Data processing system, data processing method and data processing program
The data processing system addresses response inconsistencies in stream processing systems by using a second information processing device to convert request identifiers, ensuring unique identification of requests and responses, even during recovery from failures.
Patent Information
- Application Number
- JP2021100356
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-06-16
- Publication Date
- 2025-05-16
- Estimated Expiration
- 2041-06-16
AI Technical Summary
In stream processing systems, response inconsistencies may occur when requests are sent to external systems during failures, leading to potential data inconsistencies during recovery processes.
A data processing system comprising a first and a second information processing device, where the first device sends processing requests to the second device, which converts identifiers in the requests to reversible conversion identifiers, allowing for distinct handling of repeated requests during recovery, thereby preventing response inconsistencies.
The system effectively prevents response inconsistencies by ensuring that each request and its corresponding response can be uniquely identified, even if the same request is resent during recovery processes.
Smart Images

Figure 0007678290000001 
Figure 0007678290000002 
Figure 0007678290000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a data processing system, a data processing method, and a data processing program. [Background technology]
[0002] In recent years, stream processing systems have been used that execute a number of tasks (task programs) in a chain for IoT (Internet of Things) data received from a number of sensors and output the data.
[0003] In such a stream processing system, for example, if a need arises during the execution of a task on IoT data (hereinafter also simply referred to as data), a processing request (hereinafter also referred to as a request) is sent to another information processing system (hereinafter also referred to as an external system or a specific information processing system). In this case, the external system executes processing corresponding to the received request and sends the processing result (hereinafter also referred to as a response) to the stream processing system. After that, the stream processing system processes the response received from the external system in the task (for example, see Patent Document 1). [Prior art documents] [Patent documents]
[0004] [Patent Document 1] Special Publication No. 2019-535058 Summary of the Invention [Problem to be solved by the invention]
[0005] Here, in the above-described stream processing system, for example, the internal state of each task is periodically saved in order to ensure fault tolerance. Specifically, in the stream processing system, for example, the internal state of each task is saved by following a distributed checkpoint method in which the internal state of each task is saved for each task. Then, in the case where a fault such as a network disconnection occurs in the stream processing system, for example, the saved internal state is used to restore the state before the fault occurred (hereinafter also referred to as recovery), and each task is executed again from the restored state.
[0006] However, for example, if a request was being sent to an external system when a failure occurred, requests and responses may be sent and received again between the stream processing system and the external system as a result of the execution of the above-mentioned recovery. As a result, inconsistencies in responses may occur in the stream processing system.
[0007] Therefore, in one aspect, the present invention aims to provide a data processing system, a data processing method, and a data processing program that make it possible to prevent inconsistencies from occurring in responses sent from an external system. [Means for solving the problem]
[0008] In one aspect of an embodiment, a data processing system has a first information processing device and a second information processing device accessible to the first information processing device, wherein the first information processing device, in response to receiving data to be processed, executes a task on the received data, and when a process of making a first processing request to another information processing system is executed in conjunction with the execution of the task on the data, transmits the first processing request to the second information processing device, and in response to receiving the first processing request, the second information processing device converts a first identifier included in the first processing request into a first converted identifier that can be reversibly converted, transmits the first processing request including the converted first converted identifier to the other information processing system, and in response to receiving a first execution result of the process corresponding to the first processing request, reconverts the first converted identifier included in the first execution result to the first identifier, and transmits the first execution result including the reconverted first identifier and the first converted identifier to the first information processing device. Effect of the Invention
[0009] According to one aspect, it is possible to prevent inconsistencies in responses sent from external systems. [Brief description of the drawings]
[0010] [Figure 1] FIG. 1 is a diagram illustrating the configuration of an information processing system 10. As shown in FIG. [Diagram 2] FIG. 2 is a diagram illustrating the distributed checkpoint method. [Diagram 3] FIG. 3 is a diagram illustrating the distributed checkpoint method. [Figure 4] FIG. 4 is a diagram illustrating the distributed checkpoint method. [Diagram 5] FIG. 5 is a diagram illustrating the distributed checkpoint method. [Figure 6] FIG. 6 is a diagram illustrating the distributed checkpoint method. [Figure 7] FIG. 7 is a diagram illustrating the distributed checkpoint method. [Figure 8] FIG. 8 is a diagram illustrating the distributed checkpoint method. [Figure 9] FIG. 9 is a diagram for explaining access to the external system 3. As shown in FIG. [Figure 10] FIG. 10 is a diagram for explaining access to the external system 3. As shown in FIG. [Figure 11] FIG. 11 is a diagram for explaining access to the external system 3. As shown in FIG. [Figure 12] FIG. 12 is a diagram illustrating the hardware configuration of the information processing device 1. As shown in FIG. [Figure 13] FIG. 13 is a diagram for explaining the hardware configuration of access proxy execution device 2. As shown in FIG. [Figure 14] FIG. 14 is a functional block diagram of the information processing device 1. [Figure 15] FIG. 15 is a block diagram of the functions of access proxy execution device 2. [Figure 16] FIG. 16 is a flowchart outlining stream data processing in the first embodiment. [Figure 17] FIG. 17 is a flowchart illustrating details of stream data processing in the first embodiment. [Figure 18] FIG. 18 is a diagram illustrating an overview of stream data processing in the first embodiment. [Figure 19] FIG. 19 is a flowchart illustrating details of stream data processing in the first embodiment. [Figure 20] FIG. 20 is a flowchart illustrating details of stream data processing in the first embodiment. [Figure 21] FIG. 21 is a flowchart illustrating details of stream data processing in the first embodiment. [Figure 22] FIG. 22 is a flowchart illustrating details of stream data processing in the first embodiment. [Figure 23]FIG. 23 is a flowchart illustrating details of stream data processing in the first embodiment. [Figure 24] FIG. 24 is a flowchart illustrating details of stream data processing in the first embodiment. [Diagram 25] FIG. 25 is a flowchart illustrating details of stream data processing in the first embodiment. [Figure 26] FIG. 26 is a diagram for explaining a specific example of the request management information 131 and the request management information 231. In FIG. [Figure 27] FIG. 27 is a diagram for explaining a specific example of a request. [Figure 28] FIG. 28 is a diagram illustrating a specific example of a response. [Figure 29] FIG. 29 is a diagram for explaining access to the external system 3. As shown in FIG. [Diagram 30] FIG. 30 is a diagram illustrating an overview of stream data processing in the second embodiment. [Diagram 31] FIG. 31 is a flowchart illustrating details of stream data processing in the second embodiment. [Diagram 32] FIG. 32 is a flowchart illustrating details of stream data processing in the second embodiment. [Diagram 33] FIG. 33 is a flowchart illustrating details of stream data processing in the second embodiment. [Diagram 34] FIG. 34 is a flowchart illustrating details of stream data processing in the second embodiment. [Diagram 35] FIG. 35 is a diagram for explaining a specific example of the request management information 131. As shown in FIG. [Diagram 36] FIG. 36 is a diagram for explaining access to the external system 3. As shown in FIG. [Figure 37] FIG. 37 is a diagram illustrating an overview of stream data processing in the third embodiment. [Figure 38]FIG. 38 is a flowchart illustrating details of stream data processing in the third embodiment. [Figure 39] FIG. 39 is a flowchart illustrating details of stream data processing in the third embodiment. [Diagram 40] FIG. 40 is a flowchart illustrating details of stream data processing in the third embodiment. [Diagram 41] FIG. 41 is a flowchart illustrating details of stream data processing in the third embodiment. [Diagram 42] FIG. 42 is a diagram for explaining access to the external system 3. As shown in FIG. [Diagram 43] FIG. 43 is a diagram illustrating an overview of stream data processing in the fourth embodiment. [Diagram 44] FIG. 44 is a flowchart illustrating details of stream data processing in the fourth embodiment. [Diagram 45] FIG. 45 is a flowchart illustrating details of stream data processing in the fourth embodiment. [Figure 46] FIG. 46 is a diagram illustrating a specific example of the request ID information 232. As shown in FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0011] [Configuration of the information processing system in the first embodiment] First, a description will be given of the configuration of the information processing system 10. FIG 1 is a diagram for explaining the configuration of the information processing system 10.
[0012] The information processing system 10 includes, for example, a queue 4a (hereinafter also referred to as storage device 4a) and a queue 4b (hereinafter also referred to as storage device 4b) that accumulate sensor data DT transmitted from a plurality of sensors (not shown), and an information processing device 1 (hereinafter also referred to as stream processing device 1), which is one or more physical machines or virtual machines that executes tasks for each piece of sensor data DT accumulated in the queues 4a and 4b. The sensor data DT is, for example, data such as the measured value of a thermometer in a factory or the current position of a vehicle. Hereinafter, the queues 4a and 4b are collectively referred to simply as queues 4.
[0013] Specifically, the information processing device 1 performs a process (hereinafter also referred to as stream data processing or data processing) of executing tasks T1, T2, and T3 in a chain manner for each piece of sensor data DT, as shown in Fig. 1. Hereinafter, each task including tasks T1, T2, and T3 will be collectively referred to simply as task T. Note that, although the following description will be given of a case in which three tasks (tasks T1, T2, and T3) are executed in a chain manner in the information processing device 1, the information processing device 1 may execute a number of tasks other than three in a chain manner.
[0014] More specifically, the information processing device 1 executes task T1 for the sensor data DT accumulated in queue 4a, and executes task T2 for the sensor data DT accumulated in queue 4b, as shown in Fig. 1. Then, the information processing device 1 executes task T3 for the sensor data DT (execution result of task T1) on which task T1 has been executed, and the sensor data DT (execution result of task T2) on which task T2 has been executed. After that, the information processing device 1 outputs the sensor data DT (execution result of task T3) on which task T3 has been executed, to a previously designated operation terminal (not shown), etc.
[0015] Note that the stream data processing may be performed in parallel, for example, in a plurality of information processing devices 1. That is, for example, in the plurality of information processing devices 1, distributed stream data processing may be performed in which tasks T1, T2, and T3 are executed in parallel for each piece of sensor data DT. Also, each of tasks T1, T2, and T3 may be shared and executed by, for example, a plurality of information processing devices 1.
[0016] Furthermore, as shown in FIG. 1, the information processing device 1 has, for example, a storage device T1a that stores internal states such as the execution results of task T1 (hereinafter also referred to as the internal state of task T1), a storage device T2a that stores internal states such as the execution results of task T2 (hereinafter also referred to as the internal state of task T2), and a storage device T3a that stores internal states such as the execution results of task T3 (hereinafter also referred to as the internal state of task T3).
[0017] Here, the information processing system 10 periodically saves the internal state of each task, for example, for the purpose of ensuring fault tolerance. Specifically, the information processing system 10 saves the internal state of each task, for example, by following a distributed checkpoint method that saves the internal state of each task for each task. Then, when a fault such as a network disconnection occurs, the information processing system 10 uses the saved internal state to restore the state before the fault occurred, and executes each task again from the restored state. The distributed checkpoint method will be described below.
[0018] [Distributed Checkpoints] 2 to 8 are diagrams for explaining the distributed checkpoint method.
[0019] In the information processing device 1, barrier markers are transmitted and received between tasks to notify each task of the timing for saving its internal state. Specifically, each task saves its internal state in response to receiving a barrier marker.
[0020] More specifically, the information processing device 1 periodically transmits and receives each barrier marker including a barrier marker BM1 and a barrier marker BM2 together with the sensor data DT, for example, as shown in Fig. 2. The barrier marker BM1 is a barrier marker transmitted to a task T3 via a task T1, and the barrier marker BM2 is a barrier marker transmitted to a task T3 via a task T2.
[0021] Then, for example, as shown in FIG. 3, when the barrier marker BM2 reaches the task T2, the task T2 saves in the memory device T0 the internal state that was stored in the memory device T2a at the time of receiving the barrier marker BM2.
[0022] Subsequently, for example, as shown in FIG. 4, when the barrier marker BM2 reaches the task T3, the task T3 waits until the barrier marker BM1 reaches it.
[0023] That is, task T3 is a task executed for sensor data DT (the execution result of task T1) obtained by executing task T1 and sensor data DT (the execution result of task T2) obtained by executing task T2. Therefore, task T3 waits until barrier markers arrive from all of the transmission sources of sensor data DT for task T3 (task T1 and task T2).
[0024] After that, for example, when the barrier marker BM1 reaches the task T1, the task T1 saves in the memory device T0 the internal state that was stored in the memory device T1a at the time of receiving the barrier marker BM1.
[0025] Then, for example, when the barrier marker BM1 reaches the task T3 as shown in Fig. 5, the task T3 determines that barrier markers have arrived from all of the transmission sources of the sensor data DT for the task T3, and saves the internal state stored in the memory device T3a at the time of receiving the barrier marker BM1 in the memory device T0 as shown in Fig. 6. Furthermore, in this case, the task T3 transmits, for example, a barrier marker BM3 to the next task T.
[0026] This enables the information processing device 1 to align the sensor data DT to be processed between tasks before saving the internal state.
[0027] Here, for example, when a checkpoint is completed by the arrival of the barrier marker BM3 at the last task in the information processing device 1 and this checkpoint was the last checkpoint completed before the occurrence of a failure, in case recovery is performed due to the occurrence of the failure, the information processing device 1 returns to the storage device T1a the internal states transmitted from the storage device T1a with the arrival of the barrier marker BM1 among the internal states stored in the storage device T0, as shown in Fig. 7. In addition, in this case, the information processing device 1 returns to the storage device T2a the internal states transmitted from the storage device T2a with the arrival of the barrier marker BM2 among the internal states stored in the storage device T0. In addition, in this case, the information processing device 1 returns to the storage device T3a the internal states transmitted from the storage device T3a with the arrival of the barrier markers BM1 and BM2 among the internal states stored in the storage device T0.
[0028] In other words, when the information processing device 1 performs recovery following the occurrence of a failure, it restores the states of tasks T1, T2, and T3 to the states before the failure occurred by using the internal states saved according to the distributed checkpoint method.
[0029] Then, as shown in FIG. 8, the information processing device 1 processes the sensor data DT immediately after the barrier markers BM1 and BM2 in order, and completes recovery by catching up with the read position of the sensor data DT at the time of the occurrence of the failure.
[0030] As a result, even when the information processing device 1 executes recovery following the occurrence of a failure, it is possible to prevent the occurrence of inconsistencies before and after the recovery.
[0031] [Access to external systems] Next, a description will be given of access to the external system 3. Fig. 9 to Fig. 11 are diagrams for explaining access to the external system 3.
[0032] First, there will be explained the configurations of the information processing device 1, the access proxy execution device 2, and the external system 3. Fig. 9 is a diagram for explaining the configurations of the information processing device 1, the access proxy execution device 2, and the external system 3.
[0033] The access proxy execution device 2 (hereinafter also referred to as information processing device 2) is, for example, one or more physical machines or virtual machines, and transmits requests to the external system 3 and receives (waits for) responses from the external system 3 on behalf of the information processing device 1. The access proxy execution device 2 performs stream data processing, for example, in the same way as the information processing device 1, and ensures fault tolerance by a distributed checkpoint method.
[0034] Furthermore, the external system 3 is, for example, a database system that manages various data that the information processing device 1 does not store as the internal state of each task.
[0035] Then, for example, when the information processing device 1 needs to transmit a request to the external system 3 in conjunction with the execution of a task on the sensor data DT, the information processing device 1 transmits the request that needs to be transmitted to the access proxy execution device 2.
[0036] Specifically, for example, when a need arises during the execution of a task on the sensor data DT, the information processing device 1 accumulates a request to the external system 3 in a queue 5a as shown in FIG.
[0037] Next, access proxy execution device 2, for example, acquires the requests accumulated in queue 5a and accumulates them in queue 6a. Then, external system 3, for example, executes a process corresponding to the requests accumulated in queue 6a.
[0038] Thereafter, the external system 3 accumulates multiple responses to the request in queue 6b, for example, in response to the completion of execution of a process corresponding to the request. Then, the access proxy execution device 2, for example, acquires the multiple responses accumulated in queue 6b and accumulates them in queue 5b. Furthermore, the information processing device 1, for example, sequentially acquires the multiple responses accumulated in queue 5b, and sequentially executes tasks for each acquired response.
[0039] That is, when the information processing device 1 needs to send a request to the external system 3 in conjunction with the execution of a task on the sensor data DT, it sends the request to the access proxy execution device 2. In this case, the information processing device 1 transmits and receives requests and responses to and from the access proxy execution device 2 via queues 5a and 5b. Moreover, the access proxy execution device 2 transmits and receives requests and responses to and from the external system 3 via queues 6a and 6b.
[0040] As a result, even if it becomes necessary to transmit a request to the external system 3, the information processing device 1 does not need to wait until it receives a response from the access proxy execution device 2, and can continue executing the task for the next and subsequent sensor data DT. Similarly, even if it becomes necessary to transmit a request to the external system 3, the access proxy execution device 2 does not need to wait until it receives a response from the external system 3, and can continue transmitting the next and subsequent requests.
[0041] Therefore, the information processing device 1 can suppress, for example, a waiting time of a barrier marker (for example, a waiting time of the barrier markers BM1 and BM2 in the task T3 described in FIG. 2 and the like) that occurs in association with an access to the external system 3. Therefore, the information processing device 1 can suppress, for example, a decrease in the processing speed of each task.
[0042] Next, a description will be given of a sequence chart diagram when accessing the external system 3. Fig. 10 and Fig. 11 are sequence chart diagrams when accessing the external system 3. Hereinafter, the queue 5a and the queue 5b will be collectively referred to simply as queue 5, and the queue 6a and the queue 6b will be collectively referred to simply as queue 6.
[0043] 10, when the need arises during the execution of a task on the sensor data DT, the information processing device 1 transmits a request (1) to the external system 3 via the queue 5, the access proxy execution device 2, and the queue 6. Then, when the external system 3 receives the request (1) transmitted from the information processing device 1, it executes processing corresponding to the received request (1).
[0044] Thereafter, for example, when the external system 3 generates responses (1) and (2) as responses corresponding to the request (1) transmitted from the information processing device 1, it transmits response (1) to which information indicating that it is not the final response corresponding to request (1) has been added to the information processing device 1 via the queue 6, the access proxy execution device 2, and the queue 5. In this case, the external system 3 also transmits response (2) to which information indicating that it is the final response corresponding to request (1) has been added to the information processing device 1 via the queue 6, the access proxy execution device 2, and the queue 5.
[0045] Then, when the information processing device 1 receives response (1), it waits until it receives a further response corresponding to request (1) in accordance with the information added to the received response (1). Furthermore, when the information processing device 1 receives response (2), it ends waiting for a response corresponding to request (1) in accordance with the information added to the received response (2).
[0046] Here, for example, if a request was being sent to the external system 3 when a failure occurred, the request may be sent again to the external system 3 or a response may be sent again to the information processing device 1 as recovery is performed. Therefore, in the information processing device 1, for example, an inconsistency in the response to the request may occur.
[0047] Specifically, as shown in FIG. 11, for example, when a failure occurs in the access proxy execution device 2 and recovery is performed, and the internal state of the access proxy execution device 2 returns to time CP, the access proxy execution device 2 may re-acquire request (1), which was acquired after time CP, from queue 5 and transmit it to the external system 3 via queue 6.
[0048] Therefore, in this case, request (1) is sent twice to the external system 3, and the external system 3 sends to the information processing device 1 not only responses (1) and (2) corresponding to the first request (1), but also response (1) (hereinafter also referred to as response (1) (re-attempt)) and response (2) (hereinafter also referred to as response (2) (re-attempt)) corresponding to the second request (1) (hereinafter also referred to as request (1) (re-attempt)).
[0049] In this case, for example, if the responses arrive at the information processing device 1 in the order of response (1), response (1) (again), response (2), and response (2) (again), the information processing device 1 receives each of the responses received up to the reception of response (2), which has information added indicating that it is the final response corresponding to request (1), (response (1), response (1) (again), and response (2)) as responses corresponding to request (1).
[0050] In other words, in this case, the information processing device 1 will receive responses (1) and (1) (again), which are identical responses, as responses corresponding to request (1), resulting in an inconsistency in the responses corresponding to request (1).
[0051] Therefore, in this embodiment, the information processing device 1 executes a task for the received sensor data DT in response to receiving the sensor data DT to be processed, for example. Then, when a process for making a request to an external system 3 (hereinafter also referred to as another information processing system) is executed in association with the execution of the task for the sensor data DT, the information processing device 1 transmits the request to the access proxy execution device 2.
[0052] On the other hand, in response to receiving a request, the access proxy execution device 2 in this embodiment converts the identifier included in the request into a converted identifier that can be reversibly converted. Specifically, the access proxy execution device 2 generates a converted identifier, which is a new identifier, by adding a timestamp (hereinafter also referred to as an additional identifier) to the identifier included in the request (e.g., a request ID). Then, the access proxy execution device 2 transmits the request including the converted converted identifier to the external system 3.
[0053] Furthermore, in response to receiving a response corresponding to the request, the access proxy execution device 2 reconverts the converted identifier included in the response into an identifier. Then, the access proxy execution device 2 transmits a response including the reconverted identifier and the converted identifier to the information processing device 1.
[0054] That is, the access proxy execution device 2 in this embodiment converts the identifier included in the request into a conversion identifier, so that even if the same request is sent multiple times as a result of the execution of recovery, each request is sent in a distinguishable state to the external system 3. Then, when the access proxy execution device 2 receives a response from the external system 3, it transmits the received response to the information processing device 1 in a distinguishable state in which the request corresponding to that response remains distinguishable.
[0055] This allows the information processing device 1 in this embodiment to identify responses corresponding to each request as responses corresponding to different requests, even if the same request is sent multiple times as a result of the execution of recovery. This allows the information processing device 1 to identify responses that need to be discarded (responses that are the same as responses already received) from the responses received from the access proxy execution device 2, making it possible to prevent inconsistencies from occurring in the responses.
[0056] [Hardware configuration of information processing system] Next, a description will be given of the hardware configuration of the information processing system 10. Fig. 12 is a diagram illustrating the hardware configuration of the information processing device 1. Fig. 13 is a diagram illustrating the hardware configuration of the access proxy execution device 2.
[0057] First, the hardware configuration of the information processing device 1 will be described.
[0058] 12, the information processing device 1 includes a CPU 101 which is a processor, a memory 102, an I / O interface 103, and a storage medium 104. Each unit is connected to one another via a bus 105.
[0059] The storage medium 104 has, for example, a program storage area (not shown) that stores a program 110 for performing stream data processing. The storage medium 104 also has, for example, an information storage area 130 that stores information used when performing stream data processing. The storage medium 104 may be, for example, a hard disk drive (HDD) or a solid state drive (SSD).
[0060] The CPU 101 executes a program 110 loaded from the storage medium 104 into the memory 102 to process stream data.
[0061] The I / O interface 103 is, for example, an interface device such as a network interface card, and is capable of accessing the access proxy execution device 2 via a network such as the Internet.
[0062] Next, the hardware configuration of access proxy execution device 2 will be described.
[0063] 13, the access proxy execution device 2 comprises a CPU 201 which is a processor, a memory 202, an I / O interface 203, and a storage medium 204. Each unit is connected to one another via a bus 205.
[0064] The storage medium 204 has, for example, a program storage area (not shown) that stores a program 210 for performing stream data processing. The storage medium 204 also has, for example, an information storage area 230 that stores information used when performing stream data processing. The storage medium 204 may be, for example, an HDD or SSD.
[0065] The CPU 201 executes a program 210 loaded from the storage medium 204 into the memory 202 to process stream data.
[0066] The I / O interface 203 is, for example, an interface device such as a network interface card, and is capable of accessing the information processing device 1 and the external system 3 via a network such as the Internet.
[0067] [Functions of information processing systems] Next, a description will be given of the functions of the information processing system 10. Fig. 14 is a block diagram of the functions of the information processing device 1. Fig. 15 is a block diagram of the functions of the access proxy execution device 2.
[0068] First, a functional block diagram of the information processing device 1 will be described.
[0069] As shown in FIG. 14, the information processing device 1 realizes various functions including a data receiving unit 111, an application task execution unit 112 (hereinafter also referred to as the application task execution unit 112), a request management unit 113, a request sending unit 114, a response receiving unit 115, a consistency determination unit 116, and a duplication determination unit 117, by organic cooperation between hardware such as a CPU 101 and a memory 102 and a program 110.
[0070] Furthermore, the information processing device 1 stores, for example, request management information 131, execution time unique information 132, and response specific information 133 in the information storage area .
[0071] The data receiving unit 111, for example, sequentially acquires the sensor data DT accumulated in the queue 4. In addition, the data receiving unit 111 receives, for example, the sensor data DT transmitted from a previous task (a previous task operating in the same information processing device 1 or another information processing device 1).
[0072] The application task execution unit 112 executes, for example, a task for the sensor data DT received by the data receiving unit 111 .
[0073] For example, when it becomes necessary to transmit a request to the external system 3 as a result of the execution of a task by the application task execution unit 112, the request management unit 113 generates a request to the external system 3. Then, for example, the request management unit 113 stores information about the generated request in the information storage area 130 as request management information 131.
[0074] Request sending section 114 sends, for example, the request generated by request management section 113 to access proxy execution device 2.
[0075] The response receiving unit 115 receives, for example, a response transmitted from the access proxy execution device 2 (a response corresponding to a request transmitted by the request transmitting unit 114). Specifically, the response receiving unit 115 sequentially acquires the responses accumulated in the queue 5, for example.
[0076] The consistency determination unit 116 determines whether or not the response received by the response receiving unit 115 is a response that should be discarded, for example, from the content included in the response received by the response receiving unit 115. Then, when the consistency determination unit 116 determines that the response received by the response receiving unit 115 is a response that should be discarded, for example, the consistency determination unit 116 discards the response received by the response receiving unit 115.
[0077] The duplication determination unit 117 determines whether or not the response received by the response receiving unit 115 is the same as a response previously received (hereinafter also referred to as a duplicate response) from, for example, the content included in the response received by the response receiving unit 115. Then, for example, when the duplication determination unit 117 determines that the response received by the response receiving unit 115 is a duplicate response, it discards the response received by the response receiving unit 115. The execution time unique information 132 and the response identification information 133 will be described later.
[0078] Next, a functional block diagram of access proxy execution device 2 will be described.
[0079] As shown in FIG. 15, the access proxy execution device 2 realizes various functions including a request receiving unit 211, a duplicate filter unit 212, a unique information generation unit 213, a request management unit 214, a request processing unit 215, an identifier conversion unit 216, a request sending unit 217, a response receiving unit 218, a unique information extraction unit 219, a specific information addition unit 220, a response processing unit 221, and a response sending unit 222, by organic cooperation between hardware such as a CPU 201 and a memory 202 and a program 210.
[0080] Furthermore, access proxy execution device 2 stores, for example, request management information 231 and request ID information 232 in information storage area 230.
[0081] The request receiving unit 211 receives, for example, a request transmitted from the information processing device 1. Specifically, the request receiving unit 211 sequentially acquires the requests accumulated in the queue 5, for example.
[0082] The duplicate filter unit 212 determines whether or not the request received by the request receiving unit 211 is the same as a request previously received (hereinafter also referred to as a duplicate request) based on, for example, the contents of the request received by the request receiving unit 211. Then, for example, when the duplicate filter unit 212 determines that the request received by the request receiving unit 211 is a duplicate request, the duplicate filter unit 212 discards the request received by the request receiving unit 211.
[0083] The unique information generating unit 213 generates execution time unique information corresponding to a request, for example, by the request receiving unit 211. The execution time unique information is, for example, a timestamp indicating the time when the request receiving unit 211 received the request.
[0084] The request management unit 214 stores, for example, information corresponding to the request received by the request receiving unit 211 as request management information 231 in the information storage area 130 .
[0085] The request processing unit 215 converts the request received by the request receiving unit 211, for example, in accordance with the external system 3 to which the request is to be sent.
[0086] The identifier conversion unit 216 converts, for example, an identifier included in a request received by the request receiving unit 211 into a conversion identifier that can be reversibly converted. The identifier is, for example, a request ID that can identify each request.
[0087] Specifically, the identifier conversion unit 216 generates a converted identifier by adding the runtime unique information generated by the unique information generation unit 213 to the identifier included in the request received by the request reception unit 211, for example.
[0088] The request sending unit 217 sends, for example, a request including the converted identifier converted by the identifier conversion unit 216 to the external system 3 .
[0089] The response receiving unit 218 receives, for example, a response transmitted from the external system 3 (a response corresponding to a request transmitted by the request transmitting unit 217). Specifically, the response receiving unit 218 sequentially acquires the responses accumulated in the queue 6, for example.
[0090] The unique information extraction unit 219 extracts, for example, the runtime unique information from the response received by the response reception unit 218. Then, the unique information extraction unit 219 adds the extracted runtime unique information to the response received by the response reception unit 218, for example.
[0091] The specific information addition unit 220, for example, acquires response specific information corresponding to the response received by the response receiving unit 218. The response specific information is, for example, a storage position (offset) in the queue 6 where the response received by the response receiving unit 218 was stored. Then, the specific information addition unit 220 adds the acquired response specific information to the response received by the response receiving unit 218, for example.
[0092] For example, the response processing unit 221 converts the response to which the unique information extraction unit 219 has added the runtime unique information (the response to which the specific information addition unit 220 has added the response specific information), as necessary.
[0093] The response transmission unit 222 transmits, for example, the response converted by the response processing unit 221 to the information processing device 1. The request ID information 232 will be described later.
[0094] [Outline of the first embodiment] Next, an outline of the first embodiment will be described. Figures 16 and 17 are flow charts for explaining an outline of stream data processing in the first embodiment. Also, Figure 18 is a diagram for explaining an outline of stream data processing in the first embodiment.
[0095] As shown in FIG. 16, the information processing device 1 waits until it receives the sensor data DT (NO in S11), for example.
[0096] Then, when the sensor data DT is received (YES in S11), the information processing device 1 starts, for example, executing a task for the sensor data DT (S12).
[0097] Next, for example, when it becomes necessary to transmit a request to the external system 3 in conjunction with the execution of a task on the sensor data DT (YES in S13), the information processing device 1 transmits a request to the external system 3 to the access proxy execution device 2 (S14).
[0098] If it is not necessary to transmit a request to the external system 3 (NO in S13), the information processing device 1 does not execute the process of S14.
[0099] Meanwhile, access proxy execution device 2 waits until it receives a request transmitted from information processing device 1, for example, as shown in FIG. 17 (NO in S21).
[0100] Then, when a request transmitted from information processing device 1 is received (YES in S21), access proxy execution device 2 converts, for example, the identifier included in the request into a conversion identifier that can be reversibly converted (S22).
[0101] Furthermore, the access proxy execution device 2 transmits, for example, a request including the translation identifier to the external system 3 (S23).
[0102] Thereafter, the access proxy execution device 2 waits, for example, until a response is received from the external system 3 (NO in S24).
[0103] Then, when a response is received from the external system 3 (YES in S24), the access proxy execution device 2, for example, re-converts the converted identifier included in the response into an identifier (S25).
[0104] Furthermore, access proxy execution device 2 transmits, for example, a response including the reconverted identifier and the converted identifier to information processing device 1 (S26).
[0105] This allows the information processing device 1 in this embodiment to distinguish responses corresponding to each request as responses corresponding to different requests, even if the same request is sent multiple times as a result of the execution of recovery. This allows the information processing device 1 to identify responses that need to be discarded, out of the responses received from the access proxy execution device 2, and prevents inconsistencies from occurring in the responses.
[0106] Specifically, as shown in Fig. 18, for example, when the same requests, request (1) and request (1) (re-attempt), occur due to the execution of recovery, a conversion identifier (hereinafter also referred to as a first conversion identifier) generated from an identifier included in request (1) is added to request (1), and another conversion identifier (hereinafter also referred to as a second conversion identifier) generated from an identifier included in request (1) (re-attempt) is added to request (1) (re-attempt). That is, a first conversion identifier and a second conversion identifier capable of distinguishing and identifying each request are added to each of request (1) and request (1) (re-attempt).
[0107] Then, each of response (1) and response (2) is transmitted to information processing device 1 while holding, for example, a first conversion identifier capable of identifying request (1) corresponding to each response, and each of response (1) (repeated) and response (2) (repeated) is transmitted to information processing device 1 while holding a second conversion identifier capable of identifying request (1) (repeated) corresponding to each response.
[0108] Thereafter, for example, when a response (1) (re-attached) is received after a response (1), the information processing device 1 determines that the identifier in the response (1) and the identifier in the response (1) (re-attached) are the same, but the first conversion identifier in the response (1) and the second conversion identifier in the response (1) (re-attached) are different, and discards the response (1) (re-attached).Furthermore, for example, when a response (2) (re-attached) is received after a response (1), the information processing device 1 determines that the identifier in the response (1) and the identifier in the response (2) (re-attached) are the same, but the first conversion identifier in the response (1) and the second conversion identifier in the response (2) (re-attached) are different, and discards the response (2) (re-attached).
[0109] That is, in this case, the information processing device 1 determines that a request (1) (repeated) that is the same request as the request (1) has occurred, and discards, for example, the response (1) (repeated) and the response (2) (repeated).
[0110] On the other hand, for example, when response (2) is received after response (1), the information processing device 1 determines that the identifier held by response (1) and the identifier held by response (2) are identical, and that the first conversion identifier held by response (1) and the first conversion identifier held by response (2) are identical, and does not discard response (2).
[0111] [Details of the first embodiment] Next, details of the first embodiment will be described. Figures 19 to 25 are flow charts explaining the details of stream data processing in the first embodiment. Figures 26 to 28 are diagrams explaining the details of stream data processing in the first embodiment.
[0112] [Task execution process] First, among the stream data processing, a process for executing a task on the sensor data DT (hereinafter, also referred to as a task execution process) will be described.
[0113] As shown in FIG. 19, the data receiving unit 111 waits until it receives the sensor data DT (NO in S31), for example.
[0114] Then, when the sensor data DT is received (YES in S31), the application task execution unit 112 starts, for example, the execution of a task for the received sensor data DT (S32).
[0115] Next, when it becomes necessary to transmit a request to the external system 3 in association with the execution of a task on the sensor data DT (YES in S33), the request management unit 113 generates a request ID for identifying the request to the external system 3 (S34). Then, the request management unit 113 stores information including the request ID generated in the processing of S34 as request management information 131 in the information storage area 130 (S35). A specific example of the request management information 131 will be described below.
[0116] [Example of request management information (1)] Fig. 26 and Fig. 35 are diagrams for explaining specific examples of the request management information 131 and the request management information 231. Specifically, Fig. 26(A), Fig. 26(C), and Fig. 35 are diagrams for explaining a specific example of the request management information 131. Also, Fig. 26(B) is a diagram for explaining a specific example of the request management information 231.
[0117] The request management information 131 shown in FIG. 26(A) etc. has, for example, items such as a "request ID," in which a request ID for identifying each request is set, a "timestamp," in which the time at which the request management information 131 corresponding to each request was generated is set, and a "timeout time," in which the timeout time corresponding to each request (the time at which waiting for a response corresponding to each request is to be ended) is set.
[0118] Specifically, in the request management information 131 shown in FIG. 26(A), "e4fde84d-eedc-3500-a9a0-000000000000" is set as the "request ID," "1614593910" is set as the "timestamp," and "1614594510" is set as the "timeout time."
[0119] 19, the request management unit 113 generates a request to the external system 3 from, for example, each piece of information included in the request management information 131 stored in the information storage area 130 (S36). A specific example of the request will be described below.
[0120] [Example of a request (1)] Fig. 27 is a diagram for explaining a specific example of a request. Specifically, Fig. 27(A) is a diagram for explaining a specific example of a request (hereinafter, also referred to as request RQ1) generated in the process of S36. Also, Fig. 27(B) is a diagram for explaining a specific example of a request (hereinafter, also referred to as request RQ2) in which execution time unique information is added to a request ID in the process of S55 described later.
[0121] In the request RQ1 shown in Fig. 27(A), for example, in "requestId," information of "request ID" included in the request management information 131 described in Fig. 26(A) is set. Also, in the request RQ1 shown in Fig. 27(A), for example, in "eventTime," a time corresponding to the time of "timestamp" included in the request management information 131 described in Fig. 26(A) is set.
[0122] Specifically, in the request RQ1 shown in FIG. 27(A), for example, "e4fde84d-eedc-3500-a9a0-000000000000" is set as the "requestId", and "1614593910128" is set as the "eventTime".
[0123] Returning to FIG. 19, the request sending unit 114 sends, for example, the request generated in the process of S36 to the queue 5 (S37).
[0124] On the other hand, for example, if the execution of a task for the sensor data DT does not require transmission of a request to the external system 3 (NO in S33), the request management unit 113 and the request transmission unit 114 do not perform the processes in S34 and thereafter.
[0125] [First timeout process] Next, among the stream data processes, a process performed when a timeout occurs in a request in the information processing device 1 (hereinafter, also referred to as a first timeout process) will be described.
[0126] 20, the request management unit 113 waits until the determination timing comes (NO in S41), for example. The determination timing may be a regular timing such as at one-minute intervals, for example.
[0127] Then, when the determination timing arrives (YES in S41), the request management unit 113, for example, refers to the request management information 131 stored in the information storage area 130, and determines whether or not there is a request for which a timeout has occurred (S42).
[0128] Specifically, the request management unit 113 refers to the request management information 131 described with reference to FIG. 26(A), for example, and determines whether or not there is a request for which the time set in the "timeout" has already passed.
[0129] As a result, if it is determined that a request for which a timeout has occurred exists (YES in S42), the request management unit 113 deletes, for example, the request management information 131 stored in the information storage area 130, that corresponds to the request for which it is determined that a timeout has occurred (S43).
[0130] Thereafter, the request management section 113 notifies the application task execution section 112 of information indicating that a timeout has occurred for the request determined to exist in the process of S42 (S44).
[0131] On the other hand, if it is determined that there is no request for which a timeout has occurred (NO in S42), the request management unit 113 does not perform the processes in S43 and thereafter.
[0132] [Request reception process] Next, of the stream data processing, a process that is performed when access proxy execution device 2 receives a request transmitted from information processing device 1 (hereinafter also referred to as request reception process) will be described.
[0133] As shown in FIG. 21, the unique information generating unit 213 waits until the request receiving unit 211 receives the request transmitted from the information processing device 1 (NO in S51).
[0134] Specifically, the unique information generation unit 213 waits, for example, until the request reception unit 211 acquires a request from the queue 5. Note that the request reception unit 211 may acquire requests at regular intervals, for example, at one-second intervals.
[0135] Then, when the request receiving unit 211 receives a request transmitted from the information processing device 1 (YES in S51), the unique information generating unit 213 generates, for example, runtime unique information corresponding to the received request (S52).
[0136] Specifically, the unique information generating unit 213 generates, for example, a timestamp indicating the time when the request receiving unit 211 received the request as the execution time unique information.
[0137] Next, the request management unit 214 stores each piece of information including the execution time unique information generated in the process of S52 as the request management information 231 in the information storage area 230 (S53). A specific example of the request management information 231 will be described below.
[0138] [Example of request management information (2)] The request management information 231 shown in FIG. 26(B) has, for example, items such as a "request ID" in which a request ID that identifies each request is set, a "timestamp" in which the time at which the request management information 231 corresponding to each request was generated is set, and a "timeout time" in which the timeout time corresponding to each request (the time at which waiting for a response corresponding to each request is to be stopped) is set.
[0139] Specifically, in the request management information 231 shown in FIG. 26(B), "e4fde84d-eedc-3500-a9a0-000000000000" is set as the "request ID," "1614593910" is set as the "timestamp," and "1614594510" is set as the "timeout time."
[0140] Returning to FIG. 21, the request processing unit 215 generates (converts) the payload of the request received in the process of S51 (S54).
[0141] Specifically, the request processing unit 215 converts the request received in the process of S51, for example, into a state that can be identified by the external system 3 that transmits the request.
[0142] Then, the identifier conversion unit 216 converts, for example, the request ID included in the request received in the process of S51 (S55).
[0143] Specifically, the identifier conversion unit 216 adds the runtime unique information generated in the process of S52 to the request ID included in the request received in the process of S51. A specific example of the request after the process of S55 is described below.
[0144] [Example of request (2)] In the request RQ2 shown in FIG. 27(B), for example, a request ID converted to a timestamp at the end (for example, a timestamp indicating the time when the request receiving unit 211 received the request) is set in "requestId".
[0145] Specifically, in the request RQ2 shown in FIG. 27(B), for example, "e4fde84d-eedc-3500-a9a1-614593910764" is set as the "requestId".
[0146] Returning to FIG. 21, the request transmission unit 217 transmits, for example, the request whose request ID has been converted in the process of S55 to the queue 6 (S56).
[0147] [Second timeout process] Next, among the stream data processing, a process carried out when a timeout occurs in a request in access proxy execution device 2 (hereinafter also referred to as second timeout processing) will be described.
[0148] 22, the request management unit 214 waits until the determination timing comes (NO in S61), for example. The determination timing may be a regular timing such as at one-minute intervals, for example.
[0149] Then, when the determination timing arrives (YES in S61), the request management unit 214, for example, refers to the request management information 231 stored in the information storage area 230, and determines whether or not there is a request for which a timeout has occurred (S62).
[0150] Specifically, the request management unit 214 refers to the request management information 231 described with reference to FIG. 26(B), for example, and determines whether or not there is a request for which the time set in the "timeout" has already passed.
[0151] As a result, if it is determined that a request for which a timeout has occurred exists (YES in S62), the request management unit 214 deletes, for example, the request management information 231 stored in the information storage area 230, that corresponds to the request for which it is determined that a timeout has occurred (S63).
[0152] Then, the request management unit 214 transmits information indicating that a timeout has occurred for the request determined to exist in the process of S62 to the response processing unit 221 (S64).
[0153] On the other hand, if it is determined that there is no request for which a timeout has occurred (NO in S62), the request management unit 214 does not perform the processes in S63 and thereafter.
[0154] [First response reception process] Next, of the stream data processing, a process that is performed when the access proxy execution device 2 receives a response sent from the external system 3 (hereinafter, also referred to as a first response reception process) will be described.
[0155] 23, the unique information extraction unit 219 waits, for example, until the response reception unit 218 receives a response transmitted from the external system 3 (NO in S71). Specifically, the unique information extraction unit 219 waits, for example, until the response reception unit 218 acquires a response from the queue 6. Note that the response reception unit 218 may acquire responses at regular intervals, for example, at one-second intervals. A specific example of a response received in the processing of S71 will be described below.
[0156] [Example of response (1)] Fig. 28 is a diagram for explaining a specific example of a response. Specifically, Fig. 28(A) is a diagram for explaining a specific example of a response received in the process of S71 (hereinafter also referred to as response RS1). Also, Fig. 28(B) is a diagram for explaining a specific example of a response after conversion in the process of S79 described later (hereinafter also referred to as response RS2).
[0157] In response RS1 shown in FIG. 28(A), for example, a request ID converted to a timestamp at the end (a timestamp indicating the time when the request receiving unit 211 received the request) is set in "requestId". In addition, in response RS1 shown in FIG. 28(A), for example, "hasNext" is set indicating whether each response is a final response or not. A final response is the last response sent from the external system 3 among the responses corresponding to each request. In addition, "hasNext" is set, for example, to "false" indicating that each response is a final response, or "true" indicating that each response is not a final response.
[0158] Specifically, in the response RS1 shown in FIG. 28(A), for example, "e4fde84d-eedc-3500-a9a1-614593910764" is set as the "requestId" and "false" is set as the "hasNext".
[0159] Returning to FIG. 23, when a response sent from the external system 3 is received (YES in S71), the unique information extraction unit 219 restores the request ID included in the received response (S72).
[0160] Specifically, the unique information extraction unit 219 identifies, for example, a portion of the request ID included in the response received by the response reception unit 218 other than the execution time unique information as the restored request ID.
[0161] Furthermore, the unique information extraction unit 219 adds, for example, the runtime unique information contained in the response received in the process of S71 to the response received in the process of S71 (S73).
[0162] Specifically, the unique information extraction unit 119 sets the execution time unique information in the items other than the request ID in the response received in the process of S71.
[0163] Thereafter, the request management unit 214, for example, refers to the request management information 231 stored in the information storage area 230 and determines whether or not request management information 231 corresponding to the request including the request ID restored in the processing of S72 exists (S74).
[0164] As a result, if it is determined that request management information 231 exists corresponding to the request including the request ID restored in the processing of S72 (YES in S74), the request management unit 214 adds, for example, necessary information about the request corresponding to the response received in the processing of S71 to the response received in the processing of S71 (S76).
[0165] Specifically, the request management unit 214 adds, for example, to the response received in the processing of S71, information necessary for processing the response in the information processing device from among the information contained in the request corresponding to the response received in the processing of S71 (the request received in the processing of S51) (for example, information for identifying the task that output the request, etc.).
[0166] Next, the request management unit 214 determines, for example, whether or not the response received in the process of S71 is the final response (S77).
[0167] Specifically, the request management unit 214 determines whether or not "hasNext" in the response received in the process of S71 is "false", for example.
[0168] As a result, if it is determined that the response received in the processing of S71 is the final response (YES in S77), the request management unit 214 deletes, for example, the request management information 231 stored in the information storage area 230 that was determined to exist in the processing of S74 (S78).
[0169] On the other hand, if it is determined in the process of S71 that the response received is not the final response (NO in S77), the request management unit 214 does not perform the process of S78.
[0170] Then, the response processing unit 221 generates (converts) the payload of the response to which the information has been added in the process of S76 (S79). A specific example of the response after the conversion in the process of S79 will be described below.
[0171] [Response example (2)] In the request RQ2 shown in Fig. 28(B), for example, the request ID reconverted by resetting "0" to the end (the request ID restored in the processing of S72) is set in "requestId". Also, in the request RQ2 shown in Fig. 28(B), for example, the execution time unique information is set in "acceptedTime".
[0172] Specifically, in the request RQ2 shown in FIG. 28(B), for example, “requestId” is set to “e4fde84d-eedc-3500-a9a0-000000000000”, “hasNext” is set to “false”, and “acceptedTime” is set to “1614593910764”.
[0173] Returning to FIG. 23, the response transmission unit 222 transmits the response converted in the process of S79 to the queue 5 (S80).
[0174] On the other hand, if it is determined that there is no request management information 231 corresponding to the request including the request ID restored in the processing of S72 (NO in S74), the request management unit 214 discards the response received in the processing of S71 (S75).
[0175] [Second response reception process] Next, among the stream data processes, a process that the information processing device 1 performs when it receives a response transmitted from the external system 3 (hereinafter, also referred to as a second response reception process) will be described.
[0176] As shown in FIG. 24, for example, consistency determination section 116 waits until response reception section 115 receives a response transmitted from access proxy execution device 2 (NO in S81).
[0177] Specifically, the consistency determination unit 116 waits, for example, until the response reception unit 115 acquires a response from the queue 5. Note that the response reception unit 115 may acquire a response at regular intervals, for example, at one-second intervals.
[0178] Then, when the response receiving unit 115 receives a response sent from the access proxy execution device 2 (YES in S81), the consistency determination unit 116, for example, refers to the request management information 131 stored in the information storage area 130 and determines whether or not there is request management information 131 corresponding to the request ID included in the received response (S82).
[0179] As a result, when it is determined that the request management information 131 corresponding to the request ID included in the response received in the process of S81 exists (YES in S82), the consistency determination unit 116 determines, for example, whether or not the request management information 131 determined to exist includes runtime unique information 132 (S84). The runtime unique information 132 is runtime unique information stored in the information storage area 130 in the process of S93, which will be described later.
[0180] Then, if it is determined that the request management information 131 includes the execution time unique information 132 (NO in S84), the consistency determination unit 116 determines, for example, whether the response received in the processing of S81 is the final response (S91), as shown in FIG. 25.
[0181] As a result, if it is determined that the response received in the processing of S81 is the final response (YES in S91), the consistency determination unit 116 deletes, for example, the request management information 131 stored in the information storage area 130 that was determined to exist in the processing of S82 (S92).
[0182] On the other hand, if it is determined that the response received in the processing of S81 is not the final response (NO in S91), the consistency determination unit 116 stores, for example, the runtime unique information included in the response received in the processing of S81 as runtime unique information 132 in the information storage area 130 (S93).
[0183] Specifically, as shown in FIG. 26C, for example, the consistency determining unit 116 stores the execution time unique information 132 in the information storage area 130 as part of the request management information 131 determined to exist in the process of S82.
[0184] Then, after the process of S92 or S93, the consistency determining unit 116 notifies the application task executing unit 112 of the response received in the process of S81, for example (S94).
[0185] On the other hand, if it is determined that the request management information 131 contains the runtime unique information 132 (YES in S84), the consistency determination unit 116 determines whether the runtime unique information contained in the response received in the processing of S81 matches the runtime unique information 132 contained in the request management information 131 determined to exist in the processing of S82 (S85).
[0186] As a result, if it is determined that the runtime unique information included in the response received in the processing of S81 matches the runtime unique information 132 included in the request management information 131 determined to exist in the processing of S82 (YES in S85), the consistency determination unit 116, for example, notifies the application task execution unit 112 of the response received in the processing of S81 (S86).
[0187] After that, the consistency determining unit 116 determines, for example, whether or not the response received in the process of S81 is the final response (S87).
[0188] As a result, if it is determined that the response received in the processing of S81 is the final response (YES in S87), the consistency determination unit 116 deletes, for example, the request management information 131 stored in the information storage area 230 that was determined to exist in the processing of S82 (S88).
[0189] On the other hand, if it is determined in the process of S81 that the response received is not the final response (NO in S87), the request management unit 214 does not perform the process of S88.
[0190] Furthermore, if it is determined that the runtime unique information included in the response received in the processing of S81 does not match the runtime unique information 132 included in the request management information 131 determined to exist in the processing of S82 (NO in S85), the consistency determination unit 116, for example, discards the response received in the processing of S81 (S83).
[0191] That is, even if a response having the same request ID as a previously received response is received, if the execution time unique information is different, the consistency determining unit 116 determines that the same response as the previously received response has been transmitted again. In this case, the consistency determining unit 116 discards the received response without notifying the application task executing unit 112.
[0192] This enables the information processing device 1 to prevent inconsistencies from occurring in the response sent from the external system 3.
[0193] In addition, if the consistency determination unit 116 determines that there is no request management information 131 corresponding to the request ID included in the response received in the processing of S81 (NO in S82), the consistency determination unit 116 similarly discards the response received in the processing of S81 (S83).
[0194] In this way, the information processing device 1 in this embodiment executes a task for the received sensor data DT in response to receiving the sensor data DT to be processed, for example. Then, when a process of making a request to the external system 3 is executed in association with the execution of the task for the sensor data DT, the information processing device 1 transmits the request to the access proxy execution device 2.
[0195] On the other hand, in response to receiving a request, the access proxy execution device 2 in this embodiment converts the identifier included in the request into a conversion identifier that can be reversibly converted. Then, the access proxy execution device 2 transmits the request including the converted conversion identifier to the external system 3.
[0196] Furthermore, in response to receiving a response corresponding to the request, the access proxy execution device 2 reconverts the converted identifier included in the response into an identifier. Then, the access proxy execution device 2 transmits a response including the reconverted identifier and the converted identifier to the information processing device 1.
[0197] That is, by converting the identifier included in the request into a conversion identifier, the access proxy execution device 2 in this embodiment transmits each request in a distinguishable state to the external system 3 even if the same request is transmitted multiple times in association with the execution of recovery. Then, when the access proxy execution device 2 receives a response from the external system 3, it transmits the received response to the information processing device 1 in a distinguishable state in which the request corresponding to the response remains distinguishable.
[0198] This allows the information processing device 1 in this embodiment to distinguish responses corresponding to each request as responses corresponding to different requests, even if the same request is sent multiple times as a result of the execution of recovery. This allows the information processing device 1 to identify responses that need to be discarded, out of the responses received from the access proxy execution device 2, and prevents inconsistencies from occurring in the responses.
[0199] [Outline of the second embodiment] Next, an outline of the second embodiment will be described. Fig. 29 is a diagram for explaining access to the external system 3. Fig. 30 is a diagram for explaining an outline of stream data processing in the second embodiment. Note that the following describes the differences from the stream data processing in the first embodiment.
[0200] As shown in FIG. 29, for example, when the internal state of the access proxy execution device 2 returns to the time CP due to recovery following the occurrence of a failure, the access proxy execution device 2 may re-acquire from queue 6 the response (1) that was acquired after time CP, and transmit it to the information processing device 1 via queue 5.
[0201] Therefore, in this case, the information processing device 1 receives two responses, response (1) and response (1) (again), each having the same content, and a mismatch occurs between the responses corresponding to the request (1).
[0202] 30, the access proxy execution device 2 in the second embodiment adds information indicating the order of transmission from the external system 3 (the order of arrival in the queue 6) to the response received from the external system 3. Then, the access proxy execution device 2 transmits the response to which the information indicating the order of transmission from the external system 3 has been added to the information processing device 1.
[0203] Thereafter, when the information processing device 1 receives a response from the access proxy execution device 2, it determines whether or not the transmission order of the received response is later than the transmission order of the other responses already received. As a result, when it is determined that the transmission order of the received response is not later than the transmission order of the other responses already received, the information processing device 1 discards the received response.
[0204] In other words, if the transmission order of the received response is not later than the transmission order of the already received response, the information processing device 1 determines that the already received response has been resent from the access proxy execution device 2, and discards the received response without notifying the application task execution unit 112.
[0205] This makes it possible for the information processing device 1 in this embodiment to more effectively prevent inconsistencies from occurring in responses transmitted from the external system 3.
[0206] [Details of the second embodiment] Next, details of the second embodiment will be described. Figures 31 to 34 are flow charts for explaining the details of the stream data processing in the second embodiment. Also, Figure 35 is a diagram for explaining the details of the stream data processing in the second embodiment.
[0207] [First response reception process] First, the first response reception process in the second embodiment will be described.
[0208] As shown in FIG. 31, for example, the unique information extraction unit 219 waits until the response reception unit 218 receives a response transmitted from the external system 3 (NO in S101).
[0209] Then, when the response receiving unit 218 receives a response transmitted from the external system 3 (YES in S101), the unique information extracting unit 219 recovers, for example, the request ID included in the received response (S102).
[0210] Furthermore, the unique information extraction unit 219 and the specific information addition unit 220 add, for example, the execution time unique information and the response specific information contained in the response received in the process of S101 to the response received in the process of S101 (S103).
[0211] Specifically, the unique information extraction unit 119 sets the execution time unique information in the response received in the process of S101 (items other than the request ID of the response received in the process of S101), for example.
[0212] Furthermore, the specific information adding unit 220 specifies, as response specific information, for example, a storage position (offset) in the queue 6 where the response received in the process of S101 was stored. Then, the specific information adding unit 220 sets the response specific information in the response received in the process of S101, for example.
[0213] That is, the specific information addition unit 220 adds, for example, response specific information to the response received in the processing of S101, which is information indicating the order in which the response received in the processing of S101 was sent from the external system 3 (the order in which it arrived at the queue 6).
[0214] Thereafter, the request management unit 214, for example, refers to the request management information 231 stored in the information storage area 230 and determines whether or not request management information 231 corresponding to the request including the request ID restored in the processing of S102 exists (S104).
[0215] As a result, if it is determined that request management information 231 exists corresponding to the request including the request ID restored in the processing of S102 (YES in S104), the request management unit 214 adds necessary information about the request corresponding to the response received in the processing of S101 to the response received in the processing of S101 (S106).
[0216] Next, it is determined whether the response received in the process of S101 is the final response (S107).
[0217] As a result, if it is determined that the response received in the processing of S101 is the final response (YES in S107), the request management unit 214 deletes, for example, the request management information 231 stored in the information storage area 230, that was determined to exist in the processing of S104 (S108).
[0218] On the other hand, if it is determined in the process of S101 that the response received is not the final response (NO in S107), the request management unit 214 does not perform the process of S108.
[0219] Then, the response processing unit 221 generates (converts) a payload of the response to which the information has been added in the process of S106 (S109).
[0220] Thereafter, the response transmission unit 222 transmits the response converted in the process of S109 to the queue 5 (S110).
[0221] On the other hand, if it is determined that no request management information 231 exists corresponding to the request including the request ID restored in the processing of S102 (NO in S104), the request management unit 214 deletes the request management information 231 determined to exist in the processing of S104 (S105).
[0222] [Third response reception process] Next, among the stream data processing in the second embodiment, we will explain the processing performed when the information processing device 1 receives a response sent from the external system 3, which is performed before the second response reception processing (hereinafter also referred to as the third response reception processing).
[0223] As shown in FIG. 32, duplication determination section 117 waits, for example, until response reception section 115 receives a response sent from access proxy execution device 2 (NO in S111).
[0224] Then, when the response receiving unit 115 receives a response sent from the access proxy execution device 2 (YES in S111), the duplication determination unit 117, for example, refers to the request management information 131 stored in the information storage area 130 and determines whether or not there is request management information 131 corresponding to the request ID included in the received response (S112).
[0225] As a result, when it is determined that the request management information 131 corresponding to the request ID included in the response received in the process of S111 exists (YES in S112), the duplication determination unit 117 determines, for example, whether or not the request management information 131 determined to exist includes response specific information 133 (S114). The response specific information 133 is response specific information stored in the information storage area 130 in the process of S133 described later.
[0226] Then, if it is determined that the request management information 131 does not include the response identification information 133 (NO in S114), the duplication determination unit 117 determines whether the response received in the processing of S111 is the final response (S131), as shown in FIG. 34.
[0227] As a result, if it is determined that the response received in the processing of S111 is the final response (YES in S131), the duplication determination unit 117 deletes, for example, the request management information 131 stored in the information storage area 130 that was determined to exist in the processing of S112 (S132).
[0228] On the other hand, if it is determined that the response received in the processing of S111 is not the final response (NO in S131), the duplication determination unit 117 stores, for example, the response identification information included in the response received in the processing of S111 as response identification information 133 in the information storage area 130 (S133).
[0229] Specifically, as shown in FIG. 35, for example, the request management unit 113 stores the response identification information 133 in the information storage area 130 as part of the request management information 131 determined to exist in the process of S112.
[0230] Then, after the process of S132 or S133, the duplication determining unit 117 notifies the application task executing unit 112 of the response received in the process of S111, for example (S134).
[0231] On the other hand, if it is determined that the request management information 131 contains response identification information 133 (YES in S114), the request management unit 113 determines, for example, whether the response identification information included in the response received in the processing of S111 is greater than the response identification information 133 included in the request management information 131 determined to exist in the processing of S112 (S115).
[0232] As a result, if it is determined that the response identification information included in the response received in the processing of S111 is greater than the response identification information 133 included in the request management information 131 determined to exist in the processing of S112 (YES in S115), the duplication determination unit 117 updates, for example, the response identification information 133 included in the request management information 131 determined to exist in the processing of S112 to the response identification information included in the response received in the processing of S81 (S116).
[0233] Then, as shown in FIG. 33, the duplication determining unit 117 notifies the application task executing unit 112 of the response received in the process of S111 (S121).
[0234] Thereafter, the duplication determination unit 117 determines whether or not the response received in the process of S111 is the final response (S122).
[0235] As a result, if it is determined that the response received in the processing of S111 is the final response (YES in S122), the duplication determination unit 117 deletes, for example, the request management information 131 stored in the information storage area 230 that was determined to exist in the processing of S112 (S123).
[0236] On the other hand, if it is determined in the process of S111 that the response received is not the final response (NO in S122), the request management unit 214 does not perform the process of S123.
[0237] Furthermore, if the duplication determination unit 117 determines that the response identification information included in the response received in the processing of S111 is not greater than the response identification information 133 included in the request management information 131 determined to exist in the processing of S112 (NO in S115), it discards the request received in the processing of S111 (S113).
[0238] In other words, if the response identification information of the response received in processing S81 is not greater than the response identification information 133 stored in the information storage area 130, the information processing device 1 determines that the already received response has been resent from the access proxy execution device 2, and discards the response received in processing S81.
[0239] This makes it possible for the information processing device 1 in this embodiment to more effectively prevent inconsistencies from occurring in responses transmitted from the external system 3.
[0240] In addition, the duplication determination unit 117 similarly discards the request received in the processing of S111 (S113) if it determines that there is no request management information 131 corresponding to the request ID included in the response received in the processing of S111 (NO in S112).
[0241] [Outline of the third embodiment] Next, an overview of the third embodiment will be described. Fig. 36 is a diagram for explaining access to an external system 3. Fig. 37 is a diagram for explaining an overview of stream data processing in the third embodiment. Note that the following describes the differences from the stream data processing in the first embodiment.
[0242] As shown in FIG. 36, for example, when the internal state of the access proxy execution device 2 returns to the time CP due to recovery following the occurrence of a failure, the access proxy execution device 2 re-acquires from queue 5 the request (1) that was obtained after time CP and transmits it to the external system 3 via queue 6, and also re-acquires from queue 6 the response (1) that was obtained after time CP and transmits it to the information processing device 1 via queue 5.
[0243] In this case, there is a possibility that the access proxy execution device 2 will obtain response (1) (re-attempt) before request (1) (re-attempt). Therefore, even if the access proxy execution device 2 receives response (1) (re-attempt), there is a possibility that the received response (1) (re-attempt) will be discarded because there is no request management information 231 corresponding to request (1) (re-attempt). Therefore, in this case, the information processing device 1 will not be able to receive all responses to request (1), and an inconsistency in the responses corresponding to request (1) will occur.
[0244] Therefore, in the third embodiment, when the access proxy execution device 2 receives a response sent from the external system 3 prior to the request sent from the information processing device 1, as shown in FIG. 37, it waits until it receives a request corresponding to the received response.
[0245] This makes it possible for the information processing device 1 in this embodiment to more effectively prevent inconsistencies from occurring in responses transmitted from the external system 3.
[0246] [Details of the third embodiment] Next, the details of the third embodiment will be described. Figures 38 to 41 are flow charts explaining the details of stream data processing in the third embodiment. Note that, hereinafter, a response that is waiting until a request is received is also called a waiting response.
[0247] [Request reception process] Next, a request receiving process in the third embodiment will be described.
[0248] As shown in FIG. 38, for example, the request management unit 214 waits until the request receiving unit 211 receives the request transmitted from the information processing device 1 (NO in S141).
[0249] Then, when the request receiving unit 211 receives a request sent from the information processing device 1 (YES in S141), the request management unit 214, for example, refers to the request management information 231 stored in the information storage area 230 and determines whether or not there is request management information 231 corresponding to the request ID included in the request received in the processing of S141 (S142).
[0250] As a result, if it is determined that there is request management information 231 corresponding to the request ID included in the request received in the processing of S141 (YES in S142), the request management unit 214 discards the request received in the processing of S141 (S143), for example.
[0251] On the other hand, if it is determined that there is no request management information 231 corresponding to the request ID included in the request received in S141 (NO in S142), the request management unit 214 determines whether or not there is a waiting response corresponding to the request ID included in the request received in S141, for example, in a waiting response queue (not shown) (S144).
[0252] As a result, if it is determined that a standby response corresponding to the request ID included in the request received in the processing of S141 exists (YES in S144), the request management unit 214 acquires, for example, one of the standby responses determined to exist, from the top (S145).
[0253] Next, the response transmission unit 222 transmits, for example, the standby response acquired in the process of S145 to the queue 5 (S146).
[0254] Specifically, the response transmission unit 222 generates (converts) the payload of the standby response acquired in the process of S145, for example, and transmits the converted response to the queue 5.
[0255] Then, when all standby responses corresponding to the request ID included in the request received in the processing of S141 have been acquired (YES in S147), the access proxy execution device 2 ends the request reception processing.
[0256] On the other hand, if all standby responses corresponding to the request ID included in the request received in the process of S141 have not been acquired (NO in S147), the request management unit 214 performs the processes from S145 onwards again.
[0257] Furthermore, if it is determined that there is no standby response corresponding to the request ID included in the request received in the processing of S141 (NO in S144), the unique information generation unit 213 generates, for example, runtime unique information corresponding to the request received in the processing of S141 (S151), as shown in FIG. 39.
[0258] Next, the request management unit 214 stores each piece of information including the execution time unique information generated in the process of S151 as the request management information 231 in the information storage area 230 (S152).
[0259] Then, the request processing unit 215 generates (converts) the payload of the request received in the process of S152 (S153).
[0260] Furthermore, the identifier conversion unit 216 converts, for example, the request ID included in the request received in the process of S152 (S154).
[0261] Thereafter, the request sending unit 217 sends the request, the request ID of which has been converted in the process of S154, to the queue 6 (S155).
[0262] [First response reception process] Next, the first response reception process in the third embodiment will be described.
[0263] As shown in FIG. 40, for example, the unique information extraction unit 219 waits until the response reception unit 218 receives a response transmitted from the external system 3 (NO in S161).
[0264] Then, when the response receiving unit 218 receives a response transmitted from the external system 3 (YES in S161), the unique information extracting unit 219 recovers, for example, the request ID included in the received response (S162).
[0265] Furthermore, the unique information extraction unit 219 adds, for example, the runtime unique information contained in the response received in the process of S161 to the response received in the process of S161 (S163).
[0266] Thereafter, the request management unit 214, for example, refers to the request management information 231 stored in the information storage area 230 and determines whether or not there is request management information 231 corresponding to the request including the request ID restored in the processing of S162 (S164).
[0267] As a result, if it is determined that request management information 231 exists corresponding to the request including the request ID restored in the processing of S162 (YES in S164), the request management unit 214 adds, for example, necessary information about the request corresponding to the response received in the processing of S161 to the response received in the processing of S161 (S166).
[0268] Next, the request management unit 214 determines whether or not the response received in the process of S161 is the final response (S167).
[0269] As a result, if it is determined that the response received in the processing of S161 is the final response (YES in S167), the request management unit 214 deletes, for example, the request management information 231 stored in the information storage area 230 that was determined to exist in the processing of S164 (S168).
[0270] On the other hand, if it is determined in the process of S161 that the received response is not the final response (NO in S167), the request management unit 214 does not perform the process of S168.
[0271] Then, the response processing unit 221 generates (converts) a payload of the response to which the information has been added in the process of S166 (S169).
[0272] Thereafter, the response transmission unit 222 transmits the response converted in the process of S169 to the queue 5 (S170).
[0273] On the other hand, if it is determined that there is no request management information 231 corresponding to the request including the request ID restored in the processing of S162 (NO in S164), the request management unit 214 adds, for example, the response received in the processing of S161 to a queue for waiting responses as a waiting response (S165).
[0274] That is, in this case, the request management unit 214 sets the response received in the process of S161 as a standby response, and waits for a request corresponding to the response received in the process of S161.
[0275] This makes it possible for the information processing device 1 in this embodiment to more effectively prevent inconsistencies from occurring in responses transmitted from the external system 3.
[0276] [Response deletion process] Next, among the stream data processes in the third embodiment, a process of deleting a standby response stored in a standby response queue (hereinafter, also referred to as a response deletion process) will be described.
[0277] 41, the request management unit 214 waits until the determination timing comes (NO in S171), for example. The determination timing may be a regular timing such as at one-minute intervals, for example.
[0278] Then, when the determination timing arrives (YES in S171), the request management unit 214 determines, for example, whether or not there is a waiting response among the waiting messages stored in the waiting response queue that has been stored in the queue for a predetermined time or more (S172).
[0279] As a result, if it is determined that there is a standby response that has been stored in the queue for a predetermined time or more (YES in S172), the request management unit 214 deletes, for example, from among the waiting messages stored in the standby response queue, a standby response that has been stored in the queue for a predetermined time or more (S173).
[0280] On the other hand, if it is determined that there is no waiting response that has been stored in the queue for a time equal to or longer than the predetermined time (NO in S172), the request management unit 214 does not perform the process of S173.
[0281] [Outline of the Fourth Embodiment] Next, an outline of the fourth embodiment will be described. Fig. 42 is a diagram for explaining access to the external system 3. Fig. 43 is a diagram for explaining an outline of stream data processing in the fourth embodiment. Note that the following describes the differences from the stream data processing in the first embodiment.
[0282] As shown in FIG. 42, for example, when the internal state of the information processing device 1 returns to the time point CP by recovery following the occurrence of a failure, the information processing device 1 retransmits to the queue 5 the requests that were transmitted after the time point CP.
[0283] Therefore, in this case, request (1) is sent twice to the external system 3, and the external system 3 sends response (1) (again) corresponding to request (1) (again) to the information processing device 1, in addition to response (1) corresponding to request (1). That is, in this case, the information processing device 1 receives response (1) and response (1) (again), which are responses with the same content, and an inconsistency occurs in the response corresponding to request (1).
[0284] 43, when access proxy execution device 2 in the fourth embodiment receives a request from information processing device 1, it stores the request ID included in the received request. Then, when access proxy execution device 2 receives another request having the stored request ID from information processing device 1, it discards the other received request without sending it to queue 6.
[0285] That is, when access proxy execution device 2 receives a request having the same request ID as a previously received request, it determines that the same request has been sent from information processing device 1, and discards the received request.
[0286] This makes it possible for the information processing device 1 in this embodiment to more effectively prevent inconsistencies from occurring in responses transmitted from the external system 3.
[0287] [Details of the fourth embodiment] Next, details of the fourth embodiment will be described. Figures 44 and 45 are flow charts for explaining details of stream data processing in the fourth embodiment. Also, Figure 46 is a diagram for explaining details of stream data processing in the fourth embodiment.
[0288] [Request reception process] Next, a request receiving process in the fourth embodiment will be described.
[0289] As shown in FIG. 44, for example, the request management unit 214 waits until the request receiving unit 211 receives the request transmitted from the information processing device 1 (NO in S181).
[0290] Then, when the request receiving unit 211 receives a request sent from the information processing device 1 (YES in S181), the request management unit 214, for example, refers to the request ID information 232 stored in the information storage area 230 and determines whether or not there is request management information 231 corresponding to the request ID included in the request received in the processing of S181 (S182).
[0291] As a result, if it is determined that there is request management information 231 corresponding to the request ID included in the request received in the processing of S181 (YES in S182), the request management unit 214, for example, discards the request received in the processing of S181 (S183).
[0292] On the other hand, if it is determined that there is no request management information 231 corresponding to the request ID included in the request received in the process of S181 (NO in S182), the request management unit 214 stores, for example, the request ID included in the request received in the process of S181 as request ID information 232 in the information storage area 230 (S184). A specific example of the request ID information 232 will be described below.
[0293] [Example of Request ID information] FIG. 46 is a diagram illustrating a specific example of the request ID information 232. As shown in FIG.
[0294] In the request ID information shown in FIG. 46, "b0db6333-6196-4a76-ab0a-ced4d6dd9593" and "7fc986ee-49b4-4c31-b09b-2ca0ba1df969" are stored as "request IDs."
[0295] Returning to FIG. 44, the unique information generating unit 213 generates, for example, execution time unique information corresponding to the request received in the process of S181 (S185).
[0296] Next, the request management unit 214 stores each piece of information including the execution time unique information generated in the process of S185 as the request management information 231 in the information storage area 230 (S186).
[0297] Then, the request processing unit 215 generates (converts) the payload of the request received in the process of S181 (S187).
[0298] Furthermore, the identifier conversion unit 216 converts, for example, the request ID included in the request received in the process of S181 (S188).
[0299] Thereafter, the request sending unit 217 sends the request, the request ID of which has been converted in the process of S188, to the queue 6 (S189).
[0300] [ID deletion process] Next, among the stream data processes in the fourth embodiment, a process of deleting a request ID included in the request ID information 232 (hereinafter, also referred to as an ID deletion process) will be described.
[0301] 45, the request management unit 214 waits until the determination timing comes (NO in S191), for example. The determination timing may be a regular timing such as at one-minute intervals.
[0302] Then, when the time for judgment arrives (YES in S191), the request management unit 214 judges, for example, whether or not there is a request ID included in the request ID information 232 stored in the information storage area 230, for which the time that has elapsed since it was stored as the request ID information 232 is greater than or equal to a predetermined time (S192).
[0303] As a result, if it is determined that a request ID exists for which the time that has elapsed since it was stored as request ID information 232 is greater than or equal to a predetermined time (YES in S192), the request management unit 214, for example, deletes from among the request IDs included in the request ID information 232 stored in the information storage area 230, the request ID for which the time that has elapsed since it was stored as request ID information 232 is greater than or equal to a predetermined time (S193).
[0304] On the other hand, if it is determined that there is no request ID for which the time that has elapsed since it was stored as the request ID information 232 is equal to or longer than the predetermined time (NO in S192), the request management unit 214 does not perform the process of S193.
[0305] The above embodiment can be summarized as follows:
[0306] (Appendix 1) A data processing system having a first information processing device and a second information processing device accessible to the first information processing device, The first information processing device is In response to receiving data to be processed, execute a task for the received data; when a process of making a first processing request to another information processing system is executed in association with the execution of a task on the data, the first processing request is transmitted to the second information processing device; The second information processing device is In response to receiving the first processing request, converting a first identifier included in the first processing request into a first converted identifier that is reversibly convertible; Transmitting the first processing request including the converted first converted identifier to the other information processing system; In response to receiving a first execution result of the processing corresponding to the first processing request, re-converting the first converted identifier included in the first execution result into the first identifier; transmitting the first execution result including the reconverted first identifier and the first converted identifier to the first information processing device; A data processing system comprising:
[0307] (Appendix 2) In Appendix 1: The second information processing device is in response to receiving a second processing request to be executed on another information processing system in association with execution of a task on the data, converting a second identifier included in the second processing request into a second converted identifier that can be reversibly converted; Transmitting the second processing request including the converted second converted identifier to the other information processing system; In response to receiving a second execution result of the processing corresponding to the second processing request, re-converting the second converted identifier included in the second execution result into the second identifier; Transmitting the second execution result including the reconverted second identifier and the second converted identifier to the first information processing device; The first information processing device is determining whether the first identifier and the second identifier are identical in response to receiving the first execution result and the second execution result; If it is determined that the first identifier and the second identifier are identical, it is determined whether the first translation identifier and the second translation identifier are identical; discarding one of the first execution result and the second execution result when it is determined that the first conversion identifier and the second conversion identifier are different; A data processing system comprising:
[0308] (Appendix 3) In Appendix 2: When the first information processing device determines that the first conversion identifier and the second conversion identifier are different, the first information processing device discards an execution result received later from the second information processing device, out of the first execution result and the second execution result. A data processing system comprising:
[0309] (Appendix 4) In Appendix 2: The second information processing device is generating the first conversion identifier by adding a first additional identifier indicating a generation timing of the first conversion identifier to the first identifier in response to receiving the first processing request; In response to receiving the first execution result, splitting the first conversion identifier included in the first execution result into the first identifier and the first additional identifier; transmitting the first execution result including the first identifier and the first additional identifier to the first information processing device; A data processing system comprising:
[0310] (Appendix 5) In Appendix 4: The second information processing device is In response to receiving the second processing request, generating the second translation identifier by adding a second additional identifier indicating a generation timing of the second translation identifier to the second identifier; In response to receiving the second execution result, splitting the second converted identifier included in the second execution result into the second identifier and the second additional identifier; transmitting the second execution result including the second identifier and the second additional identifier to the first information processing device; The first information processing device is determining whether the first identifier and the second identifier are identical in response to receiving the first execution result and the second execution result; When it is determined that the first identifier and the second identifier are identical, it is determined whether or not the first additional identifier and the second additional identifier are identical; discarding one of the first execution result and the second execution result when it is determined that the first additional identifier and the second additional identifier are different. A data processing system comprising:
[0311] (Appendix 6) In Appendix 4: the first additional identifier is a time stamp indicating a time when the second information processing device received the first processing request, the second additional identifier is a timestamp indicating a time when the second information processing device received the second processing request; A data processing system comprising:
[0312] (Appendix 7) In Appendix 2: the second information processing device transmits to the first information processing device the first execution result including the first identifier and a transmission order of the first execution result from the other information processing system; A data processing system comprising:
[0313] (Appendix 8) In Appendix 7: the second information processing device transmits the second execution result, including the second identifier and a transmission order of the second execution result from the other information processing system, to the second information processing device; The first information processing device is determining whether the first identifier and the second identifier are identical in response to receiving the first execution result and the second execution result; When it is determined that the first identifier and the second identifier are identical, the transmission order corresponding to the first execution result is compared with the transmission order corresponding to the second execution result; discarding the second execution result when the second execution result is received after the first execution result and the transmission order corresponding to the second execution result is not after the transmission order corresponding to the first execution result; A data processing system comprising:
[0314] (Appendix 9) In Appendix 1: when the second information processing device receives the first execution result before the first processing request, the second information processing device waits until the second information processing device receives the first processing request. A data processing system comprising:
[0315] (Appendix 10) In Appendix 2: The second information processing device is In response to receiving the first processing request, storing the first identifier included in the first processing request in a storage unit; In response to receiving the second processing request, determining whether the first identifier stored in the storage unit and the second identifier included in the second processing request are identical; discarding the second processing request without transmitting it to the other information processing system when it is determined that the first identifier and the second identifier are identical; A data processing system comprising:
[0316] (Appendix 11) A data processing method in a data processing system having a first information processing device and a second information processing device accessible to the first information processing device, comprising: In response to receiving data to be processed, execute a task for the received data; when a process of making a first processing request to another information processing system is executed in association with the execution of a task on the data, the first processing request is transmitted to the second information processing device; The first information processing device executes the process, In response to receiving the first processing request, converting a first identifier included in the first processing request into a first converted identifier that is reversibly convertible; Transmitting the first processing request including the converted first converted identifier to the other information processing system; In response to receiving a first execution result of the processing corresponding to the first processing request, re-converting the first converted identifier included in the first execution result into the first identifier; executing a process of transmitting the first execution result including the reconverted first identifier and the first converted identifier to the first information processing device; A data processing method, wherein processing is executed by the second information processing device.
[0317] (Appendix 12) In response to receiving a first processing request from another information processing device to a specific information processing system, converting a first identifier included in the first processing request into a first converted identifier that is reversibly convertible; Transmitting the first processing request including the converted first converted identifier to the specific information processing system; in response to receiving a first execution result of a process corresponding to the first processing request from the specific information processing system, re-converting the first conversion identifier included in the first execution result into the first identifier; transmitting the first execution result including the reconverted first identifier and the first converted identifier to the other information processing device; A data processing program that causes a computer to execute a process.
[0318] (Appendix 13) In response to receiving data to be processed, execute a task for the received data; when a process of making a first processing request to a specific information processing system is executed in association with the execution of a task on the data, the first processing request is transmitted to another information processing device; when a process of making a second processing request to a specific information processing system is executed in association with the execution of a task on the data, the second processing request is transmitted to the other information processing device; determining whether the first identifier and the second identifier are identical in response to receiving from the other information processing device a first execution result of a process corresponding to the first processing request, the first execution result including a first identifier included in the first processing request and a first converted identifier obtained by the other information processing device by converting the first identifier in a reversible manner, and a second execution result of a process corresponding to the second processing request, the second execution result including a second identifier included in the second processing request and a second converted identifier obtained by the other information processing device by converting the second identifier in a reversible manner; If it is determined that the first identifier and the second identifier are identical, it is determined whether the first translation identifier and the second translation identifier are identical; discarding one of the first execution result and the second execution result when it is determined that the first conversion identifier and the second conversion identifier are different; A data processing program that causes a computer to execute a process. [Explanation of symbols]
[0319] 1: Information processing device 2a: Queue 2b: Queue 10: Information processing system DT: Sensor data T1: Task T1a: Storage device T2: Task T2a: Storage device T3: Task T3a:Storage device
Claims
1. A data processing system having a first information processing device and a second information processing device accessible to the first information processing device, The first information processing device is In response to receiving data to be processed, execute a task for the received data; when a process of making a first processing request to another information processing system is executed in association with the execution of a task on the data, the first processing request is transmitted to the second information processing device; The second information processing device is In response to receiving the first processing request, converting a first identifier included in the first processing request into a first converted identifier that is reversibly convertible; Transmitting the first processing request including the converted first converted identifier to the other information processing system; In response to receiving a first execution result of the processing corresponding to the first processing request, re-converting the first converted identifier included in the first execution result into the first identifier; transmitting the first execution result including the reconverted first identifier and the first converted identifier to the first information processing device; A data processing system comprising:
2. In claim 1, The second information processing device is In response to receiving a second processing request to be executed on another information processing system in association with execution of a task on the data, converting a second identifier included in the second processing request into a second converted identifier that can be reversibly converted; Transmitting the second processing request including the converted second converted identifier to the other information processing system; In response to receiving a second execution result of the processing corresponding to the second processing request, re-converting the second converted identifier included in the second execution result into the second identifier; Transmitting the second execution result including the reconverted second identifier and the second converted identifier to the first information processing device; The first information processing device is determining whether the first identifier and the second identifier are identical in response to receiving the first execution result and the second execution result; If it is determined that the first identifier and the second identifier are identical, it is determined whether the first translation identifier and the second translation identifier are identical; discarding one of the first execution result and the second execution result when it is determined that the first conversion identifier and the second conversion identifier are different; A data processing system comprising:
3. In claim 2, When the first information processing device determines that the first conversion identifier and the second conversion identifier are different, the first information processing device discards an execution result received later from the second information processing device, of the first execution result and the second execution result. A data processing system comprising:
4. In claim 2, The second information processing device is generating the first conversion identifier by adding a first additional identifier indicating a generation timing of the first conversion identifier to the first identifier in response to the reception of the first processing request; In response to receiving the first execution result, splitting the first conversion identifier included in the first execution result into the first identifier and the first additional identifier; transmitting the first execution result including the first identifier and the first additional identifier to the first information processing device; A data processing system comprising:
5. In claim 4, The second information processing device is In response to receiving the second processing request, generating the second conversion identifier by adding a second additional identifier indicating a generation timing of the second conversion identifier to the second identifier; In response to receiving the second execution result, splitting the second conversion identifier included in the second execution result into the second identifier and the second additional identifier; transmitting the second execution result including the second identifier and the second additional identifier to the first information processing device; The first information processing device is determining whether the first identifier and the second identifier are identical in response to receiving the first execution result and the second execution result; If it is determined that the first identifier and the second identifier are identical, it is determined whether or not the first additional identifier and the second additional identifier are identical; discarding one of the first execution result and the second execution result when it is determined that the first additional identifier and the second additional identifier are different. A data processing system comprising:
6. In claim 2, the second information processing device transmits to the first information processing device the first execution result including the first identifier and a transmission order of the first execution result from the other information processing system; A data processing system comprising:
7. In claim 6, the second information processing device transmits the second execution result, including the second identifier and a transmission order of the second execution result from the other information processing system, to the second information processing device; The first information processing device is determining whether the first identifier and the second identifier are identical in response to receiving the first execution result and the second execution result; When it is determined that the first identifier and the second identifier are the same, the transmission order corresponding to the first execution result is compared with the transmission order corresponding to the second execution result; discarding the second execution result when the second execution result is received after the first execution result and the transmission order corresponding to the second execution result is not after the transmission order corresponding to the first execution result; A data processing system comprising:
8. In claim 1, when the second information processing device receives the first execution result before the first processing request, the second information processing device waits until the second information processing device receives the first processing request. A data processing system comprising:
9. In claim 2, The second information processing device is In response to receiving the first processing request, storing the first identifier included in the first processing request in a storage unit; In response to receiving the second processing request, determining whether the first identifier stored in the storage unit and the second identifier included in the second processing request are identical; discarding the second processing request without transmitting it to the other information processing system when it is determined that the first identifier and the second identifier are identical; A data processing system comprising:
10. A data processing method in a data processing system having a first information processing device and a second information processing device accessible to the first information processing device, comprising: In response to receiving data to be processed, execute a task for the received data; when a process of making a first processing request to another information processing system is executed in association with the execution of a task on the data, the first processing request is transmitted to the second information processing device; The first information processing device executes the process, In response to receiving the first processing request, converting a first identifier included in the first processing request into a first converted identifier that is reversibly convertible; Transmitting the first processing request including the converted first converted identifier to the other information processing system; In response to receiving a first execution result of the processing corresponding to the first processing request, re-converting the first converted identifier included in the first execution result into the first identifier; executing a process of transmitting the first execution result including the reconverted first identifier and the first converted identifier to the first information processing device; A data processing method, wherein processing is executed by the second information processing device.
11. In response to receiving a first processing request from a first information processing device to another information processing system, converting a first identifier included in the first processing request into a first converted identifier that can be reversibly converted; Transmitting the first processing request including the converted first converted identifier to the other information processing system; in response to receiving a first execution result of the processing corresponding to the first processing request from the other information processing system, re-converting the first conversion identifier included in the first execution result into the first identifier; transmitting the first execution result including the reconverted first identifier and the first converted identifier to the first information processing device; A data processing program for causing a second information processing device to execute a process.
Citation Information
Patent Citations
Information processing device, test method of information processing system and program
JP2013041352A
Transaction resumption program, information processing device and transaction resumption method
JP2014238677A
Identification information application system and identification information application method
JP2018156553A
Micro-batch snapshot and state management
JP2019535058A
Database system, server device, program, and information processing method
WO2017042890A1