Data management method and data management program

Abstract

To dynamically optimize a storage structure of a data store according to state update, and to accelerate access to the data store.SOLUTION: A data management device 100 creates high frequency state item list information 120 in which high frequency state items having high frequency access are listed on the basis of frequency of access to a KVS 110 when a prescribed data processing program performs data processing. When writing state information containing values of high frequency state items to the KVS 110, the data management device 100 refers to the high frequency state item list information 120, and determines whether the state information corresponds to the high frequency state item. When it is determined that the state information corresponds to the high frequency state item, the data management device writes the state information of a plurality of high frequency state items while grouping the same to one record of the KVS 110. On the other hand, when it is determined that the state information does not correspond to the high frequency state item, processing of writing the state information to one record of the KVS 110 is performed.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a data management method and a data management program. [Background technology]

[0002] Digital twins make it possible to represent actual operating objects as models in a virtual space. For example, real-world IoT devices such as connected cars and smartphones send state information about their own state (situation), such as their location and speed, in an event message to a data center. The data center then constructs a digital twin of the real-world object (IoT device) from the state information contained in the received event message. The data center updates state items based on the received event message, and the user logic associated with the state item update executes a processing program, thereby performing processing for each event.

[0003] IoT devices send event messages containing state information to data centers when they update their state or periodically. Because event messages are an infinite, non-stop event stream, data centers must be able to receive huge amounts of event messages without shutting down their systems. Data centers manage the received event messages in a data store such as an LSM-tree type KVS. LSM-tree stands for Log Structured Merge tree, and KVS stands for Key Value Store. An LSM-tree type KVS has slow data read (READ) speeds but fast write speeds, making it suitable for state management of stream processing of event messages in digital twins.

[0004] Conventionally, method 1 for storing states in a digital twin data store is to store each state item of the digital twin as one entry, while method 2 is to collect all states for each digital twin in one entry.

[0005] Prior art techniques include, for example, filtering and acquiring stream data using statistical information and managing stream data characteristics from schema definitions. Then, by determining whether to cache relation data based on the data rate, technology maintains real-time processing and reduces the amount of external data duplication. Another technique performs deduplication by storing only unique data, reduces index operations within the cache when accessing data by updating cache index entries, and reduces the number of accesses to the cache index of the flash cache. Another technique updates programs without shutting down the system by storing an association between an event type and a plug-in when a control message is received, and then executing the plug-in corresponding to the event type when an event message is received. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-328716 [Patent Document 2] U.S. Patent No. 10,210,087 [Patent Document 3] Patent Publication No. 2019-133579 Summary of the Invention [Problem to be solved by the invention]

[0007] In conventional technologies, the majority of the time spent processing event message streams is spent accessing state data stored in KVS, making it difficult to improve data processing performance. Here, with conventional state storage method 1, accessing many small state items results in a read wait each time, lengthening processing time. Furthermore, with method 2, if there is a large, infrequently used state item, this state item is read and written every time along with frequently used state items, resulting in problems with high I / O and backup costs.

[0008] Furthermore, in a digital twin, the state stored in the data store is a system directly connected to the real world. In a digital twin, state access patterns change dynamically in response to changes that occur in the real world. In addition, the processing logic that processes event messages is also replaced dynamically and non-stop. For this reason, digital twins are limited in that the state storage structure cannot be optimized in advance.

[0009] In one aspect, the present invention aims to be able to dynamically optimize the storage structure of a data store in response to state updates and to speed up access to the data store. [Means for solving the problem]

[0010] According to one aspect of the present invention, a data management method and a data management program are proposed in which, when a specified data processing program processes data, it creates high-frequency state item list information that lists high-frequency state items that are accessed frequently based on the access frequency to a data store, and when writing state information including the values ​​of the high-frequency state items to the data store, it references the high-frequency state item list information to determine whether the state information corresponds to the high-frequency state item, and if it is determined that the state information corresponds to the high-frequency state item, it groups and writes the state information for multiple high-frequency state items into one record in the data store, and if it is determined that the state information does not correspond to the high-frequency state item, it writes the state information into one record in the data store. [Effects of the Invention]

[0011] According to one aspect of the present invention, it is possible to dynamically optimize the storage structure of a data store in response to state updates, thereby achieving the effect of speeding up access to the data store. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is an explanatory diagram illustrating an example of a data management method according to an embodiment. [Figure 2] Figure 2 is an explanatory diagram of the processing overview of digital twins. [Figure 3] FIG. 3 is an explanatory diagram of the data management function of the existing technology. [Figure 4A] Figure 4A is an explanatory diagram of an example of how state information of a digital twin is stored using existing technology. (Part 1) [Figure 4B] Figure 4B is an explanatory diagram of an example of how state information of a digital twin is stored using existing technology. (Part 2) [Figure 5] FIG. 5 is an explanatory diagram of another example of storing state information of a digital twin. [Figure 6] FIG. 6 is an explanatory diagram of the functions of the data management device according to the embodiment. [Figure 7] FIG. 7 is an explanatory diagram of a processing example of a WRITE operation of the data management device. [Figure 8] FIG. 8 is an explanatory diagram of a processing example of a READ operation of the data management device. [Figure 9] FIG. 9 is an explanatory diagram of an example of a process for determining high frequency state items in the data management device. [Figure 10] FIG. 10 is a diagram showing a management table for each type of digital twin in the data management device. [Figure 11] FIG. 11 is an explanatory diagram of user logic selection for each type of data management device. [Figure 12] FIG. 12 is a chart showing an example of access statistical information by type of data management device. [Figure 13] FIG. 13 is a chart showing an example of a list of high frequency state items by type managed by the data management device. [Figure 14] FIG. 14 is a diagram illustrating an example of a hardware configuration of a data management device. [Figure 15] FIG. 15 is a flowchart of an example of the overall processing of an event performed by the data management device. [Figure 16]FIG. 16 is a flowchart of an example of message processing by the data management device. [Figure 17] FIG. 17 is a flowchart of an example of a WRITE access process performed by the data management device. [Figure 18] FIG. 18 is a flowchart of an example of a READ access process performed by the data management device. [Figure 19A] FIG. 19A is a flowchart of an example of a process for creating a list of high-frequency state items performed by the data management device (part 1). [Figure 19B] FIG. 19B is a flowchart of an example of a process for creating a list of high-frequency state items performed by the data management device (part 2). [Figure 19C] FIG. 19C is a flowchart of an example of a process for creating a list of high-frequency state items performed by the data management device (part 3). [Figure 20] FIG. 20 is a flowchart of an example of a flush process performed by the data management device. [Figure 21A] FIG. 21A is an explanatory diagram of a READ / WRITE process according to the existing technology. [Figure 21B] FIG. 21B is an explanatory diagram of the READ / WRITE process according to the embodiment. [Figure 22A] FIG. 22A is an explanatory diagram of a maintenance process for grouped state items according to the existing technology. [Figure 22B] FIG. 22B is an explanatory diagram of the maintenance process for grouped state items according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the disclosed data management method and data management program will be described in detail with reference to the drawings.

[0014] (An example of a data management method according to an embodiment) 1 is an explanatory diagram showing an example of a data management method according to an embodiment. The data management device 100 is, for example, a computer that performs processing to collect statistics on frequently accessed state items in a data store, group the frequently accessed state items, and record them as a single entry in the data store. The data store is, for example, the KVS 110 or cache memory of the data management device 100.

[0015] The data management device 100 receives and processes state update event messages sent from IoT devices and updates them as state data of the digital twin. In addition, the data management device 100 executes a predetermined function (operation) by the user logic corresponding to the updated state.

[0016] Fig. 1(a) shows an example of a logical state structure of a digital twin. Fig. 1(b) shows the storage layout of the KVS 110 as a data store of the data management device 100, and the high-frequency state item list information 120.

[0017] The logical state structure of the digital twin shown in Figure 1(a) is a theoretical representation of event messages for each IoT device external to the data management device 100, and shows the digital twin (DT) for each IoT device.

[0018] Each IoT device sends an event message including state information of a state (situation) that changes over time to the data management device 100. For example, an IoT device (DT1) with id "car2 (vehicle 2)" sends the state "speed 42 (km / h)" and the state "position" as state information at a certain time.

[0019] The data management device 100 according to the embodiment performs the following processes. (1) The data management device 100 collects data on the frequency of access to state items, performs predetermined statistical processing, and generates high-frequency state item list information 120 for state items that frequently access the data store. The high-frequency state item list information 120 is a table listing of state items that are frequently accessed. The access frequency to a state item can be calculated, for example, based on the frequency of read and write access to the KVS 110. Furthermore, in the case of a KVS 110 with a cache (cache memory), it can be calculated based on the frequency of read and write access to the cache.

[0020] In the explanation using FIG. 1, for convenience, an example will be described in which the data management device 100 extracts the states of event messages that are sent frequently from among the event messages sent by the IoT devices (DT1 to DT4) as high-frequency state items.

[0021] 1(b), the data management device 100 includes position, speed, and wheel1_speed (speed of wheel 1) as high-frequency state items in the high-frequency state item list information 120. It also includes wheel1_pressure (air pressure of wheel 1), wheel2_speed (speed of wheel 2), and wheel2_pressure (air pressure of wheel 2).

[0022] For example, the data management device 100 determines the state information of event messages sent frequently from the IoT devices (DT1 to DT4) in Fig. 1(a) as high-frequency state items, and creates high-frequency state item list information 120. The high-frequency state item list information 120 is allocated in the memory of the data management device 100.

[0023] The items "position" and "speed" in the high-frequency state item list information 120 shown in FIG. 1(b) are created based on a state item H1 that is frequently transmitted from, for example, an IoT device with an ID of "car2 (passenger car 2)." The high-frequency state item list information 120 also has items of wheel1_speed (wheel 1 speed), wheel1_pressure (wheel 1 air pressure), wheel2_speed (wheel 2 speed), and wheel2_pressure (wheel 2 air pressure). These items are created based on a state item H2 that is frequently transmitted from, for example, an IoT device with an ID of "truck4 (truck 4)."

[0024] In the above explanation, an example was given in which high frequency state item list information 120 was created based on the id "car2 (passenger car 2)" and id "truck4 (truck 4)" of specific IoT devices. State items H that are sent frequently for each "car" or "truck" type (attribute) such as the type of vehicle can be set as high frequency state items in the high frequency state item list information 120. Furthermore, state items H that are sent simply with high frequency may be set as high frequency state items in the high frequency state item list information 120, regardless of the type of IoT device.

[0025] Furthermore, the data management device 100 dynamically changes and optimizes the storage structure of state items through periodic maintenance of the high frequency state item list information 120, thereby responding to changes in access patterns to state items.

[0026] (2) The data management device 100 refers to the high-frequency state item list information 120 and groups high-frequency state items that are accessed frequently in a specific digital twin X. The data management device 100 groups the grouped states into (X, * ) in one record of the KVS 110. On the other hand, the data management device 100 records other state items, that is, less frequently accessed state items, in one record of the KVS 110 as individual entries.

[0027] For example, in the example of the logical state structure of the digital twin DT shown in FIG. 1(a), the state item H1 "speed: 42" and "position: ..." of the id "car2" correspond to the high frequency state items in the high frequency state item list information 120 in FIG. 1(b). In this case, as shown in FIG. 1(b), the data management device 100 stores one entry (X, * ), and set the key to (car2, * ) The data management device 100 also groups a plurality of state items H1 {speed: 42, position: . . .} relating to the speed and position of car2 into one entry (one record) 110-1 of the KVS 110 as Value.

[0028] Also, the state item H2 of the id "truck4" in FIG. 1(a) corresponds to the high frequency state item set in the high frequency state item list information 120 in FIG. 1(b). The data management device 100 stores one entry (X, * ) and the key is (truck4, * ) The data management device 100 also groups a plurality of state items H2 related to the speed and tire pressure of truck 4 as Value into one entry (one record) 110-5 of the KVS 110.

[0029] Furthermore, the state items L1 "roof (roof): 0 (no roof)" and "winker (turn signal direction): ..." of the id "cabrio3" shown in FIG. 1(a) are data transmissions that occur infrequently, and are not set in the high-frequency state item list information 120 of FIG. 1(b). In this case, the data management device 100 stores an entry 110-3 with a key of (cabrio3, roof) and a value of {0} as an individual entry in the KVS 110, as shown in FIG. 1(b). The data management device 100 also stores an entry 110-4 with a key of (cabrio3, winker) and a value of {...}. In this way, when the low-frequency state item L1 includes multiple states, the data management device 100 creates an individual entry for each state.

[0030] Furthermore, among the state information of id "bus5" in FIG. 1(a), speed corresponds to the high-frequency state item H3 in FIG. 1(b). However, refuel_history (refueling history (date)) corresponds to the low-frequency state item L2, and is not set in the high-frequency state item list information 120 in FIG. 1(b). In this case, the data management device 100, as shown in FIG. 1(b), stores one entry (X, * ) and set the key to (bus5, * ), and stores state item H3 with the value being the speed of bus 5 in one entry (one record) 110-6 of the KVS 110. In addition, the data management device 100 saves a low-frequency state item L2 with the key being (bus5, refuel_history) and the value being refueling history (date and time) in a separate entry 110-7 of the KVS 110.

[0031] Here, if the data management device 100 is configured to access a KVS 110 with a cache, the control unit (CPU) of the data management device 100 prefetches the high-frequency state item list information 120 into the cache at the beginning of event processing. This allows the data management device 100 to read multiple state items that are accessed frequently with a single READ access to the KVS 110.

[0032] (3) In response to dynamic changes in the high-frequency state item list information 120, the data management device 100 asynchronously updates the high-frequency state item list information 120 when writing (WRITE) state items to the KVS 110.

[0033] The high-frequency state item list information 120 is determined by selecting state items from state information sent frequently from IoT devices. The data management device 100 references the high-frequency state item list information 120 and stores the state items for the KVS 110 as groups or individual entries. This allows the data management device 100 to perform best-effort maintenance of the storage structure of the KVS 110 asynchronously with write operations to the KVS 110, and does not incur unnecessary overhead for updating the storage structure of the KVS 110.

[0034] (Digital Twin Overview) Figure 2 is an explanatory diagram of the outline of digital twin processing. An outline of digital twin data processing performed by the data management device 100 will be explained. IoT devices 200 in the real world sequentially transmit their own state information as event messages to the data management device 100, which is a data center. The data center is composed of, for example, a group of servers or a cloud.

[0035] The IoT device 200 is, for example, a connected car or a smartphone, and in the example of Fig. 2, the IoT device 200 is a connected car such as a bus or a truck. The IoT device 200 transmits an event message 210 including state information consisting of an identification ID and a state (condition) to the data management apparatus 100.

[0036] The horizontal axis in FIG. 2 represents time, and shows how the IoT devices 200 transmit their respective state information as event messages 210 to the data management device 100. Looking at the order in which the event messages 210 arrive at the data management device 100, the data management device 100 first receives an event message 210a from a light vehicle (id: kei2). Next, the data management device 100 receives an event message 210b from a truck (id: truck4), and then receives an event message 210c from a bus (id: bus1). The state information included in each of the event messages 210a to 210c includes states set for each IoT device 200, such as speed, current location, number of luggage, and number of occupants. The IoT device 200 transmits an event message 210 with its state updated, for example, every time the state is updated or periodically.

[0037] The data management device 100 operates continuously without system shutdown in order to receive stream-like event messages 210 from a huge number of IoT devices 200 located throughout the country, for example. The vertical axis in the figure shows the digital twins DT corresponding to the respective IoT devices 200.

[0038] The data management device 100 has, for example, multiple user logics 220 (220a to 220n) for each type (attribute) of the IoT device 200. The user logic 220 is a predetermined data processing program that runs on the data management device 100, and updates the state of the digital twin DT based on the state information included in the event message 210 received from the IoT device 200. The user logic 220 may also execute a processing program for each application that is pre-associated with the updated state item, and output logic processing information 230, which is the processing result.

[0039] For example, upon receiving an event message 210a for a light vehicle (id: kei2), the data management device 100 activates the user logic 220a corresponding to the attribute (light vehicle) of the state information, and updates the state of the digital twin DT1. The user logic 220a may also output logic processing information 230a corresponding to the update of the light vehicle's state (in the illustrated example, speed, current location, etc.). For example, the user logic 220a outputs optimal route information to the destination as logic processing information 230a to a navigation function that provides guidance on the light vehicle's travel, based on the updated current state.

[0040] Furthermore, upon receiving an event message 210b for a truck (id: truck4), the data management device 100 activates the user logic 220b corresponding to the attribute (truck) and updates the state of the digital twin DT2. The user logic 220b may also output logic processing information 230b corresponding to updates to the truck state (in the illustrated example, speed, number of packages, etc.). For example, the user logic 220b outputs state information for each update as logic processing information 230b to an operation management function that manages the operation of trucks.

[0041] 2, the data management device 100 constructs a digital twin DT of a real-world object based on state information included in an event message 210 received from an IoT device 200. This digital twin DT corresponds to a physical facility (IoT device 200) that is actually operating in the real world constructed as a model in a virtual space. Each time an event message 210 is received, the data management device 100 updates the state information of the KVS 110, which stores information about the digital twin DT.

[0042] (Example of data management configuration using existing technology) Figure 3 is an explanatory diagram of a data management function of existing technology. For convenience, the same reference numerals are used in Figure 3 to designate components that are the same as those in the embodiment. A data management device 300 located in a data center is configured, for example, by distributed processing of multiple servers. The data management device 300 shown in Figure 3 broadly includes a message queue 301 functional unit and an event processing unit 302 functional unit.

[0043] The message queue 301 receives and processes the event message 210 sent by the IoT device 200, and outputs it to the event processing unit 302. The event processing unit 302 is made up of multiple processing threads (for example, multiple servers) that process the event message 210 in a distributed manner.

[0044] The event processing unit 302 includes the functional units of a message processing unit 311, a logic management unit 312, a cache flush unit 313, and the user logic 220, as well as a cache memory 330 and a KVS 110. The event processing unit 302 sequentially executes the following processes 1 to 3.

[0045] 1. The message processing unit 311 reads an event message for an object belonging to its own thread from the message queue 301 and deserializes it. The message processing unit 311 stores the data of each updated state item included in the event message in the cache memory 330.

[0046] 2. The logic management unit 312 calls the user logic 220 associated with the updated state item. The called user logic 220 executes the read / write processing of state information to the cache memory 330 of the KVS 110, which is performed by the event processing unit 302.

[0047] 3. During a write access, the cache flush unit 313 writes (flushes) the state information data recorded in the cache memory 330 to the KVS 110 for each entry, and then deletes the written data from the cache memory 330. The KVS 110 holds state information for each IoT device 200 that has stream-processed the event message for each distributed node. The event processing unit 302 periodically backs up the data stored in the KVS 110 to the backup server 350 or the like.

[0048] (Example of storing digital twin state information using existing technology) Figures 4A and 4B are explanatory diagrams of examples of storing state information for digital twins using existing technology. The examples of logical state structures for digital twins shown in Figure 4A(a) and Figure 4B(a) are the same as those in Figure 1(a).

[0049] 4A shows method 1 for storing the state of a digital twin. In method 1, the individual state items of each digital twin DT are stored as individual entries in the KVS 110.

[0050] According to method 1, state items of a digital twin can be individually accessed and rewritten in the KVS 110. For example, when processing update data for multiple state items contained in an event message sent from the IoT device 200, only the entries (records) required for the update can be accessed in the KVS 110.

[0051] However, method 1 has the drawback that it is costly to find entries within the KVS 110 for each access. For example, if there are many small entries, as in entry area E, the number of entries per state increases, and each time many small state items are accessed, a READ wait occurs, lengthening the processing time and increasing the cost of accessing the KVS 110.

[0052] Figure 4B shows method 2 for storing the state of a digital twin. In method 2, one KVS entry is created for each digital twin DT, and all state items of the corresponding digital twin are stored in one KVS entry. According to method 2, only one access to the KVS 110 is required per event message sent from the IoT device 200.

[0053] However, this method has the drawback of infrequently used large state items resulting in I / O overhead. For example, state values ​​are not limited to numbers and strings; they can become large with complex content such as list structures. Even large state values ​​are still subject to I / O to the KVS 110. For example, if there is a large, infrequently used state item, it will be read and written every time along with frequently used state items, resulting in high I / O and backup costs. In the example of Figure 4B, the state of DT4's id "bus5" includes two state items: "speed" and "refuel_history (refueling history (date)) F." However, speed is updated frequently, while refueling history is updated infrequently. In Method 2, these two values ​​(speed and refueling history) are set for a single KVS 110 entry (key "bus5"), so the infrequently used state item (refueling history F) causes overhead in KVS 110 I / O, especially when reading.

[0054] Due to its characteristics, the KVS 110 cannot partially update values ​​that have already been written, so updating the state requires deleting and re-adding data, which makes method 2 expensive. Also, if differential backup processing is applied to the KVS 110, all of the multiple states for each entry are updated, making the differential backup meaningless.

[0055] Digital twins have unique circumstances regarding state storage in the KVS 110, as they are "systems directly connected to the real world." For example, in digital twins, state access patterns change dynamically due to changes that occur in the real world. In addition, event message reception must be non-stop, and the replacement of user logic 220 corresponding to event messages must also be performed non-stop and dynamically. For this reason, the KVS 110 cannot optimize the storage structure in advance.

[0056] Here, with regard to state storage in the KVS 110, since the above methods 1 and 2 each have problems, a hybrid that combines methods 1 and 2 is considered.

[0057] Fig. 5 is an explanatory diagram of another example of storing digital twin state information. Fig. 5 corresponds to a hybrid method that combines existing methods 1 and 2.

[0058] In method 3, state items H1, H2, and H3 that are frequently accessed in digital twin "X" are grouped together, and (X, * ) in the KVS 110 (corresponding to method 2). In addition, the state items L1 and L2, which are not accessed frequently, are stored in the KVS 110 as individual entries (corresponding to method 1).

[0059] However, with Method 3, it is not possible to determine in advance whether Method 1 or 2 should be selected for each state item. In this regard, the access pattern for the state of the KVS 110 changes dynamically due to changes in the IoT device 200 that occur in the real world and additions and modifications to the user logic 220. Thus, with Method 3, in order to dynamically determine whether Method 1 or 2 should be selected and to enable the addition of new state items, it is necessary to create and store in the KVS 110 metadata G, which describes the storage structure for the KVS 110. The metadata G in the example of Figure 5 consists of information on entry items to which Method 1 is applied (speed, position, wheel speed, etc.) and information on entry items to which Method 2 is applied (presence or absence of a roof, blinker indication direction, refueling history, etc.).

[0060] Furthermore, in method 3, if an attempt is made to dynamically change the storage structure, it becomes necessary to process the metadata changes and the changes to the storage structures of all related digital twins exclusively and collectively, resulting in a large overhead.

[0061] The data management method of the embodiment solves the problems that each of the methods 1 to 3 has. In the embodiment, the following processes (1) to (3) are carried out.

[0062] (1) The data management device 100 collects data on the frequency of access to state items, performs predetermined statistical processing, and generates information 120 on frequently accessed state items (list of frequently accessed state items).

[0063] (2) The data management device 100 refers to the high-frequency state item list information 120, groups high-frequency state items that frequently access the KVS 110 in a specific digital twin X, and defines the grouped states as (X, * ) in the KVS 110. On the other hand, the data management device 100 stores other state items, that is, less frequently accessed state items, in the KVS 110 as individual entries.

[0064] (3) In response to dynamic changes in the high-frequency state item list information 120, the data management device 100 asynchronously updates the high-frequency state item list information 120 when writing state items to the KVS 110.

[0065] The data management device 100 prefetches into the cache memory 330 high-frequency state items that are accessed frequently by the KVS 110 at the beginning of each event message processing operation. Furthermore, the data management device 100 reads from the cache memory 330 on the second and subsequent accesses to the KVS 110, so no access to the KVS 110 occurs. For example, even if the number of high-frequency state items increases, only one access to the KVS 110 is required for each event message processing operation. Furthermore, by prefetching into the cache memory 330 at the beginning of this first processing operation, the waiting time can be reduced.

[0066] On the other hand, for infrequent state items that do not frequently access the KVS 110, the data management device 100 accesses the KVS 110 each time an event message is processed. However, because the frequency of access to the KVS 110 is low, it does not have a significant impact on overall performance. For example, infrequent state items are not included in entries that access the KVS 110 as a group like high-frequency state items, so it is possible to prevent unnecessary I / O and data transfer.

[0067] In this way, the data management device 100 references the high-frequency state item list information 120 and controls access to the KVS 110 by frequency. This enables the data management device 100 to shorten the time it takes to access state information stored in the KVS 110, thereby speeding up digital twin event processing.

[0068] Furthermore, in order to dynamically change the storage structure of state items, the data management device 100 does not require metadata indicating the storage structure of the KVS 110 in real time. The data management device 100 of this embodiment executes the high-frequency state item list information 120 asynchronously during normal write operations to the KVS 110. As a result, the data management device 100 does not need to access metadata, as in method 3, and no unnecessary overhead is generated for maintenance such as updating the storage structure of the KVS 110.

[0069] For example, the states included in the high-frequency state item list information 120 are often updated when an event message is received from an IoT device 200 in a digital twin. For this reason, the data management device 100 executes the process for receiving an event message and the update process for maintaining the high-frequency state item list information 120 in a superimposed manner, and performs a single write process to the KVS 110 of the results of these superimposed processes. This prevents unnecessary write processes from occurring to the KVS 110 due to maintenance of the high-frequency state item list information 120. The maintenance is performed to dynamically optimize the storage structure of the KVS 110 by updating the high-frequency state items set in the high-frequency state item list information 120 in response to changes in the high-frequency state items over time.

[0070] Furthermore, when the access pattern to state items changes, the data management device 100 changes the state storage structure of the KVS 110 accordingly, thereby always maintaining an optimal storage structure of the KVS 110. In this case, no additional access to the KVS 110 occurs in order to change the storage structure, and changing the state storage structure does not cause delays in message processing or the like.

[0071] (Functions of the Data Management Device of the Embodiment) Fig. 6 is an explanatory diagram of the functions of a data management device according to an embodiment. The data management device 100, which is placed in a data center, is configured, for example, by distributed processing of a plurality of servers. The data management device 100 shown in Fig. 6 mainly includes a message queue 601 functional unit and an event processing unit 602 functional unit.

[0072] The message queue 601 receives and processes the event message 210 sent by the IoT device 200, and outputs it to the event processing unit 602. The event processing unit 602 is made up of multiple processing threads (for example, multiple servers) that perform distributed processing of the event message.

[0073] The event processing unit 602 includes the following functional units: a prefetch unit 610 , a message processing unit 611 , a plug-in management unit 612 , a cache flush unit 613 , a grouping gateway unit 620 , and a user logic 220 , as well as a cache memory 630 and a KVS 110 .

[0074] The prefetch unit 610 performs prefetching. For example, if the high frequency state item list information 120 exists, the prefetch unit 610 prefetches at the beginning of the event message processing for the object belonging to its own thread. At this time, the prefetch unit 610 prefetches the entry (X, * ) is prefetched from the KVS 110 to the cache memory 630.

[0075] The message processing unit 611 deserializes the event message read from the message queue 601. The message processing unit 611 outputs to the grouping gateway unit 620 the data of each state item whose state has been updated and is included in the event message.

[0076] The plug-in management unit 612 calls the user logic 220 associated with the updated state item. The called user logic 220 outputs access (READ / WRITE) information for each state item to the grouping gateway unit 620.

[0077] When writing state information to the cache memory 630, the cache flush unit 613 performs flush processing to write this state information data for each entry to the KVS 110. The cache flush unit 613 then deletes the written data from the cache memory 630.

[0078] The grouping gateway unit 620 controls access to read and write to the KVS 110. The grouping gateway unit 620 includes a write unit 621, a read unit 622, an access statistics information unit 623, a high frequency determination unit 624, and a high frequency state item list creation unit 625.

[0079] The write unit 621 writes state information to the cache memory 630. The read unit 622 reads state information from the cache memory 630 and processes it.

[0080] The access statistical information unit 623 obtains predetermined access statistical information relating to the access frequency of the WRITE unit 621 and the READ unit 622 based on state updates, and outputs the access statistical information to the high frequency determination unit 624. For example, the statistical value of the access statistical information is the update frequency of the state item.

[0081] The high frequency determination unit 624 determines high frequency state items from the state information written to the WRITE unit 621 by state updates based on the statistical values ​​of the access statistical information. The high frequency state item list creation unit 625 creates high frequency state item list information 120 that lists multiple high frequency state items determined to be high frequency by the high frequency determination unit 624. The high frequency state item list creation unit 625 creates the high frequency state item list information 120, for example, periodically.

[0082] The grouping gateway unit 620 refers to the high-frequency state item list information 120 created by the high-frequency state item list creation unit 625, and accesses the cache memory 630. At this time, when the WRITE unit 621 writes to the cache memory 630, the grouping gateway unit 620 groups multiple high-frequency state items into one entry, and creates individual entries for low-frequency state items, as described above.

[0083] The write operation by the write unit 621 of the grouping gateway unit 620 and the read operation by the read unit 622 will be described below.

[0084] (WRITE operation) 7 is an explanatory diagram of a processing example of a WRITE operation of the data management device. Fig. 7 shows an operation example in order when the IoT device 200 performs a WRITE operation on the bus to write speed state information (bus5, speed).

[0085] As described above, every time the data management device 100 receives an event message, the prefetch unit 610 at the beginning of the process reads the state items (bus5, * ) is prefetched from the KVS 110 to the cache memory 630.

[0086] 1. First, the user logic 220 makes a WRITE access to the grouping gateway unit 620 for the state item speed (updated value is 42).

[0087] 2. The grouping gateway unit 620 records the accesses of the state item speed in the access statistics information 640 of the access statistics information unit 623.

[0088] 3.1 The grouping gateway unit 620 checks whether a state item is a high-frequency state item using the high-frequency determination unit 624. For example, if high-frequency state item list information 120 exists, the high-frequency determination unit 624 checks whether the state item "speed" is included in this high-frequency state item list information 120. Here, immediately after starting or restarting the data management device 100, the high-frequency state item list information 120 may not exist. 3.2 Grouping If the state item "speed" does not exist in the high frequency state item list information 120, the gateway unit 620 checks whether or not the state items of the state information prefetched onto the cache memory 630 include "speed".

[0089] 4. The grouping gateway unit 620 determines that speed is a high-frequency state item if it exists in the high-frequency state item list information 120 or the cache memory 630. In this case, the WRITE unit 621 of the grouping gateway unit 620 updates the grouped state item in the cache memory 630 with the value "42" of the given state item speed.

[0090] 5.1 On the other hand, if speed does not exist in the high-frequency state item list information 120 or the cache memory 630, the grouping gateway unit 620 deletes the state item speed from the grouped state items in the cache memory 630. For example, if speed is recorded in the grouped state items in the cache memory 630, the state item speed that is no longer frequently accessed is deleted to keep up with changes in state items that are accessed frequently.

[0091] 5.2 The grouping gateway unit 620 stores (bus5, speed) as a new individual entry in the cache memory 630. The grouping gateway unit 620 suspends writing to the KVS 110 until flush control is performed.

[0092] 6. After this, the cache flush unit 613 writes out to the KVS 110 the high-frequency state item H1 grouped by the grouping gateway unit 620 and the low-frequency state item L1 of the individual entry.

[0093] The data management device 100 also performs the same procedures as those in 1. to 6. above when performing a WRITE operation on other state information, for example, (bus5, latitude). * Regardless of how many high-frequency state items H1 included in the KVS 110 are accessed, one prefetch and one flush are executed for each access to the KVS 110.

[0094] The data management device 100 executes the following process when the user logic 220 makes a WRITE access to the low-frequency state item L1 (bus5, refuel_history). After confirming that the state item (bus5, refuel_history) is not a high-frequency state item, the grouping gateway unit 620 reads the grouped state item (bus5, * ) and the grouping gateway unit 620 refers to (bus5, * ) does not include refuel_history. Here, the grouping gateway unit 620 checks whether the high frequency state item list information 120 has been changed recently and whether (bus5, * ) may not have been updated. Then, the grouping gateway unit 620 checks whether refuel_history is (bus5, * ), the current WRITE of (bus5, refuel_history) is not performed. *If refuel_history is not included in bus5, then (bus5, refuel_history) is stored in cache memory 630.

[0095] The above WRITE operation allows the data management device 100 to operate for a certain period of time even if the high frequency state item list information 120 does not exist. Also, the high frequency state item list information 120 does not need to be made non-volatile for fault tolerance. For example, when the data management device 100 is started up or restarted after recovery, (X, * ) simply becomes empty, and normal operation is possible. Furthermore, the data management device 100 updates the high-frequency state item list information 120 each time an event message is processed, and does not need to access the KVS 110 to change the high-frequency state item list information 120. The event processing unit 602 periodically backs up the data stored in the KVS 110 to a backup server 650 or the like.

[0096] (READ operation) Fig. 8 is an explanatory diagram of a processing example of a READ operation of the data management device. Fig. 8 shows an operation example in which the IoT device 200 reads state information (bus5, refuel_history) of the refueling history on the bus.

[0097] 1. First, the user logic 220 performs READ access to the grouping gateway unit 620 for the state item refuel_history.

[0098] 2. The grouping gateway unit 620 records the accesses in the state item refuel_history in the access statistics information 640 of the access statistics information unit 623.

[0099] 3. The grouping gateway unit 620 retrieves the grouped state items (bus5, *) The data management device 100 completes the process if the state item refuel_history is found in the cache memory 630. On the other hand, if refuel_history is not found, the data management device 100 executes the process in 4.

[0100] 4. The READ unit 622 of the grouping gateway unit 620 reads the state information (bus5, refuel_history) from the cache memory 630. If the state information is not found, the data management apparatus 100 executes the process in 5.

[0101] 5. The user logic 220 reads the state information (bus5, refuel_history) from the KVS 110. At this time, the user logic 220 also stores the read state information (bus5, refuel_history) in the cache memory 630.

[0102] In the case of the above-mentioned READ, the data management device 100 does not need to access the high-frequency state item list information 120, and can correctly READ even if there is no corresponding state item in the high-frequency state item list information 120. Furthermore, the data management device 100 can READ the corresponding state information even if the state information set in the high-frequency state item list information 120 does not match the storage layout of the state information in the KVS 110.

[0103] (Example of determining the list of high frequency state items) 9 is an explanatory diagram of an example of a process for determining high-frequency state items in the data management device 100. An example of determining the high-frequency state item list information 120 by the high-frequency determining unit 624 of the data management device 100 will be described.

[0104] The high frequency determination unit 624 collects information to determine high frequency state items, such as the number of events processed by the data management device 100, the number of accesses for each state item, the total size of WRITE (written) data, etc. The high frequency determination unit 624 calculates the state usage and access cost based on this collected information.

[0105] The high frequency determination unit 624 calculates the usage a of a state by dividing the number of accesses by the number of events. The access cost b is calculated by dividing the total data size by the number of accesses. The entire square (area) shown in Figure 9(a) indicates the access cost of one state, and the shaded area indicates the usage.

[0106] The high frequency determination unit 624 determines the high frequency state item based on, for example, the following: In the example of Fig. 9(b), the high frequency determination unit 624 determines the state item whose ratio of utilization a (shaded area) to access cost b (entire area) of one state is greater than a preset threshold and which contains the largest number of state items as the high frequency state item.

[0107] In the example of FIG. 9(b), the high frequency determination unit 624 determines the four state items on the left (wheel1_speeed to speed) to be high frequency state items, and determines that the one state item on the right (refuel_history) is not a high frequency state item. For example, the high frequency determination unit 624 determines the state item speed_history, which has both a high access cost b and a high usage a, and the state item winker, which has a low access cost b and a low usage a, as high frequency state items. In contrast, the high frequency determination unit 624 does not determine the state item refuel_history, which has a high access cost b but a low usage a ratio, as a high frequency state item. The high frequency determination unit 624 changes and determines the high frequency state items in the high frequency state item list information 120 each time a WRITE process is executed.

[0108] (Examples of digital twin processing by type) To prevent the amount of statistical data from becoming too large, the data management device 100 may classify digital twins by type and have one high-frequency state item list information 120 for each type. The type corresponds to the type or attribute of the IoT device 200.

[0109] 10 is a diagram showing a management table for each type of digital twin in the data management device. The data management device 100 executes different user logic 220 depending on the type, attribute, etc. of the LoT device 200. The data management device 100 classifies digital twins by type and has high-frequency state item list information 120 for each type.

[0110] FIG. 10(a) shows a digital twin and type relationship management table 1001. In each record of the digital twin and type relationship management table 1001, a type name is set for each digital twin ID. For example, the type name "car" is set for the digital twin ID "car2," and the type name "bus" is set for the digital twin ID "bus5." The digital twin ID corresponds to the ID of the state information included in the event message received by the data management device 100.

[0111] 10(b) shows a management table 1002 for type names and user logic to be executed. In each record of the management table 1002 for type names and user logic to be executed, the user logic to be executed for each type name is set. For example, the user logic to be executed, "User Logic 1," is set for the type "Car," and the user logic to be executed, "User Logic 2," is set for the type "Bus."

[0112] The data management device 100 stores management tables 1001 and 1002 shown in Figures 10(a) and 10(b) in a memory or the like. When the data management device 100 receives an event message, it refers to Figure 10(a) based on the ID of the state information included in the event message and determines the type name corresponding to the digital twin ID. Next, it refers to Figure 10(b) and determines the user logic to be executed that corresponds to the type name. This allows the data management device 100 to execute user logic 220 for each type.

[0113] 11 is an explanatory diagram of user logic selection by type of data management device. In the example shown in Fig. 11, the user logic includes a car user logic 220a, a bus user logic 220b, and a truck user logic 220c corresponding to three user logics 220 by type.

[0114] Corresponding to the type-specific user logic 220, the grouping gateway unit 620 has access statistical information 640 for each type and state item list information 120. An access statistical information unit 623 of the grouping gateway unit 620 creates vehicle access statistical information 640a, bus access statistical information 640b, and truck access statistical information 640c. A high-frequency state item list creation unit 625 of the grouping gateway unit 620 creates vehicle high-frequency state item list information 120a, bus high-frequency state item list information 120b, and truck high-frequency state item list information 120c.

[0115] When the data management device 100 receives an event message, it references the management tables 1001 and 1002 in FIGS. 10(a) and 10(b) and executes one user logic 220 (220a to 220c) corresponding to the type corresponding to the processing of the event message. The data management device 100 then selects access statistical information 640 (640a to 640c) and high-frequency state item list information 120 (120a to 120c) corresponding to the type of user logic 220 (220a to 220c) to be executed. The grouping gateway unit 620 of the data management device 100 performs WRITE and READ access using the access statistical information 640a to 640c corresponding to the type-specific user logic 220a to 220c and the high-frequency state item list information 120a to 120c.

[0116] With the above configuration, the data management device 100 can execute different user logic 220 depending on the type of digital twin, attributes, etc. Furthermore, since the amount of statistical data (access statistical information 640, high frequency state item list information 120) is divided by type when processing event messages, it is possible to prevent a single piece of statistical information from becoming overwhelming, and it becomes possible to easily manage the statistical information.

[0117] (Example of statistical information managed by the data management device) Fig. 12 is a chart showing examples of access statistical information by type of data management device. The access statistical information unit 623 creates access statistical information for cars 640a shown in Fig. 12(a), access statistical information for buses 640b shown in Fig. 12(b), and access statistical information for trucks 640c shown in Fig. 12(c) for the types explained in Fig. 11.

[0118] For example, in the vehicle access statistical information 640a, the access statistical information unit 623 sets values ​​for the following items for each state item: first detection index, last detection index, number of accesses, and total size. The access statistical information unit 623 also sets an event index. The event index is the number of event messages processed by the data management device 100.

[0119] The first detection index and the last detection index are storage areas in the message queue 601 when the event processing unit 602 reads one event message from the message queue 601. The access statistics information unit 623 checks the last detection index for each event processing. The access statistics information unit 623 calculates the number of events based on the values ​​of the first detection index and the event index. The access statistics information unit 623 also calculates the access frequency based on the values ​​of the first detection index and the last detection index and the number of accesses.

[0120] Fig. 13 is a chart showing an example of high frequency state item list information by type managed by the data management device. The high frequency state item list creation unit 625 creates high frequency state item list information 120a for vehicles in Fig. 13(a), high frequency state item list information 120b for buses in Fig. 13(b), and high frequency state item list information 120c for trucks in Fig. 13(c) for the types in Fig. 11.

[0121] For example, to explain the contents of the high frequency state item list information 120a for vehicles, the high frequency state item list creating unit 625 sets the type to vehicle, and sets position (current location) and speed (speed) values ​​as state items.

[0122] (Example of hardware configuration of data management device) 14 is a diagram showing an example of the hardware configuration of the data management device 100. The data management device 100 can be configured as a computer such as a server made up of general-purpose hardware shown in FIG.

[0123] The data management device 100 has a CPU (Central Processing Unit) 1401, a memory 1402, and a network I / F (Interface) 1403. The data management device 100 also has a recording medium I / F 1404, a recording medium 1405, a portable recording medium I / F 1406, and a portable recording medium 1407. The components are connected to each other via a bus 1400.

[0124] The CPU 1401 functions as a control unit that controls the entire data management device 100. The CPU 1401 may have multiple cores. The memory 1402 includes, for example, a read-only memory (ROM), a random access memory (RAM), and a flash ROM. Specifically, for example, the flash ROM stores an OS program, the ROM stores application programs, and the RAM is used as a work area for the CPU 1401. The programs stored in the memory 1402 are loaded into the CPU 1401, causing the CPU 1401 to execute the coded processes.

[0125] The network I / F 1403 is connected to the network NW via a communication line, and is connected to an external computer via the network NW. For example, when the data management device 100 is configured with multiple servers 301, 302, and 350 as shown in FIG. 3, the external computer is connected to each of the servers 301, 302, and 350 via the network NW. The network I / F 1403 serves as an interface between the network NW and the inside of the device, and controls the input and output of data from the external computer. For example, a modem or a LAN adapter can be used as the network I / F 1403.

[0126] The recording medium I / F 1404 controls reading / writing of data from / to the recording medium 1405 under the control of the CPU 1401. The recording medium 1405 stores data written under the control of the recording medium I / F 1404. Examples of the recording medium 1405 include a magnetic disk and an optical disk.

[0127] The portable recording medium I / F 1406 controls reading / writing of data from / to the portable recording medium 1407 under the control of the CPU 1401. The portable recording medium 1407 stores data written under the control of the portable recording medium I / F 1406. Examples of the portable recording medium 1407 include a CD (Compact Disc)-ROM, a DVD (Digital Versatile Disk), and a USB (Universal Serial Bus) memory.

[0128] In addition to the above-mentioned components, the data management device 100 may also include, for example, an input device, a display, and the like.

[0129] The function of the event processing unit 602 shown in Fig. 6 can be realized by the CPU 1401 shown in Fig. 14 executing a program. The function of the cache memory 630 shown in Fig. 6 can be realized by using, for example, the memory 1402 shown in Fig. 14. The function of the KVS 110 shown in Fig. 6 can be realized by using, for example, the recording medium 1405 and portable recording medium 1407 shown in Fig. 14. The access statistical information 640 and high frequency state item list information 120 shown in Fig. 6 can be provided, for example, in the recording areas of the memory 1402, recording medium 1405, and portable recording medium 1407 shown in Fig. 14.

[0130] (Example of processing by data management device) Next, an example of processing by the data management device 100 will be described with reference to Fig. 15 to Fig. 20. The processing shown in Fig. 15 to Fig. 20 is executed by the CPU 1401, which is the control unit of the data management device 100.

[0131] (Overall processing example) 15 is a flowchart of an example of the overall processing of an event performed by the data management device 100. The data management device 100 continues to perform loop processing for each step shown in FIG. 15 (steps S1502 to S1508) until a request to stop the processing is received.

[0132] First, the data management device 100 waits to receive an event message (step S1502), and upon receiving the event message, proceeds to processing in step S1503. When the data management device 100 receives an event message, it sets the ID of the digital twin corresponding to the message destination to the digital twin type object $0.

[0133] Next, the data management device 100 deserializes the received event message (step S1503). Next, the data management device 100 deserializes the grouped state items ($0, * ) is prefetched from the KVS 110 (step S1504).

[0134] Next, the data management device 100 performs message processing for the received event message (step S1505). Next, the data management device 100 selects the user logic 220 registered with type $0 and executes the processing of the selected user logic 220 (step S1506). The selected user logic 220 executes the WRITE access or READ access processing for the event message.

[0135] Next, the data management device 100 creates high-frequency state item list information 120 from the access statistical information 640 (step S1507). For example, the data management device 100 creates the high-frequency state item list information 120 periodically at predetermined intervals, rather than each time an event message is processed. After this, the data management device 100 flushes the cache data in the cache memory 630 to the KVS 110 and updates the KVS 110 (step S1508). After this, in step S1509, if there is no request to stop the device, the data management device 100 returns to the processing of step S1501 to continue the above processing, and if there is a request to stop processing, the data management device 100 ends the above processing (step S1509).

[0136] (Message processing example) Fig. 16 is a flowchart of an example of message processing by the data management device. Fig. 16 shows the details of the processing of step S1505 in Fig. 15. The processing content shown in Fig. 16 corresponds to the message processing of existing digital twins. The data management device 100 loops the processing shown in Fig. 16 (step S1602) until processing for each state item described in the event message is completed.

[0137] In step S1602, the data management device 100 updates the value of the state item managed in the digital twin with the value included in the event message (WRITE access, step S1602). Note that if the state item appears for the first time, the data management device 100 creates a new state item on the digital twin. After this, in step S1603, if the processing for each state item described in the event message is incomplete, the data management device 100 returns to the processing of step S1601 to continue processing. On the other hand, if the processing for all state items is completed, the data management device 100 ends the above processing (step S1603).

[0138] (WRITE access processing example) Figure 17 is a flowchart of an example of WRITE access processing performed by the data management device. Figure 17 shows an example of processing in which WRITE access is made to state item $x of digital twin object $0 under the control of user logic 220 in step S1506 of Figure 15. First, the data management device 100 records access to $x in the access statistics information 640 (step S1701). The processing of step S1701 is executed once for each state item and once for each event to be processed.

[0139] Next, the data management device 100 references the high-frequency state item list information 120 and determines whether high-frequency state item list information exists (step S1702). If a high-frequency state item list exists in the high-frequency state item list information 120 (step S1702: Yes), the data management device 100 proceeds to processing in step S1703. On the other hand, if a high-frequency state item list does not exist in the high-frequency state item list information 120 (step S1702: No), the data management device 100 proceeds to processing in step S1706.

[0140] In step S1703, the data management device 100 determines whether or not the state item $x is included in the high-frequency state item list information 120 (step S1703). If the state item $x is included in the high-frequency state item list information 120 (step S1703: Yes), the data management device 100 proceeds to the processing of step S1704. On the other hand, if the state item $x is not included in the high-frequency state item list information 120 (step S1703: No), the data management device 100 proceeds to the processing of step S1708.

[0141] In step S1704, the data management system 100 checks whether $x is the grouped state item in the cache memory 330. * It is determined whether $x is included in the cache memory 330 (step S1704). * If $x is included in the cache memory 330 (step S1704: Yes), the data management system 100 proceeds to the process of step S1705. * If not included (step S1704: No), the data management device 100 proceeds to the process of step S1709.

[0142] In step S1705, the data management device 100 calculates $x and its value. * The data is written to the cache memory 330 (step S1705), and the above processing is terminated.

[0143] In step S1706, the data management system 100 checks whether $x is in the cache memory 330. * It is determined whether $x is included in the cache memory 330 (step S1706). * If $x is included in the cache memory 330 (step S1706: Yes), the data management system 100 proceeds to the process of step S1705. * If not included (step S1706: No), the data management apparatus 100 proceeds to the process of step S1707.

[0144] In step S1707, the data management system 100 stores $x and its value in the cache memory 330 (step S1707), and ends the above processing.

[0145] In step S1708, the data management device 100 sets $x * (step S1708), and the process proceeds to step S1707. * If $x does not exist in the data management system 100, the data management system 100 does not execute the deletion process.

[0146] In step S1709, the data management device 100 deletes the grouped state item ($0, $x) from the KVS 110 and the cache memory 330 (step S1709), and proceeds to the processing of step S1705.

[0147] (Example of READ access processing) Figure 18 is a flowchart of an example of READ access processing performed by the data management device. Figure 18 shows an example of processing for READ access to state item $x of digital twin $0 under the control of user logic 220 in step S1506 of Figure 15. First, the data management device 100 records access to $x in the access statistics information 640 (step S1801). The processing of step S1801 is executed once for each state item and once for each event to be processed.

[0148] Next, the data management apparatus 100 checks whether the state item $x is in the cache memory 330. * It is determined whether the state item $x is included in the cache memory 330 (step S1802). * If the state item $x is included in the cache memory 330 (step S1802: Yes), the data management system 100 proceeds to the process of step S1803. * If not included (step S1802: No), the data management apparatus 100 proceeds to the process of step S1804.

[0149] In step S1803, the data management apparatus 100 sets the state item $x * (step S1803), and the above processing ends.

[0150] In step S1804, the data management device 100 determines whether or not the state item $x is included in the cache memory 330 (step S1804). If the state item $x is included in the cache memory 330 (step S1804: Yes), the data management device 100 proceeds to the processing of step S1805. On the other hand, if the state item $x is not included in the cache memory 330 (step S1804: No), the data management device 100 proceeds to the processing of step S1806.

[0151] In step S1805, the data management system 100 reads the state item $x from the cache memory 330 (step S1805), and ends the above processing.

[0152] In step S1806, the data management device 100 reads the grouped state item ($0, $x) from the KVS 110 (step S1806), and ends the above processing.

[0153] (Example of creating a list of high frequency state items) 19A to 19C are flowcharts showing an example of a process for creating a high-frequency state item list information performed by a data management device. Fig. 19A shows the overall process for creating a high-frequency state item list information, and shows details of the process of step S1507 in Fig. 15.

[0154] 19A, the data management device 100 first determines whether a predetermined time has passed since system startup or the previous creation of high-frequency state item list information 120 (step S1901). If the predetermined time has passed since system startup or the previous creation of high-frequency state item list information 120 (step S1901: Yes), the data management device 100 proceeds to processing in step S1902. On the other hand, if the system has not started or the predetermined time has not passed since the previous creation of high-frequency state item list information 120 (step S1901: No), the data management device 100 proceeds to processing in step S1905.

[0155] In step S1902, the data management device 100 performs loop processing by type using the access statistical information 640 by type of digital twin (step S1902). During the loop processing, processing is performed to create high frequency state item list information 120 of one type (step S1903).

[0156] Thereafter, in step S1903, if the process of creating all types of high-frequency state item list information 120 is incomplete, the data management device 100 returns to the process of step S1902 to continue processing. On the other hand, if the process of creating all types of high-frequency state item list information 120 is completed, the data management device 100 ends the above process (step S1904).

[0157] In step S1905, the data management device 100 does not create the high frequency state item list information 120 (if it has already been created, it does not update it, step S1905). After this, the data management device 100 ends the above processing.

[0158] Figures 19B and 19C show the details of the process in step S1903 of FIG. 19A. In FIG. 19B, the data management device 100 acquires each data of a table S of one type of access statistical information 640, an event index E, and a utilization threshold C, and executes the following process. As shown in FIG. 12, in the table S of the access statistical information 640, data of state item (name) n, first detection index f, last detection index l, access count c, and size total s are stored in a columnar manner for each vertical state item. The data management device 100 uses the last detection index l to execute the record only once in one event message.

[0159] First, the data management device 100 sets the total cost TC (total cost) to 0 and the weighted utilization WU (weightened usefulness) to 0 (step S1910). Next, the data management device 100 performs a loop process for each row (n, f, l, c, s) of the table S for a plurality of state items (step S1911).

[0160] In this loop process, the data management device 100 calculates the utilization U and the access cost AC for one state item (step S1912). The data management device 100 calculates the utilization (access probability) U (usefulness) based on, for example, c / (E - f). The data management device 100 calculates the access cost AC (access cost) based on, for example, s / c as the average size of the state item.

[0161] Next, the data management device 100 determines whether the utilization U exceeds the threshold C (C < U) (step S1913). If the utilization U exceeds the threshold C (step S1913: Yes), the data management device 100 proceeds to the process of step S1914. On the other hand, if the utilization U does not exceed the threshold C (step S1913: No), the data management device 100 proceeds to the process of step S1915.

[0162] In step S1914, the data management device 100 adds n to the output F of the high frequency state item list information 120 (table S) of the type being processed. The data management device 100 also deletes (n, f, l, c, s) from table S. The data management device 100 also calculates the weighted utilization WU, for example, based on WU + U × AC. The data management device 100 also calculates the total cost TC, for example, based on TC + AC (step S1914).

[0163] In step S1915, if the processing for all other types is incomplete, the data management device 100 returns to the processing of step S1911 to continue the above processing, and if the processing for all types is completed, proceeds to the processing of step S1920 (Figure 19C).

[0164] 19C, the data management device 100 performs a loop process on each row (n, f, l, c, s) of table S in a predetermined order, for example, in ascending order starting from the smallest state item (step S1920). For example, the data management device 100 determines the order based on (1-c / (Ef)) x s / c.

[0165] In this loop processing, the data management device 100 calculates the utilization U, access cost AC, weighted utilization WU', and total cost TC' for one state item (step S1921). The data management device 100 calculates the utilization U based on, for example, c / (Ef). The data management device 100 calculates the access cost AC based on, for example, s / c. The data management device 100 calculates the weighted utilization WU' based on, for example, WU+U×AC. The data management device 100 calculates the total cost TC' based on, for example, TC+AC.

[0166] Next, the data management device 100 determines whether the overall utilization (WU’ / TC’) exceeds the threshold value C (C < WU’ / TC’) (step S1922). If the overall utilization exceeds the threshold value C (step S1922: Yes), the data management device 100 proceeds to the process of step S1923. On the other hand, if the overall utilization does not exceed the threshold value C (step S1922: No), the data management device 100 proceeds to the process of step S1924.

[0167] In step S1923, the data management device 100 adds n to the output F of the high-frequency state item list information 120 (table S) of the type being processed. Also, the data management device 100 sets the overall utilization WU’ as the weighted utilization WU. Further, the data management device 100 sets the total cost TC’ as the total cost TC (step S1923).

[0168] In step S1924, if the processing for all other types is incomplete, the data management device 100 returns to the process of step S1920 to continue the above processing. If the processing for all types is completed, the data management device 100 ends the above processing. The data management device 100 outputs the high-frequency state item list information F (120) by type at the end of this processing and proceeds to the process of step S1904 in FIG. 19A.

[0169] (Flash process of cache memory) FIG. 20 is a flowchart of an example of a flash process performed by the data management device. FIG. 20 shows an example of the flash process of the cache memory 630 in step S1508 of FIG. 15. First, the data management device 100 determines whether the high-frequency state item list information 120 exists (step S2001). If the high-frequency state item list information 120 exists (step S2001: Yes), the data management device 100 proceeds to the process of step S2002. On the other hand, if the high-frequency state item list information 120 does not exist (step S2001: No), the data management device 100 proceeds to the process of step S2003.

[0170] In step S2002, the data management device 100 determines whether the probability (e.g., the above-mentioned access probability U) of each high-frequency state item set in the high-frequency state item list information 120 is lower than a predetermined threshold (e.g., 3%) (step S2002). If the probability of the state item exceeds the threshold (step S2002: No), the data management device 100 proceeds to the processing of step S2003 because the state item is high-frequency. On the other hand, if the probability of the state item is below the threshold (step S2002: Yes), the data management device 100 proceeds to the processing of step S2004, where the state item is low-frequency and this state item is excluded.

[0171] In step S2004, the data management device 100 * ) (step S2004). In step S2005, the data management device 100 performs a WRITE access to ($0, $x) again (see FIG. 17, step S2005). This WRITE access deletes the low-frequency state item from the grouped high-frequency state items. In step S2006, the data management device 100 performs a loop process on each entry $x of ($0, $x). * If there is any processing remaining for each entry $x of ($0, * ) is completed, the process proceeds to step S2003.

[0172] In step S2003, the data management apparatus 100 writes (flushes) the updated entry in the cache memory 630 to the KVS (step S2003), and ends the above processing.

[0173] (Comparison of existing technology and embodiment) Next, the present embodiment will be compared with the existing technology. First, the READ / WRITE process will be compared with FIG. 21A and FIG. 21B.

[0174] FIG. 21A is an explanatory diagram of a READ / WRITE process according to the existing technology. In FIG. 21A, the state items with high frequency described above are grouped (X, * ) An example of method 3 is shown below.

[0175] In the data management device 300 of the existing technology, the storage location of the KVS 110 entry is determined by the metadata 2101. For this reason, in the existing technology, the metadata 2101 and the entry 2102 are each cached in the cache memory 330. In this case, when the state is updated by the user logic 220, the gateway unit 2210 determines the entry structure in the cache memory 330 based only on the layout of the metadata, and performs READ / WRITE access to the state items of the KVS 110. In the existing technology, the layout indicated by the metadata (storage structure of the KVS 110) is required. Furthermore, when changing the grouping of state items based on the frequency of the state items, it is necessary to process the change in the storage structure of the KVS 110 and the change in the metadata together and exclusively, which results in a large overhead in data processing.

[0176] Figure 21B is an explanatory diagram of READ / WRITE processing according to an embodiment. In the READ processing of Figure 21B(a), the grouping gateway unit 620 of the data management device 100 checks the grouping state item. At this time, the data management device 100 prefetches the entry 2102 into the cache memory 630 in advance, allowing access in a short time. Furthermore, if there is no grouping state item, the grouping gateway unit 620 can access the KVS 110 as an individual entry.

[0177] In the WRITE process of FIG. 21B(b), if there is high frequency state item list information 120, the grouping gateway unit 620 of the data management device 100 can refer to it and perform maintenance on the grouped state items.

[0178] As described above, according to the embodiment, by grouping entries of frequently occurring state items, even if there are a large number of state items, only one access to the KVS 110 is required for each event message processing, thereby improving access efficiency. Furthermore, by prefetching event information in advance at the beginning of event message processing, waiting time can be further reduced. Furthermore, by treating infrequent state items as individual entries, access occurs each time, but because the access is infrequent, it does not affect overall performance. Furthermore, because infrequent state items are not included in the grouped frequently occurring state items, unnecessary I / O and data transfer can be prevented. These features shorten the time required to access state information stored in the KVS 110, thereby speeding up digital twin event processing.

[0179] Next, the maintenance processing of grouped state items will be compared using FIGS. 22A and 22B.

[0180] 22A is an explanatory diagram of maintenance processing of grouped state items according to the existing technology. When performing maintenance processing of grouped state items, the event processing unit 302 of the data management device 300 of the existing technology needs to exclusively execute the following steps 1 to 3.

[0181] 1. The input of the event message 210 is temporarily stopped (step S2201). Also, the data backup of the KVS 110 to the backup server 650 is temporarily stopped (step S2202).

[0182] 2. Update the metadata of the KVS 110 (step S2203). In the example shown, the metadata v3 is updated with the update metadata v4.

[0183] 3. The storage structure of all digital twins classified as the same type is updated collectively (step S2204). The processing load of this process increases as the number of digital twins increases.

[0184] 22B is an explanatory diagram of the maintenance process of the grouping state item according to the embodiment. In the embodiment, the grouping gateway unit 620 of the data management device 100 1. The high frequency determination unit 624 simply determines the high frequency state items (step S2210).

[0185] The data management device 100 then performs the actual maintenance asynchronously for each digital twin. The data management device 100 performs this maintenance in conjunction with a specified WRITE process. Each digital twin can have a different storage structure. Here, the states included in the high-frequency state item list information 120 are often updated when the digital twin receives an event message from the IoT device 200. For this reason, the data management device 100 executes the process when an event message is received and the update process for maintaining the high-frequency state item list information 120 in a superimposed manner, and performs a single write process to the KVS 110 of the results of these superimposed processes. This prevents unnecessary WRITE processes from occurring to the KVS 110 for the maintenance of the high-frequency state item list information 120.

[0186] In the embodiment described above, when a specific data processing program processes data, the data management device 100 creates a list of frequently accessed state items based on the access frequency of the data store. When writing state information including the values ​​of frequently accessed state items to the data store, the data management device 100 references the frequently accessed state item list and determines whether the state information corresponds to a frequently accessed state item. If the data management device 100 determines that the state information corresponds to a frequently accessed state item, it groups and writes the state information for multiple frequently accessed state items into a single record in the data store. On the other hand, if the data management device 100 determines that the state information does not correspond to a frequently accessed state item, it writes the state information into a single record in the data store. This allows the data management device 100 to dynamically change and optimize the storage structure of the data store in response to changes in the access frequency of each state item to the data store. The data management device 100 groups and stores frequently updated state information in the data store, allowing state information for multiple state items to be written with a single access to the data store. For example, the access time to the data store when processing event messages input from a huge number of IoT devices can be shortened, and the event processing of the digital twin can be accelerated. Furthermore, the data management device 100 does not require access to special metadata, and does not incur unnecessary overhead processing for changing the storage structure of the data store.

[0187] Furthermore, the data management device 100 creates high-frequency state item list information that lists multiple high-frequency state items whose access frequency is higher than a predetermined threshold based on the access frequency to the data store for each state item. This allows the data management device 100 to dynamically update the high-frequency state item list information according to the access frequency of the high-frequency state items.

[0188] The data store of the data management device 100 can be a KVS (Key Value Store) and cache memory. In this case, the data management device 100 performs writing processing to the cache memory and flushes the cache data from the cache memory to the KVS at a predetermined time. As a result, the data management device 100 can dynamically change the storage structure in the cache memory, group frequently updated state information, and then reflect this in the storage structure of the KVS 110.

[0189] Furthermore, the data management device 100 includes a process for prefetching grouped state information stored in the KVS into cache memory in response to input state information at the beginning of processing of a predetermined data processing program. This allows the data management device 100 to further speed up event message processing by prefetching.

[0190] Furthermore, the data management device 100 updates the high-frequency state item list information based on fluctuations in the access frequency to the data store during the creation process. The data management device 100 writes the updated high-frequency state item list information asynchronously while writing state information to the data store. This allows the data management device 100 to change the storage structure of the data store in response to changes in the access pattern to state items, thereby always maintaining the high-frequency state item list information in an optimal state. Furthermore, the storage structure of the data store can always be maintained optimally. This change in the storage structure of the data store does not require additional access to the data store, preventing the effects of message processing delays and the like.

[0191] Furthermore, the data management device 100 determines high-frequency state items based on the cost of accessing the data store for each state item, and the number and frequency of data processing. This allows the data management device 100 to correctly identify high-frequency state items for each state item based on the input status and usage of event messages, etc.

[0192] Furthermore, the data management device 100 creates high-frequency state item list information for each type of external device that outputs state information containing state item values. The data management device 100 performs the determination process by referencing high-frequency state item list information corresponding to each type of external device. For example, the data management device 100 uses a predetermined data processing program to process multiple event messages input from multiple external devices of different types, each processed separately by message type. The data management device 100 performs the determination process by referencing high-frequency state item list information for each type when writing state information containing state item values ​​to a data store using the predetermined data processing program, and determining whether one or more state items included in the event message are high-frequency state items. This allows the data management device 100 to process data based on input from a wide variety of external devices, such as IoT devices, and access a data store appropriate for each type of external device.

[0193] Furthermore, when reading state information including values ​​of state items from the data store, the data management device 100 reads state information grouped into records of the data store or individual state information. This allows the data management device 100 to easily read from the data store by optimizing the state storage structure when writing to the data store. Furthermore, when reading, there is no need to refer to the list of high-frequency state items.

[0194] The data management method described in the embodiment of the present invention can be realized by having a processor, such as a server, execute a prepared program. This method is recorded on a computer-readable recording medium, such as a hard disk, flexible disk, CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disk), or flash memory, and is executed by being read from the recording medium by a computer. This method may also be distributed via a network, such as the Internet.

[0195] The following additional notes are provided regarding the above-described embodiment.

[0196] (Note 1) When a predetermined data processing program processes data, it creates a list of high-frequency state items based on the access frequency to the data store, and When writing state information including the value of the high-frequency state item into the data store, refer to the high-frequency state item list information and determine whether the state information corresponds to the high-frequency state item; If it is determined that the state information corresponds to the high frequency state item, the state information of the plurality of high frequency state items is grouped and written into one record of the data store; If it is determined that the state information does not correspond to the high-frequency state item, write the state information to one record in the data store. A data management method characterized in that processing is executed by a computer.

[0197] (Appendix 2) The creation process is as follows: creating the high frequency state item list information, which lists a plurality of high frequency state items whose access frequency is higher than a predetermined threshold, based on the access frequency of each of the state items to the data store; 2. The data management method according to claim 1,

[0198] (Supplementary Note 3) The data store is a KVS (Key Value Store) and a cache memory, The writing process includes: to the cache memory, flushing cache data from the cache memory to the KVS at a predetermined time; 3. The data management method according to claim 1 or 2,

[0199] (Supplementary Note 4) At the beginning of the processing of the predetermined data processing program, a process is included in which the grouped state information stored in the KVS is prefetched into the cache memory in response to the input state information. 4. The data management method according to claim 3,

[0200] (Appendix 5) The creation process is as follows: updating the high frequency state item list information based on a change in the access frequency to the data store; asynchronously during the process of writing the state information to the data store, the updated high frequency state item list information being superimposed thereon; 5. The data management method according to any one of Supplementary Notes 1 to 4.

[0201] (Appendix 6) The creation process is as follows: determining the high frequency state items based on the cost of accessing the data store for each of the state items and the number and frequency of data processing; 6. A data management method according to any one of appendices 1 to 5.

[0202] (Appendix 7) The creation process is as follows: creating a list of high frequency state items for each type of external device that outputs state information including the values ​​of the state items; The determination process includes: The high frequency state item list information corresponding to the type of the external device is referenced. 7. The data management method according to any one of appendices 1 to 6.

[0203] (Supplementary Note 8) The predetermined data processing program processes a plurality of event messages input from a plurality of external devices of different types, each of which is processed according to its type; The determination process includes: when writing state information including the value of the state item to the data store by the predetermined data processing program, referencing the list information of high frequency state items by type, and determining whether or not one or more of the state items included in the event message correspond to the high frequency state items; 8. The data management method according to claim 7,

[0204] (Supplementary Note 9) The external device is an IoT device, The event message is periodically input, the event message including the state information of the state and value detected by the IoT device. 9. The data management method according to claim 8,

[0205] (Supplementary Note 10) When reading state information including values ​​of the state items from the data store, the state information grouped into records of the data store or the state information individually is read. 10. The data management method according to any one of Supplementary Notes 1 to 9.

[0206] (Supplementary Note 11) When a predetermined data processing program processes data, based on the frequency of access to the data store, high frequency state item list information is created that lists the high frequency state items that are accessed frequently, When writing state information including the value of the high-frequency state item into the data store, refer to the high-frequency state item list information and determine whether the state information corresponds to the high-frequency state item; If it is determined that the state information corresponds to the high frequency state item, the state information of the plurality of high frequency state items is grouped and written into one record of the data store; If it is determined that the state information does not correspond to the high-frequency state item, write the state information to one record in the data store. A data management program that causes a computer to execute processing. [Explanation of symbols]

[0207] 100 Data management device 110 KVS 120 High Frequency State Item List Information 200 IoT devices 210 Event Messages 220 User Logic 230 Logic Processing Information 601 Message Queue 602 Event Processing Unit 610 Prefetching section 611 Message Processing Unit 612 Plugin Management Unit 613 Cache Flush Unit 620 Grouping Gateway Unit 621 WRITE section (writing section) 622 READ section (read section) 623 Access Statistics Department 624 High Frequency Judgment Unit 625 High Frequency State Item List Creation Department 630 cache memory 640 Access Statistics 650 Backup Server 1401 CPU 1402 memory 1403 Network Interface 1405 Recording media 1407 Portable recording media DT Digital Twin H1 High Frequency State Items L1 Low Frequency State Items

Claims

1. When a predetermined data processing program processes data, based on the frequency of access to the data store, high frequency state item list information is created that lists the high frequency state items that have a high access frequency; When writing state information including the value of the high-frequency state item into the data store, refer to the high-frequency state item list information and determine whether the state information corresponds to the high-frequency state item; If it is determined that the state information corresponds to the high frequency state item, the state information of the plurality of high frequency state items is grouped and written into one record of the data store; If it is determined that the state information does not correspond to the high-frequency state item, write the state information to one record in the data store. A data management method characterized in that processing is executed by a computer.

2. The creation process includes: creating the high frequency state item list information, which lists a plurality of high frequency state items whose access frequency is higher than a predetermined threshold, based on the access frequency of each state item of the state information to the data store; 2. The data management method according to claim 1.

3. the data store is a KVS (Key Value Store) and a cache memory; The writing process includes: to the cache memory, flushing cache data from the cache memory to the KVS at a predetermined time; 3. The data management method according to claim 1 or 2.

4. At the beginning of processing of the predetermined data processing program, a process of prefetching the grouped state information stored in the KVS into the cache memory in response to the input state information is included.

4. The data management method according to claim 3.

5. The creation process includes: updating the high frequency state item list information based on a change in the access frequency to the data store; asynchronously during the process of writing the state information to the data store, the updated high frequency state item list information being superimposed thereon; 5. The data management method according to claim 1, wherein the data management method is a data management method for managing data.

6. The creation process includes: determining the high-frequency state items based on the cost required to access the data store for each state item of the state information, and the number and frequency of data processing; 6. The data management method according to claim 1, wherein the data management method is a data management method for managing data.

7. The creation process includes: creating a list of high frequency state items for each type of external device that outputs the state information; The determination process includes: The high frequency state item list information corresponding to the type of the external device is referenced.

7. The data management method according to claim 1, wherein the data management method is a data management method for managing data.

8. the predetermined data processing program processes a plurality of event messages input from a plurality of external devices of different types, each of which is processed according to its type; The determination process includes: when writing the state information to the data store by the predetermined data processing program, referencing the high frequency state item list information by type, and determining whether one or more pieces of state information included in the event message correspond to the high frequency state items; 8. The data management method according to claim 7.

9. When reading the state information from the data store, the state information may be grouped into records in the data store or may be read individually.

9. The data management method according to claim 1, wherein the data management method is a data management method for managing data.

10. When a predetermined data processing program processes data, based on the frequency of access to the data store, high frequency state item list information is created that lists the high frequency state items that have a high access frequency; When writing state information including the value of the high-frequency state item into the data store, refer to the high-frequency state item list information and determine whether the state information corresponds to the high-frequency state item; If it is determined that the state information corresponds to the high frequency state item, the state information of the plurality of high frequency state items is grouped and written into one record of the data store; If it is determined that the state information does not correspond to the high-frequency state item, write the state information to one record in the data store. A data management program that causes a computer to execute processing.

Images