Public data determination device, data publication system, public data determination method, and program

Abstract

To obtain a disclosed data determination device capable of efficiently determining disclosed data.SOLUTION: A disclosed data determination device 1 comprises a high frequency access determination part 121 that calculates access frequency to disclosed data using access information indicating the history of access to disclosed data that has already been disclosed and detects high frequency access data that is disclosed data with high access frequency using the calculated access frequency, a similarity calculation part 122 that calculates the similarity between first attribute information indicating the attributes of the high frequency access data and second attribute information indicating the attributes of transaction data, and a data selection part 123 that determines the transaction data that is similar to the high frequency access data based on the similarity as disclosed data to be added.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a public data determination device, a data publication system, a public data determination method, and a program for determining data to be published.

Background Art

[0002] In recent years, in order to promote the utilization of various data including open data, the amount of data to be published has been increasing. The data to be published is, for example, published as a set of data. When publishing a dataset, information about the dataset and information indicating the location of the dataset are generated as a data catalog. For example, when a user acquires a dataset and inputs information about the dataset to be acquired as a search condition, each data item in the data catalog is compared with the search condition, and if there is a data item that matches the search condition, the information of the data catalog including the matched data item is displayed as a search result.

[0003] Patent Document 1 discloses a technique for generating tags assigned to data as a data catalog. In the technique described in Patent Document 1, when a developer of an application used for data analysis inputs a search keyword to acquire data, the tag names of the tags that match the search keyword are displayed. When the developer of the application selects a tag name from the displayed tag names, the data corresponding to the selected tag name is displayed, and the developer of the application can download desired data from the displayed data.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the technology described in Patent Document 1, the recipients of the data are application developers, not general users, so all collected data is subject to provision. On the other hand, when data is made available to a wider range of users, generally, data to be made available is selected from the collected data, and the selected data is made public. The data to be made public is determined, for example, by manual selection, but unlike data made available to application developers, the needs of users are diverse for data to be made publicly available, making it difficult and time-consuming to manually determine appropriate data to be made public. For this reason, it is desirable to determine the data to be made public efficiently. However, Patent Document 1 does not disclose a method for determining the data to be made public.

[0006] This disclosure is made in view of the above and aims to provide a public data determination device that can efficiently determine public data. [Means for solving the problem]

[0007] To solve the above-mentioned problems and achieve the objectives, the public data determination device relating to this disclosure is a public data determination device in a data disclosure system that discloses public data selected from transaction data, and comprises a high-frequency access determination unit that calculates the access frequency of public data using access information that shows the history of access to already disclosed public data, and detects high-frequency access data, which is public data with a high access frequency, using the calculated access frequency. The public data determination device further comprises a similarity calculation unit that calculates the similarity between first attribute information that shows the attributes of high-frequency access data and second attribute information that shows the attributes of transaction data, and a data selection unit that determines transaction data similar to high-frequency access data as additional public data based on the similarity. [Effects of the Invention]

[0008] This disclosure has the effect of enabling efficient determination of publicly available data. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example configuration of the data publication system according to Embodiment 1. [Figure 2] Figure showing an image of the data catalog for Embodiment 1. [Figure 3] A diagram showing an example of data items in the data catalog of Embodiment 1. [Figure 4] This figure shows an example of a search request screen displayed on the user terminal of Embodiment 1. [Figure 5] A diagram showing an example of the display screen for the search results of Embodiment 1. [Figure 6] A flowchart showing an example of the process for determining the public dataset in Embodiment 1. [Figure 7] This figure shows an example configuration of a computer system that implements the public data determination device of Embodiment 1. [Figure 8] This figure shows an example configuration of the data disclosure system according to Embodiment 2. [Figure 9] This figure shows an example configuration of the data disclosure system according to Embodiment 3. [Figure 10] This figure shows an example configuration of the data disclosure system according to Embodiment 4. [Figure 11] This figure shows an example of a search request screen that accepts disclosure requests for Embodiment 4. [Modes for carrying out the invention]

[0010] The following describes in detail, with reference to the drawings, the public data determination device, data publication system, public data determination method, and program according to the embodiment.

[0011] Embodiment 1. Figure 1 is a diagram showing an example configuration of a data disclosure system according to Embodiment 1. The data disclosure system 100 of this embodiment comprises a data disclosure determination device 1 and a data disclosure device 3. The data disclosure system 100 discloses to users, for example, various data acquired in a smart city, data acquired at various plants including power plants, and data acquired by various sensors, as a dataset. Note that the data disclosed by the data disclosure system 100 is not limited to the examples described above and may be any kind of data.

[0012] The data publication system 100 publishes public data selected from transaction data. Specifically, the data publication system 100 acquires transaction data from the transaction data management device 4 which manages the transaction data, determines which data to publish as a dataset from the acquired transaction data, and stores the dataset to be published. Transaction data is the data that forms the basis of the public dataset to be published, and may include sensor data acquired by sensors, and may also include at least one of the following: video data acquired by surveillance cameras and data acquired at power plants. Transaction data is not limited to these examples, and may also include, for example, various data acquired in smart cities, data acquired at various plants other than power plants, and records of entry and exit to buildings and venues in smart cities. Furthermore, transaction data may include data acquired at multiple power plants and data acquired at multiple smart cities. Transaction data is not limited to the examples described above.

[0013] The transaction data management device 4 includes a transaction data acquisition unit 41 that acquires transaction data, a transaction data storage unit 42 that stores the transaction data acquired by the transaction data acquisition unit 41, and a transmission / reception unit 43 that transmits and receives data with other devices.

[0014] The data catalog management device 2 manages the data catalogs of the data sets to be published. A data catalog is information including one or more data items related to a data set for each data set to be published, and includes, for example, attribute information indicating the attributes of the published data set and information indicating the location of the published data set. FIG. 2 is a diagram showing an image of the data catalog of the present embodiment. A data catalog is generated for each data set. The data catalog of each data set is also called data set information. The data catalog includes information on each data item such as, for example, an explanation of the data set, a storage location (information indicating the location), a creator, and distribution information, as shown in FIG. 2.

[0015] The published data sets are selected by administrators, operators, developers, etc. at the initial stage of the operation of the data publishing system 100. The data catalogs corresponding to the initial published data sets are generated, for example, by being input by administrators, operators, developers, etc. or by manually acquiring information from other devices. After the start of the operation of the data publishing system 100, as will be described later, the published data sets added by the data publishing system 100 are determined. The data catalogs corresponding to the added published data sets are automatically generated based on transaction data, master data, etc. Note that the master data is information related to transaction data and is managed by the master data management device 5.

[0016] FIG. 3 is a diagram showing an example of data items in the data catalog of the present embodiment. As shown in FIG. 3, the data items to be cataloged, that is, the sources of the data items included in the data catalog are, for example, master data, transaction data, setting information, and others. In the example shown in FIG. 3, an example where the transaction data is sensor data acquired by a sensor is shown. Note that the sensor may include a device that acquires images such as a camera. The setting information is, for example, information set by an operator, a manager, or the like. The setting information may be stored in the master data management device 5 together with the master data, or may be stored in other locations not shown. For example, when the source of the data is other, it is input by an operator, a manager, or the like at the time of generating the data catalog, or is generated by a defined process.

[0017] Among the data items to be cataloged, the data items acquired from the master data include, for example, sensor information (sensor name, ID (Identifier),...), observation target (target name), observation activity (project name), and the like. The project name corresponds to the project when the sensor data is acquired for some project. Instead of the project name, information indicating the purpose for which the sensor data is acquired may be used. FIG. 3 is an example, and the data items in the data catalog are not limited to the example shown in FIG. 3.

[0018] Among the data items to be cataloged, those obtained from transaction data include, for example, date and time (observation date and time), sensor name, etc. Among the data items to be cataloged, those obtained from configuration information include, for example, the creator of the dataset among the data items in the dataset details. Among the data items to be cataloged, those obtained from other sources include, for example, license information (usage conditions, scope of publication, publication region, data protection requirements, warranty, etc.), distribution information (distribution start date, dataset size, etc.), public URL (Uniform Resource Locator), dataset details (title, description), etc. The public URL is generated by a defined algorithm when the data catalog is generated. The public URL is generated randomly, for example. Transaction data and master data include identification information to identify transaction data, such as sensor name and sensor number, and this identification information allows transaction data and master data to be associated. Similarly, for information obtained from configuration information and other sources, identification information is included for information that needs to be associated with transaction data.

[0019] Note that the data sources shown in Figure 3 are just examples, and the data sources for each data item are not limited to those shown in Figure 3. For example, license information may be included in the master data, in which case the source of the license information may be the master data.

[0020] Returning to the explanation of Figure 1, the data catalog management device 2 comprises a transmission / reception unit 21 that transmits and receives data with other devices, a data catalog storage unit 22 that stores data catalogs, an access detection unit 23 that detects access to data catalogs, an access information storage unit 24 that stores the detection results by the access detection unit 23 as access information, and an editing acceptance unit 25 that accepts editing of data catalogs. The access information indicates the history of access to publicly available datasets that have already been made public. In detail, for example, the access information is information that indicates the history of publicly available datasets being provided to a user, and includes information that identifies the data catalog that was accessed, i.e., requested for acquisition, from the data publishing device 3, and the corresponding time.

[0021] The master data management device 5 comprises a transmission / reception unit 51 that transmits and receives data with other devices, and a master data storage unit 52 that stores master data. As described above, the master data is information related to transaction data, and if the transaction data is data acquired by a sensor, it includes the name of the sensor, etc.

[0022] The data publishing device 3 of the data publishing system 100 provides a website that accepts requests to obtain published datasets. The following describes an example of how published datasets are made available to users via a website; however, the method of publishing published datasets is not limited to the example using a website. The data publishing device 3 sends information to the user terminal 6, which is the user's terminal, to display a search acceptance screen, which is a screen for accepting requests to obtain published datasets. Although Figure 1 shows one user terminal 6, there are no particular restrictions on the number of user terminals 6, and the number of user terminals 6 is not limited to the example shown in Figure 1.

[0023] The user terminal 6 displays a search request screen based on the information received from the data publishing device 3. Figure 4 shows an example of the search request screen displayed on the user terminal 6 in this embodiment. The user enters, for example, search criteria for searching for the desired published dataset in the search criteria input field 201 shown in Figure 4. Upon receiving the search criteria, the user terminal 6 sends a search request including the search criteria to the data publishing device 3.

[0024] When the data publishing device 3 receives a search request from the user terminal 6 to search for a public dataset, it refers to the data catalogs managed by the data catalog management device 2 according to the search request and retrieves data catalogs that match the search criteria. For each retrieved data catalog, the data publishing device 3 extracts information about the data items to be presented to the user from the data catalog as presented information and sends the presented information to the user terminal 6 as a search result. The presented information includes at least a part of the data catalog and can be said to be a data catalog itself. Furthermore, the presented information is information about the public dataset corresponding to the retrieved data catalog.

[0025] Figure 5 shows an example of the display screen for search results in this embodiment. The user refers to each piece of information displayed as a search result on the user terminal 6 and selects the information corresponding to the public dataset the user desires by checking the checkbox 202 provided for each piece of information. When the user presses, for example, the download start button 203, the user terminal 6 determines that the selection result has been confirmed and sends information indicating the information selected by the user as the selection result to the data publishing device 3. The data publishing device 3 finds the location of the public dataset corresponding to the selection result received from the user terminal 6 from the corresponding data catalog and sends the public dataset to the user terminal 6. As a result, the public dataset selected by the user is provided to the user. Note that Figures 4 and 5 are examples, and the information, layout, and specific strings displayed on each display screen are not limited to these examples; any screen that can accept input similar to that in Figures 4 and 5 is acceptable.

[0026] In this way, the user is provided with a public dataset selected by the user from among the public datasets whose data item information in the data catalog matches the search criteria specified by the user.

[0027] Returning to the explanation of Figure 1, the data publishing device 3 comprises a transmission / reception unit 31 that transmits and receives data with other devices, a publishing processing unit 32 that publishes the published dataset, and a published data storage unit 33 that stores the published data. The publishing processing unit 32 publishes the published dataset via a website, for example. For example, the publishing processing unit 32 receives a request to search for the published dataset via the website and transmits the selection result to the user terminal 6 via the website.

[0028] The public data determination device 1 of the data publication system 100 obtains access information from the data catalog management device 2 and determines the public data set by selecting a public data set from the transaction data based on the access information. The public data determination device 1 generates a data catalog corresponding to the determined public data set using the corresponding transaction data and master data, and transmits the generated data catalog to the data catalog management device 2. When the data catalog management device 2 receives the data catalog from the public data determination device 1, it stores the received data catalog in the data catalog storage unit 22. As a result, the data catalog corresponding to the public data determined by the public data determination device 1 is registered.

[0029] The public data determination device 1 comprises a transmission / reception unit 11 that transmits and receives data with other devices, a public data determination unit 12 that determines a public data set, and a catalog generation unit 13 that generates a data catalog corresponding to the public data set determined by the public data determination unit 12. The public data determination unit 12 comprises a high-frequency access determination unit 121, a similarity calculation unit 122, and a data selection unit 123.

[0030] The high-frequency access determination unit 121 uses access information to calculate the access frequency of public datasets (public data) and uses the calculated access frequency to detect high-frequency access data, which are public datasets with high access frequency. In detail, the high-frequency access determination unit 121 obtains access information from the data catalog management device 2 via the transmission / reception unit 11 and uses the access information to detect data catalogs with high access frequency. As described above, the access information includes information identifying the data catalog accessed by the data publishing device 3 and the corresponding time. The data publishing device 3 accesses the data catalog when searching for a data catalog that matches the search criteria, and when it receives a selection result from the user terminal 6 and obtains the location of the dataset corresponding to the selection result from the data catalog. Here, the access information includes information related to the latter, i.e., when obtaining the location of the dataset corresponding to the selection result from the data catalog. In the following, access information is assumed to include information about access when retrieving the location of the dataset corresponding to the selection result from the data catalog, that is, information about access to the data catalog corresponding to the public dataset selected by the user as the dataset to be retrieved. However, it is not limited to this, and access information may also include information about access when a data catalog matching the search criteria is searched. In other words, it may include information about access to the data catalog corresponding to the public dataset matching the search criteria specified by the user.

[0031] In the following example, we will describe how to use information about access to a data catalog detected by the data catalog management device 2 as access information. However, instead, the data publishing device 3 may acquire information about access to a published dataset as access information. That is, the data publishing device 3 may include an access information detection unit that detects information identifying the accessed published dataset and the corresponding time as access information by detecting the selection result received from the user terminal 6 or the published dataset read from the published data storage unit 33, and an access information storage unit that stores the access information. In this case, the high-frequency access determination unit 121 acquires access information from the data publishing device 3 via the transmission / reception unit 11.

[0032] Since accessing a data catalog means accessing the corresponding public dataset, the high-frequency access determination unit 121 detects, based on access information, public datasets that have been accessed more than a threshold over a certain period of time as high-frequency accessed public datasets, i.e., popular public datasets.

[0033] The similarity calculation unit 122 calculates the similarity between first attribute information, which indicates the attributes of the frequently accessed public dataset (frequently accessed data), and second attribute information, which indicates the attributes of the transaction data. Specifically, the similarity calculation unit 122 uses a data catalog corresponding to the frequently accessed public dataset, transaction data, and corresponding master data to calculate the similarity between the attribute information of the frequently accessed public dataset and the attribute information of the transaction data. The first attribute information is at least a part of the attribute information in the data catalog corresponding to the frequently accessed public dataset.

[0034] The data selection unit 123 determines transaction data similar to frequently accessed public datasets based on similarity to be added as public data. The data selection unit 123 transmits the determined public dataset and a notification indicating that the public dataset has been added to the data publishing device 3 via the transmission / reception unit 11. The transmission / reception unit 31 of the data publishing device 3 stores the public dataset received from the public data determination device 1 in the public data storage unit 33 and outputs a notification indicating that the public dataset has been added to the publishing processing unit 32.

[0035] Furthermore, the data selection unit 123 notifies the catalog generation unit 13 of information indicating the transaction data corresponding to the public dataset to be added. The catalog generation unit 13 generates a data catalog of the transaction data corresponding to the notified information and transmits the generated data catalog to the data catalog management device 2 via the transmission / reception unit 11. Specifically, the catalog generation unit 13 generates a data catalog corresponding to the public dataset to be added, as determined by the data selection unit 123, using master data containing transaction data that forms the basis of the public data and information about the transaction data, as well as a data catalog of a public dataset with high-frequency access similar to the public dataset in question.

[0036] Furthermore, one or more of the data catalog management device 2, transaction data management device 4, and master data management device 5 may be included in the data publication system 100.

[0037] Furthermore, the separation of each device shown in Figure 1 is just one example, and two or more of the devices shown in Figure 1 may be implemented as a single device. For example, the public data determination device 1 and the data publication device 3 may be integrated, or the public data determination device 1 and the data catalog management device 2 may be integrated, or the public data determination device 1 and the data catalog management device 2 may be integrated, or the public data determination device 1, the data catalog management device 2, and the data publication device 3 may be integrated. When multiple devices are integrated, a transmitting and receiving unit may not be used for transmitting and receiving data in these devices. Also, some of the functional units that make up each device may be provided in other devices. For example, the catalog generation unit 13 of the public data determination device 1 may be provided in the data catalog management device 2. Also, each device shown in Figure 1 may be further divided into multiple devices. For example, the public data storage unit 33 may be provided in a public data management device (not shown).

[0038] Next, the method for determining the public dataset in this embodiment will be described. Figure 6 is a flowchart showing an example of the process for determining the public dataset in this embodiment. As shown in Figure 6, the public data determination device 1 calculates the access frequency (step S1). In detail, the high-frequency access determination unit 121 obtains access information from the data catalog management device 2 via the transmission / reception unit 11, and uses the access information to calculate the access frequency for each data catalog, i.e., for each public dataset. The access frequency is, for example, the number of accesses within a certain period of time.

[0039] Next, the public data determination device 1 selects public datasets (public data) with high access frequency (step S2). Specifically, the high-frequency access determination unit 121 selects public datasets whose access frequency is above a threshold, for example. Alternatively, the high-frequency access determination unit 121 may select public datasets with high access frequency using the results of statistical processing of the access frequency of each public dataset. For example, the mean and standard deviation of the access frequency of each public dataset may be calculated, and the access frequency that is greater than the mean and whose standard deviation is above the threshold may be set as the threshold. Furthermore, the high-frequency access determination unit 121 may sort the public datasets in order of access frequency and select a predetermined number of the top public datasets with the highest number of accesses. The method for selecting public datasets with high access frequency is not limited to these examples, as long as it is based on access frequency.

[0040] Next, the public data determination device 1 sets one of the selected public datasets as the data to be processed (step S3). Specifically, the data selection unit 123 sets one of the public datasets selected by the high-frequency access determination unit 121 as the data to be processed.

[0041] Next, the public data determination device 1 acquires the transaction data and the corresponding master data (step S4). Specifically, the data selection unit 123 acquires the non-public transaction data stored in the transaction data storage unit 42 of the transaction data management device 4 via the transmission / reception unit 11 and 43, and acquires the corresponding master data from the master data storage unit 52 of the master data management device 5 via the transmission / reception unit 11 and 51. Note that in the case of time-series data such as sensor data, transaction data may be acquired continuously and without breaks, but here, for example, the time-series data is stored as data for unit periods similar to the public dataset. For example, if one day's worth of data is used as the unit of the public dataset, each transaction data is accompanied by the observation date and time (in this example, the observation year, month, and day, as it is on a daily basis) and the sensor name corresponding to the sensor data. Alternatively, if the transaction data is stored as data for a period longer than the period corresponding to the public dataset, the transaction data may be divided into periods of the same length as the public dataset and the same processing may be performed. In addition, the public data determination device 1 may acquire data from the transaction data that corresponds to the attribute information described later, instead of acquiring the entire transaction data.

[0042] Next, the public data determination device 1 calculates the similarity between the attribute information of the data to be processed and the attribute information of the transaction data (step S5). Specifically, the similarity calculation unit 122 obtains data items used for calculating similarity from the data catalog storage unit 22 as attribute information from the data catalog corresponding to the data to be processed, via the transmission / reception unit 11 and the transmission / reception unit 21. The similarity calculation unit 122 also obtains data items used for calculating similarity from the master data storage unit 52 as attribute information from the master data corresponding to the transaction data, via the transmission / reception unit 11 and the transmission / reception unit 51. In addition, the similarity calculation unit 122 obtains the sensor name, observation date and time, etc., as attribute information from the transaction data itself. For each transaction data, the similarity calculation unit 122 calculates the similarity between the attribute information of the data to be processed and the attribute information of the transaction data.

[0043] Similarity is calculated, for example, based on the determination of whether the information of the data item in the first attribute information matches the information of the data item in the second attribute information for each data item included in the attribute information of the data to be processed (first attribute information) and the attribute information of the transaction data (second attribute information). Similarity may be determined such that the more data items for which the information of the data item in the first attribute information matches the information of the data item in the second attribute information, the higher the similarity. For example, the number of matching data items may be used as the similarity, or a weight coefficient may be set for each data item, and the similarity may be the sum of the weight coefficients of the matching data items. The weight coefficient may be determined by operators, administrators, etc., or past access may be analyzed using machine learning to determine the correlation between each data item and the number of accesses, and the weight coefficient of data items with a high correlation to the number of accesses may be set to be larger.

[0044] Furthermore, attribute information matching is not limited to exact matching; matching may be determined if the difference between the two is within a defined range. For example, if a data item consists of multiple pieces of information, matching may be determined even if only a part of that information matches. For instance, if transaction data is sensor data and the data item is the observation date and time, and the data to be processed was observed on July 20th of a certain year, the similarity calculation unit 122 may determine that the attribute information of the observation date and time matches for all transaction data observed in July of the same year, or it may also determine that the attribute information of the observation date and time matches for transaction data observed in July of a different year. In other words, the attribute information used to calculate similarity is not limited to the information itself obtained from data catalogs, transaction data, master data, etc., but may also be information calculated using the obtained information. For example, the season (spring, summer, autumn, winter) of the observation date and time may be calculated from the observation date and time and transaction data of the data to be processed, and a determination may be made as one of the data items whether the season matches.

[0045] The data items used to calculate the similarity score include, but are not limited to, at least some of the following: observation date and time (same day, time, day of the week, season, etc.), sensor installation location, sensor name, sensor type, observation target, observation activity (project name), and purpose of data acquisition.

[0046] Next, the public data determination device 1 determines whether or not there is transaction data with a high degree of similarity (step S6). Specifically, the data selection unit 123 determines, for example, whether or not there is transaction data whose similarity is equal to or greater than a threshold.

[0047] If there is transaction data with a high degree of similarity (Step S6 Yes), the public data determination device 1 determines the transaction data with a high degree of similarity to be a public dataset (Step S7). Specifically, the data selection unit 123 determines, for example, transaction data whose similarity is equal to or greater than a threshold to be added as a public dataset.

[0048] Next, the public data determination device 1 determines whether or not there are any unprocessed public data sets among the selected public data sets (step S8). Specifically, the data selection unit 123 determines whether or not there are any public data sets among the public data sets selected in step S2 that have not been selected as data to be processed.

[0049] If there are any unprocessed public datasets among the selected public datasets (Step S8 Yes), the public data determination device 1 changes the data to be processed (Step S9) and repeats the process from Step S4. The public data determination device 1 may store the attribute information used for processing from the transaction data and master data acquired in the first Step S4 in a storage unit (not shown), and read the stored attribute information in the second and subsequent Step S4s.

[0050] If there are no unprocessed public datasets among the selected public datasets (Step S8 No), the public data determination device 1 generates a data catalog corresponding to the determined public dataset (Step S10) and terminates the process. Specifically, the data selection unit 123 notifies the catalog generation unit 13 of information indicating the transaction data determined to be added as a public dataset, and the processing target data with a high similarity in attribute information to the transaction data. Based on the notification, the catalog generation unit 13 generates a data catalog corresponding to the transaction data.

[0051] For example, the catalog generation unit 13 obtains information on data items to be used in the data catalog from the master data storage unit 52, via the transmission / reception unit 11 and the transmission / reception unit 51, from the master data storage unit 52, which corresponds to the notified transaction data. Then, for data items in the data catalog whose source is transaction data and master data, the catalog generation unit 13 obtains the corresponding information from the transaction data and the master data. For data items whose source is configuration information and others, the catalog generation unit 13 obtains the information corresponding to the transaction data if it is possible to obtain it, and for data items for which information corresponding to the transaction data cannot be obtained, it copies and uses the information from the processing target data with a high degree of similarity. Furthermore, for data items that are generated by a predetermined algorithm for each public dataset, such as public URLs, the catalog generation unit 13 generates information according to the predetermined algorithm. In this way, a data catalog corresponding to the determined public dataset is generated. The generated data catalog is stored in the data catalog storage unit 22 via the transmission / reception unit 11 and the transmission / reception unit 21. Furthermore, the public data determination device 1 transmits the determined public dataset to the data publishing device 3, and the data publishing device 3 stores the received public dataset in the public data storage unit 33. Alternatively, the public data determination device 1 may notify the data publishing device 3 of information indicating the determined public dataset (transaction data), and the public data determination device 1 may store the public dataset in the public data storage unit 33 by obtaining the corresponding transaction data from the transaction data management device 4.

[0052] Furthermore, if the answer in step S6 is No, the public data determination device 1 proceeds to step S8.

[0053] The process illustrated in Figure 6 may be performed periodically, or it may be performed only when instructed to do so by an operator, administrator, or other person. Note that Figure 6 is just one example, and the processing procedure is not limited to the example shown in Figure 6.

[0054] In step S10 shown in Figure 6, a data catalog is automatically generated, but some parts of the automatically generated data catalog may not be from the actual public dataset. For example, if the license information of the added public dataset is copied from a highly similar public dataset, it may differ from the license information corresponding to the actual added public dataset. In such cases, the operator or administrator edits the data catalog using the data catalog management device 2. Specifically, the editing reception unit 25 receives the input of the edits and reflects the received edits in the data catalog of the data catalog storage unit 22.

[0055] Furthermore, while the data catalog for the added public dataset was automatically generated in the example described above, the data catalog for the added public dataset may also be configured by an operator, administrator, or other person, similar to the data catalog for the initial public dataset.

[0056] Next, the hardware configuration of the public data determination device 1 of this embodiment will be described. In this embodiment, the public data determination device 1 functions as a computer system when a program (computer program) describing the processing in the public data determination device 1 is executed on the computer system. Figure 7 is a diagram showing an example of the configuration of a computer system that realizes the public data determination device 1 of this embodiment. As shown in Figure 7, this computer system comprises a control unit 101, an input unit 102, a storage unit 103, a display unit 104, a communication unit 105, and an output unit 106, which are connected via a system bus 107.

[0057] In Figure 7, the control unit 101 is, for example, a processor such as a CPU, which executes a program describing the processing in the public data determination device 1 of this embodiment. The input unit 102 consists of, for example, a keyboard and mouse, and is used by the user of the computer system to input various types of information. The storage unit 103 includes various types of memory such as RAM (Random Access Memory) and ROM (Read Only Memory), and storage devices such as a hard disk, and stores the program to be executed by the control unit 101, necessary data obtained during the processing, etc. The storage unit 103 is also used as a temporary storage area for the program. The display unit 104 consists of a display, LCD (Liquid Crystal Display Panel), etc., and displays various screens to the user of the computer system. The communication unit 105 is a receiver and transmitter that perform communication processing. The output unit 106 is a printer, speaker, etc. Note that Figure 7 is just one example, and the configuration of the computer system is not limited to the example in Figure 7. For example, the computer system does not have to have an output unit 106.

[0058] Here, an example of the operation of the computer system until the program of this embodiment becomes executable will be described. In a computer system with the above configuration, for example, the program is installed in the storage unit 103 from a CD-ROM or DVD-ROM set in a CD (Compact Disc)-ROM drive or DVD (Digital Versatile Disc)-ROM drive (not shown). When the program is executed, the program read from the storage unit 103 is stored in the main memory area of ​​the storage unit 103. In this state, the control unit 101 performs processing as the public data determination device 1 of this embodiment according to the program stored in the storage unit 103.

[0059] In the above explanation, a program describing the processing in the public data determination device 1 is provided using a CD-ROM or DVD-ROM as the recording medium. However, the explanation is not limited to this, and depending on the configuration of the computer system, the capacity of the program to be provided, a program provided via a transmission medium such as the Internet via the communication unit 105 may also be used.

[0060] For example, the program of this embodiment causes the data publishing system to execute a determination step that calculates the access frequency of published data using access information that shows the history of access to already published published data, and uses the calculated access frequency to detect high-frequency access data, which is published data that is accessed frequently; a calculation step that calculates the similarity between first attribute information that shows the attributes of high-frequency access data and second attribute information that shows the attributes of transaction data; and a data selection step that determines transaction data similar to high-frequency access data as published data to be added based on the similarity.

[0061] The public data determination unit 12 and catalog generation unit 13 shown in Figure 1 are realized by the execution of a program stored in the storage unit 103 shown in Figure 7 by the control unit 101 shown in Figure 7. The storage unit 103 shown in Figure 7 is also used to realize the public data determination unit 12 and catalog generation unit 13. The transmitting / receiving unit 11 is realized by the communication unit 105 shown in Figure 7. The public data determination device 1 may be realized by multiple computer systems. For example, the public data determination device 1 may be realized by a cloud system.

[0062] The data catalog management device 2 can also be implemented, for example, by the computer system shown in Figure 7. The access detection unit 23 shown in Figure 1 is implemented by the control unit 101 shown in Figure 7 executing a program stored in the storage unit 103 shown in Figure 7. The storage unit 103 shown in Figure 7 is also used to implement the access detection unit 23. The transmitting / receiving unit 21 shown in Figure 1 is implemented by the communication unit 105 shown in Figure 7. The data catalog storage unit 22 and access information storage unit 24 shown in Figure 1 are part of the storage unit 103. The editing reception unit 25 shown in Figure 1 is implemented by the input unit 102 shown in Figure 7. The data catalog management device 2 may be implemented by multiple computer systems. For example, the data catalog management device 2 may be implemented by a cloud system.

[0063] The data publishing device 3 can also be implemented, for example, by the computer system shown in Figure 7. The publishing processing unit 32 shown in Figure 1 is implemented by executing a program stored in the storage unit 103 shown in Figure 7 by the control unit 101 shown in Figure 7. The storage unit 103 and communication unit 105 shown in Figure 7 are also used to implement the publishing processing unit 32. The transmitting / receiving unit 31 shown in Figure 1 is implemented by the communication unit 105 shown in Figure 7. The published data storage unit 33 shown in Figure 1 is part of the storage unit 103. The data publishing device 3 may be implemented by multiple computer systems. For example, the data publishing device 3 may be implemented by a cloud system.

[0064] Similarly, the transaction data management device 4 and the master data management device 5 are also implemented by, for example, the computer system shown in Figure 7. The transaction data management device 4 and the master data management device 5 may be implemented by multiple computer systems. For example, the transaction data management device 4 and the master data management device 5 may be implemented by a cloud system.

[0065] Furthermore, as described above, two or more of the public data determination device 1, data catalog management device 2, data publication device 3, transaction data management device 4, and master data management device 5 may be implemented by a single computer system.

[0066] As described above, the public data determination device 1 of this embodiment detects popular public datasets based on the access frequency of the public datasets and determines transaction data with similar attribute information to the popular public datasets as the public datasets to be added. This allows the public data determination device 1 of this embodiment to provide public datasets that meet user needs. Furthermore, the public data determination device 1 of this embodiment can efficiently determine public data (public datasets) without requiring operators, administrators, etc., to determine the public data. Additionally, the public data determination device 1 of this embodiment can efficiently generate data catalogs by automatically generating data catalogs for public datasets.

[0067] Embodiment 2. Figure 8 shows an example of the configuration of the data publication system according to Embodiment 2. The data publication system 100a of this embodiment is the same as the data publication system 100 of Embodiment 1, except that it includes a data publication determination device 1a instead of the data publication determination device 1. Components having the same functions as those of Embodiment 1 are denoted by the same reference numerals as in Embodiment 1, and redundant descriptions are omitted. The differences from Embodiment 1 will be mainly described below.

[0068] The public data determination device 1a of this embodiment is the same as the public data determination device 1, except that a catalog generation notification unit 14 is added. When the catalog generation unit 13 generates a data catalog, it notifies the catalog generation notification unit 14 of this fact, and the catalog generation notification unit 14 notifies the data catalog administrator that the data catalog has been generated. Specifically, for example, the catalog generation notification unit 14 generates a catalog generation notification indicating that a data catalog has been generated, and sends the generated catalog generation notification to the administrator terminal 7, which is the terminal of the data catalog administrator. The catalog generation notification may include, for example, information for identifying the generated data catalog. The information for identifying the data catalog may also be identification information for the corresponding public dataset. The catalog generation notification may be generated as an email, or it may be information for display as a pop-up notification to the administrator terminal 7, or it may be information for display by application software for managing data catalogs or application software such as an SNS (Social Networking Service). The catalog generation notification may also be information for notifying the administrator terminal 7 of the generation of the data catalog by sound, such as voice. Catalog generation notifications may be provided through both visual and auditory means.

[0069] The administrator can recognize that a data catalog has been generated by checking the catalog generation notification using the administrator terminal 7. This allows the administrator to check the data catalog on the data catalog management device 2 and edit the data catalog as needed.

[0070] The public data determination device 1a of this embodiment is implemented, similar to the public data determination device 1 described in Embodiment 1, by, for example, the computer system shown in Figure 7. The catalog generation notification unit 14 is implemented by the control unit 101 and the communication unit 105 shown in Figure 7.

[0071] In Figure 8, a catalog generation notification is sent to the administrator terminal 7. However, instead of this, or in addition to this, a catalog generation notification may be sent to the data catalog management device 2 to notify the administrator that the data catalog has been generated. In this case, information indicating that the catalog has been generated will be displayed on a display unit (not shown) in the data catalog management device 2, allowing the administrator and operators to recognize that the data catalog has been generated.

[0072] Furthermore, although the catalog generation notification unit 14 is provided within the public data determination device 1a in Figure 8, the data catalog management device 2 may also include a catalog generation notification unit, and when a data catalog is added, the catalog generation notification unit may send a catalog generation notification to the administrator terminal 7. Alternatively, when a data catalog is added, the data catalog management device 2 may display information indicating that a data catalog has been generated on a display unit (not shown).

[0073] Embodiment 3. Figure 9 shows an example of the configuration of a data publication system according to Embodiment 3. The data publication system 100b of this embodiment is the same as the data publication system 100 of Embodiment 1, except that it includes a data publication determination device 1b instead of a data publication determination device 1. Components having the same functions as those in Embodiment 1 are denoted by the same reference numerals as in Embodiment 1, and redundant descriptions are omitted. The differences from Embodiment 1 will be mainly described below.

[0074] The public data determination device 1b of this embodiment is the same as the public data determination device 1, except that a deletion determination unit 15 is added. The deletion determination unit 15 determines public data sets to be deleted using access frequency. Specifically, the deletion determination unit 15 obtains access information from the data catalog management device 2 via the transmission / reception unit 11 and uses the access information to determine public data sets to be deleted. Note that the access information may be obtained from the data publication device 3, as in Embodiment 1. For example, the deletion determination unit 15 determines at least one of the public data sets to be deleted: public data sets with an access frequency below a threshold, and public data sets that have not been accessed for a certain period of time or longer. The deletion determination unit 15 notifies the data publication device 3 and the data catalog management device 2 via the transmission / reception unit 11 of information indicating the public data sets to be deleted. The data publication device 3 deletes the notified public data sets from the public data storage unit 33, and the data catalog management device 2 deletes the data catalog corresponding to the notified public data sets from the data catalog storage unit 22.

[0075] In the example shown in Figure 8, the deletion determination unit 15 is provided in the public data determination device 1b, but the deletion determination unit may also be provided in the data publication device 3 or the data catalog management device 2.

[0076] In this embodiment, access information is used to delete public datasets that are accessed infrequently, thereby reducing the amount of storage capacity used to store public datasets and data catalogs.

[0077] The public data determination device 1b of this embodiment is implemented, similar to the public data determination device 1 described in Embodiment 1, by, for example, the computer system shown in Figure 7. The deletion determination unit 15 is implemented by the control unit 101 shown in Figure 7.

[0078] Embodiment 4. Figure 10 shows an example of the configuration of a data publication system according to Embodiment 4. The data publication system 100c of this embodiment is the same as the data publication system 100 of Embodiment 1, except that it includes a data publication determination device 1c instead of a data publication determination device 1. Components having the same functions as those in Embodiment 1 are denoted by the same reference numerals as in Embodiment 1, and redundant descriptions are omitted. The differences from Embodiment 1 will be mainly described below.

[0079] The public data determination device 1c of this embodiment is the same as the public data determination device 1, except that a requested data search unit 16 is added. In this embodiment, the public processing unit 32 of the data public device 3 receives a public request from a user terminal 6 for the public disclosure of data that is not included in the public dataset. When the public processing unit 32 of the data public device 3 receives a public request, it transmits the public request to the public data determination device 1c via the transmitting / receiving unit 31. When the requested data search unit 16 receives the public request via the transmitting / receiving unit 11, it searches for transaction data corresponding to the request conditions in the public request, and if there is transaction data corresponding to the request conditions, it determines that transaction data to be added as the public dataset. In detail, the requested data search unit 16 searches for the requested data from the transaction data stored in the transaction data management device 4 and the master data stored in the master data management device 5.

[0080] Figure 11 shows an example of a search acceptance screen for receiving disclosure requests in this embodiment. The disclosure processing unit 32 of the data disclosure device 3 displays the search acceptance screen exemplified in Figure 11 on the user terminal 6, for example, instead of the search acceptance screen shown in Figure 4 of Embodiment 1. The user enters the request conditions, which are information about the data to be disclosed, in the request input field 204. The user terminal 6 sends the disclosure request, including the request conditions, to the data disclosure device 3. Note that Figure 11 is just an example, and the displayed search acceptance screen is not limited to the example shown in Figure 11.

[0081] The request data search unit 16 searches transaction data and master data using the request conditions in the public request as search keywords to determine whether there is any transaction data that matches the request conditions. Note that the above-mentioned setting information and information with an acquisition source of "other" may also be used as search targets. If the request data search unit 16 finds transaction data that matches the request conditions, it determines that the transaction data will be added to the public dataset, notifies the catalog generation unit 13 of information indicating the public dataset, and acquires the transaction data and transmits it to the data publication device 3 as a public dataset. The catalog generation in the catalog generation unit 13 is the same as in Embodiment 1.

[0082] As described above, in this embodiment, a user requests the publication of data not present in the public dataset, and if corresponding transaction data exists, that transaction data is added to the public dataset. Therefore, it can also accommodate small needs.

[0083] In the example shown in Figure 10, the request data search unit 16 is provided in the public data determination device 1c, but the request data search unit may also be provided in the data publication device 3. In this case, if the request data search unit finds transaction data that matches the publication request, it notifies the catalog generation unit 13 of information indicating the transaction data via the transmission / reception unit 31 and the transmission / reception unit 11.

[0084] The public data determination device 1c of this embodiment is implemented, similar to the public data determination device 1 described in Embodiment 1, by, for example, the computer system shown in Figure 7. The requested data retrieval unit 16 is implemented by the control unit 101 shown in Figure 7.

[0085] The configurations shown in the above embodiments are merely examples, and it is possible to combine them with other known technologies, combine different embodiments, and omit or modify parts of the configuration without departing from the gist of the invention.

[0086] The various aspects of this disclosure are summarized below as an appendix.

[0087] (Note 1) A public data determination device in a data publishing system that publishes public data selected from transaction data, A high-frequency access determination unit calculates the access frequency of the publicly available data using access information that shows the history of access to the publicly available data already made public, and uses the calculated access frequency to detect high-frequency access data, which is the publicly available data with a high access frequency. A similarity calculation unit calculates the similarity between first attribute information indicating the attributes of the high-frequency access data and second attribute information indicating the attributes of the transaction data. A data selection unit that determines transaction data similar to the high-frequency access data based on the similarity to be added as the public data, A public data determination device characterized by comprising the following features. (Note 2) From among the publicly available data that matches the information of each data item in the data catalog showing information about the aforementioned publicly available data and the search conditions specified by the user, the publicly available data selected by the user is provided to the user. The data catalog includes attribute information indicating the attributes of the publicly available data and information indicating the location of the publicly available data. The first attribute information is at least a part of the attribute information in the data catalog corresponding to the high-frequency access data, The public data determination device according to Appendix 1, characterized in that the access information is information indicating the history of the public data being provided to the user. (Note 3) A catalog generation unit generates a data catalog corresponding to the publicly available data to be added, determined by the data selection unit, using the transaction data that forms the basis of the publicly available data, master data containing information about the transaction data, and the data catalog of the high-frequency access data similar to the publicly available data. The public data determination device according to Appendix 2, characterized by comprising the following features. (Note 4) When the catalog generation unit generates the data catalog, a catalog generation notification unit notifies the data catalog administrator that the data catalog has been generated. The public data determination device according to Appendix 3, characterized by comprising the above. (Note 5) A deletion determination unit that determines the publicly available data to be deleted using the aforementioned access frequency. A public data determination device according to any one of the appendices 1 to 4, characterized by comprising: (Note 6) A request data search unit searches for transaction data corresponding to the request conditions in a disclosure request that requests the disclosure of data that is not publicly available, and if there is transaction data corresponding to the request conditions, determines that transaction data to be added as the publicly available data. A public data determination device according to any one of the appendices 1 to 5, characterized by comprising: (Note 7) The public data determination device according to any one of the appendices 1 to 6, characterized in that the similarity is calculated based on the determination result of whether or not the information of the data item in the first attribute information and the information of the data item in the second attribute information match for each data item included in the first attribute information and the second attribute information. (Note 8) The public data determination device according to Appendix 7, characterized in that the transaction data includes sensor data acquired by the sensor. (Note 9) The public data determination device according to Appendix 8, characterized in that the data item includes at least one of the following: the date and time of observation of the transaction data, the type of sensor, and the purpose of observation of the sensor. (Note 10) The public data determination device according to any one of appendices 1 to 9, characterized in that the transaction data includes at least one of video data acquired by a surveillance camera and data acquired at a power plant. [Explanation of symbols]

[0088] 1,1a,1b,1c Public data determination device, 2 Data catalog management device, 3 Data publication device, 4 Transaction data management device, 5 Master data management device, 6 User terminal, 7 Administrator terminal, 11,21,31,43,51 Transmit / receive unit, 12 Public data determination unit, 13 Catalog generation unit, 14 Catalog generation notification unit, 15 Deletion determination unit, 16 Request data search unit, 22 Data catalog storage unit, 23 Access detection unit, 24 Access information storage unit, 25 Edit acceptance unit, 32 Publication processing unit, 33 Public data storage unit, 41 Transaction data acquisition unit, 42 Transaction data storage unit, 52 Master data storage unit, 100,100a,100b,100c Data publication system, 121 High-frequency access determination unit, 122 Similarity calculation unit, 123 Data selection unit.

Claims

1. A public data determination device in a data publishing system that publishes public data selected from transaction data, A high-frequency access determination unit calculates the access frequency of the publicly available data using access information that shows the history of access to the publicly available data already made public, and uses the calculated access frequency to detect high-frequency access data, which is the publicly available data with a high access frequency. A similarity calculation unit calculates the similarity between first attribute information indicating the attributes of the high-frequency access data and second attribute information indicating the attributes of the transaction data. A data selection unit that determines the transaction data similar to the high-frequency access data based on the similarity score as the public data to be added, A public data determination device characterized by comprising the following features.

2. From among the publicly available data that matches the information of each data item in the data catalog showing information about the aforementioned publicly available data and the search conditions specified by the user, the publicly available data selected by the user is provided to the user. The data catalog includes attribute information indicating the attributes of the publicly available data and information indicating the location of the publicly available data. The first attribute information is at least a part of the attribute information in the data catalog corresponding to the high-frequency access data, The public data determination device according to claim 1, characterized in that the access information is information indicating the history of the public data being provided to the user.

3. A catalog generation unit generates a data catalog corresponding to the publicly available data to be added, determined by the data selection unit, using the transaction data that forms the basis of the publicly available data, master data containing information about the transaction data, and the data catalog of the high-frequency access data similar to the publicly available data. The public data determination device according to claim 2, characterized by comprising:

4. When the catalog generation unit generates the data catalog, a catalog generation notification unit notifies the data catalog administrator that the data catalog has been generated. The public data determination device according to claim 3, characterized by comprising:

5. A deletion determination unit that determines the publicly available data to be deleted using the aforementioned access frequency. A public data determination device according to any one of claims 1 to 4, characterized by comprising:

6. A request data search unit searches for transaction data corresponding to the request conditions in a disclosure request that requests the disclosure of data that has not been made public, and if there is transaction data corresponding to the request conditions, it determines that transaction data to be added as the public data. A public data determination device according to any one of claims 1 to 4, characterized by comprising:

7. The public data determination device according to any one of claims 1 to 4, characterized in that the similarity is calculated based on the determination result of whether or not the information of the data item in the first attribute information and the information of the data item in the second attribute information match for each data item included in the first attribute information and the second attribute information.

8. The public data determination device according to claim 7, characterized in that the transaction data includes sensor data acquired by the sensor.

9. The public data determination device according to claim 8, characterized in that the data item includes at least one of the observation date and time of the transaction data, the type of sensor, and the purpose of observation of the sensor.

10. The public data determination device according to claim 7, characterized in that the transaction data includes at least one of video data acquired by a surveillance camera and data acquired at a power plant.

11. A publishing processing unit that publishes public data selected from transaction data, A high-frequency access determination unit calculates the access frequency of the publicly available data using access information that shows the history of access to the publicly available data already made public, and uses the calculated access frequency to detect high-frequency access data, which is the publicly available data with a high access frequency. A similarity calculation unit calculates the similarity between first attribute information indicating the attributes of the high-frequency access data and second attribute information indicating the attributes of the transaction data. A data selection unit that determines the transaction data similar to the high-frequency access data based on the similarity score as the public data to be added, A data publishing system characterized by having the following features.

12. A method for determining public data to be published by a data publishing system that publishes public data selected from transaction data, A determination step of using access information that shows the history of access to the publicly available data already made public to calculate the access frequency of the publicly available data, and using the calculated access frequency to detect high-frequency access data, which is the publicly available data with a high access frequency, A calculation step of calculating the similarity between first attribute information indicating the attributes of the high-frequency access data and second attribute information indicating the attributes of the transaction data, A data selection step in which transaction data similar to the high-frequency access data based on the similarity is determined to be added as the public data, A method for determining publicly available data, characterized by including the following:

13. A data publishing system that publishes public data selected from transaction data, A determination step of using access information that shows the history of access to the publicly available data already made public to calculate the access frequency of the publicly available data, and using the calculated access frequency to detect high-frequency access data, which is the publicly available data with a high access frequency, A calculation step of calculating the similarity between first attribute information indicating the attributes of the high-frequency access data and second attribute information indicating the attributes of the transaction data, A data selection step in which transaction data similar to the high-frequency access data based on the similarity is determined to be added as the public data, A program characterized by causing the execution of a specific action.

Images