Data analysis device, data analysis method, and program

Abstract

To provide a data analysis device, a data analysis method, and a program capable of automatically generating meta-data.SOLUTION: A data analysis device 1X mainly includes analysis query generation means 52X, insight generation means 53X, and meta-data generation means 54X. The analysis query generation means 52X generates an analysis query for analyzing data from the data. The insight generation means 53X generates an insight of the data on the basis of the data and the analysis query. The meta-data generation means 54X generates meta-data on the data on the basis of the insight.SELECTED DRAWING: Figure 15

Description

[Technical Field]

[0001] The present disclosure relates to the technical fields of a data analysis device, a data analysis method, and a program that perform processing related to data analysis. [Background technology]

[0002] Systems that use metadata are known. For example, Patent Document 1 discloses a technology for creating a natural language analysis model using metadata assigned by a user. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-051423 Summary of the Invention [Problem to be solved by the invention]

[0004] Manually generating metadata useful for data management requires a huge amount of work, and there is a limit to how much useful metadata can be added manually.

[0005] In view of the above-mentioned problems, one of the objects of the present disclosure is to provide a data analysis device, a data analysis method, and a program that are capable of automatically generating metadata. [Means for solving the problem]

[0006] One aspect of the data analysis device is an analytical query generation means for generating an analytical query for analyzing the data from the data; an insight generation means for generating insights into the data based on the data and the analytical query; a metadata generating means for generating metadata of the data based on the insight; The data analysis device has:

[0007] One aspect of the data analysis method includes: The computer generating an analytical query from the data to analyze the data; generating insights into the data based on the data and the analytical query; generating metadata for the data based on the insights; It is a data analysis method.

[0008] One aspect of the program is generating an analytical query from the data to analyze the data; generating insights into the data based on the data and the analytical query; The program causes a computer to execute a process of generating metadata for the data based on the insight. [Effects of the Invention]

[0009] One example of the effect of the present disclosure is that metadata can be automatically generated. [Brief explanation of the drawings]

[0010] [Figure 1] The configuration of the data analysis system is shown. [Figure 2] 1 shows the hardware configuration of a data analysis device. [Figure 3] FIG. 10 is a diagram illustrating an outline of a metadata generation process. [Figure 4] 2 is an example of a functional block of a processor. [Figure 5] 1 shows an example of a functional block of a data analysis unit. [Figure 6] This section provides an overview of analytical query generation using LLM. [Figure 7] A concrete example of generating analytical queries using LLM is given below. [Figure 8] 10 shows an overview of a process for generating insights using a chart recommendation model or a question-answering model. [Figure 9] A specific example of insight generation processing using a chart recommendation model and a question-answering model will be given. [Figure 10] 10 is a display example of a data display screen for a user to check data and metadata of the data. [Figure 11] 10 is a display example of the data display screen after the metadata addition button is selected. [Figure 12] This is an example of the data display screen after the hyperlink "Highlight 1" is selected. [Figure 13] 10 is an example of a flowchart illustrating an overview of processing executed by the data analysis apparatus. [Figure 14] The configuration of the data analysis system is shown. [Figure 15] FIG. 2 is a functional block diagram of the data analysis device. [Figure 16] 10 is a flowchart illustrating an example of a processing procedure of the data analysis device. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of a data analysis device, a data analysis method, and a program will be described with reference to the drawings. Hereinafter, a "query" refers to a natural language inquiry (including questions and hypothesis statements). An "answer" refers to a natural language sentence output by a system in response to a query, or text data representing a natural language sentence. An "insight" into the data being analyzed refers to information that suggests something about the data obtained by analyzing the data being analyzed, such as data that provides useful information for answering a query about the data being analyzed. A "data catalog" refers to a searchable inventory of an organization's data assets, and uses metadata, which is data that describes or summarizes the data.

[0012] First Embodiment (1) System Configuration 1 shows the configuration of a data analysis system 100. The data analysis system 100 mainly includes a data analysis device 1, an input device 2, a display device 3, and a storage device 4.

[0013] The data analysis device 1 generates metadata for the data registered in the data catalog 5 and outputs the generated metadata. Examples of metadata output include registering the metadata in the data catalog 5 and displaying the generated metadata. Hereinafter, data analyzed by the data analysis device 1 to generate metadata will also be referred to as "data to be analyzed." The data to be analyzed is assumed to be a table (database) containing multiple records.

[0014] The data analysis device 1 performs data communication with the input device 2, the display device 3, and the storage device 4 via a communication network or by direct wireless or wired communication.

[0015] The input device 2 is an interface that accepts user input, which is external input, and includes, for example, a touch panel, buttons, a keyboard, a voice input device, etc. The input device 2 supplies input information generated based on the user input to the data analysis apparatus 1.

[0016] The display device 3 is, for example, a display, a projector, or the like, and performs a predetermined display based on the display information supplied from the data analysis device 1.

[0017] The storage device 4 is a memory that stores various information necessary for the processing executed by the data analysis device 1. The storage device 4 stores a data catalog 5 and multiple pieces of data 6 (6A, 6B, ...) registered in the data catalog 5. The data catalog 5 includes at least metadata associated with each piece of data 6 to make each piece of data 6 searchable. The metadata included in the data catalog 5 includes general metadata (file name, data source, data format, schema, creation date, etc.) that exists by default, as well as metadata generated by the data analysis device 1. The data 6 is a database that can be used by an organization (such as a company) that manages the data analysis system 100, and is searchable data whose metadata is registered in the data catalog 5. In FIG. 1, data 6A and data 6B are shown as examples of the data 6.

[0018] The storage device 4 may store various types of information required for processing by the data analysis device 1, in addition to the data catalog 5 and the data 6. The storage device 4 may store, for example, model information (configuration information) for configuring a machine-learned large language model (LLM), model information for configuring any natural language understanding model used in natural language processing, and the like.

[0019] The model information includes various parameters of the machine-learned deep learning model, such as the layer structure, the neuron structure of each layer, the number and size of filters in each layer, and the weights of each element of each filter.

[0020] Here, we will explain the definition of a large-scale language model, or language model. A language model is a machine learning model that learns the relationships between words in a sentence and generates related strings from a target string. By using a language model that has been trained with sentences and sentences from various contexts, it is possible to generate related strings with appropriate content related to the target string.

[0021] For example, let us consider a case where a language model is used in question answering. The language model accepts the input question "What kind of country is Japan?" as a target string. The accepted question is also called a "prompt." The language model generates a string such as "Japan is an island country in the Northern Hemisphere..." as an answer to the question.

[0022] The learning method of the language model is not particularly limited, but as an example, the language model may be learned to output at least one sentence that includes an input character string.

[0023] As a specific example, the language model may be a Generative Pre-trained Transformer (GPT) that outputs a sentence including an input string by predicting a string that is likely to follow the input string, or ChatGPT based on GPT. Other language models include, for example, Text-to-Text Transfer Transformer (T5), Bidirectional Encoder Representations from Transformers (BERT), RoBERTa (Robustly optimized BERT approach), and Efficiently Learning an Encoder that Classifies Token Replacements Accurately.

[0024] The storage device 4 may be a storage device such as a hard disk connected to or built into the data analysis device 1, or may be a storage medium such as a flash memory. The storage device 4 may also be a server device that performs data communication with the data analysis device 1. In this case, the storage device 4 may be composed of multiple server devices.

[0025] The configuration of the data analysis system 100 shown in FIG. 1 is an example, and various modifications may be made to the configuration. For example, the input device 2 and the display device 3 may be configured as an integrated device. In this case, the input device 2 and the display device 3 may be configured as a tablet terminal integrated with the data analysis device 1. The data analysis device 1 may be connected to or have a built-in sound output device such as a speaker that outputs sound, and output information by sound. The data analysis device 1 may also be configured from multiple devices. In this case, the multiple devices that make up the data analysis device 1 exchange information necessary to execute pre-assigned processing between these multiple devices.

[0026] (2) Hardware configuration of data analysis equipment 2 shows the hardware configuration of the data analysis device 1. The data analysis device 1 includes, as hardware components, a processor 11, a memory 12, and an interface 13. The processor 11, the memory 12, and the interface 13 are connected via a data bus 19.

[0027] The processor 11 executes a predetermined process by executing a program stored in the memory 12. The processor 11 is a processor such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a TPU (Tensor Processing Unit). The processor 11 may be composed of multiple processors. The processor 11 is an example of a computer.

[0028] The memory 12 is composed of various types of volatile and non-volatile memories, such as a RAM (Random Access Memory) and a ROM (Read Only Memory). The memory 12 also stores programs for the data analysis device 1 to execute various processes. The memory 12 is also used as a working memory, and temporarily stores information acquired from the storage device 4. The memory 12 may also function as the storage device 4. Similarly, the storage device 4 may also function as the memory 12 of the data analysis device 1. The programs executed by the data analysis device 1 may be stored in a storage medium other than the memory 12.

[0029] The interface 13 is an interface for electrically connecting the data analysis device 1 to other devices. These interfaces may be wireless interfaces such as network adapters for wirelessly transmitting and receiving data to and from other devices, or may be hardware interfaces for connecting to other devices via cables or the like.

[0030] The hardware configuration of the data analysis device 1 is not limited to the configuration shown in Fig. 2. For example, the data analysis device 1 may include at least one of an input device 2 or a display device 3. Furthermore, the data analysis device 1 may be connected to or have a built-in sound output device such as a speaker.

[0031] (3) Processing Overview FIG. 3 is a diagram showing an outline of the metadata generation process performed by the data analysis apparatus 1. As shown in FIG.

[0032] As shown in FIG. 3, after acquiring data to be analyzed, the data analysis apparatus 1 generates an analytical query, which is an analytical query for analyzing the data to be analyzed. Then, the data analysis apparatus 1 generates insights into the data to be analyzed based on the analytical query and the data to be analyzed. Then, the data analysis apparatus 1 generates metadata from the insights. Here, the data analysis apparatus 1 may generate various types of data as metadata. As shown in FIG. 3, examples of metadata generated by the data analysis apparatus 1 include a summary, which is text data summarizing the data to be analyzed, a chart visualizing the data to be analyzed, tags to be added to the data to be analyzed, analysis types effective for analyzing the data to be analyzed, and highlights of the data to be analyzed. A highlight refers to, for example, a portion of the data to be analyzed that best represents the data to be analyzed (a notable object) and may further include supplemental information related to that portion. A summary is an example of "text data summarizing data."

[0033] Here, the effect of automatic generation of metadata by the data analysis device 1 will be explained in more detail.

[0034] Generally, when building a data catalog to promote data utilization, the following three aspects are important: recall, search, and understanding. Here, from the recall perspective, a data catalog is required to enable users to recall what they can learn using the data catalog. Furthermore, from the search perspective, a data catalog is required to enable searches that allow users to easily find desired data. From the understanding perspective, a data catalog is required to enable users to smoothly grasp the contents of the data and analysis results. Building a data catalog that satisfies these requirements for each perspective allows users to smoothly find desired data through the data catalog, thereby favorably promoting the analysis of accumulated data and data-driven decision-making. On the other hand, simply using a superficial data profile as metadata results in a data catalog that is insufficient in terms of recall, search, and understanding. Furthermore, manually adding useful metadata is costly and has limitations. Taking the above into consideration, the data analysis device 1 in this embodiment automatically generates useful metadata for the data being analyzed, taking into account insights into the data being analyzed.

[0035] (3) Functional Blocks 4 is an example of a functional block of the processor 11. The processor 11 functionally includes a data analysis unit 15 and a UI (User Interface) control unit 16.

[0036] The data analysis unit 15 refers to various information stored in the storage device 4 and the memory 12, and generates metadata for data to be analyzed selected from the data 6 registered in the data catalog 5. In this case, the data analysis unit 15 may select the data to be analyzed based on user input information supplied from the UI control unit 16, or may select the data to be analyzed from the data 6 registered in the data catalog 5 based on a predetermined rule (including random sampling). Furthermore, when any of the data 6 is updated, the data analysis unit 15 may generate metadata for the updated data 6, and update the metadata associated with the updated data 6 in the data catalog 5 based on the generation result. Then, the data analysis unit 15 supplies the metadata generation result to the UI control unit 16.

[0037] The UI control unit 16 controls the reception of user input and the display of information to be viewed by the user. For example, the UI control unit 16 may supply information specifying data to be analyzed to the data analysis unit 15 based on input information (i.e., external input) supplied from the input device 2. The UI control unit 16 also generates display information based on the metadata generation result generated by the data analysis unit 15, and supplies the generated display information to the display device 3, thereby controlling the display of the display device 3. Specific processing by the UI control unit 16 will be described later with reference to display examples.

[0038] Fig. 5 shows an example of functional blocks of the data analysis unit 15. Functionally, the data analysis unit 15 has a data acquisition unit 51, an analytical query generation unit 52, an insight generation unit 53, a metadata generation unit 54, and an output unit 55. Note that in Fig. 5, blocks where data is exchanged are connected by solid lines, but the combination of blocks where data is exchanged is not limited to that in Fig. 3. The same applies to other functional block diagrams described later.

[0039] The data acquisition unit 51 reads any of the data 6 registered in the data catalog 5 from the storage device 4 as data to be analyzed. For example, when the data analysis unit 15 receives information specifying data to be analyzed from the UI control unit 16, it reads the specified data 6 from the storage device 4 as data to be analyzed. In another example, the data analysis unit 15 detects data 6 for which metadata has not been generated or for which metadata needs to be updated, and reads the detected data 6 from the storage device 4 as data to be analyzed. Note that the data analysis unit 15 may sequentially read each of the data 6 registered in the data catalog 5 from the storage device 4 as data to be analyzed. The data acquisition unit 51 supplies the acquired data to be analyzed to the analytical query generation unit 52 and the insight generation unit 53.

[0040] The analytical query generation unit 52 generates an analytical query based on the analysis target data supplied from the data acquisition unit 51. A method for generating an analytical query will be described later. The analytical query generation unit 52 supplies the generated analytical query to the insight generation unit 53.

[0041] The insight generation unit 53 generates insights into the analysis target data based on the analysis target data supplied from the data acquisition unit 51 and the analysis query supplied from the analysis query generation unit 52. The method of generating insights will be described later. The insight generation unit 53 supplies the generated insights to the metadata generation unit 54.

[0042] The metadata generation unit 54 generates metadata for the data to be analyzed based on the insights supplied from the insight generation unit 53. In this case, the metadata generation unit 54 generates metadata including, for example, at least one of a summary, a chart, a tag, an analysis type, and a highlight related to the data to be analyzed. Note that if the insights generated by the insight generation unit 53 correspond to a summary and a chart, the metadata generation unit 54 may include the summary and the chart in the metadata. The metadata generation unit 54 supplies the generated metadata to the output unit 55. The output unit 55 registers the metadata generated by the metadata generation unit 54 in the data catalog 5 as metadata for the data to be analyzed, in association with the data to be analyzed.

[0043] The data analysis unit 15 and the UI control unit 16 described in FIG. 4 , and the data acquisition unit 51, the analytical query generation unit 52, the insight generation unit 53, the metadata generation unit 54, and the output unit 55 described in FIG. 5 can be realized, for example, by the processor 11 executing a program. Alternatively, the necessary programs may be recorded in any nonvolatile storage medium and installed as needed to realize each component. At least some of these components may not necessarily be realized by software programs, but may be realized by any combination of hardware, firmware, and software. At least some of these components may be realized using a user-programmable integrated circuit, such as an FPGA (Field-Programmable Gate Array) or a microcontroller. In this case, the integrated circuit may be used to realize a program consisting of the above-described components. At least some of the components may be configured by an ASSP (Application Specific Standard Produce), an ASIC (Application Specific Integrated Circuit), or a quantum processor (quantum computer control chip). In this way, each component may be realized by various hardware. The same applies to other embodiments described later. Furthermore, each of these components may be realized by the cooperation of multiple computers, for example, using cloud computing technology.

[0044] (4) Analytical query generation Next, the analytical query generation process executed by the analytical query generation unit 52 will be described. The analytical query generation unit 52 analyzes the data to be analyzed and generates an analytical query. In this case, the analytical query generation unit 52 may generate the analytical query using any method. For example, the analytical query generation unit 52 may generate the analytical query using LLM. Hereinafter, generation of an analytical query using LLM will be described as a representative example.

[0045] Figure 6 shows an overview of analytical query generation using LLM.

[0046] The analytical query generation unit 52 first generates a prompt to be input to the LLM from the data to be analyzed. In this case, the analytical query generation unit 52 outlines the contents of the data to be analyzed and generates a prompt that is a sentence that instructs the generation of a query appropriate for the outlined data to be analyzed. In the example of FIG. 6, the analytical query generation unit 52 generates a prompt that includes a first sentence indicating the outlined data to be analyzed and the following second sentence: "What kind of analysis would likely produce interesting results from the above data? Please output {n} queries in natural language." Here, the "simplified data to be analyzed" is, for example, text data that summarizes a table, which is the data to be analyzed, by column or row. The above prompt is an example of "text data that includes an outline of the data and requests the generation of a predetermined number of queries based on the outline."

[0047] Next, the analytical query generation unit 52 inputs a prompt to the LLM to acquire one or more analytical queries from the LLM. For example, if machine-learned configuration information (model information) of the LLM is stored in the storage device 4, the analytical query generation unit 52 inputs a prompt to the LLM configured by referring to the machine-learned parameters, etc., indicated by the configuration information, and acquires text data output by the LLM in response to the input. This text data indicates text representing one or more queries in response to the prompt. In the example of FIG. 6, the LLM outputs text data representing n queries ("Query 1," "Query 2," "Query 3," ..., "Query n") because the prompt specifies that n queries (n is an integer greater than or equal to 1) should be output. Here, the number n of analytical queries may be any number. Using multiple analytical queries improves the comprehensiveness of subsequent analyses. "n" is an example of a predetermined number.

[0048] The device that executes the LLM may be an external device capable of data communication with the data analysis device 1. In this case, the analytical query generation unit 52 transmits an execution instruction signal for the LLM, including a prompt, to the external device via the interface 13, and receives a response signal, including text data output by the LLM, from the external device via the interface 13. The external device that receives the execution instruction signal inputs the prompt included in the execution instruction signal to the LLM, and transmits a response signal, including text data output by the LLM, to the data analysis device 1. Similarly, when a processing block other than the analytical query generation unit 52 uses the LLM or another model, the data analysis device 1 may obtain the execution results of the model from the external device instead of executing the model itself.

[0049] Figure 7 shows a specific example of generating an analytical query using LLM. The data to be analyzed shown in Figure 7 is a table with the following column names: "Month (year)," "Page Name," "URL," and "Number of Page Views."

[0050] In this case, the analytical query generation unit 52 provides the prompt with an opening template sentence of "Please analyze the data having the following columns," and an ending template sentence of "What kind of analysis of the above data would likely produce interesting results? Please output two queries in natural language." Here, the number of queries "n" in Figure 6 is set to "2."

[0051] Furthermore, the analytical query generation unit 52 provides sequence data "Column Name: [Month (Year), Page Name, URL, Number of Page Views]" in which the column names of the data to be analyzed are concatenated with commas, and an explanation of each column name, between the opening and closing template sentences. The explanation of each column name is, for example, a sentence including the category (variable type) indicated by each column name and a sample of the field belonging to each column (which may be all fields). In the example of FIG. 7, the following explanations corresponding to the column names "Month (Year)," "Page Name," "URL," and "Number of Page Views" are provided in the prompt. "Month (year) is a time variable and has values ​​such as March 2023, July 2023, etc." "Page name is a categorical variable, with values ​​such as xxxxx, yyyyy, etc." "URL is a categorical variable, with values ​​such as www.xxxxx.html, www.yyyyy.html, etc." "Pageviews are a quantitative variable, a series of integer values."

[0052] The analytical query generation unit 52 may determine the type of the variable by any method. For example, the analytical query generation unit 52 may determine the category by referring to a lookup that indicates combinations of column name candidates and variable types.

[0053] Next, the analytical query generation unit 52 inputs the above-mentioned prompt into the LLM, and thereby acquires two queries (query 1 and query 2) output by the LLM as analytical queries. Here, the analytical query generation unit 52 acquires query 1 "analyze the trend in the number of page views by month" and query 2 "compare the number of page views by page name" as analytical queries.

[0054] As described above, the analytical query generation unit 52 obtains an analytical query suitable for analyzing the data to be analyzed from the execution result of the LLM, which specifies a prompt using the column names of the data to be analyzed. This allows the analytical query generation unit 52 to have the LLM understand the outline of the data, and then automatically generate a recommended analytical query. Furthermore, by combining this with the generation of insights in the subsequent stage, analysis based on various perspectives becomes possible.

[0055] (5) Generate insights Next, the insight generation process executed by the insight generation unit 53 will be described. The insight generation unit 53 generates insights using the data to be analyzed and an analytical query. An insight is generated for each analytical query. In this case, the insight generation unit 53 may generate insights using any method. For example, the insight generation unit 53 may generate insights using at least one of a chart recommendation model that recommends charts that are considered useful to humans, or a question-answering model that outputs answers to input queries. Hereinafter, the generation of insights using a chart recommendation model or a question-answering model will be described as a representative example.

[0056] Figure 8 shows an overview of the insight generation process. The process shown in Figure 8 is performed for each analytical query.

[0057] First, the insight generation unit 53 extracts some columns or rows of the analysis target data related to the analytical query based on the relevance with the analytical query, and uses the extracted some columns or rows in subsequent analysis processing. In FIG. 8, "analysis target data (before extraction)" refers to the analysis target data before extraction based on the above-mentioned relevance, and "analysis target data (after extraction)" refers to data consisting of columns or rows extracted from the analysis target data based on the above-mentioned relevance. For example, the insight generation unit 53 extracts columns (or rows instead of columns) of the analysis target data whose relevance with the analytical query is equal to or greater than a predetermined threshold. The above-mentioned threshold is stored in advance in the memory 12, for example.

[0058] Here, the above-mentioned relevance is an arbitrary index value that represents the degree of relevance between the analytical query and each column or row that constitutes the data to be analyzed. For example, the above-mentioned relevance may be the similarity between the character string of the analytical query and the sequential data of each column or row, or the similarity between the embedding vectors of the analytical query and the above-mentioned sequential data. Here, the sequential data is a character string in which the elements of the column or row are concatenated in series, and may be in any format.

[0059] The insight generation unit 53 may perform the extraction process based on the relevance described above only when a predetermined condition is satisfied, such as when the data to be analyzed is huge and not suitable for input (prompt) to a chart recommendation model or a question-answering model. For example, when the number of characters in the data to be analyzed is equal to or greater than a predetermined threshold, the insight generation unit 53 determines that the current data to be analyzed is not suitable for input to a chart recommendation model or a question-answering model, and performs the extraction process based on the relevance described above.

[0060] Next, the insight generation unit 53 inputs data based on the extracted analysis target data and the analytical query into the chart recommendation model or the question answering model. Then, the insight generation unit 53 acquires insights based on the data output by the chart recommendation model or the question answering model.

[0061] Here, the chart recommendation model is any model that, when a query and a table are input, outputs aggregated data (charts) that are data obtained by aggregating the table related to the query. For example, it is a learning model based on deep learning, and an example of a chart recommendation model includes QuickInsights. Parameters and the like for configuring the chart recommendation model are stored in advance in the memory 12, etc., and the insight generation unit 53 configures the chart recommendation model by referring to the parameters and the like, and inputs the extracted analysis target data and the analysis query into the chart recommendation model. Then, the insight generation unit 53 acquires, as an insight, the chart output by the chart recommendation model in response to the above-mentioned input.

[0062] Preferably, the insight generation unit 53 may generate a summary of the data to be analyzed by verbalizing the chart output by the chart recommendation model, and may adopt the summary as an insight.

[0063] In this case, in a first example, the insight generation unit 53 inputs a prompt to an LLM such as ChatGPT to instruct the chart output by the chart recommendation model to be verbalized (converted into text). Then, the insight generation unit 53 acquires the answer output by the LLM such as ChatGPT in response to the above input as a summary of the data to be analyzed.

[0064] In a second example, the insight generation unit 53 uses a data-to-text model (document transformation model) based on the chart (assumed to be a table here) output by the chart recommendation model and the analytical query to acquire text data describing the chart according to the content of the analytical query as a summary of the data to be analyzed. In this case, the insight generation unit 53 converts the chart into sequence data, combines the analytical query with the sequence data of the chart as information representing the context, and inputs the combined data into the data-to-text model. The insight generation unit 53 then acquires the text output by the data-to-text model based on the above input as a summary of the data to be analyzed. Here, the data-to-text model is a model that learns the relationship between sequence data to which information representing the context is added and text data representing the explanation of the sequence data. The data-to-text model is, for example, a learning model based on deep learning, and an example of a data-to-text model includes T5 (Text-to-Text Transfer Transformer). Parameters and the like for constructing the Data-to-Text model are stored in advance in the memory 12, etc., and the insight generation unit 53 constructs the Data-to-Text model by referring to the parameters and the like.

[0065] A question-answering model is any model that outputs an answer to a query when the query is input. For example, it is a learning model based on deep learning, and examples of question-answering models include ChatGPT and Unified SKG. Parameters and the like for configuring the question-answering model are pre-stored in the memory 12, etc. The insight generation unit 53 configures the question-answering model by referring to the parameters and the like, and inputs a prompt based on the extracted data to be analyzed and the analytical query into the question-answering model. The insight generation unit 53 then acquires the answer output by the question-answering model in response to the above input as an insight. The acquired answer corresponds to a summary of the data to be analyzed.

[0066] Fig. 9 shows a specific example of insight generation processing using a chart recommendation model and a question-answering model. Here, an example is shown using the analysis target data and analytical queries (Query 1 and Query 2) shown in Fig. 7.

[0067] The insight generation unit 53 inputs the analysis target data regarding the number of page views for each page name per month and query 1 "Analyze the trend in the number of page views per month" into the chart recommendation model. As a result, the chart recommendation model outputs a graph regarding the number of page views per month. The insight generation unit 53 also generates text that is verbalized from the graph output by the chart recommendation model using ChatGPT or the like. This text is used as a summary of the analysis target data. The insight generation unit 53 then acquires, as insights, the graph output by the chart recommendation model and a summary of the analysis target data that is a verbalization of the graph. Note that the insight generation unit 53 may extract columns or rows of the analysis target data based on the relevance with query 1, and use only the extracted columns or rows in the above-described processing.

[0068] The insight generation unit 53 also inputs data including the above-mentioned analysis target data and query 2 "compare the number of page views for each page name" into the question-answering model. The data input to the question-answering model is a query requesting a comparison of the number of page views for each page name of the above-mentioned analysis target data. As a result, the question-answering model outputs an answer representing a summary of the analysis target data. The insight generation unit 53 then acquires the answer output by the question-answering model as an insight. Note that the insight generation unit 53 may extract columns or rows of the analysis target data based on the relevance with query 2, and use only the extracted columns or rows in the above-mentioned processing.

[0069] In this way, the insight generation unit 53 can suitably generate insights based on the analysis target data and each analytical query.

[0070] Here, a supplementary explanation will be given regarding the selection of a model to be used by the insight generating unit 53.

[0071] In a first example, the insight generation unit 53 may select whether to use a chart recommendation model or a question-answering model for each analytical query. In this case, the insight generation unit 53 inputs a prompt to an LLM such as ChatGPT asking whether to use the chart recommendation model or the question-answering model when using a target analytical query. Then, the insight generation unit 53 selects whether to use the chart recommendation model or the question-answering model based on the answer output by the LLM in response to the input.

[0072] In a second example, the insight generation unit 53 may use both the chart recommendation model and the question-answering model and integrate their execution results. In this case, for example, the insight generation unit 53 acquires text that integrates a summary generated using the chart recommendation model and a summary generated using the question-answering model as an insight into the data to be analyzed. In the above-mentioned summary integration, the insight generation unit 53 generates, for example, a prompt that instructs the user to integrate the summary generated using the chart recommendation model and the summary generated using the question-answering model, and inputs the prompt into the LLM. Then, the insight generation unit 53 acquires the answer output by the LLM in response to the above-mentioned input as an insight representing the final summary of the data to be analyzed.

[0073] (5) Generate Metadata Next, a description will be given of the metadata generation process executed by the metadata generation unit 54. The metadata generation unit 54 generates metadata based on the insights generated by the insight generation unit 53. In this case, for example, the metadata generation unit 54 may use the summary and / or chart, which is the insight, as metadata as is. In another example, the metadata generation unit 54 may generate tags to be added to the analysis target data, analysis types effective for analyzing the analysis target data, highlights of the analysis target data, etc., from the summary, which is the insight.

[0074] For example, the metadata generation unit 54 inputs a prompt to the LLM instructing it to generate tags and analysis types from the summary, which is an insight. Then, based on the response output by the LLM in response to the above input, the metadata generation unit 54 determines the tags and analysis types to be used as metadata for the data to be analyzed.

[0075] Conventionally, tags and analysis types have had to be generated manually, requiring an operator to visually check the contents of the data to be analyzed and manually assign tags and generate analysis types. In contrast, in this embodiment, the data analysis device 1 uses the generated insights to automatically assign tags and analysis types without manually analyzing the contents of the data. These tags and analysis types are preferably used as metadata that makes it easier for users to search for desired data.

[0076] In another example, the metadata generation unit 54 inputs a prompt to the LLM instructing it to generate highlights from the insight summaries. The metadata generation unit 54 then determines highlights to be used as metadata for the analysis target data based on the text data output by the LLM in response to the above input. Generally, it can be difficult for a user to interpret the summaries of individual analysis target data. In such cases, the user can easily understand the analysis results obtained by viewing the highlights and determine whether or not to view more detailed information. In this way, by generating highlights from insight summaries, the data analysis device 1 can support the user's understanding, facilitate a smooth path to analysis of items of interest, and improve the user experience.

[0077] (6) Display example 10 is a display example of a data display screen that allows a user to check data and metadata of the data. The UI control unit 16 generates display information for displaying the data display screen based on the processing results generated by the data analysis unit 15, and transmits the generated display information to the display device 3 via the interface 13, thereby causing the display device 3 to display the data display screen. Here, the UI control unit 16 mainly has a data display area 61, a metadata display area 62, and a metadata addition button 63.

[0078] The UI control unit 16 displays details of data 6 (here, data X) specified by the user in a data display area 61, and displays metadata associated in the data catalog 5 with the data X displayed in the data display area 61 in a metadata display area 62. Here, the data analysis device 1 has not performed metadata generation processing for the data X, and only the default metadata for the data X is recorded in the data catalog 5.

[0079] When the UI control unit 16 detects that the metadata addition button 63 has been selected, it supplies an instruction to the data analysis unit 15 to execute a metadata generation process specifying the data X displayed in the data display area 61 as the data to be analyzed. Then, the data analysis unit 15 receives the execution instruction and executes the metadata generation process with the data X as the data to be analyzed. The UI control unit 16 is an example of a "display control means."

[0080] 11 is an example of the display on the data display screen after the metadata addition button 63 is selected. The data analysis unit 15 performs a metadata generation process using data X as the data to be analyzed, and the UI control unit 16 additionally displays the metadata of data X generated by the data analysis unit 15 in the metadata display area 62.

[0081] In the example of FIG. 11, the data analysis unit 15 generates a tag, an analysis type, and a highlight for data X as metadata for data X. Here, the data analysis unit 15 generates, for the tag, a value such as "AA" for the location "Location" for data X. The data analysis unit 15 also generates "access analysis" and "segmentation" as values ​​for the analysis type for data X. The data analysis unit 15 also generates at least "Highlight 1" as one of the highlights for data X, which indicates that the "target of interest" is "page views" and the "insight" is "decreasing monthly." Here, a hyperlink for displaying the insight used to generate "Highlight 1" is provided in the value column for "Highlight 1."

[0082] According to the display example of FIG. 11, the user can conveniently check the generated metadata.

[0083] 12 shows an example of the display on the data display screen after the hyperlink for "Highlight 1" is selected. In this case, the UI control unit 16 pops up an insight display window 64, which displays the insight used to generate "Highlight 1," on the data display screen in association with "Highlight 1."

[0084] In this case, the UI control unit 16 detects that the hyperlink for "Highlight 1" has been selected, and obtains the insight for "Highlight 1" generated by the data analysis unit 15 from the data analysis unit 15. Here, as shown in Fig. 9, the data analysis unit 15 generates a chart and a summary based on query 1 and the data to be analyzed, and generates "Highlight 1" from the generated summary. Therefore, the UI control unit 16 displays the insight display window 64 showing the chart and summary shown in Fig. 9.

[0085] Thus, according to the display example of Figure 12, if a user shows interest in a generated highlight, insights about the highlight can be displayed, allowing the user to check the details of the analysis results as needed.

[0086] In the above display example, the data analysis device 1 generates metadata for the data 6 manually specified by the user, but this is not limiting. For example, the data analysis device 1 may execute a metadata generation process when any data 6 is updated or when tag presets are changed. In this way, when the data analysis device 1 determines that a predetermined condition is met that indicates the need for metadata generation or updating, it automatically generates metadata and updates the data catalog 5, thereby keeping the data catalog 5 in accordance with the latest data 6.

[0087] (7) Processing flow FIG. 13 is an example of a flowchart showing an outline of the processing executed by the data analysis apparatus 1.

[0088] First, the data analysis device 1 acquires data to be analyzed (step S11). In this case, the data analysis device 1 selects the data to be analyzed from the data 6 stored in the storage device 4.

[0089] The data analysis apparatus 1 then generates an analytical query based on the analysis target data acquired in step S11 (step S12). The data analysis apparatus 1 then generates insights into the analysis target data based on the analysis target data acquired in step S11 and the analytical query generated in step S12 (step S13). The data analysis apparatus 1 then generates metadata for the analysis target data based on the insights generated in step S13 (step S14). The data analysis apparatus 1 then outputs the metadata generated in step S14 (step S15). In this case, the data analysis apparatus 1 associates the metadata with the analysis target data and registers it in the data catalog 5, and displays information about the generated metadata on the display device 3.

[0090] Second Embodiment 14 shows the configuration of a data analysis system 100 A. The data analysis system 100 A mainly includes a data analysis device 1 A and a terminal device 8. The data analysis device 1 A and the terminal device 8 perform data communication via a network 7.

[0091] The data analysis apparatus 1A is one or more devices that function as a server (including a cloud server), and performs processing related to the data analysis executed by the data analysis apparatus 1 in the first embodiment. In this case, the data analysis apparatus 1A receives input information from the terminal apparatus 8 via the network 7, which the data analysis apparatus 1 receives from the input device 2 in the first embodiment. The data analysis apparatus 1A also transmits display information that the data analysis apparatus 1 transmitted to the display device 3 in the first embodiment to the terminal apparatus 8 via the network 7. The data analysis apparatus 1A also stores the data catalog 5 and data 6, or references the data catalog 5 and data 6 via the network 7.

[0092] The terminal device 8 is a terminal having an input function, a display function, and a communication function, and functions as the input device 2 and the display device 3 in the first embodiment. The terminal device 8 may be, for example, a personal computer, a tablet terminal, a PDA (Personal Digital Assistant), or the like. The terminal device 8 transmits input information generated based on the received user input to the data analysis device 1A via the network 7. Furthermore, when the terminal device 8 receives display information from the data analysis device 1A, it displays information based on the display information.

[0093] The data analysis apparatus 1A according to the second embodiment can preferably perform the input processing and output processing that the data analysis apparatus 1 according to the first embodiment performs on the user of the terminal device 8.

[0094] Third Embodiment 15 is a functional block diagram of a data analysis apparatus 1X. The data analysis apparatus 1X mainly includes an analytical query generation means 52X, an insight generation means 53X, and a metadata generation means 54X. The data analysis apparatus 1X may be composed of multiple devices.

[0095] The analytical query generation means 52X generates an analytical query for analyzing the data from the data. The analytical query generation means 52X can be, for example, the analytical query generation unit 52 in the first or second embodiment.

[0096] The insight generation means 53X generates an insight into the data based on the data and the analytical query. The insight generation means 53X can be, for example, the insight generation unit 53 in the first or second embodiment.

[0097] The metadata generating means 54X generates metadata of the data based on the insight. The metadata generating means 54X can be, for example, the metadata generating unit 54 in the first or second embodiment.

[0098] 16 is an example of a flowchart executed by the data analysis apparatus 1X. The analytical query generation means 52X generates an analytical query for analyzing data from the data (step S21). The insight generation means 53X generates an insight of the data based on the data and the analytical query (step S22). The metadata generation means 54X generates metadata of the data based on the insight (step S23).

[0099] The data analysis device 1X according to the third embodiment can automatically generate metadata suitable for recall, search, and understanding based on insights into data.

[0100] In addition, part or all of the above-described embodiments (including variations, the same applies below) may also be described as, but are not limited to, the following supplementary notes. Furthermore, not only the devices, methods, and storage media described in the supplementary notes, but also various hardware, software, various recording means (including storage media) for recording software, or systems may be made to depend on part or all of the configurations described in the supplementary notes, as long as they do not deviate from the above-described embodiments.

[0101] [Appendix 1] an analytical query generation means for generating an analytical query for analyzing the data from the data; an insight generation means for generating insights into the data based on the data and the analytical query; a metadata generating means for generating metadata of the data based on the insight; A data analysis device having: [Appendix 2] the insight generation means generates text data summarizing the data as the insight; The data analysis device described in Appendix 1, wherein the metadata generation means generates the metadata based on the text data, the metadata including at least one of tags related to the data, analysis types related to the data, and highlights related to the data. [Appendix 3] the analytical query generation means inputs text data, which includes an outline of the data and requests the generation of a predetermined number of queries according to the outline, into a language model that has been machine-learned to output an answer to a prompt when the prompt is input, as the prompt to the language model, and obtains the predetermined number of queries output by the language model as the analytical queries. [Appendix 4] the data is a table, The data analysis device described in Appendix 1, wherein the insight generation means extracts some columns or rows from the data based on the relevance of each column or row of the data to the analytical query, and generates the insight based on the extracted some columns or rows. [Appendix 5] The data analysis device according to claim 1, wherein the insight generation means acquires, as the insight, at least one of a chart output by a chart recommendation model based on the data and the analytical query and text data that verbalizes the chart. [Appendix 6] 2. The data analysis device according to claim 1, wherein the insight generation means acquires, as the insight, text data output by a question-answering model based on the data and the analytical query. [Appendix 7] The data is searchable data registered in a data catalog, 2. The data analysis apparatus according to claim 1, further comprising an output means for updating the data catalog based on the metadata. [Appendix 8] 2. The data analysis device according to claim 1, further comprising a display control means for displaying the metadata together with the data on a display device. [Appendix 9] The computer generating an analytical query from the data to analyze the data; generating insights into the data based on the data and the analytical query; generating metadata for the data based on the insights; Data analysis methods. [Appendix 10] generating an analytical query from the data to analyze the data; generating insights into the data based on the data and the analytical query; A program that causes a computer to execute a process of generating metadata for the data based on the insight. [Appendix 11] A storage medium storing the program of Appendix 10.

[0102] In each of the above-described embodiments, the program can be stored using various types of non-transitory computer-readable media and supplied to a computer processor or the like. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic storage media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). The program may also be supplied to a computer by various types of transitory computer-readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer-readable medium can supply the program to a computer via a wired communication path such as an electric wire or optical fiber, or via a wireless communication path.

[0103] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various modifications within the scope of the present invention that would be understood by those skilled in the art can be made to the configuration and details of the present invention. In other words, the present invention naturally includes various modifications and alterations that would be possible for those skilled in the art based on the entire disclosure, including the claims, and the technical ideas. Furthermore, the disclosures of the above-cited patent and non-patent documents are incorporated herein by reference. [Explanation of symbols]

[0104] 1, 1A, 1X Data Analysis Equipment 2 Input devices 3 Display device 5. Data Catalog 6(6A,6B,...) data 7 Network 8 Terminal Equipment 100, 100A Data Analysis System

Claims

1. an analytical query generation means for generating an analytical query for analyzing the data from the data; an insight generation means for generating insights into the data based on the data and the analytical query; a metadata generating means for generating metadata of the data based on the insight; A data analysis device having:

2. the insight generation means generates text data summarizing the data as the insight; The data analysis device according to claim 1 , wherein the metadata generation means generates the metadata including at least one of a tag related to the data, an analysis type related to the data, and a highlight related to the data, based on the text data.

3. 2. The data analysis device according to claim 1, wherein the analytical query generation means inputs text data, which includes an outline of the data and requests the generation of a predetermined number of queries according to the outline, to a language model that has been machine-learned to output an answer to a prompt when the prompt is input, as the prompt to the language model, and acquires the predetermined number of queries output by the language model as the analytical queries.

4. the data is a table, 2. The data analysis device according to claim 1, wherein the insight generation means extracts some columns or rows from the data based on the relevance of each column or row of the data to the analytical query, and generates the insight based on the extracted some columns or rows.

5. 2. The data analysis device according to claim 1, wherein the insight generation means acquires, as the insight, at least one of a chart output by a chart recommendation model based on the data and the analytical query and text data that verbalizes the chart.

6. The data analysis device according to claim 1 , wherein the insight generation means acquires, as the insight, text data output by a question-answering model based on the data and the analytical query.

7. The data is searchable data registered in a data catalog, The data analysis apparatus according to claim 1 , further comprising an output unit that updates the data catalog based on the metadata.

8. The data analysis device according to claim 1 , further comprising a display control means for displaying the metadata together with the data on a display device.

9. The computer generating an analytical query from the data to analyze the data; generating insights into the data based on the data and the analytical query; generating metadata for the data based on the insights; Data analysis methods.

10. generating an analytical query from the data to analyze the data; generating insights into the data based on the data and the analytical query; A program that causes a computer to execute a process of generating metadata for the data based on the insight.

Images