Data processing device, data processing method and data processing program

Abstract

To provide a data processing device capable of extracting the feature of data in the data to be processed.SOLUTION: A data processing device includes: a replicating unit which replicates a first table storing data to be analyzed, and which creates a second table; a joining unit which cross-joins the first table and the second table based on an identification string for identifying the data, and which creates a new third table; a calculating unit which calculates, in the third table created by the joining unit, a difference between the string of explanatory variables in the first table and the string of explanatory variables in the second table, and which stores the difference in the string newly created in the third table; and an extracting unit that extracts a feature quantity for the string created by the calculating unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a data processing device, a data processing method, and a data processing program. [Background technology]

[0002] Materials are designed by repeatedly producing prototypes while adjusting the composition based on the experience of the material developer. For example, Patent Document 1 discloses a material design device, a material design method, and a material design program for polymer materials that simultaneously satisfy multiple desired material properties. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2021 / 045058 Summary of the Invention [Problem to be solved by the invention]

[0004] The experimental data from past material prototypes reflects the thinking of material developers. However, there was no technology available to analyze what thinking the material developers had based the past experimental data on when prototyping materials, or in other words, what characteristics exist in the data to be processed, such as experimental data.

[0005] The present disclosure has been made in consideration of the above points, and aims to provide a data processing device, a data processing method, and a data processing program that are capable of extracting data characteristics from data to be processed. [Means for solving the problem]

[0006] A data processing device according to a first aspect of the present disclosure includes a duplication unit that duplicates a first table storing data to be analyzed to generate a second table; a join unit that cross-joins the first table and the second table based on an identification column for identifying the data to generate a new third table; a calculation unit that calculates a difference between a column of explanatory variables in the first table and a column of explanatory variables in the second table in the third table generated by the join unit and stores the difference in a newly generated column in the third table; and an extraction unit that extracts features from the column generated by the calculation unit.

[0007] A data processing device according to a second aspect of the present disclosure is the data processing device according to the first aspect, wherein the extraction unit extracts, as a feature, information related to the identification sequence used by the combination unit for cross-combining.

[0008] A data processing device according to a third aspect of the present disclosure is the data processing device according to the first aspect, wherein the extraction unit extracts the feature amount by dimensionality reduction processing.

[0009] A data processing device according to a fourth aspect of the present disclosure is the data processing device according to the first aspect, wherein the extraction unit extracts features using a result of unsupervised learning that generates latent variables.

[0010] A data processing device according to a fifth aspect of the present disclosure is the data processing device according to the first aspect, wherein the extraction unit extracts the feature amount using a result of supervised learning.

[0011] A data processing method according to a sixth aspect of the present disclosure includes a processor that copies a first table that stores data to be analyzed to generate a second table, cross-links the first table and the second table based on an identification column for identifying the data to generate a new third table, calculates the difference between a column of explanatory variables in the first table and a column of explanatory variables in the second table in the generated third table, stores the difference in a newly generated column in the third table, and executes a process of extracting features from the generated column.

[0012] A data processing program according to a seventh aspect of the present disclosure causes a computer to execute the following process: duplicating a first table in which data to be analyzed is stored to generate a second table; cross-linking the first table and the second table based on an identification column for identifying the data to generate a new third table; calculating a difference between a column of explanatory variables in the first table and a column of explanatory variables in the second table in the generated third table; storing the difference in a newly generated column in the third table; and extracting features from the generated column. [Effects of the Invention]

[0013] According to the present disclosure, it is possible to provide a data processing device, a data processing method, and a data processing program that are capable of extracting data characteristics from data to be processed. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a diagram illustrating an overview of a data processing device according to an embodiment of the disclosed technology. [Figure 2] FIG. 2 is a block diagram showing a hardware configuration of the data processing device. [Figure 3] FIG. 2 is a block diagram illustrating an example of a functional configuration of a data processing device. [Figure 4] FIG. 10 illustrates an example of a replication process. [Figure 5] FIG. 10 is a diagram illustrating an example of a joining process. [Figure 6] FIG. 10 is a diagram illustrating an example of calculation processing. [Figure 7] FIG. 10 illustrates an example of extraction processing. [Figure 8] 10A and 10B are diagrams illustrating the effect of data processing by the data processing device. [Figure 9] FIG. 10 is a diagram showing a state in which feature amounts are stored in a table when extraction processing is performed directly on processing target data without performing join processing and calculation processing. [Figure 10] 10 is a graph showing the relationship between the feature amounts shown in FIG. 9. [Figure 11]10 is a flowchart showing the flow of data processing by the data processing device. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of the present disclosure will be described below with reference to the drawings. The same reference numerals are used throughout the drawings to designate identical or equivalent components and parts. The dimensional proportions of the drawings are exaggerated for illustrative purposes and may differ from the actual proportions.

[0016] 1 is a diagram showing an overview of a data processing device according to this embodiment. The data processing device 10 is a device that executes data processing on processing target data and outputs the processing results. The data processing device 10 is, for example, a personal computer that stores a computer program that executes data processing.

[0017] In this embodiment, the data processing device 10 accepts data in a table format as data to be processed. The data in a table format may be of any type, such as data stored in a database or entered in a spreadsheet. The data to be processed may include, for example, past test data for the prototyping of a product, such as the blending amounts of materials used to manufacture a certain product, material characteristics (such as molecular weight, softening point, and particle shape) obtained from experimental results such as analysis or simulation results, material processing time, and test results for the prototype product.

[0018] The data processing device 10 then executes processing to extract features of the processing target data as data processing for the processing target data, and outputs the extracted features as processed data.

[0019] The data processing device 10 performs data processing on the target data to extract features of the target data, and outputs the extracted features, thereby making it possible to present what features the target data possesses. If the target data is past test data, it is possible to present what features the past test data possesses. Note that, as will be described later, the data processing device 10 does not simply perform feature extraction processing on the target data, but performs preprocessing on the target data before executing feature extraction processing.

[0020] Next, a description will be given of the hardware configuration of the data processing device 10. Fig. 2 is a block diagram showing the hardware configuration of the data processing device 10.

[0021] 2, the data processing device 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage 14, an input unit 15, a display unit 16, and a communication interface (I / F) 17. Each component is connected to each other via a bus 19 so as to be able to communicate with each other.

[0022] The CPU 11 is a central processing unit that executes various programs and controls each component. That is, the CPU 11 reads a program from the ROM 12 or the storage 14 and executes the program using the RAM 13 as a work area. The CPU 11 controls the above components and performs various arithmetic operations in accordance with the program recorded in the ROM 12 or the storage 14. In this embodiment, the ROM 12 or the storage 14 stores a data processing program that executes data processing on data to be processed.

[0023] The ROM 12 stores various programs and various data. The RAM 13 temporarily stores programs or data as a working area. The storage 14 is configured with a storage device such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory, and stores various programs including the operating system and various data.

[0024] The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used to perform various inputs.

[0025] The display unit 16 is, for example, a liquid crystal display, and displays various information. The display unit 16 may also function as the input unit 15 by adopting a touch panel system.

[0026] The communication interface 17 is an interface for communicating with other devices, and uses standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark).

[0027] When executing the above data processing program, the data processing device 10 uses the above hardware resources to realize various functions. The functional configuration realized by the data processing device 10 will be described below.

[0028] FIG. 3 is a block diagram showing an example of the functional configuration of the data processing device 10. As shown in FIG.

[0029] 3, the data processing device 10 has, as functional components, a duplication unit 101, a combination unit 102, a calculation unit 103, and an extraction unit 104. Each functional component is realized by the CPU 11 reading and executing a data processing program stored in the ROM 12 or the storage 14.

[0030] The duplication unit 101 executes a duplication process to duplicate data to be processed. That is, the duplication unit 101 duplicates data to be processed in a table format. The data to be processed in a table format is an example of a first table of the present disclosure, and the table generated by the duplication process is an example of a second table of the present disclosure. The duplication unit 101 may store the table generated by the duplication process in the RAM 13 or the storage 14.

[0031] FIG. 4 is a diagram illustrating an example of a duplication process performed by the duplication unit 101. The duplication unit 101 duplicates table 200A to generate table 200B having the exact same data. The columns “Test Set” and “#” are columns storing information for identifying experimental data. Of these, the column “Test Set” is an identification column storing identification information used in the binding process and arithmetic process described below. The columns “NR,” “BR,” and “BR molecular weight” are columns storing experimental data values. The experimental data values ​​are examples of explanatory variables in the present disclosure. In the following description, table 200A will also be referred to as “table A,” and table 200B will also be referred to as “table B.” In the following description, the column “Test Set” will be referred to as column (a), the column “NR” as column (b1), the column “BR” as column (b2), and the column “BR molecular weight” as column (b3).

[0032] The structure of the table-format data to be processed is not limited to this example. In Fig. 4, table 200A has only one identification column called "Test Set," but the data to be processed may have multiple identification columns.

[0033] Furthermore, the copying unit 101 may perform any pre-processing such as feature design such as ratio calculation, normalization, non-linear conversion, etc., before the copying process.

[0034] Furthermore, prior to the replication process, the replication unit 101 may display on the display unit 16 a user interface that allows the user to select the target data to be replicated.

[0035] The combining unit 102 performs a combining process to cross-join the target data and the duplicated data of the target data based on the identification column. Specifically, the combining unit 102 performs a combining process to generate new data by cross-joining the target data and the duplicated data of the target data based on the identification column, using the identification information in the target data as a key. The table generated by the combining process is an example of a third table of the present disclosure. The combining unit 102 may store the table generated by the combining process in the RAM 13 or the storage 14.

[0036] 5 is a diagram showing an example of join processing by the join unit 102. The join unit 102 cross-joins the tables 200A and 200B for all combinations of rows that have the same column (a) in the tables 200A and 200B to generate a new table 200C. In the example shown, the join unit 102 joins the data in the first row of the table 200A with four rows of data in the table 200B where the column (a) is "A0001". Similarly, the join unit 102 joins the data in the second to fourth rows of the table 200A with four rows of data in the table 200B where the column (a) is "A0001". That is, in the example shown, the join unit 102 generates 16 rows of data where the column (a) is "A0001" through join processing.

[0037] When performing the combining process, the combining unit 102 may or may not perform the combining process on all of the data to be processed.

[0038] Furthermore, during the combining process, the combining unit 102 may display on the display unit 16 a user interface that allows the user to select key identification information.

[0039] The calculation unit 103 executes a calculation process for calculating an arbitrary column of the data generated by the join process by the join unit 102. Specifically, as the calculation process, the calculation unit 103 performs subtraction on the same columns in the two original tables for the data generated by the join process, and stores the subtraction result in a new column. The column in which the subtraction result is stored is also called a difference column.

[0040] 6 is a diagram showing an example of calculation processing by calculation unit 103. The "ΔNR" column is a difference column that stores values ​​obtained by subtracting the values ​​in the "NR" column derived from Table B from the values ​​in the "NR" column derived from Table A. The "ΔBR" column is a difference column that stores values ​​obtained by subtracting the values ​​in the "BR" column derived from Table B from the values ​​in the "BR" column derived from Table A. The "Δmolecular weight" column is a difference column that stores values ​​obtained by subtracting the values ​​in the "BR molecular weight" column derived from Table B from the values ​​in the "BR molecular weight" column derived from Table A.

[0041] The calculation unit 103 performs a calculation process for the same column of table 200C, subtracting the value derived from table B from the value derived from table A, and storing the results in a new column. In the example of FIG. 6, in the first row, the value derived from table A and the value derived from table B are the same, so the calculation unit 103 stores 0 in the "ΔNR", "ΔBR", and "Δmolecular weight" columns. From the second row onwards, there is a difference between the values ​​derived from table A and the values ​​derived from table B, so the calculation unit 103 stores the values ​​obtained by subtracting the value derived from table B from the value derived from table A in the "ΔNR", "ΔBR", and "Δmolecular weight" columns, respectively.

[0042] The calculation unit 103 may perform any pre-processing such as feature design such as ratio calculation, normalization, non-linear conversion, etc. before the subtraction.

[0043] The extraction unit 104 executes extraction processing to extract a feature for each difference column in which the result of the calculation processing by the calculation unit 103 is stored. Specifically, the extraction unit 104 extracts, for each difference column, information related to the identification column used by the combining unit 102 to cross-join the two tables as a feature. In other words, the extraction unit 104 extracts, for each difference column, information expressing the difference between the identification columns used by the combining unit 102 to cross-join the two tables as a feature. In other words, the extraction unit 104 extracts, for each difference column, information common to each element of the identification column used by the combining unit 102 to cross-join the two tables as a feature. The extraction unit 104 can extract features by any method, but for example, it extracts features by dimensionality reduction processing (ICA (Independent Component Analysis), PCA (Principal Component Analysis), LSI (Latent Semantic Indexing), LDA (Linear Discriminant Analysis), IA, MDS, NMF, etc.), unsupervised learning (t-SNE) that generates latent variables, supervised learning (NN), etc. The extracted features can be used for data analysis of the data to be processed.

[0044] Here, an example will be shown in which the extraction process is performed after the joining process and the calculation process are performed on the processing target data.

[0045] 7 is a diagram showing an example of extraction processing by the extraction unit 104. The "IC component 1," "IC component 2," and "IC component 3" columns in table 200C in FIG. 7 are columns in which feature amounts corresponding to the values ​​stored in the "ΔNR," "ΔBR," and "Δmolecular weight" columns are stored. In this embodiment, the extraction unit 104 extracts feature amounts by ICA from the values ​​stored in the "ΔNR," "ΔBR," and "Δmolecular weight" columns using any method as described above, and stores the extracted feature amounts in the "IC component 1," "IC component 2," and "IC component 3" columns, respectively.

[0046] Before the extraction process, the extraction unit 104 may perform any pre-processing such as normalization or nonlinear transformation on each difference sequence.

[0047] Fig. 8 is a diagram illustrating the effect of data processing by the data processing device 10. Fig. 8 shows a graph of the relationship between the IC component 1 column and the IC component 3 column shown in Fig. 7, a graph of the relationship between the IC component 2 column and the IC component 3 column, and a graph of the relationship between the IC component 1 column and the IC component 2 column.

[0048] 8, it can be seen that there is a predetermined relationship between the feature quantities extracted as a result of data processing by the data processing device 10. In other words, when the extraction process is performed by performing the combining process and the arithmetic process on the processing target data, it becomes easy to understand the characteristics of the feature quantities extracted from the processing target data.

[0049] On the other hand, an example will be shown in which extraction processing is performed directly on the processing target data without performing join processing and calculation processing.

[0050] 9 shows the state of feature amounts stored in table 200A when extraction processing is performed directly on the data to be processed without performing linkage processing or calculation processing. Even in this case, feature amounts are extracted for the values ​​stored in the "ΔNR", "ΔBR", and "Δmolecular weight" columns by ICA. FIG. 10 shows graphs of the relationship between IC component 1 column and IC component 3 column shown in FIG. 9, the relationship between IC component 2 column and IC component 3 column, and the relationship between IC component 1 column and IC component 2 column.

[0051] As shown in the graph in Figure 10, if extraction processing is performed directly on the target data without performing linkage processing and calculation processing, no relationship can be found in the extracted feature quantities. In other words, if extraction processing is performed directly on the target data without performing linkage processing and calculation processing, it is difficult to understand the characteristics of the feature quantities extracted from the target data.

[0052] The data processing device 10 according to this embodiment performs a combining process and an arithmetic process on the data to be processed, and then executes an extraction process to extract features, thereby making it possible to grasp what technical thoughts or hypotheses are expressed as parameters in the data to be processed.

[0053] Next, the operation of the data processing device 10 will be described.

[0054] 11 is a flowchart showing the flow of data processing by the data processing device 10. The CPU 11 reads out a data processing program from the ROM 12 or storage 14, loads it into the RAM 13, and executes it, thereby performing data processing.

[0055] In step S101, the CPU 11 acquires data to be processed. The data to be processed is data in a table format. The data to be processed may be, for example, past test data, and may include information such as the blending amounts of materials for manufacturing a certain product and the processing time of the materials.

[0056] Following step S101, in step S102, the CPU 11 executes a replication process for replicating the table of the data to be processed. Details of the replication process are as explained above as the process of the replication unit 101.

[0057] Following step S102, in step S103, the CPU 11 executes a join process to cross-join the target data and the data obtained by duplicating the target data. Details of the join process are as described above as the process of the join unit 102.

[0058] Following step S103, in step S104, the CPU 11 executes arithmetic processing for performing an operation on any column of the data generated in the joining processing. Details of the arithmetic processing are as described above as the processing of the calculation unit 103.

[0059] Following step S104, in step S105, CPU 11 executes extraction processing to extract feature quantities from each difference sequence in which the results of the arithmetic processing are stored. Details of the extraction processing are as described above as the processing by extraction unit 104.

[0060] The data processing device 10 according to this embodiment executes a series of processes to extract features from the data to be processed, thereby making it possible to understand what technical thoughts or hypotheses are expressed as parameters in the data to be processed.

[0061] Although the embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person skilled in the art of the present disclosure can conceive of various modifications or alterations within the scope of the technical idea described in the claims, and it is understood that these modifications or alterations also naturally fall within the technical scope of the present disclosure.

[0062] Furthermore, the effects described in the above embodiments are explanatory or exemplary and are not limited to those described in the above embodiments. In other words, the technology according to the present disclosure may achieve other effects that are obvious to a person skilled in the art of the present disclosure from the description in the above embodiments, in addition to or instead of the effects described in the above embodiments.

[0063] In the above embodiments, the data processing performed by the CPU after reading the software (program) may be performed by various processors other than the CPU. Examples of such processors include programmable logic devices (PLDs) whose circuit configuration can be changed after fabrication, such as field-programmable gate arrays (FPGAs), and dedicated electrical circuits, such as application-specific integrated circuits (ASICs), which are processors with circuit configurations specifically designed to perform specific processing. Data processing may be performed by one of these processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). The hardware structure of these processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor devices.

[0064] In addition, in each of the above embodiments, the data processing program is described as being pre-stored (installed) in a ROM or storage, but this is not limiting. The program may be provided in a form recorded on a non-transitory recording medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The program may also be downloaded from an external device via a network. [Explanation of symbols]

[0065] 10 Data processing device 101 Reproduction Department 102 Joint 103 Arithmetic section 104 Extraction part

Claims

1. a duplication unit that duplicates a first table in which data to be analyzed is stored to generate a second table; a join unit that cross-joins the first table and the second table based on an identification column for identifying the data to generate a new third table; a calculation unit that calculates a difference between a column of explanatory variables of the first table and a column of explanatory variables of the second table in the third table generated by the combining unit, and stores the difference in a newly generated column in the third table; an extraction unit that extracts a feature from the sequence generated by the calculation unit; A data processing device comprising:

2. The data processing device according to claim 1 , wherein the extraction unit extracts, as a feature, information related to the identification sequence used by the combination unit for cross-combining.

3. The data processing device according to claim 1 , wherein the extraction unit extracts the feature quantity by dimensionality reduction processing.

4. The data processing device according to claim 1 , wherein the extraction unit extracts the feature quantities using the results of unsupervised learning that generates latent variables.

5. The data processing device according to claim 1 , wherein the extraction unit extracts the feature quantity using a result of supervised learning.

6. The processor: A second table is generated by duplicating a first table in which data to be analyzed is stored; generating a new third table by cross-joining the first table and the second table based on an identification column for identifying the data; calculating a difference between the column of explanatory variables of the first table and the column of explanatory variables of the second table in the generated third table, and storing the difference in a newly generated column in the third table; Extract features from the generated sequence A data processing device that performs the processing.

7. On the computer, A second table is generated by duplicating a first table in which data to be analyzed is stored; generating a new third table by cross-joining the first table and the second table based on an identification column for identifying the data; calculating a difference between the column of explanatory variables of the first table and the column of explanatory variables of the second table in the generated third table, and storing the difference in a newly generated column in the third table; Extract features from the generated sequence A data processing program that executes processing.

Images