Data processing device, data processing method, and data processing program

Abstract

Provided is a data processing device comprising: a duplication unit that duplicates a first table in which data to be analyzed is stored to generate a second table; a linking unit that generates a new third table by cross-linking the first table and the second table on the basis of an identification string for identifying the data; a computation unit that, in the third table generated by the linking unit, computes the difference between a column of explanatory variables in the first table and a column of explanatory variables in the second table, and stores the difference in a column newly generated in the third table; and an extraction unit that extracts a feature amount with respect to the column generated by the computation unit.

Description

Data processing device, data processing method and data processing program

[0001] The present disclosure relates to a data processing device, a data processing method, and a data processing program.

[0002] Materials are designed by repeatedly making prototypes while adjusting the composition based on the experience of the material developer. For example, International Publication No. 2021 / 045058 discloses a material design device, a material design method, and a material design program for polymer materials that simultaneously satisfy multiple desired material properties.

[0003] The experimental data from past material prototypes reflects the thinking of material developers. However, there was no technology available to analyze what thinking the material developers had based the past experimental data on when prototyping materials, or in other words, what characteristics exist in the data to be processed, such as experimental data.

[0004] The present disclosure has been made in consideration of the above points, and aims to provide a data processing device, a data processing method, and a data processing program that are capable of extracting data characteristics from data to be processed.

[0005] A data processing device according to a first aspect of the present disclosure includes a duplication unit that duplicates a first table storing data to be analyzed to generate a second table; a join unit that cross-joins the first table and the second table based on an identification column for identifying the data to generate a new third table; a calculation unit that calculates the difference between a column of explanatory variables in the first table and a column of explanatory variables in the second table in the third table generated by the join unit and stores the difference in a newly generated column in the third table; and an extraction unit that extracts features from the column generated by the calculation unit.

[0006] A data processing device according to a second aspect of the present disclosure is the data processing device according to the first aspect, wherein the extraction unit extracts, as a feature, information related to the identification sequence used by the combination unit for cross-combining.

[0007] A data processing device according to a third aspect of the present disclosure is the data processing device according to the first aspect, wherein the extraction unit extracts the feature amount by dimensionality reduction processing.

[0008] A data processing device according to a fourth aspect of the present disclosure is the data processing device according to the first aspect, wherein the extraction unit extracts features using a result of unsupervised learning that generates latent variables.

[0009] A data processing device according to a fifth aspect of the present disclosure is the data processing device according to the first aspect, wherein the extraction unit extracts the feature amount using a result of supervised learning.

[0010] A data processing method according to a sixth aspect of the present disclosure includes a processor that copies a first table in which data to be analyzed is stored to generate a second table, cross-links the first table and the second table based on an identification column for identifying the data to generate a new third table, calculates the difference between a column of explanatory variables in the first table and a column of explanatory variables in the second table in the generated third table, stores the difference in a newly generated column in the third table, and executes a process of extracting features from the generated column.

[0011] A data processing program according to a seventh aspect of the present disclosure causes a computer to generate a second table by duplicating a first table in which data to be analyzed is stored, generate a new third table by cross-linking the first table and the second table based on an identification column for identifying the data, calculate a difference between a column of explanatory variables in the first table and a column of explanatory variables in the second table in the generated third table, store the difference in a newly generated column in the third table, and execute a process of extracting features from the generated column.

[0012] According to the present disclosure, it is possible to provide a data processing device, a data processing method, and a data processing program that are capable of extracting data characteristics from data to be processed.

[0013] 10 is a diagram illustrating an overview of a data processing device according to an embodiment of the disclosed technology. FIG. 11 is a block diagram illustrating a hardware configuration of the data processing device. FIG. 12 is a block diagram illustrating an example of a functional configuration of the data processing device. FIG. 13 is a diagram illustrating an example of a replication process. FIG. 14 is a diagram illustrating an example of a join process. FIG. 15 is a diagram illustrating an example of a calculation process. FIG. 16 is a diagram illustrating an example of an extraction process. FIG. 17 is a diagram illustrating the effect of data processing by the data processing device. FIG. 18 is a diagram illustrating a state in which feature amounts are stored in a table when extraction processing is performed directly on data to be processed without performing join processing and calculation processing. FIG. 19 is a graph illustrating the relationship between the feature amounts shown in FIG. 9. FIG. 11 is a flowchart illustrating the flow of data processing by the data processing device.

[0014] An example of an embodiment of the present disclosure will be described below with reference to the drawings. The same reference numerals are used throughout the drawings to designate identical or equivalent components and parts. The dimensional proportions of the drawings are exaggerated for illustrative purposes and may differ from the actual proportions.

[0015] 1 is a diagram showing an overview of a data processing device according to this embodiment. The data processing device 10 is a device that executes data processing on processing target data and outputs the processing results. The data processing device 10 is, for example, a personal computer that stores a computer program for executing data processing.

[0016] In this embodiment, the data processing device 10 accepts data in a table format as data to be processed. The data in a table format may be of any type, such as data stored in a database or entered in a spreadsheet. The data to be processed may include, for example, past test data for the prototyping of a product, such as the blending amounts of materials used to manufacture a certain product, material characteristics (such as molecular weight, softening point, and particle shape) obtained from experimental results such as analysis or simulation results, material processing time, and test results for the prototype product.

[0017] The data processing device 10 then executes processing to extract features of the processing target data as data processing for the processing target data, and outputs the extracted features as processed data.

[0018] The data processing device 10 performs data processing on the target data to extract features of the target data, and outputs the extracted features, thereby making it possible to present what features the target data possesses. If the target data is past test data, it is possible to present what features the past test data possesses. Note that, as will be described later, the data processing device 10 does not simply perform feature extraction processing on the target data, but performs preprocessing on the target data before executing feature extraction processing.

[0019] Next, a description will be given of the hardware configuration of the data processing device 10. Fig. 2 is a block diagram showing the hardware configuration of the data processing device 10.

[0020] 2, the data processing device 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage 14, an input unit 15, a display unit 16, and a communication interface (I / F) 17. Each component is connected to each other via a bus 19 so as to be able to communicate with each other.

[0021] The CPU 11 is a central processing unit that executes various programs and controls each component. That is, the CPU 11 reads a program from the ROM 12 or the storage 14 and executes the program using the RAM 13 as a work area. The CPU 11 controls the above components and performs various arithmetic processing in accordance with the program recorded in the ROM 12 or the storage 14. In this embodiment, the ROM 12 or the storage 14 stores a data processing program that executes data processing on data to be processed.

[0022] The ROM 12 stores various programs and various data. The RAM 13 temporarily stores programs or data as a working area. The storage 14 is configured with a storage device such as a hard disk drive (HDD), a solid state drive (SSD), or a flash memory, and stores various programs including an operating system and various data.

[0023] The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used to input various types of information.

[0024] The display unit 16 is, for example, a liquid crystal display, and displays various information. The display unit 16 may be a touch panel type and function as the input unit 15.

[0025] The communication interface 17 is an interface for communicating with other devices, and uses standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark).

[0026] When executing the above data processing program, the data processing device 10 uses the above hardware resources to realize various functions. The functional configuration realized by the data processing device 10 will be described below.

[0027] FIG. 3 is a block diagram showing an example of the functional configuration of the data processing device 10.

[0028] 3, the data processing device 10 has, as its functional components, a duplication unit 101, a combination unit 102, a calculation unit 103, and an extraction unit 104. Each functional component is realized by the CPU 11 reading and executing a data processing program stored in the ROM 12 or the storage 14.

[0029] The duplication unit 101 executes a duplication process to duplicate data to be processed. That is, the duplication unit 101 duplicates data to be processed in a table format. The data to be processed in a table format is an example of a first table of the present disclosure, and the table generated by the duplication process is an example of a second table of the present disclosure. The duplication unit 101 may store the table generated by the duplication process in the RAM 13 or the storage 14.

[0030] FIG. 4 is a diagram illustrating an example of a duplication process performed by the duplication unit 101. The duplication unit 101 duplicates table 200A to generate table 200B having exactly the same data. The columns "Test Set" and "#" are columns in which information identifying experimental data is stored. Of these, column "Test Set" is an identification column in which identification information used in the binding process and calculation process described below is stored. Columns "NR," "BR," and "BR molecular weight" are columns in which experimental data values ​​are stored. The experimental data values ​​are examples of explanatory variables in the present disclosure. In the following description, table 200A will also be referred to as "table A," and table 200B will also be referred to as "table B." In the following description, column "Test Set" will be referred to as column (a), column "NR" as column (b1), column "BR" as column (b2), and column "BR molecular weight" as column (b3).

[0031] The structure of the table-format data to be processed is not limited to this example. In Fig. 4, table 200A has only one identification column called "Test Set," but the data to be processed may have multiple identification columns.

[0032] Furthermore, the copying unit 101 may perform any pre-processing such as feature design such as ratio calculation, normalization, non-linear conversion, etc., before the copying process.

[0033] Furthermore, prior to the copying process, the copying unit 101 may display on the display unit 16 a user interface that allows the user to select processing target data that is the target of the copying process.

[0034] The combiner 102 performs a combine process to cross-join the target data and the duplicated data of the target data using the identification column as a reference. Specifically, the combiner 102 performs a combine process to generate new data by cross-joining the target data and the duplicated data of the target data using the identification column as a reference, using the identification information in the target data as a key. The table generated by the combine process is an example of a third table of the present disclosure. The combiner 102 may store the table generated by the combine process in the RAM 13 or the storage 14.

[0035] FIG. 5 is a diagram showing an example of join processing by the join unit 102. The join unit 102 cross-joins tables 200A and 200B for all combinations of rows with the same column (a) in tables 200A and 200B to generate a new table 200C. In the example shown, the join unit 102 joins four rows of data in which column (a) of table 200B is "A0001" with the data in the first row of table 200A. Similarly, the join unit 102 joins four rows of data in which column (a) of table 200B is "A0001" with the data in the second to fourth rows of table 200A. That is, in the example shown, the join unit 102 generates 16 rows of data in which column (a) is "A0001" through join processing.

[0036] When performing the combining process, the combining unit 102 may perform the combining process on all data of the processing target data, or may not perform the combining process on all data of the processing target data.

[0037] Furthermore, during the combining process, the combining unit 102 may display on the display unit 16 a user interface that allows the user to select key identification information.

[0038] The calculation unit 103 executes a calculation process for calculating an arbitrary column of the data generated by the join process by the join unit 102. Specifically, as the calculation process, the calculation unit 103 performs subtraction on the same columns in the two original tables for the data generated by the join process, and stores the subtraction result in a new column. The column in which the subtraction result is stored is also referred to as a difference column.

[0039] 6 is a diagram showing an example of calculation processing by the calculation unit 103. The "ΔNR" column is a difference column that stores values ​​obtained by subtracting the values ​​in the "NR" column from Table B from the values ​​in the "NR" column from Table A. The "ΔBR" column is a difference column that stores values ​​obtained by subtracting the values ​​in the "BR" column from Table B from the values ​​in the "BR" column from Table A. The "Δmolecular weight" column is a difference column that stores values ​​obtained by subtracting the values ​​in the "BR molecular weight" column from Table B from the values ​​in the "BR molecular weight" column from Table A.

[0040] The calculation unit 103 performs a calculation process for the same column of table 200C, subtracting the value derived from table B from the value derived from table A, and storing the results in a new column. In the example of Fig. 6, in the first row, the value derived from table A and the value derived from table B are the same, so the calculation unit 103 stores 0 in the "ΔNR", "ΔBR", and "Δmolecular weight" columns. From the second row onwards, there is a difference between the values ​​derived from table A and the values ​​derived from table B, so the calculation unit 103 stores values ​​obtained by subtracting the value derived from table B from the value derived from table A in the "ΔNR", "ΔBR", and "Δmolecular weight" columns, respectively.

[0041] The calculation unit 103 may perform any pre-processing such as feature design such as ratio calculation, normalization, non-linear conversion, etc. before the subtraction.

[0042] The extraction unit 104 executes extraction processing to extract features for each difference column in which the results of the calculation processing by the calculation unit 103 are stored. Specifically, the extraction unit 104 extracts, as features, information related to the identification column used by the combining unit 102 to cross-join the two tables for each difference column. In other words, the extraction unit 104 extracts, as features, information that expresses the difference between the identification columns used by the combining unit 102 to cross-join the two tables for each difference column. In other words, the extraction unit 104 extracts, as features, information that is common to each element of the identification column used by the combining unit 102 to cross-join the two tables for each difference column. The extraction unit 104 can extract features by any method, but as an example, it extracts features by dimensionality reduction processing (ICA (Independent Component Analysis), PCA (Principal Component Analysis), LSI (Latent Semantic Indexing), LDA (Linear Discriminant Analysis), IA, MDS, NMF, etc.), unsupervised learning (t-SNE) that generates latent variables, supervised learning (NN), etc. The extracted features can be used for data analysis of the data to be processed.

[0043] Here, an example will be shown in which the extraction process is performed after the joining process and the calculation process are performed on the processing target data.

[0044] 7 is a diagram showing an example of extraction processing by the extraction unit 104. The "IC component 1," "IC component 2," and "IC component 3" columns in table 200C in FIG. 7 are columns in which feature quantities corresponding to the values ​​stored in the "ΔNR," "ΔBR," and "Δmolecular weight" columns are stored. In this embodiment, the extraction unit 104 extracts feature quantities by ICA from the values ​​stored in the "ΔNR," "ΔBR," and "Δmolecular weight" columns using any method as described above, and stores the extracted feature quantities in the "IC component 1," "IC component 2," and "IC component 3" columns, respectively.

[0045] Before the extraction process, the extraction unit 104 may perform any pre-processing such as normalization or nonlinear transformation on each difference sequence.

[0046] 8 is a diagram illustrating the effect of data processing by the data processing device 10. Fig. 8 shows a graph of the relationship between the IC component 1 column and the IC component 3 column shown in Fig. 7, a graph of the relationship between the IC component 2 column and the IC component 3 column, and a graph of the relationship between the IC component 1 column and the IC component 2 column.

[0047] 8, it can be seen that there is a predetermined relationship between the feature quantities extracted as a result of data processing by the data processing device 10. In other words, when the extraction process is performed by performing the combining process and the arithmetic process on the processing target data, it becomes easy to understand the characteristics of the feature quantities extracted from the processing target data.

[0048] On the other hand, an example will be shown in which extraction processing is performed directly on the processing target data without performing join processing and calculation processing.

[0049] 9 shows the state of feature amounts stored in table 200A when extraction processing is performed directly on the data to be processed without performing join processing or calculation processing. Even in this case, feature amounts are extracted for the values ​​stored in the "ΔNR," "ΔBR," and "Δmolecular weight" columns by ICA. FIG. 10 shows graphs of the relationship between column IC component 1 and column IC component 3 shown in FIG. 9, the relationship between column IC component 2 and column IC component 3, and the relationship between column IC component 1 and column IC component 2.

[0050] 10, when extraction processing is performed directly on the target data without performing linking processing and calculation processing, it is clear that no relationship can be found in the extracted feature quantities. In other words, when extraction processing is performed directly on the target data without performing linking processing and calculation processing, it is difficult to grasp the characteristics of the feature quantities extracted from the target data.

[0051] The data processing device 10 according to this embodiment performs combining and arithmetic processing on the data to be processed, and then executes extraction processing to extract features, thereby making it possible to grasp what technical thoughts or hypotheses are expressed as parameters in the data to be processed.

[0052] Next, the operation of the data processing device 10 will be described.

[0053] 11 is a flowchart showing the flow of data processing by the data processing device 10. The CPU 11 reads a data processing program from the ROM 12 or the storage 14, loads it into the RAM 13, and executes it, thereby performing data processing.

[0054] In step S101, the CPU 11 acquires data to be processed. The data to be processed is data in a table format. The data to be processed may be, for example, past test data, and may include information such as the blending amounts of materials for manufacturing a certain product and the processing time of the materials.

[0055] Following step S101, the CPU 11 executes a replication process for replicating the table of the data to be processed in step S102. Details of the replication process are as described above as the process of the replication unit 101.

[0056] Following step S102, in step S103, the CPU 11 executes a join process to cross-join the target data and the replicated data of the target data. Details of the join process are as described above as the process of the join unit 102.

[0057] Following step S103, in step S104, the CPU 11 executes a calculation process for calculating an arbitrary column of the data generated by the joining process. Details of the calculation process are as described above as the processing of the calculation unit 103.

[0058] Following step S104, in step S105, the CPU 11 executes an extraction process to extract feature quantities from each difference sequence in which the results of the calculation process are stored. Details of the extraction process are as described above as the process performed by the extraction unit 104.

[0059] The data processing device 10 according to this embodiment executes a series of processes to extract features from the data to be processed, thereby making it possible to understand what technical thoughts or hypotheses are expressed as parameters in the data to be processed.

[0060] Although the embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person skilled in the art of the present disclosure can conceive of various modifications or alterations within the scope of the technical idea described in the claims, and it is understood that these modifications or alterations also naturally fall within the technical scope of the present disclosure.

[0061] Furthermore, the effects described in the above embodiments are explanatory or exemplary and are not limited to those described in the above embodiments. In other words, the technology according to the present disclosure may achieve other effects that are obvious to a person skilled in the art of the present disclosure from the description in the above embodiments, in addition to or instead of the effects described in the above embodiments.

[0062] In each of the above embodiments, the data processing performed by the CPU after reading the software (program) may be performed by various processors other than the CPU. Examples of such processors include a PLD (Programmable Logic Device) whose circuit configuration can be changed after manufacture, such as an FPGA (Field-Programmable Gate Array), and a dedicated electrical circuit, such as an ASIC (Application Specific Integrated Circuit), which is a processor having a circuit configuration designed specifically to perform specific processing. Furthermore, data processing may be performed by one of these various processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). Furthermore, the hardware structure of these various processors is, more specifically, an electrical circuit combining circuit elements such as semiconductor elements.

[0063] In addition, although the above embodiments have described an embodiment in which the data processing program is pre-stored (installed) in a ROM or storage, the present invention is not limited to this. The program may be provided in a form recorded on a non-transitory recording medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The program may also be downloaded from an external device via a network.

[0064] The disclosure of Japanese Patent Application No. 2024-72837, filed on April 26, 2024, is incorporated herein by reference in its entirety. In addition, all documents, patent applications, and technical standards described herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually indicated to be incorporated by reference.

Claims

1. A data processing device comprising: a duplication unit that duplicates a first table storing data to be analyzed to generate a second table; a join unit that cross-joins the first table and the second table based on an identification column for identifying the data to generate a new third table; a calculation unit that calculates the difference between a column of explanatory variables in the first table and a column of explanatory variables in the second table in the third table generated by the join unit and stores the difference in a newly generated column in the third table; and an extraction unit that extracts features from the column generated by the calculation unit.

2. The data processing device according to claim 1, wherein said extraction unit extracts, as a feature, information related to said identification sequence used by said combination unit for cross-combination.

3. The data processing device according to claim 1, wherein the extraction unit extracts features by dimensionality reduction processing.

4. The data processing device according to claim 1, wherein the extraction unit extracts features using the results of unsupervised learning that generates latent variables.

5. The data processing device according to claim 1, wherein the extraction unit extracts features using the results of supervised learning.

6. A data processing device comprising a processor that performs the following processes: duplicating a first table in which data to be analyzed is stored to generate a second table; cross-linking the first table and the second table based on an identification column for identifying the data to generate a new third table; calculating the difference between the column of explanatory variables of the first table and the column of explanatory variables of the second table in the generated third table, storing the difference in a newly generated column in the third table; and extracting features from the generated column.

7. A data processing program that causes a computer to execute the following processes: generate a second table by duplicating a first table in which data to be analyzed is stored; generate a new third table by cross-linking the first table and the second table based on an identification column for identifying the data; calculate the difference between the column of explanatory variables of the first table and the column of explanatory variables of the second table in the generated third table, store the difference in a newly generated column in the third table; and extract features from the generated column.

Images