Analytical device

Abstract

To provide an analysis device capable of efficiently excluding an outlier and accurately performing multiple regression analysis.SOLUTION: An analysis device 1 comprises: a calculation unit 11 which calculates an abnormality degree for each of a plurality of datasets composed of a plurality of explanatory variables and a plurality of objective variables; a setting unit 12 which sets a threshold value on abnormality degrees for identifying an exclusive dataset not subject to multiple regression analysis from the plurality of datasets; an analysis unit 13 which performs multiple regression analysis using the dataset left after the exclusive dataset identified on the basis of the threshold value among the plurality of datasets is excluded, and calculates a determination coefficient; a determination unit 14 which determines the minimization of the number of exclusive datasets and the maximization of the determination coefficient; and a regression equation acquisition unit which acquires a regression equation reflected with the results of the determination unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] This disclosure relates to an analytical instrument. [Background technology]

[0002] A method of multiple regression analysis is known in which the relationship between the dependent variable and the independent variables is expressed by a relational equation, and the dependent variable is estimated using this relational equation. For example, Patent Document 1 discloses a multiple regression analysis device that divides multiple datasets into strata based on the independent variables and the thresholds for those independent variables, and performs multiple regression analysis on the divided datasets. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-064707 [Overview of the project] [Problems that the invention aims to solve]

[0004] When outliers are included in the multiple datasets being analyzed for multiple regression, the estimation accuracy of the multiple regression analysis decreases. To perform multiple regression analysis accurately, it is necessary to appropriately remove outliers from the multiple datasets. However, if users manually remove outliers, the efficiency of the work involved in multiple regression analysis decreases. The technology disclosed in Patent Document 1 does not address this problem.

[0005] This disclosure enables efficient exclusion of outliers and accurate multiple regression analysis. [Means for solving the problem]

[0006] The analytical device according to this disclosure comprises: a calculation unit that calculates the degree of anomaly for each of a plurality of datasets composed of a plurality of explanatory variables and a dependent variable; a setting unit that sets a threshold for the degree of anomaly to identify excluded datasets from the plurality of datasets that are to be excluded from the multiple regression analysis; an analysis unit that performs the multiple regression analysis using the datasets remaining after the excluded datasets identified based on the threshold have been excluded from the plurality of datasets, and calculates the coefficient of determination; a determination unit that determines the minimization of the number of excluded datasets and the maximization of the coefficient of determination; and a regression equation acquisition unit that obtains a multiple regression equation that reflects the results of the determination unit. [Effects of the Invention]

[0007] The analytical device described herein enables efficient exclusion of outliers and accurate multiple regression analysis. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing the configuration of an analytical apparatus according to an embodiment. [Figure 2] This figure shows a data group, which is an example of multiple datasets. [Figure 3] This is a flowchart showing the processing performed by the analytical device according to the embodiment. [Figure 4] This figure shows a specific example of the processing performed by the analytical apparatus according to the embodiment. [Modes for carrying out the invention]

[0009] Embodiments of this disclosure will be described in detail below with reference to the drawings. In each drawing, the same or corresponding elements are denoted by the same reference numerals. For clarity of explanation, redundant explanations will be omitted where necessary.

[0010] (Configuration of analyzer 1) Referring to Figure 1, the analysis device 1 according to this embodiment will be described. Figure 1 is a block diagram showing the configuration of the analysis device 1. The analysis device 1 includes a data acquisition unit 10, a calculation unit 11, a setting unit 12, an analysis unit 13, a determination unit 14, a regression equation acquisition unit 15, and a display control unit 16.

[0011] The analysis device 1 includes a processor, memory, and storage device (not shown in the diagram). The storage device stores a program on which the processing according to this embodiment is implemented. The processor loads the program into memory and executes the program to realize each of the functions, such as the data acquisition unit 10 described above. For example, a CPU (Central Processing Unit) may be used as the processor. Some or all of each functional component may be implemented with dedicated hardware. Also, some or all of each functional component may be implemented with multiple information processing devices or circuits. These multiple information processing devices or circuits may be centrally located or distributed.

[0012] The data acquisition unit 10 acquires multiple datasets, each consisting of multiple explanatory variables and a target variable. The data acquisition unit 10 may acquire these datasets from a storage device (not shown) or from other devices via a network (not shown).

[0013] Each of the multiple datasets (hereinafter referred to as "each dataset") has multiple explanatory variables and a target variable. For example, each dataset has M explanatory variables X1 to XM and one target variable Y, where "M" is an integer greater than or equal to 2. The explanatory variables X1 to XM can be represented by multidimensional vectors. The data acquisition unit 10 acquires N such datasets, where "N" is an integer greater than or equal to 2. The N datasets are associated with identification information to identify each dataset. The identification information is, for example, an ID assigned to each dataset.

[0014] FIG. 2 is a diagram showing a data group G1 which is an example of a plurality of data sets acquired by the data acquisition unit 10. Here, an example of M = 4 and N = 30 is shown, but M and N are not limited thereto. The data group G1 has data sets with ID = D01 to D30, in which an ID, explanatory variables X1 to X4, and an objective variable Y are associated. Note that the degree of abnormality shown in the rightmost column of the figure is calculated by the calculation unit 11 described later.

[0015] Returning to FIG. 1, the description will be continued. The calculation unit 11 calculates the degree of abnormality for each of the plurality of data sets acquired by the data acquisition unit 10. As described above, each data set has multi-dimensional explanatory variables X1 to X4. The calculation unit 11 calculates the degree of abnormality in multiple dimensions of the explanatory variables of each data set. The calculation unit 11 can calculate the degree of abnormality using a method such as Isolation Forest which is a kind of unsupervised learning, for example. An example of the calculated degree of abnormality is shown in FIG. 2 as described above.

[0016] The setting unit 12 sets a threshold value related to the degree of abnormality for specifying an exclusion data set that is excluded from the object of multiple regression analysis from a plurality of data sets. The exclusion data set corresponds to the outliers described above. The set threshold value is used to exclude the exclusion data set from the plurality of data sets in the analysis unit 13. The setting unit 12 can set the threshold value using a previously provided value. The setting unit 12 may set the threshold value using a value input from the user via an input unit (not shown). Further, the setting unit 12 may set an upper limit threshold value and a lower limit threshold value. For example, the setting unit 12 may set the upper limit threshold value and the lower limit threshold value based on at least one of the maximum value and the minimum value of the degree of abnormality. For example, the setting unit 12 may fix the upper limit threshold value to the maximum value of the degree of abnormality and set the lower limit threshold value to an arbitrary value, etc.

[0017] Further, when the determination unit 14 determines that the threshold value does not satisfy a predetermined condition, the setting unit 12 newly sets the threshold value. Thereby, the setting unit 12 newly sets a threshold value different from the previously set threshold value.

[0018] The analysis unit 13 performs multiple regression analysis using some or all of the plurality of data sets acquired by the data acquisition unit 10. First, the analysis unit 13 identifies an exclusion data set that is excluded from the target of multiple regression analysis from the plurality of data sets based on the threshold value set by the setting unit 12. Next, the analysis unit 13 excludes the exclusion data set from the plurality of data sets. Then, the analysis unit 13 performs multiple regression analysis using the data sets remaining after the exclusion data set is excluded from the plurality of data sets, and calculates the coefficient of determination.

[0019] A specific explanation will be given using the example of the data group G1 described above. For example, assume that the analysis unit 13 identifies three data sets as exclusion data sets based on the threshold value. The analysis unit 13 excludes the three exclusion data sets from the data group G1, performs multiple regression analysis using the remaining 27 data sets, and calculates the coefficient of determination. Also, when a new threshold value is set in the setting unit 12, the analysis unit 13 performs multiple regression analysis using the newly set threshold value. Thereby, the analysis unit 13 performs multiple regression analysis on different data sets to be analyzed, and calculates the coefficient of determination corresponding to each data set.

[0020] The determination unit 14 determines whether a predetermined condition is satisfied in the multiple regression analysis based on the threshold value set by the setting unit 12. The predetermined condition is that the number of exclusion data sets is minimized and the coefficient of determination is maximized. The determination unit 14 makes the above determination using the setting of the threshold value in the setting unit 12 and the result of the multiple regression analysis in the analysis unit 13 using the threshold value. Thereby, the determination unit 14 determines whether the minimization of the number of exclusion data sets and the maximization of the coefficient of determination are achieved. Also, the determination unit 14 can determine whether the threshold value satisfies the above predetermined condition. The determination unit 14 identifies a threshold value that satisfies the predetermined condition, and outputs information on the threshold value to the regression equation acquisition unit 15.

[0021] If the determination unit 14 determines that the threshold does not satisfy the predetermined conditions, it causes the setting unit 12 and the analysis unit 13 to repeat the process. In this way, the determination unit 14 identifies a threshold that satisfies the predetermined conditions through trial and error. In this manner, the determination unit 14 minimizes outliers and maximizes the estimation accuracy of the multiple regression analysis.

[0022] The regression equation acquisition unit 15 acquires a multiple regression equation that reflects the results of the determination unit 14. For example, the regression equation acquisition unit 15 acquires information on thresholds that satisfy predetermined conditions. As a result, the regression equation acquisition unit 15 acquires a multiple regression equation obtained by a multiple regression analysis using the thresholds. The regression equation acquisition unit 15 may output the acquired multiple regression equation to a display unit or the like via the display control unit 16. As a result, the analysis device 1 can present the user with a multiple regression equation with improved estimation accuracy.

[0023] The display control unit 16 displays information related to the processing of the analysis device 1 on a display unit (not shown). The display information may include, but is not limited to, the number of excluded datasets, the coefficient of determination, or the multiple regression equation obtained by the regression equation acquisition unit 15. The display control unit 16 may also display the information on an information terminal used by the user.

[0024] (Processing by analyzer 1) Next, with reference to Figure 3, the processes performed by the analyzer 1 will be explained. Figure 3 is a flowchart of the processes performed by the analyzer 1. Here, the overall flow of the process will be explained, and specific examples of the process will be described later. First, the data acquisition unit 10 acquires multiple datasets consisting of multiple explanatory variables and a target variable (S1). Next, the calculation unit 11 calculates the anomaly score for each dataset (S2). Subsequently, the setting unit 12 sets thresholds for the anomaly score to identify datasets to be excluded (S3).

[0025] Next, the analysis unit 13 identifies the datasets to be excluded from the multiple datasets based on the set threshold (S4). The analysis unit 13 excludes the excluded datasets from the multiple datasets, performs multiple regression analysis using the datasets remaining after the excluded datasets have been excluded, and calculates the coefficient of determination (S5).

[0026] Next, the determination unit 14 determines whether the number of excluded datasets has been minimized and the coefficient of determination has been maximized (S6). If it is determined that the number of excluded datasets has been minimized and the coefficient of determination has been maximized (YES in S6), the regression equation acquisition unit 15 acquires a multiple regression equation that reflects the result of the determination unit 14 (S7). If it is determined that the number of excluded datasets has not been minimized and the coefficient of determination has not been maximized (NO in S6), the analyzer 1 returns to the process in step S3 and repeats the subsequent processes. The display control unit 16 may display information related to the above-described processes on a display unit (not shown) as appropriate.

[0027] Referring to Figure 4, the processing performed by the analysis device 1 described above will be explained in detail. Figure 4 is a diagram showing a specific example of the processing performed by the analysis device 1. Here, it is assumed that the data acquisition unit 10 has acquired data group G1 shown in Figure 2. In Figure 4, data groups G2 and G3 represent the data sets that remain after exclusion data sets identified based on thresholds have been removed from data group G1.

[0028] The left side of Figure 4 shows the datasets for each of the data groups G1 to G3. Unlike Figure 2, the datasets are shown in order from smallest to largest anomaly. The explanatory variables X1 to X4 are not shown. The right side of Figure 4 shows information regarding the multiple regression analysis using data groups G1 to G3. The number of datasets N is the number of datasets acquired by the data acquisition unit 10. The number of datasets N is 30 and is common to all data groups G1 to G3.

[0029] Threshold Th is an example of a threshold set by the setting unit 12. Here, the setting unit 12 sets an upper threshold Th1 and a lower threshold Th2 as threshold Th. Number of excluded datasets E is the number of datasets excluded from data group G1. Number of excluded datasets E corresponds to the number of excluded datasets identified by the analysis unit 13 based on the thresholds set by the setting unit 12. Coefficient of determination R 2 This is the coefficient of determination calculated by the analysis unit 13. The analysis device 1 minimizes the number of excluded datasets E and calculates the coefficient of determination R 2 Calculate the threshold Th that maximizes [the value].

[0030] First, the setting unit 12 sets the initial values ​​of the upper threshold Th1 and the lower threshold Th2. As shown in the area of ​​data group G1 on the right side of the figure, the setting unit 12 sets the upper threshold Th1 and the lower threshold Th2 to 0.11 and -0.11, respectively. Data group G1 does not contain any datasets where the anomaly score is greater than the upper threshold Th1, nor any datasets where the anomaly score is less than the lower threshold Th2. Therefore, the number of excluded datasets E is 0. The analysis unit 13 performs multiple regression analysis using all 30 datasets of data group G1. Here, the analysis unit 13 calculates the coefficient of determination R 2 Let's assume that 0.633 was calculated as such.

[0031] Next, as shown in the area of ​​data group G2 on the right side of the figure, the setting unit 12 sets the upper threshold Th1 and lower threshold Th2 to 0.11 and -0.05, respectively. Based on these thresholds, the analysis unit 13 identifies the data sets to be excluded from data group G1. The analysis unit 13 identifies the data sets with IDs D07, D06, and D15 as the data sets to be excluded. Therefore, the number of data sets to be excluded E is 3. The analysis unit 13 performs multiple regression analysis using the data group G2 that remains after excluding these three data sets.

[0032] Here, the analysis unit 13 calculates the coefficient of determination R 2 We assume that 0.566 was calculated. Compared to the case of data group G1, the coefficient of determination R2 The value has become smaller. Therefore, the determination unit 14 determines that the set threshold is not a threshold that satisfies the predetermined conditions. The determination unit 14 instructs the setting unit 12 and the analysis unit 13 to perform the process again.

[0033] Then, as shown in the area of ​​data group G3 on the right side of the figure, the setting unit 12 sets the upper threshold Th1 and lower threshold Th2 to 0.11 and 0.01, respectively. Based on these thresholds, the analysis unit 13 identifies the datasets with IDs D07, D06, D15, ..., D28, and D12 as excluded datasets. Therefore, the number of excluded datasets E is 10. The analysis unit 13 performs multiple regression analysis using the remaining data group G3 after excluding these 10 datasets.

[0034] Here, the analysis unit 13 calculates the coefficient of determination R 2 We will assume that 0.694 was calculated. Compared to the case of data groups G1 and G2, the coefficient of determination R 2 This is the largest value. Therefore, the determination unit 14 determines that the set threshold minimizes the number of excluded datasets E, and the coefficient of determination R 2 The determination unit 14 determines that it maximizes the threshold. The determination unit 14 outputs the threshold information to the regression equation acquisition unit 15. The regression equation acquisition unit 15 obtains the threshold information from the determination unit 14 and obtains a multiple regression equation that reflects the result of the determination unit 14.

[0035] As described above, the analysis device 1 according to this embodiment determines the minimization of the number of excluded datasets and the maximization of the coefficient of determination, and obtains a multiple regression equation that reflects the results of the determination. In this way, the analysis device 1 can assist in the elimination of outliers in multiple regression analysis. By using the analysis device 1, users can efficiently improve the estimation accuracy of multiple regression analysis. Furthermore, the analysis device 1 can prevent the variability in accuracy that occurs when users manually eliminate outliers.

[0036] When the above program is loaded into a computer, it includes a set of instructions (or software code) for causing the computer to perform the functions described in the embodiments. The program may be stored in various types of non-transitory computer-readable media or tangible storage media. By way of example and not limitation, non-transitory computer-readable media or tangible storage media include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD), or other memory technologies, CD-ROM, digital versatile disc (DVD), Blu-ray (registered trademark) disc, or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage, or other magnetic storage devices. Also, the program may be transmitted on various types of transitory computer-readable media or communication media. By way of example and not limitation, transitory computer-readable media or communication media include electrical, optical, acoustic, or other forms of propagated signals.

[0037] Note that the present disclosure is not limited to the above embodiments and can be appropriately changed without departing from the spirit.

Explanation of Reference Numerals

[0038] 1: Analysis device, 10: Data acquisition unit, 11: Calculation unit, 12: Setting unit, 13: Analysis unit, 14: Determination unit, 15: Regression equation acquisition unit, 16: Display control unit, E: Number of excluded data sets, G1 - G3: Data groups, N: Number of data sets, R 2 : Coefficient of determination, Th: Threshold value, Th1: Upper limit threshold value, Th2: Lower limit threshold value, X1 - X4 and XM: Explanatory variables, Y: Objective variable

Claims

1. A calculation unit that calculates the anomaly score for each of several datasets, each of which consists of a dataset containing a set of multiple explanatory variables and a dependent variable. A setting unit for setting a threshold for the degree of anomaly in order to identify excluded datasets, which are datasets that are excluded from the multiple datasets to be used in multiple regression analysis, An analysis unit performs the multiple regression analysis using the datasets remaining after the exclusion datasets identified based on the threshold have been removed from the aforementioned plurality of datasets, and calculates the coefficient of determination. A determination unit that determines the minimization of the number of excluded datasets and the maximization of the coefficient of determination, The system includes a regression equation acquisition unit that acquires a multiple regression equation that reflects the results of the determination unit. Analyzer.

2. If the determination unit determines that the number of excluded datasets has not been minimized and the coefficient of determination has not been maximized, The setting unit sets a new threshold different from the threshold, The analysis unit performs the multiple regression analysis based on the newly set threshold. The analytical apparatus according to claim 1.

Images