Machine learning method and machine learning device

Abstract

The machine learning method includes: a data set division process of dividing a period into a learning period and a verification period, and dividing a data set into learning data and verification data; a candidate learning period division process for dividing the candidate learning period into a plurality of divided learning periods; a learning period evaluation process for generating a learning period evaluation model for each divided learning period using learning data corresponding to the divided learning periods, and calculating an evaluation index for the learning period evaluation model for each divided learning period using verification data; a learning period selection process in which the candidate learning period division process and the learning period evaluation process are repeated until a predetermined end condition is satisfied, and when the end condition is satisfied, a divided learning period having a high evaluation index is selected as a learning target period; and a model output process for outputting a learning model that has been machine-learned using the learning data corresponding to the learning target period.

Description

Technical Field

[0001] The present invention relates to a machine learning method and a machine learning device. Background Art

[0002] In the past, a method called cross validation was used when performing machine learning on a data set. Cross validation is to divide a data set into learning data and validation data, use the learning data to perform machine learning on a learning model, and use the validation data to evaluate the performance of the machine-learned learning model (see, for example, Patent Documents 1 and 2).

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2019-003408

[0006] Patent Document 2: Japanese Patent Application Publication No. 2019-061565 Summary of the invention

[0007] Problems to be solved by the invention

[0008] When cross validation is used, if the learning data contains unique data, for example, data with characteristics significantly different from other data, the performance of the learning model may be degraded due to the influence of the unique data.

[0009] The present invention is made in view of the above-mentioned problems, and its purpose is to provide a machine learning method and a machine learning device that can improve the performance of a learning model based on machine learning.

[0010] Solutions to Solve Problems

[0011] In order to achieve the above-mentioned object, a machine learning method according to one embodiment of the present invention is a machine learning method for machine learning of a learning model using a computer and a data set composed of data at each time point acquired according to the temporal change of a prescribed phenomenon during a prescribed period arranged in a time series. In the machine learning method, the following processing is performed:

[0012] Data set segmentation processing, dividing the above period into a learning period and a verification period, setting the above learning period as an initial candidate learning period, and segmenting the above data set into learning data corresponding to the above learning period and verification data corresponding to the above verification period;

[0013] A candidate learning period division process is performed to divide the candidate learning period into a plurality of divided learning periods;

[0014] A learning period evaluation process, using the learning data corresponding to the segmented learning period to perform machine learning of the learning model for each segmented learning period, thereby generating a learning period evaluation model for each segmented learning period, and using the verification data to respectively verify the learning period evaluation model for each segmented learning period, thereby calculating an evaluation index of the learning period evaluation model for each segmented learning period;

[0015] a learning period selection process, setting the split learning period with the higher evaluation index as the next candidate learning period and repeating the candidate learning period division process and the learning period evaluation process until a prescribed end condition is satisfied, and when the end condition is satisfied, selecting the split learning period with the higher evaluation index as the learning target period; and

[0016] The model output process outputs the learning model that has been machine-learned using the learning data corresponding to the learning target period as a learned model.

[0017] In addition, a machine learning method according to another embodiment of the present invention is a machine learning method for performing machine learning of a learning model using a computer and a data set composed of data at each time point acquired according to the temporal change of a prescribed phenomenon during a prescribed period arranged in a time series. In the machine learning method, the following processing is performed:

[0018] Data set segmentation processing, dividing the above period into a learning period and a verification period, setting the above learning period as an initial candidate learning period, and segmenting the above data set into learning data corresponding to the above learning period and verification data corresponding to the above verification period;

[0019] The learning data segmentation process divides the learning period into a plurality of sub-learning periods, and segments the learning data into a plurality of sub-learning data corresponding to the plurality of sub-learning periods;

[0020] A candidate learning period division process is performed to divide the plurality of sub-learning periods included in the candidate learning period into a plurality of divided learning periods;

[0021] A learning period evaluation process, using the sub-learning data corresponding to the sub-learning periods contained in the split learning period, performing machine learning of the learning model for each of the split learning periods, thereby generating a learning period evaluation model for each of the split learning periods, and using the verification data to respectively verify the learning period evaluation model for each of the split learning periods, thereby calculating an evaluation index of the learning period evaluation model for each of the split learning periods;

[0022] a learning period selection process, setting the above-mentioned split learning period with a high evaluation index as the next above-mentioned candidate learning period and repeating the above-mentioned candidate learning period division process and the above-mentioned learning period evaluation process until a prescribed end condition is satisfied, and when the above-mentioned end condition is satisfied, selecting the above-mentioned sub-learning period contained in the above-mentioned split learning period with a high evaluation index as the learning object period; and

[0023] The model output process outputs the learning model that has been machine-learned using the sub-learning data corresponding to the learning target period as a learned model.

[0024] Effects of the Invention

[0025] According to the machine learning method of the scheme of the present invention, through the learning period selection process, the candidate learning period division process and the learning period evaluation process are repeatedly performed, so that the divided learning period with a high evaluation index of the learning period evaluation model is selected as the learning object period, and the learning data corresponding to the learning object period is used to perform machine learning of the learning model. Therefore, the performance of the learning model based on machine learning can be improved.

[0026] Other problems, structures, and effects than those described above will become apparent from the modes for carrying out the invention described later. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 1 is a structural diagram showing an example of the machine learning device 1 .

[0028] Figure 2 2 is a diagram showing a hardware configuration of an example of the computer 200 .

[0029] Figure 3 This is a flowchart showing an example of the machine learning method S1 according to the first embodiment.

[0030] Figure 4 is a flowchart showing an example of the machine learning method S1 according to the first embodiment (continued Figure 3 ).

[0031] Figure 5 This is a diagram showing an execution example of the machine learning method S1 according to the first embodiment.

[0032] Figure 6 is a diagram showing an example of executing the machine learning method S1 according to the first embodiment (continued Figure 5 ).

[0033] Figure 7 It is a diagram showing an example of calculation of the evaluation index in the machine learning method S1 according to the first embodiment.

[0034] Figure 8 This is a flowchart showing an example of the machine learning method S2 according to the second embodiment.

[0035] Fig. 9 is a flowchart showing an example of the machine learning method S2 according to the second embodiment (continued Figure 8 ).

[0036] Fig.10 This is a diagram showing an execution example of the machine learning method S2 according to the second embodiment.

[0037] Fig.11 is a diagram showing an example of executing the machine learning method S2 according to the second embodiment (continued Fig.10 ).

[0038] Fig.12 It is a diagram showing an example of calculation of an evaluation index in the machine learning method S2 according to the second embodiment. DETAILED DESCRIPTION

[0039] Hereinafter, the embodiments for implementing the present invention will be described with reference to the accompanying drawings. Hereinafter, in the description for achieving the purpose of the present invention, the necessary scope is schematically shown, and the scope required for the description of the relevant parts of the present invention is mainly described, and the description of the parts based on the known technology is omitted.

[0040] (First embodiment)

[0041] Figure 1 1 is a block diagram showing an example of a machine learning device 1 according to the first embodiment. The machine learning device 1 is a device capable of executing a machine learning method S1 for machine learning a learning model M using a data set D, and is, for example, a general-purpose or dedicated computer (see the following). Figure 2 )constitute.

[0042] The data set D is composed of data at each time point acquired during a specified period P according to the temporal change of a specified phenomenon, and is arranged in a time series. The specified phenomenon is, for example, a phenomenon in which one or more variables representing the state of the observed object change over time, such as a mechanical phenomenon, an electromagnetic phenomenon, a thermal phenomenon, a sound phenomenon, a chemical phenomenon, a physiological phenomenon, etc., and may also be a natural phenomenon or an economic phenomenon. The variables representing the state of the observed object with each phenomenon are measured by various sensors at a specified time period, for example, and the measured variables are digitized as data, collected and accumulated in the order of the time series, thereby constituting the data set D used in the machine learning method S1. In addition, the data set D may be collected in the past or in real time.

[0043] The machine learning device 1 includes a control unit 10 , a storage unit 11 , an input unit 12 , an output unit 13 , a communication unit 14 , and a device connection unit 15 as main components.

[0044] The control unit 10 functions as the machine learning unit 100 by executing the machine learning program 110 stored in the storage unit 11, and performs various processes performed by the machine learning method S1. In addition to storing the machine learning program 110 and various data used in the machine learning program 110, the storage unit 11 also stores, for example, an operating system (OS), other programs and data, etc. The input unit 12 accepts various input operations, and the output unit 13 outputs various information via a display screen or voice, thereby functioning as a user interface. The communication unit 14 is connected to a wired or wireless network and functions as a communication interface for sending and receiving various data with other devices (not shown). The device connection unit 15 functions as a connection interface with external devices such as scanners and printers, measuring devices such as sensors, and storage media such as USB memories, CD-ROMs, and DVDs.

[0045] In the machine learning method S1, machine learning of the learning model M is performed using a data set D, and the learned model is provided to any utilization system. When the data set D is, for example, a data set based on variables of a monitored object or a controlled object in a machine or device (or one of the components of the machine or device), the learned model is used to infer the state of the monitored object (abnormality, failure, their precursors, etc.) or control the controlled object by inputting unknown variables (which may be collected in the past or in real time).

[0046] The learning model M is a model that implements any machine learning algorithm and machine learning method. It can be a model for processing classification problems such as binary classification or multi-value classification, or a model for processing regression problems. The learning model M can include tree-type models such as decision trees and regression trees, integrated learning such as bagging and boosting, deep neural networks, regression neural networks, convolutional neural networks, neural network types such as LSTM (including deep learning), hierarchical clustering, non-hierarchical clustering, clustering such as k-nearest neighbor method and k-means method, principal component analysis, factor analysis, multivariate analysis such as logistic regression, support vector machines, etc., but is not limited to these examples. In addition, the learning model M can also be applied to any of teacher learning, teacher-free learning and reinforcement learning as a machine learning method.

[0047] Figure 2 2 is a diagram showing a hardware configuration of an example of a computer 200. The computer 200 is an example of a device constituting the machine learning device 1, and is configured as a general-purpose or special-purpose computer.

[0048] like Figure 2 As shown, the computer 200 includes, as its main components, a bus 210, a processor 212, a memory 214, an input device 216, an output device 217, a display device 218, a storage device 220, a communication I / F (interface) unit 222, an external device I / F unit 224, an I / O (input and output) device I / F unit 226, and a media input and output unit 228. In addition, the above-mentioned components can be omitted as appropriate depending on the purpose of using the computer 200.

[0049] The processor 212 is composed of one or more arithmetic processing devices (CPU (Central Processing Unit), MPU (Micro-processing unit), DSP (digital signal processor), GPU (Graphics Processing Unit), etc.), and works as the control unit 10 that manages the entire computer 200. The memory 214 stores various data and programs 230, and is composed of, for example, a volatile memory (DRAM, SRAM, etc.) that functions as a main memory, a non-volatile memory (ROM), a flash memory, etc.

[0050] The input device 216 is composed of, for example, a keyboard, a mouse, a numeric keypad, an electronic pen, etc., and functions as the input unit 12. The output device 217 is composed of, for example, a sound (voice) output device, a vibration device, etc., and functions as the output unit 13. The display device 218 is composed of, for example, a liquid crystal display, an organic EL display, electronic paper, a projector, etc., and functions as the output unit 13. The input device 216 and the display device 218 can also be integrally configured like a touch panel display. The storage device 220 is composed of, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), etc., and functions as the storage unit 11. The storage device 220 stores various data required for the execution of the operating system and the program 230.

[0051] The communication I / F unit 222 is connected to a network 240 such as the Internet or an intranet by wire or wireless, and functions as a communication unit 14 that performs data transmission and reception with other computers in accordance with a predetermined communication standard. The external device I / F unit 224 is connected to an external device 250 such as a camera, a printer, a scanner, a reader / writer by wire or wireless, and functions as a device connection unit 15 that performs data transmission and reception with the external device 250 in accordance with a predetermined communication standard. The I / O device I / F unit 226 is connected to an I / O device 260 such as various sensors and actuators, and functions as a device connection unit 15 that performs transmission and reception of various signals and data such as detection signals based on sensors and control signals to actuators with the I / O device 260. The media input / output unit 928 is composed of a drive device such as a DVD (Digital Versatile Disc) drive, a CD (Compact Disc) drive, a memory card slot, and a USB connector, and functions as a device connection unit 15 for reading and writing data on a medium (non-temporary storage medium) 970 such as a DVD, CD, memory card, or USB memory.

[0052] In the computer 200 having the above structure, the processor 212 calls the program 230 stored in the storage device 220 to the memory 214 for execution, and controls various parts of the computer 200 via the bus 210. In addition, the program 230 may be stored in the memory 214 instead of the storage device 220. The program 230 may also be recorded in the medium 270 in the form of an installable file or an executable file, and provided to the computer 200 via the medium input and output unit 228. The program 230 may also be provided to the computer 200 by downloading it via the network 240 with the help of the communication I / F unit 222. In addition, the computer 200 may also use hardware such as FPGA (field-programmable gate array) and ASIC (application specific integrated circuit) to realize various functions realized by the execution of the program 230 by the processor 212.

[0053] The computer 200 is an electronic device of any form, such as a fixed-type computer or a portable computer. In addition, the computer 200 may be a client-type computer, a server-type computer, or a cloud-type computer.

[0054] Next, refer to Figures 3 to 7 Details of the machine learning method S1 performed by the control unit 10 (machine learning unit 100 ) of the machine learning device 1 having the above-described configuration will be described.

[0055] Figure 3 and Figure 4 This is a flowchart showing an example of the machine learning method S1 according to the first embodiment. Figure 5 and Figure 6 This is a diagram showing an execution example of the machine learning method S1 according to the first embodiment. Figure 7 It is a diagram showing an example of calculation of the evaluation index in the machine learning method S1 according to the first embodiment.

[0056] First, in step S100 (data set acquisition processing), the machine learning device 1 acquires a data set D including data at each time point in a predetermined period P in a time series order. The data set D may be data stored in the storage unit 11, data received from another device via the communication unit 14, data recorded in a storage medium referenced via the device connection unit 15, or data measured by a measuring device via the device connection unit 15.

[0057] Next, in step S110 (data set segmentation processing), period P is divided into a learning period PL and a verification period PT, the learning period PL is set as the initial candidate learning period PL1, and the data set D is segmented into learning data DL corresponding to the learning period PL and verification data DT corresponding to the verification period PT.

[0058] The period P is divided into a learning period PL (e.g., 90%) and a verification period PT (e.g., 10%) according to a predetermined data ratio such as "9:1", for example. In this case, the verification period PT may be discretely set in the time series of the period P. For example, when the period P is 35 days, it is more preferable to disperse the verification period PT in the time series of the period P such as one day in the early stage (1st to 12th day), one day in the middle stage (13th to 24th day), and one day in the late stage (25th to 35th day), and the verification period PT may be set to the 6th day, the 18th day, and the 30th day. In this way, the deviation of the data contained in the verification data can be suppressed.

[0059] The initial candidate learning period PL1 indicates the candidate learning period PL1 obtained when the candidate learning period division process S130 and the learning period evaluation processes S140 and S141 are repeatedly executed in the learning period selection processes S150 to S152 described later. i (where i=1, 2, ..., R (R is an integer greater than or equal to 2)). "i" is a variable (index) indicating the number of repetitions, and in the data set segmentation process S110, it is initialized to the number of repetitions i=1. In addition, "R" indicates the upper limit of the number of repetitions, and in this embodiment, the repetition upper limit R=3 is used for explanation.

[0060] Next, in step S130 (candidate learning period division processing), the candidate learning period PL i Divide into multiple split learning periods PLD i，j (wherein j=1, 2, ..., S (S is an integer greater than or equal to 2)) "j" is a number used to identify a plurality of divided learning periods PLD i，j In addition, "S" indicates that the learning period PLD is divided into multiple segments. i，j In this embodiment, the number of divisions is set to S = 2 for explanation. Therefore, in this embodiment, the initial candidate learning period PL1 when the number of repetitions i is "1" is divided into two split learning periods PLD corresponding to the number of divisions S being "2". 1，1 、PLD 1，2 .

[0061] Alternatively, the learning period PLD may be divided into a plurality of i，j During each segmentation learning period PLD i，j During the study period PL i The time series is divided into multiple split learning periods PLD in such a way that each of the discrete periods is included i，j For example, when the candidate learning period PL1 is 32 days excluding the 6th, 18th, and 30th days among 35 days, the divided learning period PLD 1，1 Includes 16 days from 1st to 5th day, 19th to 29th day, split learning period PLD 1，2 The learning period PLD is divided into 16 days including the 7th to 17th day and the 31st to 35th day. i，j The deviation of the data contained in each.

[0062] Next, in steps S140 and S141 (learning period evaluation processing), the divided learning period PLD is used. i，j The corresponding segmented learning data DLD as the learning data DL i，j For each split learning period PLD i，j Perform machine learning on the learning model M to generate PLD for each split learning period i，j Evaluate the model during learning ML i，j . And, use the verification data DT to perform each segmentation learning period PLD i，j Evaluate the model during learning ML i，j Verification of the PLD during each segmentation learning period i，j Calculate the evaluation model ML during learning i，j Evaluation index EL i，j .

[0063] Evaluating ML models during learning i，j Evaluation index EL i，j For example, accuracy, precision, recall, specificity, F value, etc., a single evaluation index can be used, or multiple evaluation indexes can be used in combination. i，j Using the precision rate to illustrate, when the number of repetitions i is "1", the split learning period PLD is calculated. 1，1 Evaluate the model during machine learning 1，1 The accuracy (evaluation index EL 1，1 =36.5%), and using split learning period PLD 1，2 Evaluate the model during machine learning 1，2 The accuracy (evaluation index EL 1，2 =23.6%).

[0064] Next, in steps S150 to S152 (learning period selection process), it is determined whether to end the repetitive process of the candidate learning period division process S130 and the learning period evaluation process S140 and S141 according to a predetermined end condition (step S150). The end condition is determined, for example, by whether the number of repetitions i exceeds the repetition upper limit R (in this embodiment, R=3) when the number of repetitions i is increased, the evaluation index EL i，j It is judged whether it deviates from a predetermined limit value (such as a lower limit value) or the like.

[0065] Furthermore, when the termination condition is not satisfied (step S151: No), the evaluation index EL is set to 0 in order to continue the repetitive processing. i，j High split learning period PLD i，j Set as the next candidate learning period PL i (Step S151) and returns to step S130, thereby repeating the candidate learning period division process S130 and the learning period evaluation process S140, S141 until the end condition is satisfied. For example, in this embodiment, when the number of repetitions i is "1", the evaluation index EL is set to 1，1 High split learning period PLD 1，1 The next candidate learning period PL2 is set. In addition, when the number of divisions S is "2", the evaluation index EL i，j High one split learning period PLD i，j Set as the next candidate learning period PL i However, when the number of divisions S is "3 or more", the evaluation index EL i，j The highest PLD during a split learning period i，jSet as the next candidate learning period PL i , or by evaluation index EL i，j The PLD will be divided into multiple learning periods in order from high to low i，j Set as the next candidate learning period PL i .

[0066] On the other hand, when the termination condition is satisfied (step S151: Yes), the repetitive processing is terminated and the evaluation index EL is set to i，j High split learning period PLD i，j The learning target period PLF is selected (step S152). In addition, when the number of divisions S is "2", the evaluation index EL i，j High one split learning period PLD i，j The learning target period PLF is set, but when the number of divisions S is "3 or more", the evaluation index EL i，j The highest PLD during a split learning period i，j Set as the learning object period PLF, you can also use the evaluation indicator EL i，j The PLD will be divided into multiple learning periods in order from high to low i，j Set the learning target period PLF.

[0067] Then, in step S160 (first model output processing), the learning model M that has been machine-learned using the learning data DL corresponding to the learning object period PLF (hereinafter referred to as "learning object data DLF") is output as the first learned model MF1. The output object of the first learned model MF1 may be the storage unit 11 or another device. The first learned model MF1 includes, for example, learned coefficients and parameters for characterizing the learning model M. In addition, when the second learned model MF1 is output in the second model output processing step S190 described later, the output of the first learned model MF1 may be omitted.

[0068] Next, in step S170 (learning data division processing), the learning period PLC other than the learning target period PLF is divided into a plurality of sub-learning periods PLS k (k=2, 3, ..., T (T is an integer greater than or equal to 3)), and the learning data DLC other than the learning object data DLF corresponding to the learning object period PLF is divided into a plurality of sub-learning periods PLS k Corresponding multiple sub-learning data DLS k "T" means splitting into multiple sub-learning data DLS k In this embodiment, the number of divisions is set to T = 7 (= 2 R -1=23 -1) and is divided into 7 sub-learning data DLS k (k=2, 3, ..., 8) for explanation.

[0069] Next, in steps S180 and S181 (addition period selection process), the plurality of sub-learning data DLS are used. k Sub-learning data DLS k The first learned model MF1 is verified respectively, thereby verifying each sub-learning period PLS k Calculate the evaluation index EF of the first learning model MF1 k Then, the evaluation index EF of the first learning model MF1 is k Sub-learning period PLS that meets the specified additional conditions k The PLA is selected as the target period for addition. For example, the additional conditions are used for the evaluation index EF k The threshold of the value is calculated according to the evaluation index EF k As a result, a sub-learning period PLS can be selected as the additional target period PLA k You can also select multiple sub-learning periods PLS k .exist Figure 6 and Figure 7 In the example of , the case where two sub-learning periods PLS2 and PLS6 are selected as the additional target period PLA is illustrated.

[0070] Then, in step S190 (second model output processing), the learning model M that has been machine-learned using the learning data DL corresponding to at least one of the learning target period PLF and the additional target period PLA is output as the second learned model MF2. That is, the learning model M that has been machine-learned using the learning data DLF corresponding to the learning target period PLF and the sub-learning data DLS corresponding to the additional target period PLA is output as the second learned model MF2. k (exist Figure 6 and Figure 7 In the example, at least one of DLS2 and DLS6 performs machine learning of the learning model M and outputs it as the second learned model MF2, thereby completing a series of processing. As the output object of the second learned model MF2, it can be the storage unit 11 or other devices, similarly to the first learned model MF1. In the second learned model MF2, similarly to the first learned model MF1, for example, learned coefficients and parameters for characterizing the learning model M are included.

[0071] In addition, the second learned model MF2 can use the sub-learning data DLS corresponding to the addition target period PLA kTo additionally perform machine learning (re-learning) of the first learned model MF1, the learning object data DLF and the sub-learning data DLS corresponding to the additional object period PLA may be used. k The machine learning of the initialized learning model M is performed. In addition, as the second learned model MF2, the learning object data DLF and the sub-learning data DLS can be k The machine learning of the learning model M can be performed in any combination to output multiple second learning completion models MF2. For example, the learning model M that has been machine learned using the learning object data DLF and the sub-learning data DLS2, DLS6, the learning model M that has been machine learned using the learning object data DLF and the sub-learning data DLS2, and the learning model M that has been machine learned using the learning object data DLF and the sub-learning data DLS6 can be output.

[0072] As described above, according to the machine learning method S1 of the present embodiment, the candidate learning period division process S130 and the learning period evaluation processes S140 and S141 are repeatedly performed through the learning period selection processes S150 to S152, thereby the learning period evaluation model ML is generated. i，j Evaluation index EL i，j High split learning period PLD i，j The learning target period PLF is selected, and the learning target data DLF corresponding to the learning target period PLF is used to perform machine learning of the learning model M. Therefore, the performance of the learning model based on machine learning can be improved.

[0073] Furthermore, by adding the period selection processing steps S180 and S181, the evaluation index EF k Sub-learning period PLS that meets the specified additional conditions k The additional target period PLA is selected, and the sub-learning data DLS corresponding to the additional target period PLA is used. k Perform machine learning on the learning model M. Therefore, it is possible to use the sub-learning data DLS k While suppressing overlearning caused by using learning object data DLF for machine learning, the performance of the learning model is improved.

[0074] (Second embodiment)

[0075] The basic structure of the machine learning device 1 of this embodiment is the same as that of the first embodiment, but the details of the machine learning method S2 performed by the control unit 10 (machine learning unit 100) of the machine learning device 1 are different from the machine learning method S1 in the first embodiment. Specifically, in the first embodiment, the learning data segmentation process S170 is performed after the first model output process S160, while in the second embodiment, the learning data segmentation process S220 is performed after the data set segmentation process S210, which is different. Figures 8 to 12 Details of the machine learning method S2 performed by the control unit 10 (machine learning unit 100 ) of the machine learning device 1 according to the present embodiment will be described.

[0076] Figure 8 and Fig. 9 This is a flowchart showing an example of the machine learning method S2 according to the second embodiment. Fig.10 and Fig.11 This is a diagram showing an execution example of the machine learning method S2 according to the second embodiment. Fig.12 It is a diagram showing an example of calculation of an evaluation index in the machine learning method S2 according to the second embodiment.

[0077] First, in step S200 (data set acquisition processing), similarly to step S100 , the machine learning device 1 acquires a data set D including data at each time point in a predetermined period P in a time series order.

[0078] Next, in step S210 (data set segmentation processing), as in step S110, the period P is divided into a learning period PL and a verification period PT, the learning period PL is set to the initial candidate learning period PL1, and the data set D is segmented into learning data DL corresponding to the learning period PL and verification data DT corresponding to the verification period PT.

[0079] The initial candidate learning period PL1 shows the candidate learning period PL1 when the candidate learning period division process S230 and the learning period evaluation processes S240 and S241 are repeatedly executed in the learning period selection process S250 to S252 described later. i (i=1, 2, ..., R (R is an integer greater than 2)). "i" is a variable (index) indicating the number of repetitions, and in the data set segmentation process S210, it is initialized to the number of repetitions i=1. In addition, "R" indicates the upper limit of the number of repetitions, and in this embodiment, the repetition upper limit R=3 is used for explanation.

[0080] Next, in step S220 (learning data division processing), the learning period PL is divided into a plurality of sub-learning periods PLS. k(k=1, 2, ..., T (S is an integer greater than or equal to 2)), the learning data DL is divided into a plurality of sub-learning periods PLS k Corresponding multiple sub-learning data DLS k "T" means splitting into multiple sub-learning data DLS k In this embodiment, the number of divisions is set to T = 8 (= 2 R =2 3 ) and is divided into 8 sub-learning data DLS k (k=1, 3, ..., 8) for explanation.

[0081] Next, in step S230 (candidate learning period division processing), the candidate learning period PL i PLS with multiple sub-learning periods k Divide into multiple split learning periods PLD i，j (wherein j=1, 2, ..., S (S is an integer greater than or equal to 2)) "j" represents a number for identifying a plurality of divided learning periods PLD i，j In addition, "S" indicates that the learning period PLD is divided into multiple segments. i，j In this embodiment, the number of divisions is set to S = 2 for explanation. Therefore, in this embodiment, the initial candidate learning period PL1 when the number of repetitions i is "1" is divided into two split learning periods PLD corresponding to the number of divisions S being "2". 1，1 、PLD 1，2 , but at this time it is divided into four sub-learning periods PLS1 to PLS4, namely, the split learning period PLD 1，1 , and a divided learning period PLD including four sub-learning periods PLS5 to PLS8 1，2 .

[0082] Alternatively, the learning period PLD may be divided into a plurality of i，j During each segmentation learning period PLD i，j During the study period PL i The time series includes discrete periods so that multiple sub-learning periods PLS k The time series in PLD is randomly divided into multiple splits during the learning period i，j For example, it is also possible to divide the learning period PLD 1，1 Contains four sub-learning periods PLS1, PLS3, PLS5, PLS7, and split learning period PLD 1，2 The learning period PLS2 is divided into four sub-learning periods PLS4, PLS6, and PLS8. kThe deviation of the data contained in each.

[0083] Next, in steps S240 and S241 (learning period evaluation processing), the learning period PLD is divided using i，j PLS included in the sub-learning period k The corresponding sub-learning data DLS k For each split learning period PLD i，j Perform machine learning on the learning model M to generate PLD for each split learning period i，j Evaluate the model during learning ML i，j Then, the validation data DT is used to perform each segmentation learning period PLD separately. i，j Evaluate the model during learning ML i，j Verification of the PLD during each segmentation learning period i，j Calculate the evaluation model ML during learning i，j Evaluation index EL i，j .

[0084] Next, in steps S250 to S252 (learning period selection process), similarly to steps S150 to S152, it is determined whether to end the repetitive process of the candidate learning period division process S230 and the learning period evaluation process S240 and S241 according to a predetermined end condition (step S250). The end condition is determined, for example, by whether the number of repetitions i exceeds the repetition upper limit R (in this embodiment, R=3) when the number of repetitions i is increased, the evaluation index EL i，j It is judged whether it deviates from a predetermined limit value (such as a lower limit value) or the like.

[0085] Furthermore, when the termination condition is not satisfied (step S251: No), the evaluation index EL is set to i，j High split learning period PLD i，j Set as the next candidate learning period PL i (Step S251) The process returns to step S230, whereby the candidate learning period division process S230 and the learning period evaluation processes S240 and S241 are repeated until the end condition is satisfied.

[0086] On the other hand, when the termination condition is satisfied (step S251: Yes), the repetitive processing is terminated and the evaluation index EL is set to i，j High split learning period PLD i，j PLS included in the sub-learning period k The learning target period PLF is selected (step S252).

[0087] Then, in step S260 (first model output process), similarly to step S160, the sub-learning data DLS corresponding to the learning target period PLF is used. k The learning model M that has undergone machine learning (hereinafter referred to as “learning object data DLF”) is output as a first learned model MF1.

[0088] Next, in steps S280 and S281 (additional period selection processing), a plurality of sub-learning data DLS other than the learning target data DLF corresponding to the learning target period PLF are used. k The individual sub-learning data DLS k The first learned model MF1 is verified respectively, thereby verifying each sub-learning period PLS k Calculate the evaluation index EF of the first learning model MF1 k Then, the evaluation index EF of the first learning model MF1 is k Sub-learning period PLS that meets the specified additional conditions k Select PLA as the additional target period. Fig.11 and Fig.12 In the example of , the case where two sub-learning periods PLS2 and PLS6 are selected as the additional target period PLA is illustrated.

[0089] Then, in step S290 (second model output process), the sub-learning data DLS corresponding to at least one of the learning target period PLF and the additional target period PLA is used. k The learning model M subjected to machine learning is output as the second learned model MF2. That is, the learning object data DLF corresponding to the learning object period PLF and the sub-learning data DLS corresponding to the additional object period PLA are used. k (exist Fig.11 and Fig.12 In the example, at least one of DLS2 and DLS6 performs machine learning on the learning model M and outputs it as the second learned model MF2, thus completing a series of processing.

[0090] As described above, according to the machine learning method S2 of this embodiment, the candidate learning period division process S230 and the learning period evaluation processes S240 and S241 are repeatedly performed through the learning period selection processes S250 to S252, thereby the learning period evaluation model ML is generated. i，j Evaluation index EL i，j High split learning period PLD i，j PLS included in the sub-learning period kThe learning target period PLF is selected, and the learning target data DLF corresponding to the learning target period PLF is used to perform machine learning of the learning model M. Therefore, the performance of the learning model based on machine learning can be improved.

[0091] Furthermore, by adding the period selection processing steps S280 and S281, the evaluation index EF k Sub-learning period PLS that meets the specified additional conditions k The additional target period PLA is selected, and the sub-learning data DLS corresponding to the additional target period PLA is used. k Perform machine learning on the learning model M. Therefore, it is possible to use the sub-learning data DLS k While suppressing overlearning caused by using learning object data DLF for machine learning, the performance of the learning model is improved.

[0092] (Other embodiments)

[0093] The present invention is not limited to the above-described embodiments, and can be implemented with various modifications within the scope of the present invention. Moreover, these modifications are all included in the technical concept of the present invention.

[0094] In the above-mentioned embodiment, the machine learning methods S1 and S2 performed by the control unit 10 (machine learning unit 100) of the machine learning device 1 are described as being implemented by a single device, but the various processes included in the machine learning methods S1 and S2 can also be implemented by multiple devices by distributing them in multiple devices.

[0095] In the above-mentioned embodiment, in the flowcharts showing the details of the machine learning methods S1 and S2, the execution order of each step can be appropriately changed, and some steps can be omitted. For example, in the machine learning method S1, the processing after step S170 can be omitted, and in the machine learning method S2, the processing after step S280 can be omitted.

[0096] In the above-mentioned embodiment, the machine learning methods S1 and S2 are described as being performed in accordance with various conditions and settings such as end conditions, additional conditions, data ratio, repetition upper limit value R, number of divisions S, number of divisions T, etc., but these conditions and settings can be changed appropriately, for example, they can be configured to be able to be changed on the display screen through the input unit 12 and the output unit 13 that function as a user interface.

[0097] In the above-mentioned embodiment, the data set segmentation processing S110 and S210 in the machine learning methods S1 and S2 is a process of segmenting the data set D into learning data DL and verification data DT, and the so-called holdout validation method is adopted. However, in the machine learning methods S1 and S2, cross-validation methods such as k-split cross-validation, stratified k-split cross-validation, Monte Carlo segmentation cross-validation, time series segmentation cross-validation, sliding cross-validation, etc. can also be appropriately combined.

[0098] Industrial Applicability

[0099] The present invention can be used in a machine learning method and a machine learning device.

[0100] Description of Reference Numerals

[0101] 1…machine learning device, 10…control unit, 11…storage unit, 12…input unit, 13…output unit, 14…communication unit, 15…device connection unit, 100…machine learning unit, 110…machine learning program, 200…computer.

Claims

1. A machine learning method, wherein a computer is used to perform machine learning of a learning model using a data set composed of data at each time point acquired according to a temporal change of a prescribed phenomenon during a prescribed period arranged in a time series, wherein the following processing is performed: Data set segmentation processing, dividing the period into a learning period and a verification period, setting the learning period as an initial candidate learning period, and segmenting the data set into learning data corresponding to the learning period and verification data corresponding to the verification period; A candidate learning period division process of dividing the candidate learning period into a plurality of divided learning periods; A learning period evaluation process, using the learning data corresponding to the divided learning period to perform machine learning of the learning model for each of the divided learning periods, thereby generating a learning period evaluation model for each of the divided learning periods, and using the verification data to respectively verify the learning period evaluation model for each of the divided learning periods, thereby calculating an evaluation index of the learning period evaluation model for each of the divided learning periods; A learning period selection process, wherein the split learning period with the high evaluation index is set as the next candidate learning period and the candidate learning period division process and the learning period evaluation process are repeated until a predetermined end condition is satisfied, and when the end condition is satisfied, the split learning period with the high evaluation index is selected as the learning object period; and The model output process outputs the learning model that has been machine-learned using the learning data corresponding to the learning target period as a learned model.

2. A machine learning method, wherein a computer is used to perform machine learning of a learning model using a data set composed of data at each time point acquired according to the temporal change of a prescribed phenomenon during a prescribed period arranged in a time series, wherein the following processing is performed: Data set segmentation processing, dividing the period into a learning period and a verification period, setting the learning period as an initial candidate learning period, and segmenting the data set into learning data corresponding to the learning period and verification data corresponding to the verification period; A learning data segmentation process, dividing the learning period into a plurality of sub-learning periods, and segmenting the learning data into a plurality of sub-learning data corresponding to the plurality of sub-learning periods; A candidate learning period division process of dividing the plurality of sub-learning periods included in the candidate learning period into a plurality of divided learning periods; A learning period evaluation process, using the sub-learning data corresponding to the sub-learning periods contained in the split learning period to perform machine learning of the learning model for each of the split learning periods, thereby generating a learning period evaluation model for each of the split learning periods, and using the verification data to respectively verify the learning period evaluation model for each of the split learning periods, thereby calculating an evaluation index of the learning period evaluation model for each of the split learning periods; A learning period selection process, wherein the split learning period with the high evaluation index is set as the next candidate learning period and the candidate learning period division process and the learning period evaluation process are repeated until a prescribed end condition is satisfied, and when the end condition is satisfied, the sub-learning period contained in the split learning period with the high evaluation index is selected as the learning object period; and The model output process outputs the learning model that has been machine-learned using the sub-learning data corresponding to the learning target period as a learned model.

3. The machine learning method according to claim 1, wherein: Further processing is performed as follows: A learning data segmentation process, dividing the learning period other than the learning object period into a plurality of sub-learning periods, and segmenting the learning data other than the learning data corresponding to the learning object period into a plurality of sub-learning data corresponding to the plurality of sub-learning periods; and The additional period selection process uses each of the plurality of sub-learning data to verify the learned model, thereby calculating the evaluation index of the learned model for each sub-learning period, and selecting the sub-learning period whose evaluation index of the learned model satisfies the specified additional condition as the additional object period. The model output process outputs the learning model that has been machine-learned using the learning data corresponding to at least one of the learning target period and the additional target period as a learned model.

4. The machine learning method of claim 2, wherein: Further additional period selection processing is performed, and the learning completion model is verified respectively using each of the plurality of sub-learning data except the sub-learning data corresponding to the learning object period, thereby calculating the evaluation index of the learning completion model for each sub-learning period, and the sub-learning period whose evaluation index of the learning completion model meets the specified additional conditions is selected as the additional object period, The model output process outputs the learning model that has been machine-learned using the sub-learning data corresponding to at least one of the learning target period and the additional target period as a learned model.

5. The machine learning method of claim 1, wherein: The candidate learning period division processing divides the candidate learning period into a plurality of the divided learning periods so that each of the plurality of divided learning periods includes a discrete period in the time series of the candidate learning period.

6. The machine learning method of claim 2, wherein: The candidate learning period division processing randomly divides the time series of the plurality of sub-learning periods into the plurality of divided learning periods such that each of the plurality of divided learning periods includes a discrete period in the time series of the candidate learning periods.

7. The machine learning method according to any one of claims 1 to 6, wherein: The data set division process discretely sets the verification period in the time series of the period when dividing the period into the learning period and the verification period.

8. A machine learning device, comprising a machine learning unit, wherein the machine learning unit performs machine learning of a learning model using a data set composed of data at each time point acquired during a specified period according to a temporal change of a specified phenomenon arranged in a time series, wherein: The machine learning unit performs the following processing: Data set segmentation processing, dividing the period into a learning period and a verification period, setting the learning period as an initial candidate learning period, and segmenting the data set into learning data corresponding to the learning period and verification data corresponding to the verification period; A candidate learning period division process of dividing the candidate learning period into a plurality of divided learning periods; A learning period evaluation process, using the learning data corresponding to the divided learning period to perform machine learning of the learning model for each of the divided learning periods, thereby generating a learning period evaluation model for each of the divided learning periods, and using the verification data to respectively verify the learning period evaluation model for each of the divided learning periods, thereby calculating an evaluation index of the learning period evaluation model for each of the divided learning periods; A learning period selection process, wherein the split learning period with the high evaluation index is set as the next candidate learning period and the candidate learning period division process and the learning period evaluation process are repeated until a predetermined end condition is satisfied, and when the end condition is satisfied, the split learning period with the high evaluation index is selected as the learning object period; and The model output process outputs the learning model that has been machine-learned using the learning data corresponding to the learning target period as a learned model.

9. A machine learning device, comprising a machine learning unit, wherein the machine learning unit performs machine learning of a learning model using a data set composed of data at each time point acquired during a specified period according to a temporal change of a specified phenomenon arranged in a time series, wherein: The machine learning unit performs the following processing: Data set segmentation processing, dividing the period into a learning period and a verification period, setting the learning period as an initial candidate learning period, and segmenting the data set into learning data corresponding to the learning period and verification data corresponding to the verification period; A learning data segmentation process, dividing the learning period into a plurality of sub-learning periods, and segmenting the learning data into a plurality of sub-learning data corresponding to the plurality of sub-learning periods; A candidate learning period division process of dividing the plurality of sub-learning periods included in the candidate learning period into a plurality of divided learning periods; A learning period evaluation process, using the sub-learning data corresponding to the sub-learning periods contained in the split learning period to perform machine learning of the learning model for each of the split learning periods, thereby generating a learning period evaluation model for each of the split learning periods, and using the verification data to respectively verify the learning period evaluation model for each of the split learning periods, thereby calculating an evaluation index of the learning period evaluation model for each of the split learning periods; A learning period selection process, wherein the split learning period with the high evaluation index is set as the next candidate learning period and the candidate learning period division process and the learning period evaluation process are repeated until a prescribed end condition is satisfied, and when the end condition is satisfied, the sub-learning period contained in the split learning period with the high evaluation index is selected as the learning object period; and The model output process outputs the learning model that has been machine-learned using the sub-learning data corresponding to the learning target period as a learned model.

Images