Continuous learning system and continuous learning method

Abstract

A continuous learning system is provided with an acquisition unit (12), a learning unit (14), a compression unit (16), and a storage unit (30), in which a prediction model (32) and past data information (34) are stored, the learning unit and the compression unit calculate the prediction model and past data information using the stored prediction model and past data information and additional data acquired by the acquisition unit (12), and store the calculated prediction model and past data information. The compression unit calculates, as information of the past data, statistics of the past data that makes it possible to obtain a learning result equivalent to a case where the past data itself acquired in the past by the acquisition unit is used when the learning unit learns in conjunction with the next-level additional data in the next-level learning.

Description

Technical Field

[0001] This disclosure relates to a continuous learning system for learning predictive models. Background Technology

[0002] For example, as described in Patent Document 1, it discloses that in a continuous learning system, in order to limit the amount of learning data and improve the learning accuracy of the prediction model, useful data is selected from the supplementary data added as learning data each time and saved.

[0003] According to this continuous learning system, since the amount of learning data stored is limited, the learning time for learning a predictive model during continuous learning can be shortened. Furthermore, the capacity of the storage device for storing the learning data can also be reduced.

[0004] Existing technical documents Patent documents Patent Document 1: Japanese Patent Application Publication No. 2019-144872 Summary of the Invention

[0005] However, in the aforementioned continuous learning system, due to the limited amount of learning data (hereinafter referred to as past data), there is a problem of forgetting the knowledge gained from past data, i.e., catastrophic forgetting, which leads to a decrease in learning accuracy.

[0006] Furthermore, continuous learning systems present the following problems: Because the amount of past data saved increases with each learning iteration, even with a limit on the amount of past data saved, the storage capacity for past data increases with each learning iteration, thus lengthening the learning time required for the predictive model. Additionally, the past data stored in storage consists of learning data selected from the appended data each time, raising the risk of leakage of personal or confidential information if the learning data contains such information.

[0007] One objective of this disclosure is to avoid the risks of catastrophic forgetting, leakage of personal information and confidential information in continuous learning systems, and to suppress learning costs such as storage capacity of learning data and learning time, thereby enabling high-precision learning and prediction models.

[0008] One aspect of the continuous learning system disclosed herein is a continuous learning system that learns a prediction model that makes predictions based on input data. The continuous learning system includes an acquisition unit, a learning unit, a compression unit, and a storage unit.

[0009] The acquisition unit acquires additional data, and the learning unit learns the prediction model based on this additional data, information from past data used in the previous learning stage, and the prediction model obtained from the previous learning stage. The compression unit calculates information from past data used by the learning unit in the next learning stage based on the additional data, information from past data, and the prediction model obtained by the learning unit from the previous learning stage.

[0010] The storage section stores the prediction model learned by the learning section and the information of past data calculated by the compression section. Furthermore, the learning section and the compression section use the additional data obtained by the acquisition section and the information of the prediction model and past data stored in the storage section to calculate the information of the prediction model and past data.

[0011] Furthermore, the compression unit calculates the statistics of past data as information about past data. These statistics enable the learning unit to obtain the same learning results as when it uses the past data itself acquired by the acquisition unit as supplementary data when learning in the next level.

[0012] Thus, in the continuous learning system disclosed herein, the information of past data stored in the storage unit is not a selection of past data, but rather a statistical measure of past data.

[0013] As is well known, a statistic is a quantity calculated based on a dataset. In this disclosure, in the compression unit, this statistic is calculated as a statistic of past data, which enables the learning unit to obtain the same learning results as when using the past data itself when learning in the next level by incorporating the additional data of the next level.

[0014] That is, the statistics of past data are calculated in a way that yields the same learning results as when the learning department studies the additional data (t+1) of the next level together with the past data (0), (1), (2), ... (t) in the next level of learning.

[0015] Therefore, in the compression unit, all past data acquired by the acquisition unit as supplementary data can be compressed for continuous learning. Thus, in the learning unit, based on the supplementary data, statistics of past data used in previous learning stages, and the predictive model learned in previous stages, the risks of catastrophic forgetting, leakage of personal information, and confidential information can be avoided, and the predictive model can be learned with high accuracy.

[0016] Furthermore, in this disclosure, personal information refers, for example, to ordinary pedestrians reflected in images when vehicle driving data is used in the learning data. Additionally, confidential information refers, for example, to information that should only be seen by persons with higher-level authority when internal company documents are used in the learning data (text). However, the form of learning data is not limited to images and text; it can also be other forms such as audio or natural language.

[0017] Furthermore, since statistics of past data are stored in the storage unit, the storage capacity of past data stored in the storage unit can be reduced compared to storing past data as is in the storage unit. Therefore, according to the continuous learning system of this disclosure, compared with the prior art described above, learning costs can be suppressed; more specifically, the storage capacity of past data stored in the storage unit and the time required for the learning unit to learn the prediction model can be suppressed. Attached Figure Description

[0018] Figure 1 This is a block diagram illustrating the overall structure of the continuous learning system in the implementation method.

[0019] Figure 2 This is a block diagram illustrating the structure of the prediction model for the implementation method.

[0020] Figure 3 This is an explanatory diagram illustrating the continuous learning operation in the continuous learning system of the implementation method.

[0021] Figure 4 This is an explanatory diagram showing the processing operation of the learning unit in the first embodiment.

[0022] Figure 5 This is an explanatory diagram that compares the amount of data in the storage unit with the amount of data in the case where the compression unit retains sufficient statistics calculated by the compression unit of the first embodiment and the amount of data in the case where all past data is retained.

[0023] Figure 6 This is an explanatory diagram showing the processing operation of the learning unit in the second embodiment.

[0024] Figure 7 This is an explanatory diagram showing the processing operation of the learning unit in the third embodiment.

[0025] Figure 8 This is a block diagram illustrating the configuration of the cloud and vehicle in other implementation methods. Detailed Implementation

[0026] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0027] [The Composition of a Continuous Learning System] The continuous learning system 1 in this embodiment is a computer system implemented using a general-purpose computer such as a personal computer and peripheral devices. For example... Figure 1 As shown, the continuous learning system 1 includes a control unit 10, an input unit 20, an output unit 22, a communication control unit 24, and a storage unit 30.

[0028] The input unit 20 uses input devices such as a keyboard and mouse to input various instruction information, such as "processing start," to the control unit 10 in response to the input operations performed by the operator. Furthermore, the input unit 20 has the function of inputting additional data (described later) with a weight ratio γt relative to past data.

[0029] Furthermore, the output unit 22 is implemented by a display device such as a liquid crystal display, a printing device such as a printer, etc. Additionally, the communication control unit 24 is implemented by a NIC (Network Interface Card), etc., and controls the communication between the control unit 10 and external devices such as servers connected via the network.

[0030] Next, the storage unit 30 is implemented using semiconductor memory elements such as RAM (Random Access Memory) and flash memory, or storage devices such as hard disks and optical disks. The storage unit 30 stores the processing program used for continuous learning and various data used in the execution of the processing program.

[0031] In addition, the storage unit 30 also stores the predictive model 32 generated during continuous learning processing and sufficient statistics 34 of past data used to generate the predictive model 32. The control unit 10 is implemented using a CPU (Central Processing Unit) or the like, and executes the processing program stored in the storage unit 30. Thus, the control unit 10 functions as the acquisition unit 12, the learning unit 14, the compression unit 16, and the model correction unit 18, performing continuous learning processing. Furthermore, these functional units can be installed separately or in part on different hardware. For example, the learning unit 14 and the compression unit 16 can be installed as separate devices from the other functional units.

[0032] Here, the acquisition unit 12 acquires the learning data input from the input unit 20 or the communication control unit 24 as supplementary data for learning, and transmits it to the learning unit 14 and the compression unit 16. The learning unit 14 uses the supplementary data acquired by the acquisition unit 12, the prediction model 32 learned in the previous stage, and the sufficient statistics 34 of the past data to learn the prediction model 32.

[0033] like Figure 2As shown, the prediction model 32 generated in the learning unit 14 is a known model that includes a feature extractor φ (or φt) and a linear predictor gt containing learned parameters. Furthermore, in the prediction model 32, the input data input from the input unit 20 or the communication control unit 24 during learning is processed by the feature extractor φ (or φt) and the linear predictor gt, and output as a prediction result.

[0034] Furthermore, in the learning of the prediction model 32 in the learning unit 14, the prediction model 32 stored in the storage unit 30 and sufficient statistics 34 of past data are used. Also, whenever the learning unit 14 learns the prediction model 32, the prediction model 32 stored in the storage unit 30 is updated to the learned prediction model 32.

[0035] Next, the compression unit 16 uses the additional data obtained by the acquisition unit 12, the prediction model 32 learned in the previous stage, and the sufficient statistics 34 of the learning data used in the previous stage, i.e., the past data, to calculate the sufficient statistics 34 of the past data, including the additional data.

[0036] Furthermore, in the calculation of sufficient statistics 34 in compression unit 16, similarly to the learning of prediction model 32 in learning unit 14, the prediction model 32 stored in storage unit 30 and sufficient statistics 34 of past data are used. In addition, whenever sufficient statistics 34 are calculated by compression unit 16, sufficient statistics 34 of past data stored in storage unit 30 are updated to the calculated sufficient statistics 34.

[0037] Furthermore, the model correction unit 18 has the function of changing the configuration of the linear predictor gt in the prediction model 32. That is, for example, when the loss function used in the learning unit 14 to learn the prediction model 32 is the squared error, and the prediction model 32 is a linear regression model with an arbitrary feature extractor, when the number of classes of the learning data increases from 1 to K (where K≥2), the linear predictor gt is corrected as follows.

[0038] [Formula 1] [Continuous learning processing] Next, as Figure 3 As shown, the continuous learning process in the control unit 10 is implemented by repeatedly learning the prediction model 32 at each learning time, such as the learning time t of the previous level, the learning time t+1 of the next level, and so on.

[0039] That is, in the continuous learning process at time t, the learning unit 14 calculates the prediction model 32(t) based on the prediction model 32(t-1) generated in the previous learning time based on past data including additional data, the sufficient statistics 34(t-1) of the past data, and the current additional data 36(t).

[0040] In addition, the compression unit 16 calculates sufficient statistics 34(t) of past data based on the statistics 34(t-1) of past data including the additional data calculated during the previous learning stage, the prediction model 32(t-1) generated during the previous learning stage, and the additional data 36(t) of this time.

[0041] Next, in the continuous learning process at time t+1, the learning unit 14 calculates the prediction model 32(t+1) based on the prediction model 32(t) generated in the previous learning time based on past data including additional data, the sufficient statistics 34(t) of the past data, and the additional data 36(t+1) of this time.

[0042] In addition, the compression unit 16 calculates sufficient statistics 34(t+1) of past data based on the statistics 34(t) of past data including supplementary data calculated during the previous learning stage, the prediction model 32(t) generated during the previous learning stage, and the supplementary data 36(t+1) of this time.

[0043] Therefore, in the continuous learning system 1 of this embodiment, the prediction model 32 and the sufficient statistics 34 of past data in the storage unit 30 are repeatedly calculated and updated according to each learning time t, t+1, ... of the continuous learning process.

[0044] [Effect] As described above, the past data used in the continuous learning process of this embodiment is not all the learning data used in past continuous learning, but a sufficient statistical measure of that learning data.

[0045] Therefore, unlike when all the learning data used in the past continuous learning is stored in the storage unit 30 as past data, the amount of past data stored in the storage unit 30 will not increase every time the prediction model 32 is learned.

[0046] Therefore, the continuous learning system 1 according to this embodiment can reduce the storage capacity of the storage unit 30 used to store past data. Furthermore, it can reduce the processing load in continuous learning processing and shorten the time required for continuous learning processing. Therefore, the continuous learning system 1 according to this embodiment can reduce these learning costs.

[0047] Furthermore, as is well known, a sufficient statistic is a statistic that provides exactly the same information as the original dataset during model learning. Therefore, in the learning unit 14, by using the sufficient statistic 34 of past data stored in the storage unit 30, the prediction model 32 can be learned with the same accuracy as when the original dataset is used as past data. Therefore, the continuous learning system 1 according to this embodiment can avoid the risks of catastrophic forgetting, leakage of personal information and confidential information, and can learn the prediction model 32 with high accuracy.

[0048] Furthermore, in the continuous learning system 1 of this embodiment, there is a model correction unit 18 that can change the configuration of the linear predictor gt in the prediction model 32, so it can not only cope with the situation of additional learning data, but also the situation of additional classes.

[0049] Next, more detailed structural examples of the learning unit 14 and compression unit 16 of this embodiment will be described in the following first to third embodiments. Furthermore, the definitions of terms and symbols used in the following descriptions of the first to third embodiments are listed in Tables 1 and 2 below.

[0050] [Table 1] [Table 2] Furthermore, the fixed feature extractor listed in Table 1, the L2 regularization coefficient at time t, the kernel function, and the dimension of the feature space listed in Table 2 are pre-specified by the user via the input unit 20 or the communication control unit 24. However, by pre-setting default values, the continuous learning system 1 can operate even if the user does not specify them. Here, as the default value for the fixed feature extractor, in the first and second embodiments described later, it can be set to, for example, a Random Fourier Feature. In addition, as the default value for the L2 regularization coefficient at time t, a grid search can be performed, for example, from 1e-6 to 1e6. In addition, as the default value for the kernel function, in the third embodiment described later, it can be set to, for example, an RBF kernel function. In addition, as the default value for the dimension of the feature space, in the first and second embodiments described later, it can be set to, for example, 10000.

[0051] [First Implementation Method] In the first embodiment, the processing operations of the learning unit 14 and the compression unit 16 when the sufficient statistics 34 of the past data are in matrix form as shown in Table 1 will be explained. Figure 4As shown, in the learning process of this embodiment executed by the learning unit 14, sufficient statistics 34: Sxx,t-1, Sxy,t-1 of past data are input into the loss function Lt. Supplementary data 36: Xt, yt are input into the loss function Lt after passing through the feature extractor φ and the linear predictor gt. Then, the parameters β of the linear predictor gt are repeatedly updated in a manner that minimizes the loss function Lt, and the learning completion parameter βt is calculated.

[0052] The specific forms of the feature extractor φ, the linear predictor gt, and the loss function Lt are determined by the prediction model 32. For example, when the loss function Lt is a squared error and the prediction model 32 is a linear regression model with an arbitrary feature extractor φ, the prediction model 32 is described as follows.

[0053] [Equation 2] In this case, the parameters that minimize the loss function Lt described below are determined in the learning section 14. Here, an example of ridge regression is shown.

[0054] [Formula 3] In addition, in the compression unit 16, the sufficient statistics of past data, supplementary data Xt, yt, and the importance ratio of supplementary data to past data γt in the first embodiment shown in Table 1 are taken as inputs, and the sufficient statistics of past + supplementary data are calculated as follows.

[0055] [Formula 4] Furthermore, the importance ratio γt of the supplementary data is a parameter input from the outside via the input unit 20. Therefore, the operator can operate the input unit 20 to specify the ratio of prioritizing past / supplementary data, for example, to perform compression that gives more emphasis to the supplementary data.

[0056] <Experimental Results> Figure 5 The results of experiments conducted to confirm the effectiveness of this embodiment are shown. For example... Figure 5 As shown, the sufficient statistics of past data calculated by the compression unit 16 as described above will not increase in data volume each time continuous learning processing is performed, as is the case when all learning data is accumulated as past data. Instead, it will be a fixed amount of data.

[0057] Furthermore, when the prediction model 32 is learned using all the learning data as past data, the "accuracy" of the learning precision increases as the amount of past data increases, but in this embodiment, it can also be confirmed that the learning precision changes in the same way.

[0058] Therefore, it can be seen that, according to this embodiment, when the amount of sufficient statistical data stored in the storage unit 30 as past data is kept constant, catastrophic forgetting can be completely avoided, just as in the case of storing all learning data as past data.

[0059] in addition, Figure 5 This represents the experimental results showing the learning accuracy of the prediction model 32 when using the MNIST dataset with 10,000 additional data points. Furthermore, in this experiment, a fixed feature extractor was used: Random Fourier features, feature dimension D = 5000, and L2 regularization coefficients were set to perform grid search within the range of 1e-5 to 1.

[0060] <Variation Example> In this variation, we will explain the case where the loss function Lt is a squared error and the prediction model 32 is a linear model with an arbitrary feature extractor φ.

[0061] Additionally, as examples of such models, SVM can be cited, with its corresponding loss function being smoothed hinge loss, and logistic regression, with its corresponding loss function being cross-entropy loss, but an example of SVM is shown here.

[0062] In this case, prediction model 32 is described as follows.

[0063] [Formula 5] Then, in the learning section 14, the parameters that minimize the loss function Lt described below are determined.

[0064] [Formula 6] Additionally, regarding the smoothed hinge loss, refer to the non-patent literature "Luo, Junru, Hong Qiao, and Bo Zhang. Learning with Smooth Hinge Losses.", 2021. In these derivations, dwell conditions and second-order Taylor approximations can be used.

[0065] Next, in the compression unit 16, the sufficient statistics of the past data shown in Table 1, the supplementary data Xt, yt, L2 regularization coefficient λt, learning completion parameter βt, and the importance ratio of the supplementary data to the past data γt are taken as inputs, and the sufficient statistics of the past + supplementary data are calculated as follows.

[0066] [Formula 7] [Second Implementation] In the second embodiment, the processing operations of the learning unit 14 and the compression unit 16 will be described when the sufficient statistics 34 of the past data are in the form of synthetic data in the feature space shown in Table 1.

[0067] like Figure 6 As shown, in the learning process of this embodiment executed by the learning unit 14, sufficient statistics 34 of past data: Φt-1, yt-1 are input into the loss function Lt after passing through the linear predictor gt. Additional data: Xt, yt are input into the loss function Lt after passing through the feature extractor φ and the linear predictor gt. Then, the parameters β of the linear predictor gt are repeatedly updated in a manner that minimizes the loss function Lt, resulting in the learning completion parameters βt.

[0068] The specific forms of the feature extractor φ, the linear predictor gt, and the loss function Lt are determined by the prediction model 32. For example, when the loss function Lt is a squared error and the prediction model 32 is a linear regression model with an arbitrary feature extractor φ, the prediction model 32 is described as Equation 2 of the first embodiment.

[0069] In this case, the parameters that minimize the loss function Lt described below are determined in the learning section 14. Here, an example of ridge regression is shown.

[0070] [Formula 8] In addition, in the compression unit 16, the sufficient statistics of past data, supplementary data Xt, yt, and the importance ratio of supplementary data to past data γt in the second embodiment shown in Table 1 are taken as inputs, and the sufficient statistics of past + supplementary data are calculated as follows.

[0071] Specifically, calculate [Equation 9] As [Equation 10] After performing singular value decomposition, specify any R×R orthogonal matrix U (usually choosing U=I (identity matrix) is sufficient) and calculate the corresponding sufficient statistics (synthetic data form in feature space) as follows.

[0072] [Equation 11] [Third Implementation Method] In the third embodiment, the processing operations of the learning unit 14 and the compression unit 16 are explained when the sufficient statistics of past data are in the form of synthetic data in the input space shown in Table 1.

[0073] like Figure 7 As shown, in the learning process of this embodiment executed by the learning unit 14, the sufficient statistics 34 of past data: Xt-1, yt-1, wt-1 are input into the loss function Lt after passing through the feature extractor φt and the linear predictor gt. Supplementary data Xt and yt are also input into the loss function Lt after passing through the feature extractor φt and the linear predictor gt. Then, the parameters α of the linear predictor gt are repeatedly updated in a manner that minimizes the loss function Lt, and the learning completion parameter αt is calculated.

[0074] The specific forms of the feature extractor φt, the linear predictor gt, and the loss function Lt are determined by the prediction model 32. For example, when the loss function Lt is the squared error and the prediction model 32 is a linear regression model with arbitrary feature extractors (kernel functions), the prediction model 32 is described as follows.

[0075] [Equation 12] In this case, the parameters that minimize the loss function Lt described below are determined in the learning section 14. Here, an example of kernel function ridge regression is shown.

[0076] [Equation 13] Furthermore, in the compression unit 16, the sufficient statistics of past data, supplementary data Xt, yt, L2 normalization coefficient λt, and importance ratio of supplementary data to past data γt in the first embodiment shown in Table 1 are taken as input, and the sufficient statistics of the past + supplementary data are calculated. Specifically, the sufficient statistics that minimize the loss function described below are obtained through repeated calculations.

[0077] [Formula 14] <Variation Example> In this variation, we illustrate the case where the loss function Lt is the squared error and the prediction model 32 is a linear model with arbitrary feature extractors (kernel functions).

[0078] Additionally, SVM can be cited as an example of such a model, with the corresponding loss function being smoothed hinge loss, and logistic regression, with the corresponding loss function being cross-entropy loss, but an example of SVM is shown here.

[0079] In this case, prediction model 32 is described as follows.

[0080] [Formula 15] Then, in the learning section 14, the parameters that minimize the loss function Lt described below are determined.

[0081] [Formula 16] Next, in the compression unit 16, the sufficient statistics of the past data shown in Table 1, the supplementary data Xt, yt, L2 regularization coefficient λt, learning completion parameter αt, and the importance ratio of the supplementary data to the past data γt are taken as input, and the sufficient statistics of the past + supplementary data are calculated. Specifically, the sufficient statistics that minimize the loss function are obtained by iterative calculation.

[0082] [Equation 17] [Other Implementation Methods] The embodiments and specific examples of this disclosure have been described above, but this disclosure is not limited to the above embodiments and can be implemented in various ways.

[0083] For example, in the above embodiment, it was explained that sufficient statistics of past data are calculated in the compression unit 16 as information of past data used in the previous stage of learning. However, it is not necessary to calculate sufficient statistics as information of past data. That is, as information of past data, any statistics of past data that enable the learning unit 14 to learn the additional data in the next stage together with the previous stage of learning will yield the same learning result as the case where the past data itself acquired by the acquisition unit is used.

[0084] Alternatively, multiple functions of a single component in the above embodiments can be achieved through multiple components, or a single function of a single component can be achieved through multiple components. Furthermore, multiple functions of multiple components can be achieved through a single component, or a single function achieved by multiple components can be achieved through a single component. Additionally, a portion of the structure in the above embodiments can be omitted. Furthermore, at least a portion of the structure in the above embodiments can be added to or replaced relative to the structures of other above embodiments.

[0085] In addition to the aforementioned continuous learning system, this disclosure can also be implemented in various ways, such as a system that incorporates a continuous learning system as a component, a program for enabling a computer to function as a continuous learning system, a non-transitional physical recording medium such as a semiconductor memory that records the program, and a continuous learning method.

[0086] Furthermore, in other embodiments, the continuous learning system 1 can also operate collaboratively with multiple vehicles. For example, such as... Figure 8As shown, Cloud 100 has a continuous learning system 1, and a data collection vehicle 200 collaborates with a prediction model-equipped vehicle 300. The data collection vehicle 200 acquires identification images from camera 201.

[0087] Additionally, a label is assigned to the acquired sign image by a human via the input unit 202. The label may indicate, for example, temporary stop. The data collection vehicle inputs the sign image and label as an additional training dataset 203 to the communication control unit 24 in the cloud. The acquisition unit 12 of the control unit 10 acquires the additional training dataset 203 via the communication control unit 24. The learning unit 14 updates the prediction model 32 based on the acquired additional training dataset 203, the prediction model 32 serving as the sign recognition model, and the sufficient statistics 34. Furthermore, the compression unit 16 updates the sufficient statistics 34 based on the acquired additional training dataset 203, the prediction model 32 serving as the sign recognition model, and the sufficient statistics 34. The model correction unit 18 corrects the prediction model 32, for example, if additional sign types are desired. With this structure, a sign recognition model is learned based on the sufficient statistics, generating a fully learned sign recognition model 303.

[0088] Subsequently, the communication control unit 24 outputs (deploys) the learned sign recognition model 303 to the prediction model-equipped vehicle 300. The prediction model-equipped vehicle 300 inputs the sign image obtained from the camera 301 to the learned sign recognition model 303, and displays the sign recognition result output from the learned sign recognition model 303 on the display device 302 mounted on the prediction model-equipped vehicle 300. Thus, the occupants of the prediction model-equipped vehicle can confirm the sign recognition result displayed on the display device 302 such as an LCD screen.

[0089] Furthermore, if the output unit 22 of the continuous learning system 1 is equipped with a display device, the calculated statistics can also be displayed by the display device.

Claims

1. A continuous learning system for learning a prediction model that makes predictions based on input data. The continuous learning system has the following features: The acquisition department is configured to acquire additional data; The learning unit is configured to: learn the prediction model based on the additional data, information from past data used in the previous learning stage, and the prediction model obtained from the previous learning stage; The compression unit is configured to: calculate information about the past data used by the learning unit in the next learning stage based on the additional data, information about the past data, and the prediction model learned by the learning unit in the previous stage; as well as The storage unit is configured to store information about the prediction model learned by the learning unit and the past data calculated by the compression unit. The learning unit and the compression unit use the appended data, the prediction model stored in the storage unit, and the information from the past data to calculate the prediction model and the information from the past data. The compression unit calculates the statistics of the past data as information about the past data. These statistics enable the learning unit to obtain the same learning results as when the additional data acquired by the acquisition unit in the next learning stage is used as the additional data.

2. The continuous learning system according to claim 1, The compression unit is configured to calculate a sufficient statistic of the past data as the statistic of the past data.

3. The continuous learning system according to claim 1 or 2, The prediction model has a fixed feature extractor and a linear predictor.

4. The continuous learning system according to claim 2, The learning unit is configured to learn the prediction model by combining sufficient statistics of the past data calculated by the compression unit with the additional data.

5. The continuous learning system according to claim 1 or 2, The system includes an input section that receives the importance ratio of the additional data relative to the previous data. The compression unit is configured to set the ratio of past data to additional data when calculating information about the past data, based on the importance ratio input via the input unit.

6. The continuous learning system according to claim 2, The compression unit is configured to use sufficient statistics of any one of the following forms: matrix form, synthetic data form in feature space, and synthetic data form in input space, as sufficient statistics of the past data.

7. The continuous learning system according to claim 6, The compression unit is configured such that, when the prediction model is a linear regression model containing a fixed feature extractor, the compression unit calculates a strict sufficient statistic in the form of the matrix or the synthetic data in the feature space as a sufficient statistic for the past data.

8. The continuous learning system according to claim 6, The compression unit is configured such that, when the prediction model is a linear model containing a fixed feature extractor, the compression unit calculates an approximate sufficient statistic in matrix form as a sufficient statistic for the past data.

9. The continuous learning system according to claim 6, The compression unit is configured such that, when the prediction model is a kernel function model, the compression unit calculates an approximate sufficient statistic in the form of synthetic data in the input space as a sufficient statistic for the past data.

10. The continuous learning system according to claim 1 or 2, It includes a model correction unit configured to change the model structure of the prediction model according to the category of the input data.

11. A continuous learning method for learning a prediction model that makes predictions based on input data. In the aforementioned continuous learning method, Obtain additional data. Based on the additional data, information from past data used in the previous learning stage, and the prediction model obtained from the previous learning stage, the prediction model is learned. Based on the additional data, the information from the past data, and the prediction model learned in the previous stage, the information from the past data used in the next stage of learning is calculated. The learned prediction model and the calculated information from the past data are stored separately. Using the additional data, the stored prediction model, and the information from the past data, calculate the information from the prediction model and the past data. The statistics of the past data are calculated as information about the past data, such that when learning in the next level of learning is combined with the additional data of the next level, the same learning results can be obtained as when the past data itself is used as the additional data.

Images