Machine learning system and machine learning method

Abstract

To enable acquisition of new knowledge while suppressing forgetfulness of old knowledge.SOLUTION: A machine learning system comprises: a learning data division unit 11 which distributes learning data to a first characteristic group and a second characteristic group; a first learning process unit 131 which learns with a first model by using the learning data of the first characteristic group; a second learning process unit 132 which learns with a second model by using the learning data of the second characteristic group; and a learning process coupling unit 133 which performs learning by combining the learning of the first learning process unit 131 and the learning of the second learning process unit 132.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a machine learning system and a machine learning method. [Background technology]

[0002] One of the elemental technologies of AI (artificial intelligence) is machine learning, in which a computer learns large amounts of data. Conventionally, a technique for classifying (grouping) training data by performing machine learning has been known (see, for example, Patent Document 1). Patent Document 1 states that "machine learning generates a trained model that classifies data into one of multiple object classes." [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-52936 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, when AI (artificial intelligence) is made to learn new things that need to be memorized, it has the problem that while it can remember the new things (knowledge), it forgets the old knowledge that it had previously learned and acquired (so-called catastrophic forgetting).

[0005] Therefore, when acquiring new knowledge, it is necessary to prevent forgetting of old knowledge. However, the conventional technology described in Patent Document 1 does not take into consideration preventing forgetting of old knowledge at all.

[0006] The present invention has been made in consideration of the above circumstances, and aims to provide a machine learning system and a machine learning method that can acquire new knowledge while suppressing the forgetting of old knowledge. [Means for solving the problem]

[0007] The machine learning system of the present invention for solving the above problems includes a training data dividing unit that divides training data into a first characteristic group and a second characteristic group, a first learning process unit that performs training with a first model using the training data of the first characteristic group, a second learning process unit that performs training with a second model using the training data of the second characteristic group, and a learning process combining unit that performs training by combining the learning of the first learning process unit and the learning of the second learning process unit. The learning data division unit divides the learning data using intermediate data between the teacher model and the student model, where the first model is a teacher model and the second model is a student model. .

[0008] Further, the machine learning method of the present invention for solving the above problems includes: The training data division unit is The training data is divided into a first characteristic group and a second characteristic group, and then: The first learning process is training a first model using training data of a first characteristic group; The second learning process is A second model is trained using the training data of the second feature group, and then The learning process connection part Learning is performed by combining learning with the first model and learning with the second model. Furthermore, when the first model is a teacher model and the second model is a student model, the learning data division unit divides the learning data using intermediate data between the teacher model and the student model. [Effects of the Invention]

[0009] According to the present invention, it is possible to acquire new knowledge while suppressing the forgetting of old knowledge. Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a functional block diagram showing an example of the configuration of a machine learning system according to a first embodiment of the present invention; [Figure 2] 1 is a flowchart showing an example of a machine learning processing procedure in the machine learning system according to Example 1 of the present invention. [Figure 3] FIG. 1 is a diagram schematically illustrating a flow of machine learning processing in a machine learning system according to a first embodiment of the present invention. [Figure 4] FIG. 10 is a functional block diagram illustrating a configuration example of a machine learning system according to a second embodiment of the present invention. [Figure 5] 10 is a flowchart showing an example of a machine learning processing procedure in the machine learning system according to Example 2 of the present invention. [Figure 6] FIG. 10 is a diagram schematically illustrating the flow of machine learning processing in a machine learning system according to a second embodiment of the present invention. [Figure 7] FIG. 10 is a functional block diagram illustrating a configuration example of a machine learning system according to a third embodiment of the present invention. [Figure 8] 11 is a flowchart showing an example of a machine learning processing procedure in the machine learning system according to Example 3 of the present invention. [Figure 9] FIG. 10 is a diagram schematically illustrating the flow of machine learning processing in a machine learning system according to a third embodiment of the present invention. [Figure 10] FIG. 10 is a functional block diagram illustrating a configuration example of a machine learning system according to a fourth embodiment of the present invention. [Figure 11] FIG. 10 is a diagram schematically illustrating the flow of machine learning processing in a machine learning system according to a fifth embodiment of the present invention. [Figure 12] FIG. 10 is a diagram showing a table of quality determination criteria using confidence values ​​in a machine learning system according to a fifth embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, modes for carrying out the present invention (hereinafter referred to as "embodiments") will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functions or configurations are designated by the same reference numerals, and redundant explanations will be omitted.

[0012] <One embodiment of the present invention> A machine learning system and a machine learning method according to an embodiment of the present invention are suitable for use in, for example, an external environment recognition technology (system) for automatic driving of automobiles. Hereinafter, the machine learning system and the machine learning method according to the present embodiment will be described using an example in which they are applied to external environment recognition technology for automatic driving, but are not limited to application examples to external environment recognition technology for automatic driving.

[0013] The machine learning system and method according to the present embodiment are, for example, a technology used when an externally introduced model with a certain degree of recognition capability is used as a teacher model for the purpose of creating a highly accurate student model by additional learning using the introduced model. In this embodiment, the model before additional learning becomes the teacher model, and the model after additional learning becomes the student model.

[0014] Here, as an example, we will explain the case where a model capable of recognizing the environment around the vehicle during the day is introduced from outside as a model before additional learning (teacher model), and by performing additional learning, a more accurate model that can recognize the environment around the vehicle both during the day and at night is created as a model after additional learning (student model).

[0015] When daytime landscape images around the vehicle (hereinafter sometimes simply referred to as "daytime images") are used as the teacher model, in order to create more accurate landscape images as student models, it is preferable to perform additional learning using, for example, learning data of nighttime landscape images around the vehicle (hereinafter sometimes simply referred to as "night images").

[0016] If this additional learning is performed on a teacher model of daytime images using only learning data of nighttime images, the new knowledge of nighttime images will be overwritten on the old knowledge of daytime images. As a result, the old knowledge of daytime images, which is the knowledge acquired by the old model, will be forgotten and only the new knowledge of night images will be remembered, making it impossible to create a highly accurate student model.

[0017] Therefore, in the machine learning system and machine learning method according to this embodiment, additional learning is performed by dividing the learning data and learning means into those for old knowledge and those for new knowledge. This makes it possible to acquire new knowledge while suppressing the forgetting of old knowledge, thereby improving the accuracy of the model after additional learning (student model). Furthermore, when creating the model after additional learning (student model), there is no need to create a model from scratch, and knowledge acquired by the old model (old knowledge) can be effectively utilized, thereby reducing model development costs.

[0018] Below, a specific example of this embodiment will be described in which additional learning is performed by dividing the learning data and learning means into those for old knowledge and those for new knowledge in order to acquire new knowledge while suppressing the forgetting of old knowledge.

[0019] [Example 1] Example 1 is a basic configuration example of a machine learning system according to this embodiment. Fig. 1 is a functional block diagram showing a configuration example of a machine learning system according to Example 1 of the present invention.

[0020] (System configuration example) 1, a machine learning system 10 according to the first embodiment has a system configuration including, as functional units, a learning data dividing unit 11, a learning data storage unit 12, and a learning unit 13. A model capable of recognizing the daytime environment around a vehicle from the outside is introduced into this machine learning system 10 as a pre-learning model (teacher model).

[0021] The additional pre-learning model (teacher model) introduced into the machine learning system 10 is provided to the training data division unit 11 and the training unit 13 (particularly, the first training process unit 131 described later). The introduction of the additional pre-learning model into the machine learning system 10 and the provision of the introduced additional pre-learning model to the training data division unit 11 and the training unit 13 are the same in each of the embodiments described later.

[0022] The learning data dividing unit 11 divides the prepared learning data into a plurality of characteristic groups, for example, two characteristic groups, a first characteristic group and a second characteristic group (so-called grouping). Here, for example, image data of learning images including day images (images of scenery around the vehicle during the daytime) and night images (images of scenery around the vehicle during the nighttime) is prepared in advance, and the image data is provided to the learning data dividing unit 11 as learning data. The content of the learning data provided to the learning data dividing unit 11 is the same in each of the embodiments described later.

[0023] The training data dividing unit 11 divides the training data, which is image data including daytime images and nighttime images, into a first characteristic group for the daytime images and a second characteristic group for the nighttime images. This grouping can be performed, for example, in the training data dividing unit 11, based on the magnitude of the confidence value obtained as the inference result after performing inference using an externally provided pre-training model (teacher model).

[0024] The learning data storage unit 12 includes a first storage unit 121 and a second storage unit 122.

[0025] The first storage unit 121 stores the image data of daytime images grouped into the first characteristic group by the learning data division unit 11 as learning data without correct answer values. The second storage unit 122 stores the image data of nighttime images grouped into the second characteristic group by the learning data division unit 11 as learning data with correct answer values.

[0026] The learning unit 13 includes a first learning process unit 131 , a second learning process unit 132 , and a learning process combining unit 133 .

[0027] The first learning process unit 131 performs learning using a previous model (teacher model) as a first model, using the learning data of the first characteristic group, i.e., image data of daytime images, which are grouped by the learning data division unit 11 and stored in the first storage unit 121. An example of the learning by the first learning process unit 131 is distillation learning.

[0028] Here, distillation learning is learning the learning results of the teacher model, that is, learning in such a way that the learning data does not have a correct answer value and the difference between the two models, i.e., the teacher model (old model) and the image data of the daytime image, is eliminated.

[0029] The second learning process unit 132 performs learning using a new model (student model) as a second model, using the learning data of the second characteristic group, i.e., image data of night images, which are grouped by the learning data division unit 11 and stored in the second storage unit 122. The learning by the second learning process unit 132 can be exemplified by normal learning.

[0030] Here, normal learning refers to learning in which the learning data has a correct value and the parameters of the teacher model are updated to compensate for the deviation of the teacher model from the correct value. Therefore, normal learning can also be called correct value learning, which learns the correct value. However, the learning of the second learning process unit 132 is not limited to normal learning, and distillation learning can also be used.

[0031] The learning process combination unit 133 performs learning by combining the learning (distilled learning) of the first learning process unit 131 and the learning (normal learning or distilled learning) of the second learning process unit 132. Then, as the learning result of the learning process combination unit 133, an additionally learned model (student model) is created.

[0032] As described above, in the machine learning system 10 according to the first embodiment, additional learning is performed by dividing the learning data and learning means into those for old knowledge (for example, daytime images and distilled learning) and those for new knowledge (for example, nighttime images and normal learning). This makes it possible to acquire new knowledge (for example, knowledge of nighttime images) while suppressing forgetting of old knowledge (for example, knowledge of daytime images), thereby improving the accuracy of the new model (student model), which is a model after additional learning.

[0033] A machine learning system 10 according to a first embodiment, shown as a functional block diagram in Fig. 1, can be configured by a computer including a processor such as a CPU (Central Processing Unit) and storage units such as a ROM (Read Only Memory), a RAM (Random Access Memory), and an HDD (Hard Disk Drive). Each function of the machine learning system 10 is realized by the processor executing a program stored in the ROM in advance. This also applies to each of the embodiments described below.

[0034] (Example of processing procedure) Next, an example of a machine learning processing procedure (machine learning method) executed under the control of a computer processor in the machine learning system 10 according to the first embodiment will be described with reference to the flowchart of Fig. 2. Fig. 3 schematically shows the flow of the machine learning processing in the machine learning system 10 according to the first embodiment.

[0035] A processor of a computer (hereinafter simply referred to as "processor") first executes the function of the training data division unit 11. Specifically, the processor performs inference on prepared training data using an externally provided pre-training model (teacher model), and calculates a confidence value X of the training data using the pre-training model (step S11).

[0036] Next, the processor divides the training data into two characteristic groups, one with relatively high confidence value X and the other with relatively low confidence value X, based on the calculated confidence value X (step S12). Specifically, the processor divides the training data into the first characteristic group and the second characteristic group by comparing the confidence value X with a predetermined first threshold value TH1 and a second threshold value TH2 that is smaller than the first threshold value TH1 (TH1>TH2).

[0037] In this grouping, if the confidence value X is greater than the first threshold TH1 (X>TH1), the training data of daytime images is classified as distilled training data (first characteristic group). Distilled training data is an old model characteristic with no correct answer value. Also, if the confidence value X is equal to or less than the first threshold TH1 and greater than the second threshold TH2 (TH1≧X>TH2), the training data of nighttime images is classified as normal training data (second characteristic group). The normal training data is a new model characteristic with a correct answer value.

[0038] Next, the processor executes the function of the learning unit 13. Specifically, the processor performs inference using the learning data of the old model (daytime image) and the learning data of the new model (nighttime image) (step S13), and then performs distillation learning between the old model (teacher model) and the new model (student model) using the learning data of the old model characteristics (daytime image) without a correct answer value (step S14).

[0039] Next, the processor performs normal learning (correct value learning) on ​​the new model (student model) using the learning data of the new model characteristics with correct values ​​(step S15), and then determines whether the new model has achieved the desired accuracy (step S16).

[0040] If the processor determines that the new model has reached the desired accuracy (YES in S16), it ends the series of machine learning processes described above. If the processor determines that the new model has not reached the desired accuracy (NO in S16), it returns to step S13 and repeats the subsequent processes until the new model reaches the desired accuracy.

[0041] As described above, a series of machine learning processes in the machine learning system 10 according to Example 1 makes it possible to acquire new knowledge (e.g., night images) while suppressing the forgetting of old knowledge (e.g., daytime images), and also makes it possible to create a new model that achieves the desired accuracy through machine learning.

[0042] [Example 2] Example 2 is a modification of Example 1, and is an example in which information contained in training data that is unnecessary for creating a new model is made unnecessary to be trained. Fig. 4 is a functional block diagram showing an example of the configuration of a machine learning system according to Example 2 of the present invention.

[0043] (System configuration example) As shown in FIG. 4, the machine learning system 10 according to the second embodiment has a system configuration including a learning data dividing unit 11A, a learning data storage unit 12A, and a learning unit 13 as functional units.

[0044] Comparing the machine learning system 10 according to the second embodiment with the machine learning system 10 according to the first embodiment, the configurations of the training data dividing unit 11A and the training data storage unit 12A are different, but the configuration of the training unit 13 is the same.

[0045] Specifically, the learning data dividing unit 11A divides the learning data prepared in advance into a plurality of characteristic groups, for example, into three characteristic groups: a first characteristic group, a second characteristic group, and a third characteristic group.

[0046] Here, for example, when image data of training images including daytime images and nighttime images is prepared in advance as training data necessary for creating a new model, the training data may contain information unnecessary for creating the new model, which may be one of the factors that reduce the accuracy of the new model.

[0047] Therefore, the training data division unit 11A classifies the training data, including the daytime images and nighttime images necessary for creating a new model, into a first characteristic group and a second characteristic group, and further classifies information contained in the training data that is not necessary for creating a new model into a third characteristic group that does not require learning output (grouping). This grouping can be performed, for example, by performing inference in the training data division unit 11A using an externally provided pre-training model (teacher model), and based on the magnitude of the confidence value obtained as a result of the inference.

[0048] The learning data storage unit 12A includes a first storage unit 121, a second storage unit 122, and a third storage unit 123.

[0049] The first storage unit 121 stores the training data of daytime images grouped into the first characteristic group by the training data division unit 11A as training data without correct answers. The second storage unit 122 stores the training data of nighttime images grouped into the second characteristic group by the training data division unit 11A as training data with correct answers. The third storage unit 123 stores information grouped into the third characteristic group by the training data division unit 11A that is not required for creating a new model as information not required for learning.

[0050] The learning data of daytime images stored in the first storage unit 121 and the learning data of nighttime images stored in the second storage unit 122 are used for additional learning, as in the case of Example 1. On the other hand, the information stored in the third storage unit 123, i.e., information that is neither the learning data of daytime images nor the learning data of nighttime images, is processed as not requiring learning because it is not necessary for creating a new model.

[0051] In the machine learning system 10 according to the second embodiment described above, information unnecessary for creating a new model is grouped into a third characteristic group, and the third characteristic group is deemed unnecessary for creating a new model and therefore does not require learning. This allows a new model to be created by performing additional learning using learning data of daytime images and learning data of nighttime images that do not contain information unnecessary for creating a new model, thereby further improving the accuracy of the new model (student model) that is the model after additional learning.

[0052] (Example of processing procedure) Next, an example of a machine learning processing procedure executed under the control of a computer processor in the machine learning system 10 according to the second embodiment will be described with reference to the flowchart of Fig. 5. Fig. 6 schematically shows the flow of the machine learning processing in the machine learning system 10 according to the second embodiment.

[0053] The processor first executes the function of the training data division unit 11. Specifically, the processor performs inference on prepared training data using an externally provided pre-training model (teacher model), and calculates a confidence value X of the training data using the pre-training model (step S21).

[0054] Next, the processor classifies the training data into three characteristic groups based on the calculated confidence value X, namely, high, medium, and low confidence values ​​X (step S22). Specifically, the processor classifies the training data into a first characteristic group, a second characteristic group, and a third characteristic group by comparing the confidence value X with a predetermined first threshold value TH1 and a second threshold value TH2 that is smaller than the first threshold value TH1 (TH1>TH2).

[0055] In this grouping, if the confidence value X is greater than the first threshold TH1 (X>TH1), the training data of daytime images is classified as distilled training data (first characteristic group). Distilled training data is an old model characteristic with no correct answer value. Also, if the confidence value X is equal to or less than the first threshold TH1 and greater than the second threshold TH2 (TH1≧X>TH2), the training data of nighttime images is classified as normal training data (second characteristic group). The normal training data is a new model characteristic with a correct answer value.

[0056] Furthermore, if the confidence value X is equal to or less than the second threshold value TH2 (TH2≧X), the information is classified as information that does not belong to either the training data of daytime images or the training data of nighttime images (third characteristic group). Information that does not belong to either the training data of daytime images or the training data of nighttime images is information that is not required for creating a new model, and therefore, information classified as the third characteristic group is processed as information that does not require training by the training unit 13.

[0057] Next, the processor executes the function of the learning unit 13. Specifically, the processor performs inference using the learning data of the old model (daytime images) and the learning data of the new model (nighttime images) (step S23), and then performs distillation learning between the old model (teacher model) and the new model (student model) using the learning data of the old model characteristics that does not have a correct answer value (step S24).

[0058] Next, the processor performs normal learning (correct value learning) on ​​the new model (student model) using the learning data of the new model characteristics with correct values ​​(step S25), and then determines whether the new model has achieved the desired accuracy (step S26).

[0059] If the processor determines that the new model has reached the desired accuracy (YES in S26), it ends the series of machine learning processes described above. If the processor determines that the new model has not reached the desired accuracy (NO in S26), it returns to step S23 and repeats the subsequent processes until the new model reaches the desired accuracy.

[0060] As described above, by a series of machine learning processes in the machine learning system 10 according to Example 2, information unnecessary for creating a new model is grouped into a third characteristic group, and learning is not required for this third characteristic group, thereby enabling further improvement in the accuracy of the new model (student model), which is the model after additional learning.

[0061] [Example 3] Example 3 is a modification of Example 1, and is an example in which forgetting of an old model (teacher model) is detected and control is performed to suppress the forgetting. Fig. 7 is a functional block diagram showing an example of the configuration of a machine learning system according to Example 3 of the present invention.

[0062] (System configuration example) As shown in FIG. 7, the machine learning system 10 according to the third embodiment has a system configuration including a learning data dividing unit 11, a learning data storage unit 12, and a learning unit 13A as functional units.

[0063] Comparing the machine learning system 10 according to Example 3 with the machine learning system 10 according to Example 1, the configurations of the learning data dividing unit 11 and the learning data storage unit 12 are the same, but the configuration of the learning unit 13A is different. Specifically, the learning unit 13A has a learning process ratio control unit 134 in addition to a first learning process unit 131, a second learning process unit 132, and a learning process combining unit 133.

[0064] The learning process proportion control unit 134 has a function of automatically maintaining the balance between the old model (in this example, a daytime image) and the new model (in this example, a nighttime image), and controls the learning rate (weight update rate) of distilled learning based on the inference result of the old model (teacher model) in the first learning process unit 131 and the inference result of the old model (teacher model) in the second learning process unit 132. Specifically, the learning process proportion control unit 134 detects the degree of forgetting about the old model from the difference between the inference result of the old model in the first learning process unit 131 and the inference result of the old model in the second learning process unit 132 for the old model (daytime image), and performs control to increase the learning rate of distilled learning when the difference between both inference results is higher than a predetermined threshold.

[0065] As described above, in the machine learning system 10 according to the third embodiment, control is performed to add a new model (night image) while suppressing forgetting of the old model (day image) under the control of the learning process proportion control unit 134. This makes it possible to maintain a balance between the old model (day image) and the new model (night image), thereby further improving the accuracy of the new model (student model), which is a model after additional learning.

[0066] (Example of processing procedure) Next, an example of a machine learning processing procedure executed under the control of a computer processor in the machine learning system 10 according to the third embodiment will be described with reference to the flowchart of Fig. 8. Fig. 9 schematically shows the flow of the machine learning processing in the machine learning system 10 according to the third embodiment.

[0067] The processor first executes the function of the training data division unit 11. Specifically, the processor performs inference on prepared training data using an externally provided pre-training model (teacher model), and calculates a confidence value X of the training data using the pre-training model (step S31).

[0068] Next, the processor divides the training data into two characteristic groups, one with relatively high confidence value X and the other with relatively low confidence value X, based on the calculated confidence value X (step S32). Specifically, the processor divides the training data into the first characteristic group and the second characteristic group by comparing the confidence value X with a predetermined first threshold value TH1 and a second threshold value TH2 that is smaller than the first threshold value TH1 (TH1>TH2).

[0069] In this grouping, if the confidence value X is greater than the first threshold TH1 (X>TH1), the training data of daytime images is classified as distilled training data (first characteristic group). Distilled training data is an old model characteristic with no correct answer value. Also, if the confidence value X is equal to or less than the first threshold TH1 and greater than the second threshold TH2 (TH1≧X>TH2), the training data of nighttime images is classified as normal training data (second characteristic group). The normal training data is a new model characteristic with a correct answer value.

[0070] Next, the processor executes the function of the learning unit 13. Specifically, the processor performs inference on the old model (daytime image) using the learning data of the old model and the learning data of the new model (step S33).

[0071] Next, the processor detects the difference between the inference results of the old model and the new model for the old model (daytime image) as information indicating the degree of forgetting of the old model (daytime image), and determines the learning rate of distillation learning by comparing the difference between the two inference results with a predetermined threshold (step S34).

[0072] It is preferable that the difference between the inference results of the old model and the inference results of the new model is 0, i.e., that both inference results match. However, in this example, the processor compares the difference between both inference results with a predetermined threshold value, and if the difference between both inference results is higher than the threshold value, it controls to increase the learning rate (weight update rate) of distillation learning.

[0073] Next, the processor performs distillation learning between the old model (teacher model) and the new model (student model) using the learning data of the old model characteristics (daytime image) without a correct answer value and the learning rate determined in step S34 (step S35).

[0074] Next, the processor performs normal learning (correct value learning) on ​​the new model (student model) using the learning data of the new model characteristics with correct values ​​(step S36), and then determines whether the new model has achieved the desired accuracy (step S37).

[0075] If the processor determines that the new model has reached the desired accuracy (YES in S37), it ends the series of machine learning processes described above. If the processor determines that the new model has not reached the desired accuracy (NO in S37), it returns to step S33 and repeats the subsequent processes until the new model reaches the desired accuracy.

[0076] As described above, a series of machine learning processes in the machine learning system 10 according to Example 3 can maintain a balance between the old model (daytime image) and the new model (nighttime image), thereby further improving the accuracy of the new model (student model), which is the model after additional learning.

[0077] [Example 4] The fourth embodiment is an example of application to an automated driving (AD) system for automobiles and an advanced driver assistance system (ADAS). Fig. 10 is a functional block diagram showing an example of the configuration of a machine learning system according to the fourth embodiment of the present invention.

[0078] The machine learning system according to the fourth embodiment is applied to an autonomous driving system or an advanced driving assistance system for an automobile 100, and has a system configuration in which a learning data division unit 11 and a learning data storage unit 12 are mounted on the automobile 100, and a learning unit 13 is arranged on the cloud 200 side, which is capable of exchanging data with the automobile 100 via a network 300 such as the Internet. The learning data division unit 11 can be configured using, for example, an autonomous driving ECU (Electronic Control Unit).

[0079] In addition to a learning data dividing unit 11 and a learning data storage unit 12, the automobile 100 is also provided with a sensor 101 such as a camera that acquires information about the surroundings of the automobile 100, a communication unit 102, a model recognition unit 103, and a judgment / vehicle body control unit 104. In addition to a learning unit 13, the cloud 200 is also provided with a communication unit 201.

[0080] On the automobile 100 side, a sensor 101 such as a camera captures an image of the periphery of the automobile 100 and supplies the image data obtained to a training data dividing unit 11 as training data, and also supplies the image data to a model recognition unit 103 .

[0081] The functions of the learning data dividing unit 11 and the learning data storage unit 12 are the same as those in the first embodiment. That is, the learning data dividing unit 11 takes in image data supplied from the sensor 101 as learning data, and divides the learning data into two characteristic groups: a first characteristic group (e.g., learning data of daytime images) and a second characteristic group (e.g., learning data of nighttime images).

[0082] In the learning data storage unit 12, the first storage unit 121 stores image data of daytime images grouped into the first characteristic group by the learning data division unit 11 as learning data without correct answer values. The second storage unit 122 stores image data of nighttime images grouped into the second characteristic group by the learning data division unit 11 as learning data with correct answer values.

[0083] The communication unit 102 transmits the learning data of the first characteristic group stored in the first storage unit 121 and the learning data of the second characteristic group stored in the second storage unit 122 to the communication unit 201 of the cloud 200 via a network 300 such as the Internet.

[0084] On the cloud 200 side, the communication unit 201 receives the learning data of the first characteristic group and the learning data of the second characteristic group transmitted from the communication unit 102 on the automobile 100 side, and supplies them to the learning unit 13.

[0085] The learning unit 13 performs distilled learning on an old model (teacher model) using the learning data of the first characteristic group, and performs normal learning (or distilled learning) on ​​a new model (student model) using the learning data of the second characteristic group.The learning unit 13 then performs learning by combining the learning (distilled learning) of the first learning process unit 131 and the learning (normal learning or distilled learning) of the second learning process unit 132, and creates an additionally learned model (student model) as a result of this learning.

[0086] The communication unit 201 transmits the additionally trained model (student model) created by the learning unit 13 to the communication unit 102 on the automobile 100 side via a network 300 such as the Internet.

[0087] On the automobile 100 side, the communication unit 102 receives the additionally trained model (student model) transmitted from the communication unit 201 on the cloud 200 side, and supplies it to the model recognition unit 103. The model recognition unit 103 performs inference on the additionally trained model (student model) supplied from the communication unit 102 using image data supplied from the sensor 101.

[0088] The inference result of the model recognition unit 103 is supplied to the judgment and vehicle body control unit 104. The judgment and vehicle body control unit 104 performs, for example, a process of determining whether or not there is an obstacle ahead of the automobile 100 based on the inference result of the model recognition unit 103, and performs vehicle body control, i.e., automatic driving control, based on the determination result.

[0089] As described above, in the machine learning system according to the fourth embodiment, for example, the machine learning system according to the first embodiment, which can improve the accuracy of a new model (student model), is applied to an automatic driving system for automobiles and an advanced driving assistance system. This makes it possible to provide a new model with higher accuracy to the automatic driving system for automobiles and the advanced driving assistance system.

[0090] In the machine learning system according to Example 4, the case where the machine learning system according to Example 1 is applied to an automatic driving system for automobiles or an advanced driver assistance system has been described as an example, but it is also possible to apply the machine learning system according to Example 2 or Example 3.

[0091] [Example 5] Example 5 is a modification of Example 4, and is an example having a plurality of teacher models. Fig. 11 is a diagram schematically illustrating the flow of machine learning processing in a machine learning system according to Example 5 of the present invention.

[0092] As in the case of the machine learning system according to the fourth embodiment, a learning data dividing unit 11 that divides learning data according to model characteristics is mounted on the automobile 100. The learning data dividing unit 11 is configured using an autonomous driving ECU. The learning data divided by the learning data dividing unit 11 is stored (held) in a learning data storage unit 12 that is mounted on the automobile 100 like the learning data dividing unit 11.

[0093] In distillation learning in the learning unit 13 arranged on the cloud 200 side, multiple teacher models, for example, old model A (teacher model), old model B (teacher model), and new model (teacher model), assigned by the learning data division unit 11 on the automobile 100 side are used. What is meant here is that the number of teacher models assigned by the learning data division unit 11 is not limited to one, but may be multiple. The new model (teacher model) is created in advance.

[0094] In the machine learning system according to the fifth embodiment, distillation learning is performed using a plurality of teacher models to improve accuracy, and a quality check is performed on a new model (student model) that is a model after additional learning. For the quality check of this new model (student model), for example, a confidence value obtained as a result of inference using an old model (teacher model) in the learning data division unit 11 can be used.

[0095] Figure 12 shows a table of quality assessment criteria using confidence values. Here, the training data is the old model (e.g., daytime images) and the new model (e.g., nighttime images). Typically, the confidence value of the old model (daytime images) is relatively high compared to the teacher model, and the confidence value of the new model (nighttime images) is relatively low. Here, for convenience, the confidence value for the teacher model is denoted as XH for the old model and XL for the new model.

[0096] Ideally, the confidence value of the student model (new model) should match the confidence value of the teacher model (old model), but in reality they do not. Therefore, when checking the quality, a predetermined setting value a is set for the old model (daytime image) and a predetermined setting value b is set for the new model (nighttime image).

[0097] Then, in the case of the old model (daytime image), if the confidence value X for the student model (new model) is greater than (confidence value XH for the teacher model - set value a) (X>XH-set value a), the quality of the new model (student model) is determined to be good. Also, in the case of the new model (night image), if the confidence value X for the student model (new model) is greater than (confidence value XL for the teacher model + set value b) (X>XL+set value b), the quality of the new model (student model) is determined to be good.

[0098] As described above, in the machine learning system according to the fifth embodiment, when applied to an automated driving system for automobiles or an advanced driver assistance system, distillation learning is performed using a plurality of teacher models to improve accuracy, and the quality of the new model (student model) is checked. This makes it possible to provide a new model with higher accuracy and quality for the automated driving system for automobiles or the advanced driver assistance system.

[0099] In addition, in the machine learning system of Example 5, which is intended to be installed in automobile 100, distillation learning is performed using multiple teacher models to improve accuracy, but the use of multiple teacher models to improve accuracy is not limited to machine learning systems intended to be installed in automobile 100.

[0100] <Modification> The present invention is not limited to the examples of the above-described embodiments, and various other applications and modifications are possible as long as they do not deviate from the gist of the present invention as set forth in the claims. For example, the above-described embodiments have described the system configuration in detail and specifically to clearly explain the present invention, and the present invention is not necessarily limited to systems that include all of the described configurations. Furthermore, it is also possible to add, delete, or replace part of the configuration of the present embodiments with other configurations.

[0101] In the above-described embodiment, the grouping of the training data in the training data division unit 11 is performed using the confidence value obtained as the inference result in the training data division unit 11 as the parameter for grouping the training data. However, the parameter for grouping is not limited to the confidence value. For example, it is also possible to group the training data using intermediate data between the first model, the teacher model, and the second model, the student model, as a parameter. In this case, the same effects and advantages as when using the confidence value can be obtained. Furthermore, it is also possible to use the difference in the final result (large difference: new model, small difference: old model), the difference in feature amount (old model characteristics and training data), or the class type of the training data as a parameter. In this case, the same effects and advantages as when using the confidence value can be obtained.

[0102] In the above-described embodiment, the first and second characteristic groups divided by the learning data dividing unit 11 are exemplified by a combination of daytime landscape images (daytime images) and nighttime landscape images (nighttime images). However, the combination is not limited to this. For example, the first and second characteristic groups may be a combination of highway landscape pixels and urban landscape pixels. [Explanation of symbols]

[0103] 10...machine learning system, 11,11A...learning data division unit, 12,12A...learning data storage unit, 13,13A...learning unit, 100...automobile, 121...first storage unit, 122...second storage unit, 123...third storage unit, 131...first learning process unit, 132...second learning process unit, 133...learning process combination unit, 134...learning process ratio control unit, 200...cloud, 300...network

Claims

1. a training data dividing unit that divides training data into a first characteristic group and a second characteristic group; a first learning process unit that uses the learning data of the first characteristic group to learn a first model; a second learning process unit that uses the learning data of the second characteristic group to learn a second model; a learning process combination unit that performs learning by combining the learning of the first learning process unit and the learning of the second learning process unit, The machine learning system is characterized in that, when the first model is a teacher model and the second model is a student model, the training data division unit divides the training data using intermediate data between the teacher model and the student model.

2. the learning used in the first learning process unit is distillation learning, The learning used in the second learning process is normal learning (correct answer learning) or distillation learning. The machine learning system of claim 1 .

3. The learning data division unit sorting the training data using a confidence value obtained as a result of the inference using the first model; If the confidence value is greater than a predetermined first threshold, the device is classified into the first characteristic group; If the confidence value is equal to or less than the first threshold value and greater than a second threshold value that is smaller than the first threshold value, the device is assigned to the second characteristic group. The machine learning system of claim 1 .

4. The learning data division unit If the confidence value is equal to or less than the second threshold value, the device is classified as a third characteristic group that does not require learning. The machine learning system according to claim 3 .

5. a learning process rate control unit that controls a learning rate of the learning process coupling unit; The learning process rate control unit controls the learning rate of the learning process combination unit based on the inference result of the teacher model in the first learning process unit and the inference result of the teacher model in the second learning process unit when the first model is used as a teacher model. The machine learning system of claim 1 .

6. the learning data division unit is mounted on a vehicle, The first learning process unit, the second learning process unit, and the learning process coupling unit are provided on a cloud side that can exchange data with the automobile. The machine learning system of claim 1 .

7. The first learning process unit performs learning with a plurality of the first models using the learning data sorted into the first characteristic groups by the learning data division unit. The machine learning system of claim 6 .

8. A training data division unit divides the training data into a first characteristic group and a second characteristic group; Then, a first learning process unit uses the training data of the first characteristic group to train a first model, and a second learning process unit uses the training data of the second characteristic group to train a second model; Thereafter, a learning process combination unit performs learning by combining the learning using the first model and the learning using the second model; A machine learning method characterized in that, when a first model is a teacher model and a second model is a student model, the training data division unit divides the training data using intermediate data between the teacher model and the student model.

Images