Model learning device, model learning method, and program

Abstract

To provide a model learning apparatus configured to modify a model of AI in accordance with a request from a user, a model learning method and a program.SOLUTION: In a system including a user terminal, home appliances, and a server, a model learning apparatus 10 includes: a retrieval unit 11 which retrieves a factor of prediction using a model trained with a training dataset; a confirmation unit 12 which presents the factor of prediction and confirms whether it is necessary or not to retrain the model; a configuration unit 13 which reconfigures the training dataset on the basis of the factor of prediction; and a learning unit 14 which retrains the model with the reconfigured training dataset.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a model learning device, a model learning method, and a program, and more particularly to a model learning device, a model learning method, and a program that learn a model by supervised machine learning. [Background technology]

[0002] In recent years, various home appliances that operate based on judgment criteria and behavior patterns provided by programs, such as smart speakers, pet robots, and robot vacuum cleaners, have been developed. These home appliances are also called AI (Artificial Intelligence) home appliances or AI-equipped home appliances. The advanced functions of AI-equipped home appliances are provided by a processor reading and executing a program (called a trained model or simply a model) created by machine learning. For example, an AI model is constructed using a neural network such as a CNN (Convolutional Neural Network). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-30738 A Summary of the Invention [Problem to be solved by the invention]

[0004] The operation of general-purpose consumer electronics products does not always match the specific needs of a user.

[0005] The present invention has been made in consideration of the above problems, and has as its object to make it possible to modify an AI model in accordance with the needs of a user. [Means for solving the problem]

[0006] A model learning device according to one aspect of the present invention includes a search means for searching for prediction factors of a model trained using a training dataset, a confirmation means for presenting the prediction factors and confirming whether or not it is necessary to retrain the model, a construction means for reconstructing the training dataset based on the prediction factors, and a learning means for retraining the model using the reconstructed training dataset.

[0007] A model training method according to one aspect of the present invention searches for factors of predictions made by a model trained using a training dataset, presents the factors of prediction and confirms whether or not it is necessary to retrain the model, reconstructs the training dataset based on the factors of prediction, and retrains the model using the reconstructed training dataset.

[0008] A program according to one aspect of the present invention causes a computer to search for prediction factors of a model trained using a training dataset, present the prediction factors and confirm whether or not it is necessary to retrain the model, reconstruct the training dataset based on the prediction factors, and retrain the model using the reconstructed training dataset. Effect of the Invention

[0009] According to one aspect of the present invention, the AI ​​model can be modified according to the user's needs. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating a schematic configuration of a system to which a model learning device according to all of the embodiments is applied. [Diagram 2] FIG. 1 is a block diagram showing a configuration of a model learning device according to a first embodiment. [Diagram 3] 4 is a flowchart showing an example of the operation of the model learning device according to the first embodiment. [Figure 4]FIG. 4 is a diagram showing an example of a search for prediction factors by the model learning device according to the first embodiment. [Diagram 5] FIG. 11 is a block diagram showing the configuration of a model learning device according to a second embodiment. [Figure 6] FIG. 11 is a sequence diagram showing the flow of processes executed by each element of a system including a model learning device according to a second embodiment. [Figure 7] FIG. 11 is a block diagram showing the configuration of a model learning device according to a third embodiment. [Figure 8] FIG. 11 is a sequence diagram showing the flow of processes executed by each element of a system including a model learning device according to a third embodiment. [Figure 9] FIG. 11 is a diagram showing an example of an operation performed by a user on a user terminal to input learning data in a system including a model learning device according to a third embodiment. [Figure 10] FIG. 1 is a diagram illustrating a schematic configuration of a system to which a model learning device according to all of the embodiments is applied. [Figure 11] 11 is a diagram illustrating an example of a hardware configuration of a user terminal included in the system illustrated in FIG. 1 or FIG. [Figure 12] FIG. 2 is a diagram illustrating an example of a hardware configuration of a model learning device according to the first to third embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] (Common to all embodiments) First, an example of a system configuration common to all the embodiments will be described.

[0012] (System 1) FIG. 1 illustrates an example of a configuration of a system 1 to which any one of model learning devices 10, 20, and 30 according to embodiments 1 to 3 described below is applied. As illustrated in FIG. 1, the system 1 includes a user terminal 100, a home appliance 200, and a server 300. The user terminal 100, the home appliance 200, and the server 300 are connected to each other via a wireless communication network. In the following description, it is assumed that the user terminal 100 and the home appliance 200 are located in a user's home, and the server 300 is located on the network, for example.

[0013] The user terminal 100 is an information device carried by a user. For example, the user terminal 100 is a smartphone or a tablet terminal. The user terminal 100 connects to the home appliance 200 by wireless communication technology such as Wi-Fi (registered trademark) or Bluetooth (registered trademark).

[0014] An application for remotely controlling the home appliance 200 is installed in the user terminal 100. The user inputs an instruction to the home appliance 200 into a UI (User Interface) of the application, whereby an instruction command is transmitted from the user terminal 100 to the home appliance 200. This allows the user to remotely control the home appliance 200 using the user terminal 100.

[0015] Furthermore, the user terminal 100 is used to transmit a request for improving the operation of the home appliance 200 to the home appliance 200 or the server 300, as will be described in detail in the first to third embodiments.

[0016] The home appliance 200 is a general consumer appliance or device that operates based on criteria and behavioral patterns provided by a program. The home appliance 200 is a so-called AI home appliance. In one example, the home appliance 200 is a robot vacuum cleaner that has a function of moving by itself and cleaning floors while avoiding various obstacles in the home.

[0017] In the following embodiments 1 to 3, it is assumed that the home appliance 200 is a robot vacuum cleaner. However, the home appliance 200 is not limited to a robot vacuum cleaner. In another example, the home appliance 200 may be a pet robot, a communication robot, or a small drone. Alternatively, the system 1 may include any electronic device equipped with AI instead of the home appliance 200. In one example, the system 1 may include a biometric authentication device that performs face authentication or iris authentication, or an estimation device that predicts a situation / state, instead of the home appliance 200. In this case, in the following description, the "home appliance 200" can be read as an "electronic device", a "biometric authentication device", or an "estimation device".

[0018] The server 300 is constructed on the Internet, a local network, or a network of an intermediate scale. The server 300 may be an on-premise server or a cloud server. The server 300 transmits various data such as update programs to the home appliance 200 via the network on a regular or irregular basis.

[0019] In addition, the server 300 creates an AI model that defines the judgment criteria and behavioral patterns of the home appliance 200, and transmits the created model to the home appliance 200. Here, the AI ​​model is a program that allows the computer of the home appliance 200 to execute advanced judgments such as prediction, inference, classification, optimization, understanding, or recognition, and is created by machine learning of a large amount of learning data. For example, a robot vacuum cleaner recognizes obstacles scattered on the floor of a home, predicts collisions, and controls the motor of the driving wheel to avoid the obstacles. This allows the robot vacuum cleaner to move not only in a simple straight line, but also while cleaning (sweeping) a room or the entire house.

[0020] [Embodiment 1] The first embodiment will be described with reference to FIGS.

[0021] (Model learning device 10) The configuration of the model learning device 10 according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the configuration of the model learning device 10. In one example, each function of the model learning device 10 described below is realized by a server 300. However, some or all of the functions of the model learning device 10 may be realized by a control unit of a home appliance 200 instead of the server 300.

[0022] As shown in FIG. 2, the model learning device 10 includes a search unit 11, a confirmation unit 12, a configuration unit 13, and a learning unit .

[0023] The search unit 11 searches for factors of prediction by a model trained using a training data set. The model is trained using, for example, a training data set stored in a training database (not shown), and is, for example, AI software installed in the home appliance 200. The training database is provided in the server 300 or the home appliance 200. The training database may be provided in another device with which the model learning device 10 can communicate. The search unit 11 is an example of a search means.

[0024] In one example, the search unit 11 uses a sensitivity analysis technique. Generally, sensitivity analysis measures the magnitude of the effect that a change in a certain event (parameter) has on a result (function). Here, the search unit 11 measures the magnitude of the effect that a change in learning data used for model learning has on a prediction (e.g., class classification). For this reason, an influence function is used as an index representing the magnitude of the influence on the prediction. The influence function represents the accuracy (score) of the prediction using the test data.

[0025] The search unit 11 quantitatively calculates the change in the value of the influence function when each parameter of the influence function is changed. Then, the search unit 11 searches for learning data (i.e., prediction factors) that have a large effect on prediction by the model from a learning data set stored in a learning database (not shown). Each parameter of the influence function corresponds one-to-one with the learning data that constitutes the learning data set. In one example, learning data that has a large effect is data in which the change in the value of the influence function due to the presence or absence of that learning data is equal to or greater than a threshold. An example of a method for searching for learning data (i.e., prediction factors) that have a large effect on prediction by the model will be described later (FIG. 4).

[0026] The search unit 11 outputs information indicating the training data that has a large influence on the prediction by the model to each of the confirmation unit 12 and the configuration unit 13.

[0027] The confirmation unit 12 presents the factors of the prediction to confirm whether or not it is necessary to retrain the model. The confirmation unit 12 is an example of a confirmation means. Here, the presentation means that information is made known through the five senses. Specifically, the confirmation unit 12 clearly indicates the factors of the prediction by a means of presenting any one of characters, symbols, numbers, designs, sounds, light emission, and vibrations, or a combination thereof.

[0028] In one example, the confirmation unit 12 receives information indicating the learning data that has a large influence on the prediction by the model from the search unit 11. The confirmation unit 12 presents the learning data that has a large influence on the prediction by the model as a factor of the prediction by the above-mentioned means. For example, the confirmation unit 12 displays the learning data that is a factor of the prediction on the screen of the user terminal 100. Then, the confirmation unit 12 transmits a message to the user terminal 100 asking whether or not re-learning is necessary. Furthermore, the confirmation unit 12 transmits a request for a response to the confirmation of whether or not re-learning is necessary to the user terminal 100. When the confirmation unit 12 receives a response from the user terminal 100 that re-learning is necessary, it instructs the configuration unit 13 to reconstruct the learning dataset.

[0029] The configuration unit 13 reconfigures the training data set based on the prediction factors. In one example, the configuration unit 13 reconfigures the training data set due to receiving a response to the confirmation of the need for retraining. The configuration unit 13 is an example of a configuration means.

[0030] In one example, when the configuration unit 13 receives an instruction to reconstruct the training data set from the confirmation unit 12, the configuration unit 13 receives information indicating training data that has a large influence on predictions made by the model from the search unit 11. The configuration unit 13 removes training data that has a large influence on predictions made by the model from the training data set stored in a training database (not shown).

[0031] Thereafter, the configuration unit 13 selects, from the remaining training data, training data that constitutes a training data set for retraining the model, using an arbitrary method. For example, the configuration unit 13 may select all of the remaining training data, or may randomly select a predetermined number of training data from the remaining training data.

[0032] The configuration unit 13 reconstructs a training data set for retraining the model using the training data selected from the training database, and outputs the training data set reconstructed in this manner to the training unit 14.

[0033] The learning unit 14 re-learns the model using the reconstructed training data set. The learning unit 14 is an example of a learning means.

[0034] In one example, the training unit 14 receives the reconstructed training data set from the configuration unit 13. The training unit 14 retrains the model by applying the reconstructed training data set to the model. The retraining method is machine learning such as deep learning.

[0035] (Operation of model learning device 10) The operation of the model learning device 10 according to the first embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of processes executed by each unit of the model learning device 10. Here, it is assumed that the functions corresponding to each unit of the model learning device 10 are realized by the server 300 of the system 1 (Fig. 1).

[0036] In one example, after receiving a request for improving the operation of the home appliance 200 from the user terminal 100, the model learning device 10 starts the operation described below.

[0037] As shown in FIG. 3, first, the search unit 11 searches for factors of prediction by a model trained using a training data set (S1).

[0038] The prediction factor here is the learning data that has a large influence on the prediction by the model among the learning data used for learning the model. The search unit 11 outputs information indicating the learning data that has a large influence on the prediction by the model to each of the confirmation unit 12 and the configuration unit 13.

[0039] Next, the confirmation unit 12 presents the factors of the prediction and confirms whether or not it is necessary to re-learn the model (S2). In one example, the confirmation unit 12 transmits a message to the user terminal 100 asking whether or not it is necessary to re-learn. The confirmation unit 12 also transmits a request for a response to the confirmation of whether or not it is necessary to re-learn to the user terminal 100.

[0040] Configuration unit 13 receives a response to the confirmation as to whether re-learning is necessary from user terminal 100. If a response that re-learning is not necessary is received from user terminal 100 (No in S3-1), operation of model learning device 10 ends.

[0041] On the other hand, when a request for re-learning is received from the user terminal 100 (Yes in S3-1), the configuration unit 13 reconstructs the learning dataset based on the prediction factors (S3-2). In one example, the configuration unit 13 reconstructs the learning dataset for re-learning by excluding learning data that has a large influence on the prediction by the model from the learning dataset stored in the learning database.

[0042] Then, the configuration unit 13 outputs the reconstructed training data set to the training unit 14.

[0043] Finally, the learning unit 14 retrains the model using the reconstructed training data set (S4). In other words, the learning unit 14 applies the retrained model to the home appliance 200, thereby updating the model.

[0044] This is the end of the operation of the model learning device 10 according to the first embodiment.

[0045] (Modification) In one modified example, the learning unit 14 confirms with the user that the AI ​​model installed in the home appliance 200 is to be updated before applying the re-learned model to the home appliance 200. In one example, the learning unit 14 transmits a message confirming whether or not to update the AI ​​model installed in the home appliance 200 from the server 300 to the user terminal 100, and causes the message to be displayed on the screen of the user terminal 100.

[0046] In addition, after obtaining confirmation for updating the model installed in the home appliance 200, the learning unit 14 applies the re-learned model to the home appliance 200. First, the learning unit 14 requests the user terminal 100 to accept input of the confirmation. After receiving the confirmation input to the user terminal 100 from the user terminal 100, the learning unit 14 applies the re-learned model to the home appliance 200. Specifically, the learning unit 14 replaces the model stored in the auxiliary storage device 230 (FIG. 11) of the home appliance 200 with the re-learned model.

[0047] On the other hand, if the learning unit 14 receives a response from the user terminal 100 indicating that it is not possible to update the AI ​​model installed in the home appliance 200, the model learning device 10 repeats the processing from step S1 (FIG. 3) described above.

[0048] (An example of exploring predictive factors) An example of a method in which the search unit 11 searches for learning data that has a large effect on prediction by a model (that is, factors of prediction) will be described with reference to FIG.

[0049] In the table shown in Figure 4, the left column shows a list of the learning data stored in the learning database. The center and right columns of the same table show the degree of influence (hereinafter referred to as the influence degree) that the corresponding learning data in the left column has on a specific prediction (predictions A and B in Figure 4). In other words, it indicates the likelihood that the presence or absence of the learning data in the left column will change the prediction made by the AI ​​model. The influence degree parameter is the value of the influence function mentioned above. Prediction A and Prediction B represent different contents. For example, prediction A is to classify an object into class a and the rest, and prediction B is to classify an object into class b and the rest.

[0050] In the table shown in FIG. 4, the learning data are arranged in descending order of the degree of influence on prediction A. Therefore, the higher the learning data is in the left column of the table, the greater the influence on prediction A. For example, assume that the search unit 11 searches for three factors of prediction A. In this case, the search unit 11 searches for the three learning data framed in FIG. 4 from the list of learning data stored in the learning database based on the degree of influence on prediction A.

[0051] (Effects of this embodiment) The operation of general-purpose consumer electronics products does not necessarily match the specific needs of the user, and related technologies cannot modify the AI ​​model according to the user's needs.

[0052] According to the configuration of this embodiment, the search unit 11 searches for prediction factors by a model trained using a training data set. The confirmation unit 12 presents the prediction factors and confirms whether or not the model needs to be retrained. The configuration unit 13 reconstructs the training data set based on the prediction factors. The training unit 14 retrains the model using the reconstructed training data set. Before the AI ​​model is retrained, the prediction factors are presented and whether or not retraining is necessary is confirmed. Therefore, the user can determine whether or not retraining should be performed. This allows the AI ​​model to be modified according to the user's request.

[0053] [Embodiment 2] A second embodiment will be described with reference to Figs. 5 and 6. In the second embodiment, a model learning device 20 having a different configuration from the model learning device 10 (Fig. 2) described in the first embodiment will be described. The model learning device 20 according to the second embodiment is applied to the above-mentioned system 1 (Fig. 1). In one example, each function of the model learning device 20 described below is realized by a server 300. However, some or all of the functions of the model learning device 20 may be realized by a control unit of a home appliance 200 instead of the server 300.

[0054] In the second embodiment, the description of the first embodiment will be cited with regard to the configuration that the model learning device 20 has in common with the model learning device 10.

[0055] (Model learning device 20) The configuration of the model learning device 20 according to the second embodiment will be described with reference to Fig. 5. Fig. 5 is a block diagram showing the configuration of the model learning device 20. Each function of the model learning device 20 described below is shared between the server 300 and the home appliance 200. However, all of the functions of the model learning device 10 may be realized by only the control unit of the home appliance 200.

[0056] 5, the model learning device 20 includes a search unit 11, a confirmation unit 12, a configuration unit 13, and a learning unit 14. In addition, the model learning device 20 further includes an output unit 25.

[0057] The output unit 25 generates text data explaining the factors of the prediction. The output unit 25 is an example of an output means.

[0058] In one example, the output unit 25 receives information indicating the learning data that has a large influence on the prediction by the model from the search unit 11. The output unit 25 generates text data explaining that the learning data that has a large influence on the prediction is a factor of the prediction.

[0059] For example, the output unit 25 may use a text generation model that has been machine-learned. Specifically, the output unit 25 inputs information indicating learning data that has a large effect on the prediction to the text generation model. The text generation model outputs text data that explains the factors of the prediction based on the input information. The output unit 25 acquires the text data output by the text generation model.

[0060] Alternatively, in another example, the output unit 25 may use a template prepared in advance in a memory to generate text data explaining factors of the prediction from information indicating learning data that has a large influence on the prediction.

[0061] The output unit 25 transmits the text data thus generated to the home appliance 200. The output unit 25 instructs an audio control unit (e.g., an audio driver, a sound card, a DA conversion circuit) (not shown) of the home appliance 200 to convert the transmitted text data into audio data. Then, the output unit 25 outputs the audio data converted from the text data from a speaker of the home appliance 200. After that, the output unit 25 notifies the confirmation unit 12 that the explanation of the factors of the prediction has been completed.

[0062] In the second embodiment, the confirmation unit 12 executes confirmation of the necessity of re-learning, triggered by a notification from the output unit 25. That is, after the output unit 25 completes the explanation of the factors of the prediction, the confirmation unit 12 transmits a message to the user terminal 100 to confirm the necessity of re-learning. Therefore, the user can determine whether or not to re-learn the model after understanding the factors of the prediction. Furthermore, when the confirmation unit 12 receives a response from the user terminal 100 indicating that re-learning is necessary, the confirmation unit 12 instructs the configuration unit 13 to reconstruct the learning data set. As in the first embodiment, when the configuration unit 13 receives an instruction from the confirmation unit 12 to reconstruct the learning data set, the configuration unit 13 receives information indicating the learning data that has a large influence on the prediction by the model from the search unit 11.

[0063] (System 1 operation) An example of the operation of the system 1 (FIG. 1) equipped with the model learning device 20 according to the present embodiment 2 will be described with reference to Fig. 6. Fig. 6 is a sequence diagram showing the flow of processing executed by each unit of the system 1 according to the present embodiment 2. Here, it is assumed that the functions corresponding to each unit of the model learning device 20 are realized by the server 300.

[0064] 6, first, the user terminal 100 requests the server 300 for an explanation on the operation of the home appliance 200 (corresponding to the prediction) (S21). The user inputs the contents of the request to the user terminal 100. After the user inputs the contents of the request to the user terminal 100, a request for an explanation on the operation of the home appliance is transmitted from the user terminal 100 to the server 300.

[0065] Next, the search unit 11 of the model learning device 20 searches for factors of prediction by the model trained using the training data set (S22). The factors of prediction here are learning data that have a particularly large effect on prediction by the model, among the learning data used for training the model.

[0066] The search unit 11 outputs information indicating the training data that has a large influence on the prediction by the model to each of the confirmation unit 12 and the configuration unit 13. In addition, the search unit 11 also outputs information indicating the training data that has a large influence on the prediction by the model to the output unit 25.

[0067] The output unit 25 generates text data explaining that the learning data having a large influence on the prediction is a factor of the prediction (S23). The output unit 25 transmits the generated text data to the home appliance 200. The output unit 25 also causes the speaker of the home appliance 200 to output a voice restored from the text data. The output unit 25 notifies the confirmation unit 12 that the explanation of the factors of the prediction has been completed.

[0068] The home appliance 200 receives the text data from the confirmation unit 12. The control unit (not shown) of the home appliance 200 converts the text data into voice data in response to an instruction from the confirmation unit 12. Then, the control unit of the home appliance 200 outputs voice from a speaker (S24).

[0069] The confirmation unit 12 is notified by the output unit 25 that the explanation of the prediction factors has been completed. Thereafter, the confirmation unit 12 presents the prediction factors and confirms whether or not re-learning of the model is necessary (S25). In one example, the confirmation unit 12 transmits a message to the user terminal 100 asking whether or not re-learning is necessary. In addition, the confirmation unit 12 transmits a request for a response to the message to the user terminal 100.

[0070] The user inputs whether or not he or she wishes to re-learn into the user terminal 100. Here, it is assumed that the user wishes to re-learn. The user terminal 100 transmits the wish to re-learn to the server 300 as a response to the message (S26).

[0071] The confirmation unit 12 receives a response to the message from the user terminal 100. The confirmation unit 12 instructs the configuration unit 13 to reconstruct the learning dataset. When the configuration unit 13 receives an instruction to reconstruct the learning dataset from the confirmation unit 12, the configuration unit 13 reconstructs the learning dataset based on the prediction factors (S27). Then, the configuration unit 13 outputs the reconstructed learning dataset to the learning unit 14.

[0072] The learning unit 14 re-learns the model using the reconstructed learning data set (S28). After that, the learning unit 14 applies the re-learned model to the home appliance 200. That is, the learning unit 14 updates the AI ​​model installed in the home appliance 200 with the re-learned model.

[0073] In one modification, similar to the modification described in the first embodiment, the learning unit 14 confirms with the user that the AI ​​model installed in the home appliance 200 is to be updated before applying the re-learned model to the home appliance 200. After receiving from the user terminal 100 the confirmation input to the user terminal 100, the learning unit 14 applies the re-learned model to the home appliance 200.

[0074] This is the end of the operation of the system 1 equipped with the model learning device 20 according to the second embodiment.

[0075] (Effects of this embodiment) The operation of general-purpose consumer electronics products does not necessarily match the specific needs of the user, and related technologies cannot modify the AI ​​model according to the user's needs.

[0076] According to the configuration of this embodiment, the search unit 11 searches for prediction factors by a model trained using a training data set. The confirmation unit 12 presents the prediction factors and confirms whether or not the model needs to be retrained. The configuration unit 13 reconstructs the training data set based on the prediction factors. The training unit 14 retrains the model using the reconstructed training data set. Before the AI ​​model is retrained, the prediction factors are presented and whether or not retraining is necessary is confirmed. Therefore, the user can determine whether or not retraining should be performed. This allows the AI ​​model to be modified according to the user's request.

[0077] Furthermore, according to the configuration of this embodiment, the output unit 25 generates text data explaining the factors of the prediction. The text data explaining the factors of the prediction is converted into any form, for example, into voice data, and then output. This allows the user to understand the factors of the prediction, and therefore allows the user to more appropriately determine whether or not to retrain the AI ​​model.

[0078] [Embodiment 3] A third embodiment will be described with reference to Figs. 7 to 9. There is a strong demand for customizing home appliances as desired by users by modifying (updating) AI models. However, there are not enough experts such as data scientists and AI engineers to fully handle the customization of individual home appliances, and the cost required for such handling is also enormous. In this third embodiment, an example of a means for solving this problem will be described.

[0079] In the present embodiment 3, a model learning device 30 having a configuration different from both the model learning device 10 (FIG. 2) described in the above-mentioned embodiment 1 and the model learning device 20 (FIG. 5) described in the above-mentioned embodiment 2 will be described. The model learning device 30 according to the present embodiment 3 is applied to the above-mentioned system 1 (FIG. 1). In one example, each function of the model learning device 30 described below is realized by a server 300. However, some or all of the functions of the model learning device 30 may be realized by a control unit of a home appliance 200 instead of the server 300.

[0080] In the third embodiment, the description of the first embodiment will be cited with regard to the configuration that the model learning device 30 has in common with the model learning device 10.

[0081] (Model learning device 30) The configuration of the model learning device 30 according to the third embodiment will be described with reference to Fig. 7. Fig. 7 is a block diagram showing the configuration of the model learning device 30. Each function of the model learning device 30 described below is realized by the server 300. However, some or all of the functions of the model learning device 30 may be realized by a control unit of the home appliance 200 instead of the server 300.

[0082] 7, the model learning device 30 includes a search unit 11, a confirmation unit 12, a configuration unit 13, and a learning unit 14. In addition, the model learning device 30 further includes an output unit 25 and a receiving unit .

[0083] The output unit 25 generates text data explaining the factors of the prediction. The output unit 25 is an example of an output means.

[0084] In one example, the output unit 25 receives information indicating the learning data that has a large influence on the prediction by the model from the search unit 11. The output unit 25 generates text data explaining that the learning data that has a large influence on the prediction is a factor of the prediction.

[0085] For example, the output unit 25 may use a machine-learned text generation model. Specifically, the output unit 25 inputs information indicating learning data that has a large effect on the prediction to the text generation model. The text generation model outputs text data that explains the factors of the prediction. Then, the output unit 25 acquires the text data output by the text generation model.

[0086] Alternatively, in another example, the output section 25 may generate text data explaining the factors of the prediction using a template prepared in advance in a memory.

[0087] The receiving unit 36 ​​receives input of learning data for re-learning the model. The receiving unit 36 ​​is an example of a receiving means.

[0088] In one example, the receiving unit 36 ​​transmits a message notifying the user terminal 100 that the input of the learning data is accepted, to the user terminal 100. In addition, the receiving unit 36 ​​transmits a request for a response to the message to the user terminal 100.

[0089] The reception unit 36 ​​receives a response to the message from the user terminal 100. When the reception unit 36 ​​receives learning data input by a user from the user terminal 100, the reception unit 36 ​​outputs the received learning data to the configuration unit 13.

[0090] As in the first embodiment, when the confirmation unit 12 receives a response from the user terminal 100 indicating that re-learning is necessary, the confirmation unit 12 instructs the configuration unit 13 to reconstruct the training data set. In the third embodiment, the configuration unit 13 receives the training data input to the user terminal 100 from the reception unit 36. Furthermore, when the configuration unit 13 receives an instruction from the confirmation unit 12 to reconstruct the training data set, the configuration unit 13 receives from the search unit 11 information indicating training data that has a large influence on predictions made by the model.

[0091] The configuration unit 13 selects learning data constituting the learning data set for re-learning by the method described in the embodiment 1. Furthermore, the configuration unit 13 adds the learning data input to the user terminal 100 to the learning data set for re-learning. The configuration unit 13 outputs the learning data set reconstructed in this manner to the learning unit 14.

[0092] (System 1 operation) An example of the operation of the system 1 (FIG. 1) including the model learning device 30 according to the present embodiment 3 will be described with reference to Fig. 8. Fig. 8 is a sequence diagram showing the flow of processing executed by each unit of the system 1 according to the present embodiment 3. Here, it is assumed that the functions corresponding to each unit of the model learning device 30 are realized by the server 300.

[0093] As shown in Fig. 8, first, the user terminal 100 requests the server 300 for an explanation on the operation (corresponding to the prediction) of the home appliance 200 (S21). The user inputs the contents of the request to the user terminal 100. After the user inputs the contents of the request to the user terminal 100, a request for an explanation on the operation of the home appliance is transmitted from the user terminal 100 to the server 300.

[0094] Next, the search unit 11 of the model learning device 20 searches for factors of prediction by the model trained using the training data set (S22). The factors of prediction here are learning data that have a particularly large effect on prediction by the model, among the learning data used for training the model.

[0095] The search unit 11 outputs information indicating the training data that has a large influence on the prediction by the model to each of the confirmation unit 12 and the configuration unit 13. In addition, the search unit 11 also outputs information indicating the training data that has a large influence on the prediction by the model to the output unit 25.

[0096] The output unit 25 generates text data explaining that the learning data having a large influence on the prediction is a factor of the prediction (S23). The output unit 25 transmits the generated text data to the home appliance 200. The output unit 25 also causes the speaker of the home appliance 200 to output voice data converted from the text data. The output unit 25 notifies the confirmation unit 12 that the explanation of the factors of the prediction has been completed.

[0097] The home appliance 200 receives the text data from the confirmation unit 12. The control unit (not shown) of the home appliance 200 converts the text data into voice data in response to an instruction from the confirmation unit 12. Then, the control unit of the home appliance 200 outputs the voice data to a speaker (S24).

[0098] The confirmation unit 12 is notified by the output unit 25 that the explanation of the prediction factors has been completed. Thereafter, the confirmation unit 12 presents the prediction factors and confirms whether or not re-learning of the model is necessary (S25). In one example, the confirmation unit 12 transmits a first message to the user terminal 100 inquiring whether or not re-learning is necessary. In addition, the confirmation unit 12 transmits a request for a response to the first message to the user terminal 100.

[0099] The user inputs whether or not he or she wishes to re-learn into the user terminal 100. Here, it is assumed that the user wishes to re-learn. The user terminal 100 transmits the desire to re-learn to the server 300 as a response to the first message (S26).

[0100] The reception unit 36 ​​receives a response to the first message from the user terminal 100. When a request for re-learning is received from the user terminal 100, the reception unit 36 ​​transmits a second message to the user terminal 100 notifying that input of learning data is accepted (S31). In addition, the reception unit 36 ​​transmits a request for a response to the second message to the user terminal 100.

[0101] The user inputs learning data for re-learning the model to the user terminal 100. The user terminal 100 transmits the input learning data to the server 300 as a response to the second message (S32).

[0102] The receiving unit 36 ​​receives a response to the second message from the user terminal 100. When the receiving unit 36 ​​receives the learning data input by the user from the user terminal 100, the receiving unit 36 ​​outputs the input learning data to the configuration unit 13.

[0103] When the configuration unit 13 receives an instruction to reconstruct the learning dataset from the confirmation unit 12, the configuration unit 13 reconstructs the learning dataset based on the prediction factors (S27). Furthermore, the configuration unit 13 adds the learning data input to the user terminal 100 to the learning dataset. Then, the configuration unit 13 outputs the reconstructed learning dataset to the learning unit 14.

[0104] The learning unit 14 re-learns the model using the reconstructed learning data set (S28). After that, the learning unit 14 applies the re-learned model to the home appliance 200. That is, the learning unit 14 updates the AI ​​model installed in the home appliance 200 with the re-learned model.

[0105] In one modification, similar to the modification described in the first embodiment, the learning unit 14 confirms with the user that the AI ​​model installed in the home appliance 200 is to be updated before applying the re-learned model to the home appliance 200. After receiving from the user terminal 100 the confirmation input to the user terminal 100, the learning unit 14 applies the re-learned model to the home appliance 200.

[0106] This is the end of the operation of the system 1 equipped with the model learning device 30 according to the third embodiment.

[0107] (An example of learning data input) Fig. 9 shows an example of a method for inputting learning data to the user terminal 100. In the example shown in Fig. 9, the user performs a touch operation on the screen of the user terminal 100. Here, the image displayed on the screen of the user terminal 100 is a frame image of a video captured by the camera 250 (Fig. 11) of the home appliance 200.

[0108] The reception unit 36 ​​(FIG. 7) instructs the control unit of the home appliance 200 to transmit video data to the user terminal 100. Alternatively, the reception unit 36 ​​acquires video data from the home appliance 200 in advance. Then, the reception unit 36 ​​may transfer the video data acquired from the home appliance 200 to the user terminal 100 when accepting input of learning data for re-learning the model.

[0109] 9, the user designates an area of ​​object X that the user wishes to designate as learning data in an image displayed on the screen of the user terminal 100 by a touch operation such as dragging or tapping. However, a touch operation other than these or an operation different from a touch operation may be used. The user terminal 100 transmits data of a partial image including the area of ​​object X to the server 300 as learning data.

[0110] The functions of the user terminal 100 described in the third embodiment may be provided by the home appliance 200 instead of the user terminal 100. Specifically, the user may input learning data to the home appliance 200 (FIG. 9) by directly operating the home appliance 200 instead of the user terminal 100. In this case, the user can customize the home appliance 200 without the user terminal 100.

[0111] (Effects of this embodiment) The operation of general-purpose consumer electronics products does not necessarily match the specific needs of the user, and related technologies cannot modify the AI ​​model according to the user's needs.

[0112] According to the configuration of this embodiment, the search unit 11 searches for prediction factors by a model trained using a training data set. The confirmation unit 12 presents the prediction factors and confirms whether or not the model needs to be retrained. The configuration unit 13 reconstructs the training data set based on the prediction factors. The training unit 14 retrains the model using the reconstructed training data set. Before the AI ​​model is retrained, the prediction factors are presented and whether or not retraining is necessary is confirmed. Therefore, the user can determine whether or not retraining should be performed. This allows the AI ​​model to be modified according to the user's request.

[0113] Furthermore, according to the configuration of this embodiment, the output unit 25 generates text data explaining the factors of the prediction. The text data explaining the factors of the prediction is converted into any form, for example, into voice data, and then output. This allows the user to understand the factors of the prediction, and therefore allows the user to more appropriately determine whether or not to retrain the AI ​​model.

[0114] Furthermore, according to the configuration of this embodiment, the receiving unit 36 ​​receives input of learning data for retraining the model. The configuration unit 13 adds the input learning data to a learning data set for retraining the model. This makes it possible to obtain a model that better reflects the user's needs.

[0115] (Common to all embodiments) Here, we will explain system 2, which has a different configuration from system 1 described first. Any of the model learning devices 10 to 30 according to the above-mentioned embodiments 1 to 3 can be applied not only to system 1 but also to system 2.

[0116] Next, an example of the hardware configuration of home appliance 200, which is a common element between system 1 and system 2, will be described.

[0117] (System 2) FIG. 10 illustrates an example of the configuration of a system 2 to which any one of the model learning devices 10, 20, and 30 according to the first to third embodiments is applied. As illustrated in FIG. 10, the system 2 includes a user terminal 100 and a home appliance 200. The system 2 differs from the above-described system 1 in that the system 2 does not include a server 300 (FIG. 1). The user terminal 100 and the home appliance 200 are connected to each other via a wireless communication network. Here, both the user terminal 100 and the home appliance 200 are located in the user's home.

[0118] In the system 2, a control unit (not shown) of the home appliance 200 functions as each unit of the model learning devices 10, 20, and 30 according to the first to third embodiments. The functions of each unit of the model learning devices 10, 20, and 30 according to the first to third embodiments are realized by cooperation between a calculation unit 210 and a memory 220 (both shown in FIG. 11) of the home appliance 200. More specifically, the calculation unit 210 executes a computer program loaded into the memory 220 to execute processing corresponding to each unit of the model learning devices 10, 20, and 30 according to the first to third embodiments. The calculation unit 210 is, for example, a central processing unit (CPU), a graphics processing unit (GPU), or an application specific integrated circuit (ASIC).

[0119] In the system 2, unlike the system 1, processing is not performed on the server 300. Therefore, even if the user terminal 100 and the home appliance 200 are not connected to the Internet (or a network on which the server 300 is located), the functions of the respective parts of the model learning devices 10 to 30 described in the first to third embodiments can be provided.

[0120] (Hardware configuration of home appliance 200) 11 shows an example of a schematic hardware configuration of home appliance 200. Here, it is assumed that home appliance 200 is a robot vacuum cleaner. A robot vacuum cleaner is a high-performance vacuum cleaner that travels by itself while avoiding various obstacles in the home, and mainly cleans floors.

[0121] As shown in FIG. 11, the home appliance 200 includes a calculation unit 210, a memory 220, an auxiliary storage device 230, a timer 240, a camera 250, an input / output unit 260, a drive unit 270, and an instrument 280.

[0122] The calculation unit 210 executes information calculation processing in accordance with a computer program loaded into the memory 220. For example, the calculation unit 210 is one or more processors.

[0123] The memory 220 is a so-called main memory for storing program data being executed by the calculation unit 210. For example, the memory 220 is a RAM (Random Access Memory). The calculation unit 210 and the memory 220 work together to function as a control unit of the home appliance 200. In particular, in the above-mentioned system 2 (FIG. 10), the calculation unit 210 and the memory 220 provide functions corresponding to the respective units of the model learning devices 10 to 30 according to the above-mentioned embodiments 1 to 3.

[0124] The auxiliary storage device 230 is a storage such as a hard disk or a media drive. The auxiliary storage device 230 may be a cartridge type that is removable from the home appliance 200. The auxiliary storage device 230 stores an AI model together with various program data for operating the home appliance 200.

[0125] The timer 240 counts time. The timer 240 is used by the calculation unit 210 so that the home appliance 200 operates correctly according to a preset time schedule.

[0126] The camera 250 is used by the calculation unit 210 so that the home appliance 200 can recognize the environment and operate while avoiding obstacles. In addition, as described in the third embodiment, the video data captured by the camera 250 is also used by the user to input learning data to the user terminal 100 (FIG. 9).

[0127] Input / output unit 260 includes devices for communication with a user, such as a power button and a display of home appliance 200. Input / output unit 260 also includes a communication unit for connecting to a network.

[0128] The driving unit 270 is a movable part and auxiliary parts for operating the home appliance 200. The driving unit 270 includes a battery, a motor, wheels (driving wheels), a shaft, and brushes.

[0129] The gauge 280 measures the environmental conditions of the home appliance 200. The gauge 280 includes a pressure gauge, a thermometer, a hygrometer, an acceleration sensor, and an infrared sensor.

[0130] (Hardware configuration) Each of the components of the model learning devices 10 to 30 described in the first to third embodiments is represented by a functional block. Some or all of these components are realized by an information processing device 900 as shown in Fig. 12. Fig. 12 is a block diagram showing an example of the hardware configuration of the information processing device 900.

[0131] As shown in FIG. 12, an information processing device 900 includes, for example, the following configuration.

[0132] ·CPU(Central Processing Unit)901 ROM (Read Only Memory) 902 ·RAM(Random Access Memory)903 Program 904 loaded into RAM 903 A storage device 905 for storing a program 904 A drive device 907 for reading and writing data from the recording medium 906 A communication interface 908 for connecting to a communication network 909 An input / output interface 910 for inputting and outputting data · Bus 911 connecting each component Each of the components of the model learning devices 10 to 30 described in the first to third embodiments is realized by the CPU 901 reading and executing a program 904 that realizes these functions. The program 904 that realizes the functions of each component is, for example, stored in advance in the storage device 905 or the ROM 902, and is loaded into the RAM 903 by the CPU 901 and executed as necessary. The program 904 may be supplied to the CPU 901 via the communication network 909, or may be stored in advance in the recording medium 906, and the drive device 907 may read out the program and supply it to the CPU 901.

[0133] According to the above configuration, the model learning devices 10 to 30 described in the first to third embodiments are realized as hardware, and therefore it is possible to achieve the same effects as those described in the first to third embodiments.

[0134] [Additional Notes] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes.

[0135] (Appendix 1) A search means for searching factors of predictions made by a model trained using a training dataset; a confirmation means for presenting factors of the prediction and confirming whether or not the model needs to be retrained; A configuration means for reconstructing the learning data set based on the factors of the prediction; a training means for retraining the model using the reconstructed training data set; A model learning device equipped with

[0136] (Appendix 2) The confirmation means presents learning data that is a factor in the prediction. 2. The model learning device according to claim 1,

[0137] (Appendix 3) The learning means applies the re-learned model to the device after obtaining confirmation that the model installed in the device is updated. 3. The model learning device according to claim 1 or 2.

[0138] (Appendix 4) The system further includes an output unit for generating text data explaining factors of the prediction. 4. The model learning device according to claim 1,

[0139] (Appendix 5) The output means generates text data of an explanation regarding factors of the prediction in response to a request for an explanation regarding factors of the prediction. 5. The model learning device according to claim 4,

[0140] (Appendix 6) The constructing means constructs the learning data set excluding the learning data that is a factor of the prediction. 6. The model learning device according to claim 4 or 5.

[0141] (Appendix 7) The method further includes receiving means for receiving input of learning data for re-learning the model. 7. The model learning device according to claim 1,

[0142] (Appendix 8) The constructing means constructs the learning data set including the input learning data. 8. The model learning device according to claim 7,

[0143] (Appendix 9) Searching for factors behind predictions made by a model trained using a training dataset; presenting the drivers of the prediction to see if the model needs to be retrained; Reconstructing the learning data set based on the predictive factors; Retraining the model using the reconstructed training data set. Model training methods.

[0144] (Appendix 10) Searching for factors that influence predictions made by a model trained using a training dataset; presenting the drivers of the prediction to ascertain whether the model needs to be retrained; reconstructing the training data set based on the predictive factors; retraining the model using the reconstructed training data set; A program for causing a computer to execute the following.

[0145] (Appendix 11) The configuration means configures the training data set to modify predictions by the model. 11. The model learning device according to claim 1,

[0146] (Appendix 12) The output means outputs an explanation of the factors of the prediction in the form of voice or text. 7. The model learning device according to claim 4, wherein: [Industrial Applicability]

[0147] The present invention can be used in home appliances that are primarily used indoors, such as robot vacuum cleaners, pet robots, and small drones. [Explanation of symbols]

[0148] 1 System 2. System 10 Model learning device 11 Search section 12 Verification Section 13 Components 14 Learning Department 20 Model Learning Device 25 Output section 30 Model Learning Device 36 Reception Department 100 user terminals 200 Home appliances 300 Servers

Claims

1. A search means for searching factors of predictions made by a model trained using a training dataset; a confirmation means for presenting factors of the prediction and confirming whether or not the model needs to be retrained; A configuration means for reconstructing the learning data set based on the factors of the prediction; a training means for retraining the model using the reconstructed training data set; A receiving means for receiving input of learning data for re-learning the model; Equipped with When it is confirmed that the user desires to relearn, the reception means transmits a message to the user terminal notifying that the input of the learning data is accepted, and when the reception means receives the learning data input by the user from the user terminal, the reception means reconstructs the learning data set using the received learning data. Model learning device.

2. The confirmation means presents learning data that is a factor in the prediction.

2. The model learning device according to claim 1 .

3. The learning means applies the re-learned model to the device after obtaining confirmation that the model installed in the device is updated.

3. The model learning device according to claim 1, wherein the model learning device is a multi-layered model.

4. The system further includes an output unit for generating text data explaining factors of the prediction.

4. The model learning device according to claim 1, wherein the model learning device is a multi-layered model.

5. The output means generates text data of an explanation regarding factors of the prediction in response to a request for an explanation regarding factors of the prediction.

5. The model learning device according to claim 4.

6. The constructing means constructs the learning data set excluding the learning data that is a factor of the prediction.

6. The model learning device according to claim 4 or 5.

7. The constructing means constructs the learning data set including the input learning data.

2. The model learning device according to claim 1 .

8. A computer comprising: Searching for factors behind predictions made by a model trained using a training dataset; presenting the drivers of the prediction to see if the model needs to be retrained; Reconstructing the learning data set based on the predictive factors; Retraining the model using the reconstructed training data set.

1. A model training method, comprising: The computer, Accepting input of training data for retraining the model; When it is confirmed that the user wishes to re-learn, a message is sent to the user terminal notifying the user that the input of the learning data is accepted, and when the learning data input by the user is received from the user terminal, the learning data set is reconstructed using the received learning data. Model training methods.

9. Searching for factors that influence predictions made by a model trained using a training dataset; presenting the drivers of the prediction to ascertain whether the model needs to be retrained; reconstructing the training data set based on the predictive factors; retraining the model using the reconstructed training data set; A program for causing a computer to execute the above, Accepting input of training data for retraining the model; when it is confirmed that the user wishes to re-learn, transmitting a message to the user terminal notifying the user that the input of the learning data is accepted, and when the learning data input by the user is received from the user terminal, reconstructing the learning data set using the received learning data; A program for causing the computer to execute the above.

Images