Trained model management device, robot control system, and trained model management method

Abstract

To provide a learned model management device and a learned model management method that update a master model without reducing recognition accuracy in a user's use environment.SOLUTION: A learned model management device (10) comprises: a first storage unit (12A) that stores first learning data, and a first model that is a learned model subjected to learning processing so as to recognize an object included in input information on the basis of the first learning data; a second storage unit (12B) that stores second learning data, and a second model that is a learned model generated on the basis of the second learning data and the first model; and an update determination unit (132) that, when the second model is generated, determines whether to update the first model on the basis of the second learning data.SELECTED DRAWING: Figure 1

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to Japanese Patent Application No. 2021-129344 (filed August 5, 2021), the entire disclosure of which is incorporated herein by reference. [Technical field]

[0002] The present disclosure relates to a trained model management device and a trained model management method. [Background technology]

[0003] Conventionally, there is known a device that identifies an object contained in an image by learning using a learning image. For example, Patent Document 1 discloses a system that distributes a recognition model updated (re-learned) with a simulation video that reproduces a scene where recognition has failed to a vehicle via a network. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-43622 Summary of the Invention

[0005] A trained model management device according to an embodiment of the present disclosure includes: A first storage unit that stores first learning data and a first model that is a trained model trained and processed based on the first learning data so as to be able to recognize an object included in input information; A second storage unit that stores second learning data and a second model that is a trained model generated based on the second learning data and the first model; and an update determination unit that determines, when the second model is generated, whether or not to update the first model based on the second learning data.

[0006] A trained model management method according to an embodiment of the present disclosure includes: A trained model management method executed by a trained model management device including: a first storage unit that stores first learning data; and a first model that is a trained model that has been trained and processed so as to be able to recognize an object included in input information based on the first learning data; and a second storage unit that stores second learning data; and a second model that is a trained model generated based on the second learning data and the first model, When the second model is generated, determining whether or not to update the first model based on the second learning data. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a trained model management system including a trained model management device according to an embodiment. [Diagram 2] FIG. 2 is a schematic diagram showing a configuration example of a robot control system. [Diagram 3] FIG. 3 is a diagram showing an example of the configuration of a group of models provided as a library. [Figure 4] FIG. 4 is a sequence diagram illustrating an example of a processing procedure in the trained model management system. [Diagram 5] FIG. 5 is a sequence diagram continuing from FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, a trained model management device and a trained model management method according to an embodiment of the present disclosure will be described with reference to the drawings. In each drawing, the same or corresponding parts are denoted by the same reference numerals. In the description of this embodiment, the description of the same or corresponding parts will be omitted or simplified as appropriate.

[0009] (Overview of the trained model management system) FIG. 1 shows an example of the configuration of a trained model management system 1 including a trained model management device 10 according to the present embodiment. The trained model management device 10 is a device that manages updates to trained models provided as a library. A user uses a trained model for tasks such as identifying an object and displaying the identification result. The trained model management device 10 can also execute a task using evaluation data and a specific model provided by a user, and output the execution result. In the present embodiment, the model group includes a plurality of machine learning models (trained models) that are generated to recognize an object by machine learning. A user can use the trained model by downloading it via a network 40, for example.

[0010] The trained model management device 10 includes a communication unit 11, a first storage unit 12A, a second storage unit 12B, and a control unit 13. The control unit 13 includes an update determination unit 131, an update processing unit 132, and a detection unit 133. The trained model management device 10 may be a computer such as a server as a hardware configuration. The components of the trained model management device 10 will be described in detail later.

[0011] The trained model management device 10 may constitute a trained model management system 1 together with a trained model generation device 20 and a communication terminal device 30 connected via a network 40. The network 40 is, for example, the Internet, but may also be a LAN (Local Area Network). For example, the trained model management device 10 and the trained model generation device 20 may communicate via a LAN, and the trained model management device 10 and the communication terminal device 30 may communicate via the Internet.

[0012] The trained model generation device 20 can generate a master model or a custom model, which will be described later, by machine learning. That is, the trained model generation device 20 can generate a trained model by machine learning using training data such as first training data and second training data, which will be described later. The trained model generation device 20 also generates a new trained model (for example, a custom model, which will be described later). The trained model generation device 20 may also add new training data to update (re-train) a specified model. The training data may be created by capturing an image with a camera or the like, or may be created using CAD (Computer Aided Design) data. The training data may also be created by combining the captured training data with the CAD training data.

[0013] In this embodiment, the trained model generation device 20 can access the first storage unit 12A and the second storage unit 12B of the trained model management device 10. In this embodiment, the trained model generation device 20 is a computer separate from the trained model management device 10 and capable of communicating with the trained model management device 10, but is not limited to such a configuration. For example, the trained model generation device 20 may be realized by a computer integrated with the trained model management device 10. In this case, the trained model generation device 20 may have the configuration requirements of the trained model management device 10, and the trained model management device 10 may have the configuration requirements of the trained model generation device 20.

[0014] The communication terminal device 30 is, for example, a general-purpose terminal device capable of communication, such as a smartphone or a tablet terminal, but is not limited thereto. As another example, the communication terminal device 30 may be configured as a part of a robot control system 100 (see FIG. 2) described later. The communication terminal device 30 generates input information (for example, an image of an object to be identified) and outputs the input information to the trained model management device 10. The input information may be used as evaluation data for evaluating an updated model. The communication terminal device 30 outputs learning data provided by a user to the trained model management device 10. In addition, the communication terminal device 30 may present to the user the execution result of a task executed by the trained model management device 10 using the evaluation data and a specific model. The communication terminal device 30 includes a communication unit 31, a storage unit 32, a control unit 33, an input information generation unit 34, and a user interface unit 35. Details of the components of the communication terminal device 30 will be described later.

[0015] Here, in this embodiment, users who use the trained model management system 1 include a first user and a second user. The first user can use the communication terminal device 30-1 to perform, for example, generating a custom model specific to the first user. The second user can use the communication terminal device 30-2 to perform, for example, generating a custom model specific to the second user. The communication terminal device 30-2 may have the same configuration as the communication terminal device 30-1. In this embodiment, the communication terminal device 30-1 and the communication terminal device 30-2 have the same configuration, and will be described below as the communication terminal device 30 when no particular distinction is made.

[0016] 2 is a schematic diagram showing an example of the configuration of a robot control system 100. The robot control system 100 includes a robot 2 and a robot controller 110. The robot 2 moves a work target 8 from a work start point 6 to a work target point 7 by an end effector 2B (gripping hand) at the end of an arm 2A. In other words, the robot controller 110 controls the robot 2 so that the work target 8 moves from the work start point 6 to the work target point 7. The robot control system 100 includes a camera 4. The camera 4 captures images of items and the like within an influence range 5 that may affect the operation of the robot 2.

[0017] The robot control device 110 recognizes the work object 8 present in the space where the robot 2 performs the work, based on the image captured by the camera 4. The recognition of the work object 8 is performed using a model included in the above-mentioned model group (a trained model capable of recognizing the work object 8). Before recognizing the work object 8, the robot control device 110 downloads the trained model via the network 40 and deploys it.

[0018] Here, it is preferable that the models provided as a library can recognize various types of work objects 8 so as to meet the requirements of, for example, many users' robot control systems 100. Therefore, it is preferable that the trained model management device 10 updates the trained model, which is a master model included in the model group, so that, for example, a new product can be recognized as an object. Here, when a user wants to include a new product in the recognition targets as soon as possible, for example, the user can prepare learning data for the new product and generate a custom model using the function of the trained model generation device 20. Such a user may want to continue using a custom model that has a proven track record in recognition accuracy even after the master model has been updated. Therefore, it is preferable that the trained model management device 10 allows the user to continue using the custom model in the user's usage environment without forcibly replacing it with the master model.

[0019] As described below, the trained model management device 10 can update the master model so as to expand the recognition target by using the training data used in generating the custom model. Furthermore, the trained model management device 10 confirms an instruction to change the user's custom model to the master model, and then sets the updated master model to be used in the user's usage environment. Details of the master model and the custom model will be described later with reference to FIG. 3. Furthermore, in this embodiment, the communication terminal device 30 is described as a tablet terminal possessed by the user, but it is possible to cause a part of the robot control system 100 used by the user (such as the robot control device 110 and the camera 4) to function as the communication terminal device 30.

[0020] (Configuration of trained model management device) The components of the trained model management device 10 will be described in detail below. The communication unit 11 includes one or more communication modules connected to the network 40. The communication unit 11 may include a communication module compatible with mobile communication standards such as 4G (4th Generation) and 5G (5th Generation). The communication unit 11 may include a communication module compatible with a wired LAN standard (1000BASE-T as an example). The communication unit 11 may include a communication module compatible with a wireless LAN standard (IEEE802.11 as an example).

[0021] The first storage unit 12A and the second storage unit 12B are one or more memories. The memories are, for example, semiconductor memories, magnetic memories, optical memories, etc., but are not limited to these and may be any memories. The first storage unit 12A and the second storage unit 12B are, for example, built into the trained model management device 10, but may also be configured to be accessed from outside the trained model management device 10 via any interface. The first storage unit 12A and the second storage unit 12B may be configured with the same memory, with separate storage areas. Also, the first storage unit 12A and the second storage unit 12B may be configured with physically different memories.

[0022] At least one of the first storage unit 12A and the second storage unit 12B stores various data used in various calculations performed by the control unit 13. In addition, at least one of the first storage unit 12A and the second storage unit 12B may store results and intermediate data of various calculations performed by the control unit 13.

[0023] As described above, the first memory unit 12A and the second memory unit 12B are accessed by the trained model generation device 20. That is, the first memory unit 12A and the second memory unit 12B are shared by the trained model management device 10 and the trained model generation device 20. The first memory unit 12A and the second memory unit 12B also store a model group (multiple models) and the like. The first memory unit 12A stores a master model. The second memory unit 12B stores a custom model.

[0024] The control unit 13 is one or more processors. The processor may be, for example, a general-purpose processor or a dedicated processor specialized for a specific process, but is not limited to these and may be any processor. The control unit 13 controls the overall operation of the trained model management device 10.

[0025] Here, the trained model management device 10 may have the following software configuration. One or more programs used to control the operation of the trained model management device 10 are stored in the first storage unit 12A or the second storage unit 12B. When the program is read by the processor of the control unit 13, it causes the control unit 13 to function as an update determination unit 131, an update processing unit 132, and a detection unit 133.

[0026] When a custom model is generated, the update determination unit 131 determines whether to update the master model based on the learning data. In this embodiment, the update determination unit 131 determines to update the master model when, for example, it is determined that the target to be added has high versatility, that is, when the identification target can be expanded by updating and can be used generically. As another example, when the update determination unit 131 receives an update instruction from a manager of the model group (for example, a supplier of the model), it may determine to update the master model without determining versatility. In addition, the update determination unit 131 may determine to update the master model when, for example, the learning data used to generate the custom model can improve the recognition accuracy of the recognition target.

[0027] The update processing unit 132 executes processing related to updating the master model. More specifically, when the update determination unit 131 determines that the master model should be updated, the update processing unit 132 updates the master model by, for example, having the trained model generation device 20 execute machine learning (re-learning). The machine learning (re-learning) is executed using the learning data (first learning data) used in generating the master model and the learning data (second learning data) used in generating the custom model. In this embodiment, the trained model generation device 20 executes the update of the master model according to the update instruction of the update processing unit 132, but as another example, the update processing unit 132 may execute the update of the master model.

[0028] Furthermore, the update processing unit 132 may execute a task using the updated master model and present the execution result of the task to the user. In this embodiment, the task is object recognition. Based on the presented result, the user can evaluate, for example, precision and recall, and check whether the updated master model is inferior to the custom model.

[0029] Furthermore, when it is determined that the master model is to be updated, the update processing unit 132 may update the data indicating the user's contribution degree stored in the second storage unit 12B. When the second learning data of a certain user is used to update the master model, the user's contribution degree increases (for example, points are added). Depending on the size of the contribution degree, the user's authority to use the master model and custom models can be expanded.

[0030] The detection unit 133 detects whether the user who generated the custom model uses the updated master model instead of the custom model. When the user who generated the custom model performs an operation to create a new usage environment, the detection unit 133 may determine that the updated master model will be used instead of the custom model. The operation to create a new usage environment is, for example, downloading a trained model when adding a new work (recognition of another object) and performing new settings for deployment in the communication terminal device 30. When the user who generated the custom model continues to use the previous usage environment, the detection unit 133 may determine that the user continues to use the custom model.

[0031] (Configuration of communication terminal device) The components of the communication terminal device 30 will be described in detail below. The communication unit 31 includes one or more communication modules that connect to the network 40. The communication unit 31 may include a communication module that supports mobile communication standards such as 4G and 5G. The communication unit 31 may also include a communication module that supports a LAN standard. The communication unit 31 may include the same communication module as the communication unit 11, or may include a different communication module.

[0032] The storage unit 32 is one or more memories. The memory is, for example, a semiconductor memory, a magnetic memory, an optical memory, or the like, but is not limited to these and may be any memory. The storage unit 32 stores various data used in various calculations performed by the control unit 33. The storage unit 32 may also store results and intermediate data of various calculations performed by the control unit 33. For example, the storage unit 32 may temporarily store input information generated by the input information generation unit 34 and transmitted (uploaded) to the trained model management device 10. For example, the storage unit 32 may temporarily store a result of object recognition acquired from the trained model management device 10 via the communication unit 31.

[0033] The control unit 33 is one or more processors. The processor may be, for example, a general-purpose processor or a dedicated processor specialized for a specific process, but is not limited to these and may be any processor. The control unit 33 controls the overall operation of the communication terminal device 30.

[0034] The input information generating unit 34 generates input information. In this embodiment, the input information is an image including an object to be identified. The input information generating unit 34 may be configured to include an imaging unit such as a camera. In this embodiment, the input information generating unit 34 is a camera. For example, a user may take an image including a new object with the camera of the communication terminal device 30 to generate input information. This input information may be evaluation data used to check whether the custom model can recognize the new object.

[0035] The user interface unit 35 inputs information from the user and outputs information to the user. The user interface unit 35 includes, for example, a touch sensor. The touch sensor detects contact with the user's finger or a stylus pen, etc., and identifies the contact position. The touch sensor may be integrated with a display to configure a touch panel display. In this embodiment, the user interface unit 35 is a touch panel display. The content presented by the trained model management device 10 is displayed on the touch panel display. The user interface unit 35 may display a chat screen or the like to prompt input when inputting evaluation data and second training data, etc.

[0036] (Models) In this embodiment, the model groups stored in the first storage unit 12A and the second storage unit 12B are divided as shown in FIG. 3. The model groups are roughly divided into master models and custom models. The master model is, for example, a trained model available to all users. The master model is, for example, a trained model with high versatility. The master model is, for example, a model generated by an administrator of the model group. The custom model is, for example, a trained model generated by a specific user. The custom model may be, for example, a trained model available only to a specific user. The custom model is, for example, superior to the master model in recognizing a specific object or recognizing an object in a specific environment.

[0037] Here, as described above, in this embodiment, the users include the first user and the second user. In the following description, it is assumed that the second user generates a custom model. The master model is a trained model available to the first user and the second user. The custom model generated by the second user may be a trained model available only to the second user. In addition, if a custom model generated by the first user exists, it may be available only to the first user. In this case, the custom model of the first user may be excluded from the library (available models) of the second user. In other words, the library of the second user may be composed of a master model and a custom model generated by the second user. For example, one user (for example, the second user) may generate multiple custom models.

[0038] As shown in Fig. 3, the master model includes a general-purpose model and a specialized model. The general-purpose model is a model for recognizing objects in a large classification (first classification). The specialized model is a model for recognizing objects in a smaller classification (second classification). The second classification is a subdivision of one item of the first classification.

[0039] Also, as described above, the custom model is generated by the second learning data prepared by the user. In detail, the custom model is generated based on the master model and the second learning data. That is, in this embodiment, the custom model is generated by using an existing master model. The custom model may be generated, for example, by performing additional learning on an existing master model using the second learning data. Therefore, the custom model is related to the original master model. Such an association between the master model and the custom model is stored as related information in the first storage unit 12A and the second storage unit 12B. That is, the first storage unit 12A and the second storage unit 12B store related information that defines the relationship between multiple models.

[0040] In the example of FIG. 3, the first classification is industrial parts, stationery, etc., and the second classification is bolts, nuts, pens, scissors, etc. For example, the general-purpose model "industrial parts" is a trained model for recognizing bolts, nuts, springs, etc. For example, the specialized model "bolt" is a trained model for recognizing bolt types such as bolt A, bolt B, bolt C, etc. The specialized models "bolt" and "nut" are associated with the general-purpose model "industrial parts." The associated information may include an association between such general-purpose models and specialized models.

[0041] Here, for example, the second user may generate a custom model using the specialized model "bolt" in order to add "bolt C'", an improved product of "bolt C", to the identification target. In the example of Fig. 3, the second user's custom model "bolt" is a trained model for recognizing types of bolts such as bolt A, bolt B, and bolt C'.

[0042] In addition, if the custom model can be used only by the user who generated it, it is stored separately from the master model. In this embodiment, the custom model can be used only by the user who generated it. Therefore, the first storage unit 12A stores the master model and the first learning data used to generate the master model. Then, the second storage unit 12B stores the custom model and the second learning data used to generate the custom model. In addition, in the second storage unit 12B, the storage area may be divided for each user. As another example, the trained model management device 10 may include a different storage unit for each user. For example, the second storage unit 12B may store only the custom model generated by the second user and may be used only by the second user. In this case, the trained model management device 10 may include a third storage unit that is different from the first storage unit 12A and the second storage unit 12B and is available only to the first user.

[0043] (How to manage trained models) 4 and 5 are sequence diagrams showing an example of the processing procedure in the trained model management system 1. FIG. 5 shows the processing procedure executed following FIG. 4. With reference to FIG. 4 and FIG. 5, the processing of the trained model management method executed by the trained model management device 10 according to this embodiment will be described. As in the above example, a specific example will be described using appropriate specific examples, in which a second user generates a custom model so that "bolt C'" can be recognized from an image including "bolt C'" and other industrial parts.

[0044] The user uses the camera of the communication terminal device 30 to capture images of the "Bolt C'" and other industrial parts, and generates a captured image that is input information (step S1). This captured image becomes evaluation data that is used to evaluate the execution result of a task when a task for recognizing the "Bolt C'" is executed using a custom model or the like. The evaluation data is information on an object that can be recognized using the second learning data prepared by the second user.

[0045] The communication terminal device 30 transmits the input information (evaluation data) to the trained model management device 10 (step S2). The second storage unit 12B of the trained model management device 10 stores the evaluation data.

[0046] The user selects a master model and inputs it to the communication terminal device 30 (step S3). For example, the user may select a master model that has been used up until that point in the robot control system 100. To explain this as a specific example, the second user selects the specialized model "Bolt" that has been used up until that point.

[0047] The communication terminal device 30 transmits the user-selected master model to the trained model management device 10 (step S4). To explain this in a specific example, the communication terminal device 30 transmits information to the trained model management device 10 that the master model selected by the second user is the "bolt" of the specialized model. The first storage unit 12A and the second storage unit 12B store, as association information, the association between the custom model generated by the second user and the "bolt" of the specialized model.

[0048] The user generates second training data for generating a custom model (step S5). The second training data may be created by capturing images of the "Bolt C'" and other industrial parts using the camera of the communication terminal device 30, like the evaluation data. The second training data may also be created using CAD (Computer Aided Design) data of the "Bolt C'". The second training data may also be created by combining training data generated by capturing images and training data generated by CAD.

[0049] The communication terminal device 30 transmits the second learning data to the trained model management device 10 (step S6). The second learning data is stored in the second storage unit 12B of the trained model management device 10.

[0050] Furthermore, the communication terminal device 30 instructs the trained model generation device 20 to generate a custom model (step S7).

[0051] The trained model generation device 20 generates a custom model based on the user-selected master model and the second learning data prepared by the user (step S8). To explain this in a specific example, the trained model generation device 20 uses the "bolt" of the specialized model to perform machine learning using the second learning data stored in the accessible second storage unit 12B to generate a "bolt" of a custom model that can only be used by the second user.

[0052] The trained model generation device 20 stores the generated custom model in the second storage unit 12B of the trained model management device 10 (step S9).

[0053] The update processing unit 132 of the trained model management device 10 executes a task (recognition of an object) by using the evaluation data and the generated custom model (step S10).

[0054] The update processing unit 132 of the learned model management device 10 transmits the execution result of the task to the communication terminal device 30 (step S11). The execution result of the task is displayed on the user interface unit 35 (touch panel display) of the communication terminal device 30.

[0055] The user may evaluate the execution result of the task, and if the evaluation is not as expected, change the second learning data and regenerate the custom model. Also, steps S10 and S11 may be omitted, for example, at the instruction of the user.

[0056] The update determination unit 131 of the trained model management device 10 determines whether or not to update the master model based on the second learning data. In this embodiment, the update determination unit 131 determines to update the master model because the object to be added ("Bolt C'", an improved product of "Bolt C") is highly versatile (step S12). Here, for example, in a case where the shape of "Bolt C'" is closer to a spring than a conventional bolt and it is clear that the matching rate will decrease due to learning, the update determination unit 131 may determine not to update the master model. Also, as described above, the decision may be made by the manager of the model group.

[0057] Here, the update determination unit 131 may identify some of the master models to be updated by learning using the second learning data, among the multiple master models, based on the above update information. Information on the identified part of the master models may be output to the update processing unit 132. As another example, the update determination unit 131 may identify a trained model generated using the first learning data in which an object to be made recognizable by learning (e.g., "Bolt A", "Bolt B", "Bolt C", ...) is identical or similar to the second learning data (e.g., "Bolt C'"), as the master model to be updated. Alternatively, as another example, the update determination unit 131 may identify the master model used when generating the custom model as the master model to be updated. The update processing unit 132 updates the master model, which is the trained model identified by the update determination unit 131.

[0058] When the update determination unit 131 determines that the master model is to be updated, the update processing unit 132 of the trained model management device 10 stores the second learning data in the first storage unit 12A (step S13). That is, since the second learning data is used in updating the master model, the update processing unit 132 copies the second learning data to the first storage unit 12A. When it is determined that the master model is to be updated, the first storage unit 12A stores the second learning data in the same way as the first learning data.

[0059] Furthermore, when it is determined that the master model is to be updated, the update processing unit 132 of the trained model management device 10 updates the data indicating the contribution degree of the second user stored in the second storage unit 12B (step S14). As described above, the contribution degree expands the second user's authority to use the master model and the custom model according to the magnitude. For example, the second storage unit 12B also stores the capacity, usage fee, number of times available, and available period of the second storage unit 12B that the user can use. The update processing unit 132 may set the capacity, usage fee, number of times available, and available period of the second storage unit 12B to be advantageous according to the contribution degree. This provides an incentive for the user to use the second learning data used in generating the custom model to update the master model.

[0060] The update processing unit 132 of the trained model management device 10 instructs the trained model generation device 20 to update the master model (step S15).

[0061] The trained model generation device 20 updates the master model by learning (relearning) using the first learning data and the second learning data stored in the first storage unit 12A (step S16). To explain this in a specific example, the trained model generation device 20 re-learns the specialized model "bolt" so that "bolt C'" can also be identified. Since the specialized model "bolt" is the master model, it can be used not only by the second user but also by other users including the first user.

[0062] The trained model generation device 20 stores the updated master model in the first storage unit 12A of the trained model management device 10 (step S17).

[0063] The update processing unit 132 of the trained model management device 10 executes a task (recognition of an object) by using the evaluation data and the updated master model (step S18).

[0064] The update processing unit 132 of the learned model management device 10 transmits the execution result of the task to the communication terminal device 30 (step S19). The execution result of the task is displayed on the user interface unit 35 (touch panel display) of the communication terminal device 30.

[0065] The user may evaluate the task execution result, and if the evaluation is not as expected, may continue to use the custom model without using the updated master model. If the user decides to use the updated master model based on the evaluation, the user may download the updated master model, for example, by performing an operation to create a new usage environment. When the user performs such an operation, the communication terminal device 30 transmits the information on the use of the updated master model to the trained model management device 10 (step S20). This information is detected by the detection unit 133 of the trained model management device 10.

[0066] Here, the user may store the input information input by the robot 2 using the master model or the custom model in the first storage unit 12A or the second storage unit 12B, and use it to evaluate the updated master model. In addition, the user may store data for confirming or evaluating the learning effect used when generating a past master model or a custom model in the first storage unit 12A or the second storage unit 12B, and use it to evaluate the updated master model. In addition, the evaluation of the updated master model is not limited to being performed by the user, and may be automatically performed by, for example, the trained model management device 10.

[0067] Incidentally, even when or after updating the master model, the master model before the update may be stored in the first storage unit 12A. As a result, even if there is a defect in the updated master model, it is possible to make the robot 2 perform a task or execute the update process again using the master model before the update.

[0068] When a user uses the updated master model, the update processing unit 132 of the trained model management device 10 deletes the second learning data stored in the second storage unit 12B (step S21). By deleting the second learning data, which is relatively large in size, the free space in the second storage unit 12B increases, and the user can reduce, for example, charges according to the capacity used. In addition, the trained model management device 10 can eliminate the increase in capacity caused by the presence of a large amount of learning data for generating a custom model.

[0069] Here, even when the second user uses the updated master model instead of the custom model, the update processing unit 132 does not delete the evaluation data stored in the second storage unit 12B. This is because the evaluation data is relatively small in size and is more likely to be used by the user again than the second learning data.

[0070] As described above, the trained model management device 10 and trained model management method according to this embodiment can update the master model without degrading the recognition accuracy in the user's usage environment.

[0071] In the above, an example in which the user determines whether to update the master model when generating a custom model has been described, but the present invention is not limited to this. That is, for example, when the user uses the master model at a site such as the user's factory, the recognition accuracy of the master model may be reduced because the creation environment of the master model is different from the user's usage environment. At this time, in order to improve the accuracy of the master model, an optimized master model (second model or second master model) may be generated based on the master model (first model or first master model) using learning data acquired in the user's usage environment or learning data simulating the user's usage environment. That is, in the present invention, when the second master model is generated, the update determination unit 131 may determine whether to update the first master model based on the learning data used to learn the second master model. The second master model may be generated by connecting an additional learned model that has been subjected to a learning process regarding the user's specification environment to the first master model. The additional learned model is also called, for example, an adapter module.

[0072] In addition, the second master model is a model that has no restrictions on the user's use, and is stored in the first storage unit 12A in the same manner as the first master model. In addition, the second master model can also be called a custom model that has no restrictions on the user's use. Therefore, in the present disclosure, the custom model in the above embodiment may be read as the second master model within a range that is not inconsistent, and may be applied to the present invention.

[0073] Although the embodiments of the trained model management device 10 and the trained model management method have been described above, the present disclosure may also be embodied as a system, a program, or a storage medium on which a program is recorded. Examples of the storage medium include an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a CD-RW, a magnetic tape, a hard disk, or a memory card.

[0074] Furthermore, the implementation form of the program is not limited to application programs such as object code compiled by a compiler or program code executed by an interpreter, but may be in the form of a program module incorporated into an operating system, etc. Furthermore, the program may or may not be configured so that all processing is performed only by the CPU on the control board. The program may be configured so that a part or all of it is executed by another processing unit mounted on an expansion board or expansion unit added to the board as necessary.

[0075] Although the embodiments of the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art can make various modifications or alterations based on the present disclosure. Therefore, it should be noted that these modifications or alterations are included in the scope of the present disclosure. For example, the functions, etc. included in each component, etc. can be rearranged so as not to cause logical inconsistencies, and multiple components, etc. can be combined into one or divided.

[0076] The constituent elements and process steps described in this disclosure may be combined in any combination except those in which the features are mutually exclusive. Furthermore, each feature described in this disclosure may be replaced by an alternative feature serving the same, equivalent or similar purpose, unless expressly denied. Thus, unless expressly denied, each disclosed feature is merely one example of a generic series of the same or equivalent features.

[0077] Furthermore, the embodiments of the present disclosure are not limited to any of the specific configurations of the embodiments described above, but rather extend to any novel feature or combination of features described herein, or any novel method or process step or combination of features described herein.

[0078] In this disclosure, descriptions such as "first" and "second" are identifiers for distinguishing the configuration. In the configurations distinguished by descriptions such as "first" and "second" in this disclosure, the numbers in the configurations can be exchanged. For example, the first classification can exchange the identifiers "first" and "second" with the second classification. The identifiers are exchanged simultaneously. The configurations are still distinguished after the identifiers are exchanged. The identifiers may be deleted. The configurations from which the identifiers are deleted are distinguished by the symbols. The descriptions of identifiers such as "first" and "second" in this disclosure should not be used solely to interpret the order of the configurations or to justify the existence of identifiers with smaller numbers. For example, the identifiers "first" and "second" exchanged in the claims, such as the first model and the second model, are also included in the scope of the present invention. [Explanation of symbols]

[0079] 1. Trained model management system 2. Robot 2A Arm 2B End effector 4. Camera 5. Scope of impact 6. Starting point 7. Work target point 8 Work Object 10 Trained model management device 11 Communications Department 12A 1st memory section 12B 2nd memory section 13 Control section 20 Trained model generator 30 Communication terminal equipment 31 Communications Department 32 Storage section 33 Control Unit 34 Input information generation unit 35 User Interface Section 40 Network 100 Robot Control System 110 Robot control device 131 Update determination section 132 Update processing section 133 Detection Unit

Claims

1. A control unit is provided, The control unit generates a second model based on a first model that has been trained to be able to recognize an object and training data, and determines whether to update the first model based on the training data.

2. A trained model management device as described in claim 1, wherein the training data is data containing objects that cannot be recognized by the first model or data acquired in a user environment.

3. A trained model management device as described in claim 1, wherein the control unit evaluates the results of having the first model and the second model recognize the object, and determines whether to update the first model based on the evaluation.

4. A learned model management device as described in claim 1, wherein the control unit updates the first model when it is determined that the object added by the learning data has high versatility.

5. A trained model management device as described in claim 1, wherein the control unit generates a plurality of the second models specialized in at least one of recognizing the object specific to a user or recognizing the object in the user's recognition environment.

6. A trained model management device as described in claim 1, wherein the control unit can continue to use the second model based on input from a user regarding the recognition results of the object using the updated first model.

7. A trained model management device as described in claim 1, wherein the control unit stores the first model before updating when the first model is updated.

8. A robot control system comprising: a robot control device that controls the operation of a robot based on the results of recognizing the object using the first model or the second model managed by a learned model management device described in claims 1 to 7; and a robot controlled by the robot control device.

9. A method for managing trained models, which generates a second model based on a first model that has been trained to be able to recognize an object and training data, and determines whether to update the first model based on the training data.

Images