Learning model generation device, learning model generation method and learning model generation program

Abstract

The learning model generation apparatus of the present invention comprises: a model information display unit that can selectively display a plurality of learning models publicly available on the Internet and having the same interface; a model selection unit that selects at least one of the displayed learning models; and a learning unit that enables the selected learning model to perform machine learning on learning data including image data to generate a learned learning model.

Description

Technical Field

[0001] This invention relates to a learning model generation apparatus, a learning model generation method, and a learning model generation program, particularly to a learning model generation apparatus for performing machine learning on a learning model, a learning model generation method executed by the learning model generation apparatus, and a learning model generation program that causes a computer to execute the learning model generation method. Prior Technology

[0002] In the field of AI (Artificial Intelligence), learning models such as neural networks perform machine learning on a learning dataset to generate a learned model. For example, Japanese Patent Application Publication No. 2021-047104 discloses a learning device that generates a learning dataset and, through machine learning using the learning dataset, learns the relationship between a substrate image and the presence or absence of defects to create a learned model. Thus, to generate a learned model, it is necessary to prepare a learning dataset and a learning model that performs machine learning on the learning dataset. Summary of the Invention

[0003] Generally, learning models are built based on machine learning algorithms and learn by providing a learning dataset. Therefore, to prepare a learning model suitable for predicting objects using the learned model, the program to be processed needs to be coded according to the machine learning algorithm. Furthermore, while it is possible to generate a learned model using an existing learning model, this requires adapting the existing learning model to the learning dataset to be learned, which also necessitates coding the program. Thus, generating a learned model requires knowledge related to the learning model and time for coding the program. Therefore, it is difficult for someone without specialized knowledge to perform machine learning using the learning model, as the coding process is time-consuming.

[0004] The purpose of this invention is to provide a learning model generation device that can easily generate learned learning models.

[0005] Another objective of this invention is to provide a learning model generation method that can easily generate learned learning models.

[0006] Another objective of this invention is to provide a learning model generation program that can easily generate learned learning models.

[0007] (1) A learning model generating apparatus according to one aspect of the present invention comprises: a model information display unit that can selectively display a plurality of learning models publicly available on the Internet and having the same interface; a model selection unit that selects at least one of the displayed learning models; and a learning unit that causes the selected learning model to perform machine learning on learning data to generate a learned model.

[0008] (2) According to another aspect of the present invention, the learning model generating device performs the following steps: a model information display step, which selectively displays a plurality of learning models publicly available on the Internet and with the same interface; a model selection step, which selects at least one of the learning models from the plurality of displayed learning models; and a learning step, which causes the selected learning model to perform machine learning on the learning data and generate a learned model.

[0009] (3) According to another aspect of the present invention, the learning model generation program causes the computer to perform the following steps: a model information display step, which selectively displays a plurality of learning models publicly available on the Internet and with the same interface; a model selection step, which selects at least one of the learning models from the plurality of displayed learning models; and a learning step, which causes the selected learning model to perform machine learning on the learning data and generate a learned model.

[0010] It can easily generate learned models. Simple Explanation of the Diagram

[0011] Figure 1 is an example of an overall overview of a learning model generation system in one of the embodiments of this invention. Figure 2 shows an example of the configuration of a learning model generation device. Figure 3 is a diagram showing an example of the functional configuration of a learning model generating device in one of the embodiments of this invention. Figure 4 is a flowchart showing one example of the process of generating and processing a learning model. Figure 5 shows an example of a menu screen. Figure 6 is a flowchart showing one example of the process of generating and processing learning materials. Figure 7 shows an example of a screen generated by learning materials. Figure 8 shows an example of a data expansion settings screen. Figure 9 shows an example of an image classification setting screen. Figure 10 shows an example of an object recognition setting screen. Figure 11 shows an example of a screen displaying area classification settings. Figure 12 is a flowchart showing one example of the process of obtaining and processing a learning model. Figure 13 shows an example of the learning model selection screen. Figure 14 shows an example of a screen obtained by an external learning model. Figure 15 is a flowchart showing one example of the machine learning process. Figure 16 shows an example of a hyperparameter setting screen. Figure 17 shows an example of a screen for selecting learning materials. Figure 18 is a flowchart showing one example of the process for evaluating a learned model. Figure 19 shows an example of the evaluation screen of a learned model. Figure 20 is a flowchart showing one example of the process for comparing learned models. Figure 21 shows an example of a comparison screen of learned models. Implementation

[0012] The following description uses drawings to illustrate a learning model generation system according to one embodiment of the present invention. In the following description, identical parts are labeled with the same symbols. Their names and functions are also the same. Therefore, detailed descriptions of them will not be repeated.

[0013] (1) The structure of the learning model generation system Figure 1 is an example of an overall overview of a learning model generation system according to one embodiment of the present invention. Referring to Figure 1, the learning model generation system 1 includes a learning model generation device 100 and a server 200. The learning model generation device 100 is a general computer, such as a personal computer (PC).

[0014] The learning model generation device is connected to network 3. Network 3 is a local area network (LAN), and the connection method is not limited to wired or wireless. Furthermore, network 3 is not limited to LAN, but can also be a wide area network (WAN), a public switched telephone network (PSTN), the Internet, etc.

[0015] Network 3 is connected to Internet 5. Server 200 is connected to Internet 5. Therefore, the learning model generation device can communicate with server 200 via Network 3 and Internet 5. Furthermore, in this embodiment, the case where server 200 is connected to the Internet is used as an example for explanation, but server 200 only needs to be able to communicate with the learning model generation device, and can also be directly connected to Network 3.

[0016] Server 200 is a general-purpose computer. Its structure and functions are well-known, so they will not be described again here. Server 200 stores multiple existing learning models and exposes these learning models to the Internet 5. The learning models are deep learning networks. Server 200 exposes a library containing multiple learning models to the Internet 5. The URL (Uniform Resource Locator) of the library is exposed, making it accessible to everyone. The library may be, for example, a vendor software repository, i.e., a Python package index. Here, we will use timm (pytorch-image-models) or PyTorch Hub as an example of a library. The learning models stored in timm or PyTorch Hub in the library are deep learning networks based on PyTorch, a machine learning library described in the programming language Python.

[0017] In this embodiment, the learning model generation system 1 is shown to have one learning model generation device and one server 200. However, the number of learning model generation devices and servers 200 is not limited. The learning model generation system 1 can have one or more of each.

[0018] Figure 2 shows an example of the configuration of a learning model generation device. Referring to Figure 2, the learning model generation device 100 consists of a CPU (Central Processing Unit) 151, RAM (Random Access Memory) 152, ROM (Read Only Memory) 153, a memory device 154, an operation unit 155, a display device 156, and an input / output (I / F) 157. The CPU 151, RAM 152, ROM 153, memory device 154, operation unit 155, display device 156, and I / F 157 are connected to a bus 158.

[0019] RAM 152 is used as the operating area of ​​CPU 151. System programs are stored in ROM 153. Memory device 154 includes recording media such as hard disks and semiconductor memory, storing various programs and data. A CD-ROM (Compact Disc-ROM) 159, a recording medium for recording programs, can be mounted and removed from memory device 154. Operation unit 155 includes input devices such as a keyboard, mouse, or touch panel. Input / output interface (I / F) 157 is connected to a network, connecting CPU 151 to other computers connected to the network. Display device 156 displays various images. In this embodiment, display device 156 displays multiple screens sequentially for receiving selections of the action content of learning model generation device 100.

[0020] CPU 151 loads and executes the program stored in ROM 153 or memory device 154 in RAM 152. By executing the program stored in ROM 153 or memory device 154, CPU 151 controls display device 156 and operation unit 155 to provide a graphical user interface (GUI) to the user.

[0021] Furthermore, the example shown illustrates CPU 151 executing a program stored in memory device 154. However, the program stored in memory device 154 may include programs downloaded by CPU 151 from other computers connected to the network via the network connected to input / output I / O 157. Additionally, the program stored in memory device 154 may include programs that are rewritten by other computers connected to the network.

[0022] Figure 3 is a diagram showing an example of the functional configuration of a learning model generating device according to one embodiment of the present invention. The function shown in Figure 3 is realized by the CPU 151 of the learning model generating device 100 executing a learning model generating program stored in ROM 153 or CD-ROM 159.

[0023] The learning model generation device 100 includes a model information acquisition unit 101, a model information display unit 103, a model selection unit 105, a model acquisition unit 107, a parameter modification unit 109, a learning unit 111, an image data acquisition unit 113, a learning data generation unit 115, an evaluation data acquisition unit 117, a prediction unit 119, an evaluation unit 121, and an evaluation result display unit 123.

[0024] The model information acquisition unit 101 acquires model information related to learning models publicly available on the Internet. Here, the memory device 154 pre-memorizes one or more network addresses, which represent the Internet locations of one or more library areas publicly available on the Internet 5. The one or more library areas publicly available on the Internet 5 may include, for example, the library area possessed by the server 200.

[0025] For each of the more than one library region, model information is stored for each of the more than one learning models stored in that library region. The model information includes model identification information used to identify the learning model, and descriptive information related to the learning model.

[0026] The model information acquisition unit 101 displays URLs of one or more library areas stored in the memory device 154 on the display device 156, and selects a URL for a library area according to the operation input by the user to the operation unit 155. The model information acquisition unit 101 selects one URL from the one or more URLs displayed on the display device 156 according to the user's operation, and controls the input / output I / F 157 to access the URL selected by the user. The model information acquisition unit 101 acquires model identification information of one or more learning models stored in the library area specified by the URL, and displays the model identification information on the display device 156. Furthermore, when the user instructs the display device 156 to display model identification information, the model information acquisition unit 101 downloads the model information of the learning model for which the instruction is made. The model information acquisition unit 101 outputs the downloaded model information to the model information display unit 103. Alternatively, the model information can also be stored in an area different from the library area. For example, model information can also be stored on a computer different from server 300.

[0027] The model information display unit 103 displays one or more model information pieces input from the model information acquisition unit 101 on the display device 156. Specifically, the model information display unit 103 generates a model overview screen that arranges one or more model identification information pieces, each containing one or more model information pieces, and displays it on the display device 156. The model overview screen is a screen that can selectively display one or more model identification information pieces.

[0028] Furthermore, the model information display unit 103 generates an explanatory screen based on the explanatory information contained in the model information and displays the explanatory screen on the display device 156. The model information acquisition unit 101, according to the operation input by the user to the operation unit 155, selects one model identification information from the more than one model identification information contained in the model overview screen displayed on the display device 156. Based on the selected model identification information according to the user's operation, the model information acquisition unit 101 generates an explanatory screen containing explanatory information that is associated with the selected model identification information through the model information, and displays the explanatory screen on the display device 156. The user viewing the explanatory screen can confirm the characteristics of the learning model, such as the algorithm characteristics, specific to the learning model selected from the model identification information in the model overview screen. Therefore, information can be provided that helps to select a target learning model from the more than one model identification information contained in the model overview screen.

[0029] The model selection unit 105 freely displays model identification information from one or more model overview screens on the display device 156, and selects one model from one or more specific learning models as the learning object. The model selection unit 105 determines the learning object according to the operation input by the user to the operation unit 155. The model selection unit 105 outputs the model identification information of the learning model determined as the learning object to the model acquisition unit 107.

[0030] The model acquisition unit 107 acquires a learning model based on the model recognition information input from the model selection unit 105. The model acquisition unit 107 controls the input / output I / F 157 to access the URL of the library area containing the learning model with the model recognition information input from the model selection unit 105, and downloads the learning model specific to the model recognition information input from the model selection unit 105. The model acquisition unit 107 outputs the downloaded learning model to the learning unit 111.

[0031] Image data acquisition unit 113 acquires a plurality of image data. The image data acquired by image data acquisition unit 113 is used to enable the learning model acquired by model acquisition unit 107 to perform machine learning. The image data may be image data pre-memorized in memory device 154, or image data memorized in other recording media such as CD-ROM. Alternatively, the image data may be data published on the Internet 5. In this case, image data acquisition unit 113 controls input / output I / F 157 to download the image data published on the Internet 5. Image data acquisition unit 113 outputs the acquired plurality of image data to learning data generation unit 115.

[0032] For the learning data generation unit 115, a plurality of image data are input from the image data acquisition unit 113. The learning data generation unit 115 generates one or more learning data based on the image data. The learning data generation unit 115 outputs a portion of the learning data generated from the plurality of image data to the learning unit 111, and outputs another portion of the learning data to the evaluation data acquisition unit 117. The learning data generation unit 115 includes a data expansion unit 115a and an annotation unit 115b.

[0033] The data expansion unit 115a performs augmentation processing according to the operation input by the user to the operation unit 155. Augmentation processing is the processing performed on image data, including magnification, reduction, inversion, rotation, parallel translation, blackening, and homography transformation. By performing augmentation processing on one image data, the data expansion unit 115a generates one or more other image data. These other image data generated by the data expansion unit 115a are processed in the same way as the image data that was the object of the augmentation processing. In other words, the other image data generated by the data expansion unit 115a becomes part of the learning data.

[0034] Enlargement processing generates image data that enlarges the target object contained within the image data. Reduction processing generates image data that reduces the target object contained within the image data. Inversion processing generates image data that inverts the orientation of the target object contained within the image data. The orientation of the target object includes vertical and horizontal directions. Rotation processing generates image data that rotates the target object contained within the image data. The rotation angle includes a predetermined angle or an angle specified by the user. Parallel movement processing generates image data that moves the target object contained within the image data in parallel. The number and direction of parallel movement include a predetermined number and direction of pixels, as well as a number and direction of pixels specified by the user. Blackening processing refers to the process of transforming the pixel values ​​of useless parts of the image data other than the target object into values ​​that appear black. Homography transformation processing generates image data that uses a homography matrix to transform the target object contained in the image data into another target object. The homography transformation process transforms the target object contained in the image data before transformation into a shape that is the shape of the target object as the subject, taken from a position different from the position where the target object contained in the image data before transformation was photographed. The data expansion unit 115a stores the image data after the amplification process as new image data different from the original image data in the memory device 154.

[0035] The annotation unit 115b performs annotation processing on the image data according to the operation input by the user to the operation unit 155. The image data processed by the annotation unit 115b includes a plurality of image data input from the image data acquisition unit 113 and a plurality of image data generated by the data expansion unit 115a.

[0036] Annotation processing involves adding metadata to image data. Here, metadata includes image category information and label information. Image category information indicates the type of image data, including image classification representing the image data itself, object identification representing a rectangular region containing a target object, and region separation representing the outline of the target object.

[0037] The annotation unit 115b displays the image of the image data selected by the user on the display device 156 according to the operation input by the user to the operation unit 155. The annotation unit 115b accepts image type information input by the user to the operation unit 155 and appends the accepted image type information to the image data. Furthermore, the annotation unit 115b accepts tag information input by the user to the operation unit 155 and appends the accepted tag information to the image data. The tag information is the target variable in supervised learning. For example, when the learning model learns to identify the target object contained in the image data, the tag information is used to identify the target object. More specifically, when the target object is either a dog or a cat, the tag information represents the value of either a dog or a cat.

[0038] When the annotation unit 115b receives object recognition values ​​as image type information, it adds object recognition information to the image displayed on the display device 156 according to the operation input by the user to the operation unit 155. The operation input by the user to the operation unit 155 is an operation on a specific rectangular area in the image displayed on the display device 156. The operation on a specific rectangular area is, for example, an operation on specifying the coordinates of the two diagonals of the rectangular area. The annotation unit 115b adds information representing the rectangular area specified by the user in the image displayed on the display device 156 as object recognition information to the image data.

[0039] When the value of the received area separation is used as image type information, the annotation unit 115b adds area separation information to the image displayed on the display device 156 according to the operation input by the user to the operation unit 155. The operation input by the user to the operation unit 155 is the operation of defining the outline of a specific target object in the image displayed on the display device 156. The operation of defining the outline of a specific target object is, for example, the operation of specifying a plurality of consecutive coordinates of the outline of the target object. The annotation unit 115b adds information representing the area enclosed by the outline connecting two adjacent coordinates based on the plurality of coordinates specified by the user in the image displayed on the display device 156 as object recognition information to the image data.

[0040] When the annotation unit 115b receives the image classification value as image type information, it adds image classification information to the image displayed on the model information display unit 103. The image classification information indicates that it is not either object identification information or region separation information. When the annotation unit 115b receives the image classification value as image type information, it adds the image classification information to the image data.

[0041] The parameter modification unit 109 sets hyperparameters according to the operation input by the user to the operation unit 155. Hyperparameters are values ​​used to control the behavior of various algorithms determined by the learning model. Here, hyperparameters include the number of learning iterations, patch size, optimization function, and learning rate. Furthermore, hyperparameters may include the number of layers in the neural network, the number of neurons in each layer, kernel size, etc. The parameter modification unit 109 outputs the hyperparameters set by the user to the learning unit 111.

[0042] For the learning unit 111, a learning model is input from the model acquisition unit 107, hyperparameters are input from the parameter modification unit 109, and a plurality of learning data are input from the learning data generation unit 115. The learning unit 111 performs machine learning on the plurality of learning data according to the hyperparameters. Specifically, the learning unit 111 sets hyperparameters for the learning model, and the learning model performs machine learning on the plurality of learning data according to the hyperparameters. The image data contained in the learning data is provided to the learning model as input data, so that the model uses the values ​​of the label information contained in the learning data as forward retrieval data for machine learning. Machine learning, for example, uses methods such as error backpropagation to optimize the weight coefficients of the neural network constituting the learning model in a way that minimizes the error with the forward retrieval data.

[0043] Here, the unit of processing multiple training data for the learning model to mechanically learn is called an epoch. The number of learning iterations included in the hyperparameters indicates the upper limit of the number of times the epoch training is repeated. The learning unit 111 processes multiple training data for the learning model to mechanically learn, and repeats the epoch training the number of times determined by the hyperparameters. At the end of each epoch training, the learning unit 111 outputs the learned learning model at that point in time to the prediction unit 119. Therefore, the learned learning model is defined by the number of epoch training repetitions. In other words, the learning unit 111 outputs multiple learned learning models with different number of epoch training repetitions to the prediction unit 119. Hereinafter, the learned learning model with N epoch training repetitions (N is a positive integer) is referred to as the learned model with N epoch training repetitions.

[0044] For the evaluation data acquisition unit 117, a plurality of learning data are input from the learning data generation unit 115. The plurality of learning data input to the evaluation data acquisition unit 117 is different from the plurality of learning data input to the learning unit 111. The evaluation data acquisition unit 117 outputs the plurality of learning data input from the learning data generation unit 115 as evaluation data to the prediction unit 119 and the evaluation unit 121.

[0045] For the prediction unit 119, the self-learning unit 111 receives a plurality of learned models with different numbers of repetitions in a single training round, and the self-evaluation data acquisition unit 117 receives a plurality of evaluation data. The prediction unit 119 provides evaluation data to the learned models and obtains predicted values ​​inferred by the learned models. The prediction unit 119 obtains one predicted value for each piece of evaluation data. Therefore, the prediction unit 119 obtains a plurality of predicted values ​​corresponding to the plurality of evaluation data.

[0046] Furthermore, for multiple learned models with different repetition counts in a single training round, the prediction unit 119 provides evaluation data for each learned model and obtains predicted values ​​inferred from the learned models. Therefore, for a learned model with N repetition counts in a single training round, the prediction unit 119 obtains multiple predicted values. The prediction unit 119 outputs multiple predicted values ​​to the evaluation unit 121 for each repetition count in a single training round. In other words, the prediction unit 119 outputs a combination of the repetition count N in a single training round and the multiple predicted values ​​to the evaluation unit 121.

[0047] For the evaluation unit 121, the prediction unit 119 inputs a group of the number of repetitions N of the single-round training, which is the number of times the single-round training is repeated, and a plurality of predicted values. The self-evaluation data acquisition unit 117 inputs a plurality of evaluation data. The evaluation unit 121 evaluates the learned model for each number of repetitions of the single-round training. Here, the evaluation of the learned model for the number of repetitions N of the single-round training is explained. The evaluation unit 121 compares the predicted value corresponding to the learned model for the number of repetitions N of the single-round training with the value of the label information (correct solution) contained in the evaluation data input by the self-evaluation data acquisition unit 117. The predicted value is the value predicted by the learned model from the self-evaluation data. Therefore, one predicted value corresponds to one evaluation data. If the predicted value is the same as the value of the label information, the evaluation unit 121 judges the predicted value as a correct solution; if the predicted value is different from the value of the label information, the evaluation unit 121 judges the predicted value as a incorrect solution. Evaluation unit 121 compares multiple evaluation data with the predicted values ​​corresponding to those evaluation data, and generates evaluation indices as performance data through statistical processing. The performance data includes accuracy, reproducibility, and fit as evaluation indices. The evaluation index representing accuracy is obtained by dividing the number of correct solutions by the total number of predictions, as expressed by equation (1). The evaluation index representing reproducibility is obtained by dividing the number of correct solutions by the total number of correct solutions, as expressed by equation (2). The evaluation index representing fit is expressed by equation (3). Accuracy = (TP + TN) / (TP + TN + FP + FN) …(1) Reproducibility = TP / (TP+FN)…(2) Fit rate = TP / (TP+FP)…(3) Among them, TP (True Positive) is the number of correctly predicted solutions, TN (True Negative) is the number of correctly predicted non-solutions, FP (False Positive) is the number of non-solutions incorrectly predicted as solutions, and FN (False Negative) is the number of solutions incorrectly predicted as non-solutions.

[0048] Here, the evaluation unit 121 calculates the performance data of the learned model for the number of repetitions N in a single training round. The evaluation unit 121 outputs the performance data of the learned model for the number of repetitions N in a single training round to the evaluation result display unit 123.

[0049] For the evaluation result display unit 123, for each of the multiple learned models with different repetition numbers in a single training round, the performance data of the learned model with N repetition numbers in a single training round is input. For each of the multiple learned models with different repetition numbers in a single training round, the evaluation result display unit 123 generates an evaluation result screen representing the performance data and displays the generated evaluation result screen on the display device 156.

[0050] When the learning unit 111 generates a plurality of learned models, the evaluation result display unit 123 generates a model comparison screen that compares the plurality of learned models and displays the generated model comparison screen on the display device 156.

[0051] (2) A series of processes of learning the operation of the model generation device. Figure 4 is a flowchart illustrating one example of the learning model generation process. The learning model generation process is performed by the CPU 151 of the learning model generation device 100 executing a learning model generation program stored in RAM 152 or CD-ROM. Referring to Figure 4, the CPU 151 of the learning model generation device 100 causes the display device 156 to display the menu screen stored in the memory device 154, and advances the process to step S02 (step S01).

[0052] Figure 5 shows an example of a menu screen. Referring to Figure 5, the menu screen 510 includes buttons 511-515. Buttons 511-515 are used to receive user instructions. When the menu screen is displayed on the display device 156, the CPU 151 receives user instructions based on the user's input to the operation unit 155. When the CPU 151 receives the user's instruction to button 511, it receives an instruction to generate learning data; when it receives the user's instruction to button 512, it receives an instruction to acquire a learning model; when it receives the user's instruction to button 513, it receives a machine learning instruction; when it receives the user's instruction to button 514, it receives a learned model evaluation instruction; and when it receives the user's instruction to button 515, it receives a learned model comparison instruction.

[0053] The menu screen 510 then includes a button 517 labeled with the text string "Restore Process". When the user instructs the button 517, the CPU 151 restores the operation entered by the user when selecting learning materials in the past. In this way, the user can easily restore past instructions.

[0054] Returning to Figure 4, in step S02, CPU 151 branches the processing based on user instructions. When receiving an instruction to generate learning data, processing proceeds to step S03; when receiving an instruction to acquire a learning model, processing proceeds to step S04; when receiving a machine learning instruction, processing proceeds to step S05; when receiving an instruction to evaluate a learned model, processing proceeds to step S06; when receiving an instruction to compare learned models, processing proceeds to step S07. In step S03, the learning data generation process is executed, and processing returns to step S01. In step S04, the learning model acquisition process is executed, and processing returns to step S01. In step S05, the machine learning process is executed, and processing returns to step S01. In step S06, the learned model evaluation process is executed, and processing returns to step S01. In step S07, the learned model comparison process is executed, and processing returns to step S01. Details regarding the processing of learning data, the acquisition of learning models, machine learning, evaluation of learned models, and comparison of learned models will be described later.

[0055] Figure 6 is a flowchart showing an example of the process for generating and processing learning data. The processing for generating and processing learning data is performed by CPU 151 in step S03 of the learning model generation and processing.

[0056] Referring to Figure 6, CPU 151 acquires image data (step S11) and proceeds to step S12. The image data is used to perform machine learning on a learning model that will become the object of machine learning in the machine learning process described later. The image data may be image data pre-stored in memory device 154, or image data stored on other recording media such as CD-ROM. Alternatively, the image data may be publicly available data on the Internet. In this case, CPU 151 controls input / output I / F 157 to download image data stored on the Internet in an area specified by the user. CPU 151 assigns a name to the image data to identify it and stores it in memory device 154. Furthermore, CPU 151 stores a collection of multiple image data sets as a single data group in memory device 154. The image data is associated with a data group name used to identify the data group to which it belongs.

[0057] In step S12, it is determined whether an annotation is indicated. When the operation unit 155 receives an instruction to perform annotation processing, the CPU 151 determines that an annotation is indicated. If an annotation is indicated, the process proceeds to step S13; otherwise, the process enters a standby state. In step S13, an image data to be processed is selected, and the process proceeds to step S14. One image data to be processed is selected from a plurality of image data stored in the memory device 154. The image data is selected by the user through operation unit 155.

[0058] In step S14, the metadata is determined, and processing proceeds to step S15. Details regarding the processing of the determined metadata will be described later. The metadata includes tag information (correct answer).

[0059] In step S15, it is determined whether data amplification is indicated. When the operation unit 155 receives an instruction to perform data amplification processing, the CPU 151 determines that data amplification is indicated. If data amplification is indicated, the process proceeds to step S16; otherwise, the process proceeds to step S19. In step S16, the setting value for data amplification processing is set, and the process proceeds to step S17. In step S17, data amplification processing is performed, and the process proceeds to step S18. Details of data amplification processing will be described later; it is the process of generating one or more new image data from one image data. In step S18, the generated new image data is stored in the memory device 154.

[0060] In step S19, learning data is generated, and the process proceeds to step S20. When step S19 is executed after step S15, the learning data generated in step S19 is data that associates the metadata determined in step S14 with the image data selected in step S13. When step S19 is executed after step S18, the learning data generated in step S19 includes, in addition to the data that associates the metadata determined in step S14 with the image data selected in step S13, data that associates the metadata determined in step S14 with the image data generated in step S17. In other words, learning data containing the image data selected in step S13 and more than one type of learning data each containing one or more image data amplified in step S17 are generated. In step S20, the learning data is stored in the memory device 154, and the process proceeds to step S21.

[0061] In step S21, it is determined whether an end instruction has been accepted. When the operation unit 155 accepts an instruction to end the operation, the CPU 151 determines that an end instruction has been accepted. If an end instruction has been accepted, the process returns to the learning model generation process; otherwise, the process returns to step S13.

[0062] Figure 7 shows an example of a learning data generation screen. Referring to Figure 7, the learning data generation screen 530 includes a button 531 labeled with a text string indicating data augmentation processing and a button 532 labeled with a text string indicating annotation processing. Button 532 includes a button 532a labeled with a text string indicating image classification, a button 532b labeled with a text string indicating object recognition, and a button 532c labeled with a text string indicating region separation. Button 531 is used to receive data augmentation instructions, and buttons 532a, 532b, and 532c are used to receive annotation instructions.

[0063] Figure 8 shows an example of a data augmentation setting screen. The data augmentation setting screen 540 in Figure 8 is displayed on the display device 156 based on the user's instruction to press button 531 on the learning data generation screen 530. Furthermore, the data augmentation setting screen 540 is displayed on the display device 156 when the user sets the settings for performing data augmentation processing. Referring to Figure 8, the data augmentation setting screen 540 includes an area 542 displaying the categories of setting values ​​that are the objects to be set, and an area 543 displaying the setting values ​​for the set items. In area 542, categories for zooming in / out, inverting, rotating, parallel movement, blacking out, and homography transformation are displayed in selectable states. Checkboxes are arranged to the left of each category. When the user selects any of the multiple categories, a mark indicating that the selected object is being processed is displayed on the checkbox to the left of that category. Here, the zooming in / out category is selected as the object to be processed.

[0064] The zoom-in / zoom-out category displays the zoom-in or zoom-out processing of image data containing objects. The invert category displays the inverted processing of image data containing objects, reversing their orientation. The rotate category displays the rotated processing of image data containing objects. The parallel movement category displays the parallel movement processing of image data containing objects. The blackening category displays the blackening processing of image data containing useless pixels other than the object, transforming them to black values. The homography transformation category displays the homography transformation processing of image data containing objects, transforming them into other objects using a homography matrix.

[0065] In area 543, setting values ​​for one or more settings items belonging to the category selected as the processing object in area 542 are displayed. When a category has been selected in area 542, default settings are set for one or more settings items belonging to that category, and the default values ​​are displayed. Users can change the setting values ​​displayed in area 543 by operating the operation unit 155.

[0066] The data augmentation setting screen 540 further includes a button 540b displaying the text "Return" and a button 540c marked with the text "Next Page". When the user presses button 540b, the learning data generation screen 530 shown in FIG7 is displayed on the display device 156, and when the user presses button 540c, data augmentation processing is performed on the image data set as the processing target. The image data set as the processing target is image data stored in the memory device 154, and belongs to multiple image data belonging to image data specified by the user or a data group specified by the user.

[0067] Figure 9 shows an example of an image classification setting screen. The image classification setting screen is displayed on the display device 156 when the user sets image classification labels for image data. The image classification setting screen is displayed on the display device 156 based on the user's instruction to press button 532a on the learning data generation screen 530. Referring to Figure 9, the image classification setting screen 550 includes an area 551 for displaying annotation modes, an area 552 for displaying label information, an area 553 for displaying the images of the image data, and an area 554 for displaying an overview of the image data.

[0068] Area 554 displays a list of multiple image data belonging to the data group specified by the user for processing. Area 554 also displays thumbnails (reduced versions of the original image data) and filenames used to identify the image data. Furthermore, area 554 displays the thumbnails and filenames of the image data specified by the user for processing in reverse order to distinguish them from other images. If the user points to any of the thumbnails displayed in area 554, the image data corresponding to the pointed-out thumbnail is selected as the processing target.

[0069] The image data of the object being processed is displayed in area 553. Image category information is displayed in area 551. Area 552 can selectively display "cat" and "dog" as tag information. A checkbox is provided to the left of the tag information, allowing the user to specify either "cat" or "dog" in the tag information.

[0070] Users can set tag information using the image classification settings screen 550. For an image file named 0.jpg, the system generates image category information (image classification metadata) and tag information (cat metadata).

[0071] Figure 10 shows an example of an object recognition setting screen. The object recognition setting screen is displayed on the display device 156 when the user sets object recognition symbols in image data. The object recognition setting screen is displayed on the display device 156 based on the user's instruction to press button 532b on the learning data generation screen 530. Referring to Figure 10, the object recognition setting screen 560 includes an area 561 for displaying annotation modes, an area 562 for displaying label information, an area 563 for displaying the image data, and an area 564 for displaying a summary of the image data.

[0072] Area 564 displays a list of multiple image data belonging to the data group specified by the user for processing. Area 564 also displays thumbnails (reduced versions of the original image data) and filenames used to identify the image data. Furthermore, area 564 displays the thumbnails and filenames of the image data specified by the user for processing in reverse order to distinguish them from other images. If the user points to any of the thumbnails displayed in area 564, the image data corresponding to the pointed-out thumbnail is selected as the processing target.

[0073] An image of the object being processed is displayed in region 563. Furthermore, a rectangular region 565 surrounding the object is displayed within the image data. Region 565 is determined and displayed by the user specifying the coordinates of the two diagonal vertices of the rectangle.

[0074] In area 561, image type information is displayed for object identification. Area 562 can selectively display cats and dogs as tag information. A checkbox is provided to the left of the tag information, allowing the user to specify either cats or dogs in the tag information.

[0075] The user sets tag information using the object recognition settings screen 560. For the image data with the file name 1019.jpg, the system generates image type information display, object recognition metadata, metadata display of the position of the rectangular area 565 surrounding the target object in the image, and tag information display of the dog's metadata.

[0076] Figure 11 shows an example of a region classification setting screen. The region classification setting screen is displayed on the display device 156 when the user sets object recognition metadata for image data. The region classification setting screen is displayed on the display device 156 based on the user's instruction to press button 532c on the learning data generation screen 530. Referring to Figure 11, the region classification setting screen 570 includes an area 571 for displaying annotation modes, an area 572 for displaying label information, an area 573 for displaying images of image data, and an area 574 for displaying a summary of image data.

[0077] Area 574 displays a list of multiple image data belonging to the data group specified by the user for processing. Area 574 also displays thumbnails (reduced versions of the original image data) and filenames used to identify the image data. Furthermore, area 574 displays the thumbnails and filenames of the image data specified by the user for processing in reverse order to distinguish them from other images. If the user points to any of the thumbnails displayed in area 574, the image data corresponding to the pointed-out thumbnail is selected as the processing target.

[0078] In area 573, an image of the image data of the object being processed is displayed. Also, in the image data, the outline 575 of the target object is displayed. By having the user indicate a plurality of coordinates corresponding to the positions near the lines forming the outline of the target object, the outline 575 connecting two adjacent coordinates of each of the plurality of coordinates is determined and displayed.

[0079] In area 571, image type information is displayed as a region category. Area 572 can selectively display cats and dogs as tag information. A checkbox is provided to the left of the tag information, allowing the user to specify either cats or dogs in the tag information.

[0080] The user sets tag information and outline 575 using the area classification setting screen 570. For the image data with the file name 1000.jpg, the system generates image type information display object recognition metadata, display target object outline 575 position metadata in the image, and tag information display dog ​​metadata.

[0081] Figure 12 is a flowchart illustrating one example of the learning model acquisition process. The learning model acquisition process is performed in step S04 of the learning model generation process. Referring to Figure 12, the CPU 151 causes the display device 156 to display the learning model selection screen (step S31), and the process proceeds to step S32.

[0082] Figure 13 shows an example of a learning model selection screen. Referring to Figure 13, the learning model selection screen 580 includes buttons 581-583. Button 581 is used to accept instructions to select a learning model publicly available on the Internet. Button 582 is used to accept instructions to select a learning model stored in the memory device 154. Button 583 is used to accept instructions to end the display of the learning model selection screen.

[0083] Returning to Figure 12, CPU 151 branches its processing based on whether the learning model is obtained from the external or internal source (step S32). When CPU 151 receives an instruction from instruction button 581 in the learning model selection screen, it receives an instruction from the external learning model and proceeds to step S33. When CPU 151 receives an instruction from instruction button 582 in the learning model selection screen, it receives an instruction from the internal learning model and proceeds to step S39.

[0084] In step S33, CPU 151 accesses the URL and proceeds the process to step S34. The URL is the network address of the library area publicly available on the Internet. The library area is an area that stores multiple learning models with a unified interface. Here, the interface is a general term for the format and effect of the input and output of the learning model. Specifically, a learning model having an interface means that the learning model accepts combinations of more than one number string for each of its input and output, and that the learning model has more than one function to process the number string, and that the learning model explicitly indicates the format and function of each of its input and output. In other words, two or more learning models with the same interface display the same format of their input and output, and the two or more learning models have the same type of function and the same method of indicating the function. The learning model is a deep learning network. Here, the library area is set to timm (pytorch-image-models) or Pytorch Hub. The URL can be pre-memorized in the memory device 154 or processed by the user by inputting it into the operation unit 155.

[0085] In step S34, model recognition information for one or more learning models stored in the library area is displayed, and the process proceeds to step S35. The one or more model recognition information items are displayed on the display device 156 in a selectable manner. In step S35, it is determined whether one model recognition information item has been selected from the one or more model recognition information items. If a model recognition information item has been selected, the process proceeds to step S36; otherwise, the process proceeds to step S37. In step S36, an explanation screen is displayed on the display device 156, and the process proceeds to step S37. The explanation screen includes an area displaying explanatory information related to the learning model selected in step S35. The explanatory information includes detailed information displaying the characteristics of the learning model. Therefore, by providing explanatory information to the user who is determining whether to use the model as a machine learning object, the user can be assisted in making the decision.

[0086] In step S37, it is determined whether the selection instruction has been accepted. If the selection instruction has been accepted, the process proceeds to step S38; otherwise, the process returns to step S34. In step S39, the learning model of the model recognition information selected in step S36 is downloaded, and the process returns to the learning model generation process.

[0087] In step S39, it is determined whether a learning model has been specified. If a learning model has been specified, the system enters a standby state (NO in step S39). If a learning model has been specified, the process proceeds to step S40. If the user operates the operation unit 155 and specifies a learning model stored in the memory device 154, it is determined that a learning model has been specified.

[0088] In step S40, CPU 151 extracts the specified learning model and returns the processing to the learning model generation process. The learning model is extracted by CPU 151 executing a process that sets the state of the model as an object suitable for machine learning. For example, this involves setting the location and filename in the memory device 154 storing the specified learning model.

[0089] Figure 14 shows an example of an external learning model acquisition screen. The external learning model acquisition screen is displayed on the display device 156 during step S34 of the learning model acquisition process. Referring to Figure 14, the external learning model acquisition screen 590 includes an area 591 displaying URLs, an area 592 displaying a list of learning models, an area 593 displaying descriptive information, a button 594 labeled "Select," and a button 595 displaying "Back." In area 591, the URLs of the timm or PyTorch Hub library areas are displayed in a list. Here, the case where the URL for the VGG library area is selected is explained as an example.

[0090] In area 592, model identification information for one or more learned models stored in the library area VGG specified by the URL selected in area 591 is displayed. Here, the model identification information for VGG11, VGG13, VGG16, and VGG19 is displayed in an orderly fashion. Furthermore, the model identification information VGG16 is displayed in reverse order, showing that model identification information VGG16 has been selected.

[0091] Area 593 is an explanatory screen, displaying explanatory information for the model recognition information VGG16 selected in area 592. This explanatory information includes detailed information displaying the features of the learning model specific to the model recognition information VGG16.

[0092] When a learning model is selected in area 592, and the user presses button 594, the selected learning model is chosen as the object of machine learning. When a learning model is selected in area 592, and the user presses button 595, if no learning model is selected as the object of machine learning, the external learning model acquisition screen switches to another screen.

[0093] Alternatively, in the external learning model acquisition screen 590, the learning model can be specified using another application without the user specifying the URL and learning method. For example, the CPU 151 can access the URL by executing another application and display a list of model recognition information for the learning model. The user can then drag and drop the model recognition information onto the external learning model acquisition screen. In this case, the CPU 151 will display the model recognition information of the learning model placed on the external learning model acquisition screen 590 in area 592, and will display the URL of the library area storing the learning model in area 591.

[0094] Figure 15 is a flowchart illustrating one example of the machine learning process. The machine learning process is performed in step S05 of the learning model generation process. Referring to Figure 15, CPU 151 determines the learning model to be the object of machine learning (step S51) and proceeds to step S52. For example, when a learning model is selected in area 592 of the external learning model acquisition screen 590 shown in Figure 14, and the user instructs button 594, the learning model selected in area 592 will be determined as the object of machine learning.

[0095] In step S52, it is determined whether the hyperparameter setting instruction has been accepted. If the hyperparameter setting instruction has been accepted, the process proceeds to step S53; otherwise, it proceeds to step S54. In step S53, the hyperparameter is set, and the process proceeds to step S54. In step S54, it is determined whether learning data has been specified. If learning data has been specified, the process proceeds to step S55; otherwise, the process returns to step S52. In step S55, the learning data is obtained, and the process proceeds to step S56. Alternatively, either step S53 or step S55 can be executed first.

[0096] In step S56, machine learning is performed, and the process proceeds to step S57. CPU 151, according to the hyperparameters set in step S53, causes the learning model determined as the processing object in step S51 to perform machine learning on the learning data specified in step S54. The machine learning in step S56 is the process of learning from the learning data specified in step S54. The unit for the learning model to learn from the learning data is a single round of training. Therefore, the learned model generated by machine learning is determined by the number of repetitions of a single round of training. The number of times step S56 is executed corresponds to the number of times the single round of training is repeated.

[0097] In step S57, the learned model is stored in memory device 154, and the process proceeds to step S58. The learned model is associated with the number of repetitions in a single training round and stored in memory device 154. In step S58, it is determined whether the number of repetitions in a single training round has reached the learning count. The learning count is a value set as part of the hyperparameters set in step S53. If the number of repetitions in a single training round has reached the learning count, the process returns to the model generation process; otherwise, the process returns to step S56.

[0098] Figure 16 shows an example of a hyperparameter setting screen. The hyperparameter setting screen is displayed on the display device 156 during step S53 of the machine learning process shown in Figure 15. Referring to Figure 16, the hyperparameter setting screen 600 includes areas 601-604 for accepting settings of four types of hyperparameters. Area 601 is for accepting settings of the hyperparameter named "Learning Times." The learning times is the upper limit of the number of repetitions in a single training round. Area 602 is for accepting settings of the hyperparameter named "Patch Size." The patch size determines the maximum value of the learning data used in a single learning cycle in the learning model during machine learning.

[0099] Area 603 is the area for setting hyperparameters named "Optimization Function". The optimization function is set by selecting from a pre-prepared set of functions. Area 604 is the area for setting hyperparameters named "Learning Rate". A plurality of parameters determining the learning rate are provided, and each can be set individually.

[0100] The initial values ​​are predetermined by default in the hyperparameters. The initial values ​​are stored in the memory device 154. During the display of the hyperparameter setting screen 600, the initial values ​​are displayed in areas 601 to 604. Users can change the hyperparameters by changing the initial values, enabling one learning model to perform multiple types of machine learning.

[0101] Figure 17 shows an example of a learning data selection screen. The learning data selection screen is displayed on the display device 156 in step S55 of the machine learning process shown in Figure 15. Referring to Figure 17, the learning data selection screen 610 displays a learning data table containing learning data records. In the example shown in Figure 17, the learning data table contains 10 learning data records. Each learning data record includes a checkbox item 611, a data group item 612, a data name item 613, an identification item 614, a data count item 615, a tag count item 616, and an update date item 617.

[0102] Item 613 of the "Data Name" field sets the file name used to identify the learning materials. Item 612 of the "Data Group" field sets the group name used to identify the group to which the learning materials belong. Item 614 of the "Identification" field sets the image type information of the metadata labeled on the learning materials. Item 617 of the "Data Count" field sets the number of image data contained in the learning materials. Item 617 of the "Tag Count" field sets the number of values ​​set for the tag information of the metadata labeled on the learning materials. Item 617 of the "Update Date" field sets the date the learning materials were last updated.

[0103] When the operation unit 155 accepts an operation to select any of the learning data records, a mark is displayed in the checkbox 611 of the learning record selected by the user. Therefore, the user can confirm that the learning data record has been selected.

[0104] Furthermore, in the learning materials selection screen 610, multiple learning materials records can be selected. In this case, the collection of specific learning materials from the multiple learning materials records is used as one learning material for machine learning.

[0105] The learning materials selection screen 610 further includes a button 618 that displays the text string "Restore Process". When the user instructs the button 618, the CPU 151 restores the operation entered by the user when selecting learning materials in the past. In this way, the user can easily select the same learning materials as previously selected.

[0106] Figure 18 is a flowchart illustrating one example of the learning model evaluation process. The learning model evaluation process is performed in step S06 of the learning model generation process. Referring to Figure 18, the CPU 151 displays the learned model on the display device 156 (step S61) and proceeds to step S62. When the machine learning process shown in Figure 15 is executed, the learned model for each training iteration N is stored in the memory device 154. In step S61, the model recognition information of the learned model stored in the memory device 154 is displayed on the display device 156.

[0107] In step S62, the learned model to be evaluated is selected, and the process proceeds to step S63. Since the memory device 154 stores multiple learned models with different repetition numbers in a single round of training, for the selected learned model, the multiple learned models with different repetition numbers in a single round of training are read from the memory device 154.

[0108] In step S63, evaluation data is obtained, and the process proceeds to step S64. The evaluation data and learning data are generated simultaneously and are in the same format as the learning data.

[0109] In step S64, one piece of evaluation data is selected as the processing object, and the process proceeds to step S65. In step S65, CPU 151 makes the learned model make a prediction, and the process proceeds to step S66. CPU 151 inputs the evaluation data into the plurality of learned models with different repetition numbers of a single round of training read from memory device 154 in step S62, and obtains the prediction values ​​predicted by the plurality of learned models respectively.

[0110] In step S66, an evaluation value is determined, and the process proceeds to step S67. The predicted value is compared with the label information contained in the evaluation data to determine whether the two are consistent. For multiple learned models with different repetition numbers in a single training round, an evaluation value is determined separately. In step S67, it is determined whether there is any evaluation data that was not selected as the processing object in step S64. If there is unselected evaluation data, the process returns to step S64; otherwise, the process proceeds to step S68.

[0111] When processing proceeds to step S68, for each of the multiple learned models with different repetition numbers N in a single training round, the same number of evaluation values ​​as the evaluation data are determined. In step S68, performance data is calculated, and processing proceeds to step S69. Performance data includes, for example, accuracy, reproducibility, and fitness rate as evaluation indices. Performance data is calculated for each of the multiple learned models with different repetition numbers in a single training round. In step S69, the learned model evaluation screen is displayed, and processing returns to the learned model generation process. The user can refer to the learned model evaluation screen to select one from the multiple learned models with different repetition numbers in a single training round.

[0112] Figure 19 shows an example of a learned model evaluation screen. Referring to Figure 19, the learned model evaluation screen 620 includes areas 621-623. Area 621 displays information related to the learned model that is being evaluated. Here, in area 621, the learned model with model recognition information VGG16 is displayed as the evaluation object. Furthermore, it shows that this learned model is a machine learning model that uses learning data from the data group "probe_group".

[0113] Region 622 displays the learning curve of the learned model of the evaluation object. The learning curve is the learning curve when the evaluation index is used as the accuracy. The horizontal axis shows the number of repetitions in a single training round, and the vertical axis shows the accuracy. Region 623 displays the learning curve of the loss of the learned model of the evaluation object. The horizontal axis shows the number of repetitions in a single training round, and the vertical axis shows the loss. The loss is the value of the objective function, which shows the difference between the correct solution of the learning data and the predicted value of the learned model.

[0114] As the number of repetitions in a single training round increases, the machine learning of the learning model progresses, and the model is adjusted to be suitable for the learning data, resulting in improved accuracy. Furthermore, when the number of repetitions in a single training round exceeds a predetermined number, the accuracy falls within a specified range. Moreover, as the number of repetitions in a single training round increases, the accuracy may sometimes decrease; this change in accuracy can be used to determine overlearning.

[0115] Since users viewing the evaluation screen of the learned model can see the results of evaluating the learned model using evaluation data, they can select an appropriate number of repetitions for a single training run. Furthermore, it is easy to determine whether the learned model is overlearned.

[0116] Figure 20 is a flowchart illustrating one example of the learning model comparison process. The learning model comparison process is performed in step S07 of the learning model generation process. Referring to Figure 20, the CPU 151 displays the learned models on the display device 156 (step S71) and proceeds to step S72. When the learning model evaluation process shown in Figure 18 is executed, one learned model determined from a plurality of learned models with different repetition numbers in a single training round is stored in the memory device 154. In step S71, the model recognition information of the plurality of learned models stored in the memory device 154 is displayed on the display device 156. At least one of the learning model, hyperparameters, and learning data of the plurality of learned models is different.

[0117] In step S72, it is determined whether to select two or more learned models as comparison objects. If two or more learned models are selected, the process enters a standby state (no in step S72). If two or more learned models are selected (yes in step S72), the process proceeds to step S73.

[0118] In step S73, performance data is read out, and the process proceeds to step S74. Performance data corresponding to each of the two or more learned models is read out from memory device 154. Then, in step S74, the learning model comparison screen is displayed on display device 156, and the process returns to the learning model generation process.

[0119] Figure 21 shows an example of a learned model comparison screen. It is an example of a screen displayed when comparing a learned model with model identification information of VGG16 and a learned model with model identification information of VGG19. Referring to Figure 21, the learned model comparison screen 630 includes areas 631 and 631A displaying information related to the learned models being compared, and area 632 displaying performance data. Area 631 displays information related to the learned model with model identification information of VGG16, and area 631A displays information related to the learned model with model identification information of VGG19. Area 632 displays the performance data of the learned model with model identification information of VGG16 and the learned model with model identification information of VGG19. ACC represents the correct resolution, Recall represents the reproducibility rate, and Precision represents the accuracy.

[0120] The learned model comparison screen 630 further includes areas 633 and 634 and button 635. The confusion matrix is ​​displayed in area 634. Area 633 displays information related to the evaluation data. It compares a learned model with model identification information of VGG16 with a learned model with model identification information of VGG19, using the same evaluation data.

[0121] Therefore, users can compare the performance data of the two learned models and easily determine which learned model is more suitable. Furthermore, when the instruction button 635 is pressed, the CPU 151 displays a screen for selecting evaluation data on the display device 156. Based on the performance data obtained by evaluating the two learned models using other evaluation data selected by the user, a new learned model comparison screen is generated and displayed on the display device 156.

[0122] Therefore, by reviewing the results of evaluating two learned models using different evaluation data, users can easily determine which learned model is more suitable.

[0123] (3) The effect of implementation form According to the learning model generation apparatus 100 of the above embodiment, since multiple existing learning models publicly available on the Internet can be selectively displayed on the display device 156, it is easy to select a learning model for machine learning. Since existing learning models are used for machine learning, there is no need to code and create a new learning model. In particular, in the external learning model acquisition screen 590 shown in FIG. 14, explanatory information is displayed in area 593. Although the algorithms of the multiple existing learning models involve various aspects, by viewing the explanatory information displayed in area 593, the user can understand the algorithm of the learning model and easily select a suitable learning model from the multiple existing learning models for the purpose of prediction.

[0124] Furthermore, multiple existing learning models publicly available on the Internet are stored in libraries such as timm or PyTorch Hub. Since the interfaces of these multiple learning models stored in these libraries are identical, there is no need to change the learning data to suit the learning model. Therefore, machine learning can be performed on the learning model without creating a program to adapt the learning data to the learning model. As a result, a learning model generation device 100 that can easily generate learned learning models can be provided.

[0125] Furthermore, since the GUI used to generate learning materials is displayed on the display device 156, users can easily generate learning materials based on image data.

[0126] Furthermore, an operation screen (Figure 8, Data Augmentation Setting Screen 540) is displayed, which accepts user instructions for generating (data augmentation) other new image data based on the existing image data. In this way, users can easily generate new image data simply by inputting instructions on the operation screen to do so.

[0127] Furthermore, since the operation screens for accepting metadata attached to image data are displayed (Image Classification Setting Screen 550 in Figure 9, Object Recognition Setting Screen in Figure 10, and Region Classification Setting Screen in Figure 11), users can easily perform annotation operations simply by following the instructions on the operation screens to attach metadata to the image data. Therefore, learning materials can be easily generated without coding the program used to generate learning materials.

[0128] Furthermore, since the GUI (hyperparameter setting screen 600) used for setting hyperparameters is displayed on the display device 156, the user can easily set the hyperparameters before machine learning. Also, with the user-set hyperparameters in the user-selected learning model, the learning model performs machine learning. Therefore, there is no need to code the program used to set the hyperparameters of the learning model, making it easy to set the hyperparameters of the learning model.

[0129] The learning model generation device 100 of this embodiment does not require coding in any of the stages of generating the learning model, setting hyperparameters, or generating learning data. Therefore, even a person without programming knowledge can use the learning model for machine learning to generate a learned learning model. Furthermore, compared to creating a program, the learning model generation device 100 can generate a learned model in a short time.

[0130] (4) Summary of Implementation Forms (Item 1) The state-type learning model generation apparatus of the present invention comprises: The model information display unit can selectively display multiple learning models that are publicly available on the Internet and have the same interface; The model selection unit selects at least one of the above-mentioned learning models from the plurality of displayed learning models; and The learning department uses the selected learning model to perform rote learning on the learning materials, generating a learned model.

[0131] According to this approach, since multiple existing learning models publicly available on the Internet can be selectively displayed, it is easy to select a learning model for machine learning. Furthermore, since the interfaces of the multiple selectively displayed learning models are identical, it is not necessary to adapt the learning model to the learning data; therefore, machine learning can be performed without creating a program. As a result, a learning model generation device can be provided that can easily generate learned learning models.

[0132] (Item 2) The learning model generation device described in Item 1 can also be further equipped with: Image data acquisition unit, which acquires image data: and The learning data generation unit generates learning data based on the image data acquired by the image data acquisition unit; and The aforementioned learning materials generation unit has a graphical user interface for generating learning materials.

[0133] According to this model, since it has a graphical user interface for generating learning materials, users can easily generate learning materials from image data.

[0134] (Item 3) In the learning model generation device described in Item 2, The aforementioned graphical user interface may also have an operation screen that accepts instructions for generating other new image data based on the aforementioned image data.

[0135] In this configuration, since the operation screen displays instructions from the user to generate other new image data based on the existing image data, the user can easily generate new image data simply by inputting the instructions to generate the new image data according to the operation screen.

[0136] (Item 4) In the learning model generation device described in Item 2 or Item 3, The aforementioned graphical user interface may also have an operation screen for receiving information attached to the aforementioned image data.

[0137] In this format, since the operation screen displays the information to be attached to the image data, the user can easily generate learning materials by simply following the instructions on the operation screen to attach the information to the image data.

[0138] (Item 5) In the learning model generation device described in items 1 through 4, The aforementioned model information display unit can also further display information related to the features of the plurality of the aforementioned learning models.

[0139] According to this approach, since information related to the characteristics of the learning model is displayed, it can help the user when selecting one of several learning models as the object of machine learning.

[0140] (Item 6) The learning model generation device described in items 1 through 5 can also be further equipped with: The parameter setting unit sets the hyperparameters used by the learning unit to enable the learning model to perform machine learning; and The parameter setting section described above has a graphical user interface for setting hyperparameters.

[0141] As such, since it has a graphical user interface for setting hyperparameters, users can easily set hyperparameters.

[0142] (Item 7) The learning model generation apparatus described in items 1 through 6 can also be used for... The model selection section can select a customized model, which is a pre-prepared learning model separate from the plurality of learning models publicly available on the Internet. The interface of the customized model is the same as that of the plurality of learning models publicly available on the Internet.

[0143] According to this approach, users can easily perform machine learning on custom models that differ from the multiple learning models publicly available on the Internet.

[0144] (Item 8) The learning model generation device described in items 1 through 7 can also be further equipped with: The evaluation data acquisition department is responsible for acquiring evaluation data. The evaluation department, using the aforementioned evaluation data, evaluates the learned model generated by the aforementioned learning department; and The evaluation results display section shows the evaluation results for the learned model.

[0145] Following this approach, evaluation data is obtained, and the learned model is evaluated using this data, with the evaluation results displayed. Therefore, users can be provided with information related to machine learning, including the selection of a learning model.

[0146] (Item 9) In the learning model generation device described in Item 8, When there are multiple learned models, the evaluation result display unit can also display multiple evaluation results corresponding to the multiple learned models.

[0147] In this manner, since multiple evaluation results corresponding to multiple learned models are displayed in a sorted manner, users can easily compare multiple learned models.

[0148] (Item 10) Another aspect of the present invention, the method for generating a learning model, causes the learning model generating apparatus to perform the following steps: The model information display process allows for the selective display of multiple learning models that are publicly available on the Internet and have the same interface; The model selection step involves selecting at least one of the above learning models from the plurality of displayed learning models; and The learning process involves using the selected learning model to perform machine learning on learning data containing image data, and generating a learned model.

[0149] Based on this approach, a learning model generation method can be provided that can easily generate learned learning models.

[0150] (Item 11) Another aspect of the learning model generation program of the present invention enables the computer to perform the following steps: The model information display process allows for the selective display of multiple learning models that are publicly available on the Internet and have the same interface; The model selection step involves selecting at least one of the above learning models from the plurality of displayed learning models; and The learning process involves using the selected learning model to perform machine learning on learning data containing image data, and generating a learned model.

[0151] Based on this approach, a learning model generation program can be provided that can easily generate learned learning models.

[0152] 1: Learning Model Generation System 3: Network 5: Internet 100: Learning model generation device 101: Model Information Acquisition Department 103: Model Information Display Department 105: Model Selection Department 107: Model Acquisition Department 109: Parameter Change Department 111: Study Department 113: Image Data Acquisition Department 115: Learning Materials Production Department 115a: Data Extension Department 115b: Annotations 117: Evaluation Data Acquisition Department 119: Forecasting Department 121: Evaluation Department 123: Evaluation Results Display Department 151: CPU 152:RAM 153:ROM 154: Memory device 155: Operations Department 156: Display device 157: Input / Output (I / F) 158: Busbar 159: CD-ROM 200: Server 300: Server 510: Menu screen 511: Keypad 512: Keypad 513: Keypad 514: Button 515: Button 517: Button 530: Learning to generate images using materials 531: Key 532: Key 532a: Button 532b: Button 532c: Button 540: Settings Screen 540b: Button 540c: Buttons 542: Area 543: Area 550: Image Classification Settings Screen 551: Area 552: Area 553: Area 554: Area 560: Object Recognition Settings Screen 561: Region 562: Area 563: Region 564: Area 565: Region 570: Region Classification Setting Screen 571: Area 572: Area 573: Area 574: Area 575: Outline 580: Learning Model Selection Screen 581: Button 582: Keypad 583: Keypad 590: Image obtained by external learning model 591: Area 592: Area 593: Area 594: Keypad 595: Button 600: Hyperparameter Setting Screen 601: Area 602: Area 603: Area 604: Area 610: Learning Materials Selection Screen 611: Project 612: Project 613: Project 614: Project 615: Project 616: Project 617: Project 618: Button 620: Evaluation screen of the learned model 621: Area 622: Area 623: Area 630: Comparison screen of learned models 631: Area 631A: Area 632: Area 633: Area 634: Area 635: Button ACC: Accuracy N: Number of repetitions S01~S07: Steps S11~S21: Steps S31~S40: Steps S51~S58: Steps S61~S69: Steps S71~S74: Steps VGG: Storage Area VGG11, VGG13, VGG16, VGG19: Model Recognition Information

Claims

1. A learning model generation apparatus comprising: a model information display unit that selectively displays a plurality of learning models publicly available on the Internet, each having a uniform input and output format and a uniform type and instruction method; a model selection unit that selects at least one of the displayed learning models; and a learning unit that causes the selected learning model to perform machine learning on learning data to generate a learned model.

2. The learning model generating apparatus of claim 1 further comprises: an image data acquisition unit that acquires image data; and a learning data generating unit that generates learning data based on the image data acquired by the image data acquisition unit; and the learning data generating unit has a graphical user interface for generating learning data.

3. The learning model generating apparatus of claim 2, wherein the graphical user interface has an operation screen that accepts instructions for generating other new image data based on the image data.

4. The learning model generating apparatus as described in request item 2 or 3, wherein the aforementioned graphical user interface has an operation screen for receiving information attached to the aforementioned image data.

5. The learning model generating apparatus of any one of claims 1 to 3, wherein the model information display unit further displays information related to the features of the plurality of the learning models.

6. The learning model generating apparatus according to any one of claims 1 to 3 further comprises: a parameter setting unit that sets the hyperparameters used by the learning unit to enable the learning model to perform machine learning; and the parameter setting unit has a graphical user interface for setting the hyperparameters.

7. The learning model generating apparatus of any one of claims 1 to 3, wherein the model selection unit can select a custom model, which is a learning model prepared in advance separately from the plurality of learning models disclosed on the Internet; the interface of the custom model is the same as that of the plurality of learning models disclosed on the Internet.

8. The learning model generating apparatus according to any one of claims 1 to 3 further comprises: an evaluation data acquisition unit that acquires evaluation data; an evaluation unit that uses the evaluation data to evaluate the learned model generated by the learning unit; and an evaluation result display unit that displays the evaluation result for the learned model.

9. The learning model generating apparatus of claim 8, wherein when there are multiple learned models, the evaluation result display unit displays multiple evaluation results corresponding to the multiple learned models respectively.

10. A method for generating a learning model, which causes a computer to perform the following steps: a model information display step, which selectively displays a plurality of learning models publicly available on the Internet, with uniform input and output formats and uniform types and methods of indication; a model selection step, which selects at least one of the learning models from the plurality of displayed learning models; and a learning step, which causes the selected learning model to perform machine learning on learning data and generate a learned model.

11. A learning model generation program that causes a computer to perform the following steps: a model information display step, which selectively displays a plurality of learning models publicly available on the Internet, with uniform input and output formats and uniform types and methods of instruction; a model selection step, which selects at least one of the displayed plurality of learning models; and a learning step, which causes the selected learning model to perform machine learning on the learning data and generate a learned model.

Images