DATA MANAGEMENT SYSTEM, DATA MANAGEMENT METHOD, AND MACHINE LEARNING DATA MANAGEMENT PROGRAM

Abstract

To manage data of a learned model to response to diversified needs of a user and increase the convenience for the user.SOLUTION: An operation processing system 1 includes a management server 2, a plurality of user terminals 3, and a virtual machine server 4. The management server 2 includes an auxiliary storage unit 25 including the learned model DB53. The management server 2 generates a plurality of learned models having mechanically learned under different conditions. The data of each of the learned models is stored in the learned model DB53 in a distinguishable manner in relation to the learning data used for the mechanical learning.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a data management system, a data management method, and a machine learning data management program for managing, for example, data of a trained model using machine learning and learning data used for machine learning of the trained model. [Background technology]

[0002] In recent years, so-called trained models using AI technology have been proposed as a series of software technologies that enable computers to perform intellectual activities such as reasoning that are possible for humans. Such trained models are generally generated by performing machine learning using a large amount of training data (training data).

[0003] An example of this type of background technology is disclosed in Patent Document 1. In the learning device of the background technology, additional learning is automatically performed in response to changes in the environment or usage situation, and the accuracy of the trained model is maintained by always keeping the trained model up to date. In this way, in the learning device of the background art, only the latest trained model for a certain purpose is provided. Therefore, the learning device of the background art cannot adequately meet the demand for verifying a certain phenomenon from multiple perspectives, and has a problem in terms of usability. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-140644 A Summary of the Invention [Problem to be solved by the invention]

[0005] In view of the above-mentioned problems, the present invention aims to provide a data management system, a data management method, and a machine learning data management program that can manage trained model data to meet the diverse needs of users, thereby improving user convenience. [Means for solving the problem]

[0006] The present invention is characterized by a data management system, a data management method, and a machine learning data management program that include a learning data acquisition means for acquiring learning data to be used in machine learning, a first storage means for storing the learning data, a trained model generation means for generating a plurality of trained models of different variations that have been machine-learned under different learning conditions using the learning data, and a second storage means for linking each of the data of the plurality of trained models to the learning data used in the machine learning and storing them in a distinguishable manner. Effect of the Invention

[0007] This invention makes it possible to manage trained model data so as to meet the diverse needs of users, thereby improving user convenience. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is an explanatory diagram showing an example of the configuration of a data management system according to the present invention. [Diagram 2] FIG. 2 is an explanatory diagram showing an example of the configuration of a virtual machine server. [Diagram 3] FIG. 4 is an explanatory diagram showing an example of data stored in an auxiliary storage unit of a management server. [Figure 4] FIG. 1 is an explanatory diagram showing an example of a data structure of trained model data. [Diagram 5] FIG. 4 is an explanatory diagram showing an example of a memory map of a main storage unit of a management server. [Figure 6] FIG. 4 is an explanatory diagram of a screen configuration such as an initial screen. [Figure 7] FIG. 13 is an explanatory diagram of a screen configuration of a first member registration screen, etc. [Figure 8]FIG. 4 is an explanatory diagram of a screen configuration such as a login screen. [Figure 9] FIG. 4 is an explanatory diagram of a screen configuration of a main screen, etc. [Figure 10] FIG. 13 is an explanatory diagram of the screen configuration of a second member registration screen, etc. [Figure 11] FIG. 4 is an explanatory diagram of a screen configuration of an execution environment setting screen, etc. [Figure 12] An explanatory diagram of the screen configuration of a trained model search screen, etc. [Figure 13] FIG. 13 is an explanatory diagram of a screen configuration of a version check registration screen, etc. [Figure 14] FIG. 13 is an explanatory diagram of a screen configuration of a model change history confirmation screen, etc. [Figure 15] FIG. 13 is an explanatory diagram of a screen configuration of a data change history confirmation screen, etc. [Figure 16] FIG. 13 is an explanatory diagram of a screen configuration of a change data confirmation screen, etc. [Figure 17] FIG. 13 is an explanatory diagram of a screen configuration of an annotation management screen, etc. [Figure 18] FIG. 13 is an explanatory diagram of the screen configuration of a self-edit screen, etc. [Figure 19] FIG. 13 is an explanatory diagram of the screen configuration of an outsourcing management screen, etc. [Figure 20] FIG. 13 is an explanatory diagram of a screen configuration of a learning data management screen, etc. [Figure 21] FIG. 13 is an explanatory diagram of a screen configuration of a user confirmation screen, etc. [Figure 22] FIG. 4 is an explanatory diagram of the screen configuration of a contact screen, etc. [Diagram 23] An explanatory diagram of the screen configuration of a trained model registration screen, etc. [Figure 24] FIG. 4 is an explanatory diagram of a screen configuration of a calculation processing screen, etc. [Diagram 25] FIG. 11 is a flowchart of the data management operation of the control program of the management server. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. <System configuration>

[0010] 1 is a block diagram showing an example of a system configuration of a arithmetic processing system 1. The arithmetic processing system 1 is a arithmetic processing system for executing calculations on calculation data in a predetermined format according to a learned model that utilizes AI technology.

[0011] However, the calculation processing system 1 is also a data management system for managing data of a trained model using AI technology and teacher data (learning data) used for machine learning of the trained model.

[0012] 1, the computing system 1 includes a management server 2, a plurality of user terminals 3, and a virtual machine server 4, and the management server 2 and the plurality of user terminals 3 are connected to each other so as to be able to communicate with each other via a public line (public network) 5 such as the Internet. The management server 2 and the virtual machine server 4 are connected to each other so as to be able to communicate directly with each other via a wired or wireless connection. The management server 2 and the virtual machine server 4 may also be connected to each other via the public line 5.

[0013] The management server 2 and the virtual machine server 4 are configured with general-purpose server computers, and the user terminal 3 is configured with a general-purpose computer (terminal) such as a tablet PC, a desktop PC, a notebook (laptop) PC, a smartphone, etc. The management server 2 and the virtual machine server 4 may be configured with one server computer or multiple server computers.

[0014] The management server 2 includes a control unit 21, an input unit 22, a display unit 23, a communication unit 24, and an auxiliary storage unit 25. The input unit 22, the display unit 23, the communication unit 24, and the auxiliary storage unit 25 are each connected to the control unit 21 via a communication line (bus).

[0015] The user terminal 3 includes a control unit 31, an input unit 32, a display unit 33, a communication unit 34, and an auxiliary storage unit 35. The input unit 32, the display unit 33, the communication unit 34, and the auxiliary storage unit 35 are each connected to the control unit 31 via a communication line.

[0016] The control unit 21 includes a processor 26 and a main memory unit 27, and executes various calculations and control operations in the management server 2. The control unit 31 includes a processor 36 and a main memory unit 37, and executes various calculations and control operations in the user terminal 3.

[0017] The processors (26, 36) are arithmetic processing units including a CPU or an MPU. The main memories 27, 37 include a RAM (DRAM) and a ROM. The RAM is used as a work area and a buffer area for the processors (26, 36). The ROM stores the startup program of the management server or the user terminal 3, default values ​​for various information, and the like.

[0018] The input unit (22, 32) includes an input component that accepts operation input from a user and an input detection circuit that is interposed between the input component and the processor (26, 36). The input component is, for example, a keyboard and / or a computer mouse, and when the input component is a keyboard, it includes hardware operation buttons or keys. A touch panel may also be used as the input component. Any type of touch panel can be used, such as a capacitive type, an electromagnetic induction type, a resistive film type, or an infrared type. The input detection circuit outputs an operation signal or operation data corresponding to the operation of each input component to the processor (26, 36).

[0019] The display unit (23, 33) includes a display and a display control circuit interposed between the display and the processor (26, 36). The display may be, for example, an LCD (liquid crystal display) or an organic EL display. The display control circuit includes a GPU and a VRAM. Under the direction of the processor (26, 36), the GPU uses image generation data stored in the RAM to generate display image data in the VRAM for displaying various screens on the display, and outputs the generated display image data to the display.

[0020] The communication units (24, 34) include a communication circuit for connecting to the public line 5. The communication circuit is a wired communication circuit or a wireless communication circuit, and communicates with an external computer via the public line 5 according to instructions from the processors (26, 36).

[0021] The auxiliary storage units (25, 35) are composed of other non-volatile memories such as HDD, SSD, flash memory, EEPROM, etc., and store control programs and various data used by the processors (26, 36) to control the operation of the management server 2 or user terminals.

[0022] The auxiliary storage unit 35 of the user terminal 3 stores a user program 38 for executing various operations of the user terminal 3 in this system in response to operational inputs by the user, and user data 39 required for using this system.

[0023] The user program 38 and user data 39 are read out from the auxiliary storage unit 35 as necessary and stored (expanded) in the main storage unit 37 (RAM). The operation of the user terminal 3 is realized by the processor 36 executing the user program 38 expanded in the main storage unit 37 (RAM).

[0024] The user program 38 is configured to use at least a program that accesses the management server 2, but may be configured to use a dedicated user program. The user data 39 includes at least identification data (data such as a user ID) for identifying a user in this system.

[0025] 1 are merely examples, and are not limited thereto. For example, the management server 2 and the user terminal 3 may be provided with an interface (external storage medium connector) for reading data from various storage media (external storage media) such as an SD card or a USB memory, and writing data to the external storage medium.

[0026] Fig. 2 is a block diagram showing an example of the configuration of the virtual machine server 4. As shown in Fig. 2, the virtual machine server 4 includes two types of virtual machines 41 and 42. The virtual machines 41 and 42 are virtual computer systems (computer machines) that operate within the virtual machine server 4 using hardware elements such as a processor and a storage unit (memory such as RAM) of a server computer that constitutes the virtual machine server 4.

[0027] In this embodiment, the virtual machine server 4 includes an exclusive virtual machine 41 that runs for a specific user and a shared virtual machine 42 that runs for unspecified users. The exclusive virtual machine 41 may be realized by corresponding to specific hardware (exclusive hardware) in the virtual machine server 4, or may be realized by always reserving resources required for the exclusive virtual machine 41 from among the overall resources in the virtual machine server 4 without specifying the hardware.

[0028] In addition, in this embodiment, there are multiple dedicated virtual machines 41 and multiple shared virtual machines 42 within the virtual machine server 4, and a group of dedicated machines is formed by multiple dedicated virtual machines 41, and a group of shared machines is formed by multiple shared virtual machines 42.

[0029] In the computation processing system 1, the virtual machines 41 and 42 perform computations according to the trained models, and a certain result (inference result) is output for computation data (input) as an inference target. The trained model is a model that causes a computer to perform inference on unknown phenomena by an inductive approach using data, and by using the trained model, it is possible to cause a computer or the like to perform intellectual activities that a human being can perform.

[0030] Algorithms for trained models include, for example, regression, classification, cluster analysis (clustering), dimensionality reduction, and anomaly detection.

[0031] "Regression" refers to predicting the outcome of an unknown inference target using a "pattern" obtained from previously learned data. "Classification" refers to classifying an unknown inference target using a "classification method" obtained from previously learned data. "Classification" can be classified into two classes (two-class classification) and three or more classes (multi-classification). "Cluster analysis" refers to grouping multiple unknown inference targets (or detecting the data structure of multiple unknown inference targets) using "data similarity" obtained from previously learned data. "Dimension reduction" refers to detecting key factors by reducing the dimensions of the inference target data, i.e., reducing the number of dimensions of the data. "Anomaly detection" refers to detecting whether part or all of the inference target data is abnormal.

[0032] The trained model of the arithmetic processing system 1 of this embodiment includes one or more of the above-mentioned algorithms and can be used in various fields and for various purposes (applications), such as medical image diagnosis.

[0033] A trained model for medical image diagnosis is generated by learning from a large number of medical images. Medical images are images obtained through medical procedures such as X-rays, angiograms, computed tomography (CT), magnetic resonance imaging (MRI), scintigraphy, positron emission tomography (PET), or endoscopy. Using a trained model for medical image diagnosis makes it possible to detect lesion candidate regions, classify lesions as benign or malignant, and predict cases. The practical application process of such a trained model includes the process of generating the trained model (learning process) and the process of inference using the generated model (utilization process).

[0034] The process of generating a trained model will be briefly explained. First, data (hereinafter sometimes referred to as "original data") that is in line with the field and purpose of use of the trained model to be generated is processed into data suitable for machine learning (hereinafter sometimes referred to as "training data"). Next, a trained model is generated by subjecting the training data to machine learning that has not been subjected to machine learning (hereinafter sometimes referred to as "untrained model"). The trained model generated in this way includes parameters obtained by machine learning (hereinafter sometimes referred to as "trained parameters") and an inference program incorporating the trained parameters, and is used according to the field and purpose of use as described above. <Data Structure>

[0035] Fig. 3 is an explanatory diagram showing an example of data stored in the auxiliary storage unit 25 of the management server 2. Various data (related to the trained model) used in the calculation processing system 1 is stored (registered) in the auxiliary storage unit 25 of the management server 2. As shown in Fig. 3, an original data DB (database) 50, a learning data DB (database) 51, an untrained model DB (database) 52, a trained model DB (database) 53, a user management data DB (database) 54, a calculation data DB (database) 55, etc. are registered in the auxiliary storage unit 25 of the management server 2.

[0036] A plurality of original data that are the source of learning data are registered in the original data DB 50. The original data is, for example, image data, document data, and numerical data, which is primarily input (uploaded) by a user or an administrator of the arithmetic processing system 1 and converted and processed (primary processing) so that it can be processed by a computer. The original data is, for example, input from an external computer such as the user terminal 3 via the communication unit 24, or input from an external storage medium via an external storage medium connection unit.

[0037] In the learning data DB 51, the original data is processed (secondary processing) so as to be suitable for machine learning, and the learning data to which tag information (correct answer information) is set is registered. Examples of secondary processing include processing to remove missing values ​​and outliers from the original data. The process of setting tag information is sometimes called annotation setting. Furthermore, the learning data is linked to the corresponding original data (from which it was processed).

[0038] In the computation processing system 1 of this embodiment, there are a plurality of learning data with different variations. For example, even if the learning data are generated from the same original data, if the tag information is different (annotation settings are different), the learning data are data for the same field, the same type, and the same purpose (use), but the contents are different. In addition, the original data may be added or changed, and the learning data may be updated (changed). In this case, the learning data before the update and the learning data after the update are data with different variations. Moreover, the learning data before the update is linked to the original data before the update, and the learning data after the update is linked to the original data after the update. For example, in the case where the medical images of the lungs are taken periodically (for example, once a year) for learning data generated from the medical images of the lungs, if tag information of (being diagnosed with) lung cancer is set at a certain point in time, the learning data before the diagnosis of lung cancer and the original data linked thereto, and the learning data after the diagnosis of lung cancer and the original data linked thereto will be data of different variations. In this manner, in this embodiment, a plurality of learning data of different variations are stored in the learning data DB 51 in a distinguishable manner.

[0039] Moreover, the training data is registered as a dataset including multiple pieces of training data, and one dataset includes multiple pieces of training data that are related to each other depending on the field, type, purpose, etc. For example, when there are multiple medical images of the same type (e.g., X-ray images of lungs) taken during a certain period at the same medical facility, those images are registered as the same dataset.

[0040] Data of an unlearned model is registered in the unlearned model DB 52. However, some parameters (initial parameters) may be incorporated in advance in the unlearned model in accordance with the field of use and the purpose of use. Data of a trained model is registered in the trained model DB 53. Data of a newly registered trained model is registered in the trained model DB 53.

[0041] In the computation processing system 1 of the present embodiment, there are multiple trained models with different variations. For example, even if the same learning data is machine-learned, if the machine-learning conditions are different, the trained models will have different variations. Also, a trained model trained on learning data before the update and a trained model trained on learning data after the update will have different variations. In this way, multiple trained models with different variations are stored in the trained model DB 53 in a distinguishable manner.

[0042] 4 is an explanatory diagram showing an example of a data structure of trained model data (trained model data) 60. As shown in FIG. 4, the trained model data 60 includes at least execution data 61, model information data 62, training data 63, and change history data 64.

[0043] The execution data 61 is data for executing calculations according to the learned model in the virtual machines 41 and 42. The execution data 61 is typically data of an inference program incorporating learned parameters, but when the virtual machines 41 and 42 operate according to a general-purpose AI program, the program data is omitted, and the execution data may be data of learned parameters that can be processed by the general-purpose AI program.

[0044] The model information data 62 is data on information of a trained model (model information). The model information includes model identification information, management ID information, program language information, required library information, variation information, change time information, and changer information.

[0045] The model identification information is information for humans to identify the trained model, and is expressed as a string including words (phrases) indicating the field of use and purpose of use. The management ID information is information for managing the trained model in this system, and is expressed as a string including letters, numbers, and symbols. The program language information is information such as the name of the program language in which the trained model can be executed and the version of the program language. The required library information is information about the libraries required to execute the trained model. The variation information is information indicating the variation (version) of the trained model. For example, the version of the trained model is expressed in a distinguishable manner by a number (digit). The change time information is information about the time when the change was made after the trained model was generated (the change time). If multiple changes have been made to the trained model, the change time information includes information about the time of each change. The changer information is information about the user who made the change to the trained model, and includes information such as the name of the organization to which the user belongs and the user's name.

[0046] The learning data 63 is information about the learning data used in the process of generating a trained model. However, in this embodiment, the information about the learning data is expressed as information about a data set, not as information about individual learning data. The information about the learning data includes data identification information, origin information (generation origin information), data category ID information, data specification information, and data number information. Note that instead of or in addition to this information, the learning data itself may be used as the learning data 63.

[0047] The change history data 64 is data on information on the change history of the trained model. The change history information of the trained model includes information on the change history of the execution data 61 and information on the change history of the training data.

[0048] In addition, the trained model data 60 may be containerized (container-type virtualized) data that includes, in addition to the execution data 61, the model information data 62, the training data 63, and the change history data 64, application data for performing calculations according to the trained model and necessary library data.

[0049] Returning to Fig. 3, data (hereinafter sometimes referred to as "user management data") for managing users who use the computation system 1 is registered in the user management data DB54. The user management data includes data of user identification information (user identification information data), data of user authentication information (user authentication information data), data of user usage information (user usage information data), and data of user attribute information (user attribute information data), and the user identification information data, user identification information data, user usage information data, and user attribute information data of the same user are linked to each other.

[0050] The user identification information is information for identifying a user in the data processing system 1, such as an email address or a member ID used by the user. The user authentication information is information for performing user authentication in combination with the user identification information, such as a password. The user usage information is information such as the user's past usage record, a trained model registered by the user, training data registered by the user, and resources available to the user. The user attribute information is information indicating whether the user is a free member or a paid member, that is, information indicating the user's authority in the data processing system 1.

[0051] The calculation data DB 55 registers calculation data as an inference target when the virtual machines 41 and 42 perform calculations according to the learned model. The calculation data is data in a format corresponding to the original data, such as text data, numerical data, and image data.

[0052] Each of the learning data DB 51, the unlearned model DB 52, the trained model DB 53, and the calculation data DB 55 includes an exclusive area that is available only to specific users, and a shared area that is available to unspecified users.

[0053] Various data registered in the auxiliary storage unit 25 of the management server 2 is read out from the auxiliary storage unit 35 as necessary and loaded into the main storage unit 27 (RAM). The operation of the management server 2 is realized by the processor 26 executing the control program for the management server 2 stored in the main storage unit 27 using the data loaded into the main storage unit 27.

[0054] Fig. 5 is an explanatory diagram showing an example of a memory map 70 of the main memory unit 27 (RAM) of the management server 2. As shown in Fig. 5, the main memory unit 27 includes a program memory area 71 and a data memory area 72. The program memory area 71 stores control programs of the management server 2. The control programs include a learning data generation program 71A, a learning data management program 71B, a trained model generation program 71C, a trained model management program 71D, a trained model search program 71E, a learning data search program 71F, an original data search program 71G, an execution environment setting program 71H, and a virtual machine management program 71J.

[0055] The learning data generation program 71A is a program for performing secondary processing on the original data, setting annotations, and generating learning data (executing a learning data generation function). The learning data management program 71B is a program for registering new learning data and deleting already registered learning data (executing a learning data management function).

[0056] The trained model generation program 71C is a program for generating a trained model (performing a trained model generation function) by machine learning training data into an untrained model or an existing trained model.

[0057] The trained model management program 71D is a program for registering a generated trained model or deleting an already registered trained model (performing the trained model management function).

[0058] The trained model search program 71E is a program for accepting the specification of model search conditions for searching for trained models, and for searching for trained models that meet the model search conditions from among the trained models registered in this system (performing the trained model search function).

[0059] The learning data search program 71F is a program for receiving a designation of learning data search conditions for searching learning data, and for searching for learning data that meets the learning data search conditions from among the learning data registered in this system (exercising the learning data search function). However, the learning data is searched for in units of data sets.

[0060] The original data search program 71G is a program that accepts the specification of original data search conditions for searching original data, and searches for original data that meets the original data search conditions from among the original data registered in this system (performing the original data search function).

[0061] The execution environment setting program 71H is a program for setting an execution environment (executing an execution environment setting function) when the virtual machines 41 and 42 perform calculations according to the learned model.

[0062] The virtual machine management program 71J is a program for detecting the operating status of the virtual machines 41 and 42 and for allocating machines that perform computations (exercising a virtual machine management function).

[0063] Although not shown in the figure, the program memory area 71 also stores a communication program for communicating (sending and receiving) data with an external computer via the public line 5, an operation detection program for detecting user operation input to the input unit 22, a display program for generating display image data using image generation data stored in the RAM and outputting it to the display unit 23 (display), and programs for executing various functions provided by the management server 2.

[0064] The data storage area 72 also stores original data 72A, learning data 72B, unlearned model data 72C, learned model data 72D, user management data 72E, and calculation data 72F.

[0065] The original data 72A is newly input original data or original data read from the original data DB 50. The learning data 72B is newly generated learning data according to the learning data generation program 71A or learning data read from the learning data DB 51. The unlearned model data 72C is newly input unlearned model data or unlearned model data read from the unlearned model DB 52. The trained model data 72D is newly generated learning model data according to the trained model management program 71D or trained model data read from the trained model DB 53.

[0066] The user management data 72E is newly registered user management data, updated user management data, or user management data read out from the user management data DB 54.

[0067] The computation data 72F is newly registered computation data or computation data read from the computation data DB 55. However, the computation data 72F is only used for data management within the management server 2, and when a computation is performed, the computation data is deployed in the storage unit of the virtual machine server 40.

[0068] Although not shown in the figure, the data memory area 72 stores image generation data for generating display image data, operation data corresponding to user operation input, and other data necessary for executing the control program of the management server 2. <System operation example>

[0069] An operation example of the data processing system 1 will be described below with reference to Fig. 6 to Fig. 24. When the user terminal 3 accesses the management server 2 in response to an operational input from the user, various operation screens are displayed on the display unit 33 (display) of the user terminal 3. Each operation screen described below with reference to Fig. 6 to Fig. 24 is assumed to be displayed on the display of the user terminal 3.

[0070] 6 is an example of an operation screen that is displayed first when the user terminal 3 accesses the management server 2. The initial screen 100 includes a first user registration section (user entry section) 101 for registering user information, and a user login section 102 for a registered user to log in to the system. Each of the first user registration section 101 and the user login section 102 functions as a software key (operation button). Hereinafter, the software key will be referred to as a "button."

[0071] When the first user registration section 101 is operated, a first member registration screen 110 is displayed as shown in Fig. 7. The first member registration screen 110 is a screen for accepting input of user identification information, and includes an identification information input section 111 for inputting an email address or an arbitrary member ID, a name input section 112 for inputting the user's name, a date of birth input section 113 for inputting the user's date of birth, a gender input section 114 for inputting the user's gender, and a registration execution button 115.

[0072] The identification information input section 111, the name input section 112, the date of birth input section 113, and the gender input section 114 are text boxes (input fields) that accept text input. However, the date of birth input section 113 and the gender input section 114 may be pull-down menus.

[0073] On the first member registration screen 110, when the registration execution button 115 is operated with necessary information input in each of the input sections 111 to 114, user management data including the user identification information is registered in the user management data DB 54. Next, a password input screen for accepting password input is displayed, and when a password is input on the password input screen, the input password is registered as user authentication information linked to the user identification information. Once the user identification information and user authentication information are registered, the screen returns to the initial screen 100. A user whose user identification information and user authentication information are registered is treated as a registered user in this system.

[0074] When the user login section 102 is operated on the initial screen 100, a login screen 120 as shown in Fig. 8 is displayed. The login screen 120 is a screen for user authentication, and includes an identification information input section 121 that accepts input of user identification information, an authentication information input section 122 that accepts input of user authentication information, and a login execution button 123.

[0075] When the login execution button 123 is operated with necessary information input in each of the input sections 121 and 122, user authentication is performed. In the user authentication, if the combination of the user identification information input in the identification information input section 121 and the user authentication information input in the authentication information input section 122 matches the combination of the user identification information and the user authentication information registered in the user management data DB 54, the user authentication is successful and login is permitted.

[0076] When login is permitted, a main screen 130 is displayed as shown in Fig. 9. The main screen 130 displays the main menu of the system and operation buttons corresponding to each menu, and when each operation button is operated, the screen changes to the corresponding screen.

[0077] The main screen 130 includes a second user registration button 131, a user identification information display section 132, a membership level display section 133, a status display section 134, an execution environment setting button 135, a trained model search button 136, an annotation management button 137, a training data generation button 138, a trained model registration button 139, a trained model execution button 140, a calculation result comparison button 141, and a receipt issue button 142.

[0078] When the second user registration button 131 is operated, a second member registration screen 150 as shown in Fig. 10 is displayed. The second member registration screen 150 is a screen for registering information as a paid member (paid member information), and includes a payment method selection section 151, a card type selection section 152, a card information input section 153, and a registration execution button 154.

[0079] The payment method selection section 151 includes radio buttons for selecting either monthly payment (monthly payment) or annual payment (yearly payment), and the user can select either one of the payment methods. The card type selection section 152 allows the user to select the type (brand) of credit card using a pull-down menu. The card information input section 153 is a text box that accepts the input of card information (card number). The payment method is not limited to credit card payment, and other payment methods such as direct debit or electronic money payment may also be used.

[0080] When the payment method, card type, and card information are entered on the first membership registration screen 110 and the registration execution button 154 is operated, an examination is started to determine whether the payment is possible, and if it is determined that the payment is possible (if the examination is passed), the paid membership information is registered in the user management data DB 54.

[0081] Returning to FIG. 9, the user identification information display unit 132 displays the identification information of the currently logged-in registered user (hereinafter, sometimes simply referred to as "user"). The membership level display unit 133 displays information indicating the user's membership level, i.e., whether the user is a free member or a paid member. The status display unit 134 displays the current status information of the data processing system 1. The status information includes information such as the performance and usage time of the virtual machine, the number of models currently being trained, the number of annotations processed, the number of references and uses of trained models by the currently logged-in user, and a list of usage statuses of trained models by users other than the currently logged-in user.

[0082] The trained model search button 136 is available regardless of the membership level (even for free members). On the other hand, the execution environment setting button 135, the annotation management button 137, the training data generation button 138, the trained model registration button 139, the trained model execution button 140, the calculation result comparison button 141, and the receipt issue button 142 are available only to paid members. The operation buttons 135, 137 to 142 available only to paid members are displayed with mark images 135A, 137A to 142A indicating that they are available only to paid members. Furthermore, on the main screen 130 for free members, the operation buttons 135, 137 to 142 available only to paid members are disabled and cannot be operated.

[0083] When the execution environment setting button 135 is operated, an execution environment setting screen 160 is displayed as shown in Fig. 11. The execution environment setting screen 160 is a screen for setting an execution environment when the virtual machines 41 and 42 perform calculations according to the learned model, and includes a memory area setting unit 161, a computing capacity setting unit 162, an allocation condition setting unit 163, a registered environment display unit 164, and a usage history display unit 165.

[0084] The storage area setting unit 161 is provided to set (change or add) a storage area (occupied area) available to the user, which stores data when the virtual machines 41 and 42 perform calculations according to the learned model. The storage area setting unit 161 displays capacity information of the user's current occupied area and capacity information of free area (addable area). The storage area setting unit 161 allows the user to select a capacity (additional capacity) for additionally expanding the occupied area using a pull-down menu, with the capacity of the addable area being the upper limit.

[0085] When additional capacity is selected, the cost according to the additional capacity is displayed. The cost of adding space is displayed in monthly and annual payments, and the purchase unit can be selected from a pull-down menu.

[0086] The storage area setting section 161 is provided with a purchase button 166. When the purchase button 166 is operated with an additional capacity and a purchase unit selected, the user's occupied area is additionally expanded according to the additional capacity.

[0087] The computing power setting unit 162 is provided to set the computing power when the virtual machines 41 and 42 perform calculations according to the learned model. In the computing power setting unit 162, the OS, memory capacity, number of CPU cores and frequency, and number of GPU cores and frequency during calculation can be selected from pull-down menus.

[0088] The reservation condition setting unit 163 is provided to set conditions for reserving the virtual machines 41, 42. The reservation condition setting unit 163 includes a radio button for selecting whether to reserve an exclusive virtual machine 41 or a shared virtual machine 42. The reservation condition setting unit 163 displays the cost in the case of monthly payment and the cost in the case of annual payment according to the type of virtual machine to be reserved, and allows the purchase unit to be selected by a pull-down menu. The reservation condition setting unit 163 also includes a purchase button 167, and when the purchase button 167 is operated, a virtual machine 41 of the selected type is reserved.

[0089] The registered environment display section 164 displays a list of information on execution environments registered by the user. The information on the execution environments includes information such as the management number, computing power (memory, number of CPU cores, number of GPU cores), virtual machine allocation conditions, unit price, usage status (usage situation), cumulative usage time, and elapsed execution time. The registered environment display section 164 also has an available environment display button 168 for displaying information on available execution environments, and a registration history information display button 169 for displaying information on execution environments other than available execution environments (registration history information).

[0090] A list of information on the user's usage history is displayed in the usage history display section 165. The information on the usage history includes information such as the time (date), management number, billed cost, purchase date, payment completion date, calculation time, payment method (monthly or annual payment), contract details (monthly, annual, or pay-per-use), and unit price when the system was used (calculation was performed) in the past.

[0091] Returning to Fig. 9, when the trained model search button 136 is operated, a trained model search screen 170 as shown in Fig. 12 is displayed. The trained model search screen 170 is a screen for searching trained models registered in the computation processing system 1. The trained model search screen 170 includes a creator condition setting section 171, an analysis target condition setting section 172, an application field condition setting section 173, an other condition setting section 174, a search execution button 175, a result display section 176, an overview display button 177, a confirmation registration button 178, a detail display button 179, and a verification information display button 180.

[0092] The creator condition setting unit 171 is provided to set creator conditions, which are part of the conditions (model search conditions) for trained models to be searched, according to the creator. This creator condition setting unit 171 is provided with check boxes according to the attributes of the creator. If a check mark is displayed, the model is subject to search, and if a check mark is not displayed, the model is not subject to search. Creator attributes include the user himself / herself and creators other than the user. However, among trained models created by creators other than the user, the models to be searched are limited to publicly available trained models.

[0093] The analysis target condition setting section 172 is provided for setting analysis target conditions, which are part of the model search conditions, according to the type of analysis target, and check boxes are provided according to the type of analysis target. Analysis targets include images, numerical values, text, etc.

[0094] The application field condition setting unit 173 is provided to set application field conditions, which are part of the model search conditions, according to the type of application field, and a check box is provided according to the type of application field. The application fields include the medical field, the image recognition field, and the general-purpose field. In addition, the medical field and the general-purpose field are subdivided. For example, the medical field includes the lung image diagnosis field, the fundus image diagnosis field, the dementia prediction field, and the case prediction field, and the general-purpose field includes the multi-classification field and the linear prediction field.

[0095] The other condition setting unit 174 is provided for setting other conditions of the model search conditions, and each condition can be selected from a pull-down menu or a text box. The other conditions include the OS of the trained model, the type of program, the type of algorithm, the year of the first release, the year of the latest release, and keywords related to the trained model.

[0096] The creator condition setting section 171, the analysis target condition setting section 172, the application field condition setting section 173, and the other condition setting section 174 function as a search condition designation means for designating model search conditions in accordance with user operation input. When the search execution button 175 is operated, a trained model is searched for according to the model search conditions. The search results are displayed in a list on the result display unit 176.

[0097] The search results displayed in the result display unit 176 display various information about trained models that match the model search conditions. The trained model information displayed in the result display unit 176 includes the publisher (user who published the trained model) ID, the name and ID of the trained model, the OS and program type, the algorithm type, the application field, the source code type, the version number, and information on the release date. The result display unit 176 may display trained models with different variations of the same name, but they are distinguishable because the trained model ID, version number, release date, etc. are different.

[0098] When the summary display button 177 is operated, summary information of the trained model selected by the user is displayed in the result display unit 176, when the detail display button 179 is operated, detailed information (containing at least more information than the summary information) of the trained model selected by the user is displayed in the result display unit 176, and when the verification information display button 180 is operated, verification information (whether or not verification has been performed and the reliability score, etc.) of the trained model selected by the user is displayed in the result display unit 176. However, the detail display button 179 and the verification information display button 180 are available only to paid members.

[0099] When the confirmation registration button 178 is operated, a version confirmation registration screen 190 as shown in Fig. 13 is displayed. The version confirmation registration screen 190 is a screen for confirming the version information of the trained model selected by the user and for registering the use of the trained model for each version. The version confirmation registration screen 190 includes a publisher information display section 191 that displays the publisher ID, a model name display section 192 that displays the name of the trained model, a model ID display section 193 that displays the ID of the trained model, a change history display section 194, a model change history confirmation registration button 195, and a data change history confirmation button 196.

[0100] The information of the trained model is displayed by version in a list in the change history display unit 194. The information of the trained model displayed in the change history display unit 194 includes the version number, the change contents of the program, the identification information of the used learning data, the registration status, and the price information.

[0101] When a model change history confirmation and registration button 195 is operated with any version selected in the change history display section 194, a model change history confirmation screen 200 as shown in FIG. 14 is displayed.

[0102] The model change history confirmation screen 200 is a screen for confirming the change history of the trained model of the selected version and registering its use. The model change history confirmation screen 200 includes a version number display section 201 that displays the version number, a summary display section 202 that displays a summary of the changes made when the version was generated, a price display section 203 that displays the price of the trained model of the version, a details display section 204, a license confirmation button 205, and a use registration button 206.

[0103] Specific changes made when the version was generated are displayed in the detail display section 204. For example, the detail display section 204 displays an explanation of the differences (changes, additions, and deletions) from the version before the change, or a comment from the registrant.

[0104] When the license confirmation button 205 is operated, license information for the trained model of that version is displayed. When the usage registration button 206 is operated, the trained model of that version is registered for usage. In this way, the computation processing system 1 of this embodiment can search for and register not only the latest version but also various variations (versions) of trained models, thereby meeting the diverse needs of users.

[0105] Returning to FIG. 13, when a version is selected in the change history display section 194 and the data change history confirmation button 196 is operated, a data change history confirmation screen 210 as shown in FIG. 15 is displayed.

[0106] The data change history confirmation screen 210 is a screen for confirming the change history of the learning data used when generating the trained model of the selected version. The data change history confirmation screen 210 includes a data name display section 211 that displays the name of the learning data, a data identification information display section 212 that displays the identification information (identification ID) of the learning data, a data type display section 213 that displays the type of the learning data, a change history display section 214, and a details confirmation button 215.

[0107] Information on when the learning data was changed is displayed for each version in a list in the change history display section 214. For example, the information on when the learning data was changed includes information on the version number, resolution, time or number of pages, the type of data change (first edition, addition, deletion), and the registration date (generation date or update date).

[0108] When a detail confirmation button 215 is operated with any version selected in the change history display section 214, a change data confirmation screen 220 as shown in FIG. 16 is displayed.

[0109] The change data confirmation screen 220 is a screen for confirming the changes made when the selected version of the learning data was generated. The change data confirmation screen 220 includes a version number display section 221 that displays the version number, and a change content display section 222 that shows the specific change content. The change content display section 222 displays, for example, an explanation of the differences (changes, additions, deletions) from the version before the change, or a comment from the registrant.

[0110] As described above, in Fig. 12 to Fig. 16, a method for searching for a trained model registered in the arithmetic processing system 1 has been described, but in a similar manner, training data registered in the arithmetic processing system 1 can also be searched for. Briefly, a training data search screen is displayed, designation of training data search conditions for searching training data is accepted, training data that meets the training data search conditions is searched from a plurality of registered training data, and information on the hit training data is displayed. At this time, if there is variation data of the hit training data, information on the variation data is also displayed in addition to the hit training data.

[0111] Returning to Fig. 9, when the annotation management button 137 is operated, an annotation management screen 230 as shown in Fig. 17 is displayed. The annotation management screen 230 is a screen for managing tag information set in learning data (for each data set). The annotation management screen 230 includes an unset data display section 231, an editing data display section 232, a set data display section 233, a self-setting button 234, and an outsource button 235.

[0112] The unset data display section 231 displays a list of information on unset data. Unset data refers to data that has been subjected to secondary processing of original data and to which no tag information has been set. Information on unset data includes, for example, information on the data name, data ID, and data volume (data size). Also, a data confirmation button 236 is displayed corresponding to the unset data display section 231. When the data confirmation button 236 is operated with any unset data selected in the unset data display section 231, specific details of the unset data are displayed.

[0113] The data-in-edit display section 232 displays a list of information on learning data (data-in-edit) for which tag information is being edited. The information on the data-in-edit includes, for example, the data name, data ID, data volume, editing category (own or outsourced), and information on the outsourcee. In addition, a status check button 237 is displayed corresponding to the data-in-edit display section 232. When the status check button 237 is operated with any data-in-edit selected in the data-in-edit display section 232, the status of the data-in-edit (such as the progress of the editing work) is displayed.

[0114] The set data display section 233 displays a list of data for which tag information has been set (set data), that is, information on learning data. The information on the set data includes, for example, the data name, data ID, data volume, editing category, and information on the entrusted party. Also, a data confirmation button 238 is displayed corresponding to the set data display section 233. When the data confirmation button 238 is operated with any set data selected in the set data display section 233, the specific contents of the set data are displayed. The set data is registered in the learning data DB 51 as learning data.

[0115] When the self setting button 234 is operated, a self editing screen 240 as shown in Fig. 18 is displayed. The self editing screen 240 is a screen for the user to set annotations for learning data by himself / herself, and includes an unset data display section 241, an editing data display section 242, and a set data display section 243.

[0116] The unset data display section 241 displays a list of information on unset data, the data being edited display section 242 displays a list of information on data being edited, and the set data display section 243 displays a list of information on set data. The data displayed in each of the data display sections 241, 242, and 243 can be individually selected, and by selecting any of the data, the contents of the data are displayed, or a work screen for setting annotations (annotation assignment screen) is displayed.

[0117] It is also possible to set annotations again for the set data displayed in the set data display area 243. In this case, the set data before and after the re-setting of annotations are treated as different data.

[0118] Returning to Fig. 17, when the outsourcing button 235 is operated, an outsourcing management screen 250 as shown in Fig. 19 is displayed. The outsourcing management screen 250 is a screen for managing learning data for which annotation setting has been outsourced. The outsourcing management screen 250 includes an unconfigured data display section 251, an outsourcing destination selection section 252, a quotation request data display section 253, an ordered data display section 254, a delivered data display section 255, and a memory area confirmation section 256.

[0119] Information on unset data is displayed in a list in the unset data display section 251. Also, a data confirmation button 257 is displayed corresponding to the unset data display section 251. When the data confirmation button 257 is operated with any unset data selected in the unset data display section 251, the specific content of the unset data is displayed.

[0120] The outsourcing selection section 252 displays a list of information on outsourcing candidates (external businesses) to which outsourcing can be performed. The information on the outsourcing candidates includes, for example, outsourcing ID, outsourcing name (business name), service name, and price information. Also, corresponding to the outsourcing selection section 252, a specification entry field 258 and a quote request button 259 are displayed. When unset data is selected and one of the outsourcing candidates is selected in the outsourcing selection section 252, a desired delivery date is selected from the pull-down menu, and the quote request button 259 is operated, a quote is requested to the selected outsourcing candidate. At this time, specification information (transfer information) related to the creation of the quote can be entered in the specification entry field 258.

[0121] The quotation request data display section 253 displays a list of information on cases for which a quotation is being requested. The information on cases for which a quotation is being requested includes, for example, the name of the contractor, specifications, unset data ID, quotation deadline, status, and price information. In addition, a quotation processing section 260 is provided corresponding to the quotation request data display section 253. The quotation processing section 260 includes an inspection content confirmation button for confirming the inspection content of each case, a cancel button for canceling the inspection of each case, a pull-down menu for setting a delivery date, and an order button for ordering annotation setting. When a case for which a quotation is being requested is selected and the order button is operated, a request for annotation setting is made to the selected contractor.

[0122] The ordered data display section 254 displays a list of information on cases for which annotation setting is being requested (ordered). Information on ordered cases includes, for example, the name of the contractor, specifications, unset data ID, delivery date, status (ordered or canceled), and price information. Also, an order processing section 261 is provided corresponding to the ordered data display section 254. The order processing section 261 includes an order content confirmation button for confirming the order content of each case, and a cancellation button for canceling the request for each case.

[0123] The delivered data display section 255 displays a list of information on the delivered cases. The information on the delivered cases includes, for example, the name of the contractor, specifications, data ID, delivery date, delivery date, and price information. In addition, a delivery processing section 262 is provided corresponding to the delivered data display section 255. The delivery processing section 262 includes a delivery content confirmation button for confirming the content of the delivered data including the learning data to which tag information is set, a details confirmation button for confirming the details of the delivered cases, and an inspection and settlement button for inspecting and settling the delivered cases. When the delivered cases are inspected, the learning data included in the delivered data is registered in the learning data DB 51.

[0124] Information on the user's occupied area is displayed in the storage area confirmation section 256. Also, corresponding to the storage area confirmation section 256, a capacity change button 263 for changing the capacity of the user's occupied area is provided.

[0125] Returning to Fig. 9, when the learning data generation button 138 is operated, a learning data management screen 270 as shown in Fig. 20 is displayed. The learning data management screen 270 is a screen for managing the learning data registered in the learning data DB 51. The learning data management screen 270 includes a learning data display section 271 and a user confirmation button 272.

[0126] The learning data display section 271 displays a list of information on the learning data registered in the learning data DB 51. The learning data information includes, for example, data name, data ID, data volume, annotation setting category (own or outsourced), disclosure category (public or private), number of uses, and price information. When the user confirmation button 272 is operated with learning data with a disclosure category of "public" selected in the learning data display section 271, a user confirmation screen 280 as shown in FIG. 21 is displayed.

[0127] The user confirmation screen 280 is a screen for confirming the user of the published learning data. The user confirmation screen 280 includes a data identification information display section 281 that displays the learning data identification information including the name and ID of the learning data, a usage information display section 282, and a user contact button 283.

[0128] A list of usage information of the learning data is displayed in the usage information display section 282. The usage information of the learning data is displayed for each usage occasion, and includes information such as the user ID, the date of use, the evaluation score, the purpose, and whether or not there is a comment. When the user contact button 283 is operated with a usage occasion with a comment "present" being selected in the usage information display section 282, a contact screen 290 as shown in FIG. 22 is displayed.

[0129] The contact screen 290 is a screen for communicating with a user of the learning data. The contact screen 290 includes a user information display section 291 that displays the user's identification information, including the name or ID of the user of the learning data, a message display section 292, a message input section 293, and a message send button 294.

[0130] The message display section 292 displays messages from the user and messages from the currently logged-in user (the user who provides the learning data) in a chat format. The message input section 293 is a text box that accepts input of a new message to be sent. When the message send button 294 is operated, the message input in the message input section 293 is sent and is additionally displayed in the message display section 292.

[0131] Returning to Fig. 9, when the trained model registration button 139 is operated, a trained model registration screen 300 as shown in Fig. 23 is displayed. The trained model registration screen 300 is a screen for registering a trained model. The trained model registration screen 300 includes a registration unit 301, a registered model display unit 302, and a verification unit 303.

[0132] The registration unit 301 is provided to set the learning conditions (generation conditions) of a desired trained model and to register a trained model generated according to the learning conditions. The registration unit 301 is provided with a text box for receiving input of the name of the trained model, and also allows multiple learning conditions to be selected from a pull-down menu. The learning conditions include the OS, program type, application field, source code, version number, price, disclosure classification, and data learning mode.

[0133] In addition, a learning data search button for searching and registering learning data to be used may be provided. When the learning data search button is operated, a learning data search screen is displayed, and learning data can be searched for by specifying a learning data search condition. In addition to the hit learning data, information on variation data is also displayed on the learning data search screen, and each learning data can be individually registered for use. For example, when learning data after a lung cancer diagnosis is hit in a search, learning data before the lung cancer diagnosis may be displayed as a variation of the hit learning data, and the learning data before the lung cancer diagnosis can be registered for use to generate a trained model for detecting the presence or absence of a sign of lung cancer.

[0134] In addition, the registration unit 301 includes an upload button 304 for uploading a program from an external computer or an external storage medium, etc., and a registration button 305 for generating and registering a trained model based on the learning conditions set in the registration unit 301.

[0135] The registered model display unit 302 displays a list of information on the trained models (registered trained models) that the user has registered for use among the trained models registered in the trained model DB 53. The information on the registered trained models includes information such as a model ID, a model name, an OS, a program type, a type of algorithm, an application field, a source code, a version number, a registration date, a verification date, a verification result, an automatic version upgrade content, and a training data ID. The registered trained models displayed on the registered model display unit 302 can be individually selected. In addition, a registered model processing unit 306 is provided corresponding to the registered model display unit 302. The registered model processing unit 306 includes a detailed content confirmation button for confirming the detailed content of the registered trained model, a delete button for deleting (deregistering) the registered trained model, a training data display button for displaying the content of the training data linked to the registered trained model, and an error confirmation button for checking an error in the registered trained model.

[0136] The verification unit 303 is provided to verify the registered trained model. The verification unit 303 includes a verification execution unit 307 that executes the verification, and a data confirmation unit 308 that confirms various data. The verification execution unit 307 includes an execution button that executes the verification of the registered trained model, and a verification result reference button that displays the verification result. The data confirmation unit 308 includes a learning data display button that displays the contents of the learning data linked to the registered trained model, and a verification data display button that displays verified data (verification data).

[0137] Returning to Fig. 9, when the trained model execution button 140 is operated, a calculation processing screen 310 as shown in Fig. 24 is displayed. The calculation processing screen 310 is a screen for performing various processes related to calculations using the trained model. The calculation processing screen 310 includes a calculation data display section 311, a registered model display section 312, an available machine display section 313, a waiting job display section 314, a job in process display section 315, a processing result display section 316, a processing result confirmation button 317, and a trained model registration button 318.

[0138] The calculation data display section 311 displays a list of information on the calculation data registered in the calculation data DB 55. The information on the calculation data includes information such as a data ID, a data name, a data type, the number of data items, and a data size. In addition, a calculation data processing section 319 is provided corresponding to the calculation data display section 311. The calculation data processing section 319 includes a storage area change button for changing the user's occupied area, an upload button for uploading calculation data from an external computer or an external storage medium, and a selection button for selecting calculation data to be used for calculation.

[0139] The registered model display unit 312 displays a list of information on registered trained models that the user has registered for use. The trained model information includes information such as the model ID, model name, trained model OS, program type, algorithm type, application field, version number, and training data ID. In addition, a registered model processing unit 320 is provided corresponding to the registered model display unit 312. The registered model processing unit 320 includes a detailed content display button for displaying detailed content of the registered trained model, and a selection button for selecting a trained model to be used for calculation.

[0140] The available machine display section 313 displays a list of information on the currently available virtual machines 41, 42. The information on the virtual machines 41, 42 includes information such as the machine number, memory capacity, number of CPU cores, reservation conditions (by month or year), operation state (on standby, unused, etc.), cumulative usage time, and elapsed execution time. The virtual machines 41, 42 displayed on the available machine display section 313 can be individually selected. In addition, a computation reservation button 321 is provided corresponding to the available machine display section 313. When the computation reservation button 321 is operated by specifying an execution date and execution time from a pull-down menu, a computation job is reserved to perform a computation using the selected computation data according to the selected trained model.

[0141] The queued job display section 314 displays information on computation jobs that have been reserved but not yet executed. The processing job display section 315 displays information on computation jobs that are being executed. The computation job information includes computation data identification information (name of computation data), scheduled execution date and time, virtual machine identification information (scheduled execution machine number), and trained model identification information (trained model ID), etc. Corresponding to the queued job display section 314, a cancel button 322 for canceling the reservation of an unexecuted computation job is provided.

[0142] The processing result display section 316 displays a list of information on the processing results of the executed computation jobs. The information on the processing results of the computation jobs includes information such as computation data identification information (name of computation data), scheduled execution date and time, virtual machine identification information (scheduled execution machine number), learned model identification information (learned model ID), and an overview of the processing results (whether it ended normally or an error occurred). The processing results displayed on the processing result display section 316 can be selected individually for each computation job, and when a processing result is selected and a processing result confirmation button 317 is operated, the specific contents of the processing results are displayed. When the trained model registration button 318 is operated, a trained model registration screen 300 as shown in FIG. 23 is displayed.

[0143] Returning to Fig. 9, when the calculation result comparison button 141 is operated, the calculation processing results can be compared. Also, when the receipt issue button 142 is operated, a receipt for use of the present system can be issued. <Relearning when registering updated data>

[0144] As described above, the registration of learning data and the registration of trained models are basically performed manually by the user. However, when learning data is registered, the learning data of a previous version (pre-update data) of the learning data may already be registered in the trained model DB 53. In this case, the learning data to be registered is update data for the pre-update data already registered. When update data is registered, the pre-update data and the update data are registered in a manner that allows them to be distinguished from each other.

[0145] Furthermore, if there is a trained model that uses pre-update data (trained model before update), the trained model before update is automatically re-trained with the update data, and the trained model after update is automatically generated. Figure 25 is a flowchart of the re-learning operation of the management server 2. The re-learning operation will be described below with reference to Figure 25.

[0146] First, when new learning data is registered (step S1), the learning data DB 51 is referenced (step S2) and it is determined whether the learning data registered in step S1 is updated data (step S3). That is, it is determined whether learning data of a version previous to the learning data registered in step S1 exists.

[0147] If the learning data registered in step S1 is not update data (step S3: NO), it is registered as new learning data in the learning data DB 51 (step S4). On the other hand, if the learning data registered in step S1 is update data (step S3: YES), it is determined whether there is a history of use of the pre-update data, that is, whether there is a pre-update trained model that uses the pre-update data (step S5).

[0148] If there is a usage history of the pre-update data (step S5: YES), data of the pre-update trained model using the pre-update data is read out (step S6), an automatic re-learning process is executed on the pre-update trained model using the update data (step S7), the new trained model generated by the automatic re-learning process is registered in the trained model DB53 distinguished from the pre-update trained model (step S8), the update data is registered in the training data DB51 distinguished from the pre-update data (step S9), and the re-learning operation is terminated. Also, if there is no usage history of the pre-update data (step S5: NO), the update data is registered in the training data DB51 distinguished from the pre-update data (step S9), and the re-learning operation is terminated.

[0149] With the above configuration and operation, multiple trained models are generated through machine learning under mutually different conditions, and each of the data for the multiple trained models is linked to the learning data used in the machine learning and registered in a distinguishable manner, so that the data of the trained models can be managed to meet the diverse needs of users, improving user convenience.

[0150] In addition, each of the data of multiple trained models that are registered in a distinguishable manner can be searched according to model search conditions specified by the user, thereby improving user convenience.

[0151] Furthermore, since each of the data for the multiple trained models includes learning history data and learning data used in machine learning, information on the learning history data and learning data used in machine learning for the trained models found in the search can be presented to the user, thereby improving user convenience.

[0152] Furthermore, multiple pieces of training data are generated under mutually different conditions, and each of the multiple pieces of training data is registered in a distinguishable manner, and the trained model is generated using at least one of the multiple pieces of training data, and each of the multiple pieces of trained model data is linked to the training data used in the machine learning and registered in a distinguishable manner, so that the trained model data can be managed to meet the diverse needs of users, thereby improving user convenience.

[0153] Furthermore, each of the multiple learning data registered in a distinguishable manner can be searched according to learning data search conditions specified by the user, thereby improving user convenience.

[0154] When newly registered training data is update data corresponding to already registered training data, the trained model before the update, which was generated using the training data corresponding to the update data, is re-trained with the update data, so that the latest version of the trained model is automatically generated, thereby improving user convenience.

[0155] The present invention is not limited to the present embodiment, and may be embodied in various other ways. Furthermore, the screens and specific configurations described in the above embodiment are merely examples, and may be modified as appropriate according to the actual product.

[0156] For example, in the above-described embodiment, the various operation screens of the data processing system 1 are displayed on the display unit 33 of the user terminal 3, but the various operation screens may also be displayed on the display unit 23 of the management server 2. [Industrial Applicability]

[0157] This invention can be used in industries where a certain result is output for an input by performing calculations using a trained model using machine learning. [Explanation of symbols]

[0158] 1. Data management system 2. Management server 21...Control unit 22...Input section 23...Display section 24…Communications Department 25…Auxiliary storage section 3. User terminal 31...Control unit 32...Input section 33…Display section 4. Virtual machine server 41…Occupied Virtual Machine 42…Shared Virtual Machine 5…Public line 50…Original data DB 51…Learning data DB 52…Unlearned model DB 53…Trained model data DB 54...User management data DB 60…Trained model data 61...Execution data 62…Model information data 63…Learning data 64…Change history data

Claims

1. A learning data acquisition means for acquiring learning data to be used in machine learning; A first storage means for storing the learning data; A trained model generation means for generating a plurality of trained models having different variations by using training data of the same type or purpose in the medical field; A second storage means for storing each of the plurality of trained model data in a distinguishable manner by linking the data to the training data used in machine learning, The trained model generating means generates a trained model before a diagnosis using learning data before a certain diagnosis and a trained model after the diagnosis using learning data after the diagnosis as trained models of different variations. Data management system.

2. A model search condition designation means for accepting designation of model search conditions for searching the trained model; A model search means for searching for a trained model that meets the model search conditions from the trained model data stored in the second storage means; A model information presentation means for presenting information on trained models found by the search in the model search means to a user. The data management system according to claim 1.

3. Each of the plurality of trained model data includes learning history data and learning data used in machine learning, The model information presentation means presents to the user data on the learning history of the trained model found by the search and information on the learning data used in the machine learning. The data management system according to claim 2.

4. the learning data acquisition means acquires a plurality of learning data having different variations; the first storage means stores the plurality of pieces of learning data in a distinguishable manner; The trained model generation means generates the trained model using at least one of the plurality of training data, The second storage means associates each of the plurality of trained model data with the learning data used in the machine learning and stores the data in a distinguishable manner.

4. The data management system according to claim 1, 2 or 3.

5. a learning data search condition designation means for accepting designation of learning data search conditions for searching the learning data; a learning data search means for searching the plurality of learning data stored in the first storage means for learning data that meets the learning data search condition; A learning data information presentation means is provided for presenting to a user information on the learning data found by the search in the learning data search means.

5. The data management system according to claim 4.

6. a determination means for determining, when the learning data is acquired by the learning data acquisition means, whether the learning data is update data corresponding to the learning data stored in the first storage means; and a re-learning means for re-learning, when it is determined that the learning data acquired by the learning data acquisition means is the update data, a trained model before the update, which was generated using the learning data corresponding to the update data, with the update data.

6. The data management system according to claim 4 or 5.

7. Obtain learning data for machine learning, storing the learning data in a storage unit; Using training data of the same type or purpose in the medical field, multiple trained models with different variations are generated, A pre-diagnosis trained model using learning data before a certain diagnosis and a post-diagnosis trained model using learning data after the diagnosis are generated as trained models of different variations, The data of the plurality of trained models are linked to the learning data used in the machine learning and stored in the storage unit in a distinguishable manner. Data management methods.

8. Computer, A learning data acquisition means for acquiring learning data to be used in machine learning; A first storage means for storing the learning data in a storage unit; A trained model generation means for generating a plurality of trained models having different variations by using training data of the same type or purpose in the medical field; The second storage means functions as a second storage means for linking each of the plurality of trained model data with the learning data used in machine learning and storing the data in the storage unit in a distinguishable manner; The trained model generating means generates a trained model before a diagnosis using learning data before a certain diagnosis and a trained model after the diagnosis using learning data after the diagnosis as trained models of different variations. Data management programs.

Images