A device for providing protocols and a method for providing protocols, a device for generating evaluation models and a method for generating evaluation models, a training method, and a program.

Abstract

To provide a device and method for providing a protocol that is good for a worker by varying the number of work processes set as a protocol depending on the name of work, the worker who performs the work, and an evaluation level of the worker, an evaluation model generation device and method, a training method, and a program.SOLUTION: There is provided a work procedure in a bio-meister system. Data regarding the name of work and contents of the work performed by a worker are acquired. An evaluation level is determined from the contents of the work acquired by acquisition means using an evaluation model in which work regulations are associated with lower nodes, which is machine learned using data regarding contents of work performed in the past and evaluation results of the contents of the work by a manager as teacher data. The work regulations associated with the determined evaluation level are extracted. The extracted work regulations are configured as a protocol according to the order of the work defined by the name of the work, and the configured protocol is displayed.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention is to improve the working level of workers in various operations, and relates to a protocol (working procedure) providing device, a protocol providing method, an evaluation model generation method, an evaluation model generation device, a training method, and a program.

Background Art

[0002] Generally, there are limitations in acquiring and improving skills related to tests and inspections in biochemistry, molecular biology, etc. only through classroom learning. Therefore, experiments, practical training, etc., where actual hands-on work is carried out, have been conducted.

[0003] In such experiments and practical training, the work performed by the worker according to a pre-specified protocol (working procedure) is evaluated based on whether it is carried out as per the pre-specified protocol. The worker then repeats the work according to the pre-specified protocol based on the evaluation result to improve their skills.

[0004] However, in this pre-specified protocol, usually the work content is set in detail. In that case, for a worker with a certain working level, it may be difficult to carry out the work, or for a certain worker, it may be assumed that the work is cut short. It is not always the case that a protocol with detailed settings is a good protocol for that worker.

[0005] Also, on the premise that a good protocol is different for different workers as described above, even if work is continuously carried out based on this pre-specified protocol (i.e., even if work is continuously carried out based on the same protocol), it may not be possible to improve the working level of the worker early enough to suit the purposes of the worker and / or the administrator.

[0006] Therefore, in order to improve the work level of workers as quickly as possible, it is necessary to present a good protocol for each worker, and this good protocol should be structured by varying the number of work steps set in the protocol depending on the worker (or group) performing the work and the worker's skill level. [Overview of the project] [Problems that the invention aims to solve]

[0007] The present invention has been made in view of the above-mentioned conventional problems, and its objective is to improve the work level of workers at an early stage in a manner that suits the objectives of workers and / or managers, by varying the number of work steps set as a protocol according to the name of the work, the worker (or group) performing the work, and the evaluation level of the worker, and by presenting a protocol that is suitable for that worker. [Means for solving the problem]

[0008] In other words, the protocol provisioning device of the present invention is characterized by comprising: an acquisition means for acquiring data relating to the name of a task performed by a worker and the content of the task performed by the worker; a determination means for determining an evaluation level from the data relating to the content of the task acquired by the acquisition means, using an evaluation model that has been machine-trained using data relating to the content of previously performed tasks and the evaluation results of the content of the task by a manager as training data, wherein lower nodes are indicated as evaluation levels and work regulations are associated with the lower nodes; an extraction means for extracting work regulations associated with the evaluation level determined by the determination means; a configuration means for configuring the work regulations extracted by the extraction means as a protocol in accordance with the order of tasks defined by the name of the task; and a display control means for displaying the protocol configured by the configuration means on an external display device.

[0009] Furthermore, the method for generating an evaluation model of the present invention includes an acquisition step of acquiring data on the content of work performed in the past and the evaluation results of the content of said work by a manager as training data, and a generation step of generating an evaluation model in which data on the content of work is used as input values ​​and an evaluation level derived from said input values ​​is used as output values ​​by performing machine learning using said training data, wherein the work procedure is evaluated in accordance with the number of work procedures that were performed by workers with a predetermined work level and that were performed in a similar manner in the content of work performed in the past.

[0010] Furthermore, the training method of the present invention includes: a first acquisition step of acquiring data relating to the name of the work performed by the worker, the work level, and the content of the work performed by the worker; a first extraction step of extracting work regulations associated with the evaluation level corresponding to the work level acquired by the acquisition means from an evaluation model that has been machine-trained using data relating to the content of previously performed work and the evaluation results of the content of said work by a manager as training data, wherein lower nodes are indicated as evaluation levels and work regulations are associated with said lower nodes; a first configuration step of configuring the work regulations extracted in the first extraction step as a protocol according to the order of work defined by the name of the work; and the protocol configured in the configuration step externally The system is characterized by including: a first display control step of displaying on a display device of the unit; a second acquisition step of acquiring data relating to the content of work performed by the worker in accordance with the protocol displayed in the first display control step; a determination step of determining an evaluation level from the data relating to the content of work acquired in the second acquisition step using the evaluation model; a second extraction step of extracting work regulations associated with the evaluation level determined in the determination step; a second configuration step of configuring the work regulations extracted in the second extraction step as a protocol in accordance with the order of work defined by the name of the work; and a second display control step of displaying the protocol configured in the second configuration step on the external display device. [Effects of the Invention]

[0011] By varying the number of work steps set as a protocol based on the name of the task, the worker (or group) performing the task, and the worker's performance level, and by presenting a protocol that is suitable for that worker, it is possible to improve the worker's performance level early on in a way that meets the objectives of the worker and / or manager. [Brief explanation of the drawing]

[0012] [Figure 1] This is a schematic block diagram of the Biomeister system. [Figure 2] This is a functional configuration diagram of the BioMeister system. [Figure 3] A flowchart illustrating the work procedure in the BioMeister system. [Figure 4] This is a diagram illustrating a machine learning model (decision tree). [Figure 5] This is a diagram showing the work procedures. [Figure 6] This diagram illustrates the process of associating work procedures with an evaluation model. [Figure 7] This is a diagram illustrating the processes that make up the protocol. [Modes for carrying out the invention]

[0013] Embodiments of the present invention will be described below with reference to the drawings. Note that the following embodiments are not limiting to the present invention, and not all combinations of features described in these embodiments are essential to the solutions of the present invention. Furthermore, various other forms that do not depart from the spirit of the present invention are also included, and some of the following embodiments can be combined as appropriate.

[0014] Furthermore, although the present invention relates to a training system used for training (practice) in general, unless otherwise specified, the following explanation will use the Biomeister system, which is used for training in biochemical and molecular biology experiments and cell biology experiments, as an example.

[0015] Figure 1 is a schematic block diagram of the BioMeister System 1 according to this embodiment. As shown in Figure 1, the BioMeister System 1 comprises an information processing device 10 for the operator, an information processing device 20 for the administrator, and an application server 30.

[0016] The worker-side information processing device 10 is used for operations during worker training. The administrator-side information processing device 20 is used to present the administrator with the worker's work content (work procedure), analysis results of the work content, and evaluation results, and to generate (regenerate) an evaluation model and generate (reconstruct) a protocol based on feedback from the administrator. The application server 30 stores application data and processes the application data during worker training, displaying it on a display device connected to the worker-side information processing device 10.

[0017] While details regarding protocols will be discussed later, generally speaking, there is a trade-off between the number of work steps specified in a protocol and the ease of performing the work. Specifically, (although it may seem preferable to set up a detailed protocol), as mentioned above, if the work content is set up too finely, it may become difficult for some workers to perform the work, or some workers may skip steps. Therefore, a protocol with detailed work procedures is not necessarily a good protocol. For this reason, it is important to determine how many work procedures to set (specify) in the protocol for the work in question.

[0018] Therefore, in the Biomeister System 1 according to this embodiment, a protocol is set according to the name (content) of the work, the worker (or group) performing the work, and the worker's evaluation level, and the protocol is presented to the worker. This allows for the early improvement of the worker's work level to meet the objectives of the worker and / or manager.

[0019] In addition, as described above, on the premise that the training system of the present embodiment is used for training (practice) related to biochemical and molecular biology experiments (specifically, for example, PCR, RT-PCR, DNA sequencing, plasmid vector preparation, electrophoresis, Western blotting, etc.) and cell biology experiments (specifically, for example, animal cell culture, plant cell culture, Escherichia coli culture, immunostaining, monoclonal antibody production, screening methods, etc.), it is described as a biomaster system. However, the training system of the present embodiment can also naturally be used for training related to the manufacturing processes of food and beverages (specifically, for example, the production of beer, sake, and wine, confectionery making, etc.), traditional crafts, and various operations in the medical field (specifically, for example, kits for diagnostic reagents, cell manufacturing).

[0020] Regarding the number of work processes described above, the number of work processes set as a protocol to be displayed on the display device connected to the information processing device 10 on the operator side is shown as the instructed number of processes, and the number of work processes actually performed by the operator who confirmed the protocol is shown as the implemented number of processes.

[0021] Figure 2 is a block diagram showing the functional configuration of the biomaster system 1. As described above, the biomaster system 1 includes the information processing device 10 on the operator side, the information processing device 20 on the administrator side, and the application server 30. Hereinafter, the functional configurations of these devices and servers will be described in order.

[0022] The information processing device 10 on the operator side mainly includes a control unit 102, an operation information acquisition unit 104, a display control unit 106, an operation data storage unit 108, a work procedure (protocol) analysis unit 110, a transmission unit 112, a reception unit 114, an evaluation model acquisition unit 116, and an evaluation unit 118 as its functions.

[0023] The control unit 102 is a functional block that controls the processing of each functional block of the operator-side information processing device 10. The operation information acquisition unit 104 acquires information operated (input) by the operator using an operation device connected to the operator-side information processing device 10. The display control unit 106 controls the display device connected to the operator-side information processing device 10 to display application data (processing content and processing results), processing analysis results, evaluation results, data related to research equipment and consumables, and protocols, etc.

[0024] The operation data storage unit 108 stores operation information acquired by the operation information acquisition unit 104. The work procedure (protocol) analysis unit 110 analyzes the data on the worker's work content stored in the operation data storage unit 108. Specifically, the work procedure (protocol) analysis unit 110 compares, for example, the number of work steps set as a protocol (number of instructed steps) with the number of work steps performed by the worker (number of performed steps), and analyzes the difference (for example, missing work steps, extra work steps, etc.).

[0025] In addition, the work procedure (protocol) analysis unit 110 compares the number of clicks expected from the protocol with the number of clicks performed by the worker, and, in relation to the visual inspection work process described later, measures whether the point being observed by the worker is outside the position specified in the protocol, measures the distance between the point being observed by the worker and the position specified in the protocol, and also measures the time the point being observed by the worker remains at the position specified in the protocol, because if the point being observed by the worker does not remain at the position specified in the protocol for a certain period of time or longer, it cannot be determined that the visual inspection has been performed.

[0026] Furthermore, the results of the analysis of data regarding the worker's work content are transmitted to the administrator's information processing device 20, and to a display device connected to the worker's information processing device 10, depending on the work content and / or the worker's evaluation (level).

[0027] The transmission unit 112 is connected to the administrator's information processing device 20 and the application server 30 via a predetermined network, enabling communication between them. The transmission unit 112 transmits, for example, the worker's ID, worker's level, the name of the task, data related to the task content, the analysis results of the data related to the worker's task content, and evaluation results to the administrator's information processing device 20, and also transmits the operation (task) details to the application server 30.

[0028] The receiving unit 114 is connected to the administrator's information processing device 20 and the application server 30 via a predetermined network, enabling communication between them. The receiving unit 114 receives, for example, a protocol corresponding to the worker's level and an evaluation model (described later) from the administrator's information processing device 20, and also receives application data from the application server 30.

[0029] The evaluation model acquisition unit 116 acquires an evaluation model from the administrator's information processing device 20. The evaluation unit 118 uses the evaluation model acquired by the evaluation model acquisition unit 116 to evaluate the worker's work content based on the data on the worker's work content stored in the operation data storage unit 108. Specifically, for example, work time, number of work steps (number of steps performed), number of times the work was done incorrectly, the number of work steps using a pipette or tip appropriate to the volume instructed in the protocol, the number of work steps in which the lid of a tube, etc., was kept open, and the number of work steps involving visual confirmation are evaluated as part of the worker's work content.

[0030] Furthermore, the evaluation result determines the worker's evaluation level. This evaluation is determined using specific numerical values ​​(e.g., evaluation level 1, evaluation level 2, evaluation level 3, etc.) or letters of the alphabet. The evaluation result (determination result) is transmitted to the administrator's information processing device via the transmission unit 112, as described above. Additionally, the results of the analysis of data related to the worker's work content by the work procedure (protocol) analysis unit 110 are referenced in the evaluation by the evaluation unit 118.

[0031] As a supplement, regarding the number of steps in the visual inspection process, (generally, workers tend to look at the area they are operating with the pointing device (for example, the area they click in the case of a mouse)), it is necessary to check whether the worker is properly looking at the area they should be looking at during that task, and the number of times the worker is properly looking at the area they should be looking at is checked.

[0032] For example, consider a scenario where a tube contains water and it needs to be drawn up using a pipette. In this case, the operator will look at the button on the pipette to operate it, but they should actually be looking at two things: the water in the tube and the tip of the pipette. Therefore, the eyes should focus on the water in the tube and the tip of the pipette, and the click on the pipette's button should be performed without looking. Furthermore, a device that analyzes eye movements (for example, smart glasses, eye-tracking devices, etc., as shown by reference numeral 40 in Figure 1) can be used to check this.

[0033] The administrator-side information processing device 20 primarily comprises a control unit 202, a receiving unit 204, a protocol selection unit 206, a display control unit 208, an input information acquisition unit 210, an evaluation model generation unit 212, a work rule storage unit 214, a work rule setting unit 216, an evaluation model storage unit 218, an evaluation model selection unit 220, a protocol configuration unit 222, a protocol storage unit 224, and a transmission unit 226.

[0034] The control unit 202 is a functional block that controls the processing of each functional block of the administrator-side information processing device 20. The receiving unit 204 is communicated via a predetermined network to the worker-side information processing device 10 and the application server 30. The receiving unit 204 receives, for example, the worker's ID, worker's level, the name of the task, data related to the operation (task), the analysis results of the data related to the worker's work content, and the evaluation results (evaluation level) of the work performed by the worker from the worker-side information processing device 10.

[0035] The protocol selection unit 206 selects a protocol to be trained from a group (set) of protocols stored in the protocol storage unit 224, based on the worker's ID, the worker's level, and the name of the task. If a protocol cannot be uniquely identified, multiple protocols can be displayed on a display device connected to the worker's information processing device 10, allowing the worker to select the protocol to be trained.

[0036] The display control unit 208 controls the display device connected to the administrator's information processing device 20 to display, for example, data related to the worker's work content, the evaluation result (evaluation level) obtained from analyzing the data related to the worker's work content, information related to work regulations, and the evaluation model. The input information acquisition unit 210 acquires training data for the evaluation model (i.e., the worker's work content and the evaluation result evaluated by the administrator), work regulations associated with the evaluation model, etc., which are input by the administrator. Regarding the evaluation result evaluated by the administrator, even if it is an evaluation level, it may be the case that relatively higher points are given to work content that is judged to be good.

[0037] The evaluation model generation unit 212 generates evaluation models for each work process unit that constitutes the protocol. As described above, data on the worker's work content and the evaluation results evaluated by the manager are input as training data into the evaluation model, which is a machine learning model, to perform machine learning and generate (reconstruct) the evaluation model. The machine learning here may be performed at a predetermined timing, or it may be performed in the process of the procedure shown in Figure 3 described later. Furthermore, when performing machine learning, the training data may include the worker's unit and the content of their work as appropriate and as needed.

[0038] As a machine learning model, for example, a decision tree can be used, as shown in Figure 4. Here, a decision tree is a machine learning model that performs estimation (classification) of data by branching according to predetermined conditions. Specifically, a decision tree consists of nodes to which features (work procedures) are assigned (that is, each node has a condition for the feature (i.e., an evaluation item for branching)), and it branches so that the data is classified into one of the lower nodes according to the evaluation item for branching.

[0039] Furthermore, the lower-level nodes here are set as evaluation levels for the worker's work. The evaluation model is subjected to machine learning based on the number of workers who performed the work, as well as the evaluation results assessed by the manager. This tunes the various parameters of the evaluation model and reconstructs it. As a result, even when the same procedure is followed for a given protocol, different evaluations may be obtained.

[0040] The work procedure storage unit 214 stores detailed procedures for each task. As an example of work procedures, Figure 5 shows the work procedure for transferring reagent 1 to a tube. The work procedure setting unit 216 sets the procedures stored in the work procedure storage unit 214 to the lower nodes (evaluation levels) of the evaluation model generated by the evaluation model generation unit 212. The setting process here sets the work procedures corresponding to the evaluation level of the worker's work content for the lower nodes. It is assumed that which work procedures are set for which evaluation level (e.g., evaluation level 1, evaluation level 2, evaluation level 3, etc.) is determined in advance.

[0041] The evaluation model storage unit 218 stores evaluation models associated with work regulations corresponding to lower-level nodes (evaluation levels) for each work process that constitutes a protocol. The evaluation model selection unit 220 selects, for example, an evaluation model for a work process that constitutes a protocol of work performed by an operator from the evaluation model storage unit 218 for each work process.

[0042] When the protocol configuration unit 222 receives the work level or evaluation level of the worker to be trained and the name of the work, it first reads the evaluation models of the work processes that constitute the work related to the name of the work from the evaluation model storage unit 218, according to the number of work processes. Next, for each evaluation model, it identifies the evaluation level corresponding to the received worker's work level or the lower node of that evaluation model that corresponds to the received evaluation level. Once the lower node (evaluation level) is identified, it then retrieves the work regulations associated with that evaluation level. Finally, it sets the retrieved work regulations according to the procedure of that work.

[0043] As a supplement, the protocol configuration unit 222 configures the protocol according to the evaluation level corresponding to the received work level at the start of training, and configures the protocol according to the received evaluation level during the continuation of training. Furthermore, the work levels and evaluation levels are divided into levels from 1 to 10, for example, with levels of similar difficulty, and these levels correspond one-to-one.

[0044] This allows the work protocol for the input work name to be configured according to the work level or evaluation level of the worker who also input (received) it. Note that for lower-level nodes (evaluation levels) in the evaluation model, and for nodes with high evaluation levels, a specific work provision may not necessarily be associated; "Null" may be assigned instead. In other words, no work provision may be associated at all. By setting work provisions in this way, when a protocol is configured, if the worker's level is high, the work provision for that work process will not be configured as a protocol (it can be excluded from the protocol), and as a result, the number of work processes (instruction processes) defined in that protocol can be reduced.

[0045] The protocol storage unit 224 stores the protocol configured by the protocol configuration unit 222. The transmission unit 226 is communicated via a predetermined network to the worker's information processing device and the application server. The transmission unit 226 transmits, for example, a protocol and evaluation model corresponding to the worker's level to the worker's information processing device 10.

[0046] The application server 30 primarily comprises an application data processing unit 302 and an application data storage unit 304. During training, the application data processing unit 302 processes the application data stored in the application data storage unit 304 in response to information acquired by the operation information acquisition unit 104 (i.e., information operated (input) by the operator using an operation device connected to the operator-side information processing device 10) to generate application data to be displayed on the display device connected to the operator-side information processing device 10. The application data storage unit 304 stores this application data.

[0047] Furthermore, the functional configuration of the BioMeister System 1 is not necessarily limited to the configuration shown in Figure 2. Therefore, for example, the work procedure (protocol) analysis unit 110 and evaluation unit 118 implemented in the worker-side information processing device 10 may be implemented in the administrator-side information processing device 20.

[0048] Figure 3 is a flowchart showing the procedure for using the BioMeister System 1 when an operator is being trained. In the following, the symbol "S" in the flowchart description represents a step. That is, the processing steps S1-1 to S1-14 in the flowchart will be abbreviated as S1-1 to S1-14.

[0049] In S1-1, the Biomeister system 1 acquires the name of the task, the worker's ID, and the worker's level, which are input using an operating device connected to the worker's information processing device 10, via the operation information acquisition unit 104. Although not shown in the flowchart of Figure 3, the worker's ID, worker's level, and the name of the task are stored in the operation data storage unit 108 by the worker's control unit 102.

[0050] In S1-2, the BioMeister system 1, using the protocol configuration unit 222, configures a protocol for the task whose name was obtained in S1-1, corresponding to the evaluation level that corresponds to the worker's skill level. The configured protocol is then stored in the protocol storage unit 224 by the administrator's control unit 202.

[0051] In S1-3, the BioMeister system 1, using the protocol selection unit 206, retrieves (selects) the target protocol from the protocol storage unit 224 according to the name of the task, the acquired worker ID, and / or the worker's level. The selected protocol is then transmitted to the worker-side information processing device 10 via the transmission unit 226. In S1-4, the BioMeister system 1, using the worker-side control unit 102, retrieves application data corresponding to the received protocol from the application data storage unit 304 of the application server 30.

[0052] In S1-5, the BioMeister system 1 controls the display device connected to the worker's information processing device 10 via the worker's display control unit 106 to display the protocol selected by the protocol selection unit 206 and the application data acquired according to that protocol. After the worker confirms the protocol and application data displayed on the display device, they start the training.

[0053] In S1-6, the Biomeister system 1 acquires data on the operator's work content, which is input using the operating device, via the operation information acquisition unit 104. Furthermore, the operator's control unit 102 stores the acquired data in the operation data storage unit 108. Specifically, this data includes coordinate data from the pointing device, data related to clicks, and data analyzing eye movements.

[0054] In S1-7, the BioMeister System 1 uses the Work Procedure (Protocol) Analysis Unit 110 to determine whether the worker's training is complete based on the work procedures specified in the protocol selected in S1-4. If the BioMeister System 1 determines that the worker's training is not complete, it returns to S1-5. If it determines that the worker's training is complete, it proceeds to S1-8. The BioMeister System 1 also proceeds to S1-8 if the Operation Information Acquisition Unit 104 acquires a notification (operation information) indicating that the work has been completed, which has been input by the worker using the operating device.

[0055] In S1-8, the BioMeister System 1 uses the work procedure (protocol) analysis unit 110 to analyze the data on the worker's work content stored in the operation data storage unit 108 in S1-6. In S1-9, the BioMeister System 1 uses the evaluation model selection unit 220 to select an evaluation model corresponding to the protocol selected in S1-3.

[0056] In S1-10, the BioMeister System 1, using the evaluation unit 118, evaluates the worker's work content based on the data regarding the worker's work content stored in the operation data storage unit 108 in S1-6, using the evaluation model selected in S1-9. At that time, the BioMeister System 1 refers to the results of the work procedure (protocol) analysis unit 110 in S1-8 as needed.

[0057] In S1-11, the BioMeister System 1, in its evaluation model generation unit 212, uses data related to the worker's work content, the results of the work procedure (protocol) analysis unit 110 in S1-8, and the evaluation level evaluated by the manager as training data to perform machine learning on the evaluation model and generate an evaluation model (reconstruct the evaluation model). Regarding the machine learning here, for example, the manager may, based on the analyzed results, consider what constitutes a good work procedure from the work procedures performed by the worker, and then reflect the results of that consideration in the evaluation results (e.g., evaluation level) and load them into the evaluation model. Note that the process in S1-11 does not necessarily have to be performed using machine learning; the manager may also reconstruct the evaluation model based on the analyzed results.

[0058] In S1-12, the BioMeister system 1 uses the work procedure setting unit 216 to associate the procedures stored in the work procedure storage unit 214 with the lower nodes of the evaluation model generated by the evaluation model generation unit 212 in S1-11. In S1-13, the BioMeister system 1 uses the protocol configuration unit 222 to retrieve work procedures associated with the evaluation level from the evaluation model based on the name of the work and the evaluation level of the worker evaluated in S1-10, and further configures the retrieved work procedures according to the work procedure.

[0059] In S1-14, the BioMeister System 1, via the worker-side control unit 102 and the display unit 106, displays a pop-up on the display device connected to the worker-side information processing device 10, allowing the worker to choose whether or not to continue training. The system then branches the process based on the worker's selection. If the worker chooses to continue, the BioMeister System 1, via the worker-side control unit 102, returns the process to S1-5 and presents the worker with a protocol reconfigured according to the worker's work level evaluated in S1-10. If the worker chooses to end the work, the training ends.

[0060] In this way, even with the same task protocol and the same level of protocol, the protocol can be reconfigured (redefined) by reconstructing the evaluation model according to the number and content of tasks performed, and the worker can then conduct the training according to the reconfigured protocol in the next training session.

[0061] As a supplement, administrators can also adjust the worker IDs (i.e., workers), the number of tasks performed, and the content of the tasks that are considered when restructuring the evaluation model and protocol. For example, this applies to practical training conducted in a predetermined class unit at a university. Specifically, groups can be formed using only the IDs of the students (workers) in that class, the number of tasks (training) can be set to five, and tasks that are not considered particularly important can be excluded from the reference data (training data) when generating the evaluation model. By making these adjustments, the tasks that should be learned can be improved to a consistent level across predetermined class units.

[0062] Furthermore, by having a single worker repeatedly undergo training, it's possible to not only improve the tasks that the worker needs to learn, but also to provide them with work procedures (protocols) that are easier for them to perform. Conversely, by having a large number of workers perform a task, it's possible to develop a de facto standard protocol for that task.

[0063] In addition, based on the content of work performed in the past, if a work procedure is performed by a worker with a predetermined work level, the work procedure can be evaluated according to the number of work procedures performed using a similar procedure, and the evaluation results can be used as training data to run machine learning, thereby reconfiguring and providing the protocol (for example, if a work procedure performed by a worker with an advanced level is the same as, or judged to be the same as, a majority of the work procedures performed by workers with an advanced level are the same, then that work procedure can be evaluated as preferable, and the evaluation results can be used as training data to run machine learning, thereby reconfiguring and providing the protocol).

[0064] In this way, by adjusting the worker's ID (i.e., the worker), the number of tasks performed, and the content of the tasks, analyzing the content of the tasks performed, and performing machine learning, it is possible to generate and provide protocols that meet the objectives of the worker and / or manager. This makes it possible to improve the worker's skill level relatively quickly.

[0065] Next, we will provide further explanation regarding the processes from S1-11 to S1-13 in Figure 3. Specifically, we will provide a step-by-step explanation of the processes from the execution of machine learning in the evaluation model to the reconstruction of the protocol. As mentioned above, in the evaluation model, machine learning is executed by inputting the worker's work content and the evaluation results evaluated by the manager as training data. Once machine learning is executed, the evaluation model is reconstructed according to that training data.

[0066] For example, in a decision tree, when machine learning is performed, various parameters are tuned and the decision tree is reconstructed. Furthermore, when the decision tree is reconstructed in this way, in the Biomeister system according to this embodiment, the same work procedure may be judged as having a different evaluation level (that is, for example, a work procedure that was judged to have a high evaluation level before machine learning is performed may be judged to have a low evaluation level after machine learning is performed).

[0067] Then, once machine learning is performed and the evaluation model is reconstructed, work rules corresponding to the evaluation model are set for the subnodes (evaluation levels) of the reconstructed evaluation model. As a supplement, Figure 6 shows how this setting process works. As shown in Figure 6, work rules corresponding to each evaluation level are set in advance, and these work rules are associated with the subnodes (evaluation levels) of the evaluation model.

[0068] Specifically, Figure 6 associates a corresponding work procedure with each subnode of the decision tree relating to the work process when transferring 10 μL of reagent (i.e., each of evaluation levels 1-4) (i.e., "NULL" for evaluation level 1, "Transfer reagent 1 to a tube" for evaluation level 2, "Set the 100 μL pipette to 30 μL" and "Transfer reagent 1 to a tube" for evaluation level 3, and "Put on gloves and wipe the desk", "Set the 100 μL pipette to 30 μL" and "Transfer reagent 1 to a tube" for evaluation level 4).

[0069] Regarding the evaluation levels, as shown in Figure 6, they are defined in descending order from highest to lowest: Evaluation Level 1, Evaluation Level 2, Evaluation Level 3, and Evaluation Level 4. Furthermore, the number of associated work regulations is also set to be small (i.e., 0 items ("NULL") are associated with Evaluation Level 1, 1 item with Evaluation Level 2, 2 items with Evaluation Level 3, and 3 items with Evaluation Level 4).

[0070] When a work specification corresponding to the evaluation model is set for a lower node (evaluation level), the protocol configuration unit 222 retrieves one or more work processes included in the work to be trained from the name of the work. Here, assuming that the retrieved work processes are three work processes, the evaluation model corresponding to each of the three retrieved work processes is read from the evaluation model storage unit 218.

[0071] Figure 7 shows the case where evaluation models corresponding to each of the three work processes (i.e., work process 1, work process 2, and work process 3) are loaded. Next, the work performed by the worker is evaluated based on the content of the work performed in the previous work and the evaluation model corresponding to that work process. In the example shown in Figure 7, the evaluation model corresponding to work process 1 is judged as "evaluation level 3", the evaluation model corresponding to work process 2 is judged as "evaluation level 1", and the evaluation model corresponding to work process 3 is judged as "evaluation level 2".

[0072] The protocol component 222 evaluates the work performed by the worker and extracts the work regulations associated with the evaluated evaluation level. In the example shown in Figure 7, it extracts "B" and "C" associated with "evaluation level 3" for work process 1, "NULL" associated with "evaluation level 1" for work process 2, and "C" associated with "evaluation level 2" for work process 3.

[0073] Then, after extracting the work procedures associated with the evaluation level, the protocol configuration unit 222 constructs a protocol based on the extracted work procedures. In the example shown in Figure 7, the protocol is constructed as "Work Procedure 1's B, C + Work Procedure 2's NULL (no work procedures to be presented) + Work Procedure 3's C" based on the work procedure related to the work name and the extracted "B, C" associated with "Evaluation Level 3" of Work Procedure 1, "NULL" associated with "Evaluation Level 1" of Work Procedure 2, and "C" associated with "Evaluation Level 2" of Work Procedure 3. In this case, the display device connected to the worker's information processing device 10 will display "Work Procedure 1's B, Work Procedure 1's C, Work Procedure 3's C" as the protocol.

[0074] As described above, the protocol configuration unit 222 extracts, for each work process, one of the following according to the evaluation result (evaluation level) in the evaluation model corresponding to one or more work processes included in the work named "work": (1) work regulations corresponding to an evaluation level lower than the worker's level entered by the worker, (2) work regulations corresponding to the same evaluation level as the worker's level entered by the worker, or (3) work regulations corresponding to an evaluation level higher than the worker's level entered by the worker. The protocol is then constructed by merging (integrating) these. Therefore, even if the work name is the same, if the evaluation levels of the work processes are different, different protocols will be displayed on the display device connected to the worker's information processing device 10.

[0075] Specifically, for example, if a worker with sufficient experience in the relevant task undergoes training and achieves a high evaluation level as planned, a simplified protocol containing only the basic steps of that task will be generated and presented for the next task.

[0076] If a worker fails to input their work level (because they are unsure of their own), or if a worker who has been presented with a detailed protocol is able to conduct training, reduce work time, or decrease work errors, then a standard protocol with slightly more detailed procedures for that task will be generated and presented for the next task.

[0077] Furthermore, if the evaluation level of a task performed after being presented with a standard protocol and undergoing training is low, a more detailed protocol (i.e., a protocol with more work steps than the standard protocol) will be generated and presented for the next task. Based on the above protocol content, for reference, a simplified protocol, a standard protocol, and a detailed protocol for transferring 10 μL of reagent are shown in Table 1 below.

[0078] [Table 1]

[0079] As explained above, at predetermined times, the manager reviews the work content performed by the worker and the results of the analysis of data related to the worker's work content. The manager then inputs the evaluation results, along with the work content, into the evaluation model as training data, and runs machine learning on the evaluation model to regenerate (reconstruct) the evaluation model. Furthermore, when inputting this training data into the evaluation model, the manager considers what constitutes a good work procedure for the target audience for training (i.e., individuals, designated groups, classes, an unspecified number of people, etc.), based on the results of the analysis of data related to the worker's work content, and after reviewing the evaluation results (for example, after adjusting the evaluation level), inputs them into the evaluation model.

[0080] Furthermore, by reviewing and inputting the evaluation results (by adjusting and inputting the evaluation level), various parameters in the evaluation model are tuned, and even with the same work procedure, it may be possible to branch to different sub-nodes before and after tuning (that is, the evaluation model is reconstructed, and even with the same work procedure, there may be cases where it is judged as having a different evaluation level).

[0081] In addition, each of these subnodes is associated with a work procedure. The evaluation model extracts the work procedures associated with the subnodes that correspond to the worker's evaluation level, and then merges them according to the number of evaluation models to generate a protocol, which is then presented to the worker. This allows for the presentation of protocols that meet the manager's intentions (objectives) when workers undergo continuous training, and ultimately, the worker's work level can be improved at an early stage. When adjusting and inputting evaluation levels into the evaluation model, it is possible for not only the manager but also the worker and manager to consider together what constitutes a good work procedure (protocol), or for the worker to consider it themselves based on their own work results.

[0082] Furthermore, the worker-side information processing device 10 and the administrator-side information processing device 20 shall each be equipped with, for example, a CPU, RAM, ROM, HDD, operating device interface, network interface, etc., as part of their hardware configuration.

[0083] The CPU (Central Processing Unit) is an arithmetic processing unit that comprehensively controls each block of the information processing device. RAM (Random Access Memory) has a storage area for temporarily storing the CPU's calculation results and a load area for various control programs. ROM (Read Only Memory) stores various programs (e.g., boot programs). HDD (Hard Disk Drive) stores evaluation models, protocols, etc. The operating device interface is an interface for inputting (acquiring) data with the operating device. The network interface is connected to the LAN (Local Area Network) by wired or wireless connection, enabling information input and output with external devices.

[0084] In addition, the present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a device via a network or storage medium, and by having one or more processors in the device's computer read and execute the program. [Explanation of Symbols]

[0085] 10. Information processing device on the worker's side 20 Administrator-side information processing device 30 Application Servers 102 Control Unit 104 Operation information acquisition unit 106 Display Control Unit 108 Operation Data Storage Unit 110 Work Procedure (Protocol) Analysis Department 112 Transmitter 114 Receiving Unit 116 Evaluation Model Acquisition Unit 118 Evaluation Department 202 Control Unit 204 Receiving Unit 206 Protocol Selection Section 208 Display Control Unit 210 Input Information Acquisition Unit 212 Evaluation Model Generation Unit 214 Work Regulations Storage Unit 216 Work Regulation Setting Section 218 Evaluation Model Memory Unit 220 Evaluation Model Selection Section 222 Protocol Components 224 Procol Memory Unit 226 Transmitter 302 Application Data Processing Unit 304 Application Data Storage Unit

Claims

1. An acquisition means for acquiring data relating to the name of the work performed by the worker, and data relating to the content of the work performed by the worker, An evaluation model generated for each work process constituting the work related to the name of the work, wherein the evaluation model has been machine-trained using data on the content of work in previously performed work and data on the evaluation level of the content of said work by the manager as training data, and a determination means for determining the evaluation level of the worker for each work process included in the name of the work, wherein a work regulation corresponding to each work process is associated with each worker's evaluation level, An extraction means for extracting work regulations associated with the evaluation level determined by the determination means, according to each work process, A configuration means for merging work regulations corresponding to each work process extracted by the extraction means, according to the evaluation level determined for each work process according to the work defined by the name of the work, to configure a protocol with a variable number of work processes, Display control means for displaying the protocol configured by the above-mentioned means on an external display device. Equipped with, A protocol providing device characterized in that the data relating to the content of the aforementioned work is operation data entered using a predetermined operating device.

2. An acquisition step to acquire data relating to the name of the work performed by the worker, and data relating to the content of the work performed by the worker, A determination step in which an evaluation model is generated for each work process that constitutes the work related to the name of the work, and the evaluation model is machine-trained using data on the content of work in previously performed work and data on the evaluation level of the content of said work by the manager as training data, inputs the data on the content of work acquired in the acquisition step, and outputs the evaluation level of the worker in each work process included in the name of the work acquired in the acquisition step, thereby determining the evaluation level of the worker for each work process included in the name of the work, wherein a work regulation corresponding to each work process is associated with each evaluation level of the worker, An extraction step for extracting work regulations associated with the evaluation level determined in the above determination step, A configuration step to configure a protocol with a variable number of work processes by merging the work regulations corresponding to each work process extracted in the extraction step, according to the evaluation level determined for each work process, in accordance with the work defined by the name of the work, A display control step that causes the protocol configured in the above configuration step to be displayed on an external display device. Includes, A method for providing a protocol, which is executed by a protocol providing device, wherein the data relating to the content of the aforementioned work is operation data entered using a predetermined operating device.

3. A program that causes a computer to execute the protocol provision method described in claim 2.

4. A means for acquiring data on the content of work performed in the past, and data on the evaluation level determined by the manager for the content of the work performed in the past, as training data. A generation means that generates an evaluation model in which data related to the content of the work is used as input values ​​and an evaluation level derived from those input values ​​is used as output values ​​by performing machine learning using the training data. Equipped with, The data relating to the content of the aforementioned work is operation data entered using a predetermined operating device. The evaluation model generation device is characterized in that, in work procedures performed by workers with an advanced work level, if the majority of workers with an advanced work level have work procedures that are judged to be identical or similar, the manager will determine that such work procedure is a preferred work procedure.

5. A method for generating an evaluation model, which is performed by an evaluation model generation device, The acquisition step involves obtaining data on the content of work performed in the past, and data on the evaluation level of the content of said work by the manager, as training data. The generation step involves performing machine learning using the training data to generate an evaluation model in which data related to the content of the work is used as input values ​​and an evaluation level derived from those input values ​​is used as output values. Includes, The data relating to the content of the aforementioned work is operation data entered using a predetermined operating device. The method for generating an evaluation model is characterized in that, in work procedures performed by workers with an advanced work level, if the majority of workers with an advanced work level have work procedures that are judged to be identical or similar, the manager will determine that such work procedure is a preferred work procedure.

6. A program that causes a computer to execute the method for generating the evaluation model described in claim 5.

7. A training method performed by a system comprising an information processing device on the worker's side and an information processing device on the administrator's side, A first acquisition step involves using the worker's information processing device to acquire data relating to the name of the work performed by the worker, data relating to the work level, and data relating to the content of the work performed by the worker. A first determination step in which the data on the content of work acquired in the first acquisition step is input to an evaluation model generated by the administrator's information processing device for each work process constituting the work related to the name of the work, the evaluation model which has been machine-learned using data on the content of work in previously performed work and data on the evaluation level of the content of said work by the administrator as training data, and the evaluation level of the worker in each work process included in the name of the work acquired in the first acquisition step is output, thereby determining the evaluation level of the worker for each work process included in the name of the work, wherein a work regulation corresponding to each work process is associated with each worker's evaluation level, The administrator's information processing device performs a first extraction step of extracting work regulations associated with the evaluation level determined in the first determination step, The administrator's information processing device performs a first configuration step in which, in the first determination step, according to the work defined by the name of the work, the work regulations corresponding to each work process extracted in the first extraction step are merged according to the evaluation level determined for each work process, thereby configuring a protocol with a variable number of work processes. The operator's information processing device performs a first display control step, which causes the protocol configured in the first configuration step to be displayed on an external display device, The information processing device on the worker's side acquires data relating to the content of the work performed by the worker in accordance with the protocol displayed in the first display control step, in a second acquisition step. The administrator's information processing device inputs data relating to the content of the work acquired in the second acquisition step into the evaluation model, and performs a second determination step in which the evaluation level of the worker is determined for each work process included in the name of the work. The administrator's information processing device performs a second extraction step of extracting work regulations associated with the evaluation level determined in the second determination step, The administrator's information processing device, in the second determination step, merges the work regulations corresponding to each work process extracted in the second extraction step according to the evaluation level determined for each work process according to the work defined by the name of the work, thereby configuring a protocol with a variable number of work processes. The operator's information processing device performs a second display control step, which causes the protocol configured in the second configuration step to be displayed on the external display device. Includes, A training method characterized in that the data relating to the content of the aforementioned work is operation data entered using a predetermined operating device.

Images