Information processing system, information processing method, and program

Abstract

To provide an information processing system and the like capable of transmitting more accurate information related to a technique and further evaluating the technique.SOLUTION: An information processing system includes at least one processor that is configured to execute a program. The program includes a reading step and a first display control step. In the reading step, first image data including a technique relating to medical treatment performed by a user is read. In the first display control step, an evaluation screen for evaluating the technique is displayed so that it can be visually checked by the user. The evaluation screen includes the first image data and evaluation support information for evaluating a technique of the user in a listed manner.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] There are situations where various medical information is desired to be shared with others, especially other doctors. For example, Patent Document 1 discloses a medical support system for sharing patient information, which is an example of medical information. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-109763 Summary of the Invention [Problem to be solved by the invention]

[0004] However, even though it is possible to share medical information, as in the system disclosed in Patent Document 1, it is difficult to convey technical information about a doctor, such as the movements of his or her hands or medical instruments during surgery. This applies, for example, to situations where an experienced doctor wants to teach his or her own surgical technique to a trainee who is trying to improve his or her surgical technique. In other words, in such situations, it is difficult to convey technical information to the trainee and furthermore to evaluate the trainee's skills.

[0005] In view of the above circumstances, the present invention provides an information processing system and the like that enables more accurate transmission of information related to procedures and furthermore evaluation of procedures. [Means for solving the problem]

[0006] According to one aspect of the present invention, there is provided an information processing system. The information processing system includes at least one processor configured to execute a program to perform the following steps: In the reading step, first image data including a medical procedure performed by a user is read; In the first display control step, an evaluation screen for evaluating the procedure is displayed so as to be visible to the user; The evaluation screen includes, in a browsable manner, the first image data and evaluation support information for evaluating the user's procedure.

[0007] According to this aspect, it is possible to more accurately transmit information about the procedure and furthermore to evaluate the procedure. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a configuration diagram illustrating an information processing system 1 according to an embodiment. [Figure 2] 2 is a block diagram showing a hardware configuration of an information processing device 2 according to an embodiment. FIG. [Figure 3] FIG. 2 is a block diagram showing the hardware configuration of a user terminal 3. [Figure 4] 2 is a block diagram showing functions realized by a control unit 23 and the like in an information processing device 2 according to an embodiment. FIG. [Figure 5] 2 is a flowchart showing an outline of processing executed by the information processing system 1. [Figure 6] 2 is an activity diagram showing a specific example of processing executed by the information processing system 1. FIG. [Figure 7] FIG. 10 is a schematic diagram of a learning table T1. [Figure 8] 1 shows an example of a learning screen 6 displayed on the display unit 34 of the user terminal 3 for viewing a video including a procedure. [Figure 9] 1 shows an example of an evaluation input screen 7 displayed on the display unit 34 of the user terminal 3, where an evaluation of the procedure is input. [Figure 10] FIG. 10 is a schematic diagram of an evaluation table T2. [Figure 11] 1 shows an example of a result display screen 8 that displays the results of the evaluation analysis, and is displayed on the display unit 34 of the user terminal 3. FIG. [Figure 12] 1 shows an example of a list display screen 9 that displays a list of the skill trainings performed by the user, and is displayed on the display unit 34 of the user terminal 3. DETAILED DESCRIPTION OF THE INVENTION

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the accompanying drawings. Various features shown in the following embodiments can be combined with each other.

[0010] Incidentally, the program for realizing the software appearing in one embodiment may be provided as a non-transitory computer-readable medium, or may be provided so that it can be downloaded from an external server, or may be provided so that the program is started on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0011] In one embodiment, a "unit" may include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In one embodiment, various information is handled, and this information is represented, for example, by physical values of signal values representing voltage and current, high and low signal values as a binary bit set consisting of 0 or 1, or quantum superposition (so-called quantum bits), and communication and calculations can be performed on a circuit in the broad sense.

[0012] In addition, a circuit in the broad sense is a circuit realized by at least appropriately combining a circuit, circuitry, a processor, a memory, etc. That is, it includes an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0013] [Embodiment] 1. Hardware Configuration In this section, the hardware configuration of one embodiment will be described.

[0014] 1.1 Information Processing System 1 FIG. 1 is a configuration diagram showing an information processing system 1 according to one embodiment. The information processing system 1 includes an information processing device 2 and a user terminal 3, which are connected via a network 11. These components will be further described. Here, a system exemplified as the information processing system 1 is made up of one or more devices or components. Therefore, even the information processing device 2 or the user terminal 3 alone is an example of a system.

[0015] 1.2 Information processing device 2 2 is a block diagram showing the hardware configuration of an information processing device 2 according to an embodiment. The information processing device 2 has a communication unit 21, a storage unit 22, and a control unit (processor) 23, and these components are electrically connected via a communication bus 20 inside the information processing device 2. Each component will be further described.

[0016] The communication unit 21 is preferably a wired communication means such as USB, IEEE1394, Thunderbolt (registered trademark), wired LAN network communication, etc., but may also include wireless LAN network communication, mobile communication such as 3G / LTE / 5G, Bluetooth (registered trademark) communication, etc. as necessary. In other words, it is more preferable to implement it as a collection of multiple communication means. In one embodiment, the information processing device 2 is configured to communicate various information with the user terminal 3 via the communication unit 21 and the network 11.

[0017] The storage unit 22 stores various pieces of information defined above. This may be implemented, for example, as a storage device such as a solid state drive (SSD) that stores various programs related to the information processing device 2 executed by the control unit 23, or as a memory such as a random access memory (RAM) that stores temporarily required information (arguments, arrays, etc.) related to program calculations. Alternatively, it may be implemented as a combination of these.

[0018] The control unit 23 processes and controls the overall operations related to the information processing device 2. The control unit 23 is, for example, a central processing unit (CPU) not shown. The control unit 23 realizes various functions related to the information processing device 2 by reading out predetermined programs stored in the storage unit 22. In other words, information processing by software stored in the storage unit 22 is specifically realized by the control unit 23, which is an example of hardware.

[0019] 1.3 User terminal 3 The user terminal 3 is a terminal carried by a user. In one embodiment, the user terminal 3 is a terminal operated by a user who is a trainee doctor receiving instruction on a procedure from a supervising doctor. The user terminal 3 may be in any form, such as a smartphone, a tablet terminal, or a computer, as long as it can access the information processing device 2 via a telecommunications line.

[0020] 3 is a block diagram showing the hardware configuration of the user terminal 3. The user terminal 3 includes a communication bus 30, a communication unit 31, a storage unit 32, a control unit (processor) 33, a display unit 34, an input unit 35, and an imaging unit 36. The communication unit 31, the storage unit 32, the control unit 33, the display unit 34, the input unit 35, and the imaging unit 36 are electrically connected via the communication bus 30 inside the user terminal 3. The description of the communication unit 31, the storage unit 32, and the control unit 33 is omitted because it is the same as the description of each unit in the information processing device 2.

[0021] The display unit 34 displays a screen of a graphical user interface (GUI) that can be operated by the user. The display unit 34 may be included in the housing of the user terminal 3 or may be attached externally. Specifically, the display unit 34 may be implemented as a display device such as a CRT display, a liquid crystal display, an organic EL display, or a plasma display. It is preferable that these display devices are implemented by selectively using them depending on the type of the user terminal 3.

[0022] The input unit 35 accepts operation inputs made by the user. The operation inputs are transferred as command signals to the control unit 33 via the communication bus 30. The control unit 33 can execute predetermined control or calculations based on the transferred command signals as necessary. The input unit 35 may be included in the housing of the user terminal 3 or may be attached externally. For example, the input unit 35 may be implemented as a touch panel integrated with the display unit 34. When the input unit 35 is implemented as a touch panel, the user can input tap operations, swipe operations, etc. to the input unit 35. Instead of a touch panel, a switch button, a mouse, a QWERTY keyboard, etc. can be used as the input unit 35.

[0023] The imaging unit 36 is a so-called vision sensor (camera) configured to be able to capture information of the external world. The imaging unit 36 may be included in the user terminal 3 or may be externally attached. If it is externally attached, it may be electrically connected to the communication bus 30 using a very general standard such as a USB connection. The imaging unit 36 is configured to capture images of a procedure performed by the user. A video file including the user's procedure may be recorded in the storage unit 32 as time-series data of the captured images. Details will be described later.

[0024] 2. Functional configuration This section describes the functional configuration of one embodiment. As described above, information processing by software stored in the storage unit 22 is specifically realized by the control unit (processor) 23, which is an example of hardware, and can be executed as each functional unit included in the control unit 23.

[0025] 4 is a block diagram showing functions realized by the control unit 23 and the like in the information processing device 2 according to an embodiment. Specifically, the information processing device 2, which is an example of the information processing system 1, includes a receiving unit 231, a reading unit 232, an identifying unit 233, a generating unit 234, and a display control unit 235.

[0026] The receiving unit 231 is configured to receive various types of information. For example, the receiving unit 231 may receive image data transmitted from the user terminal 3.

[0027] The reading unit 232 is configured to read out various pieces of information that have been received by the receiving unit 231 or that have been stored in advance in the storage unit 22. For example, the reading unit 232 may read out image data that the receiving unit 231 has received.

[0028] The identification unit 233 is configured to identify various pieces of information based on the image data. For example, the identification unit 233 is configured to identify, based on the image data, first information IF1 and second information IF2 corresponding to the image data. This will be described in further detail below.

[0029] The generation unit 234 is configured to generate various display information to be displayed on the display unit 34 of the user terminal 3. The display information may be information itself generated in a manner that is visible to the user, such as a screen, an image, an icon, a message, etc., or may be rendering information for displaying a screen, an image, an icon, a message, etc. on the display unit 34 of the user terminal 3, for example.

[0030] The display control unit 235 displays the display information generated by the generation unit 234 in a manner that is visible to the user. For example, the display control unit 235 may display a screen or the like containing various information in a manner that is visible on a terminal such as the user terminal 3. This allows various pieces of information to be presented to various users who operate terminals such as the user terminal 3. Furthermore, the display control unit 235 controls visual information such as screens, images, icons, and messages to be displayed on the display unit 34 of the user terminal 3.

[0031] 3. Operation of Information Processing System 1 In this section, the information processing method of the information processing system 1 described above will be explained with reference to the drawings. The order of the processes can be changed as appropriate, multiple processes may be executed simultaneously, or some processes may be omitted. Here, a medical procedure (hereinafter simply referred to as a procedure) is a technique using hands in the medical field, and includes treatments and surgeries performed by medical professionals on patients.

[0032] 3.1 Overview 5 is a flowchart showing an outline of processing executed by the information processing system 1. In this processing, first, first image data IM1 including the procedure performed by the user is read (step S001). Next, the display control unit 235 displays an evaluation screen including the first image data IM1 and evaluation support information in a browsable manner as display information (step S002). The evaluation screen may be, for example, an evaluation input screen 7 shown in FIG. 9, which will be described later.

[0033] In summary, the information processing system 1 according to one embodiment includes at least one processor (e.g., the control unit 23), and the processor is configured to execute a program to function as each of the following units. As a reading step, the reading unit 232 reads out first image data IM1 including a medical procedure performed by a user. As a first display control step, the display control unit 235 displays an evaluation screen for evaluating the procedure so that the user can view it. Here, the evaluation screen includes the first image data IM1 and evaluation support information for evaluating the user's procedure in a configurable manner. According to this aspect, it is possible to evaluate whether the user is performing each action in the procedure and provide feedback to the user.

[0034] 3.2 Specific examples FIG. 6 is an activity diagram illustrating a specific example of processing executed by the information processing system 1. The specific example may fall within the scope of the overview described above. The specific example describes a case in which a user, a trainee doctor receiving instruction on a procedure from a supervising doctor, is supported in self-training to master the procedure. The self-training of a procedure by the user is described as a case in which the user learns the procedure by watching a model video, performs the learned procedure, and evaluates the user's own procedure using image data including the performance of the procedure. Here, performing the procedure includes not only practicing using a model or simulator, but also performing the procedure in an actual surgery. In the specific example, the evaluation support information in the overview is described as a checklist CH consisting of at least one check item. According to this aspect, the user's procedure can be evaluated in accordance with notes for each action in the procedure.

[0035] First, as a receiving step, the receiving unit 231 of the information processing device 2 receives a selection of a procedure from the user via the user terminal 3 (activity A101). In other words, as a second receiving step, the receiving unit 231 receives an input from the user selecting at least one of a plurality of procedures managed in advance. For example, the receiving unit 231 receives the selection of a procedure by receiving a selection input from the user on a screen displayed on the display unit 34 of the user terminal 3. Here, the display unit 34 may display the plurality of procedures in a browsable manner. Preferably, information indicating the relevance of the plurality of procedures may also be displayed on the display unit 34. For example, each procedure may be assigned a tag indicating the characteristics or attributes of the procedure, and a plurality of procedures with high relevance may be displayed so that the user can grasp them based on the tags. The manner of receiving the selection of a procedure is not limited thereto. In one embodiment, the procedure identified by receiving a selection input from the user is referred to as the specific procedure SM. In addition, in one embodiment, it is assumed that "vascular anastomosis" is selected as the specific procedure SM.

[0036] Next, as a second display control step, the display control unit 235 displays a learning screen 6 (activity A102) so that the user can view it, allowing the user to learn the procedure received in activity A101. Here, the learning screen 6 includes, in a visible format, second image data IM2 including a reference procedure RM (a model corresponding to the procedure), first information IF1 explaining the model's movements, and second information IF2 regarding the doctor's tacit knowledge related to the movements. In other words, the reference procedure RM is a model corresponding to the procedure related to the input, and may be a procedure performed by an experienced doctor who is in a position to instruct trainees.

[0037] Specifically, the second image data IM2 is generated by capturing an image of a procedure performed by an expert surgeon using an imaging device (not shown). Here, the procedure performed by an expert surgeon includes procedures using a model or simulator, and procedures performed in actual surgery. The second image data IM2 captured by the imaging device may be transmitted to the information processing device 2 and stored in a storage area. According to this aspect, it is possible to visually recognize the expert surgeon's tacit knowledge and thoughts when performing each action, thereby promoting learning and enabling more accurate transmission or sharing of information related to the procedure. The first information IF1 and the second information IF2 will be described in detail later.

[0038] Meanwhile, the user may use the imaging unit 36 of the user terminal 3 to image the procedure (specific procedure SM) that the user will perform. That is, the imaging unit 36 captures an image of the user's procedure (activity A103). Then, first image data IM1, which is an example of the captured image, is transmitted to the information processing device 2 via the network 11 (activity A104). Thereafter, the receiving unit 231 of the information processing device 2 receives the first image data IM1 including the specific procedure SM captured by the imaging unit 36 as a receiving step.

[0039] In other words, the imaging unit 36 of the user terminal 3 captures an image of a procedure performed by the user to generate the first image data IM1. Subsequently, as a first receiving step, the receiving unit 231 of the information processing device 2 receives the first image data IM1 from the user terminal 3 and stores it in a storage area. According to this aspect, the user can evaluate their own procedure by reading out captured data of the procedure performed by the user. Alternatively, the captured frames may be processed in real time to continuously generate the first image data IM1, and the receiving unit 231 may receive the first image data IM1. Alternatively, the first image data IM1 may be captured in advance by the imaging unit 36 or another imaging device (not shown). For example, the first image data IM1 may be stored in the storage unit 32 in advance, and the receiving unit 231 may receive the first image data IM1 by the user uploading the first image data IM1 to the information processing device 2. In this case, the storage unit 22 stores the first image data IM1 received by the receiving unit 231.

[0040] Next, as a reading step, the reading unit 232 of the information processing device 2 reads out the first image data IM1 received from the user terminal 3 in activity A104 (activity A105). Also, as a reading step, the reading unit 232 further reads out second image data IM2 including a model corresponding to the procedure related to the first image data IM1 (activity A106).

[0041] Next, as a first display control step, the display control unit 235 of the information processing device 2 displays the second image data IM2 in an evaluation screen so as to be comparable with the first image data IM1 (activity A107). Here, the evaluation screen is, for example, an evaluation input screen 7 shown in FIG. 9, which will be described later. According to this embodiment, the user's technique can be evaluated by comparing it with the technique of a skilled doctor who serves as a model.

[0042] Next, as a receiving step, the receiving unit 231 of the information processing device 2 receives input of information related to the evaluation of the procedure associated with the first image data IM1 from the user via the user terminal 3 (activity A108). Details will be described later.

[0043] Finally, as a display control step, the display control unit 235 causes the display unit 34 of the user terminal 3 to display the evaluation result based on the information input from the user in activity A108 (activity A109). Specifically, the identification unit 233 identifies the user's evaluation of the specific procedure SM based on the information input in activity A108. Next, the display control unit 235 causes the display unit 34 of the user terminal 3 to display display information generated based on the identified evaluation.

[0044] The above is the flow of processing according to the specific example.

[0045] 4. Related technical matters Next, technical matters related to the information processing explained in FIG. 6 will be explained with reference to FIGS.

[0046] (Learning table T1) FIG. 7 is a schematic diagram of the learning table T1. The learning table T1 is a lookup table that shows information about each movement of the procedure. The generation unit 234 of the information processing device 2 generates display information to be displayed on the learning screen 6 (described later) based on the information stored in the learning table T1. The learning table T1 includes first information IF1, second information IF2, and information related to the checklist CH, and each piece of information is associated with another. The learning table T1 may be stored, for example, in the storage unit 22 of the information processing device 2.

[0047] For example, the learning table T1 may include information such as a procedure ID (T11), an action ID (T12), video data (T13), doctor's thoughts (T14), and an explanation of the action (T15). For example, the "procedure ID" may correspond to the procedure presented to the user in activity A101 of FIG. 6. As shown in FIG. 7, the "video data" may be, for example, second image data IM2. The second image data IM2, which captures the supervising doctor's procedure, is stored in association with the first information IF1, which is an "explanation of the action," and the second information IF2, which is "doctor's thoughts." Note that in this embodiment, the "video data" is described as data in which the procedure is divided into individual actions and recorded, but this is not limited thereto. The video data may include data capturing multiple actions, or may be data capturing a series of actions related to the procedure. In such a case, the "video data" may have time information corresponding to each of the multiple actions included in a single captured video.

[0048] The first information IF1 is information that explains the movements of the reference technique RM (a model corresponding to the technique). The first information IF1 is, for example, information that verbalizes the method of the movements.

[0049] The second information IF2 is information regarding the doctor's tacit knowledge when performing the action related to the procedure. The second information IF2 is information verbalizing the thoughts of an experienced doctor based on his knowledge and experience, such as precautions and key points when performing the action. In other words, the first information IF1 and the second information IF2 are text information or audio information obtained by reading the text information. According to this aspect, the experienced doctor's tacit knowledge can be grasped through linguistic information. Note that the first information IF1 and the second information IF2 may be information other than text information or audio information obtained by reading the text information. For example, the first information IF1 and the second information IF2 may be visual information such as symbols. For example, the display control unit 235 of the information processing device 2 may display the first information IF1 as text information on the display unit 34 of the user terminal 3, and may also display visual information in the image data at a position corresponding to the part identified based on the first information IF1 or the second information IF2. This allows the user to more easily grasp the key points when performing the action.

[0050] Furthermore, as shown in the learning table T1, a "procedure ID" is associated with a plurality of "action IDs" and "video data." Furthermore, each piece of "video data" is associated with first information IF1 and second information IF2. In other words, a procedure is made up of a plurality of actions, and the second image data IM2 is data obtained by dividing the recorded procedure into each action. Furthermore, the first information IF1 and the second information IF2 are associated with the actions of the second image data IM2. Note that a single piece of "video data" (second image data IM2) may be associated with a plurality of pieces of first information IF1 and second information IF2. According to this aspect, the tacit knowledge and thinking of an experienced physician can be grasped for each action.

[0051] (Learning screen 6) Fig. 8 shows an example of a learning screen 6 for viewing a video including a procedure, which is displayed on the display unit 34 of the user terminal 3. The learning screen 6 is an example of a screen displayed in activity A102 of Fig. 6. The learning screen 6 includes areas 61 to 64.

[0052] Area 61 is an area where second image data IM2 corresponding to the movements in the specific procedure SM is displayed. In this embodiment, the second image data IM2 is displayed in area 61 in a manner where image data IM21 captured so that the procedure is enlarged and image data IM22 captured so that the entire hand performing the procedure is captured are combined, but this is not limited to this. For example, the second image data IM2 may be displayed in area 61 in a manner where multiple image data captured from different directions are combined into one image data. Furthermore, the second image data IM2 may be a single image. Area 62 is an area for playing back the second image data IM2.

[0053] Area 63 is an area where first information IF1 and second information IF2 corresponding to actions in the specific procedure SM are displayed. Area 63 is an area where information corresponding to actions related to second image data IM2, which is displayed in area 61, is displayed. Area 63 includes areas 631 to 634. Area 631 is an area where first information IF1 is displayed. Areas 632 to 634 are areas where second information IF2 is displayed.

[0054] In the present embodiment, the first information IF1 and the second information IF2 are displayed in the area 63, which is separate from the area where the second image data IM2 is displayed, but this is not limiting. For example, the display control unit 235 may display at least one of the first information IF1 and the second information IF2 in a manner superimposed on the second image data IM2.

[0055] As shown in Fig. 8, the learning screen 6 includes, in a browsable manner, second image data IM2, and first information IF1 and second information IF2 corresponding to the second image data IM2. The identification unit 233 of the information processing device 2 identifies the second image data IM2 corresponding to the specific procedure SM received in activity A101 of Fig. 6. Furthermore, the identification unit 233 identifies the first information IF1 and second information IF2 corresponding to the identified second image data IM2 based on the learning table T1 of Fig. 7. Next, the generation unit 234 generates display information for displaying to the user the second image data IM2, first information IF1, and second information IF2 identified by the identification unit 233 in a browsable manner.

[0056] Furthermore, the area 64 is an area for playing or changing the second image data IM2. The area 64 includes objects 641 to 643. For example, in response to the reception unit 231 receiving an input for the object 641 from the user, the display control unit 235 displays the second image data IM2 corresponding to the first information IF1 and the second information IF2 displayed in the area 63 in a played-back manner. In such a case, the display control unit 235 may display the second image data IM2 in a repeatedly played-back manner. This allows the user to repeatedly watch a video of a predetermined action that the user wants to check. Furthermore, when the reception unit 231 receives an input for the object 642 from the user, the display control unit 235 changes the second image data IM2 to be displayed in the area 61 to image data related to a previous action in the procedure, and displays the changed image data. Similarly, when the accepting unit 231 accepts input for the object 643 from the user, the display control unit 235 changes the second image data IM2 to be displayed in the area 61 to image data related to the next action in the procedure and displays the changed data. In this case, the identifying unit 233 identifies the first information IF1 and the second information IF2 corresponding to the changed image data, and the display control unit 235 displays the changed image data and the identified first information IF1 and second information IF2 in a visible manner on the display unit 34. This makes it possible to easily change the image data of the procedure displayed on the display unit 34 to image data related to any action of the user. Note that the second image data IM2 displayed on the display unit 34 in response to accepting input for the object 642 or 643 from the user may be a moving image or a still image.

[0057] Furthermore, the method of playing back the second image data IM2 is not limited to this. For example, in response to the reception unit 231 receiving from the user a selection of image data corresponding to one or more arbitrary actions among a plurality of actions included in the procedure, the display control unit 235 may display on the display unit 34 one or more image data selected by the user in a manner that repeatedly plays back the image data. According to this manner, the image data selected by the user can be repeatedly played back, allowing the user to easily check the actions that he or she wants to check.

[0058] Furthermore, the operation for playing the video or the operation for changing the image data to be displayed is not limited to the mode in which an input to an object included in the area 64 is received from the user. For example, the receiving unit 231 of the information processing device 2 may receive an input from the user to a keyboard, which is an example of the input unit 35 of the user terminal 3. According to such a mode, the user can easily perform an operation for switching the playback of the video even while actually performing a procedure.

[0059] (Evaluation input screen 7) 9 shows an example of an evaluation input screen 7 for inputting an evaluation of a procedure, which is displayed on the display unit 34 of the user terminal 3. The evaluation input screen 7 is an example of a screen displayed in activity A108 of FIG. 6. The evaluation input screen 7 includes areas 71 to 74 and an object 76.

[0060] Area 71 is an area where first image data IM1 including the user's own technique corresponding to the specific technique SM is displayed. Area 72 is an area where second image data IM2 including the reference technique RM corresponding to the specific technique SM is displayed. That is, the display control unit 235 causes the user terminal 3 to display the first image data IM1 related to the user's technique and the second image data IM2 related to the reference technique RM in a visible manner. According to this aspect, the user can visually compare their own technique with the instructor's technique and understand the difference from the instructor's technique.

[0061] Area 74 is an area where a checklist CH for the actions corresponding to the specific procedure SM is displayed. Area 74 includes area 741 and check boxes CB1 to CB3. Area 741 is an area where first information IF1, which is information that verbalizes the content of the action, is displayed.

[0062] Here, the checklist CH may be generated based on, for example, a learning table T1. The generation unit 234 of the information processing device 2 generates display information including the checklist CH based on, for example, the "doctor's thoughts T14 (IF2)" corresponding to the specific procedure SM. Note that, in one embodiment, the second information IF2 and the items included in the checklist CH are described as corresponding to each other, but this is not limited thereto. The second information IF2 and the items in the checklist CH may have different contents.

[0063] The check boxes CB1 to CB3 are user interfaces that accept input from the user regarding the evaluation of the user's own procedure. In the example of FIG. 9, check boxes CB1 and CB2 accept input from the user. Specifically, it is confirmed that the user has evaluated their own actions as being in accordance with the items in check box CB1, "Holding down the graft with forceps," and in CB2, "Using the greater curvature (dorsal) side of the needle to press down on the non-insertion side of the needle to expand the insertion point." In one embodiment, the confirmation items in check boxes CB1 to CB3 will be described as corresponding to the second information IF2.

[0064] The object 76 is an object for displaying the evaluation result on the display unit 34 of the user terminal 3. For example, by receiving an input for the object 76 from the user, the display control unit 235 of the information processing device 2 transitions the content displayed on the display unit 34 of the user terminal 3 from the evaluation input screen 7 of FIG. 9 to a result display screen 8 of FIG. 11 (described later). This allows the user to understand more detailed feedback on the evaluation of their own procedure. Furthermore, by receiving an input for the object 76 from the user, the receiving unit 231 may store information related to the input received for the check box CB in the storage unit 22. Furthermore, the identifying unit 233 may identify detailed information related to the evaluation in response to receiving an input for the object 76 from the user. Details will be described with reference to FIG. 11. Note that, in the present embodiment, an aspect has been described in which the information is stored in the storage unit 22 in response to an input for the object 76; however, this is not limited thereto. For example, the evaluation table T2 may be updated each time an input for the check box CB is received from the user.

[0065] In this embodiment, the first image data IM1 may also be in a form in which the procedure included in the data is divided into individual movements, similar to the second image data IM2. Specifically, for example, the first image data IM1 may be divided so that the multiple movements included in the first image data IM1 correspond to the divided movements in the second image data IM2. The method for dividing the first image data IM1 is not particularly limited. For example, the first image data IM1 may be divided manually. Alternatively, for example, the first image data IM1 may be divided by using an artificial intelligence equipped with a learning model that has learned image data samples as training data, inputting the first image data IM1 into the artificial intelligence, and outputting the image data in the divided form.

[0066] More preferably, the display control unit 235 may cause the display unit 34 to display the first image data IM1 divided into each action and the second image data IM2 corresponding to the action of the first image data IM1 in a synchronized manner. In other words, as a first display control step, the display control unit 235 further causes the first image data IM1 and the second image data IM2 to be displayed in a synchronized manner based on the actions included in the first image data IM1 and the second image data IM2, respectively. This manner allows the user to more easily compare the procedure performed by the user with the procedure performed by the supervising physician.

[0067] (Evaluation table T2) FIG. 10 is a schematic diagram of the evaluation table T2. The evaluation table T2 is a lookup table showing information for performing an evaluation for each action of a procedure. The evaluation table T2 has a configuration in which records are updated, for example, when an evaluation input is received from a user via the user terminal 3. Furthermore, the generation unit 234 of the information processing device 2 generates display information to be displayed on the result display screen 8 (described later) based on the information stored in the evaluation table T2. The evaluation table T2 includes, as evaluation support information, information for generating a checklist CH consisting of at least one check item. The evaluation table T2 may be stored in the storage unit 22 of the information processing device 2.

[0068] For example, the evaluation table T2 may include information such as implementation date and time (T21), user ID (T22), technique ID (T23), action ID (T24), evaluation item (T25), and evaluation (T26). The "implementation date and time" is information indicating the timing when the user performed the technique training. Furthermore, the "user ID" is information that can identify the user who performed the evaluation.

[0069] The "evaluation item (T25)" includes information for checking each movement in the procedure. For example, the "evaluation item (T25)" may include information that should be noted when comparing the user's own procedure with the reference procedure RM. The "evaluation item (T25)" is associated with, for example, each "movement ID (T24)." Here, one evaluation item may be associated with one movement ID, or multiple evaluation items may be associated with one movement ID.

[0070] "Evaluation (T26)" includes information indicating the evaluation of each action in the procedure, input from the user. For example, if the user's action satisfies the viewpoint indicated by the corresponding "Evaluation Item (T25)", "True" is stored in "Evaluation".

[0071] The evaluation table T2 is configured such that records are updated when an evaluation input is received from a user via the user terminal 3, and stores information indicating the user's evaluation of the procedure. According to this embodiment, the information is analyzed and presented to the user, thereby further supporting the user's training of the procedure.

[0072] (Result display screen 8) 11 shows an example of a result display screen 8 that shows the results of analyzing the evaluation, and is displayed on the display unit 34 of the user terminal 3. The result display screen 8 is an example of a screen that is displayed in activity A109 in FIG. 6. The result display screen 8 includes areas 81 to 83.

[0073] Area 81 is an area where the type of technique that the user has trained, which is the subject of evaluation, is displayed, for example, a specific technique SM. Area 82 is an area where information indicating an evaluation of the user's action is displayed. Identification unit 233 may identify the information based on the evaluation received from the user on evaluation input screen 7.

[0074] Area 83 is an area where an evaluation result table T3 is displayed. The evaluation result table T3 is a table that shows, for example, details of the evaluation of the user's actions related to the specific procedure SM in a manner that is visible to the user. The evaluation result table T3 may include, for example, information such as confirmation items (T31), current evaluation (T32), previous evaluation (T33), and user comparison (T34).

[0075] The "Confirmation Items (T31)" are, for example, items that require attention when performing an action, and may correspond to the contents of the check boxes CB in the checklist CH. The "Current Evaluation (T32)" corresponds to the input into the check boxes CB received from the user on the evaluation input screen 7. Furthermore, as shown in the "Previous Evaluation (T33)", the input results of the user's evaluation of the previous training may be displayed. This allows a comparison of the action with the previous training.

[0076] Furthermore, "User Comparison (T34)" may be an aspect in which information that allows a comparison result of the user with a plurality of other users to be understood is displayed. Here, the plurality of other users are, for example, users who have the same attributes as the user (such as the length of clinical training, the number of training sessions for the specific procedure SM, etc.), and may be identified by the identification unit 233.

[0077] For example, graph GR1 related to "User Comparison (T34)" shows the result that "the percentage of users who rated the actions in check item 1 that user U001 rated as 'able' as 'able' among users with the same attributes is 80%." On the other hand, graph GR3 shows the result that "the percentage of users who rated the actions in check item 1 that user U001 rated as 'not able' as 'able' among users with the same attributes is 30%." In other words, graph GR3 shows that "the actions related to check item 3 include actions that many users are likely to get wrong." Such analysis results can be identified based on evaluation table T2 in FIG. 10.

[0078] According to this aspect, by collecting information related to evaluation input received from the user, it is possible to analyze the difficulty level of each technique, thereby further supporting the user's self-training.

[0079] According to the information processing system 1 of the first embodiment, it is possible to evaluate whether the user is performing each movement in the procedure and provide feedback to the user, thereby further supporting the user's self-training.

[0080] (List display screen 9) 12 shows an example of a list display screen 9 showing a list of the manual trainings performed by the user, which is displayed on the display unit 34 of the user terminal 3. The list display screen 9 includes an area 91, objects 921 to 923, areas 931 to 933, and an area 94.

[0081] Area 91 is an area where the training of the technique that the user has trained is displayed. For example, the display control unit 235 displays the name of the specific technique SM that has been selected and accepted by the user.

[0082] Objects 921-923 are objects for displaying image data of each procedural training on the display unit 34 of the user terminal 3. For example, when the receiving unit 231 of the information processing device 2 receives input for the object 921 from the user, the display control unit 235 transitions the content displayed on the display unit 34 of the user terminal 3 from the list display screen 9 of FIG. 12 to a screen on which the user can view first image data IM1 related to the procedural training performed on December 9th. This screen preferably includes an area for displaying the first image data IM1 related to the user's own procedural technique and the second image data IM2 related to the reference procedural technique RM, as shown in FIG. 9 . Furthermore, this screen preferably includes an area for displaying a checklist CH for the actions corresponding to the specific procedural technique SM and evaluation information received from the user for the checklist CH. This aspect allows the user to easily view video data of any procedural training they have performed.

[0083] 11. In addition, by receiving an input for the object 922 from the user, the display control unit 235 of the information processing device 2 changes the content to be displayed on the display unit 34 of the user terminal 3 from the list display screen 9 of FIG. 12 to the list display screen 9 of FIG. 12 to the result display screen 8 showing the results of the manual training that the user performed on December 1st. According to this aspect, the user can easily check a detailed evaluation of any manual training that the user performed.

[0084] Areas 931 to 933 are areas where information related to each of the procedural trainings is displayed. For example, the information displayed in areas 931 to 933 may include the date of the procedural training, a self-evaluation value for the procedural training, and information regarding whether or not the training has been evaluated by a supervising physician.

[0085] Area 94 is an area where graph GR4 showing changes in the evaluation value for the user's procedure is displayed. Graph GR4 has axis 94x, which indicates the time when the procedure was performed, as the horizontal axis, axis 941y, which indicates the evaluation value, as the left vertical axis, and axis 942y, which indicates the time required for the procedure, as the right vertical axis. Display control unit 235 displays information showing changes in the user's skill level for the procedure in a manner in which the time required for the procedure and the evaluation value for the procedure are plotted. Specifically, the time on axis 94x corresponds to the date on which the procedure was performed, which are displayed in areas 931 to 933. This manner allows the user to understand changes in their skill level for the procedure.

[0086] [others] The information processing system 1 according to the above embodiment may be configured as follows.

[0087] The information processing system 1 may further include an augmented reality presentation device (not shown). The augmented reality presentation device may be configured as, for example, a head-mounted display device that a user can wear to cover the area around the user's eyes, or as smart glasses that allow the user to directly view information about the outside world while superimposing information related to augmented reality, and may be connected to the information processing device 2 and the user terminal 3 via a network 11.

[0088] In activity A103 of FIG. 6, this may be performed by a user wearing an augmented reality presentation device on their head directing their gaze toward their own procedure so that it is within the field of view of the imaging unit of the augmented reality presentation device. In such a case, the display control unit 235 of the information processing device 2 may display on the display unit of the augmented reality presentation device a state in which the external world including the captured image of the user's procedure (specific procedure SM) is superimposed on the corresponding first information IF1 and second information IF2 so that the user can visually recognize the state. Specifically, the reading unit 232 reads out the first information IF1 and second information IF2 corresponding to the specific procedure SM received in activity A101 of FIG. 6. Furthermore, the generation unit 234 generates display information to be displayed on the display unit of the augmented reality presentation device based on the read first information IF1 and second information IF2 and the image captured by the imaging unit of the augmented reality presentation device. In this embodiment, the display control unit 235 of the information processing device 2 may further include corresponding second image data IM2 as information to be superimposed on the captured image of the external world. According to this embodiment, the user can perform the procedure while checking the reference procedure RM that serves as a model.

[0089] In other words, in a third receiving step, the receiving unit 231 receives third image data IM3, which is an image of the external world visually recognized by the user. The generating unit 234 generates display information for presenting augmented reality to the user based on the user's procedure included in the third image data IM3. The augmented reality is reproduced in such a manner that first information IF1 explaining the model movement corresponding to the user's procedure and second information IF2 regarding the doctor's tacit knowledge related to the movement are superimposed on the third image data IM3. This manner allows the user to check for points of caution while comparing the state of the procedure in progress, which is expected to further improve usability.

[0090] Preferably, in the augmented reality, the medical instrument used in the reference procedure RM (a model corresponding to the procedure) or the manner of the hand manipulating the medical instrument is further reproduced in three dimensions. Specifically, the generation unit 234 generates display information to be displayed on the display unit of the augmented reality presentation device based on information related to the three-dimensional position of the tracking point in the reference procedure RM. The information related to the three-dimensional position in the reference procedure RM may be extracted from the second image data IM2 and stored in advance in association with the reference procedure RM. According to this embodiment, the user can further confirm the model procedure while comparing it with the manner of his or her own current procedure, which is expected to further improve usability.

[0091] 6, the imaging unit 36 of the user terminal 3 may simultaneously capture images of the user's procedure from different directions. In such a case, the first image data IM1 displayed on the evaluation input screen 7 of FIG. 9 may be in a form in which multiple image data captured from different directions are combined into one image data. This form allows the user to more accurately understand their own procedure.

[0092] Furthermore, in the above embodiment, the evaluation support information is described as a checklist CH, but this is not limited thereto. For example, the evaluation support information may be an object that triggers the calculation of an evaluation result. For example, when the user presses the object 76 on the evaluation input screen 7 of FIG. 9, a calculation for evaluating the user's technique based on the first image data IM1 and the corresponding second image data IM2 may be executed. Specifically, the identification unit 233 may identify information for evaluating the user's technique depending on the degree of difference between the first image data IM1 and the corresponding second image data IM2. This aspect allows the user to grasp an objective evaluation of their own technique.

[0093] Furthermore, in the above embodiment, check boxes have been described as an example of a user interface for receiving input from the user regarding the evaluation of the procedure, but this is not limited to this. The area for receiving the user's input of the evaluation may be, for example, a mode in which selection of information indicating a stage using numbers or symbols is received for each check item.

[0094] Furthermore, in the above embodiment, the user himself / herself inputs data into the checklist CH, but this is not limited thereto. For example, the receiving unit 231 of the information processing device 2 may receive data into the checklist CH from a supervising physician who is supervising the user. Specifically, the display control unit 235 may display the evaluation input screen 7 of FIG. 9 on a display unit (not shown) of a terminal carried by the supervising physician who is supervising the user. In such a case, the evaluation input screen 7 may be configured to receive comments from the supervising physician. Specifically, the evaluation input screen 7 may include a text input area (not shown), and the receiving unit 231 may receive text input from the supervising physician regarding comments on the user's procedure. This makes it possible to easily receive feedback from the supervising physician.

[0095] Furthermore, the learning screen 6 and the evaluation input screen 7 may be configured to accept questions from the user to the supervising physician. Specifically, the learning screen 6 and the evaluation input screen 7 may include a text input area (not shown), and the accepting unit 231 may accept text input related to questions from the user to the supervising physician. This allows the user to easily ask questions to the supervising physician.

[0096] Furthermore, on the evaluation input screen 7, the display control unit 235 may display the check status by the user himself / herself and the check status by the supervising physician or other users in a manner that allows the user to see them at a glance. Specifically, the evaluation input screen 7 may include an area for showing the results of the supervising physician's check, in addition to check boxes CB1 to CB3 that the user can use to check his / her own technique. In such a case, the evaluation table T2 in FIG. 10 may further include an item for storing the ID of the person who made the evaluator. This allows the user to compare his / her own evaluation of his / her technique with the evaluations of others.

[0097] Preferably, the display control unit 235 may be configured to limit the viewing range of the first image data IM1. For example, the display control unit 235 may control the first image data IM1 captured by the user to be viewable by other users and supervising physicians. More preferably, in response to receiving a selection regarding viewing restrictions from the user, the display control unit 235 may determine the users or supervising physicians who are allowed to view the first image data IM1. Furthermore, the receiving unit 231 is preferably configured to be able to receive evaluations and comments on the procedure related to the first image data IM1 from other users and supervising physicians who are allowed to view the first image data IM1. According to this aspect, the user can receive evaluations from other users and supervising physicians and can use the evaluation results to improve the procedure.

[0098] Furthermore, in the above embodiment, the display control unit 235 has been described as displaying an evaluation of the procedure for each evaluation item T25 as shown in Fig. 11, but this is not limited to this. The display control unit 235 may also display an evaluation for each element related to the procedure. Specifically, the actions related to one or more procedures may be classified into elements such as an action of inserting a needle, an action of removing a needle, and an action of changing hands with the needle, and the display control unit 235 may calculate and display an evaluation result for each element.

[0099] Furthermore, in the above embodiment, the information processing system 1 for supporting the user's self-training has been described as including the steps of learning by watching a model video, performing the learned technique, and evaluating the user's own technique using image data including the performance of the technique, but this is not limited thereto. For example, the information processing system 1 may be configured to include at least the step of learning by watching a video.

[0100] In other words, the information processing system 1 includes at least one processor (e.g., a control unit 23), and the processor is configured to execute a program to function as each of the following units: The receiving unit 231 receives an input from a user selecting at least one of a plurality of medical procedures managed in advance; the display control unit 235 displays a learning screen 6 for the user to view so as to learn the procedure; the learning screen 6 includes, in a visible format, second image data IM2 including a reference procedure RM (a model corresponding to the selected procedure), first information IF1 explaining the movements of the model, and second information IF2 regarding the doctor's tacit knowledge related to the movements; the procedure is composed of a plurality of movements, and the second image data IM2 is data in which the procedure is divided into each movement; and the first information IF1 and the second information IF2 are associated with the movements of the second image data IM2. According to this aspect, learning can be advanced by visually observing the tacit knowledge and thoughts of an experienced doctor when performing each action, and information can be transmitted or shared more accurately when learning about procedures.

[0101] The above-described embodiment may also be implemented as a distributable program. This program causes a computer to execute each step in the information processing system 1. Furthermore, an information processing method including each step of the information processing system 1 may also be implemented.

[0102] Furthermore, it may be provided in the following aspects.

[0103] (1) An information processing system comprising at least one processor, the processor being configured to execute a program to perform the following steps: a reading step reading first image data including a medical procedure performed by a user; a first display control step displaying an evaluation screen for evaluating the procedure so that the user can see it; the evaluation screen including, in a visible format, the first image data and evaluation support information for evaluating the user's procedure.

[0104] According to this aspect, it is possible to evaluate whether the user is able to perform each action in the procedure and provide feedback to the user.

[0105] (2) In the information processing system described in (1) above, the evaluation support information is a checklist consisting of at least one check item.

[0106] According to this aspect, the user's technique can be evaluated in accordance with the precautions for each operation.

[0107] (3) In the information processing system described in (1) or (2) above, in the reading step, second image data including a model corresponding to the procedure is further read out, and in the first display control step, the second image data is displayed on the evaluation screen so as to be comparable with the first image data.

[0108] According to this aspect, the user's technique can be evaluated by comparing it with the technique of an experienced doctor who serves as a model.

[0109] (4) In the information processing system described in (3) above, the first display control step further includes displaying the first image data and the second image data in a synchronized manner based on the operations contained in the first image data and the second image data, respectively.

[0110] (5) In the information processing system described in any one of (1) to (4) above, further, in the imaging step, the first image data is generated by imaging the procedure performed by the user, and in the first reception step, the first image data is received and stored in a memory area.

[0111] According to this aspect, by reading out image data of a procedure performed by a user, the user can evaluate his or her own procedure.

[0112] (6) In the information processing system described in any one of (1) to (5) above, further, in a second reception step, an input selecting at least one of a plurality of the procedures managed in advance is received from the user, and in a second display control step, a learning screen for allowing the user to learn the procedure is displayed so as to be visible to the user, wherein the learning screen includes, in a listable form, second image data including a model corresponding to the procedure, first information explaining the movements of the model, and second information regarding the doctor's tacit knowledge related to the movements.

[0113] According to this aspect, learning can be advanced by visually observing the tacit knowledge and thoughts of an experienced doctor when performing each action, and the transmission or sharing of information related to the procedure can be realized more accurately.

[0114] (7) In the information processing system described in (6) above, the first information and the second information are text information or audio information of the text information being read aloud.

[0115] According to this aspect, the tacit knowledge of an experienced doctor can be grasped through linguistic information.

[0116] (8) In the information processing system described in (6) or (7) above, the procedure is composed of a plurality of actions, the second image data is data in which the procedure is divided into each action, and the first information and the second information are associated with the actions of the second image data.

[0117] According to this aspect, the tacit knowledge and thoughts of the skilled doctor can be grasped for each action.

[0118] (9) In the information processing system described in any one of (1) to (5) above, further, in a third reception step, third image data capturing an image of the external world as seen by the user is received, and in a generation step, display information for presenting augmented reality to the user is generated based on the user's technique contained in the third image data, wherein the augmented reality is reproduced in a manner in which first information explaining a model movement corresponding to the technique and second information regarding the doctor's tacit knowledge related to the movement are superimposed on the third image data.

[0119] According to this aspect, the user can check the points that require attention while comparing with the aspect of his / her current procedure, and further improvement in usability is expected.

[0120] (10) In the information processing system described in (9) above, the augmented reality further reproduces in three dimensions the medical instrument used in the model or the manner of the hands manipulating the medical instrument.

[0121] According to this aspect, the user can compare the current aspects of the procedure with the model procedure, and further improve usability.

[0122] (11) An information processing system comprising at least one processor, the processor being configured to execute a program to perform the following steps: a reception step receives input from a user selecting at least one of a plurality of pre-managed medical procedures; a display control step displays a learning screen for the user to learn the procedure so that the learning screen is visible to the user; the learning screen includes, in a listable form, second image data including a model corresponding to the selected procedure, first information explaining the movements of the model, and second information regarding the doctor's tacit knowledge related to the movements; the procedure is composed of a plurality of movements; the second image data is data in which the procedure is divided into each of the movements; and the first information and the second information are associated with the movements of the second image data.

[0123] According to this aspect, learning can be advanced by visually observing the tacit knowledge and thoughts of an experienced doctor when performing each action, and information can be transmitted or shared more accurately when learning about procedures.

[0124] (12) In the information processing system described in (11) above, the first information and the second information are text information or audio information of the text information being read aloud.

[0125] According to this aspect, the tacit knowledge of an experienced doctor can be grasped through linguistic information.

[0126] (13) An information processing method, comprising the steps of the information processing system according to any one of (1) to (12) above.

[0127] According to this aspect, it is possible to evaluate whether the user is able to perform each action in the procedure and provide feedback to the user.

[0128] (14) A program that causes a computer to execute each step in the information processing system according to any one of (1) to (12) above.

[0129] According to this aspect, it is possible to evaluate whether the user is able to perform each action in the procedure and provide feedback to the user. Of course, this is not the case.

[0130] Finally, while various embodiments of the present invention have been described, these are presented by way of example only and are not intended to limit the scope of the invention. The novel embodiments may be embodied in various other forms, and various omissions, substitutions, and modifications may be made without departing from the spirit of the invention. Such embodiments and modifications are intended to be included within the scope and spirit of the invention, as well as within the scope of the inventions and their equivalents as defined in the accompanying claims. [Explanation of symbols]

[0131] 1: Information processing system 11: Network 2: Information processing equipment 20: Communication bus 21: Communications Department 22: Storage section 23: Control section 231: Reception 232:Reading section 233: Specific part 234 :Generation part 235: Display control unit 3: User terminal 30: Communication bus 31: Communications Department 32: Storage section 33: Control section 34:Display section 35: Input section 36: Imaging unit 4: Augmented reality device 6: Learning screen 61 :Area 62 :Area 63 :Area 631: area 632: area 633: area 634: area 64 :Area 641: Object 642: Object 643: Object 7: Evaluation input screen 71 :Area 72 :Area 73 :Area 74 :Area 741 :Area 76: Object 8:Result display screen 81 :Area 82 :Area 83 :Area 9: List display screen 921: Object 922: Object 923: Object 931: area 932: area 933: area 94 :Area 94x :axis 941y: Axis 942y: axis T1: Learning table T2: Evaluation table T3: Evaluation result table CB: Checkbox CB1: Checkbox CB2: Checkbox CB3: Checkbox CH: Checklist GR1: Graph GR3: Graph GR4:Graph SM: Specific procedure RM: Reference procedure IF1: First information IF2: Second information IM1: First image data IM2: Second image data IM21: Image data IM22: Image data IM3: Third image data

Claims

1. An information processing system, At least one processor is provided, the processor being configured to execute a program to perform the following steps: In the reading step, first image data including a medical procedure performed by a user is read; In a first display control step, an evaluation screen for evaluating the procedure is displayed so that it can be seen by the user, and the evaluation screen includes, in a visible format, the first image data and evaluation support information for evaluating the user's procedure.

2. 2. The information processing system according to claim 1, The evaluation support information is a checklist consisting of at least one check item.

3. 2. The information processing system according to claim 1, In the reading step, second image data including a model corresponding to the procedure is further read out; In the first display control step, the second image data is displayed on the evaluation screen so as to be comparable with the first image data.

4. 4. The information processing system according to claim 3, In the first display control step, the system further displays the first image data and the second image data in a synchronized manner based on the operations respectively contained in the first image data and the second image data.

5. 2. The information processing system according to claim 1, moreover, In the imaging step, the first image data is generated by imaging the procedure performed by the user; In the first receiving step, the system receives the first image data and stores it in a storage area.

6. 2. The information processing system according to claim 1, moreover, In the second receiving step, an input for selecting at least one of the plurality of procedures managed in advance is received from the user; In a second display control step, a learning screen for allowing the user to learn the procedure is displayed so that the user can see it, and the learning screen includes, in a visible format, second image data including a model corresponding to the procedure, first information explaining the movements of the model, and second information regarding the doctor's tacit knowledge related to the movements.

7. 7. The information processing system according to claim 6, The system, wherein the first information and the second information are text information or audio information obtained by reading the text information.

8. 7. The information processing system according to claim 6, The procedure is comprised of a plurality of movements, the second image data is data obtained by dividing the procedure into each of the movements, The system, wherein the first information and the second information are associated with the motion of the second image data.

9. 2. The information processing system according to claim 1, moreover, In the third receiving step, third image data obtained by capturing an image of an external environment visually recognized by the user is received; In the generation step, display information for presenting augmented reality to the user is generated based on the user's technique contained in the third image data, and the augmented reality is reproduced in a manner in which first information explaining a model movement corresponding to the technique and second information regarding the doctor's tacit knowledge related to the movement are superimposed on the third image data.

10. 10. The information processing system according to claim 9, In the augmented reality system, the medical instrument used in the model or the manner in which the hand manipulates the medical instrument is further reproduced in three dimensions.

11. An information processing system, At least one processor is provided, the processor being configured to execute a program to perform the following steps: In the receiving step, an input for selecting at least one of a plurality of medical procedures managed in advance is received from a user; In the display control step, a learning screen for the user to learn the procedure is displayed so as to be visible to the user, and the learning screen includes, in a visible format, second image data including a model corresponding to the selected procedure, first information explaining the movement of the model, and second information regarding the doctor's tacit knowledge related to the movement, The procedure is comprised of a plurality of movements, the second image data is data obtained by dividing the procedure into each of the movements, The system, wherein the first information and the second information are associated with the motion of the second image data.

12. 12. The information processing system according to claim 11, The system, wherein the first information and the second information are text information or audio information obtained by reading the text information.

13. An information processing method, comprising: A method comprising the steps of the information processing system according to any one of claims 1 to 12.

14. A program, A program that causes a computer to execute each step in the information processing system according to any one of claims 1 to 12.

Images