Device for learning to operate an operating tool and method for learning to operate an operating tool

Abstract

To provide an operation learning device of an operation tool which allows a user to recognize a difference between an operation of a teacher operation tool and an operation of a user operation tool if there is the difference, and applies a corrective force.SOLUTION: The operation learning device of the operation tool is configured to stop updating a teacher image to be displayed on a display unit when a distance between a position of a teacher operation tool associated with the teacher image and a position of a user operation tool is greater than or equal to a first criterion, and to control a force detection unit to generate a force to bring the position of the user operation tool closer to the position of the teacher operation tool when the position of the teacher operation tool associated with the teacher image and the position of the user operation tool do not match. The operation learning device is also configured to display a user operation tool image representing the user operation tool on the display unit while overlapping it with the teacher image so as to show a relation between the position of the user operation tool and the position of the teacher operation tool.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an apparatus for learning to operate a manipulator and a method for learning to operate a manipulator. [Background technology]

[0002] Advances in medical technology and medical equipment have led to an increase in abdominal surgery performed laparoscopically. Laparoscopic surgery requires training because it requires viewing three-dimensional organs and tissues such as blood vessels in two-dimensional images within a limited field of view.

[0003] In laparoscopic surgery, abdominal surgery is performed using surgical tools such as forceps under a laparoscope. Trainees learn how to operate the surgical tools by watching an instructor demonstrate how to do so.

[0004] If the trainee makes an error in operating the surgical operating tool, the teacher will teach the trainee the correct operation.

[0005] When a teacher is not present, the trainee may learn how to operate the surgical tools by watching a video of the teacher operating the surgical tools.

[0006] Therefore, a surgical training device has been proposed to help trainees learn how to operate surgical tools in laparoscopic surgery (see, for example, Patent Document 1). Patent Document 1 proposes a surgical training device that allows trainees to learn how to operate forceps and the like to adapt to images that change in viewpoint as the laparoscope moves. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-148765 Summary of the Invention [Problem to be solved by the invention]

[0008] In laparoscopic surgery, it is important to learn the correct operation of surgical tools. For example, when it comes to operating surgical tools, such as moving them to the appropriate position, it is thought to be effective to provide feedback to the trainee about the extent of the difference between the teacher's operation and the trainee's operation.

[0009] Therefore, the objective of this specification is to provide a device and method for learning to operate an operating tool that, when there is a difference between the teacher's operation of the operating tool and the user's operation of the operating tool, makes the user aware of the difference between the two operations and applies a corrective force to the user's operation so that it approaches the teacher's operation of the operating tool. [Means for solving the problem]

[0010] According to one embodiment, there is provided a device for learning to operate a manipulator, the device comprising: a display unit; a haptic unit capable of generating a force on a user manipulator operated by a user; a position detection unit for detecting the position of the user manipulator; and a control unit for displaying, on the display unit, a plurality of teacher images representing a series of operations of the teacher manipulator by a teacher while updating them sequentially, and for displaying, on the display unit, the user manipulator images representing the user manipulators superimposed on the teacher images so as to represent the relationship between the positions of the user manipulators and the positions of the teacher manipulator associated with the teacher images displayed on the display unit, wherein the control unit stops updating of the teacher images displayed on the display unit when a distance between the position of the teacher manipulator associated with the teacher image displayed on the display unit and the position of the user manipulator is equal to or greater than a first reference value, and controls the haptic device to generate a force that moves the position of the user manipulator closer to the position of the teacher manipulator when the position of the teacher manipulator associated with the teacher image displayed on the display unit does not match the position of the user manipulator.

[0011] According to another embodiment, there is provided a method for learning to operate a manipulator, which is executed by a device for learning to operate a manipulator, the device comprising: a display unit; a haptic unit capable of generating a force on a user manipulator operated by a user; a position detection unit for detecting the position of the user manipulator; and a control unit for sequentially updating and displaying on the display unit a plurality of teacher images representing a series of operations of the teacher manipulator by a teacher, and for displaying on the display unit a user manipulator image representing the user manipulator superimposed on the teacher image so as to represent the relationship between the positions of the user manipulator and the positions of the teacher manipulator associated with the teacher image displayed on the display unit, wherein the control unit stops updating of the teacher image displayed on the display unit when a distance between the position of the teacher manipulator associated with the teacher image displayed on the display unit and the position of the user manipulator is equal to or greater than a first reference value, and controls the haptic device to generate a force that moves the position of the user manipulator closer to the position of the teacher manipulator when the position of the teacher manipulator associated with the teacher image displayed on the display unit does not match the position of the user manipulator. [Effects of the Invention]

[0012] According to the device for learning to operate an operating tool and the method for learning to operate an operating tool disclosed in the present specification, if there is a difference between the teacher's operation of the operating tool and the user's operation of the operating tool, the user can be made aware of the difference between the two operations, and a corrective force can be applied to the user's operation to bring it closer to the teacher's operation of the operating tool. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a schematic diagram of a surgical operation tool operation learning device according to an embodiment of the present invention; [Figure 2] FIG. 10 is a diagram illustrating an example of an image displayed on a display device. [Figure 3] FIG. 2 is a schematic diagram of a control device. [Figure 4] 10 is an operational flowchart of an operation tool learning process of the operation learning device. [Figure 5] FIG. 10 is a diagram illustrating a teacher image management table. [Figure 6] 10 is an operational flowchart of an update determination process of the operation learning device. [Figure 7] 10 is an operational flowchart of the correction force calculation process of the operation learning device. [Figure 8] FIG. 2 is a diagram illustrating a first distance and a second distance. [Figure 9] 10A and 10B are diagrams illustrating an example of the operation of the operation learning device. [Figure 10] FIG. 10 is a diagram illustrating an example of a teacher image management table. DETAILED DESCRIPTION OF THE INVENTION

[0014] A preferred embodiment of the device for learning to operate a surgical operation tool disclosed in this specification will be described below with reference to the drawings. Figure 1 is a schematic diagram of the device for learning to operate a surgical operation tool of this embodiment.

[0015] The operation learning device 1 has a display device 20, a force-sensing device 30, an input device 40, a control device 10, and simulated surgical operation tools 60 and 61. The force-sensing device 30 is controlled by the control device 10 to generate a corrective force on the simulated surgical operation tool 60 operated by a trainee (not shown) so that the operation approaches that of the teacher's operation tool. The trainee is an example of a user, and the simulated surgical operation tools 60 and 61 are examples of user operation tools.

[0016] The simulated surgical operation tools 60, 61 simulate surgical operation tools such as forceps used in laparoscopic surgery. The simulated surgical operation tools 60, 61 have a vertically elongated shape that extends in a predetermined direction. The tips of the simulated surgical operation tools 60, 61 are movable in three dimensions and are rotatable around the longitudinal direction as the axis of rotation. The simulated surgical operation tools 60, 61 are inserted through openings 501 provided in the housing 50, and their movement in two dimensions is restricted by the openings 501. Multiple openings 501 may be arranged in the housing 50, and the multiple openings 501 may be connected to each other by slits through which the simulated surgical operation tools 60, 61 can move.

[0017] Although FIG. 1 shows only the force sensor 30 that generates a corrective force on the simulated surgical manipulation tool 60, the operation learning device 1 also has a force sensor (not shown) that generates a corrective force on the simulated surgical manipulation tool 61.

[0018] The control device 10, parts of the simulated surgical operation tools 60, 61, and the force sensing device 30 are arranged within the housing 50, but in Figure 1, the control device 10, parts of the simulated surgical operation tools 60, 61, and the force sensing device 30 are clearly shown for ease of understanding.

[0019] The control device 10 sequentially updates and displays on the display device 20 a plurality of teacher images representing a series of operations of the surgical manipulation tools by the teacher. Furthermore, the control device 10 displays on the display device 20 a simulated surgical manipulation tool image representing the simulated surgical manipulation tool 60 superimposed on the teacher image based on the positions of the simulated surgical manipulation tools 60, 61 manipulated by the trainee.

[0020] 2 is a diagram showing an example of an image 200 displayed on the display device 20. The image 200 includes a teacher image showing the operation of surgical manipulation tools 201 and 202 by a teacher. In the image 200, a simulated surgical manipulation tool image 203 showing the simulated surgical manipulation tool 60 and a simulated surgical manipulation tool image 204 showing the simulated surgical manipulation tool 61 are superimposed on the teacher image to show the relationship between the positions of the simulated surgical manipulation tools 60 and 61 and the positions of the surgical manipulation tools 201 and 202 associated with the teacher image displayed on the display device 20.

[0021] The control device 10 stops updating the teacher image displayed on the display device 20 when the distance between the position of the surgical manipulation tool associated with the teacher image and the position of the simulated surgical manipulation tools 60, 61 is equal to or greater than a first reference value.

[0022] As a result, if there is a difference between the teacher's operation of the surgical operation tools and the operation of the simulated surgical operation tools 60, 61, the updating of the teacher image is stopped, so the trainee can recognize that there is a difference between the two operations.

[0023] In addition, when the position of the surgical operating tool associated with the teacher image displayed on the display device 20 does not match the position of the simulated surgical operating tools 60, 61, the control device 10 controls the force sensing device 30 to generate a force that moves the positions of the simulated surgical operating tools 60, 61 closer to the position of the surgical operating tool associated with the teacher image.

[0024] This allows a corrective force to be applied to the trainee's operation to bring it closer to the teacher's operation of the surgical tool, and feedback can be given to the trainee as to how much difference there is between the teacher's operation and the trainee's operation.

[0025] The operation learning device 1 will be further explained below.

[0026] The display device 20 displays a teacher image and the like in the operation learning process for the surgical operation tool. The display device 20 has, for example, a liquid crystal display or a touch panel.

[0027] The input device 40 receives input of a trainee's operation and generates an operation signal. The input device 40 includes, for example, a keyboard, a mouse, or a touch panel.

[0028] The haptic device 30 is controlled by the control device 10 and is capable of generating a force on a simulated surgery operating tool 60 operated by a trainee. The haptic device 30 has a first motor 31, a second motor 32, a third motor 33, a fourth motor 34, and a parallel link mechanism 35.

[0029] The first motor 31, second motor 32, and third motor 33 each have an angle detection unit that detects the three link angles in the parallel link mechanism 35. The relationship between these three link angles is converted into a three-degree-of-freedom translational position in the x, y, and z axes of the end (tip) connected to the simulated surgical manipulation tool 60. The force applied to the simulated surgical manipulation tool 60 is converted into a torque command value for each motor by performing the reverse conversion.

[0030] One longitudinal end (tip) of the simulated surgical operation tool 60 is connected to the fourth motor 34 via a universal joint. The fourth motor 34 generates a force that rotates the simulated surgical operation tool 60 around its longitudinal axis. The fourth motor 34 also detects the amount of rotation of the simulated surgical operation tool 60 and functions as a rotation amount detection unit that detects the amount of rotation of the simulated surgical operation tool 60.

[0031] The force sensing device 30 is capable of moving the position of the fourth motor 34 in the x-axis, y-axis and z-axis directions using a parallel link mechanism 35, and is also capable of detecting the position of the fourth motor 34 (the position of the longitudinal tip of the simulated surgical operating tool 60).

[0032] The above description of the force sense device 30 is appropriately applied to a force sense device (not shown) that generates a corrective force on the simulated surgical operation tool 61.

[0033] The control device 10 includes a communication interface (IF) 11, a memory 12, and a processor 13. The communication IF 11, the memory 12, and the processor 13 are connected via a communication line 14.

[0034] The communication IF 11 has an interface circuit for connecting the control device 10 to the display device 20, the force sense device 30, the input device 40, and the like.

[0035] The memory 12 is an example of a storage unit and includes, for example, a volatile semiconductor memory and a non-volatile semiconductor memory. The memory 12 stores a computer program for the processor 13 to execute the operation learning process and data generated in the operation learning process. The memory 12 also stores a plurality of teacher images representing the operation of the teacher tool by the teacher.

[0036] The processor 13 includes one or more central processing units (CPUs) and their peripheral circuits, and may further include other arithmetic circuits such as a logic unit, a numerical calculation unit, or a graphics processing unit.

[0037] All or part of the functions of the control device 10 are functional modules realized by, for example, a computer program running on the processor 13. The processor 13 has an image control unit 131 and a force-sense control unit 132. Alternatively, the functional modules of the processor 13 may be dedicated arithmetic circuits provided in the processor 13.

[0038] 4 is an operational flowchart of the operation tool learning process of the operation learning device 1. First, the image control unit 131 reads a first teacher image from the memory 12 and causes the display device 20 to display the first teacher image. Then, after a predetermined time has elapsed, the image control unit 131 reads a second teacher image from the memory 12 and causes the display device 20 to display the second teacher image (step S101).

[0039] Next, the image control unit 131 refers to the teacher image management table from the memory 12 and obtains the position and rotation amount of the surgical operating tool associated with the second teacher image currently displayed on the display device 20 (step S102).

[0040] 5 is a diagram illustrating a teacher image management table. In the example shown in FIG. 5, teacher image management table 100 has an image number column 101 in which an image number identifying a teacher image is registered, a position determination flag column 102 in which a position determination flag indicating whether or not to determine the position of a simulated surgical manipulation tool is registered, a surgical manipulation tool position column 103 in which the position of the surgical manipulation tool is registered, a rotation amount determination flag column 104 in which a rotation amount determination flag indicating whether or not to determine the rotation amount of a simulated surgical manipulation tool is registered, and a surgical manipulation tool rotation amount column 105 in which the rotation amount of the surgical manipulation tool is registered. The position determination flag, the position of the surgical manipulation tool, the rotation amount determination flag, and the rotation amount of the surgical manipulation tool are associated with the image number and registered in teacher image management table 100.

[0041] In the position determination flag column 102, "1 (valid)" is registered when the position of the simulated surgical manipulation tool is determined, and "0 (invalid)" is registered when the position of the simulated surgical manipulation tool is not determined.

[0042] The position of the surgical manipulation tool is expressed in a first coordinate system having a predetermined position as its origin, for example, a portion that represents the surgical manipulation tool, such as the position of the tip of the surgical manipulation tool in the longitudinal direction.

[0043] In the rotation amount determination flag column 104, "1 (valid)" is registered when the rotation amount of the simulated surgical operation tool is determined, and "0 (invalid)" is registered when the rotation amount of the simulated surgical operation tool is not determined.

[0044] The rotation amount of the surgical operating tool is expressed as an angle with a predetermined rotation position as the origin.

[0045] When one teaching image includes multiple surgical manipulation tools, the position determination flag, the position of the surgical manipulation tool, the rotation amount determination flag, and the rotation amount of the surgical manipulation tool are registered in the teaching image management table for each of the multiple surgical manipulation tools. In the example shown in Figure 5, for ease of explanation, a teaching image management table 100 is shown for the case where one teaching image includes one surgical manipulation tool.

[0046] Next, the image control unit 131 acquires the position of the simulated surgical manipulation tool and the amount of rotation of the simulated surgical manipulation tool from the force feedback device 30 (step S103). The position of the simulated surgical manipulation tool represents the position of the longitudinal tip of the simulated surgical manipulation tool (the position of the fourth motor 34) and is expressed in a second coordinate system having an origin at a predetermined position of the force feedback device 30. The image control unit 131 converts the position of the simulated surgical manipulation tool from the second coordinate system to a position represented in the first coordinate system. Such conversion can be expressed as a combination of a translation matrix representing translational movement and a rotation matrix representing rotation. The amount of rotation of the simulated surgical manipulation tool is expressed as an angle, with the rotation position corresponding to the surgical manipulation tool as the origin.

[0047] Next, the image control unit 131 performs a process of determining whether or not to update the teacher image (step S104). Details of the process of determining whether or not to update the teacher image will be described later.

[0048] Next, the force-sense control unit 132 calculates the corrective force generated in the simulated surgery manipulation tool (step S105). The corrective force calculation process will be described in detail later.

[0049] Next, the image control unit 131 displays the teacher image and the simulated surgical manipulation tool image on the display device 20 (step S106). The image control unit 131 superimposes the simulated surgical manipulation tool image representing the simulated surgical manipulation tool on the teacher image and displays it on the display device 20 (see FIG. 2) so as to represent the relationship between the position of the simulated surgical manipulation tool and the position of the surgical manipulation tool associated with the teacher image displayed on the display device 20. The simulated surgical manipulation tool image is a vertically elongated image that imitates the outline of an actual manipulation tool, connecting the position of the longitudinal tip of the simulated surgical manipulation tool with the position representing the center of the opening 501 provided in the housing 50 through which the simulated surgical manipulation tool is inserted, and is superimposed on the teacher image and displayed on the display device 20.

[0050] Next, the force-sense control unit 132 controls the force-sense device 30 to generate a corrective force (step S107). Note that if the position of the simulated surgical manipulation tool coincides with the position of the surgical manipulation tool associated with the teacher image, no corrective force is generated.

[0051] Next, the image control unit 131 refers to the teacher image management table 100 stored in the memory 12 and determines whether or not there is a teacher image to be displayed next to the teacher image currently displayed on the display device 20 (step S108).

[0052] If there is a teacher image to be displayed next (step S108-Yes), the process returns to step S102. On the other hand, if there is no teacher image to be displayed next (step S108-No), the process ends.

[0053] Next, the update determination process for the teacher image will be described below with reference to Fig. 6. Fig. 6 is an operational flowchart of the update determination process of the operation learning device 1.

[0054] First, the image control unit 131 refers to the position determination flag column 102 of the teacher image management table 100 to determine whether the position determination flag associated with the image number of the teacher image currently displayed on the display device 20 is valid (step S201).

[0055] If the position determination flag is enabled (step S201-Yes), the image control unit 131 obtains a first distance between the position of the surgical manipulation tool associated with the teacher image and the position of the simulated surgical manipulation tool (step S202).

[0056] 8 is a diagram illustrating the first distance. The image control unit 131 refers to the surgical operation tool position column 103 of the teacher image management table 100 to acquire the position Pi of the surgical operation tool associated with the teacher image fi currently displayed on the display device 20 and the position Pi-1 of the surgical operation tool associated with the teacher image fi-1 displayed before the teacher image fi.

[0057] The image control unit 131 calculates the first distance as the distance L1 between the line segment Si connecting the position Pi of the surgical operating tool associated with the teacher image fi currently displayed on the display device 20 and the position Pi-1 of the surgical operating tool associated with the teacher image fi-1 displayed before the teacher image fi, and the position Ti of the simulated surgical operating tool.

[0058] Next, the image control unit 131 calculates a second distance between the position Pi+1 of the surgical manipulation tool associated with the teacher image fi+1 to be displayed after the teacher image fi currently displayed on the display device 20 and the position Ti of the simulated surgical manipulation tool (step S203). In this specification, the second distance represents the distance between the position of the surgical manipulation tool associated with the teacher image to be displayed after the teacher image displayed on the display device 20 and the position of the simulated surgical manipulation tool.

[0059] The image control unit 131 refers to the surgical operating tool position column 103 of the teacher image management table 100 to obtain the position Pi of the surgical operating tool associated with the teacher image fi currently displayed on the display device 20 and the position Pi+1 of the surgical operating tool associated with the teacher image fi+1 to be displayed next to the teacher image fi.

[0060] As shown in Figure 8, the image control unit 131 calculates the second distance as the distance L2 between the line segment Si+1 connecting the position Pi of the surgical operating tool associated with the teacher image fi currently displayed on the display device 20 and the position Pi+1 of the surgical operating tool associated with the teacher image fi+1 displayed next to the teacher image fi, and the position Ti of the simulated surgical operating tool.

[0061] Next, the image control unit 131 determines whether the second distance L2 is longer than the first distance L1 (step S204). If the second distance L2 is longer than the first distance L1 (step S204-Yes), it determines to stop updating the teacher image (step S211), and ends the series of processes.

[0062] On the other hand, if the second distance L2 is not longer than the first distance L1 (step S204-No), the image control unit 131 determines whether the second distance L2 representing the distance between the position of the surgical operating tool associated with the teacher image and the position of the simulated surgical operating tool is greater than or equal to the first reference value Lth (step S205).

[0063] If the second distance L2 is equal to or greater than the first reference value Lth (step S205-Yes), it is determined that the updating of the teacher image is to be stopped (step S211), and the series of processes is ended.

[0064] On the other hand, if the second distance L2 is not greater than the first reference value Lth (step S205-No), or if the position determination flag is invalid (step S201-No), the image control unit 131 refers to the rotation amount determination flag column 104 of the teacher image management table 100 and determines whether the rotation amount determination flag associated with the image number of the teacher image currently displayed on the display device 20 is valid (step S206).

[0065] If the rotation amount determination flag is enabled (step S206-Yes), the image control unit 131 obtains a first angle that is the difference between the rotation amount of the surgical manipulation tool associated with the teacher image and the rotation amount of the simulated surgical manipulation tool (step S207).

[0066] The image control unit 131 refers to the rotation amount column 105 of the surgical operation tool in the teacher image management table 100 and acquires the rotation amount Ri of the surgical operation tool associated with the teacher image fi currently displayed on the display device 20.

[0067] The image control unit 131 calculates a first angle M1 that is the difference between the rotation amount Ri of the surgical manipulation tool associated with the teacher image fi currently displayed on the display device 20 and the rotation amount ∪i of the simulated surgical manipulation tool.

[0068] Next, the image control unit 131 calculates a second angle, which is the difference between the rotation amount Ri+1 of the surgical manipulation tool associated with the teacher image fi+1 to be displayed next to the teacher image fi currently displayed on the display device 20, and the rotation amount Ui of the simulated surgical manipulation tool (step S208).

[0069] The image control unit 131 refers to the rotation amount column 105 of the surgical operating tool in the teacher image management table 100 and obtains the rotation amount Ri+1 of the surgical operating tool associated with the teacher image fi+1 to be displayed next to the teacher image fi currently displayed on the display device 20.

[0070] The image control unit 131 calculates a second angle M2, which is the difference between the rotation amount Ri+1 of the surgical operating tool associated with the teacher image fi+1 to be displayed next to the teacher image fi currently displayed on the display device 20, and the rotation amount ∪i of the simulated surgical operating tool.

[0071] Next, the image control unit 131 determines whether the second angle M2 is greater than the first angle M1 (step S209). If the second angle M2 is greater than the first angle M1 (step S209-Yes), it determines to stop updating the teacher image (step S211), and ends the series of processes.

[0072] On the other hand, if the second angle M2 is not greater than the first angle M1 (step S2094-No), the image control unit 131 determines whether the second angle M2, which represents the relationship between the rotation amount of the surgical operating tool associated with the teacher image and the rotation amount of the simulated surgical operating tool, is greater than or equal to the second reference value Rth (step S210).

[0073] If the second angle M2 is equal to or greater than the second reference value Rth (step S210-Yes), it is determined that the updating of the teacher image is to be stopped (step S211), and the series of processes is ended.

[0074] On the other hand, if the second angle M2 is not equal to or greater than the second reference value Rth (step S210-No), or if the rotation amount determination flag is invalid (step S206-No), it is determined that the teacher image should be updated (step S212), and the series of processes ends. This concludes the explanation of the teacher image update determination process.

[0075] Next, the corrective force calculation process will be described below with reference to Fig. 7. Fig. 7 is an operational flowchart of the corrective force calculation process of the operation learning device 1.

[0076] First, the force-sense control unit 132 refers to the position determination flag column 102 of the teacher image management table 100 to determine whether the position determination flag associated with the image number of the teacher image currently displayed on the display device 20 is valid (step S301).

[0077] If the position determination flag is enabled (step S301-Yes), the force-sense control unit 132 calculates a position correction force based on the position of the surgical manipulation tool associated with the teacher image displayed on the display device 20 and the position of the simulated surgical manipulation tool (step S302). If it is determined that the teacher image will be updated, the position correction force is calculated based on the position of the surgical manipulation tool associated with the teacher image displayed next to the currently displayed teacher image and the position of the simulated surgical manipulation tool. On the other hand, if it is determined that the teacher image will not be updated, the position correction force is calculated based on the position of the surgical manipulation tool associated with the currently displayed teacher image and the position of the simulated surgical manipulation tool.

[0078] In the example shown in FIG. 8, when it is determined that the teacher image is to be updated, the position correction force is calculated as the product of the second distance L2 and a predetermined coefficient k1. The direction of the position correction force is the direction from the position Pi of the surgical operation tool toward the line segment Si+1. On the other hand, when it is determined that the teacher image is not to be updated, the position correction force is calculated as the product of the first distance L1 and a predetermined coefficient k1. The direction of the position correction force is the direction from the position Pi of the surgical operation tool toward the line segment Si. The position correction force increases as the first distance L1 or the second distance L2 increases.

[0079] On the other hand, if the position determination flag is invalid (step S301-No), the force-sense control unit 132 sets the position correcting force to zero (step S303).

[0080] Next, the force-sense control unit 132 refers to the rotation amount determination flag column 104 of the teacher image management table 100 and determines whether the rotation amount determination flag associated with the image number of the teacher image currently displayed on the display device 20 is valid (step S304).

[0081] If the rotation amount determination flag is enabled (step S304-Yes), the force-sense control unit 132 calculates the rotation amount correction force based on the rotation amount of the surgical manipulation tool associated with the teacher image displayed on the display device 20 and the rotation amount of the simulated surgical manipulation tool (step S305), and ends the series of processes. If it is determined that the teacher image will be updated, the position correction force is calculated based on the rotation amount of the surgical manipulation tool associated with the teacher image displayed next to the currently displayed teacher image and the rotation amount of the simulated surgical manipulation tool. On the other hand, if it is determined that the teacher image will not be updated, the position correction force is calculated based on the rotation amount of the surgical manipulation tool associated with the currently displayed teacher image and the rotation amount of the simulated surgical manipulation tool.

[0082] If it is determined that the teacher image will be updated, the position correction force is calculated as the product of the second angle M2 and a predetermined coefficient k2. The direction of the rotation correction force is the direction in which the rotation position of the simulated surgical manipulation tool is directed toward the rotation position of the surgical manipulation tool associated with the teacher image fi+1 that will be displayed after the currently displayed teacher image fi. On the other hand, if it is determined that the teacher image will not be updated, the rotation correction force is calculated as the product of the first angle M1 and a predetermined coefficient k2. The direction of the rotation correction force is the direction in which the rotation position of the simulated surgical manipulation tool is directed toward the rotation position of the surgical manipulation tool associated with the currently displayed teacher image fi. The position correction force increases as the first angle M1 or the second angle M2 increases.

[0083] On the other hand, if the position determination flag is invalid (step S301-No), the force-sense control unit 132 sets the position corrective force to zero (step S303) and ends the series of processes. This concludes the description of the corrective force calculation process.

[0084] Next, an example of the operation of the teacher image update determination process and the correction force calculation process in the operation learning device 1 for a surgical manipulation tool of this embodiment will be described below with reference to Fig. 9. Fig. 9 is a diagram illustrating an example of the operation of the operation learning device 1. In the operation example shown in Fig. 9, the position determination flag is enabled, but the rotation amount determination flag is disabled. In addition, one surgical manipulation tool is included in the teacher image, and one simulated surgical manipulation tool of the operation learning device 1 is operated by the trainee.

[0085] (Example 1) The operation learning device 1 first displays the teacher image f1, then updates the image to display the teacher image f2. The operation learning device 1 acquires the position T1 of the simulated surgery tool.

[0086] The operation learning device 1 calculates a first distance L1 between a line segment S2 connecting the position P2 of the surgical operation tool associated with the teacher image f2 currently displayed on the display device 20 and the position P1 of the surgical operation tool associated with the teacher image f1 displayed before the teacher image f2, and the position T1 of the simulated surgical operation tool. The operation learning device 1 also calculates a second distance L2 between the line segment S3 connecting the position P2 of the surgical operation tool associated with the teacher image f2 currently displayed on the display device 20 and the position P3 of the surgical operation tool associated with the teacher image f3 displayed after the teacher image f2, and the position T1 of the simulated surgical operation tool.

[0087] The line segment S2 is associated with the teacher image f2, and the line segment S3 is associated with the teacher image f3.

[0088] Since the second distance L2 is longer than the first distance L1, the operation learning device 1 determines not to update the teacher image f2.

[0089] The operation learning device 1 calculates a position correction force ∨1 based on the position P2 of the surgical manipulation tool associated with the teacher image f2 displayed on the display device 20 and the position of the simulated surgical manipulation tool T1. The position correction force ∨1 is directed from the position T1 of the simulated surgical manipulation tool toward the line segment S2. Note that although the distance between the position T1 of the simulated surgical manipulation tool and the line segment S1 is shorter than the distance between the position T1 of the simulated surgical manipulation tool and the line segment S2, the position correction force ∨1 is directed toward the position P1 that indicates the minimum distance from the position T1 of the simulated surgical manipulation tool to the line segment S2 associated with the teacher image f2 displayed on the display device 20.

[0090] (Example 2) The operation learning device 1 displays the teacher image f3 and acquires the position T2 of the simulated surgical operation tool.

[0091] The operation learning device 1 calculates a first distance L1 between a line segment S3 connecting the position P3 of the surgical operation tool associated with the teacher image f3 currently displayed on the display device 20 and the position P2 of the surgical operation tool associated with the teacher image f2 displayed before the teacher image f3, and the position T2 of the simulated surgical operation tool. The operation learning device 1 also calculates a second distance L2 between the line segment S4 connecting the position P3 of the surgical operation tool associated with the teacher image f3 currently displayed on the display device 20 and the position P4 of the surgical operation tool associated with the teacher image f4 displayed after the teacher image f3, and the position T2 of the simulated surgical operation tool.

[0092] Since the second distance L2 is not longer than the first distance L1 and the second distance L2 is not equal to or greater than the first reference value Lth, the operation learning device 1 determines to update the teacher image from image f3 to image f4.

[0093] The operation learning device 1 calculates the position correction force ∨2 based on the position P4 of the surgical manipulation tool associated with the teacher image f4 displayed on the display device 20 and the position of the simulated surgical manipulation tool T2. The position correction force ∨2 is directed from the position T2 of the simulated surgical manipulation tool toward the line segment S4.

[0094] (Example 3) The operation learning device 1 displays the teacher image f4 and acquires the position T3 of the simulated surgical operation tool.

[0095] The operation learning device 1 calculates a first distance L1 between a line segment S4 connecting the position P4 of the surgical operation tool associated with the teacher image f4 currently displayed on the display device 20 and the position P3 of the surgical operation tool associated with the teacher image f3 displayed before the teacher image f4, and the position T3 of the simulated surgical operation tool. The operation learning device 1 also calculates a second distance L2 between the line segment S5 connecting the position P4 of the surgical operation tool associated with the teacher image f4 currently displayed on the display device 20 and the position P5 of the surgical operation tool associated with the teacher image f5 displayed after the teacher image f4, and the position T3 of the simulated surgical operation tool.

[0096] Since the second distance L2 is not longer than the first distance L1 and the second distance L2 is not equal to or greater than the first reference value Lth, the operation learning device 1 determines to update the teacher image from image f4 to image f5.

[0097] The operation learning device 1 calculates a position correction force ∨3 based on the position P5 of the surgical manipulation tool associated with the teacher image f5 displayed on the display device 20 and the position of the simulated surgical manipulation tool T3. The position correction force ∨3 is directed from the position T3 of the simulated surgical manipulation tool toward the line segment S5. Note that although the distance between the position T3 of the simulated surgical manipulation tool and the line segment S6 is shorter than the distance between the position T3 of the simulated surgical manipulation tool and the line segment S5, the position correction force ∨3 is directed toward the position P5 that indicates the minimum distance from the position T3 of the simulated surgical manipulation tool to the line segment S5 associated with the teacher image f5 displayed on the display device 20.

[0098] (Example 4) The operation learning device 1 displays the teacher image f5 and acquires the position T4 of the simulated surgical operation tool.

[0099] The operation learning device 1 calculates a first distance L1 between a line segment S5 connecting the position P5 of the surgical operation tool associated with the teacher image f5 currently displayed on the display device 20 and the position P4 of the surgical operation tool associated with the teacher image f4 displayed before the teacher image f5, and the position T4 of the simulated surgical operation tool. The operation learning device 1 also calculates a second distance L2 between the line segment S6 connecting the position P5 of the surgical operation tool associated with the teacher image f5 currently displayed on the display device 20 and the position P6 of the surgical operation tool associated with the teacher image f6 displayed after the teacher image f5, and the position T4 of the simulated surgical operation tool.

[0100] Since the second distance L2 is not longer than the first distance L1 and the second distance L2 is not equal to or greater than the first reference value Lth, the operation learning device 1 determines to update the teacher image from image f5 to image f6.

[0101] The operation learning device 1 calculates the position correction force ∨4 based on the position P6 of the surgical manipulation tool associated with the teacher image f6 displayed on the display device 20 and the position of the simulated surgical manipulation tool T4. The position correction force ∨4 is directed from the position T4 of the simulated surgical manipulation tool toward the line segment S6.

[0102] (Example 5) The operation learning device 1 displays the teacher image f6 and acquires the position T5 of the simulated surgical operation tool.

[0103] The operation learning device 1 calculates a first distance L1 between a line segment S6 connecting the position P6 of the surgical operation tool associated with the teacher image f6 currently displayed on the display device 20 and the position P5 of the surgical operation tool associated with the teacher image f5 displayed before the teacher image f6, and the position T5 of the simulated surgical operation tool. The operation learning device 1 also calculates a second distance L2 between the line segment S7 connecting the position P6 of the surgical operation tool associated with the teacher image f6 currently displayed on the display device 20 and the position P7 of the surgical operation tool associated with the teacher image f7 displayed after the teacher image f6, and the position T5 of the simulated surgical operation tool.

[0104] Although the second distance L2 is not longer than the first distance L1, the second distance L2 is equal to or greater than the first reference value Lth, and therefore the operation learning device 1 determines not to update the teacher image f6.

[0105] The operation learning device 1 calculates a position correction force ∨5 based on the position P6 of the surgical manipulation tool associated with the teacher image f6 displayed on the display device 20 and the position of the simulated surgical manipulation tool T4. The position correction force ∨5 is directed toward the position P6 that indicates the minimum distance between the position T5 of the simulated surgical manipulation tool and the line segment S6.

[0106] The surgical manipulation training device 1 of this embodiment can be set to determine whether or not to determine the position of the trainee's simulated surgical manipulation tool and whether or not to determine the amount of rotation of the trainee's simulated surgical manipulation tool for each teacher image.

[0107] Specifically, both the position of the trainee's simulated surgical manipulation tool and the amount of rotation of the simulated surgical manipulation tool may be determined for each teacher image, or only the position of the trainee's simulated surgical manipulation tool may be determined for each teacher image, or only the amount of rotation of the trainee's simulated surgical manipulation tool may be determined for each teacher image.Furthermore, it is also possible not to determine both the position of the trainee's simulated surgical manipulation tool and the amount of rotation of the simulated surgical manipulation tool for each teacher image.

[0108] 10 is a diagram illustrating an example of a teacher image management table. In section R1, neither the position of the trainee's simulated surgical manipulation tool nor the amount of rotation of the simulated surgical manipulation tool is determined. For example, in laparoscopic surgery, for images other than the teacher image of the operation of forming a loop of thread and wrapping it around the target site, neither the position of the trainee's simulated surgical manipulation tool nor the amount of rotation of the simulated surgical manipulation tool may be determined.

[0109] In section R2, only the position of the trainee's simulated surgical manipulation tool is determined. If the trainee's position of the simulated surgical manipulation tool does not match the position of the surgical manipulation tool determined by the teacher, the update of the teacher image stops and a position correction force is generated.

[0110] In section R3, only the rotation amount of the simulated surgical tool by the trainee is judged. If the rotation amount of the simulated surgical tool by the trainee does not match the rotation amount of the surgical tool by the teacher, the update of the teacher image stops and a rotation amount correction force is generated.

[0111] In section R4, both the position and rotation of the simulated surgical manipulation tool by the trainee are determined. If the trainee's manipulation of the simulated surgical manipulation tool does not match the manipulation of the surgical manipulation tool by the teacher, the update of the teacher image stops and a corrective force is applied to the manipulation.

[0112] This allows the trainee to effectively learn only the operations necessary for training when learning to use the simulated surgical operation tool for each teaching image.

[0113] According to the surgical operating tool operation learning device of this embodiment described above, if there is a difference between the teacher's operation of the operating tool and the user's operation of the operating tool, the user can be made aware of the difference between the two operations, and a corrective force can be applied to the user's operation to bring it closer to the teacher's operation of the operating tool.

[0114] In the present invention, the above-described embodiments of the device for learning to operate a manipulating tool and the method for learning to operate a manipulating tool can be modified as appropriate without departing from the spirit of the present invention. The technical scope of the present invention is not limited to these embodiments, but includes the inventions described in the claims and their equivalents.

[0115] For example, in the above-described embodiment, the operating tool operation learning device is used to learn the operation of surgical operating tools used in laparoscopic surgery, but it can also be used to learn the operation of other operating tools, such as operating tools used to operate vehicles or aircraft, or operating tools such as rackets used in sports.

[0116] Furthermore, in the above-described embodiment, if the trainee's operation of the simulated surgical operating tool does not match the teacher's operation of the surgical operating tool, a corrective force is applied to the operation, but the trainee may be notified that the operation does not match the teacher's operation of the surgical operating tool by using a display on a display device or a sound from an audio device along with the corrective force. [Explanation of symbols]

[0117] 1. Device for learning to operate operating tools 10 Control device 11 Communication Interface 12 Memory 13 processors 131 Image control unit 132 Force control unit 14 Communication lines 20 Display device 30 Force sensing device 31 First motor 32 Second motor 33 Third motor 34 4th motor 35 Parallel link mechanism 40 Input Devices 50 cabinets 60, 61 Simulated surgical operation tool

Claims

1. A display unit; a force sensor capable of generating a force on a user operation tool operated by a user; a position detection unit that detects the position of the user manipulation tool; a control unit that displays on the display unit, while sequentially updating, a plurality of teacher images representing a series of operations of a teacher operating tool by a teacher, and that displays on the display unit, a user operating tool image representing the user operating tool superimposed on the teacher image so as to represent the relationship between the position of the user operating tool and the position of the teacher operating tool associated with the teacher image displayed on the display unit; and The control unit When the distance between the position of the teacher operating tool associated with the teacher image displayed on the display unit and the position of the user operating tool is equal to or greater than a first reference value, stopping updating of the teacher image that is sequentially updated and displayed on the display unit, and A device for learning how to operate a manipulating tool, which controls the force sensing unit to generate a force that moves the position of the user's manipulating tool closer to the position of the teacher's manipulating tool when the position of the teacher's manipulating tool associated with the teacher image displayed on the display unit does not match the position of the user's manipulating tool.

2. Each of the plurality of teacher images is a first flag indicating whether or not a distance between the position of the teacher operating tool associated with the teacher image and the position of the user operating tool is equal to or greater than the first reference value; The control unit 2. The device for learning to operate a manipulating tool according to claim 1, wherein, when the first flag indicates that a determination is to be made, it is determined whether a distance between a position of the teacher manipulating tool associated with the teacher image displayed on the display unit and a position of the user manipulating tool is equal to or greater than the first reference value.

3. The control unit a first distance is calculated as a distance between a first line segment connecting a position of the teacher operating tool associated with a first teacher image displayed on the display unit and a position of the teacher operating tool associated with a second teacher image displayed before the first teacher image, and the position of the user operating tool; a second distance representing the distance between the position of the teacher operating tool associated with the first teacher image displayed on the display unit and the position of the teacher operating tool associated with the third teacher image displayed next to the first teacher image, and the position of the user operating tool; the second distance is greater than the first distance; Or, The operation learning device for a manipulation tool according to claim 1 , wherein updating of the teacher image is stopped when the second distance is equal to or greater than the first reference value.

4. a rotation amount detection unit that detects a rotation amount of the user operation tool; The control unit When a difference between the rotation amount of the teacher tool associated with the teacher image displayed next to the teacher image displayed on the display unit and the rotation amount of the user tool is equal to or greater than a second reference value, stopping updating of the teacher image displayed on the display unit, and 4. The device for learning to operate a manipulating tool according to claim 1, wherein, when the amount of rotation of the teacher manipulating tool associated with the teacher image displayed on the display unit does not match the amount of rotation of the user manipulating tool, the force sensor is controlled to generate a force that brings the amount of rotation of the user manipulating tool closer to the amount of rotation of the teacher manipulating tool.

5. Each of the plurality of teacher images is a second flag indicating whether or not a difference between the rotation amount of the teacher tool associated with the teacher image displayed next to the teacher image displayed on the display unit and the rotation amount of the user tool is equal to or greater than the second reference value; The control unit 5. The device for learning to operate a manipulating tool according to claim 4, wherein, when the second flag indicates that a determination is to be made, it is determined whether a difference between an amount of rotation of the teacher manipulating tool associated with the teacher image displayed next to the teacher image displayed on the display unit and an amount of rotation of the user manipulating tool is equal to or greater than a second reference value.

6. The control unit a second angle representing a difference between the amount of rotation of the teacher operating tool associated with the teacher image displayed next to the teacher image displayed on the display unit and the amount of rotation of the user operating tool is larger than a first angle representing a difference between the amount of rotation of the teacher operating tool associated with the teacher image displayed on the display unit and the amount of rotation of the user operating tool, or The operation learning device for a manipulation tool according to claim 4 , wherein updating of the teacher image is stopped when the second angle is equal to or greater than the second reference value.

7. A display unit; a force sensor capable of generating a force on a user operation tool operated by a user; a position detection unit that detects the position of the user manipulation tool; a control unit that displays on the display unit, while sequentially updating, a plurality of teacher images representing a series of operations of a teacher operating tool by a teacher, and that displays on the display unit, a user operating tool image representing the user operating tool superimposed on the teacher image so as to represent the relationship between the position of the user operating tool and the position of the teacher operating tool associated with the teacher image displayed on the display unit; A method for learning to operate an operating tool, which is executed by an operating learning device for an operating tool having the following features: The control unit When the distance between the position of the teacher operating tool associated with the teacher image displayed on the display unit and the position of the user operating tool is equal to or greater than a first reference value, stopping updating of the teacher image that is being sequentially updated and displayed on the display unit, and When the position of the teacher operating tool associated with the teacher image displayed on the display unit does not match the position of the user operating tool, the force sensor is controlled to generate a force that moves the position of the user operating tool closer to the position of the teacher operating tool. A method for learning to operate an operating tool.

Images