Image analysis device, image analysis method, image analysis program, and image analysis system

Abstract

An image analysis device, an image analysis method, an image analysis program, and an image analysis system are provided that enable quantitative evaluation of surgical skills. [Solution] The image analysis device disclosed herein includes an input unit that accepts video data of a surgeon's procedure as input. The image analysis device also includes an extraction unit that extracts, from the results of detecting the surgeon's hand movements included in the video data, at least one of the following as evaluation data: the trajectory of the hand, the total distance traveled by the hand, the number of hand movements, or the procedure time. The image analysis device also includes an evaluation unit that generates an evaluation result of the surgical skill or surgical tool based on the evaluation data.

Description

[Technical Field]

[0001] An embodiment of the present invention relates to an image analysis device, an image analysis method, an image analysis program, and an image analysis system. [Background technology]

[0002] In recent years, surgical techniques have become increasingly sophisticated thanks to advances in engineering technology, creating a demand for training physicians to improve their skills. However, traditional surgical training and education have often relied on the subjective judgment of experienced physicians, while less experienced physicians have learned intuitively, such as by "watching and learning" or "learning through experience." Therefore, analytical techniques that can quantitatively demonstrate skillful and efficient techniques have long been desired. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Takahiro Igaki et al., “Automatic Surgical Skill Assessment System Based on Concordance of Standardized Surgical Field Development Using Artificial Intelligence,” JAMA Surgery, June 7, 2023. [Non-patent document 2] Hassan Ismail Fawaz et al., “Evaluating Surgical Skills from Kinematic Data Using Convolutional Neural Networks,” Medical Image Computing and Computer Assisted Intervention, September 18, 2018.

[0004] There is a technology that evaluates surgical skills through image analysis. In the above-mentioned technology, AI (Artificial Intelligence) learns surgical videos by experienced surgeons and scores them by having the AI ​​recognize each surgical step. In addition, the above-mentioned technology proposes a deep learning model that evaluates the surgeon's level on a three-point scale based on the trajectory of the surgical robot's arm.

[0005] These techniques are scores output based on AI learning, and the evaluation method is a black box, so it is difficult to say that they explicitly represent the skillful techniques that experienced physicians consider to be effective.

[0006] In addition, skill evaluation is also being conducted by automatically acquiring the position information of surgical instruments and analyzing the surgical work process. However, detecting the position of surgical instruments can be cumbersome, requiring the introduction of a position measurement system or the attachment of markers to the instruments. Summary of the Invention [Problem to be solved by the invention]

[0007] Therefore, embodiments of the present invention provide an image analysis device, an image analysis method, an image analysis program, and an image analysis system that are capable of quantitatively evaluating surgical skills. [Means for solving the problem]

[0008] The image analysis device disclosed herein includes an input unit that receives video data of a surgeon's procedure as input. The image analysis device also includes an extraction unit that extracts, from the results of detecting the surgeon's hand movements included in the video data, at least one of the following as evaluation data: a trajectory of the hand, a total distance traveled by the hand, the number of hand movements, or a procedure time. The image analysis device also includes an evaluation unit that generates an evaluation result of a procedural skill or a surgical tool based on the evaluation data. [Brief explanation of the drawings]

[0009] [Figure 1]1 is a schematic configuration diagram of an image analysis system 1 according to a first embodiment. [Figure 2] FIG. 2 is another schematic configuration diagram of the image analysis system 1 in the first embodiment. [Figure 3] 1 is an example of a block diagram of an image analysis device 10 according to a first embodiment. [Figure 4] 3 is another example of a block diagram of the image analyzing device 10 according to the first embodiment. [Figure 5] 10 is an example of a landmark position in the first embodiment. [Figure 6] 4 is an example of evaluation data in the first embodiment. [Figure 7] 3 is a flowchart of the image analysis device 10 in the first embodiment. [Figure 8] 4 is an example of an output of an evaluation result by the image analysis device 10 in the first embodiment. [Figure 9] 1 is a hardware configuration diagram of an image analysis device 10 according to a first embodiment. [Figure 10] 10 shows an example of a comparative experiment result in the second embodiment. [Figure 11] 10 shows the results of statistical analysis of task results in the second embodiment. [Figure 12] 10 is an example of a statistical analysis result based on hand movement detection in the second embodiment. [Figure 13] 10 is another example of a statistical analysis result based on the result of hand movement detection in the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of the present invention will be described. However, the present invention can be implemented in many different forms and is not limited to the examples of the embodiments and examples shown below. Furthermore, in the specification and drawings, elements similar to those described above with reference to the previous drawings will be designated by the same reference numerals, and detailed descriptions thereof will be omitted as appropriate.

[0011] (First embodiment) FIG. 1 is a schematic diagram of an image analysis system 1 according to the first embodiment.

[0012] The image analysis system 1 includes an image analysis device 10, an imaging device 20, a display device 30, a network device 40, and a surgical training device 50. The image analysis system 1 evaluates the surgical skill of an unskilled surgeon based on, for example, image analysis.

[0013] In the image analysis system 1 of this embodiment, the image analysis device 10 performs preprocessing on a video captured by the imaging device 20 and detects hand movements. The hand movement detection may be performed using, for example, machine learning-based detection software. Furthermore, the hand movement detection may be performed in real time or non-real time.

[0014] The surgical training device 50 includes a simulated organ 60 and a training box 70 containing the simulated organ 60, allowing the surgeon to simulate surgery. The training box may have an insertion port for inserting a medical instrument such as forceps, depending on the surgery being simulated. The simulated organ 60 has a structure that mimics the layered structure of a human abdominal organ, for example, allowing the surgeon to simulate the procedure of peeling away the layered structure.

[0015] The imaging device 20 captures a video of the procedure being evaluated by the surgeon. The captured video is transmitted to a network via the network device 40 and received by the image analyzing device 10. The network in this embodiment may be configured as an Ethernet or may be configured as the Internet using a public line such as an optical communication line or a wireless communication line.

[0016] Furthermore, in this embodiment, for the sake of simplicity, the image analysis system 1 is configured to include one imaging device 20, but it may also include multiple imaging devices 20. For example, the image analysis system 1 may include two imaging devices 20, and the image analysis device 10 may perform image analysis using images of the procedure captured from above and from the side. Furthermore, images of the procedure may not only be captured of the hands, but also of the body, head, or legs, for example, from the side of the surgeon. The image analysis system 1 may detect the movements of these parts and evaluate the surgeon's skill.

[0017] As a preprocessing step for the received video, the image analysis device 10 detects the surgeon's hand movements. After detecting the hand movements, the image analysis device 10 analyzes the movements and evaluates the surgical skills. The image analysis device 10 also outputs the evaluation results of the surgical skills to the display device 30. The evaluation results of the surgical skills are output, for example, as a radar chart.

[0018] The display device 30 is, for example, a display. The display device 30 outputs the evaluation result of the surgical skill output by the image analysis device 10.

[0019] FIG. 2 is another schematic diagram of the image analysis system 1 according to the first embodiment.

[0020] In FIG. 2, image analysis system 1 includes a mobile terminal 80 such as a smartphone instead of imaging device 20. Image analysis system 1 captures images of the surgeon's procedure using mobile terminal 80, and performs preprocessing within mobile terminal 80 to detect the surgeon's hand movements. After detecting the surgeon's hand movements, mobile terminal 80 transmits the detected results to image analysis device 10 via a network. Image analysis device 10 accepts the detected hand movement results as input, so there is no need for preprocessing within the device itself. Below, image analysis system 1 will be described using the configuration in FIG. 2 unless otherwise specified.

[0021] FIG. 3 is an example of a block diagram of the image analysis device 10 according to the first embodiment.

[0022] The image analysis device 10 includes an input unit 11, an analysis unit 12, an output unit 13, and a storage unit 14. Each function of these functional blocks may be distributed and implemented in multiple physically separated configurations. The image analysis device 10 is, for example, a computer, and includes a central processing unit (CPU), a main storage device such as a random access memory (RAM), an auxiliary storage device such as a read-only memory (ROM), and a computer-readable storage medium. The analysis unit 12 is divided into two functional blocks: an extraction unit 15 and an evaluation unit 16. A series of processes for realizing each functional block is stored in the form of a program in a storage medium, and is realized when the CPU reads the program into the RAM and executes it.

[0023] The input unit 11 receives as input video data including the results of detecting the movement of the surgeon's hand from the mobile terminal 80 via a network. When the hand movement detection is performed by the image analysis device 10 as preprocessing, as in the image analysis system 1 shown in Fig. 1, the input unit 11 may receive as input raw data a video of the procedure captured by the imaging device 20, instead of the results of detecting the movement of the surgeon's hand.

[0024] The analysis unit 12 analyzes the results of the detection of the surgeon's hand movements input by the input unit 11, and evaluates the surgeon's surgical skills.

[0025] The extraction unit 15 of the analysis unit 12 extracts evaluation data from the input video data. In this example, the extraction unit 15 extracts the trajectories of the left and right hands during surgery, the total distance traveled by the left and right hands, the number of turns of the left and right hands, and the procedure time as evaluation data from the video data. The evaluation data may be at least one of these, or may be other data.

[0026] The evaluation unit 16 of the analysis unit 12 evaluates the procedural skill based on the evaluation data extracted by the extraction unit 15 and generates an evaluation result. The evaluation result may be generated, for example, by determining a threshold value for each evaluation data item in advance through hand movement analysis of an experienced physician and comparing this threshold value with the hand movement detection result of an unskilled physician. The evaluation result may also be generated using statistical analysis. The user may also determine another evaluation method when generating the evaluation result. In this case, the evaluation unit 16 generates the evaluation result according to this evaluation method.

[0027] Compared to inexperienced surgeons, experienced surgeons tend to move their hands less during surgery. Therefore, in this embodiment, the total distance of left and right hand movement during surgery is added as an evaluation item by the evaluation unit 16. For example, the evaluation unit 16 may evaluate surgical skill by comparing the total distance of left and right hand movement extracted as evaluation data with a threshold value. The threshold value used for this comparison is a design factor, and may be determined, for example, from the average total distance of hand movement required when experienced surgeons perform a similar surgery.

[0028] Furthermore, experienced surgeons tend to turn their hands less frequently during surgery compared to inexperienced surgeons. Therefore, in this embodiment, the number of left hand turns is added as an evaluation item by the evaluation unit 16. For example, the evaluation unit 16 may evaluate surgical skill by comparing the number of left hand turns extracted as evaluation data with a threshold value. The threshold value used for this comparison is also a design factor; for example, the average number of hand turns required by experienced surgeons when performing a similar surgery may be set as the threshold value.

[0029] Furthermore, skilled surgeons tend to require shorter surgery times than inexperienced surgeons. Therefore, in this embodiment, procedure time is added as an evaluation item by the evaluation unit 16. For example, procedure time can be evaluated by comparing the procedure time extracted as evaluation data with a threshold value. The procedure time may be determined as the time from the start to the end of the video, or by detecting the start and end positions of the left and right hands, or by other methods. The threshold value used for this comparison is also a design factor; for example, the average procedure time required by skilled surgeons to perform a similar surgery may be set as the threshold value.

[0030] Furthermore, experienced surgeons tend to have more efficient surgical skills than inexperienced surgeons. Therefore, in this embodiment, the efficiency of surgical skills is added as an evaluation item by the evaluation unit 16. For example, the efficiency of surgical skills can be evaluated based on how far the procedure has progressed in one surgical field development.

[0031] Furthermore, experienced physicians tend to be better at planning surgical field preparation than inexperienced physicians. Therefore, in this embodiment, the planning of surgical field preparation is added as an evaluation item by the evaluation unit 16. For example, the evaluation unit 16 may evaluate the planning of surgical field preparation from the trajectories of the left and right hands during surgical field preparation, or may use other methods. For example, any method may be used to evaluate the planning of surgical field preparation, such as efficiently preparing the surgical field using the non-dominant hand.

[0032] Furthermore, experienced surgeons tend to have better coordination between their left and right hands during surgery than inexperienced surgeons. Therefore, in this embodiment, left and right hand coordination is added as an evaluation item by the evaluation unit 16. For example, the evaluation unit 16 may evaluate the coordination from the trajectories of the left and right hands extracted as evaluation data. For example, the evaluation unit 16 compares the trajectories of the left and right hands with those of a model experienced surgeon performing a similar surgery, and evaluates the coordination between the left and right hands based on the difference. The evaluation unit 16 may calculate the similarity from the difference, or may use another method.

[0033] While non-experts tend to focus on hand movements during surgery and end up tiring due to the strain, experienced surgeons tend to compensate for hand movements by manipulating the entire body rather than just the hands. Therefore, the image analysis system 1 may add the results of detecting the movements of the surgeon's body, head, or legs to the evaluation items. For example, it is conceivable to evaluate the surgeon's skill by comparing the distance and direction of movement with a threshold value.

[0034] The output unit 13 outputs the evaluation results created by the evaluation unit 16 to the display device 30. The output unit 13 outputs the quantitative evaluation results for each evaluation item in the form of, for example, a radar chart.

[0035] The storage unit 14 stores the video data input by the input unit 11. The video data is stored, for example, in a database within the storage unit 14. The storage unit 14 also stores the evaluation data extracted by the extraction unit 15 and the evaluation results created by the evaluation unit 16 in the database, linking them to the video data.

[0036] The output unit 13 may output past video data and past evaluation results stored in the storage unit 14. For example, when a user wants to check past evaluation results, the output unit 13 outputs the evaluation results stored in the storage unit 14 to the display device 30 based on an operation from the user.

[0037] FIG. 4 is another example of a block diagram of the image analyzing device 10 according to the first embodiment.

[0038] In the configuration of image analysis system 1 as shown in Fig. 1, input unit 11 receives input of raw data, which is a video of a procedure captured by imaging device 20. After input unit 11 receives input of the raw data, detection unit 17 detects hand movement. Detection unit 17 detects hand movement using a detection program in the same manner as described above.

[0039] After the detection unit 17 detects the hand movement, this data is input to the analysis unit 12, where the same processing as in the block diagram of FIG. 3 is performed.

[0040] FIG. 5 shows an example of landmark positions in the first embodiment.

[0041] In this example, an example of detecting the movement of a right hand using landmark positions will be described. In hand movement detection, hand movement is estimated from a captured video based on landmark positions defined by detection software. In this detection software, 20 positions, such as the fingers and back of the right hand, are registered as landmark positions. When an image of an operator simulating surgery using a simulated organ 60 is captured by a mobile terminal 80, the detection software estimates the hand movement at that time in real time or non-real time. Estimation is performed for each landmark position.

[0042] The extraction unit 15 extracts evaluation data based on the landmark positions. For example, the extraction unit 15 can extract hand movements as trajectories by extracting certain landmark positions at multiple points in time in the video data.

[0043] Furthermore, the extraction unit 15 can extract the number of times the surgeon turned his / her hand from the relationship between the positions of multiple landmarks at multiple points in time. For example, if the positional relationship between the number 5 landmark position (base of the index finger) and the number 17 landmark position (base of the little finger) extracted at one point in time and these landmark positions extracted at another point in time cross on the XY coordinate, the extraction unit 15 can determine that the surgeon turned his / her hand. The method for extracting the number of times the surgeon turned his / her hand is one example, and other methods may also be used.

[0044] FIG. 6 is an example of evaluation data in the first embodiment.

[0045] FIG. 6 shows a trajectory extracted by the extraction unit 15 as evaluation data from the results of hand movement detection. For example, the mobile terminal 80 is fixed in a predetermined position during imaging, and reference XY coordinates are determined from the fixed position. This trajectory is data depicting the coordinates and trajectory at each time point for a landmark position on the left hand as a measuring point. In this example, the hand position is shown as X and Y coordinates for each second. The X axis in the figure indicates the X coordinate of the landmark position, and the Y axis indicates the Y coordinate of the landmark position. Also in this figure, each plot indicates the X and Y coordinates of a landmark position from 0 seconds when measurement started to 118 seconds, and the dashed line indicates the trajectory.

[0046] After the input unit 11 receives video data as input, the extraction unit 15 extracts the XY coordinates and trajectory at each time point for each landmark position. The evaluation unit 16 evaluates the surgical skill based on the extracted coordinates, trajectories, etc.

[0047] FIG. 7 is a flowchart of the image analysis device 10 in the first embodiment.

[0048] This flowchart explains the process by which image analysis device 10 accepts video data as input and evaluates the surgical skills of a surgeon. For the sake of explanation, an example will be given in which image analysis device 10 accepts input of a single video and performs an evaluation of the surgical skills for this image, but image analysis device 10 may also accept input of multiple videos taken from different angles and perform various skill evaluations based on these videos.

[0049] In step S1, the input unit 11 receives input of the results of detecting the movement of the surgeon's hands as video data from the mobile terminal 80 via the network. The input unit 11 also stores the received input video data in the storage unit 14. In step S2, the extraction unit 15 extracts evaluation data from the video data. In this embodiment, the extraction unit 15 extracts, from the video data, the trajectories of the left and right hands during surgery, the total distance traveled by the left and right hands, the number of turns made by the left and right hands, and the procedure time as evaluation data.

[0050] In step S3, the evaluation unit 16 creates an evaluation result based on the extracted data. The evaluation unit 16 evaluates, for example, the total distance traveled by the surgeon's left and right hands during surgery, the number of times the left hand turns, the procedure time, the efficiency of the surgical skill, the planning of the surgical field development, and the coordination of the left and right hands. In step S4, the output unit 13 outputs the evaluation result to the display device 30, for example, as a radar chart.

[0051] FIG. 8 shows an example of an output of an evaluation result by the image analysis device 10 in the first embodiment.

[0052] This example shows a radar chart output by the output unit 13 to the display device 30. This radar chart shows the evaluation results for the total distance traveled by the surgeon's left and right hands during surgery, the number of left hand turns, procedure time, efficiency of surgical skills, planning of surgical field preparation, and left and right hand coordination, using a five-point scale from 1 to 5. In this radar chart, 5 indicates the highest evaluation, and 1 indicates the lowest evaluation. This figure shows that the evaluation result for the total distance traveled by the surgeon's left and right hands during surgery is 3, the evaluation result for the number of left hand turns is 4, the evaluation result for procedure time is 3, the evaluation result for efficiency of surgical skills is 4, the evaluation result for planning of surgical field preparation is 3, and the evaluation result for left and right hand coordination is 2.

[0053] FIG. 9 is a diagram showing the hardware configuration of the image analyzing device 10 according to the first embodiment.

[0054] 9 includes a CPU 100, a main memory device 101, an auxiliary memory device 102, a network interface 103, a device interface 104, and a bus 105. The image analyzing device 10 is, for example, a computer such as a PC, and may include input devices such as a keyboard and a mouse, and an output device such as an LCD (Liquid Crystal Display) monitor, in which case they are connected via the device interface 104.

[0055] In this embodiment, a program for causing a computer to execute information processing of the image analysis device 10 is installed in the ROM 104. The image analysis device 10 loads this program into the main storage device 101 and executes it using the CPU 100. This realizes the functions of the input unit 11, analysis unit 12, output unit 13, storage unit 14, and detection unit 17 shown in FIGS. 3 and 4 within the image analysis device 10, enabling the information processing described in this embodiment. Note that data generated by this information processing is temporarily held in the main storage device 101 or stored and saved in the auxiliary storage device 102. Furthermore, the storage unit 14 is constructed on the auxiliary storage device 102.

[0056] Furthermore, the image analyzing device 10 is connected to a network via a network interface 103. Furthermore, the image analyzing device 10 controls the network interface 103 by the input unit 11 to acquire video data.

[0057] The program for image analysis device 10 can be installed, for example, by connecting external device 106 storing the program to device interface 104 and copying the program from external device 106 to auxiliary storage device 102. In this case, examples of external device 106 include a computer-readable recording medium and a recording device incorporating such a recording medium. Examples of recording media include CD-ROMs (Compact Disk Read Only Memory), CD-Rs (Compact Disk Recordable), flexible disks, DVD-ROMs (Digital Versatile Disk Read Only Memory), and DVD-Rs (Digital Versatile Disk Recordable), and examples of recording devices include HDDs. Furthermore, the program may be downloaded from a network via network interface 103.

[0058] Furthermore, in the present embodiment, an example has been described in which image analyzing device 10 evaluates the surgical skills of an inexperienced surgeon, but skill evaluation using image analyzing device 10 is not limited to this example. For example, image analyzing device 10 may be used to evaluate other procedural skills of doctors, such as evaluating endoscopic examination procedural techniques, and may also be used to evaluate the skills of other occupations, such as nurses or caregivers.

[0059] According to this embodiment, the image analyzing device 10 evaluates the surgical skill of the surgeon based on the results of detecting the hand movements of the surgeon. This allows the image analyzing device 10 to quantitatively evaluate the surgical skill of the surgeon without requiring measures such as the introduction of a position measurement system or attaching markers to surgical tools.

[0060] Furthermore, according to this embodiment, the image analysis device 10 evaluates the total distance traveled by the surgeon's left and right hands during surgery, the number of times the left hand turns, the procedure time, the efficiency of the surgical skill, the planning of the surgical field development, and the coordination of the left and right hands, based on the extracted evaluation data. This allows the surgical skill of an experienced surgeon to be objectively quantified for each detailed procedure item, making it possible to provide feedback to inexperienced surgeons and contributing to improving the learning curve.

[0061] (Second embodiment) FIG. 10 shows an evaluation experiment in the second embodiment.

[0062] FIG. 10A shows the procedure for dissecting the simulated organ 60, and FIG. 10B shows the simulated organ 60 after dissection. FIG. 10C shows a binarized image of a cross section of the simulated organ 60. The arrow in the figure indicates the first layer of the simulated organ 60, and the tip of the arrow indicates the second layer of the simulated organ 60. In this evaluation experiment, a VTT DOUBLE LAYER TYPE VTT-DBL (manufactured by KOTOBUKI Medical Co., Ltd.) was used as the simulated organ 60. The simulated organ 60 is not limited to this example, and various other simulated organs 60 may be used.

[0063] In this embodiment, the results of an evaluation experiment using the image analysis device 10 will be described. The configuration of the image analysis system 1 is the same as that shown in FIG. 1 or 2, and therefore a description thereof will be omitted. Also, the block diagram of the image analysis device 10 is the same as that shown in FIG. 3 or 4, and therefore a description thereof will be omitted. Other parts that are the same as those in the first embodiment will be omitted, and the following will mainly focus on the differences.

[0064] This evaluation experiment was conducted by dividing the participants into two groups. The participants in each group performed a procedure (hereinafter also referred to as a task) of peeling the first layer of a two-layered simulated organ 60 on a surgical training device 50. The participants in the first group performed the task using scissors with straight tips, and the participants in the second group performed the task using scissors with curved tips.

[0065] Regarding the background of the participants, the sex ratio was 6:21 (female:male), the median age was 40 years old, the median years of experience was 12 years, and all participants were right-handed. When the experimenter performed Fisher's exact probability test on the background of the participants, no statistical difference was found between the two groups.

[0066] In the ideal peeled state of the two-layer structure of the simulated organ 60, no tissue from one layer remains in the cross section of the other layer, and no tissue from one layer remains in the cross section of the other layer. In an actual task, when the first layer of the simulated organ 60 is peeled, tissue from the second layer may remain in the cross section of the first layer, and tissue from the first layer may remain in the cross section of the second layer. As shown in Figure 10C, in this evaluation experiment, images of these cross sections were binarized, and the proportion of the area of ​​the first layer's tissue in the first cross section (the proportion of dark-colored pixels) was assigned a positive point, and the proportion of the area of ​​one layer's tissue remaining in the second cross section (the proportion of dark-colored pixels) was assigned a negative point. The sum of these scores was defined as the task score. The task score may be calculated by the analysis unit 12. For example, the extraction unit 15 may binarize images included in the video data as evaluation data, and the evaluation unit 16 may calculate the task score from the binarized images.

[0067] In this evaluation experiment, in addition to calculating the task score, the experimenter measured the time taken for the task (from the start of dissection to the end), the number of times the scissors were inserted into the simulated organ 60 during the task, the number of times the simulated organ 60 was re-grasped with the forceps in the left hand, and the number of times the pulling direction of the left hand was changed (reverse pulling), as evaluation data. These evaluation data may be automatically extracted by the extraction unit 15.

[0068] Furthermore, regarding the state of this procedure, the extraction unit 15 extracted, as evaluation data, the total movement distance of the right and left hands, the number of times the right and left hands changed direction along the X axis, and the frequency of the right and left hands changing direction along the Y axis, based on the landmark positions.

[0069] In addition, the evaluation unit 16 performed statistical analysis using Wilcoxon matched pair test. The statistical analysis result was evaluated as being significantly different when p<0.05.

[0070] FIG. 11 shows the results of statistical analysis of task results in the second embodiment.

[0071] The group that performed the procedure using the curved-tip scissors had a significantly higher task score (p=0.0151). There were no significant differences in the time taken to complete the task, the number of times the scissors were inserted into the mock organ 60 during the task, or the number of times the mock organ 60 was re-grasped with the forceps in the left hand.

[0072] FIG. 12 shows an example of a statistical analysis result based on hand movement detection in the second embodiment.

[0073] As a result of statistical analysis based on the hand movement detection by the evaluation unit 16, there was no significant difference in the total movement distance between the right and left hands when using shears with straight tips and when using shears with curved tips (right hand: p=0.2021, left hand: p=0.6277). On the other hand, the frequency of direction changes of the left hand along the X-axis was significantly lower in the group using shears with curved tips.

[0074] FIG. 13 shows another example of the statistical analysis result based on the result of hand movement detection in the second embodiment.

[0075] The evaluation unit 16 divided the results obtained in Fig. 12 by the time required to complete the task, and statistically analyzed the total distance traveled by the right and left hands per second, the number of times the right and left hands changed direction along the X axis per second, and the frequency of the right and left hands changing direction along the Y axis per second. The changes in direction along the X axis and the Y axis represent hand turns.

[0076] As shown in FIG. 13, the results of the statistical analysis based on the hand movement detection by the evaluation unit 16 showed that the left hand had significantly lower movement distance (p=0.0299), frequency of direction changes along the X axis (p=0.0059), and frequency of direction changes along the Y axis (p=0.0491) in the group using shears with curved tip shapes.

[0077] Through the above-mentioned evaluation experiment, it was possible to calculate the task scores when using shears with curved tips and when using shears with straight tips, and through statistical analysis of this and statistical analysis of hand movement detection, it was possible to calculate the movement trends of the dominant hand and the non-dominant hand.

[0078] In this embodiment, the image analysis device 10 performs statistical analysis of tasks, etc., and quantitatively evaluates surgical tools. According to this embodiment, the image analysis device 10 can be used not only to evaluate surgical techniques, but also to evaluate surgical tools. [Explanation of symbols]

[0079] 1: Image analysis system, 10: Image analysis device, 11: Input unit, 12: Analysis unit, 13: Output unit, 14: Memory unit, 15: Extraction unit, 16: Evaluation unit, 17: Detection unit, 20: imaging device, 30: display device, 40: network device, 50: surgical training device, 60: Simulated organ, 70: Training box, 80: Mobile terminal, 100: CPU, 101: main memory device, 102: auxiliary memory device, 103: network interface, 104: Device interface, 105: Bus, 106: External device

Claims

1. an input unit that receives video data of a surgeon's procedure as input; an extraction unit that extracts, from the result of the detection of the surgeon's hand movement included in the video data, at least one of the trajectory of the hand, the total moving distance of the hand, the number of times the hand turns, and the procedure time as evaluation data; and an evaluation unit that generates an evaluation result of a procedural skill or a surgical tool based on the evaluation data. Image analysis device.

2. 2. The image analyzing device according to claim 1, wherein the evaluation unit generates the evaluation result by evaluating at least one of a total distance moved by the hand of the surgeon during surgery, a number of times the hand turns, a procedure time, efficiency of the procedural skill, a planned surgical field development, or coordination between the left and right hands.

3. The image analyzing device according to claim 1 , wherein the evaluation unit quantitatively evaluates the procedural skill or the surgical tool by comparing the evaluation data with a threshold or by performing statistical analysis on the evaluation data.

4. It accepts video data of the surgeon's technique as input, extracting, from the result of detecting the movement of the surgeon's hand included in the video data, at least one of the trajectory of the hand, the total moving distance of the hand, the number of times the hand turns, or the procedure time as evaluation data; Creating an evaluation result of a procedural skill or a surgical tool based on the evaluation data. An image analysis method comprising:

5. It accepts video data of the surgeon's technique as input, extracting, from the result of detecting the movement of the surgeon's hand included in the video data, at least one of the trajectory of the hand, the total moving distance of the hand, the number of times the hand turns, or the procedure time as evaluation data; Creating an evaluation result of a procedural skill or a surgical tool based on the evaluation data. An image analysis program that causes a computer to carry out an image analysis method including the steps of:

6. a surgical training device for simulating a surgical procedure using a simulated organ and a training box in which the simulated organ is placed; an image analyzer; An image analysis system comprising: The image analysis device an input unit that receives video data of a surgeon's procedure as input; an extraction unit that extracts, from the result of the detection of the surgeon's hand movement included in the video data, at least one of the trajectory of the hand, the total moving distance of the hand, the number of times the hand turns, and the procedure time as evaluation data; and an evaluation unit that generates an evaluation result of a procedural skill or a surgical tool based on the evaluation data. Image analysis system.

Images