Therapeutic moving image editing device

Abstract

To provide an image editing device capable of automatically adjusting image quality of a therapeutic image.SOLUTION: A therapeutic moving image editing device 1 comprises: a therapeutic moving image input part 3 in which a therapeutic moving image is input; a tag installation part 5 for analyzing medical appliance to be used which is a medical appliance included in the therapeutic moving image, and providing a tag related to the medical appliance to be used in the therapeutic moving image, on the basis of the medical appliance to be used; and a section image quality adjustment part 7 for adjusting the image quality of a section moving image which is a moving image of a section partitioned by the tag in the therapeutic moving image. A therapeutic moving image with the tag which is a therapeutic moving image with the tag can be cued from the part where the tag is provided, and the section image quality adjustment part adjusts the image quality of the section moving image, according to the medical appliance to be used.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] This invention relates to an editing device for surgical videos and the like.

Background Art

[0002] Japanese Patent Application Laid-Open No. 2006-43209 describes a video recording device for surgical operations. This device stores data on the names of surgical procedures or instruments, selects and specifies a particular surgical procedure or instrument existing in the index title storage section, and records an index image corresponding to the operation timing. This device enables the recorded surgical video to be saved in a state where it can be quickly searched. Therefore, this device is beneficial because it enables the surgical video to be searched and further enables the index image to be displayed.

[0003] Japanese Patent No. 5044069 describes a medical image diagnostic device. This device can obtain a surgical video in which bookmark data of a type is associated at an arbitrary timing on the time scale. This bookmark is a marker for accessing the video.

[0004] In the publication of Japanese Patent Application Laid-Open No. 2022-520701, access is made to video images of a specific surgical procedure taken by cameras (for example, the overhead cameras 125, 232123, and the table-side camera 125) and (

[0044] -

[0050] , FIG. 1) (step 802 in FIG. 8A). To identify the positions of the video images related to the surgical stages of the specific surgical procedure, technologies such as video motion detection, video tracking, shape recognition, object detection, fluid flow detection, device identification, behavior analysis, or one or more of other forms of computer-assisted situation recognition are used. For example, the video images are analyzed to identify one or more medical devices used in the surgical procedure. Based on the identification of the medical devices, specific intraoperative events are identified at the positions within the video images related to the medical devices (

[0074] ,

[0098] ,

[0148] , step 804 in FIG. 8A). Tagging is performed on the identified intraoperative events (steps 806-808 in FIG. 8A). Video analysis is performed to identify the event positions of specific intraoperative surgical events within the surgical stage (step 810 in FIG. 8A). By associating the tags with the event positions of the specific intraoperative surgical events, the tags are, for example, associated with the surgical events at the event positions in the video images (steps 812-step 816 in FIG. 8A). When the user selects a tag, a subset that matches the tag in the video image can be searched for and displayed (

[0128] -

[0230] , steps 818-step 822 in FIG. 8A) is described (

[0146] -

[0250] , FIGS. 7-8).

[0005] Surgery may take a long time. Therefore, even if tags or bookmarks are used to enable access only to the necessary parts, there is a problem that the capacity of the video itself becomes large.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0007] An object of the present invention is to provide an editing device for a surgical video that can reduce the capacity of the surgical video while maintaining the video portions necessary for surgical learning.

Means for Solving the Problems

[0008] Basically, the present invention is based on the finding that, for a surgical video, by tagging according to the medical devices in the video and adjusting the image quality according to the medical devices used in the sections delimited by the tags, the capacity of the video can be significantly reduced.

[0009] The editing device 1 for a surgical video includes a surgical video input unit 3, a tag installation unit 5, and a section image quality adjustment unit 7. The surgical video input unit 3 is an element for inputting a surgical video into the device 1. The tag installation unit 5 analyzes the medical devices (referred to as "used medical devices") included in the surgical video and is an element for providing tags related to the used medical devices to the surgical video based on the used medical devices. The surgical video provided with tags (also referred to as "tagged surgical video") can be cued from the tagged portions. The section image quality adjustment unit 7 is an element for adjusting the image quality of a section video, which is the video of the sections delimited by tags in the surgical video. The section image quality adjustment unit 7 adjusts the image quality of the section video according to the used medical devices. The editing device 1 for a surgical video can tag according to the medical devices and adjust the image quality according to the medical devices used in the sections delimited by the tags.

[0010] A preferred example of the interval image quality adjustment unit 7 is that, when the intervals delimited by tags are used as tag intervals, the interval video of the tag interval in the interval video where there is no medical device in use, i.e., the interval video of the non - existent medical device use interval, is set as a still image or an interval video with a lower image quality than that of other interval videos. The intervals during which medical devices are not used are considered to have a lower importance. Therefore, by setting the non - existent medical device use intervals as representative still images or suppressing the image quality, the capacity of the video can be reduced.

[0011] A preferred example of the surgical video editing device 1 further includes a surgical scene extraction unit 9 and an extra - surgical video image quality adjustment unit 11. The surgical scene extraction unit 9 is an element for extracting surgical scenes from surgical videos using a learned model for surgical scenes, which is a learned model obtained by training a past surgical video and information regarding the surgical scenes in the past surgical video as teacher data. The extra - surgical video image quality adjustment unit 11 is an element for setting an extra - surgical video, which is a surgical video other than the surgical scenes in the surgical video, as a still image or an interval video with a lower image quality than that of other interval videos. Through machine learning, for scenes where surgical scenes cannot be visually recognized, the capacity of the video can be reduced by setting them as still images or suppressing the image quality.

[0012] A preferred example of the surgical video editing device 1 further includes an implemented surgical procedure information adding unit 13 and an implemented surgical procedure related video extraction unit 15. The implemented surgical procedure information adding unit 13 is an element for analyzing the implemented surgical procedure, which is the surgical procedure of the surgical video, based on the surgical video and adding information regarding the implemented surgical procedure to the surgical video. The implemented surgical procedure related video extraction unit 15 is an element for extracting the implemented surgical procedure related video, which is a part related to the implemented surgical procedure in the surgical video. This device can reduce the capacity of the video by extracting the part of the implemented surgical procedure from the surgical video. By extracting the implemented surgical procedure part, the capacity of the video can be reduced.

[0013] Another invention described in this specification relates to a program for causing a computer to function as the above-described surgical video editing device, and a computer-readable non-transitory recording medium storing such a program.

Advantages of the Invention

[0014] According to this invention, it is possible to provide a surgical video editing device that can suppress the capacity of a surgical video while maintaining the video portions necessary for surgical learning.

Brief Description of the Drawings

[0015]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Embodiments for Carrying Out the Invention

[0016] Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. The present invention is not limited to the embodiments described below, and includes those appropriately modified by those skilled in the art within an obvious range from the following embodiments. In the following examples, the description will be centered on installing tags and adjusting the image quality for each video section delimited by the tags. On the other hand, an invention different from the above-described one in this specification, which adjusts the image quality of a video without delimiting the video with tags, is also described.

[0017] FIG. 1 is a block diagram for explaining an editing device for a surgical video. As shown in FIG. 1, this surgical video editing device 1 has a surgical video input unit 3 and a tag installation unit 5. This device 1 preferably further includes any one or more of an interval image quality adjustment unit 7, a surgical scene extraction unit 9, an extracorporeal surgical video image quality adjustment unit 11, an implemented surgical procedure information addition unit 13, and an implemented surgical procedure related video extraction unit 15. This device 1 may further have a person specific part removal unit 17. This device 1 may further have any one or two or more of a medical device storage unit 21, a hand information storage unit 23, a surgical procedure storage unit 25, a surgical target part storage unit 27, a surgical procedure storage unit 29, and a person specific part storage unit 31. The surgical video editing device 1 is implemented by a computer. That is, each element constituting the surgical video editing device 1 may be implemented by a computer.

[0018] A computer has an input unit, an output unit, a control unit, an arithmetic unit, and a storage unit. Each element is connected by a bus or the like so that information can be exchanged. For example, the storage unit may store a control program or various types of information. When predetermined information is input from the input unit, the control unit reads out the control program stored in the storage unit. Then, the control unit appropriately reads out the information stored in the storage unit and transmits it to the arithmetic unit. Also, the control unit appropriately transmits the input information to the arithmetic unit. The arithmetic unit performs arithmetic processing using the received various types of information and stores it in the storage unit. The control unit reads out the arithmetic result stored in the storage unit and outputs it from the output unit. In this way, various processes and steps are executed. The ones that execute these various processes are each unit and each means. The computer may have a processor, and the processor may realize various functions and various steps. The computer may be stand-alone. A part of the functions of the computer may be distributed between a server and a terminal. In that case, it is preferable that the server and the terminal can exchange information through a network such as the Internet or an intranet. The computer may include a processor and a memory connected to the processor. And the memory stores instructions, and when the instructions are executed by the processor, they may cause the computer to perform various steps or make the computer function as various elements. The computer may build a learning model by providing various types of teacher data and realize various operations by machine learning. In this case, the computer may execute various analyses and analyses using a learning model created by machine learning and deep learning of AI (artificial intelligence).

[0019] In the memory unit of the computer, for example, an analysis model (trained model) used when analyzing a surgical video may be stored. The analysis model is preferably a model that has learned a medical device used in the surgery (which may include changes in the position of the device), the surgical procedure in which the medical device is used, and the surgical steps. Note that the analysis model may further be a model that has learned the position of the hand or arm included in the surgical video (which may include changes in the position of the hand or arm), the surgical procedure corresponding to the position of the hand or arm, and the surgical steps. The computer preferably analyzes the surgical video of the surgery and performs at least one of identifying the surgical procedure of the surgery included in the surgical video data and classifying the surgical steps. The computer may classify the steps from the medical device used in the surgery and the position of the medical device (which may include changes in the position) from the surgical video. In addition to the medical device used in the surgery, the computer may classify the surgical steps from the change in the position of the hand or arm handling the medical device. Also, since it is possible to identify the surgical procedure from the combination of surgical steps including the order, the computer may identify the surgical procedure of the surgery based on the combination of steps including the classified order. At this time, the surgical procedure of the surgery may be identified with reference to the information associating the combination of steps including the order with the surgical procedure. In this way, the computer may learn various learning models by machine learning and perform various processes using the learned models.

[0020] The editing device 1 of the surgical video is a device for attaching a tag for head-out so that the surgical video can be viewed from a predetermined location of the surgical video. The surgical video may be a video related to a surgery or a surgical procedure. Examples of surgeries are surgical operations. Examples of surgical procedures are esthetics and massages. A tag, also called an index or a bookmark, is information attached to a video so that the video can be viewed from the part where the tag is added. It is preferable that the device 1 can search for the treatment video using the tag. For example, in a certain step of a certain treatment, the video can be viewed from the moment a certain medical device is used. The tag is preferably stored in the storage unit in association with, for example, the name of the medical device when the medical device is used in the treatment video. And for the display of the treatment video, the name of the medical device stored in association with the tag may be displayed together with the tag part. The treatment video provided with the tag can be retrieved from the part provided with the tag related to the medical device used using the name of the medical device related to the medical device used.

[0021] The treatment video input unit 3 is an element for inputting the treatment video. The treatment video editing device 1 is connected to a shooting unit (for example, a camera or a video camera) that shoots a video. When the treatment video is shot, the treatment video may be input to the treatment video editing device 1. Also, a video file related to the treatment video may be input to the treatment video editing device 1 by drag and drop. Furthermore, the treatment video editing device 1 may search for treatment videos on the Internet and automatically input the hit treatment videos. In this way, the treatment video is input to the treatment video editing device 1.

[0022] The tag setting unit 5 is an element for analyzing the medical devices included in the surgical video, which are the medical devices in use, and for providing tags related to the medical devices in use in the surgical video based on the medical devices in use. This apparatus 1 further includes a medical device storage unit 21. The medical device storage unit 21 stores the names of medical devices and the image data of medical devices in association with each other. The medical device storage unit 21 may further store the IDs of medical devices. The tag setting unit 5 may, for example, perform machine learning on the image data related to various medical devices, create a learning model related to medical devices, and store it in the storage unit. Then, the tag setting unit 5 may analyze the medical devices included in the surgical video using the learning model related to medical devices. The tag setting unit 5 may, for example, perform machine learning on the video data related to various surgical procedures and various surgeries, create a learning model related to various surgical procedures and various surgeries, and store it in the storage unit. Then, the tag setting unit 5 may analyze each surgical procedure in the surgical video and the actual surgery related to the surgical video using the learning model related to various surgical procedures and various surgeries.

[0023] The tag setting unit 5, for example, reads out the images (surgical images) included in the surgical video from the storage unit and reads out the image data of the medical devices from the medical device storage unit 21. Then, the tag setting unit 5 collates the surgical image with the image data of the first medical device. When the tag setting unit 5 determines that the image data of the first medical device is included in the surgical image, the tag setting unit 5 determines that the first medical device is the medical device in use. As a result of the collation process using the surgical image and the image data of the first medical device by the tag setting unit 5, when the tag setting unit 5 determines that the first medical device is not included in the surgical image, the tag setting unit 5 reads out the image data of the second medical device, which is the next medical device, from the medical device storage unit 21. Then, the tag setting unit 5 repeats the collation process between this surgical image and the image data of the medical device. By doing so, the tag setting unit 5 can identify the medical devices in the images included in the surgical video. That is, the tag setting unit 5 can identify the medical devices used at a certain timing in the surgical video.

[0024] A preferred example of the tag setting unit 5 is to analyze the image of the operator's hand included in the surgical video (surgical image), and use the medical device held by the operator's hand as the medical device to be used. For example, the surgical video may include a medical device that is not used in the surgery. In this example, even in such a case, tags can be provided for the medical device actually used in the surgery. In this example, the surgical video editing device 1 may have a hand information storage unit 23. Then, the tag setting unit 5 reads, for example, the image (surgical image) included in the surgical video from the storage unit, and reads the hand image data from the hand information storage unit 23. Then, the tag setting unit 5 collates the surgical image with the hand image data. In this way, the operator's hand in the image (surgical image) included in the surgical video can be grasped. When a plurality of medical devices are included in the surgical video (surgical image), the tag setting unit 5 may use the medical device close to the operator's hand in the surgical video (surgical image) as the medical device to be used. Also, when the hand information storage unit 23 stores the hand image data in a state where a medical device is being held, the medical device analyzed to be held by the hand in that state may be used as the medical device to be used.

[0025] A preferred example of the tag setting unit 5 is to analyze the hands of the surgeon and the assistant included in the surgical video (surgical image), and use the medical device held by the surgeon's hand as the medical device to be used. The tag setting unit 5 reads, for example, the image (surgical image) included in the surgical video from the storage unit, and reads the hand image data from the hand information storage unit 23. Then, the tag setting unit 5 collates the surgical image with the hand image data. When a plurality of hands are included in the surgical image, the tag setting unit 5 may analyze which hand is the surgeon's hand using information about the position of the hand in the surgical image and the medical device near the hand in the surgical image. For example, the hand information storage unit 23 stores the image data of the right hand and the image data of the left hand. Then, the tag setting unit 5 analyzes whether the hand included in the surgical image is the right hand or the left hand using the image data of the right hand and the image data of the left hand. Moreover, for example, the tag setting unit 5 may determine the pair of hands with a small distance from the center of the image among the plurality of pairs of hands as the pair of hands of the surgeon. Also, the pair of hands with a large distance from the center of the screen may be determined as the pair of hands of the assistant. Then, the tag setting unit 5 may analyze and store in the storage unit the medical device that is close to the surgeon's hand included in the surgical image or is analyzed to be held by the surgeon's hand as the medical device used by the surgeon. Further, the tag setting unit 5 may analyze and store in the storage unit the medical device that is close to the assistant's hand included in the surgical image or is analyzed to be held by the assistant's hand as the medical device used by the assistant.

[0026] A preferred example of the tag setting unit 5 is that based on the surgical video, it can analyze the actual surgical procedure, which is the surgical procedure of the surgical video, and add identification information regarding the surgical procedure of the actual surgery to the surgical video. Then, the surgical video with the identification information regarding the surgical procedure of the actual surgery added can be searched using the surgical procedure name regarding the actual surgical procedure.

[0027] The surgical steps mean each step in the surgery. Examples of surgical steps are deployment (e.g., lumbar deployment), fixation, screw insertion, decompression, intervertebral disc operation, and wound closure. The surgical steps are not limited to these examples and may be various steps in various surgeries and treatments.

[0028] The tag installation unit 5 may analyze the surgical process based on, for example, the type of medical device included in the surgical video (surgical image). The tag installation unit 5 may analyze the surgical process based on, for example, the movement of the medical device included in the surgical video (surgical image). The tag installation unit 5 may analyze the surgical process based on, for example, the types of the medical device and the surgical target site included in the surgical video (surgical image). The tag installation unit 5 may analyze the surgical process from information on the medical device used by the surgeon and the medical device used by the assistant among the medical devices included in the surgical video (surgical image).

[0029] The segment image quality adjustment unit 7 is an element for adjusting the image quality of a segment video, which is a video of a segment delimited by tags in the surgical video. The segment image quality adjustment unit 7 adjusts the image quality of the segment video according to the medical device used. The segment image quality adjustment unit 7 may adjust the overall image quality of a certain segment video, or may adjust the image quality of the video according to the medical device used in the segment video. The editing device 1 of the surgical video can attach tags according to the medical device, grasp the segments delimited by the tags, and further adjust the image quality according to the medical device used in the segments. For example, a learning model may be constructed by machine learning using the medical device and the importance of the video as teacher data, and a learned model may be constructed. Then, when one or more medical devices are used for a certain segment video, the segment image quality adjustment unit 7 reads the medical device used from the storage unit and inputs the medical device used into the learned model. Then, the segment image quality adjustment unit 7 can adjust the image quality of the entire segment video or the segment video according to the medical device used. The device 1 stores the medical device used and the image quality (the image quality is maximum, high, normal, low, extremely low, black-and-white image) in the storage unit, and the segment image quality adjustment unit 7 may read the image quality from the storage unit according to the medical device used and adjust the image quality of the segment video. The segment image quality adjustment unit 7 may store the surgical video with the adjusted image quality or the segment video with the adjusted image quality in the storage unit. Then, the capacity of the surgical video is reduced.

[0030] A preferable example of the interval image quality adjustment unit 7 is that, when the intervals delimited by tags are tag intervals, the interval video of the tag interval in the interval video where there is no medical device in use, i.e., the interval video of the non - existent medical device interval, is set as a still image or as an interval video with a lower image quality than that of other interval videos. The intervals during which no medical device is used are considered to have a low importance. Therefore, by setting the non - existent medical device interval as a representative still image or suppressing the image quality, the capacity of the video can be reduced. An example of the still image may be the image with the largest number of operators (surgeons, assistants, anesthesiologists, nurses, and clinical engineers) in the interval video of the non - existent medical device interval. Also, it may be an image in which the surgeon is prominently shown. The algorithm for selecting a representative image is known, and a known algorithm may be used to select the still image. Examples of those with a lower image quality than that of other interval videos are black - and - white images and images with a low resolution. Since the interval video of the non - existent medical device interval has less to be learned as a surgical video, by making it a still image or suppressing the image quality in this way, the capacity of the entire video can be significantly reduced. For example, the first interval video of the non - existent medical device interval may be an image that displays various information such as the surgical procedure being performed and the operators. Also, a sponsor may be displayed in the last interval video of the non - existent medical device interval. Further, for this sponsor display, link information to the sponsor's website may be attached. Then, by clicking on the link information or the like, it becomes possible to access the sponsor's website.

[0031] A preferred example of the surgical video editing device 1 further includes a surgical scene extraction unit 9 and a non-surgical video image quality adjustment unit 11. The surgical scene extraction unit 9 is an element for extracting surgical scenes from a surgical video using a learned model for surgical scenes, which is a learned model obtained by training a past surgical video and information on surgical scenes in the past surgical video as teacher data. In this example, the video part related to the surgical scene and the rest in the segment video can be separated, and by suppressing the image quality of the non-surgical video, the capacity of the video can be further reduced. In this example, for a certain medical device used, a learning model is constructed by machine learning using a past surgical video and information on surgical scenes in the past surgical video as teacher data. Then, the learned model can obtain the learned model regarding where the surgical scene starts and ends. By inputting the surgical video into the learned model, the surgical scenes in the surgical video can be automatically extracted. The surgical scene extraction unit 9 stores the start time and end time of the surgical scene in the storage unit. In this way, the surgical scene extraction unit 9 can extract the surgical scenes from the surgical video. The non-surgical video image quality adjustment unit 11 is an element for making the non-surgical video, which is a surgical video other than the surgical scenes in the surgical video, into a still image or a segment video with a lower image quality than that of other segment videos. The processing of the non-surgical video image quality adjustment unit 11 may be the same as the image quality adjustment of the video segment where the medical device is not present.

[0032] A preferred example of the surgical video editing device 1 further includes an actual surgical procedure information adding unit 13 and an actual surgical procedure related video extracting unit 15. The actual surgical procedure information adding unit 13 is an element for analyzing the actual surgical procedure, which is the surgical procedure of the surgical video based on the surgical video, and adding information related to the actual surgical procedure to the surgical video. As will be described later, an example of the actual surgical procedure information adding unit 13 may analyze the actual surgical procedure from the surgical video using the surgical procedure storage unit 25, the surgical target site storage unit 27, and the surgical procedure storage unit 29, add the actual surgical procedure information to the medical video or the segment video, and store it in the storage unit. Also, using the surgical video and the surgical procedure as teacher data, a learning model may be constructed by machine learning to obtain a learned model. By inputting the surgical video into the learned model, the actual surgical procedure can be analyzed. The obtained actual surgical procedure may be appropriately stored in the storage unit. The actual surgical procedure related video extracting unit 15 is an element for extracting the actual surgical procedure related video, which is a part related to the actual surgical procedure in the surgical video. For example, the computer performs machine learning using the surgical procedure and the surgical scene related thereto as teacher data to construct a learning model. Then, a learned model related to the surgical procedure can be obtained. The actual surgical procedure related video extracting unit 15 can extract the actual surgical procedure related video, which is a part related to the actual surgical procedure, by inputting the surgical video or the segment video into the learned model related to the surgical procedure. The actual surgical procedure related video extracting unit 15 stores the start time and the end time of the part related to the actual surgical procedure related video in the storage unit among the surgical video or the segment video. In this way, the actual surgical procedure related video extracting unit 15 can extract the actual surgical procedure related video. The device 1 may make the part other than the actual surgical procedure related video in the surgical video or the segment video into a still image or perform a process of suppressing the image quality. This device 1 can reduce the capacity of the video by extracting the actual surgical procedure part from the surgical video.

[0033] The treatment process memory unit 25 stores by associating treatment processes with medical devices. The tag installation unit 5 may read information regarding the medical devices used from the memory unit and read the treatment processes stored in association with the medical devices used from the treatment process memory unit 25. In this way, the tag installation unit 5 may read the treatment processes using the information necessary for analyzing the treatment processes from the treatment process memory unit 25 that stores the information necessary for analyzing the treatment processes. For example, assume that the treatment process memory unit 25 stores treatment processes in association with the medical devices used by the operator and the medical devices used by the assistant. In this case, the tag installation unit 5 reads the medical devices used by the operator and the medical devices used by the assistant from the memory unit and reads the treatment processes stored by the treatment process memory unit 25 in association with this information. The read treatment processes may be stored in the memory unit as appropriate.

[0034] Table 1 is a conceptual diagram for explaining the treatment process memory unit. As shown in Table 1, the treatment process memory unit 25 stores by associating treatment processes with medical devices. Also, as shown in Table 1, the treatment process memory unit 25 may store by associating treatment processes with medical devices and target sites. By doing so, the apparatus 1 can read the implemented treatment processes from the treatment process memory unit 25 using the information regarding the medical devices. For example, in the example of Table 1, when the medical devices used are A1, A2, A3, and A4, A may be read as the implemented treatment process. Also, in the example of Table 1, when the medical devices used are A1, A2, A3, and A4 and the target site is A5, A may be read as the implemented treatment process.

[0035]

Table 1

[0036] The treatment target site storage unit 27 stores, for example, the target site during treatment and the image of the target site in association with each other. The tag installation unit 5 may identify the treatment target site included in the treatment video (treatment image) by, for example, collating the image included in the treatment video (treatment image) with the image of the target site stored in the treatment target site storage unit 27. Examples of the treatment target site are the femur, collarbone, skull, large intestine, small intestine, and eyes. Such a collation operation can be performed by the tag installation unit 5 using machine learning.

[0037] A preferred example of the tag installation unit 5 is one that can analyze the actual treatment procedure, which is the treatment procedure of the treatment video, based on the treatment video and further provide tags related to the treatment procedure to the treatment video. Then, the treatment video with tags can be searched using the treatment procedure name related to the actual treatment process. The treatment procedure storage unit 29 may store treatment procedures in association with a plurality of treatment steps. Further, the treatment procedure storage unit 29 may store treatment procedures in association with a plurality of treatment steps and a plurality of medical devices. The tag installation unit 5 may read out information related to the treatment steps and a plurality of medical devices related to the treatment video from the treatment procedure storage unit 29, collate it with the treatment procedure storage unit 29, and read out the treatment procedures stored in association with these treatment steps and medical devices. The read treatment procedures may be stored in the storage unit in association with the treatment video.

[0038] Table 2 is a conceptual diagram for explaining the surgical procedure storage unit. As shown in Table 2, the surgical procedure storage unit 29 stores by associating a surgical procedure with a surgical process and medical devices. Also, as shown in Table 2, the surgical procedure storage unit 29 may store by associating a surgical procedure with a surgical process, medical devices, and a target site. By doing so, the apparatus 1 can read out the implemented surgical procedure from the surgical procedure storage unit 29 using the necessary information. For example, in the example of Table 2, when the surgical processes are B1, B2, B3, and B4 and the medical devices used are B5, B6, B7, and B8, B may be read out as the implemented surgical procedure. Also, in the example of Table 2, when the surgical processes are B1, B2, B3, and B4, the medical devices used are B5, B6, B7, and B8, and the target site is B9, B may be read out as the implemented surgical procedure.

[0039]

Table 2

[0040] A preferable example of the tag installation unit 5 is one that can further add link information for displaying information regarding the medical device used to the part of the medical device in the surgical video. The editing apparatus 1 of the surgical video, for example, stores in the storage unit, in association with the medical device, a description text regarding the medical device, an explanatory image, explanatory materials, access information (URL) to a website, and purchase information. Then, the editing apparatus 1 of the surgical video grasps the medical device part in the surgical images of the surgical video. When the medical device part is supported by an instruction using a touch panel or a mouse, an input for instructing the used medical device is made to the editing apparatus 1 of the surgical video. Then, the editing apparatus 1 of the surgical video reads out the link information regarding the used medical device stored from the storage unit, and is enabled to read out data related to the link information or access a website related to the link information. Then, information regarding the used medical device is displayed on the display unit of the user. In this way, for example, a user who views the surgical video can receive information on the medical device actually used or can purchase the medical device immediately.

[0041] A preferred example of the surgical video editing device 1 further includes a person specific part removing unit 17. The person specific part 7 is an element for analyzing an image part that identifies a patient or a medical staff member included in the surgical video, and for processing the surgical video so that the person specific part cannot be visually recognized when the person specific part is included in the surgical video. Since it further includes the person specific part removing unit 17, when the surgical video (surgical image) includes the face of the patient, tattoos, the face of the medical staff member, a nameplate, a hospital name, etc., it can be masked or mosaicked to ensure anonymity. In this way, a large number of videos can be made into a database after enhancing anonymity. Examples of the person specific part are the face of the patient, tattoos, the face of the medical staff member, a nameplate, and a hospital name.

[0042] For example, the person specific part removing unit 17 constructs a learning model using past surgical videos, examples of identifying the person specific part of the surgical video, and examples of processing the surgical video so that the person specific part in the surgical video cannot be visually recognized as teacher data, and performs machine learning. Then, the person specific part removing unit 17 may identify the person specific part of the input surgical video using the learning model obtained by machine learning, and process the surgical video so that the person specific part in the surgical video cannot be visually recognized. The surgical video editing device 1 may appropriately store the processed surgical video in the storage unit.

[0043] The person specific part storage unit 31 may store the learning model for identifying the person specific part of the surgical video described above. Further, the person specific part storage unit 31 may store image data related to the person specific part. In this case, the person specific part removing unit 17 collates the input surgical video (surgical image) with the image data related to the person specific part stored in the person specific part storage unit 31, and if the input surgical video (surgical image) includes an image part related to the person specific part, that part may be identified as the person specific part. Then, the person specific part removing unit 17 may perform image processing on the image part related to the person specific part included in the surgical video (surgical image) so that it cannot be visually recognized by others. Examples of the image processing are mosaicking, blurring processing, and overwriting a mask image on the image part.

[0044] The surgical video editing device 1 may create a database containing information such as the actual surgical procedure type, actual surgical procedure steps, and medical devices used for the tagged surgical video, and make the surgical video searchable based on this information. In this way, medical devices used, etc. can be indexed, and it becomes possible to easily view the necessary surgical video.

[0045] Another invention described in this specification relates to a program for causing a computer to function as the above-described surgical video editing device, and a computer-readable non-transitory recording medium storing such a program. Examples of the recording medium are a hard disk, CD, CD-ROM, floppy disk, Blu-ray disk, USB memory, and SD card.

[0046] The computer may include a processor and a memory connected to the processor. And the memory stores instructions (programs), and when the instructions are executed by the processor, the computer is caused to analyze the medical device used, which is the medical device included in the input surgical video, and provide a tag related to the medical device used to the surgical video based on the medical device used. Further, the program may cause the computer to analyze the actual surgical procedure steps, which are the surgical procedure steps of the surgical video, based on the surgical video, and further provide a tag related to the actual surgical procedure steps to the surgical video. Furthermore, the program may cause the computer to analyze the actual surgical procedure type, which is the surgical procedure type of the surgical video, and further add identification information related to the surgical procedure type to the surgical video. Furthermore, the program may cause the computer to analyze the human identification part, which is the image part for identifying the patient or medical staff included in the surgical video, and if the human identification part is included in the surgical video, process the surgical video so that the human identification part cannot be visually recognized.

[0047] FIG. 2 is a flowchart showing an example when tagging a surgical video. In the figure, S means step (process).

[0048] Video input step (S101) The surgical video is input from the surgical video input unit 3 into the surgical video editing device 1.

[0049] This step may be to read out the surgical video stored in the computer. Also, the captured surgical video may be input into the computer. The input surgical video is converted into digital data such as binary data so that the computer can perform arithmetic processing.

[0050] Medical device analysis step (S102) The tag installation unit 5 analyzes whether each surgical image included in the surgical video contains an image part related to a medical device. At this time, the tag installation unit 5 may refer to a learning model learned about medical devices. Also, the tag installation unit 5 may analyze whether the surgical image contains image data related to a medical device using the image data related to the medical device, thereby analyzing the presence or absence of a medical device included in the surgical video. When it is analyzed that no medical device is included in the surgical image, information indicating that no medical device exists may be stored in the storage unit together with the surgical image. Also, when the tag installation unit 5 analyzes that a medical device is included in the surgical image, information related to the medical device used, such as the name of the medical device used, may be stored in the storage unit together with the surgical image (and the time in the surgical video of the surgical image).

[0051] Tagging step for the medical device used (S103) When the tag installation unit 5 determines that a new medical device is included in the surgical video (when a medical device that is in the surgical image for the first time is included), a tag may be provided together with the identification information of the surgical image, such as the playback time of the surgical image. Specifically, the tag installation unit 5 associates the tag with the identification information of the surgical image and stores it in the storage unit. Information related to the medical device used may be stored in association with the tag. In this way, the tag installation unit 5 can obtain a surgical video with tags (referred to as a "tagged surgical video").

[0052] Step for analyzing the actual surgical procedure (S104) The tag installation unit 5 analyzes the actual treatment process, which is the treatment process of the treatment video, based on the treatment video, and further provides tags related to the actual treatment process to the treatment video. At this time, the tag installation unit 5 may refer to a learning model trained for the treatment process. In addition, the tag installation unit 5 may analyze the treatment process by referring to the information stored in the treatment process storage unit 25 and the treatment target site storage unit 27 using the information related to the medical device or the target site included in the treatment video (treatment image).

[0053] Tagging step for the actual treatment process (S105) When the tag installation unit 5 determines that a new actual treatment process is included in the treatment video (when the actual treatment process that first appears in the treatment image is included), tags may be provided together with the identification information of the treatment image, such as the playback time of the treatment image. Information related to the actual treatment process may be stored in association with the tag. In this way, the tag installation unit 5 can obtain a treatment video provided with tags related to the actual treatment process.

[0054] Actual treatment method analysis step (S106) The tag installation unit 5 analyzes the actual treatment method, which is the treatment method of the treatment video, based on the treatment video, and adds identification information related to the treatment method of the actual treatment to the treatment video. At this time, the tag installation unit 5 may refer to a learning model trained for the treatment method. In addition, the tag installation unit 5 may read out the actual treatment method stored in association with the actual treatment process in the actual treatment method storage unit 29 using the information related to the actual treatment process.

[0055] Actual treatment method information addition step (S107) The apparatus 1 (tag installation unit 5) may add identification information related to the treatment method of the actual treatment to the treatment video and store it in the storage unit in association with the treatment video. Since this identification information related to the treatment method of the actual treatment is also searchable information, it may be treated as a type of tag.

[0056] Interval image quality adjustment step (S108) The section image quality adjustment unit 7 adjusts the image quality of the section video, which is the video of the section delimited by tags in the surgical video. The section image quality adjustment unit 7 adjusts the image quality of the section video according to the medical device used. The section image quality adjustment unit 7 may adjust the overall image quality of a certain section video, or may adjust the image quality of the video according to the medical device used in the section video. The section image quality adjustment unit 7 reads the medical device used from the storage unit and inputs the medical device used into the learned model. Then, the section image quality adjustment unit 7 can adjust the image quality of the entire section video or the section video according to the medical device used. For example, the section image quality adjustment unit 7 makes the section video of the tag section without the medical device used in the section video, that is, the section video of the section where the medical device does not exist, into a still image. In this way, the capacity of the surgical video can be reduced by the section image quality adjustment process.

[0057] In the section image quality adjustment process, the surgical scene extraction unit 9 may automatically extract the surgical scene from the surgical video by inputting the surgical video into the learned model. The non-surgical video image quality adjustment unit 11 may make the non-surgical video, which is the surgical video other than the surgical scene in the surgical video, into a still image or a section video with a lower image quality than that of other section videos.

[0058] In the section image quality adjustment process, the implemented surgical procedure information addition unit 13 may analyze the implemented surgical procedure, which is the surgical procedure of the surgical video, based on the surgical video and add information related to the implemented surgical procedure to the surgical video. Then, the implemented surgical procedure related video extraction unit 15 may extract the implemented surgical procedure related video, which is the part related to the implemented surgical procedure in the surgical video. Then, the device 1 may make the part other than the implemented surgical procedure related video in the surgical video or the section video into a still image or perform a process of suppressing the image quality. The device 1 can reduce the capacity of the video by extracting the part of the implemented surgical procedure in the surgical video.

[0059] Human specific part determination step (S109) The human specific part removal unit 17 identifies the human specific part in the input surgical video (surgical image) using a learning model obtained by machine learning. When the human specific part removal unit 17 determines that the surgical image does not contain a human specific part, it does not perform human specific part removal. On the other hand, when the human specific part removal unit 17 determines that the surgical image contains a human specific part, it performs human specific part removal.

[0060] Human specific part removal step (S110) When the human specific part removal unit 17 determines that the surgical image contains a human specific part, it processes the surgical image so that the human specific part in the surgical image cannot be visually recognized. For example, the human specific part removal unit 17 may perform a process of making the human specific part into a mask image. In this way, the human specific part is processed so that it cannot be visually recognized from the surgical video.

[0061] FIG. 3 is a conceptual diagram showing an example of a tagged surgical video. In this example, a tag is displayed at a predetermined timing of the surgical video. And by clicking on the tag or the like, the surgical video can be browsed from the position where the tag is attached. In association with the tag, the medical device used may be displayed. In the example of FIG. 3, representative surgical images for each surgical step are displayed with a generic name (displayed small at the bottom of the video).

[0062] FIG. 4 is a conceptual diagram showing an example in which information for accessing information regarding a medical device is attached to the medical device part in a surgical video.

[0063] FIG. 5 is a conceptual diagram for explaining an example of a compressed surgical image. In this example, the first medical device non - existent interval video and the last medical device non - existent interval video are still images. Also, the image quality outside the surgical scene is suppressed.

Industrial Applicability

[0064] This invention can be used in the field of providing medical information.

Explanation of Signs

[0065] 1 Surgical video editing device 3 Surgical video input section 5 Tag setting section 7 Interval image quality adjustment section 9 Surgical scene extraction section 11 Image quality adjustment section for non-surgical motion pictures 13 Information addition section for the implemented surgical procedure 15 Extraction section for videos related to the implemented surgical procedure 17 Human specific part removal section 21 Medical device memory section 23 Hand information memory section 25 Surgical procedure memory section 27 Surgical target site memory section 29 Implemented surgical procedure memory section 31 Human specific part memory section

Claims

1. A surgical video input unit for inputting a surgical video; A tag installation unit that analyzes a medical device in use included in the surgical video and provides a tag related to the medical device in use to the surgical video based on the medical device in use; An interval video quality adjustment unit that adjusts the video quality of an interval video, which is a video of an interval delimited by the tag, among the surgical videos; comprising For the tagged surgical video, which is the surgical video provided with the tag, it is possible to cue from the portion where the tag is provided. The interval video quality adjustment unit adjusts the video quality of the interval video according to the medical device in use. An editing device for surgical videos.

2. The device according to claim 1, wherein the interval video quality adjustment unit If the interval delimited by the tag is defined as a tag interval, the interval video of the tag interval where the medical device in use is absent, which is the interval video among the interval videos, is set as a still image or as an interval video with a lower video quality than that of other interval videos.

3. The device according to claim 1, wherein using a learned model for surgical scenes, which is a learned model obtained by learning information related to surgical scenes in a past surgical video and the past surgical video as teacher data, and extracting surgical scenes from the surgical video; a non-surgical video quality adjustment unit that sets the non-surgical video, which is the surgical video other than the surgical scenes in the surgical video, as a still image or as an interval video with a lower video quality than that of other interval videos; further comprising

4. The device according to claim 1, wherein based on the surgical video, analyzing the actual surgical procedure, which is the surgical procedure of the surgical video, and adding information related to the actual surgical procedure to the surgical video; An apparatus further comprising an operation procedure-related video extraction unit that extracts an operation procedure-related video which is a part related to the actual operation procedure among the operation videos. Apparatus.

5. A computer, An operation video input unit to which an operation video is input, An analysis unit that analyzes the medical devices included in the operation video, and a tag setting unit that provides tags related to the medical devices used in the operation video based on the medical devices used, An interval video quality adjustment unit that adjusts the video quality of an interval video which is a video of an interval delimited by the tags among the operation videos, and having The tagged operation video, which is the operation video provided with the tags, can be fast-forwarded from the part where the tags are provided, The interval video quality adjustment unit adjusts the video quality of the interval video according to the medical devices used. A program for causing the computer to function as an editing device for operation videos.

6. A computer-readable non-transitory recording medium storing the program according to claim 5.

Images