Tagging device for medical treatment video

The treatment video tagging device addresses inefficiencies in existing systems by using machine learning to automatically tag and anonymize treatment videos, enhancing searchability and database compilation through automated medical device and technique identification, and ensuring patient and medical worker anonymity.

WO2024232333A1PCT designated stage expired Publication Date: 2024-11-14QUOTOMY INC
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2024/016861
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-10
Filing Date
2024-05-02
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

Existing video tagging devices for treatment videos require manual input of bookmark data and are inefficient in processing large volumes of videos, making them unsuitable for quick searching and compilation into databases.

Method used

A treatment video tagging device that analyzes medical equipment and processes within treatment videos to automatically add tags, allowing for easy searching and anonymization of individual images, utilizing machine learning to identify medical devices and techniques, and a person-specific part removal section to ensure anonymity.

Benefits of technology

Enables efficient tagging and anonymization of treatment videos, facilitating their compilation into databases and improving searchability by automatically associating medical devices and techniques with video content, while ensuring patient and medical worker anonymity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2024016861_14112024_PF_FP_ABST
    Figure JP2024016861_14112024_PF_FP_ABST
Patent Text Reader

Abstract

[Problem] To provide a device capable of automatically providing a tag to a medical treatment image. [Solution] A tagging device for a medical treatment video comprises: a medical treatment video input unit into which a medical treatment video is input; and a tag placement unit that analyzes a medical instrument for use, which is a medical instrument included in the medical treatment video, and provides, to the medical treatment video, a tag related to the medical instrument for use on the basis of the medical instrument for use. The tagged medical treatment video, which is a medical treatment video provided with the tag, can be cued from a portion provided with the tag.
Need to check novelty before this filing date? Find Prior Art

Description

Treatment video tagging device

[0001] The present invention relates to a tagging device for treatment videos.

[0002] Japanese Patent Application Laid-Open Publication No. 2006-43209 describes a surgical video recording device. This device stores name data for surgical procedures or instruments, selects and specifies a specific surgical procedure or instrument from an index title storage unit, and records an index image corresponding to the operation timing. This device can save recorded surgical procedures in a state that allows for quick search. This device is therefore useful because it makes surgical procedures searchable and can also display index images. However, this device requires that each video be manually searchable. As a result, this device cannot process a large number of surgical procedures.

[0003] Japanese Patent No. 5044069 describes a medical image diagnostic device. This device can obtain treatment videos associated with various types of bookmark data at any timing on a time scale. These bookmarks are markers for accessing the videos. This device requires a person to manually input bookmark data for each video. As a result, this device cannot process a large number of treatment videos.

[0004] JP 2006-43209 A Japanese Patent No. 5044069 A

[0005] The present invention aims to provide a device that can automatically tag treatment videos. Another aim of the present invention is to provide a device that can make image areas that identify individuals in treatment videos invisible.

[0006] This invention is based on the knowledge that by analyzing images of medical equipment included in a treatment video, tags can be automatically added to the treatment video based on the medical equipment. This invention is also based on the knowledge that it is preferable to make image parts that identify individuals invisible, because this makes it easier to create a database of treatment videos.

[0007] The treatment video tagging device 1 has a treatment video input unit 3 and a tag setting unit 5. The treatment video input unit 3 is an element for inputting treatment videos into the device 1. The tag setting unit 5 is an element for analyzing medical equipment included in the treatment video (referred to as "used medical equipment") and setting tags related to the used medical equipment in the treatment video based on the used medical equipment. Treatment videos with tags (also referred to as "tagged treatment videos") can be cue-played from the tagged part.

[0008] A preferred example of the tag setting unit 7 is one that can analyze the performed treatment steps, which are the treatment steps of the treatment video, based on the treatment video, and further set tags related to the performed treatment steps in the treatment video. Then, the treatment video with the tag set can be started from the part where the tag related to the performed treatment step is set, using the treatment step name related to the performed treatment step.

[0009] A preferred example of the tag setting unit 7 is one that can analyze the performed treatment technique, which is the treatment technique of the treatment video, based on the treatment video and add identification information related to the performed treatment technique to the treatment video. Then, the treatment video to which the identification information related to the performed treatment technique has been added can be searched using the treatment technique name related to the performed treatment technique.

[0010] A preferred example of the tag setting unit 7 is one that can further add link information for displaying information about the medical equipment used to the portion of the medical equipment used in the treatment video.

[0011] A preferred example of the treatment video tagging device 1 further includes a person-specific part removal unit 7. The person-specific part 7 is an element that analyzes person-specific parts, which are image parts that identify patients or medical professionals included in the treatment video, and, if a person-specific part is included in the treatment video, processes the treatment video so that the person-specific part is not visible.

[0012] Another invention described in this specification relates to a program for causing a computer to function as the above-mentioned treatment video tagging device, and a computer-readable non-transitory recording medium storing such a program.

[0013] According to this invention, by analyzing the images of medical equipment included in the treatment video, tags can be automatically added to the treatment video based on the medical equipment. In addition, a preferred embodiment of this invention makes it possible to make image parts that identify individuals invisible, making it easier to create a database of treatment videos.

[0014] Fig. 1 is a block diagram for explaining a tagging device for treatment videos. Fig. 2 is a flowchart showing an example of tagging treatment videos. Fig. 3 is a conceptual diagram showing an example of a treatment video to which tags have been added. Fig. 4 is a conceptual diagram showing an example in which information for accessing information about medical devices is added to a medical device portion in a treatment video.

[0015] The following describes embodiments of the present invention with reference to the drawings. The present invention is not limited to the embodiments described below, but also includes appropriate modifications of the embodiments that are obvious to those skilled in the art.

[0016] FIG. 1 is a block diagram for explaining a treatment video tagging device. As shown in FIG. 1, this treatment video tagging device 1 has a treatment video input unit 3 and a tag installation unit 5. This device 1 may further have a person-specific region removal unit 7. This device 1 may further have one or more of a medical device storage unit 11, a hand information storage unit 13, a treatment process storage unit 15, a treatment target region storage unit 17, a treatment technique storage unit 19, and a person-specific region storage unit 21. The treatment video tagging device 1 is implemented by a computer. In other words, each element constituting the treatment video tagging device 1 may be implemented by a computer.

[0017] A computer has an input unit, an output unit, a control unit, a calculation unit, and a memory unit, and each element is connected by a bus or the like to enable the exchange of information. For example, the memory unit may store a control program or various information. When predetermined information is input from the input unit, the control unit reads the control program stored in the memory unit. The control unit then reads the information stored in the memory unit as appropriate and transmits it to the calculation unit. The control unit also transmits the input information as appropriate to the calculation unit. The calculation unit performs calculation processing using the received various information and stores it in the memory unit. The control unit reads the calculation results stored in the memory unit and outputs them from the output unit. In this way, various processes and steps are performed. Each unit or means executes these various processes. A computer may have a processor, and the processor may realize various functions and steps. A computer may be standalone. A computer may have some of its functions distributed between a server and a terminal. In this case, it is preferable that the server and the terminal can exchange information via a network such as the Internet or an intranet. The computer may include a processor and a memory coupled to the processor. The memory may store instructions that, when executed by the processor, cause the computer to perform various processes or function as various elements. The computer may be provided with various training data to construct a learning model and perform various calculations through machine learning. In this case, the computer may perform various analyses using the learning model created through machine learning and deep learning of AI (artificial intelligence).

[0018] The computer's storage unit may store, for example, an analytical model (trained model) used when analyzing the video. The analytical model is preferably a model that has learned the medical equipment used in the surgery (which may include changes in the position of the equipment), the surgical procedure in which the medical equipment is used, and the surgical procedure steps. The analytical model may further be a model that has learned the hand or arm positions (which may include changes in the position of the hand or arm) included in the surgical procedure video, and the surgical procedure and surgical procedure corresponding to the hand or arm positions. The computer preferably analyzes surgical procedure videos filmed from surgical procedures and performs at least one of identifying the surgical procedure and classifying the surgical procedure steps included in the surgical procedure video data. The computer may classify the surgical procedure steps from the surgical video based on the medical equipment used in the surgery and the position of the medical equipment (which may include changes in position). Furthermore, the computer may classify the surgical procedure steps based on changes in the position of the hand or arm handling the medical equipment in addition to the medical equipment used in the surgery. Furthermore, since it is possible to identify a surgical procedure from a combination of surgical procedures including the order, the computer may identify the surgical procedure based on the combination of steps including the classified order. In this case, the surgical procedure may be identified by referring to information associating a combination of steps, including their order, with the surgical procedure. In this way, the computer may learn various learning models by machine learning and perform various processes using the learned models.

[0019] The treatment video tagging device 1 is a device for assigning tags to treatment videos to allow viewing of the treatment video from a specific point in the treatment video. The treatment video may be a video related to surgery or treatment. An example of a surgery is a surgical procedure. Examples of treatments include esthetics and massage. A tag, also called an index or bookmark, is information attached to a video to allow viewing of the video from the tagged portion. It is preferable that the device 1 allows searching of treatment videos using tags. For example, a video can be viewed from the point at which a certain medical device is used in a certain step of a certain treatment. The tag is preferably stored in a memory unit in association with the name of the medical device used in the treatment video. The name of the medical device stored in association with the tag may be displayed together with the tag portion when the treatment video is displayed. A tagged treatment video can be cue-searched from the point at which the tag related to the medical device is attached using the name of the medical device used.

[0020] The treatment video input unit 3 is an element into which treatment videos are input. The treatment video tagging device 1 is connected to a shooting unit (e.g., a camera or video camera) that shoots videos, and once a treatment video is shot, the treatment video may be input into the treatment video tagging device 1. Also, video files related to treatment videos may be input into the treatment video tagging device 1 by drag and drop. Furthermore, the treatment video tagging device 1 may search for treatment videos on the Internet and automatically input the treatment videos that are found. In this way, treatment videos are input into the treatment video tagging device 1.

[0021] The tag installation unit 5 is an element for analyzing the medical equipment included in the treatment video and providing tags related to the medical equipment in the treatment video based on the medical equipment. The device 1 further includes a medical equipment storage unit 11. The medical equipment storage unit 11 stores the names of the medical equipment and image data of the medical equipment in association with each other. The medical equipment storage unit 11 may also store the IDs of the medical equipment. For example, the tag installation unit 5 may perform machine learning on image data related to various medical equipment, create a learning model related to the medical equipment, and store the model in the storage unit. The tag installation unit 5 may then analyze the medical equipment included in the treatment video using the learning model related to the medical equipment. For example, the tag installation unit 5 may perform machine learning on video data related to various treatment steps and various treatments, create a learning model related to the various treatment steps and various treatments, and store the model in the storage unit. The tag installation unit 5 may then use the learning model related to the various treatment steps and various treatments to analyze each treatment step in the treatment video and the treatment performed in the treatment video.

[0022] The tag installation unit 5, for example, reads out images (treatment images) included in the treatment video from the storage unit and reads out image data of the medical devices from the medical device storage unit 11. Then, the tag installation unit 5 compares the treatment image with the image data of the first medical device. If the tag installation unit 5 determines that the treatment image includes image data of the first medical device, it determines that the first medical device is the used medical device. If, as a result of the comparison process performed by the tag installation unit 5 using the treatment image and the image data of the first medical device, the tag installation unit 5 determines that the treatment image does not include the first medical device, the tag installation unit 5 reads out image data of the next medical device, the second medical device, from the medical device storage unit 11. Then, the tag installation unit 5 repeats the comparison process between this treatment image and the image data of the medical device. In this way, the tag installation unit 5 can identify the medical devices in the images included in the treatment video. In other words, the tag installation unit 5 can identify the medical devices used at a certain timing in the treatment video.

[0023] A preferred example of the tag installation unit 5 is one that performs image analysis of the practitioner's hand included in the treatment video (treatment image) and identifies the medical device held by the practitioner's hand as the medical device in use. For example, the treatment video may include medical devices not used in the treatment. In this example, even in such cases, tags can be assigned to medical devices actually used in the treatment. In this example, the treatment video tagging device 1 may include a hand information storage unit 13. Then, the tag installation unit 5, for example, reads images (treatment images) included in the treatment video from the storage unit and reads image data of the hand from the hand information storage unit 13. Then, the tag installation unit 5 compares the treatment image with the image data of the hand. In this way, the practitioner's hand can be identified in the image (treatment image) included in the treatment video. If the treatment video (treatment image) includes multiple medical devices, the tag installation unit 5 may identify the medical device closest to the practitioner's hand in the treatment video (treatment image) as the medical device in use. Furthermore, when the hand information storage unit 13 stores image data of a hand holding a medical device, the medical device analyzed as being held in the hand in that state may be determined as the medical device in use.

[0024] A preferred example of the tag installation unit 5 is one that performs image analysis of the practitioner's hand and the assistant's hand included in the treatment video (treatment image), and identifies the medical device held by the practitioner's hand as the medical device in use. The tag installation unit 5, for example, reads an image (treatment image) included in the treatment video from the storage unit and reads image data of the hand from the hand information storage unit 13. The tag installation unit 5 then compares the treatment image with the image data of the hand. When multiple hands are included in the treatment image, the tag installation unit 5 may analyze which hand is the practitioner's hand using information about the position of the hand in the treatment image and information about medical devices near the hand in the treatment image. For example, the hand information storage unit 13 stores image data of the right hand and image data of the left hand. The tag installation unit 5 then uses the image data of the right hand and the image data of the left hand to analyze whether the hand included in the treatment image is the right hand or the left hand. Then, for example, the tag installation unit 5 may determine that, among multiple pairs of hands, the pair closest to the center of the image is the practitioner's hand (pair of hands). Furthermore, the pair that is farthest from the center of the screen may be determined to be the assistant's hand (pair).The tag installation unit 5 may then analyze medical devices that are close to the practitioner's hand included in the treatment image or that are image-analyzed as being held by the practitioner's hand as medical devices used by the practitioner, and store this in the storage unit.The tag installation unit 5 may also analyze medical devices that are close to the assistant's hand included in the treatment image or that are image-analyzed as being held by the assistant's hand as medical devices used by the assistant, and store this in the storage unit.

[0025] A preferred example of the tag setting unit 7 is one that can analyze the performed treatment technique, which is the treatment technique of the treatment video, based on the treatment video and add identification information related to the performed treatment technique to the treatment video. Then, the treatment video to which the identification information related to the performed treatment technique has been added can be searched using the treatment technique name related to the performed treatment technique.

[0026] The term "procedure step" refers to each step in a procedure. Examples of the procedure step include exposure (e.g., lumbar exposure), fixation, screw insertion, decompression, disc manipulation, and closure. The procedure step is not limited to these examples and may refer to various steps in various surgeries or procedures.

[0027] The tag installation unit 5 may analyze the treatment process based on, for example, the type of medical equipment included in the treatment video (treatment image). The tag installation unit 5 may analyze the treatment process based on, for example, the movement of the medical equipment included in the treatment video (treatment image). The tag installation unit 5 may analyze the treatment process based on, for example, the type of medical equipment and the treatment target area included in the treatment video (treatment image). The tag installation unit 5 may analyze the treatment process from, for example, information on the medical equipment used by the practitioner and the medical equipment used by the assistant among the medical equipment included in the treatment video (treatment image).

[0028] The treatment step storage unit 15 stores treatment steps in association with medical devices. The tag installation unit 5 reads information about the medical devices used from the storage unit, and reads the treatment steps stored in association with the medical devices used from the treatment step storage unit 15. In this way, the tag installation unit 5 reads treatment steps using information necessary for analyzing treatment steps from the treatment step storage unit 15, which stores information necessary for analyzing treatment steps. For example, assume that the treatment step storage unit 15 stores treatment steps in association with the medical devices used by the practitioner and the medical devices used by the assistant. In this case, the tag installation unit 5 reads the medical devices used by the practitioner and the medical devices used by the assistant from the storage unit, and reads the treatment steps stored in the treatment step storage unit 15 in association with this information. The read treatment steps may be stored in the storage unit as appropriate.

[0029] Table 1 is a conceptual diagram for explaining the treatment step storage unit. As shown in Table 1, the treatment step storage unit 15 stores treatment steps in association with medical devices. Also, as shown in Table 1, the treatment step storage unit 15 may store treatment steps in association with medical devices and target areas. In this manner, the device 1 can read out the treatment steps to be performed from the treatment step storage unit 15 using information about the medical devices. For example, in the example of Table 1, if the medical devices used are A1, A2, A3, and A4, A may be read out as the treatment step to be performed. Also, in the example of Table 1, if the medical devices used are A1, A2, A3, and A4 and the target area is A5, A may be read out as the treatment step to be performed.

[0030]

[0031] The treatment target area storage unit 17 stores, for example, a target area during treatment and an image of the target area in association with each other. The tag installation unit 5 may identify the treatment target area included in the treatment video (treatment image) by, for example, comparing an image included in the treatment video (treatment image) with an image of the target area stored in the treatment target area storage unit 17. Examples of treatment target areas are the femur, clavicle, skull, large intestine, small intestine, and eye. Such a comparison process can be performed by the tag installation unit 5 using machine learning.

[0032] A preferred example of the tag setting unit 7 is one that can analyze the performed treatment technique, which is the treatment technique of the treatment video, based on the treatment video and further set a tag related to the performed treatment technique in the treatment video. Then, the treatment video with the tag set can be searched using the treatment technique name related to the performed treatment step. The treatment technique storage unit 19 may store treatment techniques in association with multiple treatment steps. Furthermore, the treatment technique storage unit 19 may store treatment techniques in association with multiple treatment steps and multiple medical devices. The tag setting unit 7 may read information related to the treatment steps and multiple medical devices related to the treatment video from the treatment technique storage unit 19, compare it with the treatment technique storage unit 19, and read the treatment techniques stored in association with these treatment steps and medical devices. The read treatment techniques may be stored in the storage unit in association with the treatment video.

[0033] Table 2 is a conceptual diagram for explaining the treatment technique storage unit. As shown in Table 2, the treatment technique storage unit 19 stores treatment techniques in association with treatment steps and medical devices. Also, as shown in Table 2, the treatment technique storage unit 19 may store treatment techniques in association with treatment steps, medical devices, and target areas. In this manner, the device 1 can read out the treatment technique to be performed from the treatment technique storage unit 19 using the necessary information. For example, in the example of Table 2, if the treatment steps are B1, B2, B3, and B4 and the medical devices used are B5, B6, B7, and B8, B may be read out as the treatment technique to be performed. Also, in the example of Table 2, if the treatment steps are B1, B2, B3, and B4, the medical devices used are B5, B6, B7, and B8, and the target area is B9, B may be read out as the treatment technique to be performed.

[0034]

[0035] A preferred example of the tag setting unit 7 is one that can further add link information for displaying information about the medical equipment used to the medical equipment portion of the treatment video. The treatment video tagging device 1 stores, for example, in a memory unit, explanatory text, explanatory images, explanatory materials, website access information (URL), and purchase information related to the medical equipment, in association with the medical equipment. The treatment video tagging device 1 then identifies the medical equipment portion of the treatment image in the treatment video. When the medical equipment portion is selected using a touch panel or a mouse, an input indicating the medical equipment used is made to the treatment video tagging device 1. The treatment video tagging device 1 then reads the link information related to the medical equipment stored in the memory unit, reads data related to the link information, and enables access to a website related to the link information. Information about the medical equipment used is then displayed on the user's display. In this way, for example, a user viewing a treatment video can receive information about the medical equipment actually used and immediately purchase the medical equipment.

[0036] A preferred example of the treatment video tagging device 1 further includes a person-specific part removal unit 7. The person-specific part 7 is an element that analyzes person-specific parts, which are image parts that identify patients or medical professionals included in the treatment video, and, if a person-specific part is included in the treatment video, processes the treatment video so that the person-specific part is not visible. By further including the person-specific part removal unit 7, if a treatment video (treatment image) includes a patient's face, tattoos, a medical professional's face, a nameplate, a hospital name, etc., these can be masked or mosaicked to ensure anonymity. In this way, a large number of videos can be databased while enhancing anonymity. Examples of person-specific parts include a patient's face, tattoos, a medical professional's face, a nameplate, and a hospital name.

[0037] For example, the human specific part removal unit 7 constructs a learning model and performs machine learning using past treatment videos, examples of identifying human specific parts in treatment videos, and examples of treatment videos that have been edited to make the human specific parts invisible as training data.The human specific part removal unit 7 then uses the learning model obtained by machine learning to identify human specific parts in the input treatment video and edit the treatment video to make the human specific parts invisible.The treatment video tagging device 1 simply stores the edited treatment video in a storage unit as appropriate.

[0038] The human specific part storage unit 21 may store a learning model for identifying the human specific parts in the treatment video described above. The human specific part storage unit 21 may also store image data related to the human specific parts. In this case, the human specific part removal unit 7 may compare the input treatment video (treatment image) with image data related to the human specific parts stored in the human specific part storage unit 21, and if the input treatment video (treatment image) contains an image portion related to the human specific part, identify that portion as the human specific part. The human specific part removal unit 7 may then perform image processing on the image portion related to the human specific part included in the treatment video (treatment image) so that it cannot be viewed by other parties. Examples of image processing include applying a mosaic, blurring, or overwriting a mask image on the image portion.

[0039] The treatment video tagging device 1 may create a database containing information such as the treatment procedure, treatment steps, and medical equipment used for the tagged treatment videos, making it possible to search for treatment videos using this information. In this way, the medical equipment used can be indexed, making it easy to watch the required treatment video.

[0040] Another invention described in this specification relates to a program for causing a computer to function as the above-mentioned treatment video tagging device, and a computer-readable non-transitory recording medium storing such a program. Examples of recording media include a hard disk, a CD, a CD-ROM, a floppy disk, a Blu-ray disk, a USB memory, and an SD card.

[0041] The computer may include a processor and a memory coupled to the processor. The memory stores instructions (programs), which, when executed by the processor, cause the computer to input a treatment video, analyze the medical equipment used, which is the medical equipment included in the treatment video, and add tags related to the medical equipment used to the treatment video based on the medical equipment used. The program may also cause the computer to analyze the performed treatment steps, which are the treatment steps in the treatment video, based on the treatment video, and further add tags related to the performed treatment steps to the treatment video. The program may also cause the computer to analyze the performed treatment procedure, which is the treatment technique in the treatment video, and further add identification information related to the performed procedure to the treatment video. The program may also cause the computer to analyze person-specific features, which are image features that identify patients or medical professionals included in the treatment video, and, if the person-specific features are included in the treatment video, process the treatment video so that the person-specific features are not visible.

[0042] 2 is a flowchart showing an example of tagging a treatment video. In the figure, S represents a step (process).

[0043] Video Input Step (S101) A treatment video is input from the treatment video input unit 3 to the treatment video tagging device 1.

[0044] Used Medical Device Analysis Step (S102) The tag installation unit 7 analyzes whether each treatment image included in the treatment video contains an image portion related to a medical device. At this time, the tag installation unit 7 may refer to a learning model that has undergone machine learning on medical devices. Furthermore, the tag installation unit 7 may analyze the presence or absence of a medical device included in the treatment video by analyzing whether the treatment image contains image data related to the medical device using image data related to the medical device. If the analysis shows that the treatment image does not contain a medical device, information indicating that a medical device is not present may be stored in the storage unit together with the treatment image. Furthermore, if the tag installation unit 7 analyzes that the treatment image contains a medical device, information regarding the used medical device, such as the name of the used medical device, may be stored in the storage unit together with the treatment image (and the time in the treatment video of the treatment image).

[0045] Tagging process for medical equipment used (S103) When the tag setting unit 7 determines that a new medical equipment is included in the treatment video (when a certain medical equipment is included for the first time in a treatment image), a tag can be set along with the identification information of the treatment image, such as the broadcast time of the treatment image. Specifically, the tag setting unit 7 associates the tag with the identification information of the treatment image and stores it in the memory unit. Information about the medical equipment used may be stored in association with the tag. In this way, the tag setting unit 7 can obtain a treatment video with a tag (referred to as a "tagged treatment video").

[0046] Performed Treatment Step Analysis Step (S104) The tag setting unit 7 analyzes the performed treatment steps, which are the treatment steps of the treatment video, based on the treatment video, and further provides tags related to the performed treatment steps in the treatment video. At this time, the tag setting unit 7 may refer to a learning model that has been machine-learned regarding the treatment steps. Furthermore, the tag setting unit 7 may analyze the treatment steps by referring to the information stored in the treatment step storage unit 15 or the treatment target area storage unit 17, using information related to the medical equipment or target area included in the treatment video (treatment image).

[0047] Step of tagging the performed treatment step (S105) When the tag setting unit 7 determines that a new performed treatment step is included in the treatment video (when a treatment step is included for the first time in a treatment image), a tag can be set along with identification information of the treatment image, such as the broadcast time of the treatment image. Information related to the performed treatment step may be stored in association with the tag. In this way, the tag setting unit 7 can obtain a treatment video to which a tag related to the performed treatment step has been set.

[0048] Performed Treatment Technique Analysis Step (S106) The tag installation unit 7 analyzes the performed treatment technique, which is the treatment technique of the treatment video, based on the treatment video, and adds identification information related to the performed treatment technique to the treatment video. At this time, the tag installation unit 7 may refer to a learning model that has been machine-learned regarding the treatment technique. Furthermore, the tag installation unit 7 may use information related to the performed treatment process to read out the performed treatment technique stored in the performed treatment technique storage unit 19 in association with the performed treatment process.

[0049] The device 1 (tag installation unit 7) may add identification information related to the performed treatment technique to the treatment video and store it in the storage unit. This identification information related to the performed treatment technique is also searchable information, so it may be treated as a type of tag.

[0050] Human specific part determination step (S108) The human specific part removal unit 7 uses a learning model obtained by machine learning to identify human specific parts in the input treatment video (treatment image). If the human specific part removal unit 7 determines that the treatment image does not contain a human specific part, it does not remove the human specific part. On the other hand, if the human specific part removal unit 7 determines that the treatment image contains a human specific part, it removes the human specific part.

[0051]

[0044] If the human specific part removal unit 7 determines that the treatment image contains a human specific part, it processes the treatment image so that the human specific part is not visible in the treatment image. For example, the human specific part removal unit 7 may perform processing to turn the human specific part into a mask image. In this way, the human specific part is processed so that it is not visible in the treatment video.

[0052] FIG. 3 is a conceptual diagram showing an example of a treatment video with tags. In this example, tags are displayed at specific times in the treatment video. By clicking on the tag, the treatment video can be viewed from the tagged position. The medical equipment used may also be displayed in association with the tag. In the example of FIG. 3, representative treatment images for each treatment process are displayed with their names (displayed small at the bottom of the video).

[0053] FIG. 4 is a conceptual diagram showing an example in which information for accessing information about a medical device is added to a medical device portion in a treatment video.

[0054] The present invention can be used in the field of providing medical information.

[0055] REFERENCE SIGNS LIST 1 Treatment video tagging device 3 Treatment video input unit 5 Tag installation unit 7 Person specific part removal unit 11 Medical equipment storage unit 13 Hand information storage unit 15 Treatment process storage unit 17 Treatment target part storage unit 19 Performed treatment procedure storage unit 21 Person specific part storage unit

Claims

1. A treatment video tagging device comprising: a treatment video input unit into which a treatment video is input; and a tag setting unit that analyzes medical equipment included in the treatment video and sets a tag related to the medical equipment in the treatment video based on the medical equipment, wherein the tagged treatment video, which is the treatment video that has been tagged, can be cue-played from the part where the tag is set.

2. The device as described in claim 1, further comprising a human-specific part removal unit that analyzes human-specific parts, which are image parts that identify patients or medical professionals included in the treatment video, and, if the treatment video contains a human-specific part, processes the treatment video so that the human-specific part is not visible.

3. An apparatus as described in claim 1, wherein the tag setting unit is capable of analyzing an implemented treatment step, which is a treatment step of the treatment video, based on the treatment video, and further providing a tag relating to the implemented treatment step in the treatment video.

4. An apparatus as described in claim 1, wherein the tag installation unit is capable of analyzing an performed treatment technique, which is the treatment technique of the treatment video, based on the treatment video, and further adding identification information regarding the performed treatment technique to the treatment video.

5. A device according to any one of claims 1 to 4, wherein the tag setting unit is further capable of adding link information for displaying information about the medical equipment used to the portion of the medical equipment used in the treatment video.

6. A program for causing a computer to function as a treatment video tagging device having a treatment video input unit into which a treatment video is input, and a tagging unit that analyzes medical equipment used, which is medical equipment included in the treatment video, and assigns tags related to the medical equipment used to the treatment video based on the medical equipment used, so that a tagged treatment video, which is a treatment video that has been assigned a tag, can be cue-played from the part where the tag is assigned.

7. A non-transitory computer-readable recording medium storing the program according to claim 6.

Citation Information

Patent Citations

  • Systems and methods for analysis of surgical videos

    JP2022520701A

  • Information processing system, information processing method, and program

    WO2023002661A1