Medical system, medical database creation method, data structure of medical content data, medical practice support method, and database

Abstract

This medical system comprises: a medical knowledge database 16 for storing records including data pertaining to a medical practice on a normalized table; and a medical video analysis unit 4 for analyzing a medical video including a medical procedure video in which a medical practice using an apparatus is recorded, and extracting data corresponding to a plurality of items defined in the medical knowledge database 16. The plurality of items include at least the medical procedure video and the apparatus. The medical video analysis unit 4 analyzes speech and images that constitute a medical video, extracts the medical procedure video to generate a digest video, and extracts the apparatus appearing in the medical procedure video to generate the name of the apparatus. The medical knowledge database 16 stores the generated digest video and the name of the apparatus in the same record.

Description

Medical system, medical database creation method, data structure of medical content data, medical practice support method, and database

[0001] In recent years, advances in IT technology have made it increasingly important to effectively utilize and utilize big data in the medical field. Effective use of big data has the effect of promoting the development and spread of diagnostic and treatment methods. Data shared across medical institutions is also expected to be useful in emergency response and the realization of comprehensive community care.

[0002] Currently, medical data related to patient diagnoses and treatments is often managed by individual medical institutions. This makes it difficult to utilize medical data, both for primary use for the patient's own diagnosis and treatment, and for secondary use by research institutions, companies, and government agencies for surveys, research, and policy planning. Furthermore, due to restrictions imposed by the Personal Information Protection Act and issues of privacy and rights for doctors and patients, the utilization of various data obtained during medical procedures has generally not progressed.

[0003] Meanwhile, with the current situation in which there are many different medical device manufacturers, each company is developing new devices and equipment every day with the cooperation and support of medical professionals. Diagnosis and treatment methods using these new devices and equipment are often reported and introduced in the form of medical report videos. These medical report videos are made up of content approved by the medical professionals and patients involved, and are a package that includes not only case studies but also introductions to the devices used and recorded footage of actual use.

[0004] In recent years, medical professionals have become extremely busy due to a shortage of medical professionals and an increasing elderly population. As a result, medical professionals have had the problem of not having the opportunity to learn about newly developed diagnostic, testing, and treatment methods, as well as the new equipment used in these methods. In other words, with the current situation in which it is difficult to distribute medical data and medical professionals are short on time and mental space, medical professionals have limited opportunities to even become aware of the existence of new procedures.

[0005] In light of this, systems have been developed to improve accessibility to medical information, including medical data, and reduce the burden on medical professionals. For example, Patent Document 1 proposes a system that presents medical data, such as information on instruments to be used, to support medical professionals in selecting instruments when performing surgery. Specifically, a first file storing instruments selectable for a surgical procedure is pre-registered in a computer or the like. When the medical professional selects a surgical procedure, the system searches the first file and displays a list of instruments selectable for the selected surgical procedure on a display device. When the medical professional selects the desired instruments from the displayed list of instruments, the selected instruments are registered in a second file.

[0006] Japanese Patent Application No. Hei-236082

[0007] However, the system described in Patent Document 1 only displays medical data related to pre-registered equipment for pre-planned surgical procedures. Therefore, it is unable to recommend any medical data related to new surgical procedures and procedures that are constantly being developed. Furthermore, it does not display recorded videos of surgical procedures (such as videos of the use of equipment during a procedure) that can deepen useful knowledge in a short amount of time, making it insufficient to support medical professionals with limited time.

[0008] Therefore, the present invention aims to provide a medical system, a medical database creation method, a data structure for medical content data, a medical procedure support method, and a database that can quickly recommend medical data, including surgical procedure videos, to medical professionals that are compatible with various surgical procedures, including new techniques.

[0009] A medical system according to one aspect of the present invention includes a medical database that stores records consisting of data related to medical procedures in a normalized table, and a medical video analysis unit that analyzes medical videos, including procedural videos that record medical procedures using equipment, and extracts data corresponding to multiple items defined in the medical database.

[0010] The plurality of items include at least the procedure video and the equipment. The medical video analysis unit analyzes the audio and images constituting the medical video, extracts the procedure video to generate first identification data, and extracts the equipment appearing in the procedure video to generate second identification data. Furthermore, the first identification data and the second identification data are stored in the same record in the medical database.

[0011] A medical database creation method according to one aspect of the present invention analyzes audio and images constituting medical videos, including procedural videos recording medical procedures using equipment, extracts the procedural videos to generate first identification data, extracts the equipment appearing in the procedural videos to generate second identification data, and stores the first identification data and the second identification data in the same record in a medical database that stores records containing data related to medical procedures in a normalized table.

[0012] According to the present invention, it is possible to quickly recommend to medical personnel medical data including surgical procedure videos that correspond to various surgical procedures including new procedures.

[0013] 1 is a block diagram illustrating an example of the configuration of a medical system according to the present embodiment. FIG. 1 is a block diagram illustrating an example of a detailed configuration related to database creation of medical videos in a medical system. FIG. 2 is a block diagram illustrating an example of a detailed configuration related to generation of medical data in a medical system. FIG. 3 is a diagram illustrating an example of a scene in which information on medical procedures is input. FIG. 4 is a diagram illustrating another example of a scene in which information on medical procedures is input. FIG. 5 is a diagram illustrating an example of a medical procedure information input screen. FIG. 6 is a diagram illustrating an example of a reference information display screen. A flowchart illustrating an example of the processing flow for creating a database from medical videos. A diagram illustrating an example of data registered in a medical knowledge database. A flowchart illustrating an example of processing in generating a surgical plan. An explanatory diagram illustrating learning using an inference model.

[0014] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0015] FIG. 1 is a block diagram illustrating an example of the configuration of a medical system according to this embodiment. The medical system according to this embodiment is mainly composed of a medical video analysis and organization unit 100 and a medical data generation unit 200. The medical video analysis and organization unit 100 analyzes medical report videos (hereinafter referred to as medical videos) and organizes them into a format that can be registered in a medical knowledge database 16. The medical data generation unit 200 uses data registered in the medical knowledge database 16 to generate medical data for medical professionals.

[0016] The medical video analysis and organization unit 100 includes a control unit 1, a medical video acquisition unit 2, a network 3, a medical video analysis unit 4, a database organization unit 5, and a machine learning unit 6. The medical data generation unit 200 includes a medical knowledge database 16, a plan creation unit 11, a medical procedure information input unit 12, a device control (setting) unit 13, a display unit 14, and an in-hospital system 15.

[0017] First, the configuration of the medical video analysis and organization unit 100 will be described with reference to Fig. 2. Fig. 2 is a block diagram for explaining an example of a detailed configuration related to creating a database of medical videos in a medical system. Fig. 2 is a diagram for explaining the detailed configuration of the medical video analysis unit 100.

[0018] The medical video acquisition unit 2 acquires medical videos from the network 3. Medical videos are primarily videos created by medical professionals introducing diagnostic and treatment methods using new devices and equipment. These are intended to be compiled and edited by doctors, medical facilities, and device manufacturers for use in reports and introductions. These medical videos include case introductions, introductions to the equipment used in diagnosis and treatment, and surgical procedure recording videos (video recordings of the equipment being used).

[0019] In addition, videos of medical procedures using specific devices or equipment may be recorded in medical device systems for evidence or reports. In this case, the video may not directly contain information about diagnostic or treatment methods, but if the medical video includes video segments capturing medical procedures on specific affected areas, it may be possible to obtain information about the equipment, procedures, etc. by analyzing the image frames.

[0020] In this case, if the progress leading up to the specific medical procedure is also recorded, it is possible to analyze the images and changes in the images to infer diagnostic and treatment methods. Furthermore, since the affected area, disease name, equipment, and device can be inferred from the images, these can also be used as medical videos. Medical videos of such specific medical procedures are routinely collected at medical institutions and can easily be used as training data after necessary processing. In other words, it is easy to create an inference model that infers the type of medical procedure these videos represent, and once the videos are available, they can easily be treated as videos representing diagnostic and treatment methods using specific devices and equipment. If necessary, it is also possible to reference the medical records of the cases at the time the videos were taken. Audio recordings of diagnoses, conversations before, during, and after surgery, recorded with microphones, and information such as text entered into the devices of relevant medical professionals, can also be used. Operating room systems may also attach this information to the videos as tag information.

[0021] The medical video acquired by the medical video acquisition unit 2 is output to the medical video analysis unit 4 .

[0022] The control unit 1 controls the operation of the medical video acquisition unit 2. The control unit 11 may be configured by a processor using a CPU (Central Processing Unit) or FPGA (Field Programmable Gate Array), may operate according to a program stored in a memory (not shown) to control each unit, or may realize some or all of its functions using a hardware electronic circuit.

[0023] The medical video analysis unit 4 analyzes the medical video input from the medical video acquisition unit 2 and extracts data. The medical video analysis unit 4 includes a video ID assignment unit 4a, a voice determination unit 4b, a text determination unit 4c, a device determination unit 4d, a device usage scene determination unit 4e, and a body part determination unit 4f.

[0024] The video ID assigning unit 4a detects surgical procedure record videos included in the medical video. An ID (video ID) is assigned to each detected surgical procedure record video to uniquely identify the individual video. The video ID desirably also includes information to identify the medical video that includes the surgical procedure record video. For example, if an identification number (report number) is assigned to the medical video, the video ID is assigned as "report number + subnumber." As a specific example, if a medical video with report number "xxxxx" includes three surgical procedure videos, video IDs of "xxxxx01," "xxxxx02," and "xxxxx03" are assigned to each surgical procedure video.

[0025] The audio determination unit 4 b analyzes the audio contained in the medical video and extracts data to be registered in the database organization unit 5 .

[0026] The database organizer 5 is intended to structure and record data. This structured data, for example, has the concepts of "columns" and "rows," and is organized and recorded by each item. Because it is literally "structured," it is easy to search, aggregate, and compare. It is the most suitable data structure for data analysis. It is easy for machines to organize and search, and for humans to check, making data management easy to explain and transparent. Medical videos of medical procedures on specific affected areas (medical videos retrieved from those recorded in a specific recording area, or videos entered) are analyzed and organized into a structured database according to multiple items for each video. The extracted data includes, for example, data corresponding to items such as "case," "affected area," "patient attributes," "difficulty," and "procedure time." The extracted items are not limited to these, and may also include items that medical professionals can use as criteria when adopting surgical procedures or equipment, such as "features," "benefits," "degree of burden on the patient," and "specialty of the equipment used." Data extraction may be performed by pattern matching using a pre-registered dictionary. For example, when extracting data corresponding to an "affected area," if a word registered in the dictionary as an "affected area," such as "lung," "heart," "airway," etc., is detected, the word is extracted as the "affected area." In this case, if multiple words representing the same "affected area" are registered, they may be converted into a representative word and extracted. For example, suppose two words, "lung" and "lung," are registered as words representing "lung," and "lung" is set as the representative word. In this case, if "lung" is detected from the voice data, "lung" is extracted as the "affected area." Data extraction may also be performed using AI (artificial intelligence) such as machine learning. Furthermore, data extraction may also be performed using a combination of multiple techniques, such as pattern matching and machine learning. In this embodiment, data corresponding to a "disease" is registered in the "case" field. Hereinafter, the "case" field may also be referred to as "disease" or "case (disease)."

[0027] The text determination unit 4c analyzes characters and sentences displayed in the medical video and extracts data to be registered in the database organization unit 5. The data to be extracted is the same as that of the voice determination unit 4b. As with the voice determination unit 4b, data extraction is performed using techniques such as pattern matching and machine learning.

[0028] The equipment determination unit 4d determines the equipment that appears in the medical video and extracts data (appearing instruments) to be registered in the database organization unit 5. The equipment that appears in the medical video is determined using techniques such as pattern matching and machine learning by the inference unit 6.

[0029] Here, rather than simply detecting the type of device reflected in the image, the presence or absence of a medical device acting on the specific affected area is detected by analyzing the medical video. That is, the image patterns of the target human body part (affected area) and the surrounding human tissue are determined from each frame of the video. If deformation, discoloration, or changes are observed in the part of the medical device acting on the tissue due to contact, proximity, or other non-contact action (such as an energy device outputting energy), this indicates that the device is performing treatment in accordance with a medical procedure. That is, the device in action can be determined based on the presence or absence of changes in the image portion corresponding to the affected area or the surrounding human tissue. Furthermore, since it is impossible to determine what type of treatment was performed without knowing the type of device, information indicating the type of medical device is searched for by image determination in the image frames included in the medical video and information related to the medical video (such as medical records or information entered by medical professionals).

[0030] The device determination unit 4d can then determine the portion of the medical video that uses the device for the demonstration of a procedure, based on the information contained in the medical video. This determination can then be used to create "procedure video timing" information. For example, it can determine the time range of the medical video in which the video portion capturing the procedure falls. This allows the beginning of the video to be cue-up or cut out, allowing it to be selected as a video clip candidate, which can then be used to create "procedure video timing" information.

[0031] The device usage scene determination unit 4e determines scenes in which the devices extracted by the device determination unit 4d appear in the surgical procedure video included in the medical video. Then, it extracts the timing at which the scene appears (e.g., the elapsed time from the start of the medical video). Scene determination is expected to be performed by image pattern determination, analysis of image information captured of the process leading up to the procedure, or AI such as machine learning. In addition to timing extraction, a digest video may be created. The digest video created using machine learning techniques is performed, for example, by the inference unit 6. The machine learning at this time is trained in a way that identifies similar scenes by referring to digested clip videos from many medical device introduction videos. At this time, input information from medical professionals associated with video recording may also be referenced.

[0032] The part determination unit 4f determines the part where the procedure is being performed in the surgical procedure recording video included in the medical video. The part determination may be performed using information contained in the video (text, image, or audio), or using techniques such as pattern matching or machine learning when images are used. The part may also be extracted using the determination results of the audio determination unit 4b or the text determination unit 4c.

[0033] In addition to videos edited by medical professionals and experts, videos of medical procedures using specific devices or equipment may also be recorded for evidence or reports. The medical video of this application can also be applied to videos obtained during such medical procedures in general. While these recorded videos may not directly contain information about diagnostic or treatment methods, they can be inferred from images of the affected area and equipment that appear in the images.

[0034] Furthermore, if the progress leading up to the medical procedure for a specific affected area is recorded, such information can also be useful. In other words, it is possible to infer diagnostic and treatment methods by analyzing the video images and image changes leading up to the affected area (until the affected area to be treated appears in the image). For example, in the case of an endoscope, image features change sequentially as the endoscope is inserted and removed from a lumen or body cavity until it reaches the affected area. However, once it reaches the affected area, the image changes become less during observation, allowing for confirmation and diagnosis of the affected area. In other words, a specific affected area can be identified by detecting changes in image frames in the medical video and the image features contained in the image frames. More specifically, by analyzing the image features contained in each frame when the image changes in each frame of the endoscopic video become less frequent, it is possible to identify the affected area based on the shape and color characteristics of polyps, inflammation, etc.

[0035] In this way, by analyzing medical videos taken during the process of inserting an endoscope into a body cavity or lumen, it is possible to detect information about specific affected areas from changes in the image frames contained in the medical video. Organizing this information so that it can be structured enables rapid utilization of the video. Once this affected area is detected, changes in the affected area and image characteristics of the equipment can be detected, searched, and organized in video clips demonstrating procedures using equipment at the specific affected area, making it easier to utilize the information. Information indicating the type of equipment (appearing instrument information) can be searched and organized from the image characteristics of the equipment captured in the video and other related information, and then stored in a database, allowing medical professionals to quickly understand and determine what equipment is needed to perform similar procedures.

[0036] In this way, information about specific affected areas (such as name and location) can be determined by detecting the process of access to the affected area. For example, it is possible to determine whether the surgery was an open, laparoscopic, or gastrointestinal endoscopic procedure. In the case of laparoscopy, information about where the hole was drilled and which internal organs were visible can also be obtained. Furthermore, image changes based on the image taken during insertion can provide information corresponding to specific affected areas, such as whether the affected area is located on the abdominal, posterior, right, or left side of the organ. Videos taken during such medical procedures are routinely collected, and because there are many cases, they are easy to use as training data. Therefore, it is easy to create an inference model that infers the type of medical procedure these videos represent. Once the videos are available, they can easily be used as videos representing diagnostic and treatment methods using specific devices or equipment. If necessary, the patient's medical records at the time the video was taken can be referenced. Audio recordings of diagnoses, conversations before, during, and after surgery, recorded with microphones, and text inputs, such as text, to the devices of the medical professionals involved can also be used. Operating room systems may also attach this information to the videos as tag information. The part determination unit 4f analyzes the medical procedure information contained in the recorded medical video and other information in the vicinity thereof to determine the part where the procedure is being performed. The presence or absence of a medical device acting on a specific affected area from the medical video can be detected by determining the part of the video where the medical procedure device acts on the specific affected area based on the presence or absence of changes in the image part corresponding to the specific affected area and the human tissue in its vicinity.

[0037] In addition to analyzing the body part, the device use scene determination unit 4e also analyzes the medical video and the information accompanying it to determine the scene in which the device extracted by the device determination unit 4d is displayed.

[0038] The inference unit 6 uses the training data input to the inference model to perform machine learning on the input data to the inference model and obtain output data. The inference unit 6 includes an inference model creation unit 6a, an inference model linking unit 6b, and an inference unit 6c. The inference model creation unit 6a determines a network design and generates an inference model so that expected output data can be obtained from a large amount of training data. The inference model linking unit 6b selects an inference model suitable for obtaining the output requested by the medical video analysis unit 4 and sets it in the inference unit 6c. The inference unit 6c inputs the input data from the medical video analysis unit 4 to the set inference model. Machine learning is performed using the training data input to the inference model to obtain output data. The output data is output to the medical video analysis unit 4.

[0039] Fig. 9 is an explanatory diagram for explaining learning using an inference model. The upper part of Fig. 9 is a diagram for explaining learning for extracting featured instruments from instrument-formatted medical videos. The lower part of Fig. 9 is a diagram for explaining learning for extracting digest videos of surgical procedure videos from medical videos.

[0040] As shown in the upper part of Figure 9, when extracting featured instruments, the input data is a medical video and the output data is the device name. A large number of various device images with device name data are input to the inference model as the first training data group. The inference model is set as a model (device inference model) with a determined network design so that device names can be extracted from medical videos.

[0041] As shown in the lower part of Figure 9, when extracting a digest video, the input data is a medical video and the output data is a digest video. A large number of various endoscopic videos of specific regions are input to the inference model as the second training data group. The inference model is set as a model (digest part inference model) with a determined network design so that a digest video can be extracted from the medical video.

[0042] In this way, the inference unit 6 uses an appropriate inference model and training data in response to the output requested by the medical video analysis unit 4 to generate output data from the input data in accordance with the request. The generated output data is output to the medical video analysis unit 4, which is the requestor.

[0043] Deep learning is a multilayered version of the machine learning process using neural networks. A typical example is a forward propagation neural network, which sends information from front to back and makes a judgment. In its simplest form, it requires three layers: an input layer consisting of N1 neurons, a hidden layer consisting of N2 neurons determined by parameters, and an output layer consisting of N3 neurons corresponding to the number of classes to be discriminated. The neurons in the input and hidden layers, and those in the hidden and output layers, are connected by connection weights, and a bias value is added between the hidden and output layers, making it easy to form logic gates. While three layers are sufficient for simple discrimination, increasing the number of hidden layers makes it possible to learn how to combine multiple features during the machine learning process. In recent years, models with 9 to 152 layers have become practical due to their training time, judgment accuracy, and energy consumption.

[0044] Various well-known networks may be used for machine learning. For example, R-CNN (Regions with CNN features) or FCN (Fully Convolutional Networks) using CNN (Convolution Neural Network) may be used. This involves a process called "convolution" that compresses image features, operates with minimal processing, and is strong in pattern recognition. Furthermore, "recurrent neural networks" (fully connected recurrent neural networks) that can handle more complex information and allow information analysis whose meaning changes depending on the order or sequence of information may be used.

[0045] To realize these technologies, conventional general-purpose arithmetic processing circuits such as CPUs and FPGAs can be used, but because much of the processing in neural networks involves matrix multiplication, GPUs and Tensor Processing Units (TPUs), which are specialized for matrix calculations, may also be used. In recent years, such dedicated artificial intelligence (AI) hardware, called "neural network processing units (NPUs)," have been designed to be integrated and embeddable with CPUs and other circuits, and may even become part of the processing circuit.

[0046] Furthermore, inference models may be obtained by employing various well-known machine learning techniques, not limited to deep learning. For example, techniques such as support vector machines and support vector regression are available. Here, learning involves calculating the weights, filter coefficients, and offsets of a classifier; other techniques include using logistic regression processing. When a machine is to make a judgment, a human must teach the machine how to make the judgment. In this embodiment, a method for deriving an image judgment using machine learning is employed. However, a rule-based method for applying rules acquired by humans through experience or heuristics to make a specific judgment may also be used.

[0047] The database organizer 5 organizes the data (medical content data) extracted by the medical video analyzer 4 into a format that can be registered in the medical knowledge database 16. That is, the data extracted from the medical video is normalized and structured, and the data is grouped by associating it with preset items. The following describes a case where seven items, "case," "affected area," "patient attributes," "video ID," "instruments appearing," "treatment video timing," and "memo information," are preset items.

[0048] "Case" is synonymous with "disease." For example, "cancer" is data corresponding to a "case." "Affected area" refers to the location of a disease or injury, or the area where a procedure is performed. For example, "lungs" is data corresponding to an "affected area." "Patient attributes" corresponds to data related to patient attributes, such as the patient's age, gender, nationality, medical history, allergies, etc. "Video ID" corresponds to the video ID assigned to each surgical procedure recording video by the video ID assigning unit 4a. "Appearing instruments" correspond to the equipment appearing in the surgical procedure recording video. "Processing video timing" corresponds to the timing extracted by the equipment usage scene determining unit 4e. "Memo information" classifies data extracted by the medical video analysis unit 4 that does not correspond to "case," "affected area," "patient attributes," "video ID," "appearing instruments," or "treatment video timing." For example, data corresponding to the "difficulty," "treatment time," "characteristics," "benefits," "degree of burden on the patient," and "degree of specialization of the equipment used" of a procedure, etc., is classified as "memo information." It also includes information on the process leading up to a specific affected area (specific affected area), and this information can be used to classify how the affected area was approached, as well as the more detailed location and characteristics of the affected area. The specific affected area can be classified and recorded with more detailed information, such as whether it was approached by laparotomy, or by a laparoscope inserted into a body cavity through a hole drilled in the body, or whether it was on the abdominal side of an organ or the back, making it possible to search for more similar cases. In addition to calling this memo information, it can also be called "approach information" or "access history information."

[0049] The data extracted by the medical video analysis unit 4 is grouped by "case." The organized data 5a and 5b correspond to the groups. For example, when the medical video analysis unit 4 extracts one "case" from one medical video, one organized data 5a (or organized data 5b) is generated in the database organization unit 5.

[0050] Each organized data set 5a or 5b groups data corresponding to the following categories: "Case," "Affected Area," "Patient Attributes," "Video ID," "Instruments," "Procedure Video Timing," and "Memo Information." If multiple surgical procedure recording videos are detected for a single "case" and "affected area" and multiple video IDs are assigned, a subgroup is generated for each "video ID." Each subgroup stores data corresponding to five categories: "Patient Attributes," "Video ID," "Instruments," "Procedure Video Timing," and "Memo Information." The organized data set 5a shows an example of the organized data structure when two surgical procedure recording videos are detected for a single "case" and "affected area." As described above, the present application analyzes medical videos and organizes their content information as structured data in a table format. Since similar items are organized in the rows and columns of the table, checking specific rows and columns allows for easy confirmation and management of the data structure and data content (information contained as items), ensuring reliability. For example, even a simple task such as sorting based on the content contained in specific items can be used to compare related items and determine whether there are any discrepancies in the content.

[0051] In addition, items (matters, elements) that are not included in each video are left blank, so it is possible to determine which videos are lacking and which videos can supplement, supplement, or replace them by comparing each item other than the blanks and looking for similar ones.

[0052] Furthermore, if multiple "affected areas" are detected for one "case," a subgroup is generated for each "affected area." That is, data corresponding to each of the following items are subgrouped: "affected area," "patient attributes," "video ID," "appearing instruments," "procedure video timing," and "memo information." If multiple surgical procedure recording videos are detected for one "affected area," a subgroup is further generated for each "video ID" as described above. That is, data corresponding to each of the following items are further subgrouped: "patient attributes," "video ID," "appearing instruments," "procedure video timing," and "memo information." Organized data 5b shows an example of an organized data structure when two "affected areas" are detected for one "case." Organized data 5b shows a case where one surgical procedure recording video is detected for one "affected area," and two surgical procedure recording videos are detected for the other "affected area." That is, one "case" group has two subgroups for each "affected area," and one of the subgroups is organized into a data structure with two subgroups for each "video ID."

[0053] Next, the configuration of the medical data generation unit 200 will be described with reference to Fig. 3. Fig. 3 is a block diagram for explaining an example of a detailed configuration related to the creation of medical data in a medical system. Fig. 3 is a diagram for explaining the detailed configuration of the medical data generation unit 200.

[0054] The medical knowledge database 16 is a database that centrally stores various medical data. The medical knowledge database 16 stores data on medical videos (medical video data) organized by the database organizer 5, and data (dictionary data) published in general medical dictionaries, etc. The medical knowledge database 16 also stores data on videos of surgical procedures (in-hospital medical video data), which is data on records of diagnoses, examinations, and treatments performed in the hospital (in-hospital data). The medical video data, dictionary data, and in-hospital medical video data are normalized and structured, and grouped by "case" and "affected area."

[0055] In addition to "case" and "affected area," the following items are set in the medical knowledge database 16: Specifically, the items in which medical video data is stored include "patient attributes," "video ID," "instruments used," "treatment video timing," and "memo information." Furthermore, the items in which dictionary data is stored include "onset symptoms," "subjective symptoms," "cause information," "treatment information," and "progression information." Four subitems are set for "cause information," namely, "genetics," "constitution," "lifestyle," and "other." Three subitems are set for "treatment information," namely, "base plan," "procedure," and "equipment." Multiple records can be registered for each of "treatment information" and "progression information." The items in which in-hospital medical video data is stored are the same as those for medical video data. It is desirable that new medical video data organized by the medical video analysis and organization unit 100 be registered in the medical knowledge database 16 without delay.

[0056] The plan creation unit 11, which serves as a provision data creation unit, creates medical data from data stored in the medical knowledge database 16 and the in-hospital system 15 based on information input by medical professionals. The medical data created by the plan creation unit 11 is recommended information and reference information for medical professionals. The plan creation unit 11 is used when considering medical procedures, such as reviewing a surgical plan. When reviewing a surgical plan, the recommended information is, for example, information about equipment recommended for use in surgery. Furthermore, the reference information is information about medical data including new surgical procedure videos.

[0057] The plan creation unit 11 comprises a medical procedure-related information extraction unit 11a, an equipment selection unit 11b, and a display control unit 11c. The medical procedure-related information extraction unit 11a extracts information useful for the medical procedure from the data stored in the medical knowledge database 16, based on information input from the medical procedure-related information input unit 12. The equipment selection unit 11b selects equipment recommended for use in the medical procedure from the data stored in the medical knowledge database 16, based on information input from the medical procedure-related information input unit 12.

[0058] The display control unit 11c displays the information extracted by the plan creation unit 11 and the information input from the medical treatment-related information input unit 12 on the display unit 14. In addition, based on the information input from the medical treatment-related information input unit 12, the display control unit 11c switches the screen to be displayed on the display unit 14.

[0059] The medical treatment information input unit 12 is a component for medical personnel to input information about planned medical treatments into the plan creation unit 11. The medical treatment information input unit 12 includes an audio input unit 12a and a text input unit 12b. The audio input unit 12a is, for example, a microphone. Audio data received via the audio input unit 12a is converted into character data and output to the plan creation unit 11. The text input unit 12b is, for example, a keyboard or a touch panel. The character data input via the text input unit 12b is output to the plan creation unit 11.

[0060] The device control (setting) unit 13 performs initial settings of the devices to be used in the medical procedure based on the medical procedure plan. The display unit 14 displays the input screen of the medical procedure information input unit 12 and an output screen containing recommended information and reference information created by the plan creation unit 11 under the control of the display control unit 11c.

[0061] The in-hospital system 15 holds various data (in-hospital data) related to records of diagnoses, examinations, and treatments performed in the hospital. For example, as patient data 151, data corresponding to each of the following items is held for each patient: "Patient ID," "Medical Visit History," "Prescription History," and "Doctor in Charge." The in-hospital system 15 also includes an equipment management unit 152 that manages equipment owned by the hospital. Furthermore, the in-hospital system 15 holds videos (in-hospital medical videos) related to surgical procedures performed in the hospital. Individual in-hospital medical videos 153a are collectively managed as a video group 153.

[0062] This centralized management recording unit does not necessarily have to be an in-hospital system, but as long as strict confidentiality can be maintained, such as the management of personal information by medical institutions, the location of the recording unit does not necessarily have to be limited to an in-hospital system.

[0063] Next, a specific scenario in which medical data is generated by the medical data generation unit 200 will be described. That is, a scenario in which medical procedures are supported by this system will be described. FIG. 4A is a diagram illustrating an example of a scenario in which information about a medical procedure is input. FIG. 4A shows a scenario in which a medical doctor, a medical professional, is alone considering a surgical plan. The doctor displays a medical procedure information input screen on a computer display (display unit 14). FIG. 5A is a diagram illustrating an example of the medical procedure information input screen. As shown in FIG. 5A, the medical procedure information input screen 14a has input / display sections for, for example, "patient information," "schedule information," "medical procedure type input," "equipment information," and "reference information." Input into each input / display section is performed using a keyboard (text input unit 12b) or the computer's built-in microphone (not shown).

[0064] "Patient Information" is the section where information about the patient who will be operated on is input / displayed. The "Patient Information" input / display section has two input / display boxes for "Patient ID" and "Findings." For example, if an item corresponding to "Findings" is registered in the patient data 151 stored in the in-hospital system 15, when the ID assigned to the patient who will be operated on is entered in the "Patient ID" box, the "Findings" data for that patient registered in the in-hospital system 15 is extracted and displayed in the "Findings" box on the medical treatment information input screen 14a. The "Findings" box may be editable as well as displayed.

[0065] "Schedule Information" is the section for inputting / displaying information about the surgery schedule. The "Schedule Information" input / display section has input / display boxes for two items: "Date and Time" and "Location." These boxes can also be displayed and edited.

[0066] "Medical procedure type input" is the section where information about the type of medical procedure is input / displayed. The input / display section of "Medical procedure type input" has two input / display boxes: "Subject" and "Examination / Treatment". The "Subject" box is where the subject of the surgery is input / displayed. For example, the "disease" and "affected area" that will be the subject of the surgery are input / displayed. The "Examination / Treatment" box is where the base plan for the surgery (procedure) is input / displayed.

[0067] "Equipment Information" is the section where information about the equipment used in surgery is input / displayed. For example, when data is input into the two boxes "Subject" and "Examination / Procedure" in "Medical Procedure Type Input," data is searched in the medical knowledge database 16 using "Case (Disease)," "Affected Area," and "Base Plan" as keys. When organized data (records) that match all key items are extracted, the data registered in "Treatment Information" - "Equipment" of that organized data is displayed in "Equipment Information" on the medical procedure information input screen 14a.

[0068] "Reference Information" is the section that displays recommended surgical procedures based on the data entered in "Medical Procedure Type Input." For example, if "Disease" or "Affected Area" is entered in the "Target" box, data is searched in the medical knowledge database 16 using "Case (Disease)" and "Affected Area" as keys. When organized data (records) that match all key items are extracted, information that can identify the video of the organized data (for example, "Video ID") is displayed in "Reference Information" on the medical procedure information input screen 14a.

[0069] As shown in FIG. 4A , the “Reference Information” display section may be provided with two display boxes, for example, “Time Priority” and “Difficulty Priority.” The “Time Priority” box further extracts and displays organized data extracted from the medical knowledge database 16, the organized data having a short surgical time registered in the “Memo Information” such as “Short Time.” Similarly, the “Difficulty Priority” box further extracts and displays organized data extracted from the medical knowledge database 16, the organized data having a low surgical difficulty registered in the “Memo Information” such as “Lowest Difficulty.” Note that the display boxes provided in the “Reference Information” display section are not limited to “Time Priority” and “Difficulty Priority.” They may also be other factors that medical professionals prioritize when creating a surgical plan.

[0070] When the video ID displayed in "Reference Information" is clicked, the display screen of the computer display (display unit 14) switches from the medical treatment information input screen 14a to the reference information display screen 14b. FIG. 5B is a diagram illustrating an example of the reference information display screen. As shown in FIG. 5B, the reference information display screen 14b has display sections for, for example, "Citation Source Information," "Advantages and Features," "Reference Video," and "Video Information."

[0071] "Source information" is the section that displays the source of the clicked video ID. For example, information that can identify the source, such as the report number of the medical video that includes the video, is displayed. "Advantages and features" is the section that displays the advantages and features of the surgical procedure in the video. "Reference video" displays the surgical procedure video (clip video) that corresponds to the video ID.

[0072] The "Video Information" display section has buttons labeled "Patient," "Procedure," and "Equipment." Clicking the "Patient" button displays detailed information about the patient in a pop-up screen or similar. Clicking the "Procedure" button displays detailed information about the procedure related to the surgical procedure in a pop-up screen or similar. Clicking the "Equipment" button displays detailed information about the equipment used in the surgical procedure video in a pop-up screen or similar. Note that the display of detailed information is not limited to a pop-up screen. For example, instead of the "Patient," "Procedure," and "Equipment" buttons, detailed information display sections for each may be provided.

[0073] In this way, by using the medical system of the embodiment when considering a surgical plan, the display unit quickly displays recommended equipment for the surgical procedure entered by the medical professional and other surgical procedures applicable to the disease or affected area of ​​the surgical target. Medical professionals can plan an appropriate surgical plan while taking into account this recommended information and reference information. FIG. 4B is a diagram illustrating another example of a situation in which medical procedure information is input. FIG. 4B illustrates a situation in which a surgical plan is being considered in a conference attended by multiple medical professionals. Even when multiple medical professionals are considering a surgical plan, useful medical data can be quickly acquired using the above-described medical procedure information input screen 14a, allowing for the planning of an appropriate surgical plan. In addition to the plan during the conference, assistance during surgery may be provided by searching actual surgical footage. Depending on the disease and its origin, it may be possible to predict relationships with other areas, such as metastasis to other areas. Taking this into consideration, recommended equipment and other surgical procedures applicable to the disease or affected area of ​​the surgical target may be displayed on the display unit. Medical professionals can plan an appropriate surgical plan while taking into account this recommended information and reference information.

[0074] Next, the operation of the medical system configured as described above will be described. First, a method for creating data to be registered in the medical knowledge database 16 will be described with reference to FIG. 6. FIG. 6 is a flowchart illustrating an example of the process flow for creating a database from medical videos. The series of steps shown in FIG. 6 are executed in the medical video analysis and organization unit 100.

[0075] First, the medical video acquisition unit 2 searches the network 3 for medical videos related to the specified "disease" and acquires medical videos that meet the conditions (S1).

[0076] These medical videos are assumed to be recorded on a server (not shown) managed by a specific site. This specific site is assumed to be a medical institution, medical research institute, medical service organization, medical device manufacturer, or the like, and preferably has its content updated regularly under the supervision of experts. This ensures reliability sufficient for use as a reference for medical procedures. The present application analyzes medical videos recorded in a specific storage area and organizes the content information as structured data in a table format. Quality control based on the reliability of such a specific site also ensures the reliability of the structured data. If medical videos are successfully acquired (S2, YES), data to be registered in the database organizer 5 is extracted and organized for each acquired medical video (hereinafter referred to as a candidate video) (S3-S10). If no medical videos matching the conditions are acquired, or if data organization for all acquired medical videos has been completed (S2, NO), the series of processes related to creating data to be registered in the medical knowledge database 16 is terminated.

[0077] The procedure for extracting and organizing data for each candidate video is as follows: First, the candidate video is analyzed by the medical video analysis unit 4, and data corresponding to the "affected area" field is searched for. If the data search is successful, one organized data 5a (5b) is added to the database organization unit 5. If one piece of data corresponding to the "affected area" is extracted, one organized data 5a is added. If two or more pieces of data corresponding to the "affected area" are extracted, one organized data 5b is added. The data specified in S1 is registered in the "disease" field of the added organized data 5a (5b), and the data extracted as a result of the search is registered in the "affected area" field (S3). The data search in S3 is performed by the voice determination unit 4b and the text determination unit 4c.

[0078] Next, it is determined whether or not information about the treatment instrument can be acquired from the candidate video (S4). The determination of whether or not the treatment instrument can be acquired is performed by the instrument determination unit 4d. The instrument determination may be performed by pattern matching using audio or images, or machine learning by the inference unit 6. If it is determined that the information can be acquired (S4, YES), data about the treatment instrument that appears in the candidate video is extracted (S5). If there is information about instruments (similar instruments) that can be used as substitutes for the treatment instrument extracted from the candidate video, or information about settings for using these instruments, this is added and registered in the "appearing instruments" field (S6). On the other hand, if it is determined that information about the treatment instrument cannot be acquired (S4, NO), S5 and S6 are skipped and the process proceeds to S7.

[0079] Next, it is determined whether the candidate video contains a surgical procedure video based on changes in the images in the candidate video. For example, if the continuity of multiple consecutive images in the candidate video is good, the consecutive images are determined to be surgical procedure videos. Note that the determination of whether the candidate video contains a surgical procedure video may be performed using machine learning by the inference unit 6.

[0080] Videos determined to be surgical procedure videos are assigned video IDs by the video ID assigning unit 4a. The assigned video IDs are registered in the "video ID" field of the organized data 5a (5b) added in S3. If multiple surgical procedure videos are extracted, subgroups equal to the number of extracted surgical procedure videos are added to the organized data 5a (5b) added in S3. The video IDs assigned to each surgical procedure video are registered in the "video ID" field of each subgroup. Then, it is determined whether the treatment instrument extracted in S5 appears in the extracted surgical procedure video (S7).

[0081] If it is determined that a treatment instrument will appear (S7, YES), the device usage scene determination unit 4e extracts a scene in which a procedure using the treatment instrument appears and acquires the timing of the scene. The treatment instrument data is then registered in the "Appearing Instrument" field of the subgroup in which the video ID of the surgical procedure video in which the treatment instrument appears is registered. The acquired timing data is also registered in the "Procedure Video Timing" field of the same subgroup (S8). On the other hand, if it is determined that a treatment instrument will not appear (S7, NO), S8 is skipped and the process proceeds to S9. This "Procedure Video Timing" information allows for quick retrieval and display of videos (managed by time, etc.) demonstrating the use of the treatment instrument, so that medical professionals can refer to them. It is also possible to record time information such as the time just before the instrument appears in the image or the time from when the instrument disappears from the image. Since it is sufficient for medical professionals to quickly check the scene, only time information for cueing is sufficient. Furthermore, in some cases, a specific scene in a video may already be edited as a "procedure video," and time information that allows for retrieval and display of that video may also be used.

[0082] Depending on the medical video, information for each item in the database other than this "treatment video timing" information may also be summarized in text or the like so that it can be displayed within the image.

[0083] Next, it is determined whether or not data (other information) that has not been extracted in the steps up to S7 and that is to be registered in the organized data 5a (5b) can be extracted from the candidate video (S9). Examples of the items determined in S9 include "patient attributes," "difficulty level," "treatment time," "features," "benefits," "degree of burden on the patient," and "degree of specialization of the equipment used." The items determined in S9 are set in advance in the voice determination unit 4b and the text determination unit 4c.

[0084] If it is determined that extraction of other information is possible (S9, YES), data is extracted from the candidate video and registered in the corresponding field (S10). For example, data about the patient is registered in the "patient attributes" field, and data about "difficulty level," "treatment time," "features," "benefits," "degree of burden on the patient," and "degree of specialization of the equipment used" is registered in the "memo information" field. Once the series of processes from S3 to S10 for one candidate video is completed and the registration of the organized data 5a (5b) is completed, the process returns to S2 and proceeds to processing the next candidate video. On the other hand, if it is determined that extraction of other information is not possible (S9, NO), the process skips S10 and returns to S2 and proceeds to processing the next candidate video.

[0085] As described above, by executing the series of steps shown in S1 to S10, data is organized, for example, in a table image as shown in FIG. 7 . FIG. 7 is a diagram illustrating an example of data registered in the medical knowledge database. As shown in FIG. 7 , information useful to medical professionals is extracted from the medical videos and organized into a normalized table having the following items: “Disease,” “Affected Area,” “Multimedia Content,” “Appearing Treatment Instrument,” “Procedure Video Timing,” and “Memo Information.” The items in FIG. 7 correspond to the items constituting the organized data 5 a as follows: “Disease” corresponds to “Case,” “Affected Area” corresponds to “Affected Area,” “Multimedia Content” corresponds to “Video ID,” “Appearing Treatment Instrument” corresponds to “Appearing Instrument,” “Procedure Video Timing” corresponds to “Procedure Video Timing,” and “Memo Information” corresponds to “Memo Information.” By extracting necessary data from the medical videos acquired from the network 3 and organizing it so that it can be registered as records (assuming rows in a tabular format) in a normalized table such as that shown in FIG. 7 , the medical video information can be registered in the medical knowledge database 16. In other words, it will be possible to quickly recommend to medical professionals medical data including surgical procedure videos that correspond to various surgical procedures, including new techniques.

[0086] Next, the procedure for a medical professional to create a surgical plan will be described with reference to Fig. 8. Fig. 8 is a flowchart illustrating an example of the processing for creating a surgical plan. The series of procedures shown in Fig. 8 are executed using the medical data generation unit 200.

[0087] First, the medical professional inputs information about the patient who will be undergoing surgery into the medical treatment information input screen 14a displayed on the display unit 14 (S11). For example, the patient ID of the patient is input into the "Patient ID" input / display box shown in FIG. 5A. Next, the medical professional inputs the diagnosis results of the patient into the medical treatment information input screen 14a (S12). For example, the diagnosis results are input into the "Findings" input / display box shown in FIG. 5A. If the diagnosis results are already displayed in the "Findings" box, S12 can be skipped.

[0088] Next, a surgical procedure candidate (base plan) is input into the medical procedure information input screen 14a (S13). For example, the surgical base plan (procedure) is input / displayed in the "Examination / Treatment" box of the "Medical Procedure Type Input" shown in Fig. 5A. If the base plan is already displayed in the "Examination / Treatment" box, S13 can be skipped.

[0089] Next, the plan creation unit 11 searches the medical knowledge database 16 based on the contents input / displayed in S11 to S13, and determines whether or not there is content (record) to be recommended (S14). In S14, for example, the medical knowledge database 16 is searched using "case (disease)," "affected area," and "base plan" as keys.

[0090] If organized data (records) matching all key items are extracted (S14, YES), the instruments recommended for use in the surgery are displayed based on the extracted organized data. For example, the data registered in the "Treatment Information"-"Equipment" section of the extracted organized data is displayed in the "Equipment Information" box on the medical procedure information input screen 14a shown in FIG. 5A (S15). Furthermore, in S15, reference information regarding the surgical procedure is also generated and displayed based on the extracted organized data. For example, the data registered in the "Video ID" section of the extracted organized data is displayed in the "Reference Information" box on the medical procedure information input screen 14a shown in FIG. 5A. In this case, the "Reference Index" in FIG. 8 is the "Video ID." Note that if multiple "Video IDs" are extracted for display as reference information, the data listed in the "Memo Information" section may be sorted based on "Operation Time" or "Difficulty Level" to further narrow down the "Video IDs" to be displayed. The extracted "Video ID" data may also be sorted and displayed according to a predetermined logic, such as by shortest operation time or lowest difficulty level.

[0091] Next, it is determined whether the equipment displayed as a recommended instrument in S14 is in stock (S16). That is, the plan creation unit 11 queries the owned equipment management unit 152 of the in-hospital system 15 to determine whether the equipment in question is in stock. If it is determined that the equipment in question is not in stock (S16, YES), similar equipment that can be substituted for the equipment in question is extracted from the organized data and displayed in the "Equipment Information" box (S17). On the other hand, if it is determined that the equipment in question is in stock (S16, NO), S17 is skipped and the display content of the "Equipment Information" box remains unchanged.

[0092] On the other hand, if it is determined that there is no content (record) to be recommended after searching the medical knowledge database 16 (S14, NO), the above-mentioned steps S15 to S17 are skipped. The medical professional selects a base plan and instruments to be used while referring to the contents displayed on the reference information display screen 14b, and updates the medical treatment information input screen 14a.

[0093] If the equipment displayed in the "Equipment Information" is to be used in the surgery (S18, YES), the plan creation unit 11 instructs the equipment control (setting) unit 13 to pre-configure the equipment (S19). For example, pre-configuration is performed based on the setting values ​​registered as the initial mode for use. Note that if setting information for the equipment is registered in the medical knowledge database 16, pre-configuration may be performed based on this setting information. On the other hand, if the equipment displayed in the "Equipment Information" is not to be used in the surgery (S18, NO), the process returns to S11, for example, and the surgical plan is created again.

[0094] Once the presetting of the equipment is complete and the surgical plan is finalized based on the contents displayed on the medical treatment information input screen 14a (YES in S20), the series of operations in Fig. 8 ends. On the other hand, if the surgical plan is to be further reviewed based on the contents displayed on the medical treatment information input screen 14a (NO in S20), the process returns to S11, for example, and the surgical plan is created again.

[0095] In this way, in the medical system of the embodiment, medical video information is normalized and structured and registered in the medical knowledge database 16. In other words, medical data corresponding to various surgical procedures, including new procedures, can be quickly recommended to medical professionals in a limited time, such as when creating a surgical plan. Furthermore, because the recommended medical data includes surgical procedure videos, it becomes easier for medical professionals to visualize new procedures and procedures, lowering the barrier to adoption. In other words, it becomes possible to perform appropriate diagnosis and treatment using appropriate equipment while reducing the burden on medical professionals.

[0096] In the above description, the videos to be registered in the medical knowledge database 16 are not limited to medical videos. For example, the in-hospital medical videos 153a registered in the in-hospital system 15 may also be organized into the structure of the organized data 5a (5b) and registered in the medical knowledge database 16 in the same way as the medical videos. By registering the in-hospital medical videos 153a in the medical knowledge database 16, it is possible to quickly provide feedback on the procedures, surgical procedures, and equipment used by other medical professionals in the hospital.

[0097] The present invention is not limited to the above-described embodiments, and the components can be modified and embodied in practice without departing from the spirit of the invention. Furthermore, various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiments. For example, some of the components shown in the embodiments may be omitted. Furthermore, components from different embodiments may be appropriately combined.

[0098] In addition, even if the operational flows in the claims, specifications, and drawings are described using "first," "next," etc. for convenience, this does not mean that they must be performed in that order. Furthermore, it goes without saying that the steps that make up these operational flows can be omitted as appropriate if they do not affect the essence of the invention.

[0099] Of the technologies described here, the controls mainly described in the flowcharts can often be set by a program, and may be stored on a recording medium or a recording unit. The method of recording on this recording medium or recording unit may be recording at the time of product shipment, using a distributed recording medium, or downloading via the Internet.

[0100] In the embodiments, the parts described as "parts" (sections or units) may be configured by combining dedicated circuits or multiple general-purpose circuits, or, if necessary, by combining a processor such as a microcomputer or CPU that operates according to pre-programmed software, or a sequencer such as an FPGA. It is also possible to design the device so that an external device takes over part or all of the control, in which case a wired or wireless communication circuit is involved. Communication may be via Bluetooth (registered trademark), Wi-Fi, a telephone line, or USB. The dedicated circuit, the general-purpose circuit, and the control unit may be integrated into an ASIC.

Claims

1. A medical system comprising: a medical database that stores records consisting of data related to medical procedures in a normalized table; and a medical video analysis unit that analyzes medical videos including procedural videos in which medical procedures using equipment are recorded, and extracts data corresponding to multiple items defined in the medical database, wherein the multiple items include at least the procedural videos and the equipment; the medical video analysis unit analyzes the audio and images that make up the medical videos, extracts the procedural videos to generate first identification data, and extracts the equipment that appears in the procedural videos to generate second identification data; and the first identification data and the second identification data are stored in the same record in the medical database.

2. The medical system according to claim 1, wherein the first identification data is a digest video of the procedure video, and the second identification data is the name of the device.

3. The medical system of claim 1, wherein the plurality of items further include a disease and an affected area, the medical video analysis unit analyzes the audio and images constituting the medical video and further generates third identification data corresponding to the disease and fourth identification data corresponding to the affected area, and the medical database stores the third identification data and the fourth identification data in the same record as the first identification data and the second identification data.

4. The medical system according to claim 1, wherein the medical video analysis unit generates the first identification data from the medical video using machine learning.

5. The medical system according to claim 1, wherein the medical video analysis unit generates the second identification data from the medical video using machine learning.

6. The medical system of claim 1, further comprising: a medical procedure information input unit into which information relating to a medical procedure is input; and a provision data creation unit that searches the medical database based on the information, extracts the record containing the information, and outputs at least one of the data corresponding to the procedure video contained in the extracted record and the data corresponding to the extracted equipment.

7. A medical database creation method comprising: analyzing audio and images constituting medical videos, including procedural videos in which medical procedures using equipment are recorded; extracting the procedural videos to generate first identification data; extracting the equipment appearing in the procedural videos to generate second identification data; and storing the first identification data and the second identification data in the same record in a medical database that stores records consisting of data related to medical procedures in a normalized table.

8. A medical database creation method that analyzes medical videos that include video segments of medical procedures on specific affected areas, and organizes the medical videos into a structured database according to multiple items, and detects from the analysis of the medical videos whether or not there is a medical device that acts on the specific affected area, and selects, from the information contained in the medical videos, a section where a procedure is performed using the medical device as a candidate for a clip video.

9. A medical database creation method as described in claim 8, wherein the presence or absence of the medical device acting on the specific affected area is detected from an analysis of the medical video by determining the video portion in which the medical device is acting on the specific affected area according to the presence or absence of changes in the image portion corresponding to the specific affected area or the human tissue in its vicinity.

10. A medical database creation method according to claim 8, further comprising acquiring treatment video timing information that enables the portion of the image frames constituting the video in which the procedure is performed to be played back as the clip video.

11. The medical database creation method according to claim 8, wherein information indicating the type of medical equipment is searched for from information included in the medical video.

12. A medical database creation method as described in claim 11, wherein information indicating the type of medical equipment is searched for using text information that is the audio corresponding to the medical video or the characters contained in the image frames that make up the medical video.

13. A medical database creation method according to claim 8, wherein the specific affected area is determined by detecting changes in image frames constituting the video and image features contained in the image frames.

14. A medical database creation method according to claim 8, wherein the position information of the specific affected area is determined by detecting changes in image frames constituting the video during the process of accessing the affected area.

15. A medical database creation method that analyzes medical videos that include video segments of medical procedures on specific affected areas, and organizes the medical videos into a structured database according to multiple items, the method detecting and organizing information related to the specific affected area from changes in image frames included in the medical videos, and searching for video clips of procedures performed using medical equipment for the position of the specific affected area included in the medical videos.

16. The medical database creation method according to claim 15, further comprising searching for information indicating the type of said medical device from among information contained in said medical video.

17. A data structure of medical content data used in a computer having a control unit, a memory unit, and a display unit, and stored in the memory unit, wherein the medical content data is created by analyzing a plurality of medical videos and organizing each video into items, and the items include affected area data relating to the affected area, clip video data of a portion of a procedure using a medical device for the position of a specific affected area, and device type data indicating the type of the medical device, and the data structure of medical content data makes it possible to search for the device type data included in the corresponding medical video and the clip video data of the portion of the procedure using the medical device as output according to input of the affected area data.

18. A medical procedure support method comprising: inputting information about an affected area corresponding to a medical procedure to be performed into a database in which the medical content data having the data structure described in claim 17 is stored; outputting search results based on the input, retrieving the device type data and the clip video data of the procedure using the medical device from the database; and playing the clip video data together with the device type data on the display unit.

19. The database used in the medical procedure support method described in claim 18, wherein the database is created by determining whether or not there is a change in the image portion corresponding to the specific affected area or the human tissue in the vicinity thereof for each of the plurality of analyzed medical videos, and detecting the medical device acting on the specific affected area.

20. The database according to claim 19, wherein playback start information for a moving image clip is obtained based on information about a moving image portion in which the medical device is acting on the specific affected area.

Images