Method and electronic device for generating representative frame images of medical images

Abstract

To generate a frame image that can be representative of video of medical care executed by at least one processor.SOLUTION: A method for generating medical care video representing frame image includes the steps for: acquiring a medical care video in which blood vessels are photographed; calculating a score for each a plurality of frame images contained in medical care video; and generating medical care video representing frame images from a plurality of frame images on the basis of the score for each of the plurality of frame images.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present disclosure relates to a method for generating a representative frame image of medical video and an electronic device.

Background Art

[0002] In the medical field, lesions of a subject (e.g., a patient) to be imaged can be diagnosed by analyzing medical videos obtained using X-ray, CT (Computed Tomography) imaging, angiography, etc. Conventionally, mainly reading experts (e.g., medical staff) directly analyze medical videos to diagnose lesions. Recently, however, methods of automatically analyzing medical videos using machine learning models and the like have been used, and technical development for this is also actively underway.

[0003] On the other hand, the method of analyzing medical videos through existing machine learning models may include a stage where a frame image most suitable for the medical video (e.g., a frame image in which the target blood vessel and the surroundings are most clearly distinguished) is selected and a stage where the selected frame image is analyzed (e.g., blood vessel classification). However, if a frame image not suitable for video analysis is selected at the stage where a frame image is selected from the medical video, incorrect results may be derived at the subsequent frame image analysis stage. Along with this, in the method of automatically analyzing medical videos, technological development for generating a frame image that can represent the medical video is required.

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The present disclosure provides a method for generating a representative frame image of medical video and an electronic device for solving the above problems.

Means for Solving the Problems

[0006] This disclosure can be embodied in a variety of ways, including methods, apparatus (systems), and / or computer programs.

[0007] According to one embodiment of the present disclosure, a method for generating a representative frame image of a medical image, performed by at least one processor, may include the steps of: acquiring a medical image in which blood vessels are captured; calculating a score for each of a plurality of frame images contained in the medical image; and generating a representative frame image of the medical image from the plurality of frame images based on the scores for each of the plurality of frame images.

[0008] According to one embodiment of the present disclosure, a computer program may be provided for executing the aforementioned method for generating representative frame images of medical images on a computer.

[0009] According to one embodiment of the present disclosure, the electronic device includes a memory and at least one processor connected to the memory and configured to execute at least one computer-readable program contained in the memory, the at least one program may include instructions for acquiring a medical image of blood vessels, calculating a score for each of a plurality of frame images contained in the medical image, and generating a representative frame image of the medical image from the plurality of frame images based on the score for each of the plurality of frame images. [Effects of the Invention]

[0010] According to some embodiments of this disclosure, a representative frame image of the medical video can be generated from multiple frame images based on a score for each of the multiple frame images contained in the medical video. This allows a frame image suitable for video analysis to be used in the medical video analysis process, thereby increasing the reliability of the medical video analysis.

[0011] The effects of this disclosure are not limited to those mentioned above, and any other effects not mentioned can be clearly understood by a person with ordinary skill in the art to which this disclosure pertains ("ordinary engineer") from the wording of the claims.

[0012] Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, where similar reference numerals indicate similar elements, but are not limited thereto. [Brief explanation of the drawing]

[0013] [Figure 1] This diagram illustrates an electronic device for generating a representative frame image of a medical image according to one embodiment of the present disclosure. [Figure 2] These are drawings illustrating the configuration of an electronic device according to one embodiment of the present disclosure. [Figure 3] This is a diagram illustrating the configuration of a processor in an electronic device according to one embodiment of the present disclosure. [Figure 4] This diagram illustrates a method for generating a representative frame image of a medical video using multiple frame images contained in a medical video according to one embodiment of the present disclosure. [Figure 5] This is a diagram illustrating a method for generating a representative frame image of a medical video according to one embodiment of the present disclosure. [Figure 6] This diagram illustrates a method for generating a representative frame image of a medical video using a score vector and a weighted value vector according to one embodiment of the present disclosure. [Figure 7] This is a diagram showing an artificial neural network model according to one embodiment of the present disclosure. [Figure 8] This diagram illustrates the experimental results obtained by applying a representative frame image of medical video according to one embodiment of the present disclosure to a vascular classification model. [Figure 9] This is a diagram illustrating a method for analyzing medical images according to one embodiment of the present disclosure. [Modes for carrying out the invention]

[0014] <Summary of the Invention> According to one embodiment, the step of calculating a score for each of a plurality of frame images may include obtaining, through a machine learning model, a score vector composed of n score elements from n frame images when the medical video includes n frame images and n is a positive integer.

[0015] According to one embodiment, the step of obtaining a score vector may include calculating n score elements corresponding to each of the n frame images based on at least one of values measuring the similarity between each of the n frame images and other frame images or values measuring the image quality of each of the n frame images.

[0016] According to one embodiment, the step of obtaining a score vector may include identifying a region corresponding to a blood vessel in each of the n frame images, identifying a region that satisfies a specified condition among the regions corresponding to the blood vessels, and calculating n score elements corresponding to each of the n frame images based on the size of the region that satisfies the specified condition.

[0017] According to one embodiment, the specified condition may include a condition that the intensity of the hue of the pixels included in the region is greater than or equal to a specified size.

[0018] According to one embodiment, the step of generating a representative frame image of the medical video may include obtaining, using a specified mathematical formula, a weighted value vector composed of n weighted value elements from the score vector.

[0019] According to one embodiment, the step of generating a representative frame image of the medical video may include applying, to each of the n frame images, the corresponding weighted value element among the n weighted value elements included in the weighted value vector, and generating a representative frame image by merging the n frame images to which the corresponding weighted value elements have been applied.

[0020] According to one embodiment, the step of applying the weighting value elements corresponding to each of the n frame images may include the step of applying the weighting value elements corresponding to each of the pixels included in each of the n frame images.

[0021] According to one embodiment, the score for each of the plurality of frame images includes a score based on the intensity of the contrast agent calculated from each of the plurality of frame images, and the intensity of the contrast agent calculated from each of the plurality of frame images can include a numerical value reflecting the degree to which the region corresponding to the blood vessel into which the contrast agent is injected and the remaining region are distinguished in each of the plurality of frame images.

[0022] According to one embodiment, the step of calculating the score for each of the plurality of frame images may include the step of calculating the score for each of the plurality of frame images based on the value of the similarity measured between each of the plurality of frame images and other frame images.

[0023] According to one embodiment, the step of calculating the score for each of the plurality of frame images may include the step of calculating the score for each of the plurality of frame images based on the value of the image quality measured for each of the plurality of frame images.

[0024] According to one embodiment, the step of calculating the score for each of the plurality of frame images includes the steps of identifying the region corresponding to the blood vessel in each of the plurality of frame images, identifying the region that satisfies the specified condition among the regions corresponding to the blood vessel, and calculating the score for each of the plurality of frame images based on the size of the region that satisfies the specified condition.

[0025] According to one embodiment, the specified condition can include the condition that the intensity of the hue of the pixels included in the region is greater than or equal to the specified size.

[0026] <Detailed description of the invention> The specific details for implementing this disclosure will be described below with reference to the attached drawings. However, in the following description, if there is a risk of unnecessarily obscuring the essence of this disclosure, specific descriptions of widely known functions and configurations will be omitted.

[0027] In the attached drawings, identical or corresponding components are given the same reference numerals. Furthermore, in the following description of embodiments, the description of identical or corresponding components may be omitted. However, the omission of technical details regarding a component does not mean that such a component is not included in a particular embodiment.

[0028] The advantages and features of the disclosed embodiments, and how they are achieved, will become clearer with reference to the embodiments described below, along with the accompanying drawings. However, this disclosure is not limited to the embodiments disclosed below and may be embodied in a variety of different forms, although these embodiments are provided only to complete the disclosure and to fully inform a person of the ordinary skill of the scope of the invention.

[0029] This specification will briefly explain the terminology used and then provide a detailed description of the disclosed embodiments. The terminology used herein has been selected to the greatest extent possible to be widely used and general terms, taking into account the function of this disclosure; however, this may change depending on the intent of the articulates in the relevant field, case law, the emergence of new technologies, etc. In some cases, the applicant has arbitrarily selected terms, in which case their meaning will be described in detail in the section describing the relevant invention. Therefore, the terminology used in this disclosure should not be simply defined by its name, but rather by its meaning and the overall content of this disclosure.

[0030] In this specification, singular expressions include plural expressions unless the context clearly identifies them as singular. Similarly, plural expressions include singular expressions unless the context clearly identifies them as plural. When a part of the specification contains any component, this means that it may contain other components, not exclude them, unless otherwise stated.

[0031] Furthermore, the terms “module” or “part” as used in the specification refer to a software or hardware component, and a “module” or “part” performs some role. However, the meaning of “module” or “part” is not limited to software or hardware. A “module” or “part” may be configured to reside on an addressable storage medium, or to regenerate one or more processors. Thus, as an example, a “module” or “part” may include components such as software components, object-oriented software components, class components, and task components, and at least one of the following: processes, functions, attributes, processors, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, or variables. The components and the functions provided within a “module” or “part” may be combined with a smaller number of components and “modules” or further separated into additional components and “modules” or “parts.”

[0032] According to one embodiment of the present disclosure, “module” or “part” may be embodied in a processor and memory. “Processor” should be broadly interpreted to include general-purpose processors, central processing units (CPUs), microprocessors, digital signal processors (DSPs), controllers, microcontrollers, state machines, etc. In some environments, “processor” may refer to on-demand semiconductors (ASICs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), etc. “Processor” may refer to a combination of processing devices such as, for example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors coupled with a DSP core, or any other combination of such configurations. Also, “memory” should be broadly interpreted to include any electronic component capable of storing electronic information. "Memory" can refer to a variety of processor-readable media, such as arbitrary access memory (RAM), read-only memory (ROM), non-volatile arbitrary access memory (NVRAM), programmable read-only memory (PROM), erase-programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, magnetic or marking data storage devices, and registers. Memory is said to be in electronic communication with the processor if the processor can read / read information from or record information into it. Memory integrated into a processor is in electronic communication with the processor.

[0033] Furthermore, the terms 1st, 2nd, A, B, (a), (b), etc., used in the following examples are merely used to distinguish one component from another, and do not limit the essence, order, or sequence of the components in question.

[0034] Furthermore, in the following embodiments, when it is stated that a component is “connected,” “joined,” or “connected” to another component, it should be understood that the component may be directly connected to or linked to the other component, but that other components may be further “connected,” “joined,” or “connected” between each component.

[0035] Furthermore, the terms "comprises" and / or "comprising" used in the following embodiments do not preclude the presence or addition of one or more other components, stages, operations, and / or elements mentioned.

[0036] Various embodiments of this disclosure will be described in detail below with reference to the accompanying drawings.

[0037] Figure 1 is an illustrative diagram of an electronic device 100 for generating a representative frame image 120 of a medical image 110 according to one embodiment of the present disclosure. Referring to Figure 1, the electronic device 100 can generate (or provide) a representative frame image 120 of a medical image 110 based on the medical image 110. For example, the electronic device 100 can receive a medical image 110 and generate a representative frame image 120 using multiple frame images contained in the received medical image 110. Here, the medical image 110 is an image (e.g., video or still image) taken for the diagnosis, treatment, or prevention of a disease, and may refer to images taken of the inside / outside of a patient's body. For example, medical images 110 can include all types (modalities) of images, such as X-ray images, ultrasound images, chest radiographs, CT (Computed Tomography) images, PET (Positron Emission Tomography) images, MRI (Magnetic Resonance Imaging) images, Sonography (Ultrasound, US) images, fMRI (functional Magnetic Resonance Imaging) images, pathology tissue images (digital pathology whole slide images, WSI), and DBT (Digital Breast Tomosynthesis) images. In one embodiment, medical images 110 can include medical images of the patient's blood vessels taken while the patient has been administered a contrast agent. Furthermore, a frame image may refer to a still image that constitutes medical images 110. For example, the fact that medical video 110 contains multiple frame images may mean that medical video 110 contains multiple still images, in which case the multiple frame images may be sequentially assigned frame numbers (e.g., 1st frame image, 2nd frame image, ..., nth frame image, where n is a natural number) according to the time in which medical video 110 was taken.

[0038] Although Figure 1 does not show a storage system that can communicate with the electronic device 100, the electronic device 100 may be configured to be connected to or able to communicate with one or more storage systems. A storage system configured to be connected to or able to communicate with the electronic device 100 may include a device or cloud system that stores and manages various data related to the process of generating representative frame images 120 of medical images 110. For efficient data management, the storage system may use a database to store and manage various data. Here, the various data may include any data related to the generation of the representative frame images 120. For example, the various data may include, but are not limited to, machine learning models, training data, medical images 110, etc., related to the generation of the representative frame images 120.

[0039] The electronic device 100 can acquire medical images 110 of blood vessels. Such medical images 110 can be received via a communicationable storage medium (e.g., hospital system, local / cloud storage system, etc.). The electronic device 100 can also calculate a score for each of the multiple frame images contained in the medical image 110. Here, the score for a frame image may represent an evaluation score indicating how well the frame image in the medical image 110 is suitable for analysis for the diagnosis, treatment, prevention, etc. The method for calculating such scores for frame images will be explained in detail with reference to Figures 3 and 4. Thereafter, the electronic device 100 can generate a representative frame image 120 of the medical image 110 from the multiple frame images based on the score for each of the multiple frame images. Thus, the electronic device 100 does not select one frame image from among multiple frame images contained in the medical image 110, but rather generates a representative frame image 120 using multiple frame images contained in the medical image 110. This helps ensure that a frame image suitable for video analysis can be used in the analysis process of the medical image 110, thereby increasing the reliability of the analysis of the medical image 110.

[0040] Figure 2 is a diagram illustrating the configuration of an electronic device 100 according to one embodiment of the present disclosure. Referring to Figure 2, the electronic device 100 may include a memory 210, a processor 220, a communication module 230, and an input / output interface 240. However, the configuration of the electronic device 100 is not limited thereto. According to various embodiments, the electronic device 100 may omit at least one of the aforementioned components and may further include at least one other component. For example, the electronic device 100 may further include a display. In this case, the electronic device 100 can display at least one of the following on the display: a medical image of a blood vessel (e.g., medical image 110 in Figure 1) or a representative frame image of a medical image (e.g., representative frame image 120 in Figure 1).

[0041] The memory 210 can store various data used by at least one other component of the electronic device 100 (e.g., the processor 220). The data may include, for example, software (or programs) and input or output data for associated instructions.

[0042] Memory 210 can include any non-temporary computer-readable recording medium. In one embodiment, memory 210 can include a permanent mass storage device such as a disk drive, SSD (solid state drive), or flash memory. In another example, a permanent mass storage device such as ROM, SSD, flash memory, or disk drive may be included in the electronic device 100 as a separate permanent storage device distinct from memory 210. Memory 210 can also store an operating system and at least one program code (e.g., instruction words such as representative frame image generation installed and driven in the electronic device 100). In Figure 2, memory 210 is shown as a single memory, but this is for illustrative purposes only, and memory 210 can include multiple memories and / or buffer memories.

[0043] Software components may be loaded from a computer-readable recording medium separate from memory 210. Such a separate computer-readable recording medium may include a recording medium that can be directly connected to the electronic device 100, but may also include computer-readable recording media such as floppy drives, disks, tapes, DVD / CD-ROM drives, and memory cards. As another example, software components may be loaded into memory 210 via a communication module 230 rather than a computer-readable recording medium. For example, at least one program may be loaded into memory 210 based on a computer program (e.g., a program for data transmission such as medical images of blood vessels) installed by a file provided via the communication module 230 by a developer or a file distribution system that distributes application installation files.

[0044] The processor 220 can execute software (or programs) to control at least one other component (e.g., hardware or software component) of the electronic device 100 connected to the processor 220, and can perform a variety of data processing or calculations. According to one embodiment, as at least part of the data processing or calculation, the processor 220 can load instructions or data received from other components (e.g., communication module 230) into volatile memory, process the instructions or data stored in volatile memory, and store the resulting data in non-volatile memory.

[0045] The processor 220 can be configured to process computer program instructions by performing basic arithmetic, logic, and input / output operations. Instructions can be provided to the electronic device 100 or other external systems via memory 210 or a communication module 230. For example, the processor 220 can generate a representative frame image of a medical image. The processor 220 can then store the generated representative frame image in memory 210, display it on the display of the electronic device 100, or transmit it to an external electronic device via the communication module 230. Alternatively, the processor 220 can use the representative frame image of the medical image to perform additional analytical operations, such as classifying blood vessels. In Figure 2, the processor 220 is shown as a single processor, but this is for illustrative purposes only, and the processor 220 may contain multiple processors.

[0046] The communication module 230 can assist in establishing a direct (e.g., wired) or wireless communication channel between the electronic device 100 and an external electronic device, and in carrying out communication through the established communication channel. For example, the communication module 230 can provide a configuration or function for the electronic device 100 and an external electronic device (e.g., a user terminal or cloud server) to communicate with each other via a network. As an example, control signals, commands, data, etc., provided by the control of the processor 220 of the electronic device 100 can be transmitted to the external electronic device via the communication module 130 and the network through the communication module of the external electronic device. For example, the electronic device 100 can receive medical images of blood vessels of a subject from an external electronic device via the communication module 230.

[0047] The input / output interface 240 may be a means for interface with an input or output device (not shown) that is connected to or may be included in the electronic device 100. For example, the input / output interface 240 may include at least one of a PCI express interface or an Ethernet interface. In Figure 2, the input / output interface 240 is shown as an element configured separately from the processor 220, but is not limited to this, and the input / output interface 240 may be configured to be included in the processor 220.

[0048] According to one embodiment, the processor 220 can perform functions related to the generation of a representative frame image of a medical image. To perform functions related to the generation of a representative frame image of a medical image, the processor 220 can execute at least one computer-readable program contained in the memory 210. Here, the at least one program may include instructions for acquiring a medical image in which blood vessels are captured, calculating a score for each of the multiple frame images contained in the medical image, and generating a representative frame image of the medical image from the multiple frame images based on the score for each of the multiple frame images. For convenience of explanation in the following description, the execution of at least one program to perform functions related to the generation of a representative frame image of a medical image may be described as the processor 220 performing functions related to the generation of a representative frame image of a medical image. For example, the inclusion of instructions related to the generation of a representative frame image of a medical image in at least one program may correspond to the processor 220 performing functions related to the generation of a representative frame image of a medical image.

[0049] According to one embodiment, the processor 220 can calculate a score for each of several frame images contained in a medical image through a machine learning model that takes the medical image as input. Here, the score for a frame image may represent an evaluation score indicating how well the frame image in the medical image is suited to analysis for the diagnosis, treatment, or prevention of disease. The machine learning model may also include any model used to infer a solution for a given input. According to one embodiment, the machine learning model may include an artificial neural network model that includes an input layer, several hidden layers, and an output layer. Here, each layer may include one or more nodes. The machine learning model may also include weights associated with several nodes included in the machine learning model. Here, the weights may include any parameters associated with the machine learning model. The machine learning model in this disclosure may be a model that has been trained using a variety of learning methods. For example, various learning methods such as supervised learning, semi-supervised learning, unsupervised learning (or autonomous learning), and reinforcement learning may be used in this disclosure. In this disclosure, a machine learning model may refer to an artificial neural network model, and an artificial neural network model may refer to a machine learning model. An artificial neural network model will be explained in detail with reference to Figure 7.

[0050] Figure 3 is a diagram illustrating the configuration of a processor 220 of an electronic device according to one embodiment of the present disclosure. Referring to Figure 3, the processor 220 can generate a representative frame image (e.g., representative frame image 120 in Figure 1) of a medical image (e.g., medical image 110 in Figure 1). For this purpose, the processor 220 may include a score calculation unit 310, a weighted value calculation unit 320, and a representative frame image generation unit 330. However, the types and number of components included in the processor 220 are classified according to their functions related to the generation of a representative frame image of a medical image, and are not limited thereto. Furthermore, at least one of the components included in the processor 220 may be embodied in the form of an instruction word stored in memory (e.g., memory 210 in Figure 2).

[0051] The processor 220 can acquire medical images. According to one embodiment, the processor 220 can acquire medical images of the blood vessels of a patient while the patient has been administered a contrast agent. Here, the medical images may include multiple frame images, and the multiple frame images may be in order according to the time of acquisition. Such medical images may be received from an electronic device-connected or communicable storage system (e.g., hospital system, electronic duty record, prescription transmission system, medical image system, laboratory information system, local / cloud storage system, etc.), internal memory and / or user terminal. The received medical images may be provided to a score calculation unit 310, a weighted value calculation unit 320 and / or a representative frame image generation unit 330, and may be used to generate a representative frame image of the medical images.

[0052] The score calculation unit 310 can calculate a score for each of the multiple frame images included in the medical video. Here, the score for a frame image may represent an evaluation score indicating how well the frame image in the medical video is suitable for analysis for the diagnosis, treatment, prevention, etc. of a disease.

[0053] According to one embodiment, a score for a frame image may be calculated based on a value that measures the similarity between the frame image in question and other frame images. For example, a frame image with high similarity to other frame images may be assigned a high score. A high score assigned to a frame image in relation to similarity may mean that the frame image in question is sufficiently similar to other frame images in the sequence of frame images, and that the medical image can be analyzed by simply looking at the frame image in question. Such similarity between frame images may be calculated based on the similarity between pixels contained in each frame image, and the similarity between pixels may be calculated by at least one of the hue value or luminance value of the pixels.

[0054] According to one embodiment, a score for a frame image may be calculated based on a measured value of the image quality of that frame image. For example, a frame image with good image quality may be assigned a high score. A high score for a frame image in relation to image quality may mean that the frame image is sharp and contains objects and / or backgrounds with less movement compared to adjacent frame images, making it relatively advantageous for video analysis. The image quality of such a frame image may be calculated based on sharpness, the degree of brightness distortion, the degree of contrast distortion, etc. Sharpness means, for example, that the image has less blur, with sharper images having greater changes near the edges and blurrier images having less changes near the edges. Brightness distortion can be recognized as distortion when the average brightness of all pixels in the image exceeds a boundary value (or range) that assumes a limit to situations where the brightness of the image can be excessively extreme to one side (e.g., excessively dark or excessively bright). In this case, the lower the degree of brightness distortion, the better the image quality can be considered. Contrast refers to the difference between the dark and bright parts of an image, and accurate contrast can provide a visually sharper image. Conversely, contrast distortion means, for example, that the contrast between light and dark in an image is distorted, and the lower the degree of contrast distortion, the better the image quality can be considered. Contrast distortion can be calculated based on the ratio of the relative brightness of the dark parts to the relative brightness of the bright parts of an image, among other things.

[0055] According to one embodiment, a score for a frame image may be calculated based on a value measuring the similarity between the frame image and other frame images, and a value measuring the image quality of the frame image. For example, a score for a frame image may be determined by a combination of a value measuring the similarity between the frame image and other frame images and a value measuring the image quality of the frame image. In this case, the same or different weighting values ​​may be applied to the value measuring the similarity between the frame image and other frame images and the value measuring the image quality of the frame image.

[0056] According to one embodiment, considering that the frame image is included in a medical image of blood vessels, the score for the frame image may be calculated based on the size of the region within the vascular area (or region corresponding to a blood vessel) included in the image that satisfies specified conditions. For example, a medical image of blood vessels may be taken with a contrast agent administered to the patient's body so that the blood vessel under examination can be clearly distinguished from the surrounding area. Such a contrast agent can increase the contrast of the image by artificially increasing the difference in X-ray absorption. For example, a blood vessel area filled with contrast agent appears darker (or blacker) than the surrounding area. Accordingly, a region within the vascular area that satisfies specified conditions may indicate a region where the hue intensity of the pixels included in that region is greater than or equal to a specified size, and the larger the size of the region within the vascular area that satisfies specified conditions, the higher the score for the frame image may be calculated. In other words, a high score assigned to a frame image in relation to a blood vessel area means that the frame image was acquired when the target blood vessel was uniformly and sufficiently filled with contrast agent, and it may indicate that the frame image is one in which the blood vessel under examination can be clearly distinguished from the surrounding area.

[0057] According to one embodiment, the score calculation unit 310 can calculate a score for each of the multiple frame images contained in a medical video through a machine learning model that takes the medical video as input. For example, the score calculation unit 310 can obtain a score for each of the multiple frame images contained in a medical video through a machine learning model based on at least one of the similarity between the frame image and other frame images or the image quality of the frame image. As another example, the score calculation unit 310 can obtain a score for each of the multiple frame images contained in a medical video through a machine learning model based on the size of the region that satisfies specified conditions among the vascular regions contained in the image.

[0058] According to one embodiment, the score calculation unit 310 uses a machine learning model to mask areas identified as blood vessels (vascular areas) in each of the multiple frame images included in the medical image, and calculates a score for each of the multiple frame images included in the medical image based on the masked areas. At this time, the score for each frame image may include a score based on the intensity of the contrast agent that can be calculated from the frame image. Here, the intensity of the contrast agent that can be calculated from the frame image may include a numerical value that reflects the degree to which the area corresponding to the blood vessel into which the contrast agent was injected is distinguished from the remaining area in the frame image.

[0059] More specifically, the score calculation unit 310 first inputs each of the multiple frame images into a machine learning model, thereby obtaining multiple frame images in which the regions identified as blood vessels are masked. The machine learning model may include a model trained to perform masking on the regions identified as blood vessels in the input images and output the masked images. For example, the machine learning model may be a supervised-learned model using training data consisting of pairs of training images and images (ground truth labels) in which the regions corresponding to blood vessels (e.g., regions where contrast agent was injected) in each training image are masked. Furthermore, a machine learning model of the Semantic Segmentation series may be used to distinguish the regions identified as blood vessels in the input images, but is not limited to this; any type of machine learning model can be used. Subsequently, the score calculation unit 310 can calculate a score for each of the multiple frame images based on the masked frame images. For example, the score calculation unit 310 can calculate a score for each of the multiple frame images based on the number of pixels in the masked regions in each of the multiple masked frame images.

[0060] According to one embodiment, the score calculation unit 310 can calculate a confidence value for each of the multiple pixels contained in each of the multiple frame images included in the medical image, and calculate a score for each of the multiple frame images included in the medical image based on the calculated confidence value. For example, the score calculation unit 310 can use a filter to obtain a confidence value for each of the multiple pixels contained in the frame image, which is determined to be a vascular region. Here, the filter may be a filter that outputs a confidence value for each of the multiple pixels contained in the input image, which is determined to correspond to a tubular structure (vessel-like or tube-like structure), and may include, but is not limited to, a flange filter. Subsequently, the score calculation unit 310 can adjust (scaling) the magnitude of the confidence value calculated for the entire medical image. For example, the score calculation unit 310 can adjust the confidence value so that it is between 0 and 1. Then, the score calculation unit 310 can sum up the confidence values ​​for each of the multiple pixels, which are determined to be vascular regions, for each frame image, and calculate a score for the frame image based on the summed value.

[0061] The methods for calculating scores for frame images described above are merely examples, and the scope of this disclosure is not limited thereto. Therefore, any scoring method not described herein, such as scoring methods utilizing various filters, kernels, or models, may be applied to this disclosure.

[0062] The weighted value calculation unit 320 can obtain (or calculate) a weighted value for each of the multiple frame images contained in the medical video from the score for each of the multiple frame images contained in the medical video. For example, the weighted value calculation unit 320 can calculate a weighted value for each of the multiple frame images by applying the score for each of the multiple frame images contained in the medical video to a specified mathematical formula. In one embodiment, the specified mathematical formula can include a softmax function, a min-max normalize function, etc., and can include a form in which two or more functions are combined. Here, the score can be normalized to a value between 0 and 1 by using the min-max normalize function to set the highest score to 1 and the lowest score to 0, and the weighted value can be calculated by multiplying the normalized value by a specific value or by raising the normalized value to the power of n (where n is an integer of 2 or more) times. For example, when the normalized value is raised to the power of n times, a weighted value close to 0 can be calculated for the scores normalized to a value between 0 and 1, and the weighted value can be maintained at 1 only for the highest score.

[0063] The representative frame image generation unit 330 can generate a representative frame image of a medical video from multiple frame images based on a score for each of the multiple frame images. In one embodiment, the representative frame image generation unit 330 can generate a representative frame image by applying a weighted value corresponding to each of the multiple frame images included in the medical video and merging the multiple frame images to which the corresponding weighted values ​​have been applied. For example, the representative frame image generation unit 330 can generate a representative frame image by weighting the multiple frame images and the weighted values ​​for each of the multiple frame images. Here, applying a weighted value to a frame image can mean applying a weighted value to each pixel included in the frame image.

[0064] According to one embodiment, the representative frame image generation unit 330 can normalize the generated representative frame image. For example, the representative frame image generation unit 330 can normalize the generated representative frame image through weighted summation so that it has pixel values ​​that match the model (e.g., a vascular classification model).

[0065] Figure 4 is a diagram illustrating a method for generating a representative frame image 460 of a medical video 410 using multiple frame images contained in the medical video 410 according to one embodiment of the present disclosure. Referring to Figure 4, the processor (e.g., processor 220 in Figures 2 and 3) of an electronic device (e.g., electronic device 100 in Figures 1 and 2) for generating a representative frame image 460 of the medical video 410 can calculate a score for each of the multiple frame images contained in the medical video 410. In the following description, the medical video 410 can be described as containing n (where n is a positive integer) frame images. According to one embodiment, the processor can obtain a score vector 430 consisting of n score elements from the n frame images contained in the medical video 410 through a machine learning model 420. Here, each score element included in the score vector 430 can represent the score of the corresponding frame image, and the score of the frame image may mean an evaluation score indicating how well the frame image in the medical video 410 is suitable for analysis for the diagnosis, treatment, prevention, etc. Furthermore, while we will describe the case where the machine learning model 420 disclosed in Figure 4 is a scoring model that calculates a score for each of multiple frame images, we will not be limited to this, and the machine learning model 420 according to one embodiment of this disclosure may include a weighted value calculation model, a vascular region masking model, a vascular classification model, and the like.

[0066] According to one embodiment, in connection with obtaining the score vector 430, the processor can calculate n score elements corresponding to each of the n frame images based on at least one of the following: a measured similarity between each of the n frame images and other frame images, or a measured image quality of each of the n frame images. For example, the processor can calculate the similarity between pixels contained in each frame image based on at least one of the hue value or luminance value of pixels contained in each frame image, and can calculate the similarity between frame images based on the similarity between pixels contained in each frame image. The processor can also assign a high score to frame images that have a high similarity to other frame images. As another example, the processor can calculate the image quality of each frame image and assign a high score to frame images with good image quality. Here, image quality can be calculated by at least one of sharpness, brightness distortion, or contrast distortion. As yet another example, the processor can calculate n score elements corresponding to each of the n frame images by combining the measured similarity between each of the n frame images and other frame images and the measured image quality of each of the n frame images.

[0067] According to one embodiment, in relation to obtaining the score vector 430, the processor can calculate n score elements corresponding to each of the n frame images based on the size of the regions that satisfy a specified condition among the vascular regions (or regions corresponding to blood vessels) contained in each of the n frame images. For example, the processor can identify regions corresponding to blood vessels in each of the n frame images, identify regions that satisfy a specified condition among the regions corresponding to blood vessels, and calculate n score elements corresponding to each of the n frame images based on the size of the regions that satisfy the specified condition. Here, the specified condition may include the condition that the hue intensity of the pixels contained in the region is greater than or equal to a specified size. For example, a medical image 410 of blood vessels may be taken with a contrast agent administered to the patient's body so that the blood vessels under examination can be clearly distinguished from the surroundings, and the vascular regions filled with contrast agent appear darker (or blacker) than the surroundings. Accordingly, regions among the vascular regions that satisfy the specified condition can indicate regions where the hue intensity of the pixels contained in the region is greater than or equal to a specified size, and the larger the size of the region among the vascular regions that satisfies the specified condition, the higher the score calculated for the frame image.

[0068] In one embodiment, in relation to obtaining the score vector 430, the processor can use a machine learning model 420 to mask the regions identified as blood vessels (vascular regions) in each of the n frame images contained in the medical image 410, and calculate n score elements corresponding to each of the n frame images based on the masked regions. At this time, the score for a frame image may represent a score based on the intensity of the contrast agent that can be calculated from the frame image. Here, the intensity of the contrast agent that can be calculated from the frame image may include a numerical value that reflects the degree to which the region corresponding to the blood vessel into which the contrast agent was injected is distinguished from the remaining region in the frame image.

[0069] According to one embodiment, in relation to obtaining the score vector 430, the processor can calculate a confidence value that each of the multiple pixels contained in each of the n frame images contained in the medical image 410 is judged to be a vascular region, and based on the calculated confidence value, it can calculate n score elements corresponding to each of the n frame images. For example, the processor can use a filter (e.g., a flange filter) to obtain a confidence value that each of the multiple pixels contained in the frame image is judged to be a vascular region. Here, the filter may include a filter that outputs a confidence value that each of the multiple pixels contained in the input image is judged to correspond to a tubular structure.

[0070] Subsequently, the processor can use the specified mathematical formula 440 to obtain a weighted vector 450 composed of n weighted elements from the score vector 430. For example, the processor can input the n score elements contained in the score vector 430 into the specified mathematical formula 440 and obtain a weighted vector 450 composed of n weighted elements output from the specified mathematical formula 440. In one embodiment, the specified mathematical formula can include a softmax function, a min-max normalize function, etc., and can include a form in which two or more functions are combined.

[0071] Subsequently, the processor can generate a representative frame image 460 of the medical image 410 from the n frame images based on the score for each of the n frame images. For example, the processor can apply the corresponding weighting element from the n weighting elements obtained from the score for each of the n frame images to each of the n frame images, and then merge the n frame images to which the corresponding weighting elements have been applied to generate the representative frame image 460. For example, the processor can generate the representative frame image 460 by weighting the n frame images and the weighting elements for each of the n frame images. Here, applying the weighting element corresponding to each of the n frame images can be said to mean applying the weighting element corresponding to each pixel contained in each of the n frame images.

[0072] According to one embodiment, the processor can normalize the generated representative frame image 460. For example, the processor can normalize the generated representative frame image 460 through a weighted sum so that it has pixel values ​​that fit a model (e.g., a vascular classification model).

[0073] Figure 5 is a diagram illustrating a method for generating a representative frame image of a medical image according to one embodiment of the present disclosure. Referring to Figure 5, the processor (e.g., processor 220 in Figures 2 and 3) of an electronic device (e.g., electronic device 100 in Figures 1 and 2) for generating a representative frame image of a medical image can acquire a medical image of blood vessels in 510 steps (S510). Here, the medical image may include a medical image of the blood vessels of a subject patient taken while the subject patient has been administered a contrast agent. For example, the processor can receive medical images from an external electronic device connected via a communication module (e.g., communication module 230 in Figure 2). Here, the external electronic device may include a storage system (e.g., hospital system, electronic obligation record, prescription transmission system, medical image system, laboratory information system, local / cloud storage system, etc.) and / or a user terminal. As another example, the processor can receive medical images from the memory of the electronic device (e.g., memory 210 in Figure 2).

[0074] On a 520-point scale (S520), the processor can calculate a score for each of the multiple frame images contained in the medical image. Here, the score for a frame image can represent an evaluation score indicating how well the frame image in the medical image is suitable for analysis for purposes such as disease diagnosis, treatment, and prevention.

[0075] According to one embodiment, the processor can measure the similarity of each of a plurality of frame images to other frame images and calculate a score for each of the plurality of frame images based on the measured values. For example, the processor can assign a high score to a frame image that has a high similarity to other frame images. Such similarity between frame images may be calculated based on the similarity between pixels contained in each frame image, and the similarity between pixels may be calculated by at least one of the hue value or luminance value of the pixels.

[0076] According to one embodiment, the processor can measure the image quality for each of a plurality of frame images and calculate a score for each of the plurality of frame images based on the measured values. For example, the processor can assign a high score to frame images with good image quality. The image quality of such frame images may be calculated based on factors such as sharpness, the degree of brightness distortion, and the degree of contrast distortion.

[0077] According to one embodiment, the processor can calculate a score for each of the multiple frame images based on values ​​measuring similarity with other frame images and values ​​measuring image quality. For example, the processor can calculate a score for each of the multiple frame images by combining values ​​measuring similarity with other frame images and values ​​measuring image quality. In this case, the same or different weighting values ​​may be applied in the process of combining the measured values.

[0078] According to one embodiment, the processor can calculate a score for each of the multiple frame images based on the size of the region that satisfies specified conditions among the vascular regions (or regions corresponding to blood vessels) contained in each of the multiple frame images. For example, medical images of blood vessels may be taken with a contrast agent administered to the patient's body so that the blood vessel under examination can be clearly distinguished from the surrounding area. Such a contrast agent can increase the contrast of the image by artificially increasing the difference in X-ray absorption. For example, a blood vessel region filled with contrast agent appears darker (or blacker) than the surrounding area. Accordingly, a region within the blood vessel area that satisfies the specified conditions can indicate a region where the hue intensity of the pixels contained in that region is greater than or equal to a specified size, and the larger the size of the region within the blood vessel area that satisfies the specified conditions, the higher the score calculated for the frame image. In other words, a high score assigned to a frame image in relation to a blood vessel region means that the frame image was acquired when the target blood vessel was uniformly and sufficiently filled with contrast agent, and it can be said that the frame image is one in which the blood vessel under examination can be clearly distinguished from the surrounding area.

[0079] In one embodiment, the processor can calculate a score for each of the multiple frame images contained in the medical video through a machine learning model that takes the medical video as input. For example, the processor can obtain a score for each of the multiple frame images contained in the medical video through a machine learning model based on at least one of the following: the similarity between the frame image and other frame images, or the image quality of the frame image. In another example, the processor can obtain a score for each of the multiple frame images contained in the medical video through a machine learning model based on the size of the vascular region contained in the image that satisfies specified conditions.

[0080] According to one embodiment, the processor can use a machine learning model to mask areas identified as blood vessels (vascular regions) in each of the multiple frame images contained in the medical image, and calculate a score for each of the multiple frame images contained in the medical image based on the masked areas. In this case, the score for each frame image may include a score based on the intensity of the contrast agent that can be calculated from the frame image. Here, the intensity of the contrast agent that can be calculated from the frame image may include a numerical value that reflects the degree to which the region corresponding to the blood vessel into which the contrast agent was injected is distinguished from the remaining region in the frame image.

[0081] According to one embodiment, the processor can calculate a confidence value for each of the multiple pixels contained in each of the multiple frame images included in the medical image, and calculate a score for each of the multiple frame images included in the medical image based on the calculated confidence value. For example, the processor can use a filter (e.g., a flange filter) to obtain a confidence value for each of the multiple pixels contained in the frame image, which is determined to be a vascular region. The processor can then adjust the magnitude of the confidence value calculated for the entire medical image. For example, the processor can adjust the confidence value so that it is between 0 and 1. The processor can then sum up the confidence values ​​for each of the multiple pixels, which are determined to be a vascular region, for each frame image, and calculate a score for the frame image based on the summed value.

[0082] In 530 steps (S530), the processor can generate a representative frame image of a medical image based on the score for each of multiple frame images. According to one embodiment, the processor can obtain weighted values ​​for each of the multiple frame images contained in the medical image from the scores for each of the multiple frame images contained in the medical image. For example, the processor can calculate the weighted values ​​for each of the multiple frame images by applying the scores for each of the multiple frame images contained in the medical image to a specified mathematical formula. According to one embodiment, the specified mathematical formula can include a softmax function, a min-max normalize function, etc., and can include a form in which two or more functions are combined. Subsequently, the processor can apply the corresponding weighted values ​​to each of the multiple frame images contained in the medical image and merge the multiple frame images to which the corresponding weighted values ​​have been applied to generate a representative frame image. For example, the processor can generate a representative frame image by weighting the multiple frame images and the weighted values ​​for each of the multiple frame images together. Here, applying weighted values ​​to frame images means applying weighted values ​​to each pixel contained in the frame image.

[0083] Figure 6 is a diagram illustrating a method for generating a representative frame image of a medical image using a score vector and a weighted value vector according to one embodiment of the present disclosure. Referring to Figure 6, the processor (e.g., processor 220 in Figures 2 and 3) of an electronic device (e.g., electronic device 100 in Figures 1 and 2) for generating a representative frame image of a medical image can obtain a score vector consisting of n score elements from n (n is a positive integer) frame images contained in the medical image in 610 steps (S610). Here, each score element included in the score vector can represent the score of the corresponding frame image.

[0084] According to one embodiment, the processor can calculate n score elements corresponding to each of the n frame images based on at least one of the following: a measured similarity between each of the n frame images and other frame images, or a measured image quality of each of the n frame images. For example, the processor can calculate the similarity between pixels contained in each frame image based on at least one of the hue value or luminance value of pixels contained in each frame image, and can calculate the similarity between frame images based on the similarity between pixels contained in each frame image. The processor can also assign a high score to frame images that have a high similarity to other frame images. As another example, the processor can calculate the image quality of each frame image and assign a high score to frame images with good image quality. Here, image quality can be calculated by at least one of sharpness, brightness distortion, or contrast distortion. As yet another example, the processor can calculate n score elements corresponding to each of the n frame images by combining the measured similarity between each of the n frame images and other frame images and the measured image quality of each of the n frame images.

[0085] According to one embodiment, the processor can calculate n score elements corresponding to each of the n frame images based on the size of the regions that satisfy specified conditions among the vascular regions (or regions corresponding to blood vessels) contained in each of the n frame images. For example, the processor can identify regions corresponding to blood vessels in each of the n frame images, identify regions that satisfy specified conditions among the regions corresponding to blood vessels, and calculate n score elements corresponding to each of the n frame images based on the size of the regions that satisfy the specified conditions. Here, the specified conditions may include the condition that the hue intensity of the pixels contained in the region is greater than or equal to a specified size. For example, medical images of blood vessels may be taken with a contrast agent administered to the patient's body so that the blood vessels under examination can be clearly distinguished from the surroundings, and the vascular regions filled with contrast agent will appear darker (or blacker) than the surroundings. Accordingly, regions within the vascular region that satisfy the specified conditions can indicate regions where the hue intensity of the pixels contained in the region is greater than or equal to a specified size, and the larger the size of the region within the vascular region that satisfies the specified conditions, the higher the score calculated for the frame image.

[0086] In one embodiment, the processor can use a machine learning model to mask areas identified as blood vessels (vascular regions) in each of the n frame images, and calculate n score elements corresponding to each of the n frame images based on the masked regions. In this case, the score for a frame image may represent a score based on the intensity of the contrast agent that can be calculated from the frame image. Here, the intensity of the contrast agent that can be calculated from the frame image may include a numerical value that reflects the degree to which the region corresponding to the blood vessel in which the contrast agent was injected is distinguished from the remaining region in the frame image.

[0087] According to one embodiment, the processor can calculate a confidence value for each of the multiple pixels contained in each of the n frame images, and based on the calculated confidence values, it can calculate n score elements corresponding to each of the n frame images. For example, the processor can use a filter (e.g., a flange filter) to obtain a confidence value for each of the multiple pixels contained in the frame image, which is determined to be a vascular region. Here, the filter may include a filter that outputs a confidence value for each of the multiple pixels contained in the input image, which is determined to correspond to a tubular structure.

[0088] In 620 steps (S620), the processor can obtain a weighted vector consisting of n weighted elements from a score vector. For example, the processor can input the n score elements contained in the score vector into a specified mathematical formula and obtain a weighted vector consisting of n weighted elements output from the specified mathematical formula. In one embodiment, the specified mathematical formula can include a softmax function, a min-max normalize function, etc., and can include a form in which two or more functions are combined.

[0089] With 630 levels (S630), the processor can apply a corresponding weighting element to each of the n frame images. For example, the processor can apply a weighting element corresponding to each pixel contained in each of the n frame images.

[0090] In 640 steps (S640), the processor can merge n frame images to which weighted elements have been applied to generate a representative frame image. For example, the processor can generate a representative frame image by weighting the n frame images and the weighted elements applied to each of the n frame images.

[0091] According to one embodiment, the processor can normalize the generated representative frame image. For example, the processor can normalize the generated representative frame image through weighted summation so that it has pixel values ​​that match a model (e.g., a vascular classification model, a segmentation model, a catheter position detection model, a lesion information detection model, etc.).

[0092] Figure 7 is a diagram showing an artificial neural network model 700 according to one embodiment of the present disclosure. Referring to Figure 7, the artificial neural network model 700 is an example of a machine learning model and can represent a statistical learning algorithm or a structure that executes such an algorithm, which is embodied in machine learning technology and cognitive science based on the structure of a biological neural network.

[0093] According to one embodiment, the artificial neural network model 700 can demonstrate a machine learning model with problem-solving capabilities by having nodes, which are artificial neurons that form a network through synaptic connections, like a biological neural network, iteratively adjust the synaptic weights to learn to reduce the error between the correct output corresponding to a specific input and the inferred output. For example, the artificial neural network model 700 can include any probabilistic model or neural network model used in artificial intelligence learning methods such as machine learning and deep learning.

[0094] According to one embodiment, the aforementioned score calculation model, weighted value calculation model, vascular region masking model, and / or vascular classification model can be generated in the form of an artificial neural network model 700. For example, the artificial neural network model 700 can receive medical images of blood vessels and estimate representative frame images of the medical images based on this.

[0095] The artificial neural network model 700 can be implemented as a multi-layer perceptron (MLP) composed of multiple layers of nodes and connections between them. The artificial neural network model 700 according to this embodiment can be implemented using one of the artificial neural network model structures, including a multi-layer perceptron. The artificial neural network model 700 may consist of an input layer 720 that receives input data 710 (or input signals) from the outside, an output layer 740 that outputs output data 750 (or output signals) corresponding to the input data 710, and n hidden layers 730_1 to 730_n (where n is a positive integer) located between the input layer 720 and the output layer 740, which receive signals from the input layer 720, extract characteristics, and transmit them to the output layer 740. Here, the output layer 740 can receive signals from the hidden layers 730_1 to 730_n and output them to the outside.

[0096] The learning methods for the artificial neural network model 700 may include supervised learning, which learns to be optimized for solving a problem by inputting correct teacher signals (or labels), and unsupervised learning, which does not require teacher signals. According to one embodiment, an electronic device according to one embodiment of the present disclosure (e.g., the electronic device 100 in Figures 1 and 2) can train the artificial neural network model 700 using medical images of blood vessels.

[0097] According to one embodiment, the electronic device can generate training data for training an artificial neural network model 700. For example, the electronic device can generate a training dataset containing medical images of blood vessels. Subsequently, the electronic device can train the artificial neural network model 700 to generate representative frame images of medical images based on the generated training dataset.

[0098] According to one embodiment, the input variables of the artificial neural network model 700 can include medical images of blood vessels. In this way, when the aforementioned input variables are input through the input layer 720, the output variables output from the output layer 740 of the artificial neural network model 700 can be representative frame images of the medical images.

[0099] In this way, the artificial neural network model 700 can learn to extract the correct output corresponding to a specific input by matching multiple input variables with multiple output variables in the input layer 720 and output layer 740, respectively, and by adjusting the synaptic values ​​between nodes included in the input layer 720, hidden layers 730_1~730_n, and output layer 740. Through this learning process, the characteristics hidden in the input variables of the artificial neural network model 700 can be grasped, and the synaptic values ​​(or weights) between nodes of the artificial neural network model 700 can be adjusted to reduce the error between the output variable calculated based on the input variable and the target output. Furthermore, an electronic device can learn an algorithm that receives medical images of blood vessels as input and learn in a way that minimizes the loss with respect to the representative frame image (i.e., annotation information) of the medical image. Using the artificial neural network model 700 thus learned, the representative frame image of the medical image can be estimated.

[0100] Figure 8 is a diagram illustrating the experimental results obtained by applying a representative frame image of a medical video according to one embodiment of the present disclosure to a vascular classification model. Referring to Figure 8, we will explain how much more accurate the method of generating a representative frame image of a medical video according to one embodiment of the present disclosure and then applying the generated representative frame image to a vascular classification model is in vascular classification compared to the method of selecting the frame image that best fits an existing medical video and then applying the selected frame image to the vascular classification model. A total of 2176 medical videos were used in the experiment for the accuracy comparison, and the experimental results shown in Figure 8 represent 10 medical videos 811-820 that provided different results from each other.

[0101] Referring to Figure 8, the experimental results will be explained in detail. Of the 10 medical images 811-820, the 4th image 814 and the 10th image 820 are medical images in which the blood vessels of the subject patient were photographed without the administration of contrast agent to the patient, and are therefore unreadable. The 8th image 818 is an example where the existing method, namely, in which the most suitable frame image was selected from among the multiple frame images contained in the 8th image 818 and then applied to the vascular classification model, provided accurate results. Furthermore, for the remaining first video 811, second video 812, third video 813, fifth video 815, sixth video 816, seventh video 817, and ninth video 819, the method according to one embodiment of the present disclosure, namely, a method in which a representative frame image is generated using multiple frame images contained in a medical video (e.g., first video 811, second video 812, third video 813, fifth video 815, sixth video 816, seventh video 817, or ninth video 819), and then the generated representative frame image is applied to a vascular classification model, provided accurate results. In summary, the method according to one embodiment of the present disclosure showed 99.54% agreement with existing methods, and when comparing the 10 medical videos 811-820, which represent the 0.46% that did not agree, it can be confirmed that the method according to one embodiment of the present disclosure is more accurate than existing methods, except for two videos that were illegible.

[0102] Figure 9 is a diagram illustrating a method for analyzing medical images according to one embodiment of the present disclosure. Referring to Figure 9, a processor (e.g., processor 220 in Figures 2 and 3) of an electronic device (e.g., electronic device 100 in Figures 1 and 2) can generate a representative frame image of a medical image in 910 steps (S910). According to one embodiment, when the processor acquires a medical image containing (or captured) blood vessels, it calculates a score for each of the multiple frame images contained in the medical image, and based on the score for each of the multiple frame images, it can generate a representative frame image of the medical image from the multiple frame images. The operation of the processor in 910 steps may be the same as or similar to the operation of the processor described with reference to Figure 5.

[0103] In 920 steps (S920), the processor can classify blood vessels contained in medical images. For example, the processor can identify the type of blood vessel contained in the medical image. Through the 920 steps, the processor can identify which type of blood vessel was targeted in the medical image. In one embodiment, the processor can classify blood vessels contained in the representative frame image of the medical image by applying a machine learning model (e.g., a blood vessel classification model) to a representative frame image of the medical image.

[0104] In step 930 (S930), the processor can select a frame image to be analyzed from the medical image. According to one embodiment, the processor can select a frame image to be analyzed from among multiple frame images contained in the medical image. Here, the frame image to be analyzed may be, for example, a frame image suitable for analysis for the diagnosis, treatment, or prevention of disease. According to another embodiment, step 930 may be omitted. In this case, the processor can use a representative frame image of the medical image generated in step 910 as the frame image to be analyzed.

[0105] At 940 steps (S940), the processor can perform vascular segmentation on the frame image to be analyzed. For example, the processor can separate (or differentiate) blood vessels from the background in the frame image to be analyzed. According to one embodiment, the processor can apply a machine learning model to the frame image to be analyzed to separate the vascular region from the background region in the frame image to be analyzed. The machine learning model used in this process may be selected based on the type of blood vessel identified at 920 steps. As an example, if the identified blood vessel type is the left main coronary artery (LM), the processor can separate the vascular region from the background region in the frame image to be analyzed through the SPIDER VIEW segmentation model. As another example, if the identified blood vessel type is the left circumflex coronary artery (LCX), left anterior descending coronary artery (LAD), or right coronary artery (RCA), the processor can divide the vascular region from the background region in the frame image under analysis through a general segmentation model. As mentioned earlier, the processor can further select a machine learning model that is better suited to the identified blood vessel type, based on 920 levels of identification.

[0106] With 950 steps (S950), the processor can perform quantitative analysis of blood vessels. For example, the processor can detect lesions and calculate blood vessel diameter, FFR (Fractional Flow Reserve) value, and cardiovascular score.

[0107] The flowchart and explanation described above are merely illustrative examples and may be implemented differently in some embodiments. For example, in some embodiments, the order of the steps may be changed, some steps may be repeated, some steps may be omitted, or some steps may be added.

[0108] The aforementioned methods may be provided by computer programs stored on computer-readable recording media for execution on a computer. The media may continuously store computer-executable programs or temporarily store them for execution or download. The media may also be a variety of recording or storage means in the form of a combination of one or more hardware components, and is not limited to media directly connected to any computer system, but may be distributed on a network. Examples of media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and media configured to store program instructions, including ROM, RAM, and flash memory. Other examples of media include recording media or storage media managed by app stores and other sites, servers, etc., that supply or distribute applications and various other software.

[0109] The methods, operations, or techniques described herein may be embodied by a variety of means. For example, such techniques may be embodied in hardware, firmware, software, or a combination thereof. A person of ordinary skill will understand that the various exemplary logical blocks, modules, circuits, and algorithmic stages described in conjunction with the disclosure may be embodied in electronic hardware, computer software, or a combination thereof. To clearly illustrate such interchangeability of hardware and software, various exemplary components, blocks, modules, circuits, and stages have been generally described above in terms of their functional aspects. Whether such functions are embodied as hardware or software depends on the design requirements imposed on the particular application and the overall system. A person of ordinary skill may embodied the described functions in a variety of ways for their respective specific applications, but such embodiments should not be construed as deviating from the scope of this disclosure.

[0110] In hardware implementations, the processing units used to perform the techniques may be embodied in one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described herein, computers, or combinations thereof.

[0111] Accordingly, the various exemplary logic blocks, modules, and circuits described in conjunction with this disclosure may be embodied or performed by any combination of general-purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gates and transistor logic, discrete hardware components, or any other devices designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be embodied by a combination of computing devices, such as a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other combination of configurations.

[0112] In embodiments of firmware and / or software, the technique may be embodied in instructions stored on a computer-readable medium such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), or magnetic or marked data storage devices. The instructions may be executable by one or more processors, and may cause the processors(s) to perform specific modes of the functions described herein.

[0113] When embodied in software, the techniques described above may be stored on or transmitted through a computer-readable medium in the form of one or more instructions or codes. A computer-readable medium includes both computer storage media and communication media, encompassing any medium that facilitates the transmission of computer programs from one location to another. The storage medium may be any available medium accessible by a computer. As an unrestricted example, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to transport or store desired program code in the form of instructions or data structures and is accessible by a computer. Furthermore, any connection may appropriately be referred to as a computer-readable medium.

[0114] For example, when software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, lead wire, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, these are included within the definition of a medium. The terms "disk" and "disc" as used in this application include CDs, laserdiscs, optical discs, DVDs (digital versatile discs), floppy disks, and Blu-ray discs, where "disks" typically reproduce data magnetically, and "discs" reproduce data optically using a laser. The aforementioned combinations should also be included within the scope of computer-readable media.

[0115] The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, portable disk, CD-ROM, or any other known form of storage medium. An exemplary storage medium may be linked to the processor so that the processor can read information from or record information to the storage medium. Alternatively, the storage medium may be integrated into the processor. The processor and storage medium may reside within an ASIC. The ASIC may reside within a user terminal. Alternatively, the processor and storage medium may exist as separate components in the user terminal.

[0116] Although the embodiments described above are described as utilizing the aspects of the subject matter currently disclosed in one or more standalone computer systems, the disclosure is not limited to and may be embodied in conjunction with any computing environment, such as a network or a distributed computing environment. In other words, aspects of the subject matter may be embodied in multiple processing chips or devices, and storage may be similarly affected across multiple devices. Such devices may include PCs, network servers, and portable devices.

[0117] While this disclosure has been described in relation to some embodiments, various modifications and alterations are possible without departing from the scope of this disclosure as understandable to a person of the ordinary skill in the art to which the invention of this disclosure pertains. Such modifications and alterations should be understood to fall within the scope of the claims appended to this specification. [Explanation of Symbols]

[0118] 100 Electronic equipment 210 memory, 220 processors, 230 communication modules, 240 input / output interfaces.

Claims

1. In a method for generating representative frame images of medical images, performed by at least one processor, The stage of obtaining medical images including blood vessels, The steps include: masking the areas identified as blood vessels (vascular areas) in each of the multiple frame images contained in the medical video, and calculating a score for each of the multiple frame images contained in the medical video based on the masked areas; and The process includes generating a representative frame image of the medical image from the plurality of frame images based on a score for each of the plurality of frame images. The score for each of the aforementioned multiple frame images includes a score based on the contrast agent intensity that can be calculated from each of the multiple frame images. The intensity of the contrast agent calculated from each of the aforementioned multiple frame images includes a numerical value that reflects the degree to which the region corresponding to the blood vessel into which the contrast agent was injected is distinguishable from the rest of the region in each of the aforementioned multiple frame images. The step of generating the aforementioned representative frame image is: A process of normalizing the score for each of the multiple frame images to a predetermined range, and calculating a weighted value for each of the multiple frame images by raising the normalized score to the power of multiple times, and A method for generating a representative frame image of a medical video, comprising the step of generating a representative frame image of the medical video by weighting and summing the multiple frame images and the weight values ​​for each of the multiple frame images.

2. The step of calculating a score for each of the aforementioned multiple frame images is: A method for generating a representative frame image of a medical video according to claim 1, wherein, when the medical video includes n frame images, where n is a positive integer, the method includes the step of obtaining a score vector composed of n score elements from the n frame images through a machine learning model.

3. The step of obtaining the aforementioned score vector is: A method for generating a representative frame image of a medical image according to claim 2, comprising the step of calculating the n score elements corresponding to each of the n frame images based on at least one of the values ​​obtained by measuring the similarity between each of the n frame images and other frame images, or the values ​​obtained by measuring the image quality of each of the n frame images.

4. The step of obtaining the aforementioned score vector is: A step of identifying the region corresponding to the blood vessel in each of the n frame images, A method for generating a representative frame image of a medical image according to claim 2, comprising the steps of: identifying a region that satisfies specified conditions among the regions corresponding to the blood vessels; and calculating the n score elements corresponding to each of the n frame images based on the size of the region that satisfies the specified conditions.

5. The aforementioned specified conditions are: A method for generating a representative frame image of a medical image according to claim 4, further comprising the condition that the hue intensity of the pixels included in the region is greater than or equal to a specified size.

6. The step of generating a representative frame image of the aforementioned medical video is as follows: A method for generating a representative frame image of a medical image according to claim 2, comprising the step of obtaining a weighted value vector composed of n weighted value elements from the score vector using a specified mathematical formula.

7. The step of generating a representative frame image of the aforementioned medical video is as follows: The steps include applying the corresponding weight value element from the n weight value elements included in the weight value vector to each of the n frame images, and A method for generating a representative frame image of a medical image according to claim 6, further comprising the step of merging the n frame images to which the corresponding weighted value elements have been applied to generate the representative frame image.

8. The step of applying the corresponding weighted value element to each of the n frame images is: A method for generating a representative frame image of a medical image according to claim 7, further comprising the step of applying the corresponding weighting element to each pixel contained in each of the n frame images.

9. The step of calculating a score for each of the aforementioned multiple frame images is: A method for generating a representative frame image of a medical image according to claim 1, further comprising the step of calculating a score for each of the plurality of frame images based on a value that measures the similarity between each of the plurality of frame images and other frame images.

10. The step of calculating a score for each of the aforementioned multiple frame images is: A method for generating a representative frame image of a medical image according to claim 1, further comprising the step of calculating a score for each of the plurality of frame images based on the measured image quality of each of the plurality of frame images.

11. The step of calculating a score for each of the aforementioned multiple frame images is: Steps include identifying the region corresponding to the blood vessel in each of the aforementioned multiple frame images, A step of identifying a region among the regions corresponding to the blood vessels that satisfies the specified conditions, and A method for generating a representative frame image of a medical image according to claim 1, comprising the step of calculating a score for each of the plurality of frame images based on the size of the region that satisfies the specified conditions.

12. The aforementioned specified conditions are: A method for generating a representative frame image of a medical image according to claim 11, further comprising the condition that the hue intensity of the pixels included in the region is greater than or equal to a specified size.

13. A computer program stored on a computer-readable recording medium for performing the method described in any one of claims 1 to 12 on a computer.

14. In electronic devices, memory, and Includes at least one processor connected to the memory and configured to execute at least one computer-readable program contained in the memory, The aforementioned at least one program, We obtained medical images including blood vessels, In each of the multiple frame images contained in the medical video, the areas identified as blood vessels (vascular regions) are masked, and a score is calculated for each of the multiple frame images contained in the medical video based on the masked areas. The command includes instructions for generating a representative frame image of the medical image from the plurality of frame images, based on a score for each of the plurality of frame images. The score for each of the aforementioned multiple frame images includes a score based on the contrast agent intensity that can be calculated from each of the multiple frame images. The intensity of the contrast agent calculated from each of the aforementioned multiple frame images includes a numerical value that reflects the degree to which the region corresponding to the blood vessel into which the contrast agent was injected is distinguishable from the rest of the region in each of the aforementioned multiple frame images. The step of generating the aforementioned representative frame image is: A process of normalizing the score for each of the multiple frame images to a predetermined range, and calculating a weighted value for each of the multiple frame images by raising the normalized score to the power of multiple times, and An electronic device comprising the step of generating a representative frame image of the medical image by weighting and summing the plurality of frame images and the weight values ​​for each of the plurality of frame images.

Images