Information processing method, program, and information processing apparatus
By calculating statistical values and employing machine learning models, the method addresses the challenge of accurately extracting important frames from contrast videos, improving vascular diagnosis efficiency.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- THE UNIV OF TOKYO
- Filing Date
- 2024-11-22
- Publication Date
- 2026-06-03
AI Technical Summary
Existing methods for extracting specific frames from contrast videos, such as those used in angiography, face challenges in accurately identifying important frames while managing computational costs, particularly in diagnosing complex vascular structures like coronary arteries.
An information processing method that calculates statistical values of pixel data for each frame and uses machine learning models to extract specific frames based on these values, reducing computational cost while enhancing accuracy.
The method effectively extracts specific frames from contrast videos with high accuracy and reduces computational overhead, enabling precise diagnosis of vascular structures.
Smart Images
Figure 2026091050000001_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an information processing method, a program, and an information processing apparatus.
Background Art
[0002] In general, data of contrast examinations such as angiography (Angiography) is obtained as a video. However, in many cases, a user (medical staff) needs to select an important frame (hereinafter referred to as a "highlight image") from the video. For example, Patent Document 1 discloses a medical information processing apparatus that identifies and displays an image (frame) that a doctor wants to refer to during a procedure using a learned model.
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0004] In one aspect, an object is to provide an information processing method or the like that can suitably extract a specific frame from a contrast video.
Means for Solving the Problems
[0005] The information processing method includes acquiring a contrast video, calculating a statistical value of pixel values for each frame constituting the contrast video, and causing a computer to execute a process of extracting a specific frame from the contrast video based on the statistical value.
Effects of the Invention
[0006] In one aspect, a specific frame can be suitably extracted from a contrast video.
Brief Description of the Drawings
[0007] [Figure 1] This is a block diagram showing an example configuration of an image processing system. [Figure 2] This graph shows the variance and average value of the pixel values in each frame that makes up the contrast-enhanced video. [Figure 3] This figure shows an example of a screen that accepts the operation of assigning a class label. [Figure 4] This figure shows an example of inference processing. [Figure 5] This is a flowchart showing the steps involved in generating training data. [Figure 6] This flowchart shows the procedure for generating the first and second models. [Figure 7] This is a flowchart showing the steps involved in the inference process. [Figure 8] This graph shows the variance of pixel values in each frame that makes up a contrast-enhanced video. [Figure 9] This figure shows an example of the inference process related to Modification 2. [Modes for carrying out the invention]
[0008] The present invention will be described in detail below with reference to the drawings illustrating its embodiments. (Embodiment) Figure 1 is a block diagram showing an example configuration of an image processing system. In this embodiment, an image processing system is described that extracts a specific frame from a contrast-enhanced video based on statistical values of the pixel values of each frame constituting the contrast-enhanced video captured by the imaging device 2, and performs predetermined image processing. The image processing system includes an information processing device 1 and an imaging device 2.
[0009] The imaging device 2 is a device for imaging the inside of a living body, such as an angiography device used for angiography. The imaging device 2 is equipped with an irradiator that emits X-rays and a detector that detects the emitted X-rays, and captures a contrast-enhanced video of the blood vessels by detecting the X-rays that have passed through a subject (e.g., a person) into which a contrast agent has been administered into the blood vessels via a catheter. In this embodiment, assuming a case where PCI (Percutaneous Coronary Intervention) is performed, a configuration in which the subject's heart (coronary arteries) is imaged will be described.
[0010] Information processing device 1 is an information processing device capable of various information processing and information transmission / reception, such as a server computer, a personal computer, or a console dedicated to imaging device 2. Information processing device 1 calculates statistical values (e.g., variance and mean) of pixel values for each frame that makes up the contrast-enhanced video, and extracts frames that the user (medical professional) should focus on from the contrast-enhanced video by taking the extreme values of these statistical values (see Figure 2).
[0011] The information processing device 1 comprises a control unit 11, a main memory unit 12, a display unit 13, a communication unit 14, an input unit 15, and an auxiliary memory unit 16. The control unit 11 is a processor such as one or more CPUs (Central Processing Units), MPUs (Micro-Processing Units), or GPUs (Graphics Processing Units), and performs various information processing by reading and executing programs P stored in the auxiliary memory unit 16. The main memory unit 12 is a temporary storage area such as SRAM (Static Random Access Memory) or DRAM (Dynamic Random Access Memory), and temporarily stores data necessary for the control unit 11 to perform calculation processing. The display unit 13 is a display screen such as an organic EL (Electro-Luminescence) display, and displays images. The communication unit 14 is a communication module for performing communication-related processing, and sends and receives information with the outside. The input unit 15 is an operation interface such as a keyboard or mouse, and accepts operation input.
[0012] The auxiliary storage unit 16 is a non-volatile storage area such as a hard disk or large-capacity memory, and stores the program P (program product) and other data necessary for the control unit 11 to execute processing. The auxiliary storage unit 16 also stores the first model 51 and the second model 52. The first model 51 is a machine learning model that has been trained with predetermined training data, and when a frame of a contrast-enhanced video is input, it outputs the degree to which the input frame is suitable as a highlight image (hereinafter referred to as "highlight degree"). The second model 52 is a machine learning model that has been trained with predetermined training data, and when a frame of a contrast-enhanced video is input, it classifies the type of coronary artery (biological part) shown in the input frame.
[0013] The auxiliary storage unit 16 may be an external storage device connected to the information processing device 1. Furthermore, the information processing device 1 may be a multicomputer consisting of multiple computers, or it may be a virtual machine virtually constructed by software.
[0014] Also, in this embodiment, the information processing apparatus 1 is not limited to the above configuration, and may include, for example, an input unit that receives operation inputs, a display unit that displays images, and the like. Further, the information processing apparatus 1 may include a reading unit that reads a portable storage medium 1a such as a CD (Compact Disk)-ROM or a DVD (Digital Versatile Disc)-ROM, and reads and executes the program P from the portable storage medium 1a.
[0015] FIG. 2 is a graph showing the variance and average value of the pixel values of each frame constituting the contrast video. Hereinafter, the outline of this embodiment will be described.
[0016] The horizontal axis (first axis) of the graph shown in FIG. 2 represents the frame number of each frame of the contrast video, that is, the imaging order of the frames. The vertical axis (second axis) of the graph represents the variance and average value of the pixel values of each frame. The circles on the graph represent the points where extreme values (maximum or minimum values) are taken on the curves of the variance and average values.
[0017] In this embodiment, when the information processing apparatus 1 generates training data for machine learning and performs inference processing using a machine learning model (first model 51 and second model 52) constructed from the training data, as described later, as preprocessing, the information processing apparatus 1 performs a process of extracting important frames to be noted from the contrast video. Specifically, for each frame constituting the contrast video, the information processing apparatus 1 calculates statistical values of the pixel values (for example, luminance) of each pixel in the frame. Then, the information processing apparatus 1 extracts specific frames from the contrast video based on the calculated statistical values of each frame.
[0018] In the analysis of conventional coronary angiography videos, edge detection was the main method. Edge detection is a technique that emphasizes the contours and boundaries in an image and captures their changes to extract important frames. This method has been regarded as having the merits of being simple and having a low computational cost, but it has limitations in accurately capturing complex vascular structures and fine changes such as those of the coronary arteries. In particular, when performing a precise diagnosis of blood vessel stenosis or lesion parts, it has been pointed out that edge detection cannot ensure sufficient accuracy.
[0019] In addition to edge detection, it is conceivable to utilize a machine learning model to extract appropriate frames as highlight images from the contrast video. However, when performing image analysis using a machine learning model on all frames constituting the contrast video, there is a problem that the computational cost increases.
[0020] Therefore, in the present embodiment, frames are extracted based on the statistical values of pixel values. By using the statistical values of pixel values as a reference, important frames can be extracted with high accuracy while suppressing the computational cost.
[0021] In the present embodiment, the variance and the average value of pixel values are adopted as statistical values. Note that, in addition to the variance and the average value, for example, the median value or the like may be adopted as the statistical value. FIG. 2 shows a graph depicting the variance and the average value of pixel values of each frame.
[0022] The information processing apparatus 1 extracts one or a plurality of frames in which the statistical value takes an extreme value on the graph as frames to be focused on in the contrast video. In the present embodiment, a function called find_peaks in a library of Python (registered trademark) called scipy is used to identify the extreme points. Specifically, the information processing apparatus 1 sets the minimum distance (distance) between peaks, the peak width (width), and the prominence representing the height of the peak to "5", "2", and "0.6" respectively in the find_peaks function to identify the extreme points. Note that, when the difference in frame numbers of each identified extreme point is within 5, only the extreme point with the smaller frame number is left.
[0023] Note that, in the present embodiment, frames in which the statistical value takes an extreme value are extracted, but the present embodiment is not limited thereto. For example, the information processing apparatus 1 may extract only the frame in which the statistical value takes the maximum value. Thus, it is sufficient that the information processing apparatus 1 can extract specific frames according to the statistical values of pixel values.
[0024] Next, we will describe a method of constructing machine learning models (first model 51 and second model 52) that perform specific inference processing using frames extracted from contrast-enhanced video as training images.
[0025] Figure 3 shows an example of a screen that accepts the operation of assigning class labels. Figure 3 illustrates an example of a screen for assigning a predetermined class label to the frame extracted above. The screen includes a graph 31, stop / move buttons 32, frame image 33, first class label input field 34, second class label input field 35, and update button 36.
[0026] Graph 31, like Figure 2, is a graph showing the variance and mean value of the pixel values of each frame that makes up the contrast-enhanced video. As shown in Figure 3, the information processing device 1 displays Graph 31 with the extreme values 311 of the variance and mean value highlighted. If annotation (class label) has been assigned to the frame corresponding to the extreme value 311, the extreme value 311 is displayed as a black circle; if annotation has not been assigned, the extreme value 311 is displayed as a white circle. The information processing device 1 also displays a cursor 312 on Graph 31 indicating the position of the currently displayed frame image 33.
[0027] The stop / move button 32 is a button used to stop or move the cursor 312. When the information processing device 1 receives an input to the stop / move button 32, or receives an input to move the cursor 312 left or right by mouse operation, it moves the cursor 312 to the arbitrary extremum point 311. The information processing device 1 then displays a frame image 33 of the extremum point 311 that the cursor 312 is currently positioned at.
[0028] Frame image 33 is a display area that shows the frame at the position corresponding to the cursor 312. The information processing device 1 displays the frame of the extreme point 311 where the cursor 312 is located, and also displays the frame number, the average value of the pixel values, and the variance of that frame.
[0029] The first class label input field 34 is an input field that accepts input to select whether or not the frame image 33 is suitable as a highlight image. For example, as shown in Figure 3, the information processing device 1 displays three options in the first class label input field 34: "usable and easy to understand," "not unusable," and "unusable." The information processing device 1 accepts input to assign the correct highlight value to the frame by accepting input to select one of these options.
[0030] The second class label input field 35 is an input field that accepts input to select whether the coronary artery (biological site) shown in the frame image 33 is the left coronary artery or the right coronary artery. The user checks the frame image 33 and selects whether the coronary artery shown in the entire contrast-enhanced video is the left coronary artery or the right coronary artery. In response to the operation input to the second class label input field 35, the information processing device 1 assigns a label indicating whether it is the right coronary artery or the left coronary artery to all frames of the extreme point 311 extracted from the contrast-enhanced video.
[0031] The update button 36 is used to update the training data. When input is received for the update button 36, the information processing device 1 adds frames to the training data for machine learning via the first class label input field 34 and the second class label input field 35, to which class labels have been assigned.
[0032] As described above, the information processing device 1 accepts an input operation to assign a class label to each frame extracted from the contrast-enhanced video. This allows the information processing device 1 to generate training data for machine learning. Using this training data, the information processing device 1 generates the first model 51 and the second model 52.
[0033] Model 51 is a model that, when given a frame of a contrast-enhanced video as input, outputs a highlight level that indicates the degree to which the frame is suitable as a highlight image. For example, ConvNeXt, a type of CNN (Convolutional Neural Network), can be used as Model 51.
[0034] In this embodiment, the first model 51 is described as a CNN, but the first model 51 may be a neural network other than a CNN. Furthermore, the first model 51 may be a machine learning model other than a neural network, such as a decision tree or an SVM (Support Vector Machine). The same applies to the second model 52, which will be described later.
[0035] For example, Model 151 outputs one of three classification results for highlighting level: "Best" (usable and easy to understand), "Better" (not unusable), or "Other" (unusable). Alternatively, the problem addressed by Model 151 could be treated as a regression problem, and the highlighting level could be represented as a continuous value.
[0036] The information processing device 1 generates a first model 51 using the training data generated above. That is, the information processing device 1 generates the first model 51 by referring to the correct value of the highlight level assigned to each frame. The information processing device 1 estimates the highlight level by inputting the training frames into the first model 51 and optimizes parameters such as the weights between neurons so that the estimation result approximates the correct value. In this way, the information processing device 1 generates a first model 51 for determining whether the input frame is suitable as a highlight image.
[0037] The second model 52 is a model that, when given a frame of contrast-enhanced video as input, classifies whether the coronary artery shown in that frame is the right or left coronary artery. For example, as the second model 52, ConvNeXt, a type of CNN, can be used, similar to the first model 51.
[0038] The information processing device 1 generates a second model 52 using the training data generated above. Specifically, the information processing device 1 generates the second model 52 by referring to the correct label representing the type of coronary artery assigned to each frame. The information processing device 1 classifies the type of coronary artery by inputting the training frames into the second model 52 and optimizes parameters such as the weights between neurons so that the classification result matches the correct label. In this way, the information processing device 1 generates a second model 52 for classifying the type of coronary artery shown in the contrast-enhanced video.
[0039] Figure 4 shows an example of inference processing. Figure 4 schematically illustrates an example of inference processing using the first model 51 and the second model 52.
[0040] The information processing device 1 acquires a contrast-enhanced video from the imaging device 2, which is an image of the heart (coronary arteries) of a subject to which a contrast agent has been administered. As a preprocessing step for inference processing using the first model 51 and the second model 52, the information processing device 1 extracts specific frames from the contrast-enhanced video. Specifically, the information processing device 1 calculates statistical values (variance and mean) of the pixel values for each frame that makes up the contrast-enhanced video, and extracts one or more frames in which the statistical values take extreme values.
[0041] For example, the information processing device 1 inputs each extracted frame into the first model 51, outputting the highlight level of each frame. The information processing device 1 determines the frame with the "Best" highlight level as the highlight image. Note that in Figure 4, only one highlight image is shown as being determined to be "Best," but multiple frames may be determined to be "Best."
[0042] The information processing device 1 inputs the frame determined to be the highlight image into the second model 52, classifying whether the coronary artery shown in that frame is the right coronary artery or the left coronary artery. The information processing device 1 displays the highlight image and the classification result of whether it is the right or left coronary artery, and presents the image analysis results to the user.
[0043] The processing procedure described above is merely an example, and this embodiment is not limited thereto. For example, as shown by the dotted arrow in Figure 4, the frames extracted according to the statistical values may not be input into the first model 51, but instead directly into the second model 52 to classify the type of coronary artery.
[0044] As described above, according to this embodiment, by utilizing the statistical values of the pixel values of each frame constituting the contrast-enhanced video, specific frames of interest can be suitably extracted. Furthermore, by assigning class labels (labels representing highlight intensity and coronary artery type) to the extracted frames, training data for machine learning can be suitably generated. In addition, by inputting the extracted frames into a machine learning model (first model 51 or second model 52), video analysis can be performed on appropriate frames of the contrast-enhanced video.
[0045] Figure 5 is a flowchart showing the procedure for generating training data. Based on Figure 5, we will explain the process of generating training data for machine learning by accepting input for assigning class labels to frames extracted from contrast-enhanced video. The control unit 11 of the information processing device 1 acquires a contrast-enhanced video for training (step S11). The contrast-enhanced video is a video of a subject to which a contrast agent has been administered, and in this embodiment, it is a video of the subject's heart (coronary arteries).
[0046] The control unit 11 calculates statistical values of pixel values for each frame that makes up the acquired contrast-enhanced video (step S12). Specifically, the control unit 11 calculates the variance and average value of pixel values in each frame.
[0047] The control unit 11 extracts specific frames from the contrast-enhanced video based on the calculated statistical values (step S13). Specifically, the control unit 11 extracts one or more frames in a graph where the horizontal axis (first axis) represents the imaging order of the frames and the vertical axis (second axis) represents the statistical values of each frame, in which the statistical values take extreme values.
[0048] The control unit 11 displays a screen (see Figure 3) on the display unit 13 that includes a graph 31 showing the extreme points 311, 311, 311… corresponding to the extracted frames (step S14). The control unit 11 accepts an input to select an extreme point 311 (step S15). The control unit 11 displays a frame image 33 corresponding to the selected extreme point 311 on the screen (step S16).
[0049] The control unit 11 accepts an operation input to assign a class label to the displayed frame image 33 (step S17). Specifically, by accepting an operation input to the first class label input field 34, the control unit 11 accepts an operation input to assign a highlight degree, which represents the degree to which the displayed frame image 33 is appropriate as a highlight image. The control unit 11 also accepts an operation input to the second class label input field 35, which accepts an operation input to assign a label representing the type of coronary artery (biological site) to all frames of the contrast-enhanced video. In response to an operation input to the update button 36, the control unit 11 adds the frame images to the training data (step S18).
[0050] The control unit 11 determines whether or not to terminate the class label assignment process in response to user input (step S19). If it determines not to terminate (S19: NO), the control unit 11 returns to step S15. If it determines to terminate (S19: YES), the control unit 11 terminates the series of processes.
[0051] Figure 6 is a flowchart showing the steps involved in generating the first model 51 and the second model 52. Based on Figure 6, the process of generating the first model 51 and the second model 52 by training with training data will be explained. The control unit 11 reads out training data to which class labels have been assigned to a group of frames of the contrast-enhanced video used for training (step S31). As described above, the training data is data to which each frame of the contrast-enhanced video is associated with a highlight level, which represents the degree to which the frame is suitable as a highlight image, and a label that represents the type of blood vessel (biological part) shown in the frame.
[0052] The control unit 11 generates a first model 51 that outputs the highlight level when a frame of a contrast-enhanced video is input, based on the training data (step S32). Specifically, as described above, the control unit 11 generates a CNN as the first model 51. The control unit 11 estimates the highlight level by inputting training frames into the first model 51 and optimizes parameters such as the weights between neurons so that the estimation result approximates the correct value. In this way, the control unit 11 generates the first model 51.
[0053] Based on the training data, the control unit 11 generates a second model 52 that, when a frame of a contrast-enhanced video is input, classifies whether the blood vessel shown in that frame is the right coronary artery or the left coronary artery (step S33). Specifically, the control unit 11 generates a CNN as the second model 52. The control unit 11 inputs the training frame into the second model 52 to classify the blood vessel shown in that frame and optimizes parameters such as the weights between neurons so that the classification result matches the correct label. In this way, the control unit 11 generates the second model 52.
[0054] Figure 7 is a flowchart showing the procedure for the inference process. Based on Figure 7, the content of the inference process using the first model 51 and the second model 52 will be explained. The control unit 11 of the information processing device 1 acquires a contrast-enhanced video from the imaging device 2, which is an image of a subject who has been administered a contrast agent (step S51). The control unit 11 calculates statistical values (variance and mean) of the pixel values for each frame that makes up the acquired contrast-enhanced video (step S52). Based on the calculated statistical values, the control unit 11 extracts specific frames from the contrast-enhanced video (step S53). Specifically, the control unit 11 extracts one or more frames in which the statistical values take extreme values.
[0055] The control unit 11 inputs each extracted frame to the first model 51 and outputs a highlight level that indicates the degree to which each frame is suitable as a highlight image (step S54). Based on the output highlight level, the control unit 11 determines which frame will be used as the highlight image (step S55). The control unit 11 inputs the frame determined to be the highlight image to the second model 52 and classifies whether the blood vessel shown in that frame is the right coronary artery or the left coronary artery (step S56). The control unit 11 then completes the series of processes.
[0056] Based on the above, according to this embodiment, specific frames can be suitably extracted from contrast-enhanced video.
[0057] (Variation 1) In the above-described embodiment, a method was explained in which a highlight image is determined using a first model 51 constructed by machine learning. In this embodiment, a method is described in which a highlight image is determined from sharpness, which represents the degree of sharpness of the extreme points of the statistical values, without using the first model 51.
[0058] Figure 8 is a graph showing the variance of pixel values in each frame that makes up a contrast-enhanced video. In Figure 8, a magnified graph of the curve portion that takes a certain maximum value is schematically illustrated. This modified example will be explained based on Figure 8.
[0059] As described in the above embodiment, the information processing device 1 extracts extreme values by applying the find_peaks function to the graph curve showing the statistical values (variance in Figure 8) of the pixel values of each frame of the contrast-enhanced video. When using the find_peaks function, the prominence and peak width of the extreme value points can be calculated.
[0060] A prominence is the height of the peak or trough of a graph curve that gives an extremum, and is defined as the difference between the target extremum (maximum in Figure 8) and the preceding and succeeding extremums (minims). More specifically, if the target extremum is defined as the maximum, the prominence is defined as the difference between the target extremum and the largest of the preceding and succeeding minims.
[0061] Peak width is defined as the width of the peak or trough of a graph curve that gives an extremum. More specifically, peak width is defined as the half-width of a prominence (the width of the two points of tangency where the line bisecting the prominence touches the graph curve).
[0062] The definitions of prominence and peak width are not limited to those given above. For example, prominence may be defined as the difference between the smallest of the preceding and succeeding local minima. Similarly, peak width may be defined as the width of the base of the peak of the graph curve that gives an extremum.
[0063] For each frame extracted from the contrast-enhanced video, assuming that the statistical values are extreme, the information processing device 1 calculates the sharpness, which represents the degree of sharpness of each extreme point, based on the above-mentioned prominence and peak width. For example, the information processing device 1 calculates the sharpness as the value obtained by dividing the prominence by the peak width, as shown in the following formula (1).
[0064] Sharpness = Prominence / Peak Width …(1)
[0065] The information processing device 1 determines the highlight image based on the calculated sharpness. For example, the information processing device 1 determines the frame corresponding to the extreme point with the maximum sharpness as the highlight image. Alternatively, the information processing device 1 may determine the frame corresponding to the extreme point with sharpness equal to or greater than a threshold as the highlight image.
[0066] Based on the above, according to this modified example 1, the frame corresponding to the sharpest extreme point, that is, the frame where the statistical value changes rapidly, is determined as the highlight image. This makes it possible to suitably determine the highlight image without using the first model 51.
[0067] (Modification 2) In the above-described embodiment, a configuration was explained in which a first model 51 that outputs the degree of highlighting and a second model 52 that classifies the type of coronary artery (biological part) shown in the frame are constructed separately. In this modified example, a configuration in which both the determination of the degree of highlighting and the classification of coronary arteries are performed with a single model is described.
[0068] Figure 9 shows an example of the inference process related to Modification 2. This modification will be explained based on Figure 9.
[0069] Similar to the embodiment described above, the information processing device 1 extracts one or more frames from the contrast-enhanced video based on statistical values of the pixel values of each frame constituting the contrast-enhanced video. The information processing device 1 then determines a highlight image from the extracted frames and classifies the type of coronary artery visible in the highlight image.
[0070] In this modified example, the third model 53 is used instead of the first model 51 and the second model 52. The third model 53 is a model that, when a frame of contrast-enhanced video is input, outputs a highlight level representing the degree to which the frame is suitable as a highlight image, and a classification result that classifies the type of coronary artery shown in the frame. ConvNeXt, a type of CNN, can be used as the third model 53.
[0071] In the example screen shown in Figure 3, each frame extracted from the contrast-enhanced video is assigned a label indicating the degree of highlighting, such as "Best," "Better," or "Other," and a label indicating the type of coronary artery, such as "Right coronary artery" or "Left coronary artery." In this modified example, these labels are combined, and each frame is assigned a label such as "Best and Right Coronary Artery," "Best and Left Coronary Artery," and so on.
[0072] The information processing device 1 generates a third model 53 based on training data to which the above-mentioned integrated labels are assigned to each frame. The information processing device 1 inputs frames of the training contrast-enhanced video into the third model 53 to estimate the highlight intensity and labels representing the type of coronary artery. The information processing device 1 generates the third model 53 by optimizing parameters such as the weights between neurons so that the estimated results match the correct labels.
[0073] When performing inference processing, the information processing device 1 inputs frames extracted from the contrast-enhanced video to the third model 53, outputting labels representing the highlight intensity and the type of coronary artery. This allows the information processing device 1 to simultaneously determine the highlight image and classify the coronary arteries.
[0074] Based on the above, according to this embodiment 3, the first model 51 and the second model 52 can also be integrated.
[0075] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the present invention is indicated by the claims, not in the sense described above, and all modifications within the sense and scope equivalent to the claims are intended.
[0076] The matters described in each embodiment can be combined with each other. Furthermore, the independent and dependent claims described in the claims can be combined with each other in any combination, regardless of the form of reference. In addition, the claims use a form in which claims referencing two or more other claims (multi-claim form), but are not limited to this. A form in which multi-claims referencing at least one multi-claim (multi-multi-claim) may also be used. [Explanation of Symbols]
[0077] 1. Information Processing Device 11 Control Unit 12 Main memory 13 Display section 14 Communications Department 15 Input section 16 Auxiliary storage P Program 51 First Model 52 Second Model 53 Third Model 2. Imaging device
Claims
1. We obtain contrast-enhanced video, For each frame constituting the aforementioned contrast-enhanced video, statistical values of the pixel values are calculated. Based on the aforementioned statistical values, a specific frame is extracted from the contrast-enhanced video. An information processing method in which a computer performs the processing.
2. The aforementioned statistical value is the variance or mean of the pixel values. The information processing method according to claim 1.
3. In a graph where the first axis represents the imaging order of the frames and the second axis represents the statistical value of each frame, the frame in which the statistical value takes an extreme value is extracted. The information processing method according to claim 1.
4. The graph showing the extreme value points corresponding to the extracted frame is displayed on the display unit. When an input is received to select the aforementioned extreme point, the frame corresponding to the selected extreme point is displayed. The system accepts an input to assign a class label to the displayed frame. The information processing method according to claim 3.
5. For each of the extracted frames, the sharpness representing the degree of peaking at the extreme point is calculated based on the prominence of the extreme point on the graph corresponding to that frame and the width of the peaks or troughs of the curve of the statistical value that gives rise to the extreme point. Based on the sharpness, the frame to be used as the highlight image is determined. The information processing method according to claim 3.
6. The extracted frame is input to a model that has been trained to output the degree to which a frame is appropriate as a highlight image when the aforementioned frame is input, and the aforementioned degree is output. Based on the outputted degree, the frame to be used as the highlight image is determined. The information processing method according to claim 1.
7. The extracted frame is input to a model that has been trained to classify the biological parts shown in the frame when the aforementioned frame is input, thereby classifying the biological parts shown in the frame. The information processing method according to claim 1.
8. The aforementioned contrast-enhanced video is a video of the heart of a subject to which a contrast agent was administered. The aforementioned model classifies whether the coronary artery visible in the frame is the left coronary artery or the right coronary artery. The information processing method according to claim 7.
9. We obtain contrast-enhanced video, For each frame constituting the aforementioned contrast-enhanced video, statistical values of the pixel values are calculated. Based on the aforementioned statistical values, a specific frame is extracted from the contrast-enhanced video. A program that instructs a computer to perform a process.
10. An information processing device comprising a control unit, The control unit, We obtain contrast-enhanced video, For each frame constituting the aforementioned contrast-enhanced video, statistical values of the pixel values are calculated. Based on the aforementioned statistical values, a specific frame is extracted from the contrast-enhanced video. Information processing device.