Image processing apparatus, image processing method, and program

Abstract

To improve the accuracy of region extraction of an object for an image obtained by imaging the object.SOLUTION: An image processing apparatus according to the present disclosure comprises: an image acquisition unit that acquires an image obtained by imaging an object; a first extraction unit that extracts a first region corresponding to a region of the object using the image; a parameter acquisition unit that acquires parameters for normalizing the image on the basis of pixel values of pixels in the first region and contribution degrees indicating likelihood that the pixels are included the object; a normalization unit that acquires a normalized image obtained by normalizing the image by using the image and the parameters; and a second extraction unit that extracts the region of the object more precisely than the first region by using the normalized image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] Conventionally, images of an object such as a heart captured by an imaging device such as an ultrasound diagnostic device may be depicted with different pixel values ​​and contrast depending on the imaging conditions. Various methods have been proposed for extracting an object region from such images.

[0003] For example, in Patent Document 1, when extracting the left ventricular region from a cardiac ultrasound image taken by an ultrasound diagnostic device, anatomical landmarks such as the center of the left ventricle are first extracted. Then, pixel values ​​of pixels believed to be within the left ventricle are obtained from pixel values ​​around the extracted anatomical landmarks, and the pixel values ​​of the image are normalized based on these pixel values. This results in an image in which the pixel values ​​and contrast of the left ventricular region on the image are normalized to predetermined values ​​(normalized), and this image is used to extract the left ventricular region. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-073832 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the method described in Patent Document 1, feature points within an object are extracted, and the image is normalized based on the pixel values ​​of pixels that are deemed to be within the region based on the pixel values ​​around the extracted feature points, and the region of the object is extracted. As a result, pixels that are deemed to be within the region may include pixels in an area that is inappropriate for the object. As a result, the image may not be normalized properly, and the region may not be extracted accurately.

[0006] The technology of the present disclosure has been made in view of the above, and aims to improve the accuracy of extracting an object region from an image of the object. [Means for solving the problem]

[0007] The image processing device according to the present disclosure includes an image processing device characterized by having an image acquisition unit that acquires an image of an object, a first extraction unit that uses the image to extract a first region corresponding to the region of the object, a parameter acquisition unit that acquires parameters for normalizing the image based on pixel values ​​of pixels in the first region and a contribution indicating the probability that the pixel is included in the object, a normalization unit that uses the image and the parameters to acquire a normalized image by normalizing the image, and a second extraction unit that uses the normalized image to extract the region of the object more precisely than the first region. In addition, the image processing device according to the present disclosure includes an image processing device characterized by having an image acquisition unit that acquires an image of an object, a first extraction unit that uses the image to extract a region of the object as a first region, a parameter acquisition unit that acquires parameters for normalizing the image based on a partial region that is part of the first region, a normalization unit that uses the image and the parameters to acquire a normalized image by normalizing the image, and a second extraction unit that extracts the region of the object using the normalized image.

[0008] An image processing method according to the present disclosure includes the steps of: acquiring an image of an object; extracting a first region corresponding to a region of the object using the image; and extracting the first region. and a step of extracting a region of the object more precisely than the first region using the normalized image. The image processing method according to the present disclosure also includes a step of acquiring an image of an object, a step of extracting the region of the object as a first region using the image, a step of acquiring parameters for normalizing the image based on a partial region that is a part of the first region, a step of normalizing the image using the image and the parameters to obtain a normalized image by normalizing the image, and a step of extracting the region of the object using the normalized image. [Effects of the Invention]

[0009] According to the technology of the present disclosure, it is possible to improve the accuracy of extracting an object region from an image of the object. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing a schematic configuration of an image processing apparatus according to a first embodiment; [Figure 2] 1 is a flowchart of a process executed by an image processing apparatus according to a first embodiment; [Figure 3] FIG. 1 is a diagram showing an example of an image acquired by the image processing apparatus according to the first embodiment; [Figure 4] FIG. 10 is a block diagram showing a schematic configuration of an image processing apparatus according to a second embodiment. [Figure 5] 10 is a flowchart of a process executed by an image processing apparatus according to a second embodiment. [Figure 6] FIG. 10 is a diagram showing an example of an image of an extraction region acquired by an image processing apparatus according to a second embodiment; [Figure 7] FIG. 10 is a block diagram showing a schematic configuration of an image processing apparatus according to a third embodiment. [Figure 8] 10 is a flowchart of a process executed by an image processing apparatus according to a third embodiment. [Figure 9] FIG. 11 is a diagram showing an example of an image of an extraction region acquired by an image processing apparatus according to a third embodiment; [Figure 10] FIG. 10 is a block diagram showing a schematic configuration of an image processing apparatus according to a fourth embodiment. [Figure 11] 10 is a flowchart of a process executed by an image processing apparatus according to a fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of an image processing device disclosed in this specification will be described with reference to the drawings. The same or equivalent components, members, and processes shown in each drawing will be assigned the same reference numerals, and duplicated descriptions will be omitted as appropriate. In addition, some of the components, members, and processes will be omitted as appropriate in each drawing.

[0012] First Embodiment The image processing device according to the first embodiment is a device that acquires an image depicting an object as an input image and extracts a region of the object from the input image. The image processing device according to this embodiment takes as input an ultrasound image of a heart captured in a cardiac examination (cardiac ultrasound examination) using an ultrasound diagnostic imaging device, and extracts the region of the right ventricle from the input image, with the right ventricle being the object.

[0013] The configuration and processing of the image processing device of this embodiment will be described below with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the configuration of an image processing system (also called a medical image processing system) including the image processing device of this embodiment. The image processing system 1 includes an image processing device 100 and a database 22.

[0014] The image processing device 100 is communicably connected to a database 22 via a network 21. The network 21 includes, for example, a LAN (Local Area Network) or a WAN (Wide Area Network).

[0015] The database 22 holds and manages a plurality of images and information. The information managed by the database 22 includes images (input images, images to be processed) input to the image processing device 100 and information (model structures and parameters) of each estimator used during extraction by the coarse extraction unit 42 and the precise extraction unit 45 described below. Note that the information on the estimators may be stored in an internal memory (ROM 32 or storage unit 34) of the image processing device 100 instead of in the database 22. The image processing device 100 can acquire data held in the database 22 via the network 21.

[0016] The image processing device 100 includes a communication IF (Interface) 31, a ROM (Read Only Memory) 32, a RAM (Random Access Memory) 33, a storage unit , an operation unit 35, a display unit , and a control unit .

[0017] The communication IF 31 is configured with a LAN card or the like, and is a communication unit that realizes communication between an external device (e.g., the database 22) and the image processing device 100. The ROM 32 is configured with a non-volatile memory or the like, and stores various programs and various data. The RAM 33 is configured with a volatile memory or the like, and is used as a work memory that temporarily stores programs and data currently being executed. The storage unit 34 is configured with an HDD (Hard Disk Drive) or the like, and stores various programs and various data. The operation unit 35 is configured with a keyboard, mouse, touch panel, etc., and inputs instructions from a user (e.g., a doctor or a medical technician) to various devices. The display unit 36 ​​is configured with a display or the like, and displays various information to the user.

[0018] The control unit 40 is configured from a CPU (Central Processing Unit) or a dedicated or general-purpose processor. The control unit 40 may be configured from a GPU (Graphic Processing Unit) or FPGA (Field-Programmable Gate Array). The control unit 40 may also be configured from an ASIC (Application Specific Integrated Circuit) or the like. The control unit 40 includes an input image acquisition unit 41, a rough extraction unit 42, a normalization parameter acquisition unit 43, a normalization unit 44, and a precise extraction unit 45, which will be described later.

[0019] The input image acquisition unit 41 acquires an image to be processed (input image) to be input to the image processing device 100 from the database 22 or the storage unit 34. Note that the input image acquisition unit 41 may acquire the input image directly from a modality such as an ultrasound imaging diagnostic device, and in this case, the image processing device 100 may be implemented in the modality as part of its functions.

[0020] The rough extraction unit 42 is a first extraction unit that extracts an area of ​​an object to be extracted from the input image acquired by the input image acquisition unit 41, using a first estimator acquired from the database 22 or the storage unit 34. The area extracted by the rough extraction unit 42 is a rough extraction result of the area of ​​the object from the input image.

[0021] Normalization parameter acquisition unit 43 is a contribution acquisition unit that acquires pixel values ​​of pixels based on the object region, which is the first region extracted by rough extraction unit 42, and acquires a contribution degree that indicates the magnitude of the influence (contribution) of the pixel on the calculation of the normalization parameter. Then, normalization parameter acquisition unit 43 acquires the normalization parameter to be used in normalization unit 44, which will be described later, by calculating the normalization parameter based on the pixel value and the contribution degree. Normalization parameter acquisition unit 43 is configured to acquire the contribution degree for at least pixels corresponding to the object region, which is the first region.

[0022] The normalization unit 44 uses the normalization parameters acquired by the normalization parameter acquisition unit 43 to Then, the pixel values ​​of the input image are normalized, and a normalized image is generated using the normalized pixel values.

[0023] The precise extraction unit 45 is a second extraction unit that precisely extracts the region of the object to be extracted from the normalized image generated by the normalization unit 44, using a second estimator acquired from the database 22 or the storage unit 34. The precise extraction unit 45 is a specific example of an estimation unit that performs estimation on the normalized image.

[0024] Next, each step of the image processing method in one example of processing executed by the image processing device 100 of this embodiment will be described in detail using the flowchart in Fig. 2. As one example, the control unit 40 of the image processing device 100 starts the following processing based on a user input from the operation unit 35.

[0025] (Step S110: Obtaining input image) In step S110, the input image acquisition unit 41 acquires an input image specified by the user via the operation unit 35 from the database 22, and stores the acquired input image in the storage unit 34. Fig. 3A shows an example of an input image 1001 acquired by the input image acquisition unit 41 in step S110. The input image 1001, which is an apical four-chamber view, depicts the subject's right ventricle 1010, right atrium 1020, left ventricle 1030, and left atrium 1040.

[0026] In this embodiment, the input image acquisition unit 41 acquires, as input images, cardiac ultrasound images of the subject captured during a cardiac ultrasound examination from the database 22. The input image acquisition unit 41 may acquire input images by a method other than acquisition from the database 22. For example, the input image acquisition unit 41 may be configured to sequentially acquire, as input images, cardiac ultrasound images captured by an ultrasound diagnostic device from time to time. The control unit 40 may also display the input images acquired in step S110 on the display unit 36.

[0027] (Step S120: Rough extraction of object region) In step S120, the rough extraction unit 42 acquires from the storage unit 34 a first estimator for estimating an object region in the image using the input image acquired in step S110 as an input. Then, the rough extraction unit 42 inputs the input image acquired in step S110 to the first estimator, and extracts the object region to be extracted from the input image by rough extraction. In this embodiment, the roughly extracted region may be referred to as a first region.

[0028] Here, the process of rough extraction executed in this step will be described. In this embodiment, the estimator for extracting the right ventricle region uses a convolutional neural network (CNN), which is one of the machine learning models based on deep learning. The method uses a technique based on a neural network (CNN). That is, the relationship between the cardiac ultrasound image and the right ventricular region within the cardiac ultrasound image is learned in advance using CNN. The rough extraction unit 42 then uses the trained CNN to calculate a likelihood, between 0 and 1, for each pixel in the cardiac ultrasound image, representing the probability that the pixel is in the right ventricular region. A fixed threshold value, such as 0.5, is used, and pixels with a likelihood equal to or greater than the threshold are designated as a roughly extracted region (first region) of the right ventricle. Therefore, in this embodiment, a region with a likelihood equal to or greater than 0.5 can be designated as a region overlapping with an object region extracted by rough extraction, or as a region within the object region extracted by rough extraction. The likelihood, which represents the probability that the pixel of interest in the cardiac ultrasound image is included in the right ventricular region, can be said to indicate the degree of probability that the pixel of interest in the cardiac ultrasound image is included in the right ventricular region. That is, when the likelihood representing the probability that a pixel of interest in a cardiac ultrasound image is included in the right ventricle region is equal to or greater than a predetermined threshold, this can be rephrased as meaning that the pixel of interest in the cardiac ultrasound image is likely to be included in the right ventricle region at a predetermined probability or greater. The pixel of interest may be considered to be included in the object region in various ways, including a case where the pixel of interest has coordinates corresponding to the pixel of interest, and a case where the pixel of interest has coordinates corresponding to the periphery that forms the boundary between the inside and outside of the object. Including the pixel of interest in the object also means that the coordinates of the pixel of interest are included in a coordinate group (data set) of the internal region of the object that includes the periphery of the object. In step 120, a processed image obtained by adjusting the resolution, angle of view, gradation, etc., and performing image processing using additional information on an input image from a server such as a modality or PACS may be used as the input image to be input to the first estimator.

[0029] In this embodiment, the rough extraction unit 42 acquires a pre-trained estimator from the storage unit 34, but the estimator may also be acquired by performing the above-mentioned learning process in the image processing device 100. That is, the image processing device 100 may acquire multiple sets of cardiac ultrasound images and corresponding correct region data indicating the right ventricular region, construct an estimator for estimating the right ventricular region, and the rough extraction unit 42 may acquire the constructed estimator.

[0030] 3B shows an example of a right ventricle region 1011, which is an object region extracted by the rough extraction unit 42 in step S120. The right ventricle region 1011 extracted by the rough extraction unit 42 is a region that overlaps with the right ventricle 1010 and the surrounding area of ​​the right ventricle 1010 in the input image 1001.

[0031] (Step S130: Obtaining contribution rate) In step S130, normalization parameter acquisition unit 43 acquires pixel values ​​of pixels in the object region extracted by rough extraction in step S120. Then, normalization parameter acquisition unit 43 calculates and acquires a contribution degree that indicates the magnitude of contribution that each pixel makes to the calculation of the normalization parameter in step S140, which will be described later.

[0032] Here, the contribution degree acquired in this step will be described. In this embodiment, for pixels included in the right ventricular region, which is the object region extracted by rough extraction in step S120, a contribution degree is acquired in the range of 0 to 1 based on the likelihood calculated by the first estimator. This allows for a contribution degree corresponding to the probability that the pixel of interest in the input image is included in the right ventricular region. Furthermore, the contribution degree in this embodiment is acquired for at least the right ventricular region extracted by the rough extraction unit 42. Specifically, a threshold value is set in advance, and the contribution degree can be configured as a binary value of 1 or 0 by setting the contribution degree of pixels whose likelihood calculated by the first estimator is equal to or greater than the threshold to 1 and the contribution degree of pixels whose likelihood is less than the threshold to 0.

[0033] In this embodiment, the contribution is determined using the likelihood value of pixels in the right ventricle region extracted by rough extraction. However, the contribution may be determined using any method, such as a linear function, a nonlinear function, or a function that can be expressed in the form of y = f(x) as a projection in which the contribution y increases as the likelihood x increases. Furthermore, the contribution is not limited to a function that monotonically increases as the likelihood increases, and may be changed as appropriate, such as to be a constant within a predetermined likelihood range.

[0034] Furthermore, in this embodiment, the contribution degree is determined using the likelihood value of pixels in the right ventricular region extracted by rough extraction, but the contribution degree of pixels that deviate from the average pixel value of the pixels in the extracted right ventricular region may be set low, and the contribution degree of pixels that are close to the average may be set high. For example, the contribution degree of each pixel may be calculated in the range of 0 to 1 according to the value of a probability density function in a normal distribution based on the average pixel value and standard deviation of the pixel values ​​in the right ventricular region extracted by rough extraction. In this case, the contribution degree may be determined by additionally performing a cutoff process, such as setting the contribution degree to 0 for pixels whose difference from the average pixel value is equal to or greater than a constant multiple (e.g., 3 times) the standard deviation value.

[0035] (Step S140: Obtaining normalization parameters) In step S140, the normalization parameter acquisition unit 43 receives the normalization parameter acquired in step S130. Using the pixel value and contribution of each pixel, a normalization parameter for normalizing the input image is calculated and acquired.

[0036] Here, the normalization parameters acquired in this step will be described. In this embodiment, the pixel values ​​of each pixel in the right ventricle region extracted by rough extraction acquired in step S130 are weighted by contribution and statistical values ​​of the pixel values ​​are calculated. Here, as an example of the statistical values, the mean value μ and standard deviation value σ are calculated. Specifically, the mean value μ and standard deviation value σ are calculated using the following equations (1) and (2). These values ​​are then acquired as normalization parameters. Here, i and j are indexes representing the position of a pixel in the input image, and represent the position of the pixel in the i-th row and j-th column of the input image. Furthermore, w represents the contribution of the pixel acquired in step S130, and v represents the pixel value of the pixel. Note that in this embodiment, a case where the mean value and standard deviation value of the pixel values ​​of each pixel in the input image are calculated will be described as an example of statistical values, but a variance value may be used instead of the standard deviation value.

[0037]

number

[0038]

number

[0039] Note that, when the contribution of each pixel in the right ventricle region acquired in step S130 is composed of two values, 1 or 0, the processing in this step corresponds to processing for calculating the average value and standard deviation value using only pixels whose contribution value is 1. In other words, the processing in this step can be replaced by processing for calculating the average value and standard deviation value by limiting it to pixels whose contribution value is 1.

[0040] In the present embodiment, as an example of calculating the normalization parameter using the contribution degree, an example has been shown in which the contribution degree is used as a weight when calculating the average value and standard deviation value of pixel values ​​of the right ventricle region, but the embodiment of the present invention is not limited to this. For example, the contribution degree may be used as a criterion for determining pixels to be used in calculating the normalization parameter. Specifically, the normalization parameter acquisition unit 43 may calculate the maximum and minimum pixel values ​​for pixels whose contribution degree acquired in step S130 is equal to or greater than a predetermined value, and acquire these as the normalization parameter.

[0041] (Step S150: Normalizing the input image) In step S150, the normalization unit 44 normalizes the pixel values ​​of the input image using the normalization parameters acquired in step S140 to acquire a normalized image.

[0042] Here, the normalized image acquired in this step will be described. In this embodiment, normalization processing is performed to convert the pixel values ​​of all pixels in the input image using the normalization parameters acquired in step S140, that is, the average value and standard deviation value of the pixel values ​​of the right ventricle region, so that the average value and standard deviation value of the pixel values ​​of the right ventricle region become preset values. For example, the preset values ​​may be set to 0 for the average value and 1 for the standard deviation value. In other words, pixel value conversion processing is performed to convert the statistical value of the pixel values ​​of the right ventricle region of the input image into a predetermined statistical value. Specifically, the normalization unit 44 performs pixel value shifting and scaling processing.

[0043] Here, the mean value and standard deviation value of the normalized pixels are roughly set to 0 and 1, respectively. Although the present invention is described by way of example with respect to a case where conversion is performed so that the average and standard deviation values ​​become values ​​set in advance, the present invention is not limited to this. For example, the normalization unit 44 calculates reference values ​​for the average and standard deviation values ​​calculated by performing the processes of steps S120, S130, and S140 on a reference image different from the input image. The normalization unit 44 can then perform conversion so that the average and standard deviation values ​​of the pixels of the input image become the calculated reference values.

[0044] In the above description, an example was given in which the statistical values ​​of pixel values ​​of an input image are converted into predetermined statistical values, but the conversion may be such that the statistical values ​​of pixel values ​​of the input image approach the predetermined statistical values. Specifically, the normalization unit 44 performs a process of rounding the pixel values ​​of the image after converting the statistical values ​​of pixel values ​​of the input image into predetermined statistical values ​​to 8-bit discretized values. In this way, the normalization unit 44 performs pixel value conversion processing that approaches the statistical values ​​of pixel values ​​of the input image to the predetermined statistical values.

[0045] In this embodiment, the normalization parameters are calculated using the above equations (1) and (2) to calculate the mean and standard deviation of pixel values ​​in the right ventricle region, and the pixel values ​​are converted to match preset values. However, the present invention is not limited to this example. For example, the normalization unit 44 may calculate either the mean or the standard deviation as a normalization parameter and convert the pixel values ​​to match the preset value. For example, if the mean of the pixel values ​​is used as the normalization parameter, the normalization unit 44 may perform a conversion process to shift the pixel values ​​so that the normalized value is equal to or close to the preset value. Alternatively, if the standard deviation of the pixel values ​​is used as the normalization parameter, the normalization unit 44 may perform a conversion process to scale the pixel values ​​so that the normalized value is equal to or close to the preset value. In either case, the process can be simplified compared to using both the mean and standard deviation. The case where the mean or standard deviation of the pixel values ​​is approximately equal to the preset value may also be included in the case where the mean or standard deviation is equal to the preset value.

[0046] Furthermore, when the maximum and minimum pixel values ​​are acquired as normalization parameters in step S140, the normalization unit 44 may convert the pixel values ​​using the acquired maximum and minimum values ​​so that the pixel values ​​of the right ventricle region fall within a certain pixel value range. Specifically, the normalization unit 44 linearly converts the pixel values ​​of the input image so that the maximum pixel value of the right ventricle region becomes the upper limit value of a predetermined pixel value range and the minimum pixel value becomes the lower limit value of the predetermined pixel value range. This has the advantage of realizing a more robust normalization process for the result of the rough region estimation in step S120, such as for missing (overlooked) regions. Furthermore, the normalization unit 44 may use the upper and lower limit values ​​of a certain range of pixel value distribution as normalization parameters, instead of the maximum and minimum pixel values. Specifically, the normalization unit 44 may acquire a pixel value at a predetermined percentile when the pixel values ​​are sorted in descending order and a pixel value at a predetermined percentile when the pixel values ​​are sorted in ascending order.

[0047] In addition, the normalization unit 44 may perform pixel value conversion processing to equalize the histogram of pixel values ​​of pixels whose contribution degree is 1. Specifically, the normalization unit 44 calculates the value v(u) obtained by dividing the cumulative frequency for each pixel value u of pixels whose contribution degree is 1 by the number of pixels whose contribution degree is 1, and the minimum value v of the pixel values ​​of pixels whose contribution degree is 1. min is calculated as a normalization parameter. Then, the normalization unit 44 uses the calculated normalization parameter to equalize the histogram of pixel values ​​according to the following equation (3). Note that L in equation (3) represents the number of gradations of the pixel value, and for example, L is 256 in the case of 8 bits.

[0048]

number

[0049] 3C shows an example of a normalized image 1002 acquired by the normalization unit 44 in step S150. By the above-described normalization process, the right ventricle 1100, right atrium 1200, left ventricle 1300, and left atrium 1400 of the subject are depicted more precisely in the normalized image 1002 than the parts of the heart depicted in the input image 1001, and the right ventricle 1100 is extracted more precisely.

[0050] (Step S160: Precise extraction of object region) In step S160, the precise extraction unit 45 acquires from the storage unit 34 a second estimator for estimating the region of the object using the normalized image as input. Then, the precise extraction unit 45 inputs the normalized image acquired in step S150 to the second estimator, and extracts the region of the object to be extracted. In step S160, the region of the object extracted by the second estimator is the same as in step S120, but differs from step S120 in that the image normalized by the method of step S150 is used as input and the region of the object is extracted more accurately than the rough extraction described above.

[0051] Here, the extraction process of precise extraction executed in this step will be described. In this embodiment, the image processing device 100 uses CNN to learn in advance the relationship between the normalized cardiac ultrasound image and the right ventricular region within the cardiac ultrasound image, as in step S120. Then, using the trained CNN, the precise extraction unit 45 calculates a likelihood between 0 and 1 for each pixel in the cardiac ultrasound image, representing the probability that it is the right ventricular region, and sets a certain value, such as 0.5, as a threshold value to determine that pixels equal to or greater than the threshold are the extracted region of the right ventricle.

[0052] In this embodiment, cardiac ultrasound images with the same resolution are used as input images in steps S120 and S160. However, a low-resolution image with reduced resolution may be used as the input image in step S120, and a high-resolution image may be used as the input image in step S160 to extract the right ventricular region.

[0053] Although the above description exemplifies the case where the right ventricular region is extracted by precise extraction in step S160, the present invention is not limited to this example, and other anatomical structures may be extracted. For example, the precise extraction unit 45 may extract the position of the tricuspid valve adjacent to the right ventricular region or the apex of the heart, which is part of the right ventricular region. Furthermore, the precise extraction unit 45 is not limited to extracting anatomical regions; it may also extract abnormal regions or foreign bodies in the input image using a normalized image. Abnormal regions in the input image include tumor images, artificial objects such as placed stents, surgical scars, and artifacts resulting from imaging or reconstruction, as well as regions with specific characteristics in image findings and image feature regions that cannot be completely differentiated.

[0054] In addition to the above, the present invention can also be implemented when classifying input images. For example, the precise extraction unit 45 may execute a process to estimate the presence or absence of a disease, such as a morphological abnormality in the right ventricular region, and the type of disease. More specifically, the precise extraction unit 45 may execute a process to determine the presence or absence of right ventricular hypertrophy or valvular insufficiency. In other words, any estimation process using a normalized image is also included in the processing performed by the precise extraction unit 45 of this embodiment.

[0055] 3D shows an example of a right ventricle region 1012 extracted by the precise extraction unit 45 in step S160. The precise extraction unit 45 inputs to the estimator a normalized image 1002 in which each part of the heart is depicted more clearly than in the input image 1001. As a result, the right ventricle region 1012 extracted by the precise extraction unit 45 is a region whose shape is closer to the right ventricle 1010 in the input image 1001 than the right ventricle region 1011 extracted by the coarse extraction unit 42. In other words, the precise extraction unit 45 extracts the region of the object more precisely by using the normalized image from the coarse extraction unit 42.

[0056] The image processing device 100 of this embodiment executes the processing described above. This enables the image processing device 100 to accurately extract the right ventricular region, which is the extraction target, from a cardiac ultrasound image. Furthermore, by extracting the right ventricular region by coarse extraction and normalizing pixel values ​​using the contribution of pixels based on the extracted region, it is possible to reduce variations in pixel values ​​and contrast of the right ventricular region even when an image obtained under different imaging conditions is used as an input image. Then, by re-extracting the right ventricular region using a normalized image in which variations in pixel values ​​and contrast of the right ventricular region have been reduced (extraction by precise extraction), it is expected that a decrease in the extraction accuracy of the right ventricular region can be suppressed.

[0057] Next, a modified example of the above embodiment will be described. In the following description, the same components and processes as those of the image processing device 100 will be denoted by the same reference numerals, and detailed description thereof will be omitted.

[0058] (Variation 1-1) In the first embodiment, the image processing device 100 assumes that an input image is a cardiac ultrasound image of a subject captured by a cardiac ultrasound examination, and extracts a region of the right ventricle from the input image. However, the processing of the above embodiment can also be performed when an image of an organ other than the heart or an image captured by another modality is used as the input image.

[0059] In this modification, an example of applying the above embodiment to images obtained by other modalities is a case where a CT image is used as the input image. Another example is a case where an image of an organ other than the heart, such as the lungs, is used as the input image. Specifically, the input image acquisition unit 41 acquires a CT image of the lungs as the input image. Then, the coarse extraction unit 42 extracts a lung region from the acquired CT image by coarse extraction using a first estimator. The normalization parameter acquisition unit 43 acquires the likelihood calculated by the first estimator for the pixels in the extracted lung region as contributions. Then, the normalization parameter acquisition unit 43 calculates the average value and standard deviation value of the pixel values ​​of the pixels in the lung region as normalization parameters based on the acquired contributions. The normalization unit 44 normalizes the pixel values ​​of the CT image, which is the input image, using the normalization parameters. Then, the precise extraction unit 45 extracts the lung region from the normalized CT image by precise extraction.

[0060] The contribution can be calculated by: if the pixel value of a pixel in the lung region extracted by rough extraction is close to the statistical or theoretical value of the CT value of a typical lung region, the contribution of that pixel is calculated as high; if the pixel value deviates from that value, the contribution is calculated as low. This allows the contribution of pixels in a certain region to be calculated as low if the CT value of that region deviates from the actual CT value of the lung due to metal artifacts or the like in the input image. As a result, the normalization unit 44 can perform normalization based on pixels that indicate the actual CT value of the lung region. The modalities, regions, etc., to be processed in this modified example are not limited to the above examples.

[0061] As described above, the image processing device 100 according to this modification can extract a desired region from an input image with higher accuracy even for modalities other than cardiac ultrasound images and target regions other than the heart.

[0062] (Variation 1-2) In the first embodiment, it is assumed that the image processing device 100 extracts the area of ​​the right ventricle from an input cardiac ultrasound image of a subject captured by a cardiac ultrasound examination. However, the processing of the above embodiment can also be performed when an image other than a medical image is used as the input image.

[0063] In this modification, an example of applying the above embodiment to images other than medical images is when an image captured by a camera is used as an input image. Specifically, a case in which a human face region is extracted from an image of a person captured by a camera is exemplified. The input image acquisition unit 41 acquires the image captured by the camera as the input image. The rough extraction unit 42 extracts a human face region from the input image by rough extraction using a first estimator. The normalization parameter acquisition unit 43 acquires the likelihood calculated by the first estimator for each pixel in the extracted face region as a contribution degree. Then, the normalization parameter acquisition unit 43 calculates the average value and standard deviation value of the pixel values ​​of the pixels in the face region as normalization parameters based on the acquired contribution degrees. The normalization unit 44 normalizes the input image using the normalization parameter. Then, the precise extraction unit 45 extracts a face region from the normalized image by precise extraction. Here, the face region may be extracted by both rough extraction and precise extraction, or the extraction target may be changed. For example, the rough extraction may extract the entire face area, and the precise extraction may extract only parts of the face, such as the eyes, nose, and mouth.

[0064] As described above, according to the image processing device 100 of this modification, by applying the above-mentioned processing to an image other than a medical image as an input image, it is possible to extract a desired region from the image with higher accuracy.

[0065] (Variation 1-3) In the first embodiment, the image processing device 100 uses an estimator based on deep learning, such as a CNN, as an estimator. However, the estimator used in the above embodiment is not limited to this. For example, an estimator other than a CNN based on deep learning, such as a vision transformer, may be used. In this case, the calculated contribution degree can be used in the above processing to perform the processing of the above embodiment.

[0066] Alternatively, an estimator based on a known technique other than deep learning, such as Random Forest or Adaboost, may be used. Even in this case, the contribution level described in the first embodiment can be used to extract a region using the estimator. In this case, the region is considered to be pixels surrounded by a contour point cloud, and the estimator determines whether the input point cloud coordinates are appropriate as contour point cloud coordinates (i.e., whether they are positive or negative). Then, multiple parameters are input to the estimator, and candidates determined to be appropriate as contour point clouds are adopted. The region surrounded by the point clouds determined by the estimator to be appropriate as contour point clouds is determined to be the region extracted by rough extraction, and the contribution level is calculated so that the contribution level is higher toward the inside of the region from the contour point cloud and lower toward the outside (positions closer to the contour point cloud or outside the region). Thus, according to this modification, the image processing device 100 can perform the processing of the above embodiment even when an estimator based on a technique other than deep learning is used for region extraction.

[0067] (Variation 1-4) In the first embodiment, it is assumed that the rough extraction unit 42 inputs an input image to an estimator and extracts a region within the image, but the processing of the above embodiment is not limited to this. For example, it is also possible for the user to input information about the region to be extracted using the operation unit 35, and for the rough extraction unit 42 to extract the region based on the input information and the input image.

[0068] More specifically, the control unit 40 displays the cardiac ultrasound image of the input image acquired by the input image acquisition unit 41 on the display unit 36. Then, the user operates the operation unit 35 (such as a touch panel) to manually specify the contour point cloud of the right ventricle or landmark points as information about the right ventricle region. The rough extraction unit 42 inputs the information about the right ventricle region input by the user and the cardiac ultrasound image to the first estimator, and acquires the right ventricle region extracted by rough extraction from the input image. The subsequent processing is the same as in the first embodiment.

[0069] As a result, according to the image processing device 100 of this modified example, instead of inputting only the image to the estimator, the information on the area to be extracted input by the user is also used as input to the estimator, thereby making it possible to extract the desired area from the input image with greater accuracy.

[0070] (Variation 1-5) In the first embodiment, the normalization unit 44 normalizes the pixel values ​​of the input image using the normalization parameter to obtain a normalized image, but the processing in the above embodiment is not limited to this. For example, the user may input information about the region extracted by rough extraction and information about the likelihood of the region, and the normalization unit 44 may normalize the input image based on the information input by the user.

[0071] A specific example is shown below. First, the control unit 40 displays an ultrasound cardiac image of the input image acquired by the input image acquisition unit 41 on the display unit 36. Then, the user operates the operation unit 35 (such as a touch panel) to manually designate the region of the right ventricle as the region to be extracted by rough extraction. The user also inputs the degree of certainty that the designated region of the right ventricle is the right ventricle. Here, the normalization parameter acquisition unit 43 acquires the likelihood of each pixel, setting the likelihood of a pixel for which the user has determined a high degree of certainty to 1 and the likelihood of a pixel for which the user has determined a low degree of certainty to 0.5, for example. Then, the normalization parameter acquisition unit 43 calculates the contribution of each pixel based on the acquired likelihood, and acquires a normalization parameter based on this. The subsequent processing is the same as in the first embodiment.

[0072] As a result, according to the image processing device 100 of this modified example, pixel values ​​can be normalized based on the likelihood of the right ventricle region using information on the region extracted by rough extraction input by the user instead of extracting the region using an estimator by the rough extraction unit 42.

[0073] Second Embodiment Next, an image processing device according to a second embodiment will be described. In the following description, the same configurations and processes as those of the image processing device according to the first embodiment will be denoted by the same reference numerals, and detailed description thereof will be omitted.

[0074] As with the first embodiment, the image processing device according to the second embodiment uses an image depicting an object as an input image and extracts an object region from the input image. The image processing device 100 according to the first embodiment calculates the contribution degree based on the likelihood of each pixel based on the object region extracted by rough extraction. On the other hand, the image processing device according to the present embodiment reduces the region extracted by rough extraction and calculates the contribution degree based on the pixels of the reduced region.

[0075] In this embodiment, as in the first embodiment, an ultrasound cardiac image of a subject captured by echocardiography is used as an input image, and the right ventricle in the input image is used as the object to extract the region of the right ventricle.

[0076] The configuration of the image processing device of this embodiment will be described below with reference to Fig. 4. As shown in Fig. 4, the image processing system 2 includes an image processing device 200 and a database 22. The image processing device 200 according to this embodiment differs from the image processing device 100 according to the first embodiment in that it includes an area deformation unit 51 and a normalization parameter acquisition unit 52. The processing of each processing unit of the image processing device 200 will be described below, focusing on the differences from the processing in the first embodiment.

[0077] The area transformation unit 51 is a transformation area acquisition unit that performs transformation processing such as shrinking or enlarging the object area roughly extracted by the rough extraction unit 42, thereby acquiring a transformation area of ​​the object area.

[0078] The normalization parameter acquisition unit 52 acquires pixel values ​​of pixels based on the deformation area acquired by the area deformation unit 51, and calculates a contribution degree that indicates the magnitude of the influence (contribution) of the pixel in question on the normalization performed by the normalization unit 44. Then, the normalization parameter acquisition unit 52 calculates and acquires normalization parameters to be used by the normalization unit 44, using the acquired pixel values ​​and the calculated contribution degrees.

[0079] Next, an example of processing executed by the image processing device 200 according to this embodiment will be described in detail using the flowchart in Fig. 5. As shown in Fig. 5, the processing executed by the image processing device 200 is the same as the processing in the flowchart of the first embodiment shown in Fig. 2 except that step S210 for modifying the region extracted by rough extraction is added, and the processing content of step S220 corresponding to step S130 is different.

[0080] (Step S210: Obtaining an area obtained by modifying the roughly extracted area) In step S210, the area modification unit 51 reduces or enlarges the object area extracted by rough extraction in step S120, thereby obtaining a modified area of ​​the object area.

[0081] In this embodiment, the region deformation unit 51 acquires, as the deformation region, a region obtained by reducing the right ventricular region extracted by rough extraction in step S120 at a predetermined constant rate. Note that, instead of acquiring, as the deformation region, a region obtained by reducing the right ventricular region, the region deformation unit 51 may acquire, as the deformation region, a region obtained by enlarging the right ventricular region at a predetermined constant rate.

[0082] Furthermore, in this embodiment, the region transformation unit 51 reduces the right ventricular region at a predetermined constant rate. However, the reduction rate may be changed for each portion of the right ventricular region extracted by rough extraction in step S120. For example, the region transformation unit 51 may determine the reduction rate for each portion of the right ventricular region based on the reliability of the rough extraction of the right ventricular region. Here, the reliability of the rough extraction may be the accuracy of extraction of the object region by the rough extraction unit 42. The region transformation unit 51 may increase the reduction rate for portions with lower reliability and decrease the reduction rate for portions with higher reliability. Specifically, the region transformation unit 51 determines the reduction rate so that the reliability is high when the likelihood of each pixel of the right ventricular region calculated by the first estimator is high, and the reliability is low when the likelihood is low. Furthermore, the reliability of the rough extraction may be determined based on the spatial gradient of pixels near the contour of the right ventricular region extracted by rough extraction. For example, the reliability may be high when the spatial gradient is large, and low when the gradient is small.

[0083] 6 shows an example of a deformation region 2011 acquired by the region deformation unit 51 in step S210. Note that the input image in this embodiment is the input image 1001 in FIG. 3A. As shown in FIG. 6, the deformation region 2011 acquired by the region deformation unit 51 is a reduced region of the right ventricle region 1011 extracted by the rough extraction unit 42 in the first embodiment.

[0084] (Step S220: Obtaining contribution rate) In step S220, the normalization parameter acquisition unit 52 acquires the pixel value of a pixel based on the deformation area acquired in step S210, and calculates and acquires the contribution degree representing the contribution that the pixel makes to the calculation of the normalization parameter in step S140.

[0085] In this embodiment, the contribution degree is binarized into a value indicating the region obtained by reducing the object region and a value indicating the region outside the region obtained by reducing the object region. Specifically, the normalization parameter acquisition unit 52 acquires the contribution degree of pixels included in the right ventricle region reduced in step S210 as 1, and the contribution degree of pixels in other regions as 0. Note that the normalization parameter acquisition unit 52 may calculate the normalization parameter based on the pixel values ​​of pixels in the region reduced in step S140, without acquiring the contribution degree based on the reduced right ventricle region. This allows the normalization parameter The data acquisition unit 52 can acquire normalization parameters for normalizing the input image based on a partial region that is a part of the object region.

[0086] In this embodiment, the contribution of pixels included in the reduced right ventricular region is set to 1, and the contribution of pixels in other regions is set to 0. Furthermore, the contribution of pixels in a region between the reduced right ventricular region (i.e., the deformed region; contribution = 1) and the region of pixels outside the right ventricular region (contribution = 0) (i.e., a region outside the region due to the reduction of the right ventricular region) may be set continuously between 0 and 1. In this case, the contribution between 0 and 1 set between the deformed region and the region outside the right ventricular region may be set so that the closer the contribution is to the deformed region, the larger the value, and the closer the region is to the region outside the right ventricular region, the smaller the value. Therefore, the normalization parameter acquisition unit 52 calculates the contribution based on both the deformed region and the roughly extracted region.

[0087] Furthermore, when an enlarged region of the right ventricle extracted by rough extraction in step S210 is acquired as the deformed region, the contribution of pixels within the enlarged right ventricle region may be set to 0, and the contribution of pixels in other regions may be set to 1. In this case, the normalization unit 44 performs normalization based on the background region other than the right ventricle by setting the contribution of pixels outside the enlarged right ventricle region to 1. In this way, normalizing the pixel values ​​of the background region of the right ventricle more clearly distinguishes the background region from the right ventricle region to be extracted, thereby improving the accuracy of extraction by precise extraction in step S160.

[0088] In this embodiment, by performing the above normalization using the deformation region 2011 acquired by the region deformation unit 51, the normalization unit 44 can acquire the normalized image 1002 shown in Fig. 3C in step S150. Then, in step S160, the precise extraction unit 45 inputs the normalized image 1002 to an estimator, and can extract the right ventricle region 1012 shown in Fig. 3D.

[0089] The processing of the image processing device 200 according to this embodiment described above makes it possible to perform normalization that places more emphasis on areas that are likely to be the target object among the areas extracted by rough extraction from the input image.

[0090] Next, a modified example of the above embodiment will be described. In the following description, the same components and processes as those of the image processing device 200 will be denoted by the same reference numerals, and detailed description thereof will be omitted.

[0091] (Variation 2-1) In the second embodiment, it is assumed that an estimator based on deep learning such as CNN is used in the rough extraction unit 42 in step S120, but the user may manually specify an area to be subjected to rough extraction by operating the operation unit 35, and the rough extraction unit 42 may extract the specified area.

[0092] For example, the control unit 50 displays an ultrasound cardiac image, which is an input image acquired by the input image acquisition unit 41, on the display unit 36, and the user manually specifies the right ventricular region via the operation unit 35 (such as a touch panel), thereby acquiring information about the right ventricular region that is the target of rough extraction. Next, the region deformation unit 51 acquires, as a deformation region, a region obtained by reducing the right ventricular region depicted in the input image, based on the acquired information about the right ventricular region. Then, based on the acquired information about the right ventricular region, the normalization parameter acquisition unit 52 sets the contribution of pixels outside the region designated by the user to 0, the contribution of pixels within the deformation region to 1, and the contribution of pixels in the region between these regions continuously between 0 and 1. The subsequent processing is the same as in the second embodiment.

[0093] In the above description, the area extracted by rough extraction is manually designated by the user operating the operation unit 35. However, the rough extraction unit 42 may be configured to: Information on the right ventricular region stored in the database 22 or the storage unit 34 may be acquired as the region for coarse extraction. As a result, the image processing device 200 performs the above-described processing, and the precise extraction unit 45 can extract the right ventricular region more precisely based on the information on the right ventricular region manually specified by the user. Therefore, the image processing device 200 can use the information on the region to be extracted specified by the user for the precise extraction processing through the above-described processing.

[0094] <Third embodiment> Next, an image processing device according to a third embodiment will be described. In the following description, the same components and processes as those of the image processing device according to the above embodiments will be denoted by the same reference numerals, and detailed description thereof will be omitted.

[0095] As in the first and second embodiments, the image processing device according to the third embodiment uses an image depicting an object as an input image and extracts an object region from the input image. The image processing device 200 according to the second embodiment calculates the contribution degree based on a modified region obtained by modifying the object region extracted by rough extraction. However, the image processing device according to this embodiment calculates the contribution degree based on, for example, pixels in a region smaller than the region extracted by rough extraction.

[0096] In this embodiment, as in the first and second embodiments, a cardiac ultrasound image of a subject captured by a cardiac ultrasound examination is used as the input image, and the right ventricle in the input image is used as the target, and the area of ​​the right ventricle is assumed to be extracted.

[0097] The configuration of the image processing device of this embodiment will be described below with reference to Fig. 7. As shown in Fig. 7, the image processing system 3 includes an image processing device 300 and a database 22. The image processing device 300 according to this embodiment differs from the image processing device 100 according to the first embodiment in that it includes a reference region acquisition unit 61 and a normalization parameter acquisition unit 62. The processing of each processing unit of the image processing device 300 will be described below, focusing on the differences from the processing in the above embodiment.

[0098] The reference area acquisition unit 61 is a modified area acquisition unit that acquires, as a reference area, an area smaller than the roughly extracted area, by modifying the object area to include a specified area within the object area, which is determined based on the extraction accuracy of the object by the rough extraction unit 42.

[0099] The normalization parameter acquisition unit 62 acquires pixel values ​​of pixels based on the reference region acquired by the reference region acquisition unit 61, and calculates a contribution degree that indicates the magnitude of the influence (contribution) of the pixel on the normalization performed by the normalization unit 44. Then, the normalization parameter acquisition unit 62 calculates and acquires normalization parameters to be used by the normalization unit 44, using the acquired pixel values ​​and the calculated contribution degrees.

[0100] Next, an example of processing executed by the image processing device 300 will be described in detail using the flowchart in Fig. 8. As shown in Fig. 8, the processing executed by the image processing device 300 is different from the processing of the flowchart of the first embodiment shown in Fig. 2 in that step S310 for acquiring a reference region is added, and the processing content of step S320 corresponding to step S130 is different.

[0101] (Step S310: Obtain the reference area) In step S310, the reference area acquisition unit 61 acquires, as a reference area, an area smaller than the area extracted by rough extraction acquired in step S120 (roughly extracted area).

[0102] In this embodiment, as an example, the extracted region extracted by the rough extraction unit 42 includes Assume that there is a characteristic that the extraction accuracy of a predetermined region (here, a region including the lower 30% of the region) is high. In this case, the reference region acquisition unit 61 acquires, as the reference region, the lower 30% of the extracted region of the right ventricle region extracted by rough extraction in step S120. As another example, the reference region acquisition unit 61 may expand the lower 30% of the extracted region at a predetermined rate in the direction of forming the contour of the right ventricle, rather than the lower 30% of the extracted region, and acquire, as the reference region, the region excluding the lower 30% of the extracted region. In other words, the reference region acquisition unit 61 may acquire, as the reference region, a region of the myocardium outside the right ventricle region.

[0103] Fig. 9 shows an example of a reference region 3011 acquired by the reference region acquisition unit 61 in step S310. Note that the input image in this embodiment is the input image 1001 in Fig. 3A. As shown in Fig. 9, the reference region 3011 acquired by the reference region acquisition unit 61 is a region obtained by deleting the upper 70% of the right ventricle region 1011 extracted by the rough extraction unit 42 in the first embodiment and leaving the lower 30%.

[0104] (Step S320: Obtaining contribution rate) In step S320, the normalization parameter acquisition unit 62 acquires the pixel value of the pixel based on the reference area acquired in step S310, and calculates and acquires the contribution degree representing the contribution that the pixel makes to the calculation of the normalization parameter in step S140.

[0105] In this embodiment, the normalization parameter acquiring unit 62 acquires the contribution of pixels included in the lower 30% region of the right ventricular region extracted by rough extraction, which is the reference region calculated in step S310, as 1, and the contribution of pixels in the region of other pixels as 0. Note that the normalization parameter acquiring unit 62 may set the contribution of the pixel at the bottom of the right ventricular region extracted by rough extraction to 1, and continuously decrease the contribution within a range from 0 to 1 as it moves toward the top of the region, so that the contribution of the pixel in the lower 30% is 0.5.

[0106] In this embodiment, the normalization is performed using the reference region 3011 acquired by the reference region acquisition unit 61, so that the normalization unit 44 can acquire the normalized image 1002 shown in Fig. 3C in step S150. Then, in step S160, the precise extraction unit 45 inputs the normalized image 1002 to an estimator to extract the right ventricle region 1012 shown in Fig. 3D.

[0107] By the processing of the image processing device 300 according to this embodiment described above, a region near a certain part of the right ventricle region extracted by rough extraction is used as a reference region, and its pixel values ​​are normalized. This makes it possible for the image processing device 300 to extract the right ventricle related to the reference region from the input image with higher accuracy, even if regions with high extraction accuracy by rough extraction are biased towards certain parts.

[0108] <Fourth embodiment> Next, an image processing device according to a fourth embodiment will be described. In the following description, the same components and processes as those of the image processing device according to the above embodiments will be denoted by the same reference numerals, and detailed description thereof will be omitted.

[0109] The image processing device according to the fourth embodiment, like the first, second and third embodiments, uses an image depicting an object as an input image and extracts an object region from the input image. The image processing device 100 according to the first embodiment calculates the contribution degree using the likelihood of each pixel based on the object region extracted by rough extraction. However, the image processing device according to the fourth embodiment acquires, for example, information on the extraction accuracy of rough extraction by an estimator executed by a rough extraction unit as the contribution degree. The image processing device according to this embodiment also acquires, as the contribution degree, information on the extraction accuracy of rough extraction by an estimator executed by a rough extraction unit, as the contribution degree. Similarly, a case where an ultrasound image of a subject taken by an ultrasound examination of the heart is input is taken as an example, and the right ventricle in the input image is taken as the object, and the region of the right ventricle is extracted.

[0110] The configuration of the image processing device of this embodiment will be described below with reference to Fig. 10. As shown in Fig. 10, the image processing system 4 includes an image processing device 400 and a database 22. The image processing device 400 according to this embodiment differs from the image processing device 100 according to the first embodiment in that it includes an accuracy information acquisition unit 71 and a normalization parameter acquisition unit 72. The processing of each processing unit in the image processing device 400 will be described below, focusing on the differences from the processing in the above embodiment.

[0111] The accuracy information acquisition unit 71 acquires information about the extraction accuracy of the object region extracted by the rough extraction unit 42 through rough extraction from the database 22 or the storage unit 34 .

[0112] Based on the information related to the extraction accuracy of the object region acquired by the accuracy information acquisition unit 71, the normalization parameter acquisition unit 72 calculates a contribution indicating the magnitude of the influence (contribution) that the pixels of the object region extracted by the rough extraction unit 42 have on normalization by the normalization unit 44. Then, the normalization parameter acquisition unit 72 calculates and acquires normalization parameters to be used by the normalization unit 44 using the pixel values ​​of the pixels based on the object region and the calculated contribution.

[0113] Next, an example of processing executed by the image processing device 400 will be described in detail using the flowchart in Fig. 11. As shown in Fig. 11, the processing executed by the image processing device 400 is different from the processing of the flowchart of the first embodiment shown in Fig. 2 in that step S410 for acquiring information about rough extraction accuracy is added, and the processing content of step S420 corresponding to step S130 is different.

[0114] (Step S410: Acquisition of information regarding extraction accuracy of rough extraction) In step S410, the accuracy information acquisition unit 71 acquires information about the accuracy of the first estimator that extracts the object region in step S120. As a result, the accuracy information acquisition unit 71 acquires information about the extraction accuracy of the object region when the rough extraction unit 42 extracts the object region by rough extraction.

[0115] As a method for acquiring information about the accuracy of the first estimator in this step, a process for evaluating the accuracy of the first estimator is performed, and the evaluation result is acquired as information about the accuracy of the first estimator. Specifically, the accuracy information acquisition unit 71 first acquires an evaluation image in which the correct right ventricular region in the image is defined, separate from the input image acquired in step S110. Then, the accuracy information acquisition unit 71 acquires an estimation result of the right ventricular region using the estimator executed by the rough extraction unit 42 on the evaluation image. Then, the accuracy information acquisition unit 71 compares the correct right ventricular region in the evaluation image with the region estimation result obtained by the first estimator.

[0116] At this time, the accuracy information acquisition unit 71 acquires an evaluation result based on the comparison result, indicating that the accuracy of the first estimator is low for regions where the difference between these regions is large and high for regions where the difference is small. Here, the difference between the correct region and the estimated region can be acquired based on the magnitude of the difference in the contour positions of the regions, the likelihood value output by the first estimator, etc. Furthermore, it is desirable to acquire the difference between the correct region and the estimated region individually for each of multiple regions constituting the right ventricle. Specifically, the multiple regions can include a region close to the tricuspid valve of the right ventricle, a region close to the apex, a region between the tricuspid valve and the apex, etc. Therefore, the accuracy information acquisition unit 71 can acquire information regarding the accuracy of the first estimator for each of the multiple regions constituting the right ventricle.

[0117] In the above description, the accuracy information acquisition unit 71 determines the correct right ventricle region of the evaluation image and the first The accuracy information acquiring unit 71 may acquire a plurality of evaluation images, calculate the difference between the correct right ventricular region and the right ventricular region estimated by the first estimator for each evaluation image, and acquire an evaluation result of the accuracy of the first estimator based on the calculated difference. Specifically, the accuracy information acquiring unit 71 acquires an evaluation result of the accuracy of the first estimator based on the average value, the initial value, or the maximum value of the difference between the correct region and the estimated region for each evaluation image.

[0118] In addition, the accuracy information acquisition unit 71 may acquire, as information about the extraction accuracy of the first estimator, the proportion of parts of the right ventricular region estimated by the first estimator for each evaluation image that fail to be extracted. Specifically, the accuracy information acquisition unit 71 acquires, as information about the extraction accuracy of the first estimator, the proportion of parts close to the apex that fail to be extracted, the proportion of parts close to the tricuspid valve that fail to be extracted, and the proportion of parts between these parts that fail to be extracted, in multiple evaluation images. Here, failure to extract each part means the occurrence of extraction gaps, such as the region of each part not being extracted, or the difference between the above regions exceeding a predetermined standard, etc.

[0119] (Step S420: Obtaining contribution rate) In step S420, the normalization parameter acquisition unit 72 identifies multiple parts depicted in the region of the object extracted by rough extraction. Next, the normalization parameter acquisition unit 72 acquires pixel values ​​of pixels based on the region of the object acquired in step S120, and calculates and acquires contributions based on information related to the accuracy of the first estimator that extracts the region of the object acquired in step S410.

[0120] In this embodiment, the normalization parameter acquisition unit 72 identifies multiple parts constituting the right ventricle from the right ventricle region extracted by rough extraction in step S120, and calculates the contribution rate based on the error information of each part constituting the right ventricle acquired in step S410. Specifically, the normalization parameter acquisition unit 72 identifies the parts constituting the right ventricle from the right ventricle region extracted by rough extraction as follows: the upper 30% of the right ventricle is a part close to the apex, the lower 30% is a part close to the tricuspid valve, and the remaining parts are a part between the apex and the tricuspid valve. The normalization parameter acquisition unit 72 then calculates the contribution rate so that the contribution rate of parts with large errors related to each part constituting the right ventricle is small and the contribution rate of parts with small errors is large. In this way, the normalization parameter acquisition unit 72 calculates the contribution rate correlated with the first estimator that extracts the region of the object acquired in step S120. In this embodiment, the normalized parameter acquisition unit 72 identifies multiple parts that make up the right ventricle based on their relative positions in the entire region of the right ventricle, such as the upper and lower parts of the right ventricle, but each part may be identified using a classifier that classifies each part.

[0121] The input image in this embodiment is the input image 1001 in FIG. 3A. In step S120, the coarse extraction unit 42 extracts a right ventricular region 1011 shown in FIG. 3B. Then, by performing the normalization described above using the contribution calculated based on information related to the extraction accuracy by the coarse extraction unit 42 and the right ventricular region 1011, the normalization unit 44 can acquire the normalized image 1002 shown in FIG. 3C in step S150. Then, in step S160, the precise extraction unit 45 inputs the normalized image 1002 to an estimator, and can extract the right ventricular region 1012 shown in FIG. 3D.

[0122] As described above, the image processing device 400 according to this embodiment can calculate the contribution based on the tendency of erroneous estimation by the first estimator that performs rough extraction, thereby further increasing the contribution of pixel values ​​of regions where erroneous estimation is unlikely to occur. As a result, the image processing device 400 is expected to improve the extraction accuracy of the right ventricular region from the input image by generating a normalized image by increasing the contribution of pixel values ​​of regions that are more likely to be the correct right ventricular region.

[0123] <Other embodiments> Furthermore, the disclosed technology can be embodied as, for example, a system, a device, a method, a program, or a recording medium (storage medium), etc. Specifically, it may be applied to a system consisting of multiple devices (for example, a host computer, an interface device, an imaging device, a web application, etc.), or it may be applied to an apparatus consisting of a single device.

[0124] Needless to say, the object of the present invention can be achieved by the following: Namely, a recording medium (or storage medium) on which software program code (computer program) that realizes the functions of the above-described embodiments is recorded is supplied to a system or device. Needless to say, such a recording medium is a computer-readable recording medium. Then, a computer (or CPU or MPU) of the system or device reads and executes the program code stored on the recording medium. In this case, the recording medium on which the program code read from the recording medium is recorded constitutes the present invention.

[0125] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0126] The disclosure of this embodiment includes the following configuration, method, and program. (Configuration 1) an image acquisition unit that acquires an image of an object; a first extraction unit that extracts a first region corresponding to a region of the object using the image; a parameter acquisition unit that acquires parameters for normalizing the image based on pixel values ​​of pixels in the first region and contributions indicating the likelihood that the pixels are included in the object; a normalization unit that normalizes the image using the image and the parameters to obtain a normalized image; a second extraction unit that extracts the object region more precisely than the first region using the normalized image; having 1. An image processing device comprising: (Configuration 2) 2. The image processing device according to configuration 1, further comprising a contribution degree acquisition unit that acquires information about the contribution degree. (Configuration 3) 3. The image processing device according to configuration 1 or 2, wherein the degree of contribution has a correlation with the extraction accuracy with which the first extraction unit extracts the region of the object. (Configuration 4) the first extraction unit extracts the first region using a machine learning model trained to extract the region of the object from the image; 3. The image processing device according to claim 1, wherein the contribution is based on a likelihood of the pixel being the object with respect to the first region extracted by the machine learning model. (Configuration 5) 3. The image processing device according to claim 1, wherein the contribution is binarized into a value indicating an area obtained by reducing the first area and a value indicating an area outside the area obtained by reducing the first area. (Configuration 6) The image processing device according to any one of configurations 1 to 5, characterized in that the contribution rate is a value between 0 and 1, and an area where the contribution rate is 0.5 or more is smaller than the first area extracted by the first extraction unit. (Configuration 7) 7. The image processing device according to configuration 6, wherein the region having a contribution rate of 0.5 or more has a region overlapping with the first region extracted by the first extraction unit. (Configuration 8) 7. The image processing device according to configuration 6, wherein the region having a contribution rate of 0.5 or more is a region within the first region extracted by the first extraction unit. (Configuration 9) 9. The image processing device according to any one of configurations 1 to 8, wherein the parameter is a statistical value relating to the pixel value of each pixel of the image acquired by the image acquisition unit. (Configuration 10) 10. The image processing device according to configuration 9, wherein the statistical values ​​include at least a mean value and a variance or a standard deviation value of the pixel values. (Configuration 11) 10. The image processing device according to configuration 9, wherein the normalization unit performs pixel value conversion to bring the statistical value of the image acquired by the image acquisition unit closer to a predetermined statistical value. (Configuration 12) 10. The image processing device according to claim 9, wherein the statistical values ​​include at least an upper limit value and a lower limit value in a certain range of pixel value distribution. (Configuration 13) a modification area acquisition unit that acquires a modification area obtained by modifying the first area extracted by the first extraction unit, The contribution indicates a correlation between the pixel in the deformation region and the object. 13. The image processing device according to any one of configurations 1 to 12. (Configuration 14) The image processing device according to configuration 13, characterized in that the deformation area acquisition unit determines a ratio for deforming the first area based on the extraction accuracy of the area of ​​the object by the first extraction unit, and deforms the first area based on the ratio. (Configuration 15) The image processing device according to configuration 13, wherein the deformation area acquisition unit deforms the first area so as to include a predetermined area within the first area determined based on the extraction accuracy of the area of ​​the object by the first extraction unit. (Configuration 16) 16. The image processing device according to claim 1, wherein the image includes an ultrasound image. (Configuration 17) 17. The image processing device according to claim 1, wherein the object includes at least one of a part of a heart and a region having a predetermined characteristic included in the image. (Configuration 18) 18. The image processing device according to claim 1, wherein the first extraction unit extracts the first region from the image. (Configuration 19) 19. The image processing device according to claim 1, wherein the second extraction unit extracts the region of the object from the image more precisely than the first region. (Configuration 20) 21. The image processing device according to claim 1, wherein the second extraction unit extracts the object region more precisely by using the normalized image from the first extraction unit. 22. The image processing device according to claim 2, wherein the contribution degree obtaining unit is configured to obtain the contribution degree for at least the pixel corresponding to the first region. an image acquisition unit that acquires an image of an object; a first extraction unit that extracts a region of the object as a first region using the image; a parameter acquisition unit that acquires parameters for normalizing the image based on a partial region that is a part of the first region; a normalization unit that normalizes the image using the image and the parameters to obtain a normalized image; a second extraction unit that extracts a region of the object using the normalized image; having 1. An image processing device comprising: (Method 1) acquiring an image of an object; extracting a first region corresponding to a region of the object using the image; obtaining a parameter for normalizing the image based on pixel values ​​of pixels in the first region and contributions indicating the likelihood that the pixels are included in the object; obtaining a normalized image by normalizing the image using the image and the parameters; extracting a region of the object more precisely than the first region using the normalized image; Including, An image processing method comprising: (Method 2) acquiring an image of an object; extracting a region of the object as a first region using the image; obtaining parameters for normalizing the image based on a subregion that is a part of the first region; obtaining a normalized image by normalizing the image using the image and the parameters; extracting a region of the object using the normalized image; Including, An image processing method comprising: (program) A program for causing a computer to execute each step of the image processing method according to Method 1 or 2. [Explanation of symbols]

[0127] 100 Image processing device, 41 Input image acquisition unit, 42 Rough extraction unit, 43 Normalization parameter acquisition unit, 44 Normalization unit, 45 Precision extraction unit

Claims

1. an image acquisition unit that acquires an image of an object; a first extraction unit that extracts a first region corresponding to a region of the object using the image; a parameter acquisition unit that acquires parameters for normalizing the image based on pixel values ​​of pixels in the first region and contributions indicating the likelihood that the pixels are included in the object; a normalization unit that normalizes the image using the image and the parameters to obtain a normalized image; a second extraction unit that extracts the object region more precisely than the first region using the normalized image; having 1. An image processing device comprising:

2. The image processing apparatus according to claim 1 , further comprising a contribution degree acquisition unit for acquiring information on the contribution degree.

3. 3. The image processing apparatus according to claim 1, wherein the degree of contribution has a correlation with an extraction accuracy with which the first extraction unit extracts the region of the object.

4. the first extraction unit extracts the first region using a machine learning model trained to extract a region of the object from an image; The image processing device according to claim 1 or 2, wherein the contribution is based on a likelihood of the pixel being the object with respect to the first region extracted by the machine learning model.

5. 3. The image processing device according to claim 1, wherein the contribution is binarized into a value indicating an area obtained by reducing the first area and a value indicating an area outside the area obtained by reducing the first area.

6. 3. The image processing device according to claim 1, wherein the contribution degree is a value between 0 and 1, and an area where the contribution degree is 0.5 or more is smaller than the first area extracted by the first extraction unit.

7. The image processing device according to claim 6 , wherein the region having a contribution rate of 0.5 or more includes a region overlapping the first region extracted by the first extracting unit.

8. 7. The image processing apparatus according to claim 6, wherein the region having a contribution rate of 0.5 or more is a region within the first region extracted by the first extraction unit.

9. 3. The image processing apparatus according to claim 1, wherein the parameter is a statistical value relating to the pixel value of each pixel of the image acquired by the image acquisition unit.

10. 10. The image processing apparatus according to claim 9, wherein the statistical values ​​include at least an average value and a variance or standard deviation value of the pixel values.

11. The image processing device according to claim 9 , wherein the normalization unit performs pixel value conversion to bring the statistical value of the image acquired by the image acquisition unit closer to a predetermined statistical value.

12. The statistical value includes at least an upper limit value and a lower limit value in a certain range of pixel value distribution. The image processing device according to claim 9 ,

13. a modification area acquisition unit that acquires a modification area obtained by modifying the first area extracted by the first extraction unit, The contribution indicates a correlation between the pixel in the deformation region and the object.

3. The image processing device according to claim 1, wherein the image processing device is a computer.

14. 14. The image processing device according to claim 13, wherein the deformation area acquisition unit determines a ratio for deforming the first area based on the extraction accuracy of the area of ​​the object by the first extraction unit, and deforms the first area based on the ratio.

15. 14. The image processing device according to claim 13, wherein the deformation area acquisition unit deforms the first area so as to include a predetermined area within the first area that is determined based on the accuracy of extraction of the area of ​​the object by the first extraction unit.

16. 3. The image processing apparatus according to claim 1, wherein the image includes an ultrasound image.

17. 3. The image processing apparatus according to claim 1, wherein the object includes at least one of a part of a heart and an area having a predetermined characteristic included in the image.

18. 3. The image processing device according to claim 1, wherein the first extraction unit extracts the first region from the image.

19. 3. The image processing device according to claim 1, wherein the second extraction unit extracts the region of the object from the image more precisely than the first region.

20. 3. The image processing apparatus according to claim 1, wherein the second extraction unit extracts the region of the object more precisely than the first extraction unit by using the normalized image.

21. The image processing device according to claim 2 , wherein the contribution degree obtaining unit is configured to obtain the contribution degree for at least the pixel corresponding to the first region.

22. an image acquisition unit that acquires an image of an object; a first extraction unit that extracts a region of the object as a first region using the image; a parameter acquisition unit that acquires parameters for normalizing the image based on a partial region that is a part of the first region; a normalization unit that normalizes the image using the image and the parameters to obtain a normalized image; a second extraction unit that extracts a region of the object using the normalized image; having 1. An image processing device comprising:

23. acquiring an image of an object; extracting a first region corresponding to a region of the object using the image; obtaining a parameter for normalizing the image based on pixel values ​​of pixels in the first region and a contribution indicating a probability that the pixel is included in the object; obtaining a normalized image by normalizing the image using the image and the parameters; extracting a region of the object more precisely than the first region using the normalized image; Including, An image processing method comprising:

24. acquiring an image of an object; extracting a region of the object as a first region using the image; obtaining parameters for normalizing the image based on a subregion that is a part of the first region; obtaining a normalized image by normalizing the image using the image and the parameters; extracting a region of the object using the normalized image; Including, An image processing method comprising:

25. A program for causing a computer to execute each step of the image processing method according to claim 23 or 24.

Images