Image processing device, image processing method, and program

Abstract

This image processing device 1X is provided with a detection means 31X, an acquisition means 32X, a deformation means 33X, and a display control means 34X. The detection means 31X detects a dense part in which tissue is densely packed from a chest image. The acquisition means 32X acquires a first image obtained by cutting out a region based on the dense part from the chest image. The deformation means 33X deforms the first image on the basis of information on the density of tissue in the first image. The display control means 34X displays a second image obtained by deforming the first image on a display device. The invention can be used to assist a user in decision making, etc.

Description

Technical Field

[0001] The present disclosure relates to the technical field of an image processing apparatus, an image processing method, and a storage medium for performing image processing.

Background Art

[0002] Conventionally, an image processing system for processing chest images such as mammography images by computer image analysis has been known. For example, Patent Document 1 discloses an information processing system that generates and displays missed detection data including image data indicating an area where it is easy to miss a lesion based on a mammography image and additional data.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Regions where it is easy to miss a lesion are usually parts where tissues are dense and visibility is low. Therefore, even if, as in Patent Document 1, a region where it is easy to miss a lesion is detected and displayed, it is difficult for an examiner to confirm whether there is an abnormality in the displayed region.

[0005] In view of the above problems, one object of the present disclosure is to provide an image processing apparatus, an image processing method, and a storage medium capable of suitably displaying a location where tissues are dense in a chest image.

Means for Solving the Problems

[0006] One aspect of the image processing apparatus is detection means for detecting a dense location where tissues are dense from a chest image, Acquisition means for acquiring a first image obtained by cutting out a region based on the densely packed area from the chest image, Based on the first image above, Distribution information is generated showing the distribution of the density of the tissue in the region based on the dense area, and based on this distribution information, the density is made uniform. A deformation means for deforming the first image, A display control means for displaying a second image obtained by deforming the first image on a display device, This is an image processing device that has the following features.

[0007] One aspect of the image processing method is: From chest images, dense areas of tissue are detected. A first image is obtained by cutting out a region based on the densely packed area from the aforementioned chest image. Based on the first image above, Distribution information is generated showing the distribution of the density of the tissue in the region based on the dense area, and based on this distribution information, the density is made uniform. The first image is deformed, A second image, obtained by transforming the first image, is displayed on the display device. This is an image processing method.

[0008] One aspect of the program is: From chest images, dense areas of tissue are detected. A first image is obtained by cutting out a region based on the densely packed area from the aforementioned chest image. Based on the first image above, Distribution information is generated showing the distribution of the density of the tissue in the region based on the dense area, and based on this distribution information, the density is made uniform. The first image is deformed, This is a program that causes a computer to perform the process of displaying a second image, which is a modified version of the first image, on a display device. [Effects of the Invention]

[0009] One example of the effects of this disclosure is that it becomes possible to suitably display areas where tissue is densely packed in chest images. [Brief explanation of the drawing]

[0010] [Figure 1] The schematic configuration of the chest image processing system is shown. [Figure 2] This figure shows an overview of the image processing performed by the image processing device in the first embodiment. [Figure 3]This is an example of the functional blocks of the processor of the image processing apparatus in the first embodiment. [Figure 4] (A) An example of a target tissue mask image is shown. (B) A target tissue density image calculated based on the target tissue mask image is shown. [Figure 5] (A) A target tissue density image is shown. (B) A diagram visualizing the deformation vectors for each pixel calculated based on the target tissue density image. [Figure 6] (A) A target tissue density image clearly showing the target tissue density for each pixel is shown. (B) The correspondence between each pixel of the target tissue density image and the X-Y coordinates when the region of the target tissue density image is regarded as the X-Y coordinate space is shown. [Figure 7] An example of the first display example on the display screen is shown. [Figure 8] An example of the second display example on the display screen is shown. [Figure 9] An example of the third display example on the display screen is shown. [Figure 10] This is an example of a flowchart showing the outline of the processing executed by the image processing apparatus in the first embodiment. [Figure 11] This is a block diagram of the image processing apparatus in the second embodiment. [Figure 12] This is an example of a flowchart executed by the image processing apparatus in the second embodiment.

Embodiments for Carrying Out the Invention

[0011] Hereinafter, embodiments of an image processing apparatus, an image processing method, and a storage medium will be described while referring to the drawings.

[0012] <First Embodiment> (1) System Configuration Figure 1 shows a schematic configuration of the chest image processing system 100. As shown in Figure 1, the chest image processing system 100 is a system that deforms images representing areas with high tissue density and low visibility in the chest image of a patient subject by non-rigid deformation to improve visibility, and presents the deformed image to the examiner. The chest image processing system 100 mainly comprises an image processing device 1, a chest image generation device 2, a display device 3, and an operating device 4.

[0013] The image processing device 1 performs functions such as detecting areas of tissue density, cropping regions based on the patient's chest image supplied by the chest image generation device 2, and deforming the cropped image. The image processing device 1 also performs display control to display information based on the deformed image on the display device 3, and performs various processes based on operation signals received from the operation device 4.

[0014] The chest image generation device 2 generates a chest image of the patient's chest, which is the target area for imaging, and supplies the generated chest image to the image processing device 1. For example, the chest image generation device 2 generates a chest image (X-ray image of the breast) by performing X-ray imaging, which involves irradiating the patient's breast with X-rays and detecting the X-rays that pass through the breast. The chest image generation device 2 may be a mammography device based on the CR (Computed Radiography) method or a mammography device based on the FPD (Flat Panel Detector) method. The chest image generated by the chest image generation device 2 is digital data that can be analyzed by the image processing device 1. For example, pixels in the chest image are displayed as having larger pixel values ​​(brightness) the greater the X-ray absorption, and on the chest image, pixels with larger pixel values ​​are depicted as whiter. Note that the chest image generated by the chest image generation device 2 is not limited to mammography images, but may also be ultrasound images, MRI images, CT images, chest X-ray images, angiography images, etc.

[0015] Furthermore, the chest image generation device 2 may supply the image processing device 1 with at least one of the patient information or examination information of the patient who is the subject of the chest image, in addition to the chest image. In this case, examples of patient information include patient identification information (patient ID) for identifying the patient, and other patient attribute information (name, gender, date of birth, etc.). Examples of examination information include examination identification information (examination ID) for identifying the examination, examination date and time information, and information on examination conditions (examination site, laterality (left, right), direction (e.g., vertical direction (CC), mediolateral oblique direction (MLO), compressed breast thickness), etc.).

[0016] The display device 3 performs a predetermined display based on the display signal supplied from the image processing device 1. Examples of the display device 3 include displays such as CRTs (Cathode Ray Tubes) and LDCs (Liquid Crystal Displays), as well as projectors.

[0017] The operating device 4 generates operation signals based on operations performed by a user of the image processing device 1, such as a physician. Examples of operating devices 4 include buttons, keyboards, pointing devices such as mice, touch panels, remote controllers, voice input devices, and other arbitrary user interfaces.

[0018] Figure 1 also shows an example of the hardware configuration of the image processing device 1. The image processing device 1 mainly includes a processor 11, memory 12, and interface 13. Each of these elements is connected via a data bus 19.

[0019] The processor 11 executes predetermined processes by running programs and other data stored in memory 12. The processor 11 is a processor such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), or TPU (Tensor Processing Unit). The processor 11 may be composed of multiple processors. The processor 11 is an example of a computer.

[0020] Memory 12 consists of various volatile memories used as working memory, such as RAM (Random Access Memory) and ROM (Read Only Memory), and non-volatile memory that stores information necessary for processing by the image processing device 1. Memory 12 may also include external storage devices such as hard disks connected to or built into the image processing device 1, or it may include storage media such as removable flash memory. Memory 12 stores programs and other information necessary for the image processing device 1 to perform each of the processes in this embodiment.

[0021] Interface 13 performs interface operations between the image processing device 1 and external devices. For example, interface 13 is electrically connected to the chest image generation device 2, the display device 3, and the operating device 4. Interface 13 may be a communication interface such as a network adapter for wired or wireless communication with external devices, or it may be a hardware interface compliant with USB (Universal Serial Bus), SATA (Serial AT Attachment), etc. Interface 13 may also perform interface operations with external devices such as the chest image generation device 2, the display device 3, and the operating device 4 via a communication network such as the Internet.

[0022] The configuration of the chest image processing system 100 shown in Figure 1 is an example, and various modifications may be made.

[0023] For example, the image processing device 1 may be integrated with at least one of a chest image generation device 2, a display device 3, or an operating device 4. In another example, the image processing device 1 may include an audio output device that outputs information by sound. In yet another example, the image processing device 1 may be composed of multiple devices.

[0024] In another example, the chest image generation device 2 may be a storage device that stores pre-generated chest images of patients. In this case, the storage device is capable of data communication with the image processing device 1, and the image processing device 1 acquires, for example, a chest image of a patient specified by the operating device 4 from the storage device and performs processing according to this embodiment. The storage device may also be a server device that communicates data with the image processing device 1 via a communication network, or it may be built into the image processing device 1 as the memory 12 of the image processing device 1.

[0025] (2) Overview Figure 2 is a diagram showing an overview of the image processing performed by the image processing device 1 in the first embodiment.

[0026] When the image processing device 1 acquires a chest image of a patient from the chest image generation device 2, it performs image analysis on the acquired chest image to detect areas of high density in the breast tissue (also called "target tissue"). Here, "target tissue" refers to the tissue within the breast that appears in the chest image, and examples of target tissue include mammary glands, milk ducts (including milk ducts formed as part of mammary glands), etc. In Figure 2, the detected areas of high density are highlighted with rectangular frames.

[0027] Next, the image processing device 1 acquires an image (also called the "first crop image") obtained by cropping a region within the chest image based on the detected area of ​​the densely populated area. The first crop image may be an image obtained by cropping the detected area of ​​the densely populated area from the chest image, or it may be an image obtained by cropping a region smaller or larger than the detected area of ​​the densely populated area from the chest image so as to fit within a predetermined size.

[0028] The image processing device 1 then non-rigidly deforms the first crop image to equalize the density of the target tissue shown in the first crop image. Hereafter, the image obtained by non-rigidly deforming the first crop image will also be called the "second crop image". In this case, the image processing device 1 deforms the first crop image to equalize the degree of density of the target tissue on the image. As a result, the second crop image is an image obtained by non-rigidly deforming the first crop image so that the density of the target tissue is eliminated. The image processing device 1 then displays information based on the second crop image on the display device 3. This makes it possible for the examiner to suitably confirm areas of dense target tissue that are difficult to confirm in the first crop image using the second crop image.

[0029] (3) Functional Blocks Figure 3 shows an example of the functional blocks of the processor 11 of the image processing device 1 in the first embodiment. Functionally, the processor 11 of the image processing device 1 includes a chest image acquisition unit 30, a dense area detection unit 31, a first crop image acquisition unit 32, an image deformation unit 33, and a display control unit 34. In Figure 4, blocks where data is exchanged are connected by solid lines, but the combination of blocks where data is exchanged is not limited to this. The same applies to the diagrams of other functional blocks described later.

[0030] The chest image acquisition unit 30 receives the patient's chest image generated by the chest image generation device 2 from the chest image generation device 2 via the interface 13. The chest image acquisition unit 30 then supplies the acquired chest image to the dense area detection unit 31, the first crop image acquisition unit 32, and the display control unit 34.

[0031] The dense area detection unit 31 detects dense areas of target tissue from the chest image supplied from the chest image acquisition unit 30 and supplies the detection result to the first crop image acquisition unit 32.

[0032] Here, we will explain specific examples of detecting densely populated areas (the first and second examples) in order.

[0033] In the first example of detecting dense areas, the dense area detection unit 31 infers the dense areas of the target tissue in the chest image based on an inference model (inference engine) that outputs detection results for dense areas of the target tissue from the input chest image. The inference model described above is a model that has been trained, for example, by machine learning, and is a model that has learned the relationship between the chest image input to the inference model and the dense areas of the target tissue in that image. Specifically, the inference model is a model that has been trained to output an inference result for the dense areas of the target tissue in the input chest image when a chest image is input. The inference result output by the inference model may be positional information that indicates the region on the image surrounding the dense area of ​​the target tissue (for example, vertex coordinate information in the case of a bounding box), or it may be positional information that indicates the representative position of the dense area of ​​the target tissue on the image.

[0034] The inference model is trained using a training dataset that includes, for example, multiple records, each containing a combination of a sample chest image and ground truth information indicating the areas of high density in that chest image. The inference model may be any machine learning model (including statistical models; the same applies hereinafter), such as a neural network or a support vector machine. Representative neural network models include, for example, Fully Convolutional Network, SegNet, U-Net, V-Net, Feature Pyramid Network, Mask R-CNN, and DeepLab. When the inference model is constructed using a neural network, various parameters such as the layer structure, the neuron structure of each layer, the number and size of filters in each layer, and the weights of each element in each filter are stored as trained parameters in memory 12, etc.

[0035] Furthermore, the inference model described above may be trained to detect areas where the target tissue is densely concentrated and where lesions are suspected, as such dense areas. In this case, the inference model is trained using a training dataset that includes multiple records, each consisting of a sample chest image and ground truth information indicating areas in the chest image where the target tissue is densely concentrated and lesions are suspected.

[0036] Furthermore, the inference model may output inference results indicating multiple dense areas within the target organization. Also, if the inference model is composed of a neural network, it may output the confidence level for each detected dense area along with the location information of the detected dense areas as an inference result. In this case, for example, the dense area detection unit 31 adopts the inference results of dense areas whose confidence level is above a predetermined threshold as the detection results for dense areas to be used in subsequent processing, and supplies these detection results to the first crop image acquisition unit 32 and the display control unit 34.

[0037] In the second example of detecting dense areas, the dense area detection unit 31 calculates the degree of density of the target tissue (also called "target tissue density") for each pixel of the chest image and detects dense areas based on the target tissue density. In this case, for example, the dense area detection unit 31 may detect as a dense area an area in which a predetermined number or more pixels with a target tissue density of a predetermined threshold or higher are connected. The predetermined threshold and predetermined number are stored in, for example, memory 12. The method for detecting the target tissue and the method for calculating the target tissue density are the same as the method for detecting the target tissue and the method for calculating the target tissue density executed by the image deformation unit 33, and will be described in detail later.

[0038] The first crop image acquisition unit 32 performs a process to cut out a first crop image from the chest image supplied by the chest image acquisition unit 30 based on the detection results of dense areas of the target tissue supplied by the dense area detection unit 31, and supplies the cut-out first crop image to the image deformation unit 33.

[0039] In this case, for example, if the detection result of dense areas of the target tissue supplied by the dense area detection unit 31 indicates an area within the chest image, the first crop image acquisition unit 32 acquires a first crop image by cutting out the area indicated by the dense area detection result. In this case, the first crop image acquisition unit 32 may acquire a first crop image by cutting out an area from the chest image that is larger or smaller than the area indicated by the dense area detection result, so that the first crop image becomes a predetermined size. In this case, the first crop image acquisition unit 32 cuts out the first crop image from the chest image so that, for example, the center of the area indicated by the dense area detection result coincides with the center of the first crop image. Furthermore, if the detection result of dense areas of the target tissue supplied by the dense area detection unit 31 indicates a representative position within the chest image, the first crop image acquisition unit 32 acquires a first crop image by cutting out an area of ​​a predetermined size and shape from the chest image with the representative position as the center of the first crop image.

[0040] The image deformation unit 33 non-rigidly deforms the first crop image supplied from the first crop image acquisition unit 32 so that the density of the target tissue for each pixel becomes uniform, and supplies the deformed second crop image to the display control unit 34. In this case, the image deformation unit 33, (a) Detection of target tissue in the first crop image, (b) Calculation of the density of target tissue per pixel in the first crop image, (c) Calculation of a deformed vector field based on the density of the target tissue, and (d) Transformation of the first cropped image based on a deformed vector field (so-called image warping) These steps are executed in order. Details of the processing in the image deformation unit 33 will be described later.

[0041] The display control unit 34 controls the display on the display device 3 based on the second crop image, etc., supplied from the image deformation unit 33. In this case, the display control unit 34 generates a display signal and supplies the generated display signal to the display device 3, thereby causing the second crop image, etc., to be displayed on the display device 3. The display control unit 34 may also display a chest image that clearly indicates the detection area of ​​the dense area detected by the dense area detection unit 31, along with the second crop image, on the display device 3 based on information supplied from the chest image acquisition unit 30 and the dense area detection unit 31. An example of the display that the display device 3 displays based on the control of the display control unit 34 will be described later.

[0042] The components of the chest image acquisition unit 30, the dense area detection unit 31, the first crop image acquisition unit 32, the image deformation unit 33, and the display control unit 34 can be realized, for example, by the processor 11 executing a program. Alternatively, the necessary programs may be recorded on any non-volatile storage medium and installed as needed to realize each component. At least a portion of these components may be realized not only by software programs, but also by any combination of hardware, firmware, and software. Furthermore, at least a portion of these components may be realized using a user-programmable integrated circuit, such as an FPGA (Field-Programmable Gate Array) or a microcontroller. In this case, the program composed of the above components may be realized using this integrated circuit. At least a portion of each component may also be composed of an ASSP (Application Specific Standard Produce), ASIC (Application Specific Integrated Circuit), or quantum processor (quantum computer control chip). Thus, each component may be realized by various hardware. The same applies to other embodiments described later. Furthermore, each of these components may be realized through the collaboration of multiple computers, for example, using cloud computing technology.

[0043] (4) Details of image transformation Next, we will explain each of the above-mentioned processes (a) to (d) included in the image transformation process by the image transformation unit 33.

[0044] First, we will explain "(a) Detection of target tissue in the first crop image."

[0045] The image deformation unit 33 generates a mask image that indicates the presence or absence of target tissue for each pixel by performing threshold processing on the brightness value (pixel value) associated with each pixel of the first crop image. In this case, the image deformation unit 33 generates a mask image in which pixels with a brightness value higher than a predetermined threshold and pixels with a brightness value below the predetermined threshold have different pixel values. The above thresholds are stored in advance in memory 12, for example.

[0046] Preferably, the image deformation unit 33 performs an arbitrary filtering process (edge ​​detection process) in addition to the luminance value thresholding process described above. The filter used in the above filtering process may be any filter such as Canny, Sobel, or Laplacian. In this case, the image deformation unit 33 acquires an image obtained by inputting the image obtained by the luminance value thresholding process into the above filter as a mask image that indicates the presence or absence of target tissue pixel by pixel.

[0047] Hereafter, the mask image obtained by the process in (a) will be referred to as the "target tissue mask image," pixels in the target tissue mask image that correspond to the target tissue will be referred to as "target tissue pixels," and pixels that do not correspond to the target tissue will also be referred to as "non-target tissue pixels." The target tissue mask image is an example of "information indicating the presence or absence of tissue in each unit region."

[0048] Next, we will explain "(b) Calculation of the density of target tissue for each pixel in the first crop image."

[0049] The image deformation unit 33 calculates the density of target tissue for each pixel of the first crop image using the target tissue mask image. In this case, the image deformation unit 33 selects one pixel at a time from the first crop image, detects target tissue pixels within the offset distance for each selected pixel, and determines the sum of weights corresponding to the distance between the detected target tissue pixels and the selected pixels as the density of target tissue for the selected pixels. Hereafter, an image in which the pixel value of each pixel is the density of target tissue calculated by process (b) will also be called a "density of target tissue image".

[0050] Furthermore, the granularity of the target tissue mask image and the target tissue density image does not need to be at the granularity of one pixel in the chest image; it may be at the granularity of multiple pixels or subpixels. Also, instead of calculating the target tissue density, which is higher the higher the degree of density of the target tissue, the image deformation unit 33 may calculate an index, which is lower the higher the degree of density of the target tissue, and use this index instead of the target tissue density. The same applies when the density detection unit 31 calculates the target tissue density when detecting density areas. The target tissue density image is information that shows the distribution of the target tissue density for each unit area (one pixel, multiple pixels, or subpixels), and is an example of "distribution information".

[0051] Figure 4(A) shows an example of a target tissue mask image, and Figure 4(B) shows a target tissue density image calculated based on the target tissue mask image in Figure 4(A). For the sake of simplicity, the target tissue mask image is a 4x4 image, with the pixels in the first row and second column being target tissue pixels, and the remaining pixels being non-target tissue pixels.

[0052] In this case, the image deformation unit 33 selects each pixel of the target tissue mask image one by one. Then, if a target tissue pixel exists within a predetermined offset distance (in this case, a distance of 2 pixels) from the selected pixel, the image deformation unit 33 adds a value corresponding to the distance between the selected pixel and the target tissue pixel as the target tissue density of the selected pixel. The "value corresponding to the distance" increases as the distance decreases, and is maximized when the distance is 0 (i.e., the selected pixel and the target tissue pixel are the same pixel). In the examples of Figures 4(A) and 4(B), the target tissue density of the pixel in the 1st row, 2nd column, which is the target tissue pixel, becomes the first value, which is the maximum value. The target tissue density of the pixel that is 1 pixel away from the pixel in the 1st row, 2nd column becomes the second highest value. The target tissue density of the pixel that is 2 pixels away from the pixel in the 1st row, 2nd column becomes the third value, which is lower than the second value. The target tissue density of all other pixels becomes 0.

[0053] Furthermore, if multiple target tissue pixels exist within the offset distance, the image deformation unit 33 sets the sum of the values ​​corresponding to the distance between the selected pixel and each of the multiple target tissue pixels as the target tissue density of the selected pixel.

[0054] As a result, the image deformation unit 33 can suitably calculate the target tissue density image. Note that the granularity for calculating the target tissue density (i.e., the granularity of the target tissue density image) does not have to be one pixel, but may be multiple pixels or even subpixels.

[0055] Next, we will explain "(c) Calculation of the deformation vector field based on the density of the target tissue."

[0056] The image deformation unit 33 considers the region of the target tissue density image as a solvent and solves the diffusion equation assuming that a solute with a concentration equal to the target tissue density shown in the target tissue density image is dissolved there, calculating the movement vector for each pixel to the next step (i.e., the dissolved state). This movement vector corresponds to the deformation vector, and the space where the deformation vectors are arranged in the region of the target tissue density image corresponds to the deformation vector field. The deformation vector is a vector used to coincide an arbitrary point on the first crop image, which is the image to be deformed, with a corresponding point on the second crop image, which is the target image.

[0057] Figure 5(A) shows an image of the target tissue density, and Figure 5(B) visualizes the deformation vectors for each pixel calculated based on the target tissue density image shown in Figure 5(A). As shown in Figures 5(A) and 5(B), each deformation vector points in a direction along the direction from areas with high target tissue density (i.e., high concentration) to areas with low target tissue density. In this way, deformation vectors are calculated that diffuse the dense areas of the target tissue so that the density of the target tissue becomes uniform.

[0058] Here, a specific example of the method for calculating the deformed vector field will be explained with reference to Figures 6(A) and 6(B). Figure 6(A) shows a target tissue density image in which the density of target tissue for each pixel is shown numerically, and Figure 6(B) shows the correspondence between the target tissue density image and the XY coordinates when the region of the target tissue density image is considered as an XY coordinate space. The image deformation unit 33 then solves the diffusion equation for the target tissue density "u(x, y, t)" shown in the following equation by substituting the values ​​of "u(x, y, t)" specified by Figure 6(B).

[0059]

number

[0060] u(0,1,0)=5, u(1 / 3,1,0)=8, u(2 / 3,1,0)=5, u(1,1,0)=2, u(0,2 / 3,0)=5, u(1 / 3,2 / 3,0)=5, u(2 / 3,2 / 3,0)=5, u(1,2 / 3,0)=2, u(0,1 / 3,0)=2, u(1 / 3,1 / 3,0)=2, u(2 / 3,1 / 3,0)=2, u(1,1 / 3,0)=2, u(0,0,0)=0, u(1 / 3,0,0)=0, u(2 / 3,0,0)=0, u(1,0,0)=0.

[0061] The image deformation unit 33 then solves the diffusion equation by approximating the derivative in the form of a difference using the difference method. As a result, the image deformation unit 33 can suitably calculate the movement vector (i.e., deformation vector) in the target tissue density image.

[0062] Next, we will explain "(d) Transformation of the first cropped image based on the deformation vector field."

[0063] The image deformation unit 33 considers the movement vectors for each pixel on the target tissue density image calculated by the process in (c) as a deformation vector field from the first crop image to the second crop image, and deforms the first crop image based on the deformation vector field. In this case, the image deformation unit 33 may use any image warping technique, such as BackAward Image Warping. As a result, the image deformation unit 33 can obtain a second crop image obtained by deforming the first crop image to equalize the target tissue density.

[0064] (5) Display example Figure 7 shows a first example of the display screen that the display control unit 34 displays on the display device 3. The display control unit 34 outputs display information generated based on the chest image supplied from the chest image acquisition unit 30 and the second crop image supplied from the image deformation unit 33 to the display device 3. By transmitting the display information to the display device 3, the display control unit 34 displays the display screen shown in Figure 7 on the display device 3.

[0065] In the first display example shown in Figure 7, the display control unit 34 provides a chest image display area 70 and a second crop image display area 71 on the display screen.

[0066] Here, the display control unit 34 displays the chest image acquired by the chest image acquisition unit 30 from the chest image generation device 2 in the chest image display area 70. The display control unit 34 also overlays a frame 73 on the chest image to highlight the detected areas of dense areas, based on the detection results of dense areas supplied by the dense area detection unit 31. The display control unit 34 may also display a frame 73 that highlights the corresponding area of ​​the first crop image, based on information supplied by the first crop image acquisition unit 32.

[0067] Furthermore, the display control unit 34 displays the second crop image supplied from the image deformation unit 33 in the second crop image display area 71. In this second crop image, the density of target cells such as mammary ducts is reduced, improving the visibility of the target cells. Therefore, the examiner, who is viewing the image, can check in detail in the second crop image display area 71 whether there are any abnormalities such as lesions, especially in areas where target cells are densely concentrated in the chest image. In addition, the viewer can also understand the area of ​​the patient's entire chest corresponding to the second crop image by referring to the chest image display area 70 where the frame 73 is indicated.

[0068] Figure 8 shows a second display example of the display screen that the display control unit 34 displays on the display device 3. In the second display example, the image processing device 1 detects multiple areas of high density of target cells and generates and displays a second crop image corresponding to the detected areas of high density. In the second display example, the display control unit 34 provides a chest image display area 70 and a second crop image display area 71 on the display screen.

[0069] In the second display example, since the dense area detection unit 31 has detected multiple dense areas, the display control unit 34 displays frames 73A and 73B on the chest image, respectively, to highlight the areas of dense areas detected by the dense area detection unit 31. In this case, the image deformation unit 33 acquires a second crop image by deforming the first crop image corresponding to each of the detected areas of dense areas, and the display control unit 34 displays the second crop image supplied from the image deformation unit 33 in the second crop image display area 71, corresponding to the frames 73A and 73B that indicate the corresponding areas of dense areas. In the second display example, as an example, the display control unit 34 highlights the outer frame of the second crop image corresponding to the area of ​​dense area indicated by frame 73A, which is a solid line frame, with a solid line frame, and highlights the outer frame of the second crop image corresponding to the area of ​​dense area indicated by frame 73B, which is a dashed line frame, with a dashed line frame.

[0070] Thus, even when there are multiple detection areas of dense cells, the display control unit 34 can suitably present a second crop image that has been modified to improve the visibility of target cells in each dense cell area. Alternatively, instead of the example in Figure 8, the display control unit 34 may accept user selection of frames 73A and 73B on the display screen and display only the second crop image corresponding to the selected frame in the second crop image display area 71. This allows the display control unit 34 to clearly display the second crop image corresponding to the area of ​​interest to the viewer in the second crop image display area 71.

[0071] Figure 9 shows a third display example of the display screen that the display control unit 34 displays on the display device 3. In the third display example, instead of displaying the chest image displayed in the first and second display examples, the image processing device 1 displays information indicating the relative position of the second crop image to be displayed within the entire chest (more specifically, the breasts). In the third display example, the display control unit 34 provides a second crop image display area 71 and a crop location display area 72 on the display screen.

[0072] In this case, the display control unit 34 displays the second crop image supplied from the image deformation unit 33 in the second crop image display area 71, and also displays a schematic diagram of the breast divided into four regions (outer upper, outer lower, inner upper, and inner lower) in the crop area display area 72. In the crop area display area 72, the display control unit 34 highlights the "outer upper breast" region of the breast that corresponds to the area indicated by the second crop image, and also highlights the area indicated by the second crop image with a frame 74. In this case, the display control unit 34 may, for example, use any method to determine the three-dimensional position of an area corresponding to an arbitrary image region specified from the chest image (including a method for three-dimensionally reconstructing the chest from the chest image) to determine the relative position of the area indicated by the second crop image within the entire breast. In another example, if information showing the correspondence between each pixel of the chest image and the schematic diagram of the breast is stored in advance in the memory 12, the display control unit 34 may refer to this information to determine the relative position of the area indicated by the second crop image within the entire breast.

[0073] Thus, in the third display example, even when the chest image is not displayed, the display control unit 34 allows the viewer to understand the relative position of the area corresponding to the second crop image on the entire patient's chest, and to confirm whether or not there are any abnormalities in the chest image, particularly in areas where target cells are densely concentrated. In each display example, the display control unit 34 may further display information about the patient or examination on the display screen based on at least one of the patient information or examination information received from the chest image generation device 2 along with the chest image. Alternatively, for example, the display control unit 34 may determine a treatment method based on a model generated by machine learning the correspondence between examination information and treatment methods for chest images, and the patient's examination information, and further display the treatment method on the display screen. The method for determining the treatment method is not limited to the method described above.

[0074] (7) Processing flow Figure 10 is an example of a flowchart showing an overview of the processes performed by the image processing device 1 in the first embodiment.

[0075] First, the image processing device 1 acquires a chest image of the patient's chest from the chest image generation device 2 (step S11). Next, the image processing device 1 detects areas of high tissue density from the chest image acquired in step S11 (step S12).

[0076] The image processing device 1 then determines whether or not dense areas of target tissue have been detected in the chest image (step S13). If the image processing device 1 determines that dense areas of target tissue have been detected in the chest image (step S13; Yes), it obtains a first crop image cut out from the chest image based on the detection result of the dense areas of target tissue (step S14). The image processing device 1 then deforms the first crop image to improve the visibility of the target tissue (step S15). In this case, the image processing device 1 generates a target tissue density image, which is a map of the density of target tissue in the first crop image, and obtains a second crop image obtained by deforming the first crop image based on the target tissue density image so that the density of target tissue is made uniform.

[0077] After step S15, the image processing device 1 displays the second cropped image on the display device 3 (step S16). In this case, for example, the image processing device 1 displays a display screen on the display device 3 based on any of the first to third display examples. On the other hand, if the image processing device 1 determines in step S13 that no dense areas of target tissue were detected in the chest image (step S13; No), it displays the chest image on the display device 3 (step S17). In this case, since there are no dense areas of target tissue that are difficult for the examiner to see, the examiner can suitably confirm the condition of the patient's chest based on the chest image.

[0078] <Second Embodiment> Figure 11 is a block diagram of the image processing apparatus 1X in the second embodiment. The image processing apparatus 1X comprises a detection means 31X, an acquisition means 32X, a deformation means 33X, and a display control means 34X. The image processing apparatus 1X may be composed of multiple devices.

[0079] The detection means 31X detects areas of tissue density from the chest image. The detection means 31X can be, for example, the tissue density detection unit 31 in the first embodiment.

[0080] The acquisition means 32X acquires a first image obtained by cutting out a region based on dense areas from a chest image. The "region based on dense areas" may be a region of the chest image that directly corresponds to a dense area, or it may be a region of the chest image that has been expanded or reduced based on a dense area. The acquisition means 32X can be, for example, the first crop image acquisition unit 32 in the first embodiment.

[0081] The deformation means 33X deforms the first image based on information regarding the density of tissue in the first image. Examples of "information regarding the density of tissue in the first image" include the "target tissue mask image" and the "target tissue density image" in the first embodiment. The deformation means 33X can be, for example, the image deformation unit 33 in the first embodiment.

[0082] The display control means 34X displays a second image, which is a modified version of the first image, on the display device. Here, the "display device" may be configured integrally with the image processing device 1X, or it may be a separate device from the image processing device 1X. The display control means 34X can be, for example, the display control unit 34 in the first embodiment.

[0083] Figure 12 is an example of a flowchart showing the processing procedure in the second embodiment. The detection means 31X detects dense tissue areas from the chest image (step S21). Next, the acquisition means 32X acquires a first image by cutting out a region based on the dense tissue areas from the chest image (step S22). The deformation means 33X deforms the first image based on the information regarding tissue density in the first image (step S23). The display control means 34X displays the second image, which is the deformed first image, on the display device (step S24).

[0084] According to the second embodiment, the image processing device 1X deforms a first image relating to a dense tissue area based on information regarding the tissue density, and displays a second image obtained by deforming the first image. This allows the user to appropriately confirm the state of the dense tissue area.

[0085] In each of the embodiments described above, the program can be stored using various types of non-transitory computer-readable medium and supplied to a computer, such as a processor. Non-transitory computer-readable mediums include various types of tangible storage mediums. Examples of non-transitory computer-readable mediums include magnetic storage mediums (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage mediums (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). The program may also be supplied to the computer by various types of transient computer-readable mediums. Examples of transient computer-readable mediums include electrical signals, optical signals, and electromagnetic waves. Transitory computer-readable mediums can supply the program to the computer via wired communication channels such as electric wires and optical fibers, or via wireless communication channels.

[0086] In addition, some or all of the above embodiments (including modifications, the same applies hereinafter) may also be described as follows, but are not limited to the following.

[0087] [Note 1] A detection means for detecting dense areas of tissue from chest images, Acquisition means for acquiring a first image obtained by cutting out a region based on the densely packed area from the chest image, A deformation means for deforming the first image based on information regarding the density of the tissue in the first image, A display control means for displaying a second image obtained by deforming the first image on a display device, An image processing device having [Note 2] The image processing apparatus according to Appendix 1, wherein the display control means displays the second image and information relating to the densely populated area on the display device. [Note 3] The image processing apparatus according to Appendix 2, wherein the display control means displays the second image and the chest image with the region based on the dense area emphasized on the display device. [Note 4] The image processing apparatus according to Appendix 2, wherein the display control means displays the second image and information indicating the relative position of the region based on the dense area on the entire chest on the display device. [Note 5] The image processing apparatus according to Appendix 1, wherein the deformation means generates distribution information showing the distribution of the density of the tissue in a region based on the dense area, based on the first image, as information regarding the dense area, and deforms the first image based on the distribution information. [Note 6] The image processing apparatus according to Appendix 5, wherein the deformation means generates information indicating the presence or absence of the tissue for each unit region in the region based on the dense area, based on the first image, and generates the distribution information based on the information indicating the presence or absence of the tissue. [Note 7] The image processing apparatus according to Appendix 5, wherein the deformation means deforms the first image based on the distribution information to make the density uniform. [Note 8] The image processing apparatus according to Appendix 5, wherein the deformation means calculates a deformation vector for each unit region in the region based on the dense area based on the distribution information, and deforms the first image based on the deformation vector. [Note 9] The detection means detects the densely populated area based on the chest image and the inference model. The inference model is a machine learning model that has acquired the relationship between the chest image and the densely populated area, as described in Appendix 1 of the image processing apparatus. [Note 10] Computers From chest images, dense areas of tissue are detected. A first image is obtained by cutting out a region based on the densely packed area from the aforementioned chest image. Based on the information regarding the density of the tissue in the first image, the first image is deformed, A second image, obtained by transforming the first image, is displayed on the display device. Image processing methods. [Note 11] From chest images, dense areas of tissue are detected. A first image is obtained by cutting out a region based on the densely packed area from the aforementioned chest image. Based on the information regarding the density of the tissue in the first image, the first image is deformed, A storage medium containing a program that causes a computer to perform the process of displaying a second image, which is a modified version of the first image, on a display device.

[0088] Although the present invention has been described above with reference to embodiments, the present invention is not limited to the above embodiments. Various modifications to the structure and details of the present invention can be made as understood by those skilled in the art within the scope of the present invention. That is, the present invention includes the full disclosure, including the claims, and of course, various modifications and alterations that those skilled in the art could make in accordance with the technical idea. Furthermore, the above-mentioned patent and non-patent disclosures are incorporated herein by reference. [Explanation of Symbols]

[0089] 1. 1X Image Processing Device 2 Chest image generation device 3 Display device 4 Control device 11 processors 12 memory 13 Interfaces 100 Chest Image Processing System

Claims

1. A detection means for detecting dense areas of tissue from chest images, Acquisition means for acquiring a first image obtained by cutting out a region based on the densely packed area from the chest image, Based on the first image, a deformation means generates distribution information showing the distribution of the density of the tissue in the region based on the dense area, and deforms the first image based on the distribution information to make the density uniform, A display control means that displays a second image obtained by deforming the first image on a display device, An image processing device having

2. The image processing apparatus according to claim 1, wherein the display control means displays the second image and information relating to the densely populated area on the display device.

3. The image processing apparatus according to claim 2, wherein the display control means displays the second image and the chest image with the region based on the dense area emphasized on the display device.

4. The image processing apparatus according to claim 2, wherein the display control means displays the second image and information indicating the relative position of the region based on the dense area on the entire chest on the display device.

5. The image processing apparatus according to claim 1, wherein the deformation means generates information indicating the presence or absence of the tissue for each unit region in the region based on the dense area, based on the first image, and generates the distribution information based on the information indicating the presence or absence of the tissue.

6. The image processing apparatus according to claim 1, wherein the deformation means calculates a deformation vector for each unit region in the region based on the dense area based on the distribution information, and deforms the first image based on the deformation vector.

7. The detection means detects the densely populated area based on the chest image and the inference model. The image processing apparatus according to claim 1, wherein the inference model is a machine learning model that has learned the relationship between the chest image and the densely populated area.

8. Computers From chest images, dense areas of tissue are detected. A first image is obtained by cutting out a region based on the densely packed area from the aforementioned chest image. Based on the first image, distribution information is generated showing the distribution of the density of the tissue in the region based on the dense area, and based on the distribution information, the first image is transformed to make the density uniform. A second image, obtained by transforming the first image, is displayed on the display device. Image processing methods.

9. From chest images, dense areas of tissue are detected. A first image is obtained by cutting out a region based on the densely packed area from the aforementioned chest image. Based on the first image, distribution information is generated showing the distribution of the density of the tissue in the region based on the dense area, and based on the distribution information, the first image is transformed to make the density uniform. A program that causes a computer to perform the process of displaying a second image, which is a modified version of the first image, on a display device.

Images