Image processing device and program

Abstract

To provide a schema image corresponding to a breast shape in a mammography image, without providing a schema image different each time of taking a mammography image of an identical subject.SOLUTION: A control unit of an image processing device calculates a feature amount regarding a breast shape out of a mammography image, and on the basis of the feature amount regarding the calculated breast shape, selects a schema image corresponding to the breast shape in the mammography image from among a plurality of predetermined types of schema images.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an image processing apparatus and a program. [Background technology]

[0002] Traditionally, mammography images are taken in the MLO (Medio-Lateral Oblique) and CC (Cranio-Caudal) directions and interpreted by specialists. The positions of lesions detected during interpretation are marked on the schematic image by the reader while comparing the mammography image with a schematic image of a standard breast shape, and are then attached to the interpretation report.

[0003] Furthermore, Non-Patent Document 1 describes a technique for creating a schematic image that represents the individual breast shape by performing image recognition on the breast shape from a mammography image, rather than a standard schematic image. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Fujifilm Medical Co., Ltd., "High-speed display of high-definition digital mammography images. Supports a series of workflows from image display to report creation in one machine. Launch of new mammography-dedicated image diagnostic workstation "AMULET Bellus", [online], [searched on January 18, 2021], Internet<URL:http: / / fms.fujifilm.co.jp / news / articlenr_130307.html> Summary of the Invention [Problem to be solved by the invention]

[0005] However, because breast shape varies greatly from person to person, using a standard schematic image may result in a significant difference between the breast shape in the mammography image and the breast shape in the standard schematic image, which can be time-consuming to determine where the lesion position on the mammography image corresponds to on the schematic image. On the other hand, the technology of Non-Patent Document 1 can obtain a schematic image showing the breast shape of each mammography image, but because mammography images are taken by compressing the breast, the breast shape on each mammography image is different even for the same subject's breast. Therefore, even for the same subject's breast, the schematic image will be different for each image (each mammography image), which may cause inconveniences such as difficulty in matching the position of a lesion detected in the past on a schematic image generated based on a mammography image taken this time, which is not preferable.

[0006] An object of the present invention is to provide a schematic image corresponding to the breast shape of a mammography image, without providing a different schematic image each time a mammography image is taken of the same subject. [Means for solving the problem]

[0007] In order to solve the above problem, the image processing device of the invention described in claim 1 comprises: A calculation means for calculating a feature amount relating to a breast shape from a mammography image; a selection means for selecting a schematic image corresponding to the breast shape of the mammography image from among a plurality of predetermined types of schematic images based on the feature amount related to the breast shape calculated by the calculation means; Equipped with 、 The calculation means calculates a feature amount relating to the breast volume and a feature amount relating to the nipple position contained in the mammography image when the mammography image is an MLO image in a medial-lateral oblique direction, and calculates a feature amount relating to the breast volume contained in the mammography image when the mammography image is a CC image in a cranio-caudal direction. .

[0009] Claim 2 The invention described in claim 1 In the invention described in The feature quantity relating to the breast volume is the ratio of the length of the perpendicular line drawn from the nipple to the breast base line in the mammography image to the length of the breast base line.

[0010] Claim 3 The invention described in claim 1 or 2 In the invention described in The feature relating to the nipple position is the ratio of the length of the intersection between the perpendicular line drawn from the nipple to the breast base line and the breast base line and the lower end of the breast base line to the length of the breast base line in the mammography image.

[0011] Claim 4 The invention described in claim 1 to 3 In the invention according to any one of the above, The apparatus further comprises an output means for outputting the schematic image selected by the selection means.

[0012] Claim 5 The program of the invention described in Computer, A calculation means for calculating a feature amount relating to a breast shape from a mammography image; a selection means for selecting a schematic image corresponding to the breast shape of the mammography image from among a plurality of predetermined types of schematic images based on the feature amount relating to the breast shape calculated by the calculation means; Function as 、 The calculation means calculates a feature amount relating to the breast volume and a feature amount relating to the nipple position contained in the mammography image when the mammography image is an MLO image in a medial-lateral oblique direction, and calculates a feature amount relating to the breast volume contained in the mammography image when the mammography image is a CC image in a cranio-caudal direction. . Effect of the Invention

[0013] According to the present invention, it is possible to provide a schematic image corresponding to the breast shape of a mammography image, without providing a different schematic image for each mammography image of the same subject. [Brief description of the drawings]

[0014] [Figure 1]1 is a diagram showing an example of the overall configuration of a medical image system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram showing a functional configuration of the image processing device in FIG. [Diagram 3] 3 is a flowchart showing a schematic image selection process executed by the control unit of FIG. 2. [Figure 4] 11A to 11C are diagrams for explaining a method for calculating a feature amount relating to a breast shape in an MLO image. [Diagram 5] 11A to 11C are diagrams for explaining a method for calculating a feature amount relating to a breast shape in a CC image. [Figure 6] FIG. 13 is a diagram showing examples of schematic images of MLO images and CC images corresponding to each of categories 1 to 6. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiments described below are subject to various technically preferable limitations for carrying out the present invention. Therefore, the technical scope of the present invention is not limited to the following embodiments and illustrated examples.

[0016] (Configuration of medical image system 100) First, the configuration of a medical image system 100 including an image processing device 2 which is an embodiment of the image processing device of the present invention will be described. FIG. 1 shows the system configuration of a medical image system 100 according to this embodiment. As shown in Fig. 1, the medical image system 100 is configured to include a mammography device 1, an image processing device 2, a server device 3, and an image display device 4. These devices 1 to 4 are connected to each other so as to be able to transmit and receive data via a communication network N constructed within a medical facility, such as a LAN (Local Area Network). The number of each device is not particularly limited.

[0017] The mammography device 1 is an X-ray imaging device that takes an X-ray of a breast to generate a digital mammography image (mammography image data) and transmits the image to the image processing device 2 and the server device 3. The mammography device 1 can input various information, such as patient information and examination information, to be added to each generated mammography image from outside, and can also automatically generate the information. The patient information includes patient identification information (e.g., patient ID) for identifying the patient (subject), and information such as the patient's name, sex, and date of birth. The examination information includes examination identification information (e.g., examination ID) for identifying the examination, examination date and time, examination conditions (examination site, laterality (left, right), direction (e.g., craniocaudal (CC), mediolateral oblique (MLO)), and modality type. In this embodiment, a series of images (two MLO images, left and right, and CC images) acquired in a mammography examination of the same patient are assigned the same examination ID and managed. The mammography device 1 adds the above-mentioned patient information, examination information, and a UID (Unique ID) for identifying the image as additional information to the generated mammography image, and transmits the resulting image to the image processing device 2 and server device 3 via the communication network N.

[0018] The image processing device 2 is a computer that selects a schematic image corresponding to the breast shape of the mammography image generated by the mammography device 1 from multiple types of schematic images prepared in advance, and transmits the selected schematic image to the server device 3 in association with the UID of the mammography image.

[0019] Fig. 2 shows an example of the functional configuration of the image processing device 2. As shown in Fig. 2, the image processing device 2 includes a control unit 21, an operation unit 22, a display unit 23, a storage unit 24, and a communication unit 25, and each unit is connected via a bus 26.

[0020] The control unit 21 includes a CPU (Central Processing Unit), a RAM (Random Access Memory ) The CPU of the control unit 21 reads out various programs such as system programs and processing programs stored in the storage unit 24, loads them in the RAM, and executes various processes in accordance with the loaded programs. For example, the control unit 21 functions as a calculation unit and a selection unit by executing a schema image selection process described later.

[0021] The operation unit 22 is configured with a keyboard equipped with letter input keys, number input keys, and various function keys, etc., and a pointing device such as a mouse, and outputs a press signal of a key pressed on the keyboard and an operation signal by the mouse to the control unit 21 as input signals.

[0022] The display unit 23 is configured to include a monitor such as a CRT (Cathode Ray Tube) or LCD (Liquid Crystal Display), and displays various screens according to instructions of a display signal input from the control unit 21.

[0023] The storage unit 24 is configured, for example, with a HDD (Hard Disk Drive), a semiconductor non-volatile memory, etc. The storage unit 24 stores various programs as described above. The storage unit 24 also stores a plurality of types of schematic images of different breast shapes (identification information and image data of the schematic images) and information on classification of breast shapes corresponding to each schematic image (classifications 1 to 6 in this embodiment).

[0024] The communication unit 25 is configured by a LAN card or the like, and transmits and receives data to and from external devices connected to the communication network N via a switching hub.

[0025] The server device 3 and the image display device 4, which is a client, constitute a PACS (Picture Archiving and Communication System). The server device 3 is configured with a control unit, an operation unit, a display unit, a communication unit, a storage unit having an image DB 351, and the like. The server device 3 stores and manages the mammography images generated by the mammography device 1 in the image DB 351 in association with supplementary information (UID, patient information, and examination information) and a schematic image corresponding to the mammography image. In response to an instruction from the image display device 4, the server device 3 causes the image display device 4 to display an examination list screen for the user to select an examination to be interpreted, a viewer screen for reading and displaying the mammography image of the selected examination from the image DB 351, a report screen for displaying a schematic image corresponding to the displayed mammography image, and the like to receive an input of an interpretation report. The server device 3 then causes the image display device 4 to store the interpretation report input from the image display device 4 in association with the examination information in the image DB 351.

[0026] The image display device 4 displays an examination list screen, a viewer screen, a report screen, etc. transmitted from the server device 3, and also displays information input from these screens and transmits it to the server device 3. The image display device 4 functions as an output means for outputting a schema image selected by a schema image selection process described later.

[0027] (Operation of medical imaging system 100) Next, the operation of the medical image system 100 will be described. As described above, breast shape varies greatly from person to person, and when a standard schematic image is used to record lesions in mammography images as in the past, the breast shape in the mammography image may differ significantly from the breast shape in the standard schematic image. In this case, it is time-consuming for the user to understand where the lesion position on the mammography image corresponds to the schematic image. On the other hand, there is also a method of creating a schematic image by recognizing the breast shape from each mammography image. However, because mammography images are taken by compressing the breast, the breast shape in each mammography image will be different, even for the same subject's breast, and the schematic image will be different for each shot (each mammography image), even for the same subject's breast.

[0028] Therefore, in this embodiment, the medical imaging system 100 is equipped with an image processing device 2, which determines whether the breast shape of the mammography image generated by the mammography device 1 is classified into one of several predetermined types based on its characteristics, and selects a schema image according to the determination result from among several types of schema images with different breast shapes as the schema image corresponding to the mammography image.

[0029] 3 is a flowchart showing the flow of the schematic image selection process executed by the control unit 21 of the image processing device 2. The schematic image selection process is executed when a mammography image is received from the mammography device 1 by the communication unit 25, in cooperation with the CPU of the control unit 21 and a program stored in the storage unit 24, with the received mammography image as the processing target.

[0030] First, the control unit 21 extracts a breast region from a received mammography image (step S1). Any known method may be used to extract the breast region from the mammography image. For example, a method may be used in which a histogram analysis is performed on the mammography image to determine the brightness value of the skin line, which is the boundary between the breast region and the non-breast region, and a pixel having the brightness value from the chest wall side to the nipple side is searched for to extract the breast region based on the obtained skin line. Alternatively, for example, a Sobel filter may be used to extract the region where the brightness value (X-ray intensity) changes from the mammography image, and only the region where the degree of change in brightness value is equal to or greater than a threshold is extracted by binarization, and then contour extraction may be performed to extract the breast region.

[0031] Next, the control unit 21 detects the nipple position, breast base line, and nipple reference line from the mammography image (step S2). Figure 4 shows the nipple position a in a mammography image taken in the MLO direction (called an MLO image). mlo , breast base line L1 mlo , nipple reference line L2 mlo FIG. 5 shows the nipple position a in a mammography image in the CC direction (called a CC image). cc , breast base line L1 cc , nipple reference line L2 cc In addition, parameters in the MLO image are denoted by the subscript mlo, and parameters in the CC image are denoted by the subscript cc.

[0032] Nipple position a mlo (a cc ) can be detected using known methods described in, for example, JP-A-2015-104464, JP-A-2015-104465, etc. Breast base line L1 mlo (L1 cc ) is the line that marks the outer edge of the base of the breast on a mammogram, and the upper edge of the breast c mlo (c cc ) and the lower end of the breast d mlo (d cc ) is the breast base line L1 mlo (L1 cc In the MLO image, the upper edge of the breast region on the chest wall side of the mammography image is detected as the upper edge of the breast c mlo The upper end of the breast c mlo (c cc ) or the lower edge of the breast d mlo (d cc If the breast base line L1 cannot be detected, detect the chest wall line and mlo (L1 cc ) or the image edge on the chest wall side in the breast region of the mammography image may be the breast base line L1 mlo (L1 cc ) can also be used. Nipple reference line L2 mlo(L2 cc ) is nipple position a mlo (a cc ) to breast base line L1 mlo (L1 cc ) can be detected by drawing a perpendicular line to the

[0033] Next, the control unit 21 calculates features relating to the breast shape from the mammography image, and classifies the breast shape of the breast included in the mammography image into one of a plurality of types based on the calculated features (step S3). Although breast shapes vary greatly from person to person, they can be classified into a number of types, for example, according to the characteristics of the breast shape. There is no particular restriction on the method of classifying breast shapes. In this embodiment, an example of classifying breast shapes into the following six types based on the volume (fullness) of the breast and the position of the nipple will be described. Category 1: Low volume and flat. Category 2: More voluminous than Category 1 but slightly smaller. Category 3: Breasts hang diagonally downwards. Category 4: More voluminous and plump than Category 2. Category 5: The top of the nipple is flat and tends to droop. Category 6: Even larger in volume than Category 4, protruding upwards.

[0034] First, the breast shape in the MLO image is calculated using, for example, p mlo and q mlo An example of classification using the above method will be described. p mlo is the breast base line L1 in the MLO image. mlo Nipple position relative to length a mlo From breast base line L1 mlo The perpendicular line (nipple reference line L2 mlo In the case of the example shown in FIG. 4, it can be calculated by the following formula (1): p mloThe value is larger when the breast volume is large and the value is smaller when the volume is small.

number

number

[0035] The control unit 21 calculates the above p from the MLO image. mlo andq mlo Calculate the value of p mlo andq mlo The breast shape of the MLO image is classified into one of categories 1 to 6 by comparing the value of with a predetermined threshold as shown in Table I below. [Table 1] In addition, the p of each category 1 to 6 shown in Table I mlo, q mlo The ranges are merely examples and are not limited to those shown in Table 1.

[0036] Next, an example of classifying breast shapes in CC images will be described. Even if the breast shapes are different, such as in categories 1 to 6, the breast shapes in the CC images are similar and semicircular due to compression during imaging. Therefore, it is difficult to classify the breast shapes in the CC images into categories 1 to 6. In this example, the feature quantity p cc Based on this, we will explain how to classify them into two categories: Category 1 and other categories 2 to 6.

[0037] p cc is the breast base line L1 in the CC image. cc Nipple position relative to length a cc From breast base line L1 cc The perpendicular line (nipple reference line L2 cc In the case of the example shown in FIG. 5, it can be calculated by the following formula (3): p cc The value is larger when the breast volume is large and the value is smaller when the volume is small.

number

[0038] Next, the control unit 21 selects a schematic image corresponding to the breast shape of the mammography image from among a plurality of types of schematic images based on the breast shape classification result (step S4), and ends the schematic image selection process.

[0039] 6 is a diagram showing examples of MLO images and CC image schema images corresponding to each of categories 1 to 6. In the MLO images, since breast shapes of categories 2, 4, and 5 are similar, the same schema image is selected in this embodiment, but a different schema image may be prepared for each category and a different schema image may be selected for each category. Also, in this embodiment, the same schema image is selected for CC image categories 2 to 6, but a different schema image may be prepared for each category and a different schema image may be selected for each category. In this way, the features of the breast shape of the mammography image are calculated, and a schema image corresponding to the breast shape of the mammography image is selected from multiple types of schema images based on the calculated features.This makes it possible to provide a schema image corresponding to the breast shape of the mammography image without providing a different schema image each time a mammography image is taken for the same subject.

[0040] When the schematic image selection process is completed, the control unit 21 transmits the UID of the received mammography image and the selected schematic image to the server device 3 via the communication unit 25 in association with each other.

[0041] When the server device 3 receives the UID and the schematic image transmitted from the image processing device 2, the server device 3 stores the received schematic image in the image DB 351 in association with the UID. Furthermore, when an examination is selected from the examination list screen displayed on the image display device 4 and an instruction is given to display a viewer screen for the selected examination, the server device 3 reads out a mammography image of the selected examination from the image DB 351, transmits it to the image display device 4, and causes the image display device 4 to display the viewer screen displaying the mammography image. Furthermore, when an instruction is given to display a report screen for the selected examination, the server device 3 reads out a schematic image corresponding to the mammography image of the selected examination from the image DB 351, transmits it to the image display device 4, and causes the image display device 4 to display the report screen displaying the schematic image.

[0042] The reader reads the mammography image displayed on the viewer screen of the image display device 4, and if a lesion is detected, marks the location of the lesion on the schematic image displayed on the report screen. At this time, because a schematic image corresponding to the breast shape in the mammography image is selected, the reader can significantly reduce the effort of associating the breast shape in the mammography image with that in the schematic image and understanding where the lesion location on the mammography image corresponds to on the schematic image. In addition, because a different schematic image is not provided for each mammography image taken for the same subject, it becomes possible to easily compare the lesion location detected in the past with the lesion location detected in the mammography image taken this time on the schematic image.

[0043] Although the embodiment of the present invention has been described above, the description of the above embodiment is a preferred example of the present invention, and the present invention is not limited to this. For example, in the above embodiment, the image processing device 2 has been described as a device separate from the server device 3, but the server device 3 may have the functions of the image processing device 2. For example, when the server device 3 receives a mammography image from the mammography device 1, the control unit of the server device 3 may execute the schema image selection process shown in FIG. 3 for the received mammography image, and the selected schema image may be stored in the image DB 351 in association with the UID. Alternatively, the image display device 4 may have the functions of the image processing device 2. For example, when an examination is selected from the examination list screen displayed on the image display device 4, the control unit of the image display device 4 may read out the mammography image of the selected examination from the image DB 351, and execute the schema image selection process shown in FIG. 3 for the read out mammography image. The selected schema image may then be displayed (output) on the report screen.

[0044] In the above embodiment, after selecting a schematic image, the control unit 21 associates the UID of the received mammography image with the selected schematic image and transmits them to the server device 3 via the communication unit 25. However, a plurality of types of schematic images may be stored in the server device 3 in association with identification information of the schematic images, and after selecting a schematic image, the control unit 21 may associate the UID of the received mammography image with the identification information of the selected schematic image and transmit them to the server device 3 via the communication unit 25. The server device 3 may then associate the identification information of the received schematic image with the UID and store it in the image DB 351. The identification information of the schematic image may use the classification name (classifications 1 to 6) of the mammography image corresponding to that schematic image.

[0045] In the above embodiment, the feature quantity related to the breast shape of the mammography image is p mlo , p cc The feature quantity related to the nipple position, q mloHowever, the present invention is not limited to this, and other feature amounts may be used.

[0046] In the above description, a hard disk, a non-volatile semiconductor memory, or the like is used as a computer-readable medium for the program according to the present invention, but the present invention is not limited to this example. As other computer-readable media, a portable recording medium such as a CD-ROM can be used. Furthermore, a carrier wave can be used as a medium for providing data for the program according to the present invention via a communication line.

[0047] In addition, the detailed configuration and operation of each device constituting the medical image system may be modified as appropriate without departing from the spirit of the present invention. [Explanation of symbols]

[0048] 100 Medical Imaging Systems 1. Mammography equipment 2. Image Processing Device 21 Control section 22 Control section 23 Display section 24 Memory section 25 Communications Department 3 Server equipment 4 Image display device

Claims

1. A calculation means for calculating a feature amount relating to a breast shape from a mammography image; a selection means for selecting a schematic image corresponding to the breast shape of the mammography image from among a plurality of predetermined types of schematic images based on the feature amount relating to the breast shape calculated by the calculation means, The calculation means is an image processing device that calculates features related to the breast volume and features related to the nipple position contained in the mammography image when the mammography image is an MLO image in the medial-lateral oblique direction, and calculates features related to the breast volume contained in the mammography image when the mammography image is a CC image in the cranio-caudal direction.

2. The image processing apparatus according to claim 1 , wherein the feature amount relating to the breast volume is a ratio of a length of a perpendicular line drawn from a nipple to the breast base line in the mammography image to a length of the breast base line.

3. 3. The image processing device according to claim 1, wherein the feature relating to the position of the nipple is a ratio of the length of the intersection of a perpendicular line drawn from the nipple to the breast base line and the breast base line to the lower end of the breast base line, relative to the length of the breast base line in the mammography image.

4. 4. The image processing device according to claim 1, further comprising an output unit that outputs the schematic image selected by the selection unit.

5. Computer, A calculation means for calculating a feature amount relating to a breast shape from a mammography image; a selection means for selecting a schematic image corresponding to the breast shape of the mammography image from among a plurality of predetermined types of schematic images based on the feature amount relating to the breast shape calculated by the calculation means; Function as a The calculation means is a program that calculates features related to the breast volume and features related to the nipple position contained in the mammography image when the mammography image is an MLO image in the medial-lateral oblique direction, and calculates features related to the breast volume contained in the mammography image when the mammography image is a CC image in the cranio-caudal direction.

Images