Report creation support device

Abstract

To reduce a burden on an image-reading doctor in writing an image-reading report.SOLUTION: A report writing support device according to an embodiment comprises an identification unit and a report writing unit. When receiving an input to the effect that an image-reading order regarding a first medical image obtained by imaging a portion of a patient with a first modality is selected in response to an operation by a user, the identification unit designates a predetermined area of the first medical image as an area of interest and identifies a second medical image corresponding to the designated area of interest from among second medical images obtained by imaging the portion of the patient with a second modality different from the first modality. The report writing unit pastes the first medical image and the identified second medical image in a predetermined area of an image-reading report created for the image-reading order.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The embodiments disclosed in this specification and the drawings relate to a report creation support device.

Background Art

[0002] During breast cancer screening, a mammography device takes images of the left and right breasts of a patient while individually compressing them in the first direction, and also takes images while individually compressing the left and right breasts in the second direction. The mammography images obtained by the imaging are a total of four images: the first-direction image of the left breast, the first-direction image of the right breast, the second-direction image of the left breast, and the second-direction image of the right breast. For example, the first direction may be the cranio-caudal direction (CranioCaudal projection: CC), and the second direction may be the medio-lateral oblique direction (MedioLateral Oblique projection: MLO). Also, for example, the first direction may be the medio-lateral oblique direction (MLO), and the second direction may be the cranio-caudal direction (CC). In any case, in the following examples, the CC-direction image is also referred to as the CC image, and the MLO-direction image is also referred to as the MLO image.

[0003] Basically, these four mammography images are read in pairs of the same sides of the left and right. For example, the CC image and the MLO image of the left breast are read in a pair, and the CC image and the MLO image of the right breast are read in a pair.

[0004] In recent years, in consideration of patients with high-density breasts (dense breasts), not only mammography images taken by a mammography device but also ultrasound images taken by an ultrasound diagnostic device are used in combination for reading.

[0005] Thus, while the images to be read are diversifying, the radiologist needs to manually select the images to be read and manually input the reading results for the images to create a reading report. This is a great burden on the radiologist, and the realization of a technology that can reduce this burden is desired.

Prior Art Documents

[0006] [Patent Document 1] Japanese Patent Application Publication No. 10-108859 [Overview of the project] [Problems that the invention aims to solve]

[0007] The problem that this invention aims to solve is to reduce the burden on radiologists when preparing image interpretation reports. [Means for solving the problem]

[0008] The report creation support device according to the embodiment comprises a selection unit and a report creation unit. When the selection unit receives input that an order for image interpretation of a first medical image taken of a patient's body part with a first modality has been selected in response to user operation, it designates a predetermined area of ​​the first medical image as a region of interest and identifies a second medical image corresponding to the designated region of interest from among second medical images taken of the patient's body part with a second modality different from the first modality. The report creation unit attaches the first medical image and the identified second medical image to a predetermined area of ​​the image interpretation report created for the image interpretation order. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a block diagram showing the configuration of a medical information processing system equipped with a report creation support device according to one embodiment. [Figure 2] Figure 2 is a schematic diagram illustrating an example of a cross-reference function in one embodiment. [Figure 3] Figure 3 is a schematic diagram illustrating another example of the cross-reference function in one embodiment. [Figure 4] Figure 4 is a schematic diagram illustrating an example of a region narrowing function in one embodiment. [Figure 5]Figure 5 is a schematic diagram illustrating an example of a region narrowing function in one embodiment. [Figure 6] Figure 6 is a schematic diagram illustrating the echo scan guiding process in one embodiment. [Figure 7] Figure 7 is a schematic diagram illustrating the cross-referencing process in body mark processing in one embodiment. [Figure 8] Figure 8 is a schematic diagram illustrating the cross-referencing process in one embodiment. [Figure 9] Figure 9 is a schematic diagram illustrating an example of the lesion site enhancement function in one embodiment. [Figure 10] Figure 10 is a schematic diagram illustrating an example of the lesion site enhancement function in one embodiment. [Figure 11] Figure 11 is a schematic diagram illustrating an example of the lesion site enhancement function in one embodiment. [Figure 12] Figure 12 is a schematic diagram illustrating the image interpretation report screen in one embodiment. [Figure 13] Figure 13 is a flowchart illustrating a series of processing steps performed by a report creation support device according to one embodiment. [Modes for carrying out the invention]

[0010] The following describes in detail an embodiment of the report creation support device with reference to the drawings. Furthermore, the disclosure is merely an example, and any modifications that a person skilled in the art could easily conceive of while maintaining the spirit of the invention are naturally included within the scope of the present invention. In addition, in this specification and each figure, components that perform the same or similar functions as those described above with respect to previously shown figures are denoted by the same reference numerals, and redundant detailed descriptions may be omitted.

[0011] Figure 1 is a block diagram showing the configuration of a medical information processing system equipped with a report creation support device according to one embodiment. In this medical information processing system, a mammography imaging device 10, an ultrasound diagnostic device 30, an image storage device 50, an image observation device 60, a report creation support device 70, and a report storage device 90 are able to communicate via a network Nw. Note that the image observation device 60 may be mounted on the mammography imaging device 10, as shown by the dashed line as image observation device 20. In other words, the processing circuit (not shown) of the mammography imaging device 10 may operate to realize the functions of the image observation device 20 in addition to the functions for mammography imaging. Alternatively, the image observation device 60 may be mounted on the ultrasound diagnostic device 30, as shown by the dashed line as image observation device 40. In other words, the processing circuit (not shown) of the ultrasound diagnostic device 30 may operate to realize the functions of the image observation device 60 in addition to the functions for ultrasound diagnosis. Alternatively, the image observation device 60 may be mounted on the report creation support device 70, as shown by the dashed line as image observation device 80. In other words, the processing circuit of the report creation support device 70 may operate to implement the functions of the image observation device 60, in addition to the functions for creating the image interpretation report.

[0012] Here, the mammography device 10 is a device capable of acquiring mammography images, which are medical images. Specifically, the mammography device 10 acquires images of the patient's (subject's) left and right breasts individually while they are compressed in a first direction, and also acquires images of the same left and right breasts individually while they are compressed in a second direction. The mammography images obtained from the acquisition consist of a total of four images: a first-direction image of the left breast, a first-direction image of the right breast, a second-direction image of the left breast, and a second-direction image of the right breast. Hereafter, the first-direction image of the left breast and the first-direction image of the right breast will also be referred to as the "first mammography image." Similarly, the second-direction image of the left breast and the second-direction image of the right breast will also be referred to as the "second mammography image." When "first mammography image" and "second mammography image" are not distinguished, each of the four images will also be referred to as the "mammography image." For the first and second directions, for example, the craniocaudal projection (CC), medial-lateral projection (ML), and medial-lateral oblique projection (MLO) can be used as appropriate, with the CC direction and either the MLO direction or the ML direction being used. In this embodiment, the CC direction and the MLO direction are used as an example. In this case, the "CC direction image" and the "MLO direction image" are also referred to as the "CC image" and the "MLO image," respectively.

[0013] The ultrasound diagnostic device 30 is a device capable of acquiring ultrasound images, which are medical images. It can acquire images by transmitting and receiving ultrasound waves while an ultrasound probe (not shown) is placed at a desired position on the patient's breast. The ultrasound diagnostic device 30 may also guide the patient to the correct position within the breast where the ultrasound probe should be applied by displaying mammography images or body marks on the breast. Furthermore, the ultrasound diagnostic device 30 may be equipped with a position sensor for the ultrasound probe so that the position of the ultrasound probe can be displayed on the body marks on the breast.

[0014] The image storage device 50 is a storage device that stores multiple types of image data captured by the mammography device 10 and the ultrasonic diagnostic device 30. The image storage device 50 may also be called an image server device. Each piece of image data is a so-called DICOM (Digital Imaging Communications and Medicine) image file, which is a standard specification in the field of image examination, and has attached information. The attached information consists of multiple attached items. The attached information always includes an attached item of a patient ID for identifying the patient. In addition to the patient ID, the attached items include a modality ID for identifying the mammography device 10 and the ultrasonic diagnostic device 30, imaging date information indicating the imaging date, imaging site information indicating the imaging site, type information indicating whether the image data is a morphological image or a functional image, image collection conditions (scan conditions, imaging conditions), information on the contrast agent (type of contrast agent, injection volume, injection time, etc.), examination order (examination site, examination purpose, etc.), examination ID, and at least one of the series ID. Also, in this embodiment, the attached information is assumed to include the various attached items described above, but is not limited thereto, and may further include, for example, an image ID assigned to each piece of image data.

[0015] The image observation device 60 includes a communication interface 61, a memory 62, an input interface 63, a display 64, and a processing circuit 65. The image observation device 60 has various functions for assisting in the reading of the image data stored in the image storage device 50.

[0016] The communication interface 61 is a circuit for connecting the image observation device 60 to the network Nw and communicating with other devices. As the communication interface 61, for example, a network interface card (NIC) can be used. In the following description, the description of the communication interface 61 intervening in the communication between the image observation device 60 and other devices will be omitted.

[0017] Memory 62 consists of memory for recording electrical information, such as ROM (Read Only Memory), RAM (Random Access Memory), HDD (Hardware Disk Drive), and image memory, as well as peripheral circuits such as memory controllers and memory interfaces associated with these memories. Memory 62 stores various data sets, such as the image observation program of this image observation device, mammography images, body marks, ultrasound images, and ultrasound image acquisition position information.

[0018] The input interface 63 is implemented by a trackball, switch buttons, mouse, keyboard, touchpad (or trackpad) for inputting various instructions, commands, information, selections, and settings from the operator (user) to the main image observation device, and a touch panel display (or touchscreen) that integrates a display screen and a touchpad. The input interface 63 is connected to the processing circuit 65 and converts the input operations received from the user into electrical signals and outputs them to the processing circuit 65. In this case, the input interface 63 may display a user interface (GUI: Graphical User Interface) on the display 64 for the user to input various instructions using physical operating components such as a mouse or keyboard. In this specification, the input interface 63 is not limited to those equipped with physical operating components. For example, an electrical signal processing circuit that receives electrical signals corresponding to input operations from an external input device provided separately from the device and outputs these electrical signals to the processing circuit 65 is also included as an example of the input interface 63. In the following description, "operation of the input interface 63 by the user" is also referred to as "user operation".

[0019] The display 64 consists of a display unit for displaying medical images and the like, an internal circuit that supplies display signals to the display unit, and peripheral circuits such as connectors and cables that connect the display and the internal circuit. The display 64 can, for example, display a first mammography image stored in memory 62, a schematic diagram generated by the processing circuit 65, and a predetermined range calculated by that circuit.

[0020] The processing circuit 65 reads the image observation program stored in the memory 62 based on instructions input by the user via the input interface 63, and controls the image observation device 60 accordingly. For example, the processing circuit 65 is a processor that implements each function of the image observation device 60 according to the image observation program read from the memory 62. Here, each function may be, for example, an image processing function 66, a display control function 67, and a user support function 68. The image processing function 66 may be, for example, a skin line detection function 66a, a breast feature detection function 66b, a cross-reference function 66c, and a region narrowing function 66d. The user support function 68 may be, for example, an operation restriction function 68a, an image interpretation support function 68b, an ESG processing function 68c, and a body mark search function 68d. ESG is an abbreviation for echo scan guide, which will be described later. Note that each function may be implemented by distributing it among multiple processors as appropriate. For example, the skin line detection function 66a, breast feature detection function 66b, and cross-reference function 66c may be executed by the first processor, while the remaining functions may be executed by the second processor. Alternatively, each function may be executed by an external cloud or an image observation device 20, 40, or 80 in another device, as appropriate. For example, the image processing function 66 may be executed by an external cloud or by a processing circuit (not shown) in the image observation device 20 within the mammography imaging device 10, while the remaining display control function 67 and user support function 68 may be executed by a processing circuit 65. Alternatively, the image processing function 66 may be executed by a processing circuit 65, while the remaining display control function 67 and user support function 68 may be executed by a processing circuit (not shown) in the image observation device 40 within the ultrasound diagnostic device 30. Furthermore, the skin line detection function 66a, breast feature detection function 66b, region narrowing function 66d, and user support function 68 are optional additional features and may be omitted. If the skin line detection function 66a and the breast feature detection function 66b are omitted, for example, the user may indicate skin lines or features on the mammography image through user operation.

[0021] Next, we will describe in order the skin line detection function 66a, breast feature detection function 66b, cross-referencing function 66c, and region narrowing function 66d provided by the image processing function 66.

[0022] The skin line detection function 66a detects the skin line (breast contour) in the mammography image. The skin line detection function 66a may, for example, determine a threshold based on a histogram of pixel values ​​in the mammography image and use that threshold to detect the skin line. Alternatively, the skin line detection function 66a may detect the skin line using, for example, an edge enhancement filter or a Sobel filter.

[0023] The breast feature detection function 66b detects breast features (both ends of the breast, nipple, chest wall) from mammography images in which skin lines have been detected. The breast feature detection function 66b may, for example, detect breast features by bringing a vertical or diagonal line close to the skin line and determining the position where the line and the skin line come into contact. Alternatively, the breast feature detection function 66b may, for example, detect other breast features based on previously detected breast features.

[0024] The cross-reference function 66c selects a candidate lesion from a first mammography image obtained by compressing the patient's breast in a first direction. The selection of a candidate lesion may be performed, for example, in response to user input, or in response to the coordinates of a candidate lesion included in the results of computer-aided detection (CAD). Alternatively, the selection of a candidate lesion may be performed in response to user input on the CAD results. The term "candidate lesion" may also be referred to as "lesion." The cross-reference function 66c also calculates a target region corresponding to the location of the candidate lesion in a second mammography image obtained by compressing the breast in a second direction different from the first direction. The location of the candidate lesion is the coordinates of the selected candidate lesion (coordinates in the first mammography image). In this embodiment, as described above, the CC direction and MLO direction are used as examples of the first and second directions. Here, the first direction may be the CC direction and the second direction may be the MLO direction. Conversely, the first direction may be the MLO direction and the second direction may be the CC direction. Also, as mentioned above, CC-direction images are also called CC images, and MLO-direction images are also called MLO images.

[0025] For example, Figure 2 shows an example where the first direction is the CC direction and the second direction is the MLO direction. As shown in the upper part of Figure 2, suppose that two lesion candidates are observed in the CC image 100R and MLO image 110R of the right breast, while the CC image 100L of the right breast, the MLO image 110R of the right breast, and the MLO image 110L of the left breast are being displayed. At this time, the cross-reference function 66c selects lesion candidate 120 by enclosing one of the lesion candidates 1 and 2 in the CC image 100R of the right breast with a curve, for example, in response to user operation. Subsequently, as shown in the lower part of Figure 2, the cross-reference function 66c calculates the target region 121 in the MLO image 110R of the right breast that corresponds to lesion candidate 1.

[0026] For example, Figure 3 shows an example where the first direction is the MLO direction and the second direction is the CC direction. As shown in the upper part of Figure 3, suppose that two lesion candidates are observed in the MLO image 110R and CC image 100R of the right breast, while the MLO image 110R of the right breast, the MLO image 110L of the left breast, the CC image 100R of the right breast, and the CC image 100L of the left breast are being displayed. At this time, the cross-reference function 66c selects lesion candidate 120 by enclosing one of the lesion candidates 1 and 2 in the MLO image 110R of the right breast with a curve, for example, in response to user operation. Subsequently, as shown in the lower part of Figure 3, the cross-reference function 66c calculates the target region 121 in the CC image 100R of the right breast that corresponds to lesion candidate 1.

[0027] The region filtering function 66d narrows down the target region to the lesion candidate region corresponding to the value of the lesion candidate, based on the value of the lesion candidate or a specific value. Here, the specific value may be a preset value or a value instructed by the user, and the value may be a pixel value or a range of pixel values. Such a region filtering function 66d narrows down the lesion candidate region 122 from the target region 121 of the MLO image 110R based on the value of lesion candidate 1 in the CC image 100R, as shown in Figure 4. Alternatively, as shown in Figure 5, the region filtering function 66d narrows down the lesion candidate region 122 from the target region 121 of the CC image 100R based on the value of lesion candidate 1 in the MLO image 110R.

[0028] Furthermore, if the region narrowing function 66d obtains multiple regions as candidate lesion regions from the target region, it may narrow down the candidate lesion region to the region with the highest priority among the multiple regions. Here, any or a combination of the following (a) to (d) can be used as appropriate for assigning priority.

[0029] (a) High relevance: The highest priority is given to the lesion with the smallest difference between the lesion-nipple distance in the second mammography image and the lesion-nipple distance in the first mammography image, according to the calculation formula. This is because lesions with the most similar distances to the nipple across mammography images taken in different directions are presumed to be the same lesion. Therefore, for example, the calculation formula gives a higher priority to lesions that are closer to the nipple (because they have a higher relevance). Specifically, for example, the region filtering function 66d may assign the highest priority to the distance that is closest to the distance between the nipple and the candidate lesion in the first mammography image, among the multiple distances between the nipple and the multiple regions in the second mammography image.

[0030] (b) Tonal difference: Higher priority is given to areas with a similar tonal difference. In this case, it is preferable to compare the tonal differences after equalizing the tonal levels between images. Alternatively, the tonal differences may be compared after adjusting for dose, current, compression force, and breast thickness obtained from the image header.

[0031] (c) Size proximity: The size that is closest to the size of the target area among the sizes of multiple areas is assigned the highest priority. The size to be compared is not limited to the size obtained from the image, but may also be the size that takes into account the rigid body transformation of the breast.

[0032] (d) Position within the image: Prioritization may be given in ascending or descending order from the center of the breast outwards within the image. Alternatively, priority may be given in ascending or descending order from the top of the breast downwards within the image.

[0033] The display control function 67 is a function that displays images based on various image data generated by the image processing function 66 on the display unit, the display 64. Specifically, for example, the display control function 67 controls the display 64 to display a first mammography image that clearly shows the lesion candidate selected by the image processing function 66, and a second mammography image that includes the target area calculated by the image processing function 66 in an identifiable state. Here, "identifiable state" is not limited to methods that clearly show the target area, such as by a border, color change, or cropping, but is sufficient if the target area is identified by some method, such as showing the target area only when a button is pressed. The display control function 67 also controls the display 64 to display a third mammography image, which is a second mammography image that clearly shows the lesion candidate area, in a display area different from the first and second mammography images. Furthermore, the display control function 67 controls the display 64 to display an image based on the processing results of the user assistance function 68.

[0034] Next, we will describe in order the operation restriction function 68a, image interpretation support function 68b, ESG processing function 68c, and body mark search function 68d provided by the user support function 68.

[0035] The operation restriction function 68a accepts operations on mammography images that explicitly indicate a candidate lesion area, but ignores operations on areas other than the candidate lesion area. In other words, the operation restriction function 68a restricts operations on mammography images containing a candidate lesion area to operations on that candidate lesion area only. Here, operations on a candidate lesion area may be operations for interpreting that candidate lesion area. Furthermore, such interpretation operations may be, for example, operations to change the grayscale of the candidate lesion area, operations to change the size of the candidate lesion area, operations to add information about the candidate lesion area, click operations on the candidate lesion area, and operations to crop the candidate lesion area. For example, an operation to add information may be an operation to select a candidate lesion area in the first mammography image, then select a candidate lesion area in the second mammography image, and then add information to that candidate lesion area. The information to be added may include, for example, at least one of the following: the result of size measurement of the candidate lesion, annotations, findings, analysis results, etc. The click operation may also serve as an operation to change the gradation or size, an operation to add information, or an operation to crop. The cropping operation may also be called a trimming operation or a masking operation. Masking is the process of displaying only the candidate lesion area and hiding other areas.

[0036] The image interpretation support function 68b is a function for assisting in the interpretation of mammography images. Such an image interpretation support function 68b may, for example, perform a detection function to detect lesion candidates from a target area using computer-aided detection (CAD). The detection function is an example of a detection unit. Alternatively, for example, the image interpretation support function 68b may perform a processing function to process the mammography image in response to operations for interpreting the lesion candidate area. Processing functions that can be used as appropriate include, for example, a grayscale change function to change the grayscale of the lesion candidate area, a size change function to change the size of the lesion candidate area, and a cropping function to cut out the lesion candidate area.

[0037] The ESG processing function 68c performs ESG (Echo Scan Guide) processing to link mammography images and ultrasound examinations. ESG is a function that, when a lesion candidate is selected in the MLO image and CC image, creates and displays a body mark BM (mammary diagram) that includes MLO direction lines and CC direction lines corresponding to the location of the lesion candidate. For example, as shown in Figure 6, when a lesion candidate (circle) is selected in the MLO image 110R, a body mark BM including MLO direction lines (diagonal lines) corresponding to the location of the lesion candidate is created and displayed in the MLO image 110R. Also, as shown in Figure 6, when a lesion candidate (circle) is selected in the CC image 100R, a body mark BM including CC direction lines (vertical lines) corresponding to the location of the lesion candidate and the MLO direction lines described above is created and displayed in the CC image 100R. The intersection of the MLO direction lines and CC direction lines corresponds to the location of the lesion candidate. Such body mark BMs including MLO direction lines and CC direction lines are stored in memory 62 or image storage device 50 in association with the MLO image and CC image. With this type of ESG, during an ultrasound examination, MLO images, CC images, and body marks BM indicating the location of potential lesions can be displayed on the display 64. This makes it possible to instruct the position on the breast where the ultrasound probe should be applied, according to the potential lesions (interpretation results) on the mammography image.

[0038] The body mark search function 68d creates and updates the body mark BM based on the position of the ultrasound probe during an ultrasound examination. The created and updated body mark BM is stored in memory 62 or image storage device 50 in association with the ultrasound image. The body mark search function 68d also searches for the position on the mammography image based on the position of the ultrasound probe on the body mark BM and displays the scan area corresponding to the search result on the mammography image. For example, as shown in Figure 7, the area 124 corresponding to the position Ps of the ultrasound probe on the body mark BM becomes a long area (band-shaped area) in the vertical and horizontal directions on the MLO image 110R and CC image 100R. Here, for example, as shown in Figure 8, if a point such as a candidate lesion 120 is selected within area 124 of one CC image 100R, the cross-referencing operation restriction function 68a narrows the target area to area 125 in the other MLO image 110R, and the operation becomes even more limited. Note that "body mark search" may also be called "body mark processing". The image set shown in Figure 7, namely the ultrasound image and the MLO image 110R and CC image 100R of the right breast, in which the region 124 corresponding to the position Ps of the ultrasound probe on the body mark BM is enclosed in a frame, are stored, for example, in memory 62 or image storage device 50. Furthermore, the image set shown in Figure 8, namely the MLO image 110R and CC image 100R of the right breast, in which the region 124 corresponding to the position Ps of the ultrasound probe on the body mark BM is enclosed in a frame, and the region 125 corresponding to the lesion candidate 120 is enclosed in another frame, are stored, for example, in memory 62 or image storage device 50.

[0039] To elaborate, the body mark search function 68d acquires, in advance at the start of the ultrasound examination, the imaging position information of the ultrasound image corresponding to the position of the patient's nipple (the position information of the ultrasound probe), and associates the position of the nipple with the origin of the body mark coordinates. In addition, at the start of the ultrasound examination, imaging position information of the ultrasound image corresponding to each position on both ends of the patient's breast may be acquired in advance, and each position on both ends of the breast may be associated with the positions on both ends of the body mark. The origin of the body mark is the origin of the ZY coordinate system, where the vertical axis in the body mark is the Z axis and the horizontal axis is the Y axis, and it indicates the position of the nipple within the body mark. Each position corresponding to both ends of the body mark corresponds to each position on both ends of the patient's breast on the circumference centered on the origin, as well as the ends of the breast in the mammography image. In this way, by associating the imaging position information of the ultrasound image corresponding to characteristic breast positions (nipple, both ends of the breast) with characteristic body mark positions (origin, ends), it becomes possible to draw marks indicating the imaging position information of the ultrasound image (the position of the ultrasound probe) on the body mark.

[0040] The report creation support device 70 includes a communication interface 71, a memory 72, an input interface 73, a display 74, and a processing circuit 75.

[0041] The communication interface 71 is a circuit that connects the report creation support device 70 to the network Nw and communicates with other devices. For example, a network interface card (NIC) can be used as the communication interface 71. In the following description, the fact that the communication interface 71 is involved in communication between the report creation support device 70 and other devices will be omitted.

[0042] Memory 72 consists of memory for recording electrical information, such as ROM (Read Only Memory), RAM (Random Access Memory), HDD (Hardware Disk Drive), and image memory, as well as peripheral circuits such as memory controllers and memory interfaces associated with these memories. Memory 72 stores the report creation support program of this report creation support device, various data sets acquired for creating image interpretation reports (for example, mammography images, ultrasound images, body marks, and past image interpretation reports).

[0043] The input interface 73 is implemented by a trackball, switch buttons, mouse, keyboard, touchpad (or trackpad) for inputting various instructions, commands, information, selections, and settings from the operator (user) to the main image observation device, and a touch panel display (or touchscreen) that integrates a display screen and a touchpad. The input interface 73 is connected to the processing circuit 75 and converts the input operations received from the user into electrical signals and outputs them to the processing circuit 75. In this case, the input interface 73 may display a user interface (GUI: Graphical User Interface) on the display 74 for the user to input various instructions using physical operating components such as a mouse or keyboard. In this specification, the input interface 73 is not limited to those equipped with physical operating components. For example, an electrical signal processing circuit that receives electrical signals corresponding to input operations from an external input device provided separately from the device and outputs these electrical signals to the processing circuit 75 is also included as an example of the input interface 73. In the following description, "operation of the input interface 73 by the user" will also be referred to as "user operation".

[0044] The display 74 consists of a display unit that displays medical images and interpretation reports created from the interpretation of those medical images, an internal circuit that supplies display signals to the display unit, and peripheral circuits such as connectors and cables that connect the display and the internal circuit. It is desirable that the display 74 be provided in multiple units, such as one for displaying the medical image to be interpreted and another for displaying the interpretation report created from the interpretation of those medical images.

[0045] The processing circuit 75 reads a report creation support program stored in memory 72 based on instructions input by the user via the input interface 73, and controls the report creation support device 70 accordingly. For example, the processing circuit 75 is a processor that implements each function of the report creation support device 70 according to the report creation support program read from memory 72. Here, each function may be, for example, a report creation support function 76 and a display control function 77. The report creation support function 76 may be, for example, a related image identification function 76a, a key image attachment function 76b, and a lesion site enhancement function 76c. Each function may be implemented by distributing it among multiple processors as appropriate. For example, the report creation support function 76 may be executed by the first processor, and the display control function 77 may be executed by the second processor. Alternatively, each function may be executed by an external cloud as appropriate. For example, the report creation support function 76 may be executed by an external cloud, and the display control function 77 may be executed by the processing circuit 75. Alternatively, the report creation support function 76 may be executed by the processing circuit 75, and the display control function 77 may be executed by an external cloud.

[0046] Next, we will describe in order the related image identification function 76a, the key image attachment function 76b, and the lesion site enhancement function 76c provided by the report creation support function 76.

[0047] The related image identification function 76a is a function that, when an order for reading a medical image taken of a patient's body part with a predetermined modality is selected from the order list and a predetermined region of that medical image is designated as a region of interest (ROI), identifies a medical image corresponding to the designated region of interest from among medical images taken of the same patient's body part with a modality other than the predetermined modality mentioned above, as a related image.

[0048] For example, when an order for reading a mammography image taken of a patient's body part by the mammography device 10 is selected from the order list, and a predetermined area of ​​the mammography image is designated as a region of interest, the related image identification function 76a identifies an ultrasound image corresponding to the designated region of interest from among the ultrasound images taken of the patient's body part by the ultrasound diagnostic device 30 as a related image.

[0049] Alternatively, the related image identification function 76a, when an order for reading an ultrasound image taken of a patient's body part by the ultrasound diagnostic device 30 is selected from the order list and a predetermined region of the ultrasound image is designated as a region of interest, identifies a mammography image corresponding to the designated region of interest from among the mammography images taken of the patient's body part by the mammography device 10 as a related image.

[0050] The region of interest may be manually selected and specified by the user, or it may be automatically specified based on body marks BM associated with the medical image. The following describes the case where the region of interest is automatically specified based on body marks BM.

[0051] For example, if an order for image interpretation is selected for a mammography image acquired by the mammography imaging device 10, the related image identification function 76a may automatically specify the location of a candidate lesion indicated by the body mark BM associated with the mammography image as a region of interest using the ESG processing function 68c described above. In this case, the related image identification function 76a identifies the ultrasound image obtained when the specified region of interest (i.e., the location of the candidate lesion) is scanned by the ultrasound diagnostic device 30 as the ultrasound image corresponding to that region of interest.

[0052] When an order for image interpretation related to an ultrasound image acquired by the ultrasound diagnostic device 30 is selected, the related image identification function 76a may automatically designate the position of the ultrasound probe (i.e., the scan area of ​​the ultrasound diagnostic device 30) indicated by the body mark BM associated with the ultrasound image as the region of interest. In this case, the related image identification function 76a identifies a mammography image in which the scan area of ​​the ultrasound diagnostic device 30 indicated by the designated region of interest is clearly represented (for example, the image set of MLO image 110R and CC image 100R shown in Figure 7, or the image set of MLO image 110R and CC image 100R shown on the right side of Figure 8) as the mammography image corresponding to the region of interest.

[0053] Furthermore, the medical image to be read, which corresponds to the image interpretation order and was captured using a predetermined modality, and the medical image identified as a related image, which was captured using a modality other than the predetermined modality mentioned above, are acquired, for example, from the image storage device 50 or the image observation device 60.

[0054] According to this, when users create an interpretation report for medical images taken with a given modality, they can use medical images taken with a different modality that are related to the medical image being interpreted, without having to search for them themselves, thus reducing the effort required from the user.

[0055] For example, with this function, when creating an interpretation report for a mammography image, it is possible to identify an ultrasound image taken when the location of a candidate lesion indicated by the body mark BM was scanned by the ultrasound diagnostic device 30, based on the body mark BM associated with the mammography image, and to use this ultrasound image in conjunction with the mammography image, thereby reducing the effort required to find the ultrasound image. Furthermore, with this function, when creating an interpretation report for an ultrasound image, it is possible to identify a mammography image in which a frame that clearly represents the scan area of ​​the ultrasound diagnostic device 30 indicated by the body mark BM is superimposed, based on the body mark BM associated with the ultrasound image, and to use this ultrasound image in conjunction with the ultrasound image, thereby reducing the effort required to find the mammography image.

[0056] The key image pasting function 76b automatically pastes the medical image related to the image interpretation order selected from the image interpretation order list (the medical image to be interpreted) and the medical image identified as a related image by the related image identification function 76a described above (i.e., a medical image taken with a modality other than the medical image to be interpreted) into a predetermined position in the image interpretation report. This eliminates the need for the user to paste so-called key images into the image interpretation report themselves, thereby reducing the effort required to paste such key images. In addition, the specified region of interest may be clearly indicated in the medical image to be interpreted.

[0057] Furthermore, the key image pasting function 76b may automatically paste, as key images, past medical images of the patient that are associated with the same body mark BM corresponding to the region of interest specified by the related image identification function 76a, and past medical images that were identified as related images by the related image identification function 76a when this medical image was the subject of reading (hereinafter, these past medical images will be collectively referred to as past images) to a predetermined position on the reading report. This allows the user to perform comparative reading, comparing past images with the current image, on the reading report as well. Past images may be identified together with related images by the related image identification function 76a and acquired from the image storage device 50 or image observation device 60, or they may be identified separately from related images by the key image pasting function 76b and acquired from the image storage device 50 or image observation device 60.

[0058] Furthermore, a function similar to the function for acquiring past images may be implemented to check whether there are any medical images (hereinafter referred to as "similar images") in the image observation device 60 that are images of patients other than the patient in question and that are associated with the same body mark BM corresponding to the region of interest specified by the related image identification function 76a. If it is confirmed that there are similar images, a function may be further implemented to present a list of similar images to the user. With this, the user can select at least one similar image from the list of similar images to display medical images of similar cases on the display 74, and can perform comparative interpretation of the medical image and related images to be interpreted with the similar images. In addition to the medical images of patients other than the patient in question mentioned above, the similar images may also include medical images that were identified as related images by the related image identification function 76a when the medical image in question was the subject of interpretation.

[0059] The lesion site enhancement function 76c primarily functions when creating interpretation reports for mammography images. It highlights the location of potential lesions (in other words, areas where lesions are suspected) indicated by the body mark BM associated with the mammography image on the schematic images for MLO and CC images. Any method is acceptable for highlighting areas where lesions are suspected, as long as it makes the suspected lesion area distinguishable from other areas. Specifically, examples include enclosing the suspected lesion area in a frame as shown in Figure 9(a), or using a different color for the suspected lesion area as shown in Figure 9(b). This helps prevent overlooking potential lesions.

[0060] Here, the lesion site enhancement function 76c is described as simply enhancing the location of candidate lesions (areas where lesions are suspected) indicated by body marks BM associated with the mammography image being read. However, it is not limited to this. For example, the mammography image being read may be analyzed to identify the shape and size of the tumor in the area where lesions are suspected, and then the area where lesions are suspected may be enhanced in a manner appropriate to the shape and size of the tumor. For example, if the shape of a tumor is identified as a result of analyzing the mammography image being read, the lesion site enhancement function 76c may enhance the schematic image by surrounding the area where lesions are suspected with a frame that follows the shape of the tumor, rather than surrounding the entire area where lesions are suspected with a frame, as shown in Figure 10(a). Furthermore, if the size of the tumor is identified as being large enough to reach the danger zone (in other words, larger than a preset threshold) as a result of analyzing the mammography image being read, the lesion site enhancement function 76c may enhance the schematic image by surrounding the area where lesions are suspected with a frame and representing it in yellow, as shown in Figure 10(b). Alternatively, if analysis of the mammography image to be interpreted reveals that the tumor size has increased compared to past images shown in Figure 11(a), the lesion area enhancement function 76c may enhance the schematic image by surrounding the suspected lesion area with a frame and displaying it in red, as shown in Figure 11(b). On the other hand, if analysis of the mammography image to be interpreted reveals that the tumor size has decreased compared to past images shown in Figure 11(a), the lesion area enhancement function 76c may enhance the schematic image by surrounding the suspected lesion area with a frame and displaying it in blue, as shown in Figure 11(c).

[0061] The display control function 77 is a function that displays the medical image to be read and the reading report based on the processing results of the report creation support function 76 (i.e., a reading report that includes at least the medical image to be read and related images attached by the key image attachment function 76b) on the display 74. The display control function 77 may also be implemented as a function of the report creation support function 76.

[0062] The report storage device 90 is a storage device that stores the image interpretation reports created via the report creation support device 70. The report storage device 90 may also be called a report server device. Details of the image interpretation reports stored in the report storage device 90 will be described later with reference to Figure 12, so a detailed explanation will be omitted here.

[0063] Next, with reference to Figure 12, we will explain the image interpretation report screen (image interpretation report) 200 that is displayed on one of the displays 74 by the display control function 77. Figure 12 illustrates the image interpretation report screen 200 that is displayed when creating an image interpretation report for mammography images.

[0064] On the image interpretation report screen 200 shown in Figure 12, the user creates an image interpretation report by pasting images and entering comments on findings. The created image interpretation report is saved in memory 72 and report storage device 90.

[0065] As shown in Figure 12, the image interpretation report screen 200 includes, for example, a first area 210 for displaying patient information, a second area 220 for displaying examination information, a third area 230 for displaying information related to image interpretation, a fourth area 240 for displaying information related to image interpretation in a format different from the information displayed in the third area 230, and a fifth area 250 for displaying information related to image interpretation in a format different from the information displayed in the third area 230 and the fourth area 240.

[0066] The first area 210 displays, for example, a patient ID for identifying the patient, the patient's name, date of birth, gender, and age at the time of the examination, the patient's medical history, and the patient's diagnosis (confirmed diagnosis). The medical history and confirmed diagnosis are entered manually by the user via, for example, the input interface 73, while information other than the medical history and diagnosis is entered automatically based on, for example, the supplementary information contained in the image file related to the image interpretation order (supplementary information contained in the DICOM image file).

[0067] Area 220 displays, for example, the content of the examination, the date and time the examination was performed, the subject of the examination (examination site), information on the drugs used during the examination, information indicating the urgency of the image interpretation order, the name of the physician who interpreted the image for the order, the disease that the examination was intended to treat (intended disease name), the department that requested the examination, the name of the physician who requested the examination, comments from the department that requested the examination, comments from the radiology department that performed the examination, and comments regarding the examination.

[0068] The third region 230 displays information about the lesion. Specifically, the third region 230 displays various images attached by the key image attachment function 76b. Figure 12 illustrates a case where past images are displayed in region 231 of the third region 230, and the medical image to be read (in this case, a mammography image) and related images (in this case, an ultrasound image corresponding to the region of interest specified based on the body mark BM associated with the mammography image) are displayed in region 232 of the third region 230. However, it is not limited to this, and the third region 230 only needs to display at least the medical image to be read and related images as described above. In addition to the various images attached by the key image attachment function 76b, the third region 230 may also display images that can be attached in response to user operations (e.g., similar images) and body marks BM.

[0069] The fourth region 240 displays, for example, schematic images relating to MLO images and CC images, in which areas suspected to contain lesions by the lesion area enhancement function 76c are enhanced to be distinguishable from other areas. The user can also specify a predetermined area of ​​the schematic image displayed in the fourth region 240 via the input interface 73, thereby designating that predetermined area as a region of interest for identifying related images. In Figure 12, the image interpretation report screen 200 displayed when creating an image interpretation report for mammography images is shown, and therefore schematic images relating to MLO images and CC images are displayed in the fourth region. However, in the image interpretation report screen displayed when creating an image interpretation report for ultrasound images, for example, a body mark BM is displayed in the fourth region.

[0070] Area 5 250 displays information related to the current image interpretation. Specifically, Area 5 250 displays information about the breast, information about the mammary gland, information indicating whether a comparative interpretation was performed during the current interpretation, information showing changes over time, and findings comments. The various information displayed in Area 5 250 can be manually entered by the user, for example, via the input interface 73. Alternatively, the various information displayed in Area 5 250 may be automatically entered by analyzing the mammography image and identifying the shape and size of the tumor in the area where the lesion is suspected, as described above. For example, standard phrases corresponding to the shape and size of the tumor in the area where the lesion is suspected may be automatically entered in the findings comments. The standard phrase information mentioned above may be stored, for example, in memory 72 or an external cloud.

[0071] Furthermore, in addition to the first to fifth regions 210 to 250 described above, the image interpretation report screen 200 is also provided with a comparison button 260. The comparison button 260 is used to retrieve an image interpretation report showing the interpretation results of past images from the report storage device 90 and display it on the display 74. This allows the user to interpret medical images and related images, and to create image interpretation reports, while referring to image interpretation reports of past images. Note that the image interpretation reports of past images may not be displayed in response to the pressing of the comparison button 260, but rather, for example, they may be automatically retrieved and displayed after checking for the existence of past images and confirming that past images exist.

[0072] Next, referring to the flowchart in Figure 13, a series of processing steps performed by the processing circuit 75 of the report creation support device 70 when a user creates an interpretation report on mammography images will be explained.

[0073] First, the processing circuit 75 displays a list of image interpretation orders on the display 74 in response to the user's (radiologist's) operation (step S1). The user can confirm the image interpretation orders that have been requested of them by referring to the list of image interpretation orders displayed on the display 74. The user starts creating an image interpretation report for a predetermined image interpretation order by selecting that predetermined image interpretation order from the list of image interpretation orders displayed on the display 74.

[0074] Next, the processing circuit 75 executes a process to generate a reading report screen 200 for the reading order selected according to the user's operation (step S2). Specifically, the processing circuit 75 acquires the mammography image to be read, as requested by the reading order selected according to the user's operation, from the image storage device 50 or the image observation device 60. The processing circuit 75 then designates the location of the candidate lesion indicated by the body mark BM associated with the mammography image to be read as a region of interest, and acquires the ultrasound image obtained when the region of interest is scanned by the ultrasound diagnostic device 30 as a related image from the image storage device 50 or the image observation device 60 (step S2-1).

[0075] Next, the processing circuit 75 pastes the mammography image and ultrasound image acquired by the processing in step S2-1 onto the third region 230 of the image interpretation report screen 200. More specifically, the processing circuit 75 pastes the mammography image to be interpreted and the ultrasound image identified as a related image by the processing in step S2-1 onto the third region 230 of the image interpretation report screen 200 (step S2-2). The mammography image to be interpreted may be pasted onto the third region 230 in a state that allows the specified region of interest to be identified.

[0076] Furthermore, the processing circuit 75 highlights, in a predetermined manner, the areas on the schematic images of the MLO image and CC image displayed in the fourth region 240 of the image reading report screen 200 where lesions are suspected to be present, indicated by the body mark BM associated with the mammography image being read (step S2-3).

[0077] According to the processing steps S2-1 to S2-3 described above, the processing circuit 75 can paste the mammography image to be read and the ultrasound image obtained when the location of the candidate lesion indicated by the body mark BM associated with the mammography image is scanned by the ultrasound diagnostic device 30 into the third region 230 as key images, and generate a reading report screen 200 in which the area where the lesion indicated by the body mark BM is presumed to be located is highlighted in a predetermined manner on the schematic image relating to the MLO image and CC image displayed in the fourth region 240.

[0078] Subsequently, the processing circuit 75 displays the image interpretation report screen 200 generated by the processing in step S2 (processing in steps S2-1 to S2-3) on the display 74 (step S3). The user creates an image interpretation report by entering findings comments on the image interpretation report screen 200 displayed on the display 74 as a result of the processing in step S3. The created image interpretation report is saved in the memory 72 and the report storage device 90.

[0079] According to the series of processing steps shown in Figure 13, it is possible to omit (1) the process of identifying and selecting ultrasound images related to the mammography image to be read from among ultrasound images taken with an ultrasound diagnostic device 30, which is a different modality from the mammography device 10 that took the mammography image to be read, (2) the process of pasting the mammography image to be read and the ultrasound images identified as related images as key images into the reading report, and (3) the process of identifying the area where a lesion is suspected to be present, etc., thereby significantly reducing the burden on the user (radiologist) when creating a reading report.

[0080] Figure 13 illustrates the series of processing steps performed by the processing circuit 75 when a user creates a mammography image interpretation report. However, the processing circuit 75 performs essentially the same processing when a user creates an ultrasound image interpretation report. In this case, however, as a process equivalent to step S2-3 described above, the processing circuit 75 performs a process to display a body mark BM (in other words, a body mark BM indicating the position where the ultrasound probe was applied during the scanning of the ultrasound image) in the fourth region 240, which indicates the scan area of ​​the ultrasound image to be interpreted, pasted in the third region 230 as a key image.

[0081] According to the embodiment described above, the report creation support device 70, upon receiving input that a reading order for a first medical image taken of a patient's body part with a first modality has been selected in response to user operation, includes a related image identification function 76a (identification unit) that designates a predetermined area of ​​the first medical image as an area of ​​interest and identifies a second medical image corresponding to the designated area of ​​interest from among second medical images taken of the patient's body part with a second modality different from the first modality, and a key image pasting function 76b (report creation unit) that pastes the first medical image and the identified second medical image into a predetermined area of ​​the reading report created for the reading order. With this, when creating a reading report for a first medical image, the user can use a second medical image related to the first medical image to be read without having to find it themselves, thereby reducing the user's workload. In addition, since the user does not need to paste the key image themselves, the effort required for pasting the key image is also reduced.

[0082] Furthermore, according to one embodiment, in addition to the functions described above, the report creation support device 70 further identifies a first past image, which is an image taken in the past using the first modality of the same body part of the same patient as the first medical image being read, and which has the same body mark associated with the first medical image being read, and a second past image, which was previously identified as a second medical image corresponding to the first past image, and pastes these past images together with the current image into a predetermined area of ​​the reading report. This allows the user to perform comparative reading of past images and current images on the reading report without increasing the effort of selecting and pasting past images. This function is particularly useful when creating reading reports for patients under observation.

[0083] Furthermore, according to one embodiment, in addition to the functions described above, the report creation support device 70 is further equipped with a lesion site enhancement function 76c (enhancement unit) that, when the first medical image to be read is a mammography image, enhances the area including the location of a candidate lesion indicated by a body mark associated with the mammography image on a schematic image showing the left and right breasts in a way that makes it distinguishable from other areas. This makes it possible to remind the user not to forget to read the area where a lesion is suspected to be present, thereby preventing the oversight of a lesion.

[0084] According to at least one embodiment described above, the burden on radiologists when preparing image interpretation reports can be reduced.

[0085] While several embodiments have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be implemented in a variety of other forms, and various omissions, substitutions, modifications, and combinations of embodiments are possible without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of symbols]

[0086] 10... Mammography imaging device, 30... Ultrasound diagnostic device, 50... Image storage device, 60... Image observation device, 70... Report creation support device, 71... Communication interface, 72... Memory, 73... Input interface, 74... Display, 75... Processing circuit, 76... Report creation support function, 76a... Related image identification function, 76b... Key image attachment function, 76c... Lesion site enhancement function, 77... Display control function, 90... Report storage device.

Claims

1. In response to user operation, upon receiving input indicating that an order for image interpretation of a first medical image taken of a patient's body part with a first modality has been selected, the identification unit designates a predetermined region of the first medical image as a region of interest and identifies a second medical image corresponding to the designated region of interest from among second medical images taken of the patient's body part with a second modality different from the first modality. A report creation unit that attaches the first medical image and the identified second medical image to a predetermined area of ​​the image interpretation report created for the aforementioned image interpretation order, Equipped with, The aforementioned identification unit is a report creation support device that identifies images associated with the same body mark as the body mark associated with the first medical image.

2. The aforementioned report creation unit, The report creation support device according to claim 1, wherein a first medical image in which the designated region of interest is identifiable and the identified second medical image are affixed to a predetermined area of ​​the image interpretation report.

3. The specified part is, A report creation support device according to claim 1 or 2, which specifies the region of interest based on body marks associated with the first medical image and identifies a second medical image corresponding to the specified region of interest.

4. If the first modality is a mammography device, the first medical image is a mammography image, the second modality is an ultrasound diagnostic device, and the second medical image is an ultrasound image, The specified part is, The report creation support device according to claim 3, which designates the location of a candidate lesion indicated by a body mark associated with the mammography image as the region of interest, and identifies an ultrasound image when the designated region of interest is scanned by the ultrasound diagnostic device.

5. If the first modality is an ultrasound diagnostic device, the first medical image is an ultrasound image, the second modality is a mammography device, and the second medical image is a mammography image, The specified part is, The report creation support device according to claim 3, which designates the position of the ultrasound probe indicated by a body mark associated with the ultrasound image as the region of interest, and identifies a mammography image that identifiablely shows the scan area when the designated region of interest is scanned by the ultrasound diagnostic device.

6. The specified part is, The report creation support device according to claim 3, further specifying a first past image, which is an image of the patient's body part taken in the past with the first modality and is associated with the same body mark as the body mark associated with the first medical image, and a second past image which was previously identified as a second medical image corresponding to the first past image.

7. The aforementioned report creation unit, The report creation support device according to claim 6, further comprising attaching the identified first past image and second past image to a predetermined area of ​​the image interpretation report.

8. The specified part is, The report creation support device according to claim 6 or 7, which identifies past image interpretation reports created for the identified first past image and second past image, and displays them on a display.

9. The specified part is, The report creation support device according to claim 3, which identifies whether there are similar images of a patient different from the aforementioned patient that are associated with the same body mark as the body mark associated with the first medical image, and if such similar images exist, displays a list of such similar images on a display.

10. The report creation support device according to claim 4, further comprising an enhancement unit that highlights a region including the location of a candidate lesion indicated by a body mark associated with the mammography image on a schematic image showing the left and right breasts in a manner that makes it distinguishable from other regions.

Images