Medical support device, medical image capturing system, medical support method, and program

Abstract

To provide a medical support device, a medical image capturing system, a medical support method, and a program that can make association of an inspection image with a schema image easy.SOLUTION: A medical support device includes a processor. The processor acquires an inspection image in which the position of a virtual cross section is displayed in each of an ultrasonic image and a radiation image, which is an image including a radiation image acquired by carrying out radiation imaging of the breast in the state of being compressed by a compression member and a plurality of ultrasonic images of the breast brought into a compressed state by the compression member, the compressed state being identical to the compressed state in the radiation image, and a schema image which is an image showing a first schema, a schematic diagram showing the breast in the radiation imaging, which includes a first schema image, in which the position of a virtual cross section is displayed on the basis of the ultrasonic image. The processor executes display control to display a display image including the inspection image and the schema image.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present disclosure relates to a medical support device, a medical image capturing system, a medical support method, and a program. [Background technology]

[0002] Patent Document 1 describes a medical information providing device that acquires first image information of a subject imaged by a first medical image diagnostic device, and acquires second image information of a subject imaged by a second medical image diagnostic device prior to the first medical image diagnostic device, and is characterized by including a display position calculation unit that performs calculations to enable alignment processing between the first image and the second image by sharing position information of the first image information and the second image information between the first image and the second image, and a display control unit that reflects the position information of the second image calculated by the display position calculation unit in the display of the first image and displays it. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2023-099261 A Summary of the Invention [Problem to be solved by the invention]

[0004] In an examination of a subject's breast, two types of examinations may be performed: a radiological examination in which radiation is applied to the breast to obtain a radiological image, and an ultrasonic examination in which an ultrasonic probe is used to obtain an echo image of the breast. In this case, the examination images may be interpreted by comparing them. Furthermore, when interpreting the images, a schema, which is a schematic diagram showing the breast, may be used, and the schema and the examination image may also be compared. For example, when the position of a lesion is identified as a result of the interpretation, the position of the lesion is written in the schema. In this way, when interpreting the images while comparing multiple examination images and the schema, it has been desired to easily associate the examination images with the schema images.

[0005] The technology disclosed herein provides a medical support device, a medical image capturing system, a medical support method, and a program that can easily associate an examination image with a schematic image. [Means for solving the problem]

[0006] A first aspect of the technology disclosed herein is a medical support device that includes a processor, which acquires images including a radiographic image obtained by radiographing a breast compressed by a compression member and a plurality of ultrasound images taken of the breast compressed by a compression member in a compression state that can be considered identical to the radiographic image, where the positions of virtual cross sections are displayed in each of the ultrasound image and the radiographic image, and a schematic image including the first schema image, which is an image showing a first schema that is a schematic diagram showing the breast in radiography and where the positions of the virtual cross sections are displayed based on the ultrasound image, and performs display control to display a display image including the examination image and the schematic image.

[0007] A second aspect of the technology disclosed herein is a medical support device according to the first aspect, wherein the schema image further includes a second schema image, the second schema image being an image showing a second schema which is a schematic diagram corresponding to an ultrasound examination image of an examination method in which a probe is placed on the breast, and in which the position of a virtual cross section is displayed.

[0008] A third aspect of the technology disclosed herein is a medical support device related to the second aspect, in which, when a region of interest is specified in at least one of an ultrasound image or a radiation image, the region of interest is displayed at a corresponding position in both the first schematic image and the second schematic image.

[0009] A fourth aspect of the technology of the present disclosure is a medical support device according to the second aspect, in which the position of the virtual cross section in the second schema image is derived based on the position of the virtual cross section in the first schema and the depth of the virtual cross section in the compression direction in the ultrasound image.

[0010] A fifth aspect of the technology disclosed herein is a medical support device relating to the second aspect, in which the schema image further includes a third schema image, the third schema image being an image showing a third schema, which is a schematic diagram of the breast in radiography taken in a different direction from that of the first schema, and in which the position of the virtual cross section is displayed.

[0011] A sixth aspect of the technology of the present disclosure is a medical support device according to the fifth aspect, wherein the position of the virtual cross section in the third schema image is derived based on the position of the virtual cross section in the second schema and the distance of the virtual cross section in the ultrasound image from the subject's body surface in the anterior-posterior direction.

[0012] A seventh aspect of the technology of the present disclosure is a medical support device according to the fifth aspect, in which the radiological image includes a first radiological image and a second radiological image obtained in a different shooting direction from the first radiological image, the first schematic image corresponds to the first radiological image, the third schematic image corresponds to the second radiological image, and in the second schematic image, the position of the virtual cross section is derived based on the positions of the virtual cross sections in the first schematic image and the third schematic image.

[0013] An eighth aspect of the technique of the present disclosure is the medical support device according to the seventh aspect, in which the position of the virtual cross section in the third schematic image is displayed based on an ultrasound image.

[0014] A ninth aspect of the technology of the present disclosure is a medical support device related to the first aspect, in which a first blind area, which is an image area in an ultrasound image where the breast is not visible in an ultrasound examination, is displayed so as to be distinguishable from other areas.

[0015] A tenth aspect of the technology disclosed herein is a medical support device related to the first aspect, in which a second blind area, which is an image area in which the breast is not seen in an ultrasound examination, is displayed in a schematic image so as to be distinguishable from other areas.

[0016] An eleventh aspect of the technology disclosed herein is a medical support device relating to the first aspect, in which the schematic image includes a fourth schematic image, the fourth schematic image being an image showing a fourth schema that is a schematic diagram showing a state in which a compression member is compressing a breast, and in which the position of a virtual cross section is displayed.

[0017] A twelfth aspect of the technique of the present disclosure is a medical support device according to the eleventh aspect, in which the position of the virtual cross section in the fourth schematic image is derived based on the position of the virtual cross section in the ultrasound image.

[0018] A thirteenth aspect of the technique of the present disclosure is the medical support device according to the first aspect, in which information regarding the depth of the intersection of the virtual cross sections in the compression direction of the breast is further displayed in the displayed image.

[0019] A fourteenth aspect of the technology disclosed herein is a medical support device relating to the first aspect, in which, when a virtual cross section displayed in a schematic image based on the position of the virtual cross section in an ultrasound image is defined as a first virtual cross section, the display manner of the position of the first virtual cross section is a virtual cross section estimated based on the position of the first virtual cross section displayed in the schematic image and is distinguishable from a second virtual cross section displayed in an image other than the image in which the first virtual cross section is displayed.

[0020] A fifteenth aspect of the technology disclosed herein is a medical support device according to the first aspect, in which, when the position of a virtual cross-section displayed in a schematic image is changed, the changed virtual cross-section is displayed in a manner that makes it distinguishable from the virtual cross-section before the change.

[0021] A sixteenth aspect of the technique of the present disclosure is a medical imaging system including the medical support device according to the first aspect, a radiation imaging device, and an ultrasound imaging device.

[0022] A seventeenth aspect of the technology of the present disclosure is a medical support method including obtaining a schematic image including a radiographic image obtained by radiographing a breast compressed by a compression member and a plurality of ultrasound images taken of the breast compressed by a compression member in a compression state that can be considered identical to the radiographic image, the schematic image including an examination image in which the position of a virtual cross section is displayed in each of the ultrasound image and the radiographic image, and a first schematic image, the first schematic image being an image showing a first schema that is a schematic diagram showing a breast in radiography, the first schematic image being an image in which the position of the virtual cross section is displayed based on the ultrasound image, and executing display control to display a display image including the examination image and the schematic image.

[0023] An 18th aspect of the technology disclosed herein is a program that causes a computer to execute processing including acquiring images including a radiographic image obtained by radiographing a breast compressed by a compression member and a plurality of ultrasound images taken of the breast compressed by a compression member in a compression state that can be considered identical to the radiographic image, where the position of a virtual cross section is displayed in each of the ultrasound image and the radiographic image, and a schematic image including the first schema image, which is an image showing a first schema that is a schematic diagram showing a breast in radiography, where the position of the virtual cross section is displayed based on the ultrasound image, and executing display control to display a display image including the examination image and the schema image. Effect of the Invention

[0024] According to the technology disclosed herein, a medical support device, a medical image capturing system, a medical support method, and a program are provided that can easily associate an examination image with a schematic image. [Brief description of the drawings]

[0025] [Figure 1] FIG. 1 is a conceptual diagram showing an example of the configuration of a medical support system. [Diagram 2] 1 is an external perspective view showing an example of the configuration of a mammography apparatus; [Diagram 3] 1 is an external side view showing an example of the configuration of a mammography apparatus. [Figure 4] FIG. 2 is a diagram illustrating an example of a hardware configuration of an electrical system of a client terminal. [Diagram 5] FIG. 2 is a conceptual diagram showing an example of main functions of a processor of a client terminal. [Figure 6] 1A to 1C are conceptual diagrams showing examples of ultrasound images of cross sections of a breast in a compressed state. [Figure 7] FIG. 2 is a conceptual diagram showing an example of main functions of a processor of a client terminal. [Figure 8] FIG. 2 is a conceptual diagram showing an example of main functions of a processor of a client terminal. [Figure 9] FIG. 2 is a conceptual diagram showing an example of main functions of a processor of a client terminal. [Figure 10] FIG. 2 is a conceptual diagram showing an example of main functions of a processor of a client terminal. [Figure 11] FIG. 2 is a conceptual diagram showing an example of main functions of a processor of a client terminal. [Figure 12] FIG. 2 is a conceptual diagram showing an example of main functions of a processor of a client terminal. [Figure 13] 13 is a flowchart showing an example of a flow of a display control process. [Figure 14] 11 is a flowchart showing an example of a flow of a display control process. [Figure 15] FIG. 2 is a conceptual diagram showing an example of main functions of a processor of a client terminal. [Figure 16]FIG. 2 is a conceptual diagram showing an example of main functions of a processor of a client terminal. [Figure 17] FIG. 2 is a conceptual diagram showing an example of main functions of a processor of a client terminal. [Figure 18] FIG. 2 is a conceptual diagram showing an example of main functions of a processor of a client terminal. [Figure 19] FIG. 2 is a conceptual diagram showing an example of main functions of a processor of a client terminal. [Figure 20] FIG. 2 is a conceptual diagram showing an example of main functions of a processor of a client terminal. [Figure 21] FIG. 1 is a conceptual diagram showing an example of the configuration of a medical support system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0027] In the following description, for convenience of explanation, the height direction, width direction, and front-rear direction (also called depth direction) of the breast imaging apparatus 10 are indicated by three arrows, X, Y, and Z. First, the height direction is indicated by arrow Z, and the direction indicated by arrow Z is the upward direction of the breast imaging apparatus 10, and the opposite direction is the downward direction. The height direction is the vertical direction. The width direction is indicated by arrow X perpendicular to arrow Z, and the direction indicated by arrow X is the rightward direction of the breast imaging apparatus 10, and the opposite direction is the leftward direction. The front-rear direction is indicated by arrow Y perpendicular to arrows Z and X, and the direction indicated by arrow Y is the forward direction of the breast imaging apparatus 10, and the opposite direction is the rearward direction. That is, in the breast imaging apparatus 10, the side of the stand 20 is the rearward direction, and the opposite side where the subject A stands (see FIG. 3) is the forward direction. In the following description, expressions using sides, such as upper, lower, left, right, front, and rear, have the same meaning as expressions using directions.

[0028] In this embodiment, the "vertical direction" refers to a vertical direction including an error that is generally acceptable in the technical field to which the technology of the present disclosure belongs and does not go against the spirit of the technology of the present disclosure, in addition to a completely vertical direction. Similarly, the "horizontal direction" refers to a horizontal direction including an error that is generally acceptable in the technical field to which the technology of the present disclosure belongs and does not go against the spirit of the technology of the present disclosure, in addition to a completely horizontal direction.

[0029] As shown in FIG. 1 as an example, a medical support system 1 according to this embodiment is a system capable of providing a service including the presentation of a medical image (e.g., an examination image) to a user 18 in response to a request from the user 18. The medical support system 1 is an example of a "medical image capturing system" according to the disclosed technology. The medical support system 1 includes a breast imaging device 10, an ultrasound imaging device 40, an image database 14, and a client terminal 16. The breast imaging device 10 is a device capable of acquiring a radiological image by irradiating radiation onto the breast of a subject. The ultrasound imaging device 40 is a device capable of acquiring an ultrasound image by performing an ultrasound examination on the breast of a subject. The breast imaging device 10 and the ultrasound imaging device 40 will be described in detail later.

[0030] The image database 14 is a database capable of storing image data such as medical images. The image database 14 is connected to the mammography device 10 and the ultrasound imaging device 40 via a network 12. The network 12 is configured, for example, as at least one of a wide area network (WAN) and a local area network (LAN). Furthermore, the connection method between the mammography device 10 and the ultrasound imaging device 40 and the network 12, and the connection method between the image database 14 and the network 12 may be a wireless communication method or a wired communication method. The network 12 establishes communication between the mammography device 10 and the ultrasound imaging device 40 and the image database 14, and transmits and receives various information between the mammography device 10 and the ultrasound imaging device 40 and the image database 14. The image database 14 stores image data transmitted from the mammography device 10 and the ultrasound imaging device 40.

[0031] The client terminal 16 is a terminal operated by the user 18. The client terminal 16 is connected to the image database 14 via the network 12. The reception device 19 is connected to the client terminal 16. The reception device 19 receives instructions from the user 18. The reception device 19 has a keyboard 19A, a mouse 19B, and the like. The keyboard 19A and the mouse 19B shown in FIG. 1 are merely examples. The reception device 19 may be provided with only one of the keyboard 19A and the mouse 19B. In addition, the reception device 19 may be, for example, at least one of a proximity input device that receives a proximity input, a voice input device that receives a voice input, and a gesture input device that receives a gesture input, instead of the keyboard 19A and / or the mouse 19B. The proximity input device is, for example, a touch panel or a tablet. The client terminal 16 is an example of a "medical support device" according to the technology of the present disclosure.

[0032] A display device 17 is connected to the client terminal 16. Examples of the display device 17 include an EL (Electro-Luminescence) display and a liquid crystal display. The display device 17 displays various information (e.g., images and characters) under the control of the client terminal 16.

[0033] Here, a personal computer is shown as an example of the client terminal 16, but this is merely one example. A mobile terminal such as a smartphone or a tablet terminal may also be used as the client terminal 16. Here, an example is shown in which one client terminal 16 is connected to one image database 14 via the network 12, but this is merely one example. Needless to say, the medical support system 1 may include a plurality of client terminals 16 and a plurality of image databases 14.

[0034] As shown in Figures 2 and 3, the breast imaging device 10 according to this embodiment is a radiological imaging device that irradiates radiation onto a breast M of a subject A, who is the subject, to obtain a radiological image of the breast M. One example of the radiation is X-rays, but gamma rays may also be used. The subject A is positioned in front of the breast imaging device 10. The breast imaging device 10 is an example of a "radiological image imaging device" according to the technology of the present disclosure.

[0035] The mammography apparatus 10 is connected to a console (not shown). The console has a setting function for setting the mammography apparatus 10 in accordance with an imaging order, as well as a function for acquiring a radiation image 62 (see FIG. 5) captured by the mammography apparatus 10 and displaying the acquired radiation image.

[0036] The mammography apparatus 10 has a stand 20 and an arm 21. The stand 20 is composed of a base 20A placed on the floor of a radiography room and a support column 20B extending in the height direction from the base 20A. The arm 21 has a substantially C-shaped shape when viewed from the left side, and is connected to the support column 20B via a rotation shaft. The arm 21 is movable in the height direction relative to the support column 20B, so that the height of the arm 21 can be adjusted according to the height of the subject A. The arm 21 is also rotatable around a rotation shaft perpendicular to the support column 20B.

[0037] The arm 21 is composed of a radiation source housing section 22, a main body section 23, and an imaging table 24. A radiation source 25 is housed in the radiation source housing section 22. The radiation source housing section 22 has a housing structure having a longitudinal direction in the front-rear direction, for example. The main body section 23 integrally connects the radiation source housing section 22 and the imaging table 24. The main body section 23 holds the radiation source housing section 22 and the imaging table 24 at predetermined positions. On both sides of the main body section 23, handrails 27 are provided for the subject A to hold. The breast M of the subject A is placed on the imaging table 24. The imaging table 24 has an imaging surface 24A on which radiation that has passed through the breast M is incident. The imaging table 24 houses a radiation detector 26.

[0038] The radiation source 25 irradiates radiation towards the breast M placed on the imaging table 24. The radiation emitted from the radiation source 25 passes through the compression member 30 and then enters the breast M. The radiation detector 26 detects the radiation that has passed through the breast M and outputs a radiographic image. The radiation detector 26 is called an FPD (Flat Panel Detector). The radiation detector 26 has a scintillator that converts radiation into visible light, and may be an indirect conversion type that converts the visible light emitted by the scintillator into an electrical signal, or a direct conversion type that directly converts radiation into an electrical signal.

[0039] An irradiation field limiter 31 is provided between the radiation source 25 and the imaging table 24. The irradiation field limiter 31 is also called a collimator, and defines the irradiation field of radiation onto the imaging table 24.

[0040] A face guard 32 is attached to the radiation source housing 22. The face guard 32 is formed of or coated with a material that is not transparent to radiation, and protects the face of the subject A from radiation.

[0041] Between the imaging table 24 and the irradiation field limiter 31, a compression member 30 is provided for clamping and compressing the breast M between the imaging table 24. The compression member 30 is made of a material that transmits radiation. The compression member 30 is disposed at a position facing the imaging table 24. In this embodiment, the compression member 30 has a box shape with an open top. The compression member 30 is an example of a "compression member" according to the technology of the present disclosure.

[0042] The moving mechanism 35 supports the compression member 30 so as to be movable between the radiation source 25 and the imaging table 24. In addition, the movable part 34 is disposed between the compression member 30 and the moving mechanism 35. The movable part 34 is slidably held on a rail 28 provided on the moving mechanism 35. The rail 28 extends in the vertical direction.

[0043] The moving mechanism 35 includes, for example, a motor (not shown), a motor driver (not shown), and a feed screw mechanism (not shown). The motor rotates in response to an electrical drive signal output by the motor driver, and moves the compression member 30 via the feed screw mechanism.

[0044] The compression member 30 is attached to a movable part 34. The movable part 34 moves in the up-down direction together with the compression member 30 by a moving mechanism 35. The up-down direction functionally corresponds to the direction in which the compression member 30 moves toward the imaging table 24 (i.e., downward) and the direction in which the compression member 30 moves away from the imaging table 24 (i.e., upward). In this manner, the compression member 30 is configured to be movable in a manner that changes the distance from the imaging table 24.

[0045] An anti-scatter grid 27 is provided inside the imaging table 24. The anti-scatter grid 27 removes scattered radiation generated when radiation passes through the breast M. The anti-scatter grid 27 is disposed inside the imaging table 24, closer to the radiation source 25 than the radiation detector 26.

[0046] As described above, radiation is irradiated from the radiation source 25 while the breast M of the subject A is compressed by the compression member 30. The radiation that has passed through the breast M is detected by the radiation detector 26. As a result, a radiation image of the breast M is obtained.

[0047] In this embodiment, in addition to radiography by the mammography device 10, an ultrasound examination is performed by the ultrasound imaging device 40. The ultrasound imaging device 40 is an example of an “ultrasonic image imaging device” according to the technique of the present disclosure. The ultrasound imaging device 40 includes an ultrasound probe 42.

[0048] The ultrasonic probe 42 is moved by the user along the upper surface (the surface opposite to the side in contact with the breast M of the subject A) of the bottom plate 30A of the compression member 30, and acquires an ultrasonic image of the breast M by scanning the breast M with ultrasonic waves. Specifically, when ultrasonic imaging is performed, the ultrasonic probe 42 is moved by a medical professional along the upper surface of the bottom plate 30A of the compression member 30 with an acoustic matching member (not shown) such as echo jelly applied to the upper surface of the compression member 30 or with an acoustic matching member attached to the ultrasonic probe 42.

[0049] The ultrasonic probe 42 includes a plurality of ultrasonic transducers (not shown) arranged one-dimensionally or two-dimensionally. Each ultrasonic transducer transmits ultrasonic waves based on an applied drive signal, and outputs a reception signal by receiving ultrasonic echoes.

[0050] Each of the ultrasonic transducers is composed of a vibrator having electrodes formed on both ends of a piezoelectric material (piezoelectric body), such as a piezoelectric ceramic represented by PZT (lead zirconate titanate) or a polymer piezoelectric element represented by PVDF (polyvinylidene difluoride). When a pulsed or continuous wave drive signal is sent to the electrodes of the vibrator to apply a voltage, the piezoelectric body expands and contracts. This expansion and contraction generates pulsed or continuous wave ultrasonic waves from each vibrator, and an ultrasonic beam is formed by combining these ultrasonic waves. In addition, each vibrator expands and contracts by receiving the propagating ultrasonic waves, and generates an electric signal. These electric signals are output as ultrasonic reception signals and input to the main body (not shown) of the ultrasonic imaging device 40 via the cable 42B.

[0051] The scanning direction of the ultrasonic probe 42 may be the left-right direction of the mammography device 10 (X direction shown in FIG. 2). In this case, the ultrasonic probe 42 is disposed so that the longitudinal direction of the contact surface 42A (i.e., the transducer arrangement direction) is along the front-rear direction of the mammography device 10 (Y direction shown in FIG. 2). That is, the ultrasonic probe 42 having a one-dimensional transducer scans in the elevation direction. Then, the left-right scanning of the ultrasonic probe 42 is performed multiple times while being shifted in the front-rear direction. The multiple echo images of the breast M obtained in this manner are subjected to image reconstruction processing and are used to generate ultrasonic cross-sectional images of the breast M.

[0052] Specifically, a three-dimensional image is obtained by combining multiple echo images of the breast M. A three-dimensional image is volume data in which multiple two-dimensional slice images are stacked, and is composed of multiple voxels, which are three-dimensional pixel units. From the three-dimensional image, it is possible to extract a two-dimensional image in any cross section. For example, from the three-dimensional image, it is possible to extract a two-dimensional slice image of a transverse section (i.e., an axial section), a two-dimensional slice image of a coronal section (i.e., a coronal section), and a two-dimensional slice image of a sagittal section (i.e., a sagittal section). In this manner, an ultrasound image, which is an image showing any cross section in the three-dimensional image, is obtained.

[0053] The above-mentioned ultrasonic examination is performed in a compression state that can be considered to be the same as the compression state of the breast M in radiography. Here, the compression state that can be considered to be the same refers to compression with the same compression pressure and the same compression thickness, and more preferably includes a case where a radiographic image is acquired in a compressed state, and further, an echo image is continuously acquired without releasing the compression state. In other words, the compression state that can be considered to be the same also includes a form in which the compression is released after radiography is performed, and then the breast M is compressed under the same compression conditions and an ultrasonic examination is performed. Here, the compression conditions include the compression pressure, compression height, the type of compression member 30, the thickness of the breast M during compression, and the contact area between the compression member 30 and the breast M.

[0054] 4, the client terminal 16 includes a computer 50, a communication I / F (Interface) 58, and a bus 57. The computer 50 includes a processor 52, a storage 54, and a RAM (Random Access Memory) 56. The processor 52, the storage 54, the RAM 56, and the communication I / F 58 are connected to the bus 57. The computer 50 is an example of a "computer" according to the technology of the present disclosure, and the processor 52 is an example of a "processor" according to the technology of the present disclosure.

[0055] A memory is connected to the processor 52. The memory includes a storage 54 and a RAM 56. The processor 52 has, for example, a central processing unit (CPU) and a graphics processing unit (GPU). The GPU operates under the control of the CPU and is responsible for executing image-related processing.

[0056] The storage 54 is a non-volatile storage device that stores various programs and various parameters. Examples of the storage 54 include a flash memory (e.g., an EEPROM (Electrically Erasable and Programmable Read Only Memory) and an SSD (Solid State Drive)) and / or an HDD (Hard Disk Drive). Note that the flash memory and the HDD are merely examples, and at least one of a flash memory, an HDD magnetoresistive memory, and a ferroelectric memory may be used as the storage 54.

[0057] The RAM 56 is a memory in which information is temporarily stored, and is used as a work memory by the processor 52. Examples of the RAM 56 include a dynamic random access memory (DRAM) and a static random access memory (SRAM).

[0058] The communication I / F 58 is connected to the network 12. The communication I / F 58 is responsible for sending and receiving information to and from an external communication device (e.g., the image database 14) via the network 12. For example, the communication I / F 58 transmits information in response to a request from the processor 52 to the external communication device via the network 12. The communication I / F 58 also receives information transmitted from the external communication device, and outputs the received information to the processor 52 via the bus 57.

[0059] In the present embodiment, radiography and ultrasound examination of the breast M are performed under conditions of compression that can be considered to be the same, and as a result, a radiographic image and an ultrasound image are obtained. In this case, the user 18 may need to compare and interpret the radiographic image and the ultrasound image acquired under conditions of compression that can be considered to be the same. When interpreting the image, a schema, which is a schematic diagram showing the breast, may be used, and the schema may also be compared with a diagnostic image obtained by the examination. In this way, different types of examination images, or an examination image and a schema, cannot be simply compared because the way of viewing the image or the display contents are different. For this reason, it has been difficult to associate the examination image with the schema image (for example, to associate the position of the region of interest (for example, a lesion)).

[0060] Therefore, a display control program 54A is stored in the storage 54. The display control program 54A is a program that provides display control processing for radiological images, ultrasound images, and the like. The processor 52 reads the display control program 54A from the storage 54, and executes the read display control program 54A on the RAM 56 to perform the display control processing. The display control processing is realized by the processor 52 operating as an acquisition unit 52A, an image generation unit 52B, and an output unit 52C. The display control program 54A is an example of a "program" according to the technology of the present disclosure.

[0061] As an example, as shown in FIG. 5, in the processor 52, first, the acquisition unit 52A acquires an examination image 60 from the image database 14 via the communication I / F 58 (see FIG. 4). The examination image 60 is an image showing the examination result for the subject A. The examination image 60 includes a radiographic image 62 and an ultrasound image 64. The examination image 60 is an example of an "examination image" according to the technology of the present disclosure. The radiographic image 62 is an image obtained as a result of radiography using the breast imaging device 10 (see FIG. 2). Specifically, the radiographic image 62 is an image obtained by radiography of the breast M in a state compressed by the compression member 30 (see FIG. 2). The radiographic image 62 is an example of a "radiographic image" according to the technology of the present disclosure.

[0062] In this example, MLO (medio-lateral) imaging is performed as the imaging technique, in which the breast M is compressed from a direction inclined relative to the cranio-caudal direction of the subject and imaged, and the resulting radiographic image 62 is shown. In the following description of this embodiment, an example will be described in which MLO imaging is performed as the radiographic imaging.

[0063] Moreover, the radiation image 62 is obtained by performing imaging in a state where the radiation source 25 is held at a predetermined position (for example, a position facing the imaging table 24) with respect to the imaging stand 24. In the radiation image 62 obtained by imaging in such a state, the contour of the breast M may be less visible than in the case of so-called tomosynthesis imaging. For this reason, it may be more difficult to associate the ultrasound image 64 and / or the schematic image 66 with the radiation image 62 as described above.

[0064] The ultrasound image 64 is an image captured by compressing the breast M with the compression member 30 (see FIG. 2) under a compression state that can be considered the same as the radiation image 62. Specifically, the ultrasound image 64 is an image obtained by generating a two-dimensional image at an arbitrary cross section from a three-dimensional image obtained as a result of an ultrasound examination performed under a compression state that can be considered the same as radiation imaging. If there is no particular designation from the user 18, the cross-sectional image generated from the three-dimensional image by default is a two-dimensional image showing a cross section passing through the center of the three-dimensional image. The ultrasound image 64 is an example of an "ultrasound image" according to the technology of the present disclosure.

[0065] Here, the ultrasonic image 64 includes a plurality of ultrasonic cross-sectional images 64A to 64C. The plurality of ultrasonic cross-sectional images 64A to 64C are cross-sectional images obtained by slicing a three-dimensional image at cross sections perpendicular to each other. In the example shown in FIG. 6, the ultrasonic cross-sectional image 64A is an image showing a sagittal plane SP, which is a cross section of the breast M at an arbitrary distance from the bottom plate 30A of the compression member 30, which is a cross section perpendicular to the compression direction of the breast M (see arrow P in FIG. 6). Moreover, for example, the ultrasonic cross-sectional image 64B is an image showing a transverse plane TP, which is a cross section of the breast M parallel to the compression direction of the breast M, along the front-back direction of the subject A (direction along the B direction shown in FIG. 6). Moreover, for example, the ultrasonic cross-sectional image 64C is an image showing a coronal plane CP, which is a cross section of the breast M parallel to the compression direction of the breast M, along the left-right direction of the subject A (direction along the A direction shown in FIG. 6).

[0066] For ease of explanation, the ultrasonic cross-sectional image 64A will be referred to as a sagittal plane ultrasonic image 64A, the ultrasonic cross-sectional image 64B as a transverse plane ultrasonic image 64B, and the ultrasonic cross-sectional image 64C as a coronal plane ultrasonic image 64C.

[0067] 5, the acquisition unit 52A outputs the acquired examination image 60 to the image generation unit 52B. Here, the radiation image 62 and the ultrasound image 64 have different ranges of the breast M displayed in the images due to differences in the imaging principles. Specifically, at the tip side (i.e., the nipple side) of the breast M, a gap is likely to occur between the compression member 30 and the breast M, making it difficult to obtain reflected waves, and therefore the tip side may not be easily captured in the ultrasound image 64. In other words, there may be a loss of the breast M in the ultrasound image 64.

[0068] In addition, even on the chest wall side of the breast M, there is a limit to the scannable range due to the offset between the transducer in the ultrasonic probe 42 and the outer shape of the ultrasonic probe 42 or the plate thickness of the compression member 30. That is, in the ultrasonic probe 42, the transducer is disposed offset inward from the outer edge of the housing of the ultrasonic probe 42. Therefore, when the ultrasonic probe 42 scans inside the box-shaped compression member 30, the housing of the ultrasonic probe 42 comes into contact with the inner wall of the compression member 30, and there is a range where the ultrasonic beam from the transducer is not irradiated. Therefore, even on the chest wall side of the breast M, it may be difficult to capture it in the ultrasonic image 64.

[0069] Therefore, the image generating unit 52B detects the contour of an image region indicating the tip side of the breast M in the radiation image 62 acquired from the acquiring unit 52A. For example, the luminance value of each pixel of the radiation image 62 can be used to detect the contour in the image generating unit 52B. That is, in the radiation image 62, a boundary portion where the difference in luminance value is equal to or greater than a threshold value is detected as the contour. Note that this is merely an example, and for example, the contour may be detected by performing image recognition processing (for example, image recognition processing of a pattern matching type or image recognition processing of an AI (Artificial Intelligence) type) on the radiation image 62. Then, the image generating unit 52B reflects the contour detected in the radiation image 62 in the ultrasound image 64.

[0070] The ultrasound image 64 displays the contour line detected in the radiation image 62. Specifically, the contour line is reflected in the sagittal plane ultrasound image 64A. Here, the radiation image 62 is an image obtained by MLO imaging. Therefore, the sagittal plane ultrasound image 64A is a cross-sectional image seen from the imaging direction of the radiation image 62. Therefore, when the contour line of the radiation image 62 is reflected in the sagittal plane ultrasound image 64A, the size of the contour line in the radiation image 62 is adjusted to match the image size of the sagittal plane ultrasound image 64A, and the contour line 63A is displayed superimposed on the sagittal plane ultrasound image 64A.

[0071] As a method for deriving the contour line on the chest wall side in the ultrasound image 64, the contour line is displayed based on the limit of the scannable range caused by the offset amount between the transducer in the ultrasound probe 42 and the outer shape of the ultrasound probe 42. Since the offset amount of the transducer is predetermined in the ultrasound probe 42, the contour line is displayed on the chest wall side in the ultrasound image 64 based on the offset amount. For example, in the sagittal plane ultrasound image 64A, a linear contour line 63B is displayed superimposed at a position away from the display area on the chest wall side by an amount corresponding to the offset amount.

[0072] Then, in the sagittal plane ultrasound image 64A, the portion between the contour lines 63A and 63B and the image region of the breast M in the sagittal plane ultrasound image 64A is derived as the blind area 65. In addition, the portion between the contour lines 63A and 63B, the portion between the upper end of the image and the image region of the breast M, and the portion between the lower end of the image and the image region of the breast M are also derived as the blind area 65. Here, the blind area 65 is an image region in which the breast M is not reflected in the ultrasound examination. The image generating unit 52B displays the blind area 65 in a manner that is distinguishable from other regions of the sagittal plane ultrasound image 64A (see the area hatched with dots shown in FIG. 5). Note that here, the blind area 65 is displayed in a manner that is distinguishable from other image regions by applying a pattern to the blind area 65, but this is merely one example. For example, the blind area 65 may be shaded, colored, or flashing. Also, text indicating the blind area 65 may be displayed. The blind area 65 is an example of a "first blind area" according to the technology of the present disclosure.

[0073] 7 as an example, the acquisition unit 52A acquires a schematic image 66 from the storage 54. The schematic image 66 is an image showing a schema which is a schematic diagram of a breast M. The schematic image 66 may be a commonly used template, or may be an image created by the user 18. The schematic image 66 does not have to be stored in the storage 54, and may be acquired from an external device via the network 12.

[0074] The acquisition unit 52A outputs the acquired schema image 66 to the image generation unit 52B. The schema image 66 includes an MLO schema image 66A. The MLO schema image 66A is an image showing an MLO schema, which is a schematic diagram of the breast M in MLO imaging. In this embodiment, the MLO schema image 66A is an example of a "first schema image" according to the technology of the present disclosure. The image generation unit 52B displays a blind area superimposed on the MLO schema image 66A. Specifically, the blind area 65 derived in the sagittal plane ultrasound image 64A is adjusted to fit the size of the MLO schema image 66A and is displayed superimposed on the MLO schema image 66A. In the example shown in FIG. 7, in the MLO schema image 66A, a blind area 67 corresponding to the blind area 65 is displayed so as to be distinguishable from other areas (see the area hatched with dots shown in FIG. 7). The blind area 67 is an example of a "second blind area" according to the technology of the present disclosure.

[0075] 7, the schema image 66 also includes schema images other than the MLO schema image 66A. For example, the schema image 66 includes a CC schema image 66B. The CC schema image 66B is an image showing a CC schema, which is a schematic diagram of the breast M in CC photography. CC photography (Cranio-Caudal) is an imaging technique in which the breast M is compressed from the cranio-caudal direction of the subject A and photographed. In this embodiment, the CC schema image 66B shows a schema in a direction in which photography is not performed. In this embodiment, the CC schema image 66B is an example of a "third schema image" according to the technology of the present disclosure.

[0076] The schematic image 66 also includes a compression schematic image 66C. The compression schematic image 66C is an image showing a compression schematic diagram showing a state in which the compression member 30 is compressing the breast M. The compression schematic image 66C is an example of a "fourth schematic image" according to the technology of the present disclosure.

[0077] Furthermore, the schematic image 66 includes an ultrasound schematic image 66D. The ultrasound schematic image 66D is an image showing a schema which is a schematic diagram corresponding to an ultrasound examination image of an examination method in which an ultrasound probe 42 is placed against the breast M. The ultrasound schematic image 66D is an example of a "second schematic image" according to the technology of the present disclosure.

[0078] The examination method in which an ultrasonic probe 42 is placed against the breast M specifically refers to an examination method in which echo jelly is applied to the surface of the breast M and the ultrasonic probe 42 is placed directly against the breast M to scan it, and is a different examination method from the ultrasonic examination performed while compressed by the compression member 30 as described above.

[0079] As an example, as shown in FIG. 8, the image generating unit 52B generates a display image 68 including an inspection image 60 and a schematic image 66. The image generating unit 52B outputs the generated display image 68 to the output unit 52C. The output unit 52C outputs information indicating the display image 68 to the display device 17. Specifically, the output unit 52C executes GUI (Graphical User Interface) control for displaying the display image 68, thereby causing the display device 17 to display the display image 68. The GUI control is an example of "display control" according to the technology of the present disclosure. The display image 68 is an example of "display image" according to the technology of the present disclosure.

[0080] A display image 68 is displayed on the screen 17A of the display device 17. In the example shown in Fig. 8, an examination image 60 is displayed in the left image region of the display image 68. The examination image 60 includes a radiological image 62 and an ultrasound image 64. The ultrasound image 64 includes a sagittal plane ultrasound image 64A, a transverse plane ultrasound image 64B, and a coronal plane ultrasound image 64C. A blind area 65 is displayed in the sagittal plane ultrasound image 64A.

[0081] 8, a schematic image 66 is displayed in the image region on the right side of the display image 68. The schematic image 66 includes an MLO schematic image 66A, a CC schematic image 66B, a compression schematic image 66C, and an ultrasound schematic image 66D. A blind area 67 is displayed in the MLO schematic image 66A.

[0082] 8 is merely an example, and the layout of each image can be changed as appropriate. In addition, the display image 68 can of course display information other than the examination image 60 and the schematic image 66 (e.g., information that can identify the subject A, or information that can identify the examination content (e.g., examination date and time, examination person in charge, and / or compression conditions, etc.)). In addition, the display image 68 does not have to be displayed in one window, and can be displayed in multiple windows. In this case, each of the multiple windows can be displayed on a separate display device 17.

[0083] 8, the examination image 60 for one breast M is displayed, but this is merely an example. An embodiment may be such that the examination images 60 for each of the left and right breasts M are displayed. In this case, the schematic image 66 may be displayed for each of the examination images 60 for the left and right breasts M.

[0084] Here, the user 18 (see FIG. 1) operates the mouse 19B to specify an arbitrary portion on the ultrasound image 64 via the pointer 70. The arbitrary portion is, for example, a region in which the user 18 is interested (hereinafter also simply referred to as a "region of interest"). The following describes an example in which the region of interest is specified by the user 18. The region of interest is, for example, a region in which a lesion (e.g., a tumor) exists. In the example shown in FIG. 8, the region of interest is specified via the pointer 70 in the sagittal plane ultrasound image 64A. When the region of interest is specified via the pointer 70, a mark indicating a virtual cross section is displayed in the ultrasound image 64 at a position corresponding to the region of interest.

[0085] 8, when the pointer 70 is placed over the sagittal plane ultrasound image 64A, a cross-shaped mark 72 (hereinafter also simply referred to as "mark 72") corresponding to the position indicated by the pointer 70 is displayed. In the cross-shaped mark 72, a line segment along the left-right direction of the sagittal plane ultrasound image 64A corresponds to the virtual cross section indicated by the transverse cross-sectional ultrasound image 64B. In addition, in the cross-shaped mark 72, a line segment along the up-down direction of the sagittal plane ultrasound image 64A corresponds to the virtual cross section indicated by the coronal plane ultrasound image 64C.

[0086] Furthermore, when a region of interest is designated and a cross-shaped mark 72 is displayed in the sagittal plane ultrasound image 64A, a cross-shaped mark 74 (hereinafter also simply referred to as "mark 74") is also displayed in the transverse plane ultrasound image 64B. Similarly, a cross-shaped mark 76 (hereinafter also simply referred to as "mark 76") is also displayed in the coronal plane ultrasound image 64C. The display positions of the marks 72, 74, and 76 correspond to each other. For example, when the mark 72 is moved in the sagittal plane ultrasound image 64A, the transverse plane ultrasound image 64B and the coronal plane ultrasound image 64C are updated according to the virtual cross section after the movement. Furthermore, the positions of the marks 74 and 76 displayed in the transverse plane ultrasound image 64B and the coronal plane ultrasound image 64C, respectively, are also changed.

[0087] Although the embodiment in which the region of interest is specified in the sagittal plane ultrasonic image 64A has been described here, this is merely an example. It goes without saying that the region of interest may be specified in the transverse plane ultrasonic image 64B and the coronal plane ultrasonic image 64C.

[0088] In the following description, when there is no need to distinguish between the virtual cross sections shown in the sagittal plane ultrasound image 64A, the virtual cross sections shown in the transverse plane ultrasound image 64B, and the virtual cross sections shown in the coronal plane ultrasound image 64C, they will simply be referred to as "virtual cross sections."

[0089] 9 as an example, the image generating unit 52B displays a virtual cross section on the radiation image 62 based on the ultrasound image 64. First, the acquiring unit 52A acquires cross section position information 80 accepted via the accepting device 19. The cross section position information 80 is information capable of identifying the position of the virtual cross section in the ultrasound image 64. The acquiring unit 52A outputs the cross section position information 80 to the image generating unit 52B. The image generating unit 52B displays the cross section position on the radiation image 62 based on the cross section position information 80.

[0090] Specifically, the image generator 52B displays each virtual cross section on the radiographic image 62 according to the distance of each virtual cross section from the image end of the ultrasound image 64 indicated by the cross-section position information 80. Here, in the sagittal plane ultrasound image 64A, the distance is d2 from the top end of the transverse plane image, and the distance is d3 from the left end of the coronal plane image. As described above, the sagittal plane ultrasound image 64A is a cross-sectional image viewed from the imaging direction of the radiographic image 62. Therefore, the image generator 52B adjusts these distances d2 and d3 according to the size of the radiographic image 62, and displays each cross section superimposed on the radiographic image 62. In the example shown in FIG. 9, a cross-shaped mark 81 (hereinafter also simply referred to as "mark 81") indicating the position of the virtual cross section is displayed in the radiographic image 62.

[0091] 10, the image generator 52B displays the virtual cross section on the schematic image 66. The image generator 52B displays the position of the virtual cross section on the MLO schematic image 66A based on the ultrasound image 64. Specifically, the image generator 52B performs the same process as when the position of the virtual cross section is displayed on the radiation image 62.

[0092] More specifically, the image generator 52B adjusts the distances d2 and d3 in the sagittal plane ultrasound image 64A to match the size of the MLO schema image 66A, and displays the positions of the virtual cross sections superimposed on the MLO schema image 66A. In the example shown in FIG. 10, a cross-shaped mark 82 (hereinafter also simply referred to as "mark 82") indicating the position of the virtual cross section is displayed on the MLO schema image 66A. The line segment along the vertical direction of the image of the cross-shaped mark 82 corresponds to the virtual cross section indicated by the coronal plane ultrasound image 64C. The line segment along the horizontal direction of the image of the cross-shaped mark 82 corresponds to the virtual cross section indicated by the transverse plane ultrasound image 64B.

[0093] The image generator 52B also displays the positions of the virtual cross sections on the compressed schematic image 66C based on the cross section position information 80. Specifically, in the coronal plane ultrasound image 64C, the sagittal plane is at a distance d1 from the top end of the image and at a distance d2 from the left end of the image. The coronal plane ultrasound image 64C is a cross section image of the breast M viewed from the front of the subject A, similar to the compressed schematic image 66C. Therefore, the image generator 52B adjusts these distances d1 and d2 according to the size of the breast M in the compressed schematic image 66C, and displays the positions of the virtual cross sections superimposed on the compressed schematic image 66C. A cross mark 84 (hereinafter also simply referred to as "mark 84") indicating the position of the virtual cross section is displayed on the compressed schematic image 66C. A line segment along the compression direction of the cross mark 84 corresponds to the virtual cross section indicated by the transverse section ultrasound image 64B. The line segment of the cross mark 84 along a direction perpendicular to the compression direction corresponds to the virtual cross section shown by the sagittal plane ultrasonic image 64A.

[0094] As an example, as shown in FIG. 11, the image generating unit 52B displays the position of the virtual cross section on the ultrasound schema image 66D based on the position of the virtual cross section on the MLO schema image 66A and the cross section position information 80. Here, there is a geometric positional relationship between the MLO schema image 66A and the ultrasound schema image 66D. That is, the MLO schema image 66A is a schematic diagram of the breast M viewed from the compression direction in a state where the breast M is compressed in a direction inclined from the cranio-caudal direction. Also, the ultrasound schema image 66D is a schematic diagram of the breast M viewed from the front in a state where the breast M is not compressed. Therefore, the MLO schema image 66A is disposed below the ultrasound schema image 66D in the compression direction (here, the lower left), and the upper and lower end positions in the direction perpendicular to the compression direction of the breast M are aligned between the MLO schema image 66A and the ultrasound schema image 66D. When arranged in this manner, in the ultrasound schema image 66D, the position of the virtual cross section is located on a straight line L1 extending along the compression direction from the intersection of the virtual cross sections in the MLO schema image 66A. The image generating unit 52B derives the straight line L1 based on the geometric positional relationship between the MLO schema image 66A and the ultrasound schema image 66D as described above.

[0095] Furthermore, on the straight line L1, the position of the intersection of the virtual cross section in the ultrasonic schema image 66D is determined by the depth D1 in the compression direction of the breast M. Here, the depth D1 is the depth from the body surface of the virtual cross section shown in the sagittal plane ultrasonic image 64A. More specifically, the depth D1 is the distance in the compression direction from the contact position between the compression member 30 and the breast M in the compressed state of the breast M to the virtual cross section shown in the sagittal plane ultrasonic image 64A. That is, the depth D1 corresponds to the distance d1 in the transverse plane ultrasonic image 64B or the coronal plane ultrasonic image 64C. This is because the distance d1 is the distance from the top end of the image (i.e., the contact position with the compression member 30) to the position of the virtual cross section shown in the sagittal plane ultrasonic image 64A. The image generator 52B changes the distance d1 indicated by the cross section position information 80 according to the size of the ultrasonic schema image 66D, and calculates the depth D1. In the ultrasound schema image 66D, a position on the straight line L1 that is a depth D1 away from the boundary (i.e., the surface of the breast M) is estimated to be the position of the intersection of the virtual cross sections. This allows the position of the virtual cross section of the ultrasound schema image 66D to be estimated.

[0096] The image generating unit 52B displays the position of the virtual cross section on the ultrasound schema image 66D. In the example shown in Fig. 11, a cross-shaped mark 86 (hereinafter also simply referred to as "mark 86") is displayed on the ultrasound schema image 66D. Here, the mark 86 is displayed in a manner that allows it to be distinguished from the mark 82 displayed on the MLO schema image 66A. Specifically, the mark 82 displayed on the MLO schema image 66A is displayed with a solid line, and the mark 86 displayed on the ultrasound schema image 66D is displayed with a broken line.

[0097] Here, the position of the virtual cross section indicated by the mark 86 displayed in the MLO schema image 66A is displayed based on the cross section position information 80 of the ultrasound image 64. On the other hand, the position of the virtual cross section indicated by the mark 86 displayed in the ultrasound schema image 66D is estimated based on the position of the virtual cross section displayed in the MLO schema image 66A. That is, the mark 82 displayed based on the cross section position information 80 and the mark 86 estimated based on the position of the virtual cross section in the MLO schema image 66A are displayed so as to be distinguishable.

[0098] Further, the image generating unit 52B displays the position of the virtual cross section on the CC schema image 66B based on the position of the virtual cross section on the ultrasound schema image 66D and the cross section position information 80. Here, there is a geometric positional relationship between the ultrasound schema image 66D and the CC schema image 66B. That is, as described above, the ultrasound schema image 66D is a schematic diagram of the breast M viewed from the front. The CC schema image 66B is a schematic diagram of the breast M viewed from the compression direction in a state where the breast M is compressed from above. Therefore, the CC schema image 66B is arranged below the ultrasound schema image 66D, and the left and right ends of the ultrasound schema image 66D and the CC schema image 66B are aligned. When arranged in this way, the position of the intersection of the virtual cross section on the CC schema image 66B is located on a straight line L2 extending downward from the position of the intersection of the virtual cross section on the ultrasound schema image 66D. The image generating unit 52B derives the straight line L2 based on the geometric positional relationship between the ultrasound schema image 66D and the CC schema image 66B as described above.

[0099] Furthermore, the position of the intersection of the virtual cross section in the CC schema image 66B on the straight line L2 is determined by the distance D3 from the body surface. Here, the distance D3 is the distance from the body surface of the virtual cross section shown in the coronal plane ultrasound image 64C. More specifically, the distance D3 is the distance from the body surface of the breast M when the breast M is compressed. That is, the distance D3 corresponds to the distance d3 in the sagittal plane ultrasound image 64A. This is because the distance d3 is the distance from the left end of the image to the virtual cross section shown in the coronal plane ultrasound image 64C. The image generator 52B changes the distance d3 indicated by the cross section position information 80 according to the size of the CC schema image 66B to calculate the distance D3. In the CC schema image 66B, the position that is the distance D3 away from the boundary on the straight line L2 (i.e., the front surface of the breast M) is estimated to be the position of the intersection of the virtual cross section. This allows the position of the virtual cross section of the CC schema image 66B to be estimated.

[0100] The image generating unit 52B displays the position of the virtual cross section on the CC schema image 66B. In the example shown in Fig. 11, a cross-shaped mark 88 (hereinafter also simply referred to as "mark 88") is displayed on the CC schema image 66B. The mark 88 is displayed in a manner that allows it to be distinguished from the mark 86 displayed on the MLO schema image 66A, similar to the ultrasound schema image 66D, and the mark 88 is displayed by a dashed line.

[0101] 12, the image generator 52B generates a display image 68 including an examination image 60 in which the positions of the virtual cross sections are displayed, and a schematic image 66 in which the positions of the virtual cross sections are also displayed. The image generator 52B also generates text indicating the position of the region of interest based on the cross section position information 80. The display image 68 includes a text box 90 indicating the position of the region of interest.

[0102] As described above, the position of the region of interest in the sagittal plane ultrasound image 64A is the position of the intersection of the mark 72. Each line segment of the mark 72 indicates the position of the virtual cross section. In other words, the position of the region of interest indicates the position of the intersection of the virtual cross section, and the text box 90 indicates information regarding the position of the intersection of the virtual cross section.

[0103] The image generating section 52B outputs the generated display image 68 to the output section 52C, and the output section 52C performs GUI control for displaying the display image 68, thereby causing the display device 17 to display the display image 68.

[0104] 12, a region of interest is designated in the display image 68 via the pointer 70, and thus a mark 81 is displayed in the radiation image 62. The mark 81 corresponds to the mark 72 displayed in the sagittal plane ultrasound image 64A. That is, the mark 81 indicates the position of the virtual cross section, similar to the mark 72.

[0105] The mark 81 displayed on the radiation image 62 moves in accordance with the movement of the marks 72, 74, and 76 displayed on the ultrasound image 64. That is, the mark 81 is linked to the marks 72, 74, and 76 displayed on the ultrasound image 64. In other words, by moving the mark 81, the marks 72, 74, and 76 on the ultrasound image 64 can also be moved.

[0106] Furthermore, when a region of interest is specified via the pointer 70 in the display image 68, a mark 82 is displayed in the MLO schema image 66A. The mark 82 indicates the position of a virtual cross section of the ultrasound image 64 in the MLO schema image 66A. Here, the position of the virtual cross section in the ultrasound image 64 is a virtual cross section that is determined by specifying the region of interest via the pointer 70. Therefore, in the MLO schema image 66A, the mark 82 also indicates the position of the region of interest. Specifically, the positions of the intersections of the line segments of the cross-shaped mark 82 indicate the position of the region of interest.

[0107] Furthermore, when a region of interest is specified via the pointer 70 in the display image 68, a mark 84 is also displayed in the compression schematic image 66C. The mark 84 indicates the position of a virtual cross section of the ultrasound image 64 in the compression schematic image 66C, and the position of the intersection of the cross-shaped mark 84 indicates the position of the region of interest.

[0108] Furthermore, when a region of interest is specified via the pointer 70 in the display image 68, a mark 86 is displayed in the CC schema image 66B. The mark 86 indicates the position of a virtual cross section estimated based on the ultrasound image 64 in the CC schema image 66B, and the position of the intersection of the cross-shaped mark 86 indicates the position of the region of interest.

[0109] Furthermore, when a region of interest is specified via the pointer 70 in the display image 68, a mark 88 is also displayed in the ultrasound schematic image 66D. The mark 88 indicates the position of a virtual cross section estimated based on the ultrasound image 64 in the ultrasound schematic image 66D, and the position of the intersection of the cross-shaped mark 88 indicates the position of the region of interest.

[0110] A text box 90 is also displayed in the display image 68. The text box 90 displays text related to the region of interest designated via the pointer 70. In the example shown in Fig. 12, the following text is displayed in the text box 90: Target Depth: 3.53cm Depth: 3.8cm FOV: 6.0cm TPX plate: 0.27cm Here, Target Depth refers to the depth to the region of interest excluding the thickness of the bottom plate 30A of the compression member 30, Depth refers to the depth to the region of interest, FOV refers to the depth of field of view, and TPX plate refers to the thickness of the bottom plate 30A of the compression member 30.

[0111] In this way, when a region of interest is specified in the display image 68, the position of the virtual cross section indicating the region of interest is displayed not only in the ultrasound image 64 but also in the radiation image 62 and the schematic image 66.

[0112] Next, a display control process in the client terminal 16 according to the present embodiment will be described with reference to Fig. 13 and Fig. 14. Fig. 13 and Fig. 14 are flowcharts showing an example of the display control process. The process flow shown in Fig. 13 and Fig. 14 is an example of a "medical support method" according to the technology of the present disclosure.

[0113] 13 as an example, first, in step ST12, the acquisition unit 52A acquires the examination image 60 and the schematic image 66 from the image database 14. After the process of step ST12 is executed, the display control process proceeds to step ST14.

[0114] In step ST14, the image generating unit 52B derives the contour line of the breast M based on the radiation image 62 included in the examination image 60 acquired by the acquiring unit 52A in step ST12. After the processing of step ST14 is executed, the display control processing proceeds to step ST16.

[0115] In step ST16, the image generator 52B adjusts and displays the contour line derived in step ST14 in accordance with the size of the ultrasound image 64. After the process of step ST16 is executed, the display control process proceeds to step ST18.

[0116] In step ST18, the image generator 52B calculates the blind area 65 in the ultrasound image 64. After the process of step ST18 is executed, the display control process proceeds to step ST20.

[0117] In step ST20, the image generator 52B displays the blind area 65 calculated in step ST18 on the ultrasound image 64, and displays the blind area 67 on the MLO schema image 66A. After the process of step ST20 is executed, the display control process proceeds to step ST22.

[0118] In step ST22, the output unit 52C outputs the display image 68 generated by the image generating unit 52B to the display device 17. After the process of step ST22 is executed, the display control process proceeds to step ST24.

[0119] In step ST24, the acquisition unit 52A determines whether or not the designation of the region of interest in the display image 68 has been accepted via the acceptance device 19. If the designation of the region of interest has been accepted, the determination is affirmative, and the display control process proceeds to step ST26. If the designation of the region of interest has not been accepted, the determination is negative, and step ST24 is executed again.

[0120] In step ST26, the image generator 52B displays the cross-sectional position on the radiation image 62 based on the cross-sectional position information 80 corresponding to the position of the region of interest designated in step ST24. After the process of step ST26 is executed, the display control process proceeds to step ST28.

[0121] In step ST28, the image generator 52B displays the cross-sectional positions on the MLO schematic image 66A and the compressed schematic image 66C based on the cross-sectional position information 80. After the process of step ST28 is executed, the display control process proceeds to step ST30.

[0122] 14, in step ST30, the image generator 52B displays the cross-sectional position on the ultrasound schema image 66D based on the cross-sectional position information 80 and the MLO schema image 66A. After the process of step ST30 is performed, the display control process proceeds to step ST32.

[0123] In step ST32, the image generator 52B displays the cross-sectional position on the CC schema image 66B based on the cross-sectional position information 80 and the ultrasound schema image 66D. After the process of step ST32 is executed, the display control process proceeds to step ST34.

[0124] In step ST34, the output unit 52C outputs the display image 68, the display of which has been updated by the image generation unit 52B, to the display device 17. After the process of step ST34 is executed, the display control process proceeds to step ST36.

[0125] In step ST36, the output unit 52C determines whether or not a condition for terminating the display control process (hereinafter referred to as the "termination condition") has been satisfied. One example of the termination condition is that an instruction to terminate the display control process has been accepted. In step ST36, if the termination condition has not been satisfied, the determination is negative, and the display control process proceeds to step ST12. In step ST36, if the termination condition has been satisfied, the determination is positive, and the display control process terminates.

[0126] As described above, in the medical support system 1 according to the technique of the present disclosure, the client terminal 16 includes the processor 52, and in the processor 52, the acquisition unit 52A acquires the examination image 60 and the schematic image 66. The examination image 60 includes a radiographic image 62 and an ultrasound image 64. The ultrasound image 64 is obtained by capturing an image in a compressed state that can be considered to be the same as the radiographic image in which the radiographic image 62 was captured. The schematic image 66 includes an MLO schematic image 66A. The image generation unit 52B displays the position of the virtual cross section on the MLO schematic image 66A based on the ultrasound image 64. Then, the output unit 52C causes the display device 17 to display a display image 68. The display image 68 includes the examination image 60 and the MLO schematic image 66A in which the position of the virtual cross section is displayed.

[0127] Generally, radiography is suitable for detecting calcification in the breast M, but when the breast density is high, it is difficult to detect a tumor. On the other hand, ultrasound examination is difficult to detect calcification, but it can detect a tumor even when the breast density is high. Therefore, by performing ultrasound examination on the breast M of the same subject A under a compression state that can be considered to be the same as that of radiography, it is possible to detect both calcification and a tumor in the breast M with high breast density. In this case, a radiographic image 62 and an ultrasound image 64 are obtained, and they are compared for interpretation. Furthermore, when the radiographic imaging technique is MLO imaging, an MLO schema image 66A, which is a schematic diagram corresponding to the radiographic image 62, is used for writing the position of the findings during interpretation. In this configuration, the positions of the virtual cross sections are displayed on the radiographic image 62 and the ultrasound image 64, and further, the position of the virtual cross section is also displayed on the MLO schema image 66A. This makes it easier to grasp the positions of the virtual cross sections corresponding to each other in the examination image 60 and the schema image 66, and makes it easier to associate the examination image 60 with the schema image 66.

[0128] Furthermore, for example, an ultrasound image 64 obtained by ultrasound examination in a state where the breast M is compressed (hereinafter also simply referred to as a "compression ultrasound examination") looks different from an image obtained by a normal examination method in which an ultrasound probe 42 is placed against the breast M (hereinafter also simply referred to as a "normal examination method"), making it difficult to associate the ultrasound image 64 with the radiation image 62 or the schematic image 66. In this configuration, the position of the virtual cross section is displayed with respect to the radiation image 62 or the schematic image 66 based on the position of the virtual cross section in the ultrasound image 64. This makes it easy to associate the ultrasound image 64 obtained by ultrasound examination in a state where the breast M is compressed with the radiation image 62 and / or the schematic image 66.

[0129] In the medical support system 1 according to the present embodiment, the schema image 66 in the display image 68 includes an ultrasound schema image 66D. The ultrasound schema image 66D is an image showing a schematic diagram corresponding to an ultrasound examination image of a normal examination method. The ultrasound schema image 66D displays the position of the virtual cross section of the ultrasound image 64. In general, the user 18 is more familiar with the schema of the normal examination method than with the schema corresponding to the compression ultrasound examination. Therefore, by including the ultrasound schema image 66D displaying the position of the virtual cross section in the schema image 66, the user 18 can easily grasp the position of the virtual cross section in the ultrasound image 64.

[0130] Furthermore, in the medical support system 1 according to this embodiment, when a region of interest is specified in the ultrasound image 64, the position corresponding to the region of interest is displayed in both the MLO schema image 66A and the ultrasound schema image 66D. This makes it easy to associate the positions of the region of interest between the ultrasound image 64 and the MLO schema image 66A and the ultrasound schema image 66D, since the position of the region of interest is displayed in the MLO schema image 66A and the ultrasound schema image 66D.

[0131] In addition, in the medical support system 1 according to the present embodiment, the position of the virtual cross section in the ultrasound schema image 66D is derived based on the position of the virtual cross section in the MLO schema image 66A and the depth from the body surface in the compression direction indicated by the cross-sectional position information 80. Here, the depth from the body surface corresponds to the distance d1 from the body surface of the virtual cross section indicated by the sagittal plane ultrasound image 64A. The ultrasound image 64 obtained by the compression ultrasound examination looks different from the image obtained by the normal examination method, and it is difficult to accurately associate the ultrasound image 64 with the radiation image 62 or the schema image 66. In this configuration, the position of the virtual cross section in the ultrasound schema image 66D is derived based on the MLO schema image 66A and the depth indicated by the cross-sectional position information 80. Therefore, the accuracy of the association of the positions of the virtual cross sections is improved compared to the case where the user 18 associates them empirically.

[0132] Furthermore, in the medical support system 1 according to this embodiment, in the display image 68, the schematic image 66 includes a CC schematic image 66B. The CC schematic image 66B is an image showing a schematic diagram of the breast M in CC imaging, which is an imaging direction different from MLO imaging. The position of the virtual cross section of the ultrasound image 64 is displayed in the CC schematic image 66B. In radiation imaging, radiation imaging may be performed only from one direction (here, MLO imaging) in consideration of the effect on the subject A. Even in this case, in this configuration, the virtual cross section is displayed in the CC schematic image 66B, so that the user 18 can grasp the position of the virtual cross section in the CC schematic image 66B.

[0133] In addition, in the medical support system 1 according to this embodiment, the position of the virtual cross section in the CC schema image 66B is derived based on the position of the virtual cross section in the ultrasound schema image 66D and the distance from the body surface in the anterior-posterior direction of the subject A indicated by the cross section position information 80. Here, the distance in the anterior-posterior direction corresponds to the distance d3 from the body surface of the virtual cross section indicated by the coronal plane ultrasound image 64C. In this embodiment, since the CC schema image 66B is a schematic diagram of the breast M seen from a direction in which imaging is not performed, there is no radiation image 62 corresponding to the CC schema image 66B, and it is difficult to associate the positions of the virtual cross sections. In this configuration, the position of the virtual cross section in the CC schema image 66B is derived based on the position of the virtual cross section in the ultrasound schema image 66D and the distance from the body surface in the anterior-posterior direction. Therefore, the accuracy of the association of the positions of the virtual cross sections is improved compared to the case in which the positions of the virtual cross sections in the CC schema image 66B are associated based on the empirical rule of the user 18.

[0134] In addition, in the medical support system 1 according to the present embodiment, in the sagittal plane ultrasound image 64A, a blind area 65, which is an image area in which the breast M is not captured in the compression ultrasound examination, is displayed in a manner that is distinguishable from other areas of the sagittal plane ultrasound image 64A. The ultrasound image 64 obtained by the compression ultrasound examination is different from the ultrasound examination image of the normal examination method in that there is an image area in which the breast M is not captured. In this case, the display ranges of the radiation image 62 and the ultrasound image 64 appear to be different, making it difficult to associate them. In this configuration, the blind area 65 is displayed in the sagittal plane ultrasound image 64A, making it easy to associate the radiation image 62 with the sagittal plane ultrasound image 64A.

[0135] Furthermore, in the medical support system 1 according to this embodiment, a blind area 67, which is an image region in which the breast M is not captured in a compression ultrasound examination, is displayed in an MLO schema image 66A so as to be distinguishable from other regions of the MLO schema image 66A. If there is a defect in the ultrasound image 64 obtained by a compression ultrasound examination, the display ranges of the MLO schema image 66A and the ultrasound image 64 appear to be different, making it difficult to associate them. In this configuration, the blind area 67 is displayed in the MLO schema image 66A, making it easy to associate the ultrasound image 64 with the MLO schema image 66A.

[0136] In addition, in the medical support system 1 according to this embodiment, in the display image 68, the ultrasound schema image 66D includes a compression schema image 66C. The compression schema image 66C is a schematic diagram showing a state in which the compression member 30 is compressing the breast M. The compression schema image 66C displays the position of the virtual cross section in the ultrasound image 64. This makes it easier to grasp the position of the virtual cross section in the display image 68 when the breast M is being compressed. As a result, it becomes easier to grasp the correspondence between the examination image 60 and the schema image 66.

[0137] In addition, in the medical support system 1 according to the present embodiment, the position of the virtual cross section displayed in the compression schematic image 66C is derived based on the position of the virtual cross section in the ultrasound image 64. This improves the accuracy of the position of the virtual cross section in the compression schematic image 66C compared to the case where the virtual cross section is displayed based on the empirical rule of the user 18.

[0138] Furthermore, in the medical support system 1 according to this embodiment, a text box 90 is displayed in the display image 68. The text box 90 includes text indicating the position of the region of interest. The position of the region of interest is indicated as an intersection of the virtual cross sections. That is, the display image 68 displays information indicating the depth of the intersection of the virtual cross sections. This displays information on the depth of the virtual cross sections in the compression direction, making it easier to grasp the positions of the virtual cross sections in the examination image 60 and the schematic image 66.

[0139] In addition, in the medical support system 1 according to this embodiment, a mark 82 indicating the position of the virtual cross section displayed in the MLO schema image 66A and a mark 84 indicating the position of the virtual cross section displayed in the compression schema image 66C are displayed with solid lines. Meanwhile, a mark 86 indicating the position of the virtual cross section displayed in the CC schema image 66B and a mark 88 indicating the position of the virtual cross section displayed in the ultrasound schema image 66D are displayed with dashed lines. This makes it possible to distinguish between a virtual cross section displayed based on the position of the virtual cross section in the ultrasound image 64 and the position of the virtual cross section estimated from another schema image. As a result, the display accuracy of the position of the virtual cross section can be grasped.

[0140] (Variation 1) In the above embodiment, the radiography technique is MLO imaging, and an example in which the radiographic image 62 is obtained by MLO imaging has been described, but the technology of the present disclosure is not limited to this. In this first modified example, the radiographic image 62 is obtained by CC imaging. Then, the image generator 52B displays the position of the virtual cross section in the MLO schema image 66A, which is a schema in a direction in which imaging is not performed. In this modified example, the CC schema image 66B is an example of a "first schema image" according to the technology of the present disclosure, and the MLO schema image 66A is an example of a "third schema image" according to the technology of the present disclosure.

[0141] As an example, as shown in FIG. 15, the image generating unit 52B displays the position of the virtual cross section in the ultrasound schema image 66D based on the position of the virtual cross section in the CC schema image 66B and the cross section position information 80. Here, there is a geometric positional relationship between the CC schema image 66B and the ultrasound schema image 66D. That is, as described above, the CC schema image 66B is a schematic diagram of the breast M viewed from the compression direction in a state where the breast M is compressed from above. The ultrasound schema image 66D is a schematic diagram of the breast M viewed from the front. Therefore, the CC schema image 66B is arranged below the ultrasound schema image 66D in the compression direction, and the left and right ends of the ultrasound schema image 66D and the CC schema image 66B are aligned. When arranged in this way, the position of the intersection of the virtual cross section in the ultrasound schema image 66D is located on a straight line L3 extending along the compression direction from the position of the intersection of the virtual cross section in the CC schema image 66B. The image generator 52B derives the straight line L3 based on the geometric positional relationship between the CC schema image 66B and the ultrasound schema image 66D as described above.

[0142] Furthermore, on the straight line L3, the position of the intersection of the virtual cross section in the ultrasonic schema image 66D is determined by a depth D1 in the compression direction of the breast M. Here, the depth D1 is the depth from the body surface of the virtual cross section shown in the sagittal plane ultrasonic image 64A. As described above, the depth D1 corresponds to the distance d1 in the transverse plane ultrasonic image 64B or the coronal plane ultrasonic image 64C. The image generator 52B changes the distance d1 shown by the cross section position information 80 according to the size of the ultrasonic schema image 66D to calculate the depth D1. In the ultrasonic schema image 66D, a position on the straight line L3 away from the boundary (i.e., the surface of the breast M) is estimated to be the position of the intersection of the virtual cross section. As a result, the position of the virtual cross section of the ultrasonic schema image 66D is estimated.

[0143] The image generating unit 52B displays the position of the virtual cross section on the ultrasound schema image 66D based on the position of the virtual cross section on the CC schema image 66A and the cross section position information 80. Here, there is a geometric positional relationship between the ultrasound schema image 66D and the MLO schema image 66A. That is, the MLO schema image 66A is a schematic diagram of the breast M viewed from the compression direction in a state where the breast M is compressed in a direction inclined from the cranio-caudal direction. Also, the ultrasound schema image 66D is a schematic diagram of the breast M viewed from the front in a state where the breast M is not compressed. Therefore, the MLO schema image 66A is disposed below (here, at the lower left) the MLO schema image 66A in the virtual compression direction with respect to the ultrasound schema image 66D, and the positions of the upper and lower ends in the direction perpendicular to the compression direction of the breast M are aligned between the MLO schema image 66A and the ultrasound schema image 66D. When arranged in this manner, in the MLO schema image 66A, the position of the virtual cross section is located on a straight line L4 that extends along the compression direction from the intersection of the virtual cross sections in the ultrasound schema image 66D. The image generator 52B derives the straight line L4 based on the geometric positional relationship between the ultrasound schema image 66D and the MLO schema image 66A as described above.

[0144] Furthermore, the position of the intersection of the virtual cross section in the MLO schema image 66A on the straight line L4 is determined by a distance D3 from the body surface. Here, the distance D3 is the distance from the body surface to the virtual cross section shown in the coronal plane ultrasound image 64C. The distance D3 corresponds to the distance d3 in the sagittal plane ultrasound image 64A. The image generator 52B changes the distance d3 shown in the cross section position information 80 according to the size of the MLO schema image 66A to calculate the distance D3. In the MLO schema image 66A, a position that is the distance D3 away from the boundary on the straight line L4 (i.e., the front surface of the breast M) is estimated to be the position of the intersection of the virtual cross section. As a result, the position of the virtual cross section in the MLO schema image 66A is estimated.

[0145] The image generator 52B displays the positions of the virtual cross sections on the ultrasound schema image 66D and the MLO schema image 66A. In the example shown in Fig. 15, dashed line marks 82 and 88 are displayed on the ultrasound schema image 66D and the MLO schema image 66A, respectively.

[0146] 16 as an example, the image generating unit 52B generates a display image 68 including an examination image 60 in which the position of the virtual cross section is displayed, and a schematic image 66 in which the position of the virtual cross section is also displayed. The image generating unit 52B outputs the generated display image 68 to the output unit 52C, and the output unit 52C controls the GUI to display the display image 68, thereby causing the display device 17 to display the display image 68.

[0147] In the example shown in FIG. 16, a region of interest is designated in the display image 68 via the pointer 70 (see FIG. 12), and thus a mark 81 is displayed on the radiation image 62 obtained by CC imaging.

[0148] Furthermore, in the display image 68, a mark 82 is displayed in the MLO schematic image 66A as a result of a region of interest being designated via the pointer 70. Also, a mark 84 is displayed in the compression schematic image 66C.

[0149] Furthermore, a mark 86 is displayed on the CC schema image 66B by specifying a region of interest on the display image 68 via the pointer 70. A mark 88 is also displayed on the ultrasound schema image 66D.

[0150] Marks 81, 82, 84, 86, and 88 indicate the positions of virtual cross sections of the ultrasound image 64 in the displayed images, and the position of the intersection of the cross-shaped mark 84 indicates the position of the region of interest. In addition, in the schematic image 66, marks 84 and 86 indicating the positions of virtual cross sections indicated by the cross section position information 80 are indicated by solid lines, and marks 82 and 88 indicating the positions of virtual cross sections estimated by the image generator 52B are indicated by dashed lines, so that both are distinguishable.

[0151] In addition, since the radiation image 62 is obtained by CC imaging, a blind area 67 is displayed in the CC schematic image 66B. That is, a contour line is calculated from the radiation image 62, and the calculated contour line is reflected in the ultrasound image 64, and a blind area 65 in the ultrasound image 64 is obtained from the result. Then, the blind area 65 in the ultrasound image 64 is adjusted to fit the size of the CC schematic image 66B, and the blind area 67 is displayed in the CC schematic image 66B.

[0152] As described above, in the first modified example, the position of the virtual cross section in the ultrasound schema image 66D is derived based on the position of the virtual cross section in the CC schema image 66B and the depth from the body surface in the compression direction indicated by the cross-sectional position information 80. Here, the depth from the body surface corresponds to the distance d1 from the body surface of the virtual cross section indicated by the sagittal plane ultrasound image 64A. The ultrasound image 64 obtained by the compression ultrasound examination looks different from the image obtained by the normal examination method, and it is difficult to accurately associate the ultrasound image 64 with the radiation image 62 or the schema image 66. In this configuration, the position of the virtual cross section in the ultrasound schema image 66D is derived based on the CC schema image 66B and the depth indicated by the cross-sectional position information 80. Therefore, the accuracy of the association of the positions of the virtual cross sections is improved compared to the case where the user 18 associates them based on experience.

[0153] In the first modified example, the position of the virtual cross section in the MLO schema image 66A is derived based on the position of the virtual cross section in the ultrasound schema image 66D and the distance from the body surface in the anterior-posterior direction of the subject A indicated by the cross section position information 80. In the modified example, since the MLO schema image 66A is a schematic diagram of the breast M seen from a direction in which imaging is not performed, there is no radiation image 62 corresponding to the MLO schema image 66A, and it is difficult to associate the positions of the virtual cross sections. In this configuration, the position of the virtual cross section in the MLO schema image 66A is derived based on the position of the virtual cross section in the ultrasound schema image 66D and the distance from the body surface in the anterior-posterior direction. Therefore, the accuracy of the association of the positions of the virtual cross sections is improved compared to the case in which the positions of the virtual cross sections in the MLO schema image 66A are associated based on the empirical rule of the user 18.

[0154] (Variation 2) In the above embodiment, an example in which only either MLO imaging or CC imaging is performed in radiography has been described, but the technology of the present disclosure is not limited to this. In this second modified example, both MLO imaging and CC imaging are performed as radiography techniques, and radiographic images 62 are obtained by each imaging technique. Then, based on the MLO schema image 66A and the CC schema image 66B, the position of the virtual cross section of the ultrasound schema image 66D is displayed.

[0155] As an example, as shown in FIG. 17, the image generating unit 52B displays the positions of the virtual cross sections on an MLO schema image 66A and a CC schema image 66B based on the cross section position information 80.

[0156] The image generating unit 52B displays the virtual cross-section position on the ultrasound schema image 66D based on the position of the virtual cross-section in the MLO schema image 66A and the position of the virtual cross-section in the CC schema image 66B. As described above, there is a geometric positional relationship between the MLO schema image 66A and the ultrasound schema image 66D. The MLO schema image 66A is disposed below the ultrasound schema image 66D in the compression direction (here, the lower left), and the positions of the upper and lower ends in the direction perpendicular to the compression direction of the breast M are aligned between the MLO schema image 66A and the ultrasound schema image 66D. When disposed in this manner, the position of the virtual cross-section in the ultrasound schema image 66D is located on a straight line L5 extending from the intersection of the virtual cross-sections in the MLO schema image 66A along the compression direction.

[0157] Furthermore, there is a geometric positional relationship between the CC schema image 66B and the ultrasound schema image 66D. The CC schema image 66B is arranged below the ultrasound schema image 66D in the compression direction, and the left and right ends of the ultrasound schema image 66D and the CC schema image 66B are aligned. When arranged in this manner, the position of the intersection of the virtual cross sections in the ultrasound schema image 66D is located on a straight line L6 that extends along the compression direction from the position of the intersection of the virtual cross sections in the CC schema image 66B.

[0158] The position of the virtual cross section in the ultrasonic schema image 66D is estimated to be located at the intersection of the straight lines L5 and L6. The image generator 52B estimates the position of the virtual cross section in the ultrasonic schema image 66D using the above-mentioned relationship. This allows the position of the virtual cross section of the ultrasonic schema image 66D to be estimated.

[0159] The image generator 52B displays the positions of the virtual cross sections on the ultrasound schematic image 66D. In the example shown in Fig. 17, dashed line marks 88 are displayed on the ultrasound schematic image 66D.

[0160] 18, the image generating unit 52B generates a display image 68 including an examination image 60 in which the position of the virtual cross section is displayed, and a schematic image 66 in which the position of the virtual cross section is also displayed. The image generating unit 52B outputs the generated display image 68 to the output unit 52C, and the output unit 52C controls the GUI to display the display image 68, thereby causing the display device 17 to display the display image 68.

[0161] In the example shown in FIG. 18, a region of interest is designated in the display image 68 via the pointer 70 (see FIG. 12), and thus a mark 81A is displayed on the radiographic image 62A obtained by MLO imaging. In this modification, the radiographic image 62A obtained by MLO imaging is an example of a "first radiographic image" according to the technology of the present disclosure. Also, a mark 81B is displayed on the radiographic image 62B obtained by CC imaging. That is, the marks 81A and 81B are displayed by the image generating unit 52B based on the cross-sectional position information 80 indicating the position of the virtual cross section in the ultrasound image 64. In this modification, the radiographic image 62B obtained by CC imaging is an example of a "second radiographic image" according to the technology of the present disclosure.

[0162] Furthermore, a mark 82 is displayed on the MLO schematic image 66A as a result of a region of interest being designated on the display image 68 via the pointer 70. The MLO schematic image 66A corresponds to the radiation image 62A obtained by MLO imaging.

[0163] Also, a mark 84A is displayed on a compression schematic image 66C1 showing the state of compression in MLO imaging, and a mark 84B is displayed on a compression schematic image 66C2 showing the state of compression in CC imaging.

[0164] Furthermore, a mark 86 is displayed in the CC schematic image 66B by specifying a region of interest in the display image 68 via the pointer 70. The CC schematic image 66B corresponds to the radiation image 62B obtained by CC imaging. A mark 88 is also displayed in the ultrasound schematic image 66D.

[0165] Marks 81, 82, 84A, 84B, 86, and 88 indicate the positions of virtual cross sections of the ultrasound image 64 in the displayed images, and the position of the intersection of the cross-shaped mark 84 indicates the position of the region of interest. In addition, in the schematic image 66, marks 82, 84, 84A, 84B, and 86 indicating the positions of the virtual cross sections indicated by the cross section position information 80 are indicated by solid lines, and mark 88 indicating the position of the virtual cross section estimated by the image generator 52B is indicated by a dashed line, so that the two can be distinguished from each other.

[0166] Furthermore, since the radiation image 62 is obtained by both MLO and CC imaging, a blind area 67 is displayed in each of the MLO schematic image 66A and the CC schematic image 66B.

[0167] As described above, in the present second modified example, the position of the virtual cross section in the ultrasound schema image 66D is derived based on the positions of the virtual cross section in the MLO schema image 66A and the CC schema image 66B. This improves the accuracy of the position of the virtual cross section in the ultrasound schema image 66D compared to the case where the position of the virtual cross section in the ultrasound schema image 66D is displayed based on the empirical rule of the user 18.

[0168] In the second modified example, the positions of the virtual cross sections are displayed in the MLO schema image 66A and the CC schema image 66B based on the cross section position information 80. This improves the accuracy of the positions of the virtual cross sections in the MLO schema image 66A and the CC schema image 66B. As a result, the accuracy of the positions of the virtual cross sections in the ultrasound schema image 66D, the positions of which are estimated based on these positions, also improves.

[0169] (Variation 3) In the present third modified example, when the position of the virtual cross section displayed in the schematic image 66 is changed by the user 18 in the display image 68, the display manner of the changed virtual cross section is made distinguishable from that before the change.

[0170] As an example, as shown in FIG. 19, in a display image 68, an operation from the user 18 is accepted via the acceptance device 19, and thereby the position of the virtual cross section displayed in the schematic image 66 is changed. For example, the user 18 corrects the position of the virtual cross section previously displayed in the schematic image 66. In the example shown in FIG. 19, a mark 82 displayed in the MLO schematic image 66A is moved to the lower right via the pointer 70. Since the positions of the line segments of the mark 82 indicate the positions of the virtual cross sections, the position of the virtual cross section is changed by moving the mark 82. The display mode of the mark 82 after the virtual cross section is changed is a two-dot chain line. That is, the two-dot chain line, which is the display mode of the mark 82 after the change, is displayed so as to be distinguishable from the dashed line before the change.

[0171] Although the embodiment in which the position of the virtual cross section is changed in the MLO schema image 66A has been described here, this is merely an example. The positions of the virtual cross sections in the CC schema image 66B, the compression schema image 66C, and the ultrasound schema image 66D may be changed.

[0172] Also, although the embodiment has been described in which the line type of the mark 82 changes after the position of the virtual cross section is changed, this is merely one example. As long as the display mode before and after the change can be distinguished, the line type may be changed together with the thickness and / or color instead of the line type. Also, text may be displayed indicating that the change has been made, or the mark may be changed to a different type of mark (for example, a circle, etc.).

[0173] As described above, in the third modified example, when the position of the virtual cross section is changed in the schematic image 66, the display mode of the position of the virtual cross section after the change is displayed in a manner that is distinguishable from the state before the change. This makes it easier to distinguish that the virtual cross section has been changed by the user 18 in the schematic image 66. For example, the position of the virtual cross section estimated by the image generating unit 52B may differ from the position of the virtual cross section thought by the user 18. In this case, the user 18 may change the position of the virtual cross section on the schematic image 66, and in this configuration, it is easier to understand that the position of the virtual cross section has been changed.

[0174] (Fourth Modification) In the fourth modified example, the display contents can be switched in the compressed schematic image 66C. As an example, as shown in FIG. 20, the position of the nipple of the breast M is hidden in the compressed schematic image 66C. Specifically, when the nipple is not clearly shown in the radiation image 62 and / or the ultrasound image 64, the nipple can be hidden in the compressed schematic image 66C. In the example shown in FIG. 20, the image area showing the nipple is selected via the pointer 70 in the compressed schematic image 66C, so that the nipple is hidden.

[0175] In addition, in the compression schematic image 66C, some of the line segments of the cross-shaped mark 84 are hidden. When the position of the virtual cross section is displayed by the image generating unit 52B, if the position of the virtual cross section is inaccurate, it may be easier to associate the position of the virtual cross section with other images if the position is not displayed. In the example shown in Fig. 20, in the compression schematic image 66C, a line segment of the mark 84 corresponding to the position of the cross section indicated by the transverse sectional ultrasound image 64B is selected via the pointer 70, so that some of the line segments of the mark 84 are hidden.

[0176] As described above, in the fourth modified example, since the display contents of the compression schematic image 66C are switchable, the user 18 can change the display contents of the compression schematic image 66C.

[0177] In the above embodiment, an example in which the region of interest is designated by the user 18 has been described, but the technology of the present disclosure is not limited to this. For example, the region of interest may be designated based on a detection result of the region of interest (e.g., a lesion) obtained by image recognition processing (e.g., image recognition processing using AI (Artificial Intelligence)) on the radiation image 62 and / or the ultrasound image 64.

[0178] In the above embodiment, an example in which the display image 68 is displayed to the user 18 has been described, but the technology of the present disclosure is not limited to this. For example, information indicating the display image 68 may be output to a printer device and printed, or the display image 68 may be output to an external device (for example, an external server) and used in the process of creating an interpretation report in the external device.

[0179] In addition, in each of the above embodiments, an example is described in which the display control processing is performed by the processor 52 of the client terminal 16, but the technology of the present disclosure is not limited to this, and the device that performs the display control processing may be provided outside the client terminal 16.

[0180] In this case, as an example, a medical support system 100 may be used as shown in FIG. 21. The medical support system 100 includes an information processing device 101 and an external communication device 102. The information processing device 101 is a device in which the display control program 54A is removed from the storage 54 of the computer 50 included in the client terminal 16 described in the above embodiment. The external communication device 102 is, for example, a server. The server is realized by, for example, a mainframe. Although a mainframe is illustrated here, this is merely an example, and the server may be realized by cloud computing, or may be realized by network computing such as fog computing, edge computing, or grid computing. Here, a server is cited as an example of the external communication device 102, but this is merely an example, and at least one personal computer or the like may be used as the external communication device 102 instead of the server.

[0181] The external communication device 102 includes a processor 104, a storage 106, a RAM 108, and a communication I / F 110, and the processor 104, the storage 106, the RAM 108, and the communication I / F 110 are connected to each other via a bus 112. The communication I / F 110 is connected to the information processing device 101 via a network 114. The network 114 is, for example, the Internet. Note that the network 114 is not limited to the Internet, and may be a WAN and / or a LAN such as an intranet.

[0182] A display control program 54A is stored in the storage 106. The processor 104 executes the display control program 54A on the RAM 108. The processor 104 performs the above-mentioned display control process in accordance with the display control program 54A executed on the RAM 108.

[0183] The information processing device 101 transmits a request signal requesting the external communication device 102 to execute display control processing. The communication I / F 110 of the external communication device 102 receives the request signal via the network 114. The processor 104 performs display control processing in accordance with the display control program 54A, and transmits the processing result to the information processing device 101 via the communication I / F 110. The information processing device 101 receives the processing result (e.g., the processing result by the image generation unit 52B) transmitted from the external communication device 102 via the communication I / F 58 (see FIG. 4), and outputs the received processing result to various devices such as the display device 17.

[0184] In the example shown in FIG. 21, the external communication device 102 is an example of a "medical support device" according to the technology of the present disclosure, and the processor 104 is an example of a "processor" according to the technology of the present disclosure.

[0185] Furthermore, the display control process may be distributed and performed by a plurality of devices including the information processing device 101 and the external communication device 102.

[0186] In the above embodiment, the display control program 54A is stored in the storage 54, but the technology of the present disclosure is not limited to this. For example, the display control program 54A may be stored in a storage medium (not shown) such as an SSD or a USB memory. The storage medium is a portable, computer-readable, non-transitory storage medium. The display control program 54A stored in the storage medium is installed in the client terminal 16. The processor 52 executes the display control process according to the display control program 54A.

[0187] Furthermore, the display control program 54A may be stored in a storage device such as another computer or server connected to the client terminal 16 via a network, and the display control program 54A may be downloaded in response to a request from the client terminal 16 and installed in the client terminal 16. In other words, the program (i.e., the program product) described in this embodiment may be provided on a recording medium or may be distributed from an external computer.

[0188] It is not necessary to store all of the display control program 54A in a storage device of another computer or server connected to the client terminal 16, or in the storage 54, and only a part of the display control program 54A may be stored therein. Note that the storage medium, the storage device of another computer or server connected to the client terminal 16, and other external storage are regarded as memories that are directly or indirectly connected to the processor 52 and used.

[0189] In addition, in the above embodiment, the computer 50 is exemplified as the client terminal 16, but the technology of the present disclosure is not limited to this, and a device including an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), and / or a PLD (Programmable Logic Device) may be applied instead of the computer 50. Also, instead of the computer 50, a combination of a hardware configuration and a software configuration may be used.

[0190] As the hardware resource for executing the display control process described in the above embodiment, the following various processors can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the display control process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process, such as an FPGA, a PLD, or an ASIC. Each processor has a built-in or connected memory, and each processor executes the display control process by using the memory.

[0191] The hardware resource that executes the display control process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the display control process may be a single processor.

[0192] As an example of a configuration using one processor, first, one or more CPUs and software are combined to configure one processor, and this processor functions as a hardware resource that executes the display control process. Second, as typified by SoC (System-on-a-chip), a processor is used that realizes the functions of the entire system, including multiple hardware resources that execute the display control process, in a single IC (Integrated Circuit) chip. In this way, the display control process is realized using one or more of the above various processors as hardware resources.

[0193] More specifically, the hardware structure of these various processors can be an electronic circuit that combines circuit elements such as semiconductor elements. The above display control process is merely an example. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processing may be changed without departing from the spirit of the invention.

[0194] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above description and illustrations, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above description and illustrations omit explanations of technical common sense that do not require explanation in order to enable the implementation of the technology of the present disclosure.

[0195] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or standard was specifically and individually indicated to be incorporated by reference.

[0196] The following supplementary notes are further disclosed regarding the above embodiment.

[0197] <Appendix 1> A processor is provided. The processor is an examination image including a radiographic image obtained by radiographing a breast compressed by a compression member and a plurality of ultrasound images taken of the breast compressed by the compression member in a compression state that can be regarded as the same as the radiographic image, the positions of virtual cross sections being displayed in each of the ultrasound image and the radiographic image; a first schematic image showing the breast in the radiography, the first schematic image being an image in which the position of the virtual cross section is displayed based on the ultrasound image; and Execute display control to display a display image including the inspection image and the schematic image. Medical support equipment. <Appendix 2> The schematic image further includes a second schematic image, The second schema image is an image showing a second schema, which is a schematic diagram corresponding to an ultrasound examination image of an examination method in which a probe is placed on the breast, and in which the position of the virtual cross section is displayed. 2. A medical support device as described in appendix 1. <Appendix 3> When a region of interest is specified in at least one of the ultrasound image and the radiation image, the region of interest is displayed at a corresponding position in both the first schematic image and the second schematic image. 3. A medical support device as described in appendix 2. <Appendix 4> In the second schema image, the position of the virtual cross section is derived based on the position of the virtual cross section in the first schema and the depth of the virtual cross section in the compression direction in the ultrasound image. 4. The medical support device according to claim 2 or 3. <Appendix 5> The schematic image further includes a third schematic image, The third schema image is an image showing a third schema, which is a schematic diagram of the breast in the radiography in which the direction of imaging is different from that of the first schema, and in which the position of the virtual cross section is displayed. 5. A medical support device according to any one of claims 2 to 4. <Appendix 6> In the third schema image, the position of the virtual cross section is derived based on the position of the virtual cross section in the second schema and the distance of the virtual cross section in the ultrasound image from a body surface of the subject in the front-back direction. 6. A medical support device as described in appendix 5. <Appendix 7> the radiological images include a first radiological image and a second radiological image obtained in an imaging direction different from that of the first radiological image, the first schematic image corresponds to the first radiation image; the third schematic image corresponds to the second radiation image; In the second schematic image, the position of the virtual cross section is derived based on the positions of the virtual cross section in the first schematic image and the third schematic image. 7. The medical support device according to claim 5 or 6. <Appendix 8> In the third schematic image, the position of the virtual cross section is displayed based on the ultrasound image. 8. The medical support device according to claim 7. <Appendix 9> In the ultrasound image, a first blind area, which is an image area in which the breast is not imaged in an ultrasound examination, is displayed so as to be distinguishable from other areas. 9. A medical support device according to any one of claims 1 to 8. <Appendix 10> In the schematic image, a second blind area, which is an image area in which the breast is not imaged in an ultrasound examination, is displayed so as to be distinguishable from other areas. 10. A medical support device according to any one of claims 1 to 9. <Appendix 11> The schematic image includes a fourth schematic image, The fourth schematic image is a schematic diagram showing a state in which the compression member is compressing the breast, and is an image showing a fourth schematic image in which the position of the virtual cross section is displayed. 11. A medical support device according to any one of claims 1 to 10. <Appendix 12> In the fourth schematic image, the position of the virtual cross section is derived based on the position of the virtual cross section in the ultrasound image. 12. The medical support device according to claim 11. <Appendix 13> In the displayed image, information regarding a depth of an intersection of the virtual cross section in a compression direction of the breast is further displayed. 13. A medical support device according to any one of claims 1 to 12. <Appendix 14> When the virtual cross section displayed in the schematic image based on the position of the virtual cross section in the ultrasound image is a first virtual cross section, The display mode of the position of the first virtual cross section is a virtual cross section estimated based on the position of the first virtual cross section displayed in the schematic image, and is a display mode that can be distinguished from a second virtual cross section displayed in an image other than the image in which the first virtual cross section is displayed. 14. A medical support device according to any one of claims 1 to 13. <Appendix 15> When the position of the virtual cross section displayed on the schematic image is changed, the virtual cross section after the change is displayed in a manner that allows the virtual cross section before the change to be distinguished from the virtual cross section before the change. 15. A medical support device according to any one of claims 1 to 14. <Appendix 16> A medical support device according to any one of claims 1 to 15; A radiation image capturing device; an ultrasound imaging device; A medical imaging system comprising: [Explanation of symbols]

[0198] 1 Medical Support System 10 Mammography equipment 12 Network 14 Image Database 16 Client Terminals 17 Display device 17A screen 18 users 19 Reception equipment 19A Keyboard 19B Mouse 20 Stand 20A Pedestal 20B Post 21 Arm 22 Radiation source housing 23 Main body 24 Photo stand 24A Shooting surface 25 Radiation source 26 Radiation Detector 27 Anti-scatter grid 28 Rail 30 Compression member 30A bottom plate 31 Irradiation field limiter 32 Face Guard 34 Moving parts 35 Moving mechanism 40 Ultrasound imaging device 42 Ultrasonic probe 42A Contact surface 50 Computer 52 processors 52A Acquisition Department 52B Image generation section 52C Output section 54 Storage 54A Display control program 56 RAM 57 Bus 58 Communication I / F 60 Inspection images 62,62A,62B Radiological images 63A, 63B Contour line 64 Ultrasound Images 64A Sagittal ultrasound image 64B Cross-sectional ultrasound image 64C Coronal Ultrasound Image 65,67 Blind Area 66 Schematic Images 66A MLO Schematic Image 66B CC schematic image 66C, 66C1, 66C2 compression schematic images 66D Ultrasound Schematic Image 68 Display Images 70 Pointer 72, 74, 76, 81, 81A, 81B, 82, 84, 84A, 84B, 86, 88 Mark 80 Cross section position information 90 Text Box 100 Medical Support System 101 Information processing device 102 External communication device 104 processors 106 Storage 108 RAM 110 Communication I / F 112 Bus 114 Network A. Subject M Breast CP coronal plane SP sagittal plane TP cross section

Claims

1. A processor is provided. The processor, an examination image including a radiographic image obtained by radiographing a breast compressed by a compression member and a plurality of ultrasound images obtained by compressing the breast by the compression member under a compression state that can be regarded as being the same as that of the radiographic image, the positions of the virtual cross sections being displayed in each of the ultrasound image and the radiographic image; a first schema image which is a schematic diagram showing the breast in the radiography, the first schema image being an image in which the position of the virtual cross section is displayed based on the ultrasound image; and Execute a display control to display a display image including the inspection image and the schematic image. Medical support equipment.

2. The schematic image further includes a second schematic image, The second schema image is an image showing a second schema, which is a schematic diagram corresponding to an ultrasound examination image of an examination method in which a probe is placed on the breast, and in which the position of the virtual cross section is displayed. The medical support device according to claim 1 .

3. When a region of interest is specified in at least one of the ultrasound image and the radiation image, the region of interest is displayed at a corresponding position in both the first schematic image and the second schematic image. The medical support device according to claim 2.

4. In the second schema image, the position of the virtual cross section is derived based on the position of the virtual cross section in the first schema and the depth of the virtual cross section in the compression direction in the ultrasound image. The medical support device according to claim 2.

5. The schematic image further includes a third schematic image, The third schema image is an image showing a third schema, which is a schematic diagram of the breast in the radiation imaging in a direction different from that of the first schema, and in which the position of the virtual cross section is displayed. The medical support device according to claim 2.

6. In the third schema image, the position of the virtual cross section is derived based on the position of the virtual cross section in the second schema and the distance of the virtual cross section in the ultrasound image from a body surface of the subject in the front-back direction. The medical support device according to claim 5.

7. the radiographic image includes a first radiographic image and a second radiographic image obtained in an imaging direction different from that of the first radiographic image, the first schematic image corresponds to the first radiation image; the third schematic image corresponds to the second radiation image; In the second schematic image, the position of the virtual cross section is derived based on the positions of the virtual cross section in the first schematic image and the third schematic image. The medical support device according to claim 5.

8. In the third schematic image, the position of the virtual cross section is displayed based on the ultrasonic image. The medical support device according to claim 7.

9. In the ultrasound image, a first blind area, which is an image area in which the breast is not imaged in an ultrasound examination, is displayed so as to be distinguishable from other areas. The medical support device according to claim 1 .

10. In the schematic image, a second blind area, which is an image area in which the breast is not imaged in an ultrasonic examination, is displayed so as to be distinguishable from other areas. The medical support device according to claim 1 .

11. the schematic image includes a fourth schematic image, The fourth schematic image is an image showing a fourth schematic diagram illustrating a state in which the compression member is compressing the breast, and is an image showing the position of the virtual cross section. The medical support device according to claim 1 .

12. In the fourth schematic image, the position of the virtual cross section is derived based on the position of the virtual cross section in the ultrasound image. The medical support device according to claim 11.

13. In the displayed image, information regarding a depth of an intersection of the virtual cross section in a compression direction of the breast is further displayed. The medical support device according to claim 1 .

14. When the virtual cross section displayed in the schematic image based on the position of the virtual cross section in the ultrasound image is a first virtual cross section, The display mode of the position of the first virtual cross section is a virtual cross section estimated based on the position of the first virtual cross section displayed in the schematic image, and is a display mode that can be distinguished from a second virtual cross section displayed in an image different from the image in which the first virtual cross section is displayed. The medical support device according to claim 1 .

15. When the position of the virtual cross section displayed on the schematic image is changed, the virtual cross section after the change is displayed in a manner that allows the virtual cross section before the change to be distinguished from the virtual cross section before the change. The medical support device according to claim 1 .

16. The medical support device according to claim 1 ; A radiation image capturing device; an ultrasound imaging device; A medical imaging system comprising:

17. an examination image including a radiographic image obtained by radiographing a breast compressed by a compression member and a plurality of ultrasound images obtained by compressing the breast by the compression member under a compression state that can be regarded as being the same as that of the radiographic image, the positions of the virtual cross sections being displayed in each of the ultrasound image and the radiographic image; a first schema image which is a schematic diagram showing the breast in the radiography, the first schema image being an image in which the position of the virtual cross section is displayed based on the ultrasound image; and executing display control to display a display image including the inspection image and the schematic image; A medical support method including:

18. On the computer, an examination image including a radiographic image obtained by radiographing a breast compressed by a compression member and a plurality of ultrasound images obtained by compressing the breast by the compression member under a compression state that can be regarded as being the same as that of the radiographic image, the positions of the virtual cross sections being displayed in each of the ultrasound image and the radiographic image; acquiring a first schema image, which is a schematic diagram showing the breast in the radiography, the first schema image being an image in which the position of the virtual cross section is displayed based on the ultrasound image; and executing display control to display a display image including the inspection image and the schematic image; A program that executes a process including:

Images