Image processing device, ultrasonic diagnostic device, and computer program product

Abstract

The object of the present invention is to improve the convenience of breast image diagnosis using ultrasonic images. An image processing device of an embodiment includes a calculation unit and a display control unit. The calculation unit calculates a first distance from the region of interest on the breast X-ray image to the body surface based on the region of interest set on the breast X-ray image. The display control unit displays a mark indicating the position of the region of interest in the depth direction in the ultrasonic image based on the first distance.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS:

[0002] This application is based upon and claims the benefit of priority from Japanese Patent Application No. 2020-009215, filed on January 23, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0003] Embodiments of the present invention generally relate to an image processing device, an ultrasonic diagnostic device, and an image processing program. Background Art

[0004] In recent years, in breast image diagnosis for breast cancer examination, ultrasound image examination is performed after breast X-ray image examination. In connection with this, a function has been developed to set (point) a region of interest at a desired position on a breast X-ray image and draw a line corresponding to the position of the region of interest on a body mark of the breast. This function assists the subsequent stage of ultrasound image examination by drawing a line corresponding to the position of the region of interest, so it is hereinafter referred to as an ultrasound diagnosis auxiliary function.

[0005] However, the body mark depicting the region of interest as described above does not indicate the position of the region of interest in the depth direction, so the operator needs to read the position of the region of interest in the depth direction from the mammographic X-ray image to estimate the position on the ultrasonic image.

[0006] Prior art document: Japanese Patent Application No. 2017-086896 Summary of the invention

[0007] An object of the present invention is to improve the convenience of breast image diagnosis using ultrasonic images.

[0008] The image processing device of the embodiment includes a calculation unit and a display control unit. The calculation unit calculates a first distance from the region of interest on the breast X-ray image to the body surface based on the region of interest set on the breast X-ray image. The display control unit displays a mark indicating the position of the region of interest in the depth direction in the ultrasonic image based on the first distance.

[0009] An effect of the present invention is that the convenience of breast image diagnosis using ultrasonic images can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a block diagram showing a configuration example of the ultrasonic diagnostic apparatus according to the first embodiment.

[0011] Figure 2 This is a flowchart for explaining an example of the operation of the marker display process in the first embodiment.

[0012] Figure 3 This is a diagram showing an example of a breast X-ray image in the first embodiment.

[0013] Figure 4 This is a diagram showing an example of an ultrasonic image in the first embodiment.

[0014] Figure 5 This is a first display example in which the marker display process in the first embodiment is executed.

[0015] Figure 6 This is a second display example in which the marker display process in the first embodiment is executed.

[0016] Figure 7 This is a flowchart for explaining an example of the operation of the marker display process in the second embodiment.

[0017] Figure 8 This is a diagram for explaining an example of the relationship between the position of the region of interest on the body mark and the three-dimensional marker in the second embodiment.

[0018] Fig. 9 This is a diagram showing an example of an ultrasonic image in the second embodiment.

[0019] Fig.10 This is a first display example in which the marker display process in the second embodiment is executed.

[0020] Fig.11 This is a second display example in which the marker display process in the second embodiment is executed.

[0021] Fig.12 It is a block diagram showing a configuration example of an image processing device according to the third embodiment.

[0022] Fig.13 This is a diagram for explaining an example of the relationship between the position of the region of interest on a mammographic X-ray image and the body mark.

[0023] Description of Reference Numerals

[0024] 1 ... ultrasonic diagnostic device, 180, 250 ... processing circuit, 184, 251 ... acquisition function, 185, 252 ... calculation function, 186, 253 ... display control function, 200 ... image processing device. DETAILED DESCRIPTION

[0025] The image processing device of the embodiment includes a calculation unit and a display control unit. The calculation unit calculates a first distance from the region of interest on the breast X-ray image to the body surface based on the region of interest set on the breast X-ray image. The display control unit displays a mark indicating the position of the region of interest in the depth direction in the ultrasonic image based on the first distance.

[0026] Below, while referring to the attached Figure 1 Embodiments of the ultrasonic diagnostic apparatus and the image processing apparatus will be described in detail.

[0027] (First Embodiment)

[0028] Figure 1 It is a diagram showing a configuration example of an ultrasonic diagnostic apparatus according to the first embodiment. Figure 1 The ultrasonic diagnostic apparatus 1 includes an apparatus body 100 and an ultrasonic probe 101. The apparatus body 100 is connected to an input device 102 and a display device 103. In addition, the apparatus body 100 is connected to an external device 104 via a network NW. The external device 104 is, for example, a server equipped with a PACS (Picture Archiving and Communication Systems: medical image transmission and storage system).

[0029] The ultrasonic probe 101 performs ultrasonic scanning on a scanning area in a living body P as a subject, for example, according to control from the device body 100. The ultrasonic probe 101 includes, for example, a plurality of piezoelectric transducers, a matching layer provided on the piezoelectric transducers, and a cushion material for preventing ultrasonic waves from propagating backward from the piezoelectric transducers. The ultrasonic probe 101 is, for example, a one-dimensional array linear probe in which a plurality of ultrasonic transducers are arranged in a predetermined direction. The ultrasonic probe 101 can be detachably connected to the device body 100. The ultrasonic probe 101 may also be provided with a button to be pressed during offset processing and a freeze operation for freezing an ultrasonic image.

[0030] The plurality of piezoelectric vibrators generate ultrasonic waves based on a driving signal supplied from an ultrasonic transmitting circuit 110 (described later) possessed by the device body 100. Thus, ultrasonic waves are transmitted from the ultrasonic probe 101 to the living body P. When ultrasonic waves are transmitted from the ultrasonic probe 101 to the living body P, the transmitted ultrasonic waves are reflected one by one on the discontinuous surfaces of the acoustic impedance in the internal tissue of the living body P, and are received by the plurality of piezoelectric elements as reflected wave signals. The amplitude of the received reflected wave signal depends on the difference in acoustic impedance on the discontinuous surface where the ultrasonic wave is reflected. In addition, when the transmitted ultrasonic pulse is reflected on a surface such as a moving blood flow or a heart wall, the reflected wave signal is subjected to a frequency shift depending on the speed component of the moving body in the ultrasonic transmission direction due to the Doppler effect. The ultrasonic probe 101 receives the reflected wave signal from the living body P and converts it into an electrical signal.

[0031] Figure 1 The connection relationship between one ultrasonic probe 101 and the device main body 100 is illustrated. However, a plurality of ultrasonic probes can be connected to the device main body 100. Any one of the plurality of connected ultrasonic probes can be arbitrarily selected for ultrasonic scanning by a switching operation.

[0032] The device body 100 is a device that generates an ultrasonic image based on a reflected wave signal received by an ultrasonic probe 101. The device body 100 includes an ultrasonic transmission circuit 110, an ultrasonic reception circuit 120, an internal storage circuit 130, an image memory 140, an input interface 150, an output interface 160, a communication interface 170, and a processing circuit 180.

[0033] The ultrasonic transmission circuit 110 is a processor that supplies a driving signal to the ultrasonic probe 101. The ultrasonic transmission circuit 110 is implemented, for example, by a trigger generation circuit, a delay circuit, and a pulse generator circuit. The trigger generation circuit repeatedly generates a rated pulse for forming a transmitted ultrasonic wave at a predetermined rated frequency. The delay circuit gives each rated pulse generated by the trigger generation circuit a delay time for each piezoelectric vibrator required to determine the transmission directivity in order to focus the ultrasonic wave generated from the ultrasonic probe into a beam shape. The pulse generator circuit applies a driving signal (driving pulse) to a plurality of ultrasonic vibrators provided in the ultrasonic probe 101 at a timing based on the rated pulse. By changing the delay time given to each rated pulse by the delay circuit, the transmission direction from the surface of the piezoelectric vibrator can be arbitrarily adjusted.

[0034] In addition, the ultrasonic transmission circuit 110 can arbitrarily change the output intensity of the ultrasonic wave by the driving signal. In the ultrasonic diagnostic apparatus, by increasing the output intensity, the influence of ultrasonic wave attenuation in the living body P can be reduced. By reducing the influence of ultrasonic wave attenuation, the ultrasonic diagnostic apparatus can obtain a reflected wave signal with a large S / N ratio during reception.

[0035] Generally speaking, when ultrasonic waves propagate in a biological body P, the intensity of ultrasonic vibration (also referred to as acoustic power) corresponding to the output intensity is attenuated. The attenuation of acoustic power is caused by absorption, scattering, and reflection. In addition, the degree of reduction of acoustic power depends on the frequency of the ultrasonic wave and the distance in the radiation direction of the ultrasonic wave. For example, by increasing the frequency of the ultrasonic wave, the degree of attenuation increases. In addition, the longer the distance in the radiation direction of the ultrasonic wave, the greater the degree of attenuation.

[0036] The ultrasonic receiving circuit 120 is a processor that performs various processing on the reflected wave signal received by the ultrasonic probe 101 and generates a received signal. The ultrasonic receiving circuit 120 generates a received signal corresponding to the reflected wave signal of the ultrasonic wave obtained by the ultrasonic probe 101. Specifically, the ultrasonic receiving circuit 120 is implemented by, for example, a preamplifier, an A / D converter, a demodulator, and a beamformer. The preamplifier amplifies the reflected wave signal received by the ultrasonic probe 101 for each channel and performs gain correction processing. The A / D converter converts the reflected wave signal after gain correction into a digital signal. The demodulator demodulates the digital signal. The beamformer, for example, gives the demodulated digital signal a delay time required to determine the receiving directivity, and adds a plurality of digital signals to which the delay time is given. Through the addition processing of the beamformer, a received signal in which the reflected component from the direction corresponding to the receiving directivity is strengthened is generated.

[0037] The internal storage circuit 130 has, for example, a storage medium that can be read by a processor, such as a magnetic storage medium, an optical storage medium, or a semiconductor memory. The internal storage circuit 130 stores a program for realizing ultrasonic transmission and reception, a program related to the identification display processing described later, and various data. The program and various data may also be stored in the internal storage circuit 130 in advance. In addition, the program and various data may also be stored in a non-transitory storage medium for distribution, read from the non-transitory storage medium and installed in the internal storage circuit 130. In addition, the internal storage circuit 130 stores B-mode image data and contrast image data generated in the processing circuit 180 according to the operation input via the input interface 150. The internal storage circuit 130 can also transmit the stored image data to the external device 104 via the communication interface 170.

[0038] In addition, the internal storage circuit 130 may also be a drive device for reading and writing various information between a CD-ROM drive, a DVD drive, and a portable storage medium such as a flash memory, etc. The internal storage circuit 130 can also write the stored data to the portable storage medium and store the data in the external device 104 via the portable storage medium.

[0039] The image memory 140 includes, for example, a storage medium that can be read by a processor, such as a magnetic storage medium, an optical storage medium, or a semiconductor memory. The image memory 140 stores image data corresponding to a plurality of frames before the freeze operation input via the input interface 150. The image data stored in the image memory 140 is displayed continuously (movie display), for example.

[0040] The internal storage circuit 130 and the image memory 140 are not necessarily implemented by separate storage devices. The internal storage circuit 130 and the image memory 140 may be implemented by a single storage device. In addition, the internal storage circuit 130 and the image memory 140 may be implemented by a plurality of storage devices.

[0041] The input interface 150 receives various instructions from the operator via the input device 102. The input device 102 is, for example, a mouse, a keyboard, a panel switch, a slider switch, a trackball, a rotary encoder, an operation panel, and a touch command screen (TCS). The input interface 150 is connected to the processing circuit 180, for example, via a bus, converts the operation instructions input from the operator into electrical signals, and outputs the electrical signals to the processing circuit 180. In addition, the input interface 150 is not limited to components connected to physical operating components such as a mouse and a keyboard. For example, a circuit that receives an electrical signal corresponding to an operation instruction input from an external input device that is separately provided from the ultrasonic diagnostic apparatus 1 and outputs the electrical signal to the processing circuit 180 is also included in the example of the input interface.

[0042] The output interface 160 is, for example, an interface for outputting an electrical signal from the processing circuit 180 to the display device 103. The display device 103 is any display such as a liquid crystal display, an organic EL display, an LED display, a plasma display, a CRT display, etc. The display device 103 may also be a touch panel type display that also serves as the input device 102. The output interface 160 is connected to the processing circuit 180 via a bus, for example, and outputs the electrical signal from the processing circuit 180 to the display device 103.

[0043] The communication interface 170 is connected to the external device 104 via the network NW, for example, and performs data communication with the external device 104 .

[0044] The processing circuit 180 is, for example, a processor that functions as the core of the ultrasonic diagnostic apparatus 1. The processing circuit 180 implements functions corresponding to a program stored in the internal storage circuit 130 by executing the program. The processing circuit 180 includes, for example, a B-mode processing function 181, a Doppler processing function 182, an image generation function 183 (image generation unit), an acquisition function 184 (acquisition unit), a calculation function 185 (calculation unit), a display control function 186 (display control unit), and a system control function 187.

[0045] The B-mode processing function 181 is a function for generating B-mode data based on the reception signal received from the ultrasonic reception circuit 120. In the B-mode processing function 181, the processing circuit 180 performs envelope detection processing and logarithmic compression processing on the reception signal received from the ultrasonic reception circuit 120, and generates data (B-mode data) in which the signal intensity is expressed as brightness. The generated B-mode data is stored in a RAW data memory (not shown) as B-mode RAW data on a two-dimensional ultrasonic scanning line (raster).

[0046] Furthermore, the processing circuit 180 can perform contrast echo method, such as contrast harmonic imaging (CHI) by means of the B-mode processing function 181. That is, the processing circuit 180 can separate the reflected wave data (harmonic component or frequency-divided component) of the living body P injected with contrast agent and the reflected wave data (fundamental wave component) using the tissue in the living body P as a reflection source. Thus, the processing circuit 180 can extract the harmonic component or frequency-divided component from the reflected wave data of the living body P and generate B-mode data for generating contrast image data.

[0047] The B-mode data for generating contrast image data is data that expresses the signal intensity of the reflected wave with the contrast agent as the reflection source in terms of brightness. The processing circuit 180 can also extract the fundamental wave component from the reflected wave data of the living body P to generate B-mode data for generating tissue image data.

[0048] In addition, when performing CHI, the processing circuit 180 can extract the harmonic component by a method different from the method using the above-mentioned filtering process. In harmonic imaging, an imaging method called AM method, AM method, or a combination of AM method and PM method is performed.

[0049] In the AM method, the PM method, and the AMPM method, ultrasonic waves with different amplitudes and phases are transmitted multiple times (multiple levels) on the same scan line. As a result, the ultrasonic receiving circuit 120 generates and outputs multiple reflected wave data on each scan line. In addition, the processing circuit 180 extracts harmonic components by performing addition and subtraction processing corresponding to the modulation method on the multiple reflected wave data of each scan line. In addition, the processing circuit 180 performs envelope detection processing on the reflected wave data of the harmonic component to generate B-mode data.

[0050] For example, when the PM method is performed, the ultrasonic transmission circuit 110 transmits ultrasonic waves of the same amplitude with the phase polarity reversed twice on each scanning line, for example, (-1, 1), through the scanning sequence set by the processing circuit 180. In addition, the ultrasonic reception circuit 120 generates reflected wave data corresponding to the transmission of "-1" and reflected wave data corresponding to the transmission of "1", and the processing circuit 180 adds these two reflected wave data. As a result, the fundamental wave component is removed, and a signal mainly containing the second harmonic component is generated. In addition, the processing circuit 180 performs envelope detection processing on the signal, etc., to generate B-mode data of CHI (B-mode data for generating contrast image data).

[0051] The B-mode data of CHI is data that represents the signal intensity of the reflected wave with the contrast agent as the reflection source in terms of brightness. In addition, when the PM method is performed using CHI, the processing circuit 180 can generate B-mode data for generating tissue image data by, for example, filtering the reflected wave data corresponding to the transmission of "1".

[0052] The Doppler processing function 182 is a function for generating data (Doppler information) from which motion information based on the Doppler effect of a moving body located in a ROI (Region Of Interest) set in a scanning area is extracted by frequency analysis of a reception signal received from the ultrasonic reception circuit 120. The generated Doppler information is stored in a RAW data memory (not shown) as Doppler RAW data on a two-dimensional ultrasonic scanning line.

[0053] The image generation function 183 is a function for generating B-mode image data based on the data generated by the B-mode processing function 181. For example, in the image generation function 183, the processing circuit 180 converts (scan converts) the scan line signal string of the ultrasonic scan into a scan line signal string of a video format represented by a television, etc., and generates image data for display (display image data). Specifically, the processing circuit 180 generates two-dimensional B-mode image data (also referred to as ultrasonic image data) composed of pixels by performing RAW-pixel conversion on the B-mode RAW data stored in the RAW data memory, for example, performing coordinate conversion corresponding to the ultrasonic scanning method performed by the ultrasonic probe 101. In other words, the processing circuit 180 uses the image generation function 183 to generate a plurality of ultrasonic images (medical images) corresponding to a plurality of consecutive frames by transmitting and receiving ultrasonic waves.

[0054] Furthermore, the processing circuit 180 converts the image data into a video signal by performing various processes such as dynamic range, brightness, contrast, and gamma curve correction, and RGB conversion on the two-dimensional B-mode image data. The processing circuit 180 displays the video signal on the display device 103. In addition, the processing circuit 180 may also generate a user interface (GUI: Graphical User Interface) for the operator to input various instructions through the input device, and display the generated GUI on the display device 103.

[0055] The system control function 187 is a function for centrally controlling the operation of the entire ultrasonic diagnostic apparatus 1. For example, in the system control function 187, the processing circuit 180 controls the ultrasonic transmission circuit 110 and the ultrasonic reception circuit 120 based on parameters related to the transmission and reception of ultrasonic waves. The acquisition function 184, the calculation function 185, and the display control function 186 will be described later.

[0056] The basic structure of the ultrasonic diagnostic apparatus 1 according to the first embodiment is described above. With such a structure, the ultrasonic diagnostic apparatus 1 according to the first embodiment can reflect the lesion position calculated from the breast X-ray image on the ultrasonic image through the functions of the processing circuit described below.

[0057] In the processing circuit 180, the acquisition function 184 is a function of acquiring a breast X-ray image. Specifically, the processing circuit 180 acquires, for example, a breast X-ray image of a target patient stored in a PACS. At this time, the processing circuit 180 acquires at least one of a breast X-ray image captured in the CC direction and a breast X-ray image captured in the MLO direction for the target patient. In this embodiment, it is assumed that a breast X-ray image captured in the CC direction is acquired.

[0058] The breast X-ray image in this embodiment is, for example, an image in which a schematic diagram schematically showing a breast (hereinafter referred to as a body mark) and a region of interest are respectively associated. The body mark and the region of interest are displayed superimposed on the breast X-ray image. In addition, the expression "the breast X-ray image is associated with the region of interest" can be replaced by "the region of interest is set on the breast X-ray image".

[0059] Fig.13 This is a diagram for explaining an example of the relationship between the position of the region of interest on a mammographic X-ray image and the body mark. Fig.13 The body mark BM represents the right breast, and has a circular area representing the area of ​​the breast (hereinafter referred to as the breast area), and a substantially triangular area representing the area of ​​the armpit (hereinafter referred to as the armpit area). In addition, in the body mark, the direction from left to right is defined as the X direction, the direction from bottom to top is defined as the Y direction, and the direction perpendicular to the X direction and the Y direction and from the front to the back is defined as the Z direction.

[0060] On the body mark BM, a straight line L1 and a straight line L2 are superimposed due to the ultrasonic diagnosis auxiliary function. The straight line L1 indicates the shooting direction based on the lesion position LP1 on the breast X-ray image MG1 shot in the CC direction. The straight line L2 indicates the shooting direction based on the lesion position LP2 on the breast X-ray image MG2 shot in the MLO direction. Therefore, the intersection of the straight line L1 and the straight line L2 is the lesion position on the body mark BM estimated from the breast X-ray image MG1 and the breast X-ray image MG2.

[0061] In addition, the angle of the arm supporting the X-ray tube and the X-ray detector in the mammographic X-ray imaging device (arm angle) corresponds to the imaging direction, that is, the straight lines L1 and L2. Therefore, if the arm angle in the CC direction is set to 0 degrees, the arm angle in the MLO direction is the angle AN from the straight line L1 to the straight line L2. In addition, the arm angle can also be associated with the mammographic X-ray image.

[0062] In the processing circuit 180, the calculation function 185 is a function of calculating a first distance from the region of interest on the breast X-ray image to the body surface, and calculating a second distance from the body surface to the region of interest on the ultrasonic image. The processing circuit 180 calculates the first distance from the region of interest on the breast X-ray image to the body surface based on the region of interest set on the breast X-ray image. The first distance is calculated as an actual distance using the scale of the breast X-ray image for a straight line set on the image by the user or through image processing, for example.

[0063] For example, there is a case where the distance from the region of interest to the body surface in the CC direction and the MLO direction is inconsistent. This is caused by the inconsistency of the direction of the straight line extending from the region of interest to the body surface. In order to eliminate this inconsistency, the processing circuit 180 can also use the arm angle to infer the direction from the region of interest to the body surface on the breast X-ray image. In addition, the processing circuit 180 can also calculate the first distance based on the inferred direction.

[0064] Furthermore, the processing circuit 180 calculates a second distance from the body surface to the region of interest on the ultrasonic image based on the calculated first distance. The second distance is calculated, for example, using an operation formula that can convert the distance on the breast X-ray image and the distance on the ultrasonic image, and the operation formula takes into account the shape change of the examination part in the examination of the breast X-ray image and the shape change of the examination part in the examination of the ultrasonic image. In other words, the processing circuit 180 may also calculate the second distance using the deformation inference of the breast, which is based on the difference in body position between the examination of the breast X-ray image and the examination of the ultrasonic image.

[0065] In the processing circuit 180, the display control function 186 is a function of displaying an indicator indicating the depth of the lesion on the ultrasonic image based on the calculated distance. The processing circuit 180 displays the indicator indicating the depth of the lesion in the ultrasonic image based on the calculated first distance. More specifically, the processing circuit 180 displays the indicator indicating the depth of the lesion in an overlapping manner on the ultrasonic image using the calculated second distance.

[0066] As a method of displaying a marker on an ultrasonic image, for example, there is a method of overlapping and displaying an overlay image including the marker on the ultrasonic image (superimposed display). However, this method is not limited to the above method, and a method of directly drawing the marker in the ultrasonic image may also be used. In addition, the expression "displaying the marker on the ultrasonic image" may be replaced with "displaying the marker in the ultrasonic image", and either expression includes the meaning of the above two methods.

[0067] Furthermore, the processing circuit 180 displays the display image data generated by the image generation function 183 on the display device 103. Specifically, the processing circuit 180 may display the display image data as it is on the display device 103, or may display the display image data in parallel with or superimposed on the prescribed medical image data on the display device 103. More specifically, the processing circuit 180 may divide the display area into two and display the breast X-ray image and the ultrasonic image in parallel. In addition, the expression "displaying two images by dividing the display screen into two" may be replaced by "displaying two images on a dual screen".

[0068] Figure 21 is a flowchart for explaining an example of the operation of the marker display process in the first embodiment. The marker display process here refers to a process of displaying a lesion depth marker on an ultrasonic image. Figure 2 The processing shown is started, for example, by receiving an instruction from an operator to execute the mark display processing.

[0069] (Step ST110)

[0070] When the marker display process starts, the processing circuit 180 executes the acquisition function 184. When the acquisition function 184 is executed, the processing circuit 180 acquires a breast X-ray image. Specifically, the processing circuit 180 acquires a breast X-ray image in the CC direction from the PACS.

[0071] (Step ST120)

[0072] After acquiring the breast X-ray image, the processing circuit 180 executes the calculation function 185. When the calculation function 185 is executed, the processing circuit 180 calculates a first distance from the region of interest on the breast X-ray image to the body surface based on the region of interest set on the acquired breast X-ray image.

[0073] Figure 3 This is a diagram showing an example of a breast X-ray image in the first embodiment. Figure 3 The breast X-ray image MG10 shows the right breast with the CC direction as the shooting direction. The body mark BM1 and the region of interest LP indicating the position of the lesion are superimposed on the breast X-ray image MG10. The intersection of the two straight lines indicated by the body mark BM1 corresponds to the above-mentioned lesion position. In other words, the body mark BM1 indicates the position of the region of interest LP on the breast X-ray image MG10 in the plane orthogonal to the depth direction. In addition, in the breast X-ray image, the direction from right to left is defined as X-ray direction based on the upper right side of the breast X-ray image. MG Direction, define the direction from top to bottom as Y MG direction.

[0074] On the breast X-ray image MG10, the processing circuit 180 calculates the X-ray image from the region of interest LP. MG The position of the body surface on the mammary X-ray image MG10 is determined by detecting changes in brightness values, for example.

[0075] In this way, the processing circuit 180 calculates the first distance from the breast X-ray image. The processing performed by the calculation function 185 can be executed as an internal processing of the processing circuit 180 and may not be displayed on the display device 103.

[0076] (Step ST130)

[0077] After calculating the first distance, the processing circuit 180 calculates the second distance from the body surface to the region of interest on the ultrasonic image based on the calculated first distance. Specifically, the processing circuit 180 uses an operation formula that can convert the distance on the breast X-ray image and the distance on the ultrasonic image to calculate the second distance based on the first distance. In addition, this step is not necessary and can be omitted.

[0078] (Step ST140)

[0079] After calculating the second distance, the processing circuit 180 executes the display control function 186. If the display control function 186 is executed, the processing circuit 180 uses the calculated second distance to display an indicator (lesion depth indicator) indicating the lesion depth on the ultrasonic image. After step ST140, the indicator display process ends. In addition, the "lesion depth" can also be replaced with the "depth of the region of interest".

[0080] In addition, when step ST130 is omitted, the processing circuit 180 may also display an indicator indicating the depth of the lesion on the ultrasonic image based on the first distance. That is, regardless of whether step ST130 is omitted, the processing circuit 180 may display an indicator indicating the depth of the lesion on the ultrasonic image based on the first distance.

[0081] Figure 4 FIG. 1 is a diagram showing an example of an ultrasonic image in the first embodiment. Figure 4 A body mark BMr indicating the right breast is superimposed on the ultrasonic image US10. A probe position PP that establishes a correspondence between the position of the ultrasonic probe on the body surface and the position on the body mark BMr is superimposed on the body mark BMr. In addition, in the ultrasonic image, the direction from left to right is defined as the X direction, the direction from top to bottom is defined as the Z direction, and the direction from the front to the back, which is perpendicular to the X direction and the Z direction, is defined as the Y direction.

[0082] On the ultrasonic image US10, the processing circuit 180 sets a straight line L10 indicating a body surface position. After setting the straight line L10, the processing circuit 180 sets a straight line L11 parallel to the straight line L10 at a distance of a second distance D2 from the straight line L10 in the +Z direction. After setting the straight line L11, the processing circuit 180 sets a straight line L12 parallel to the straight line L11 at a distance of a distance d1 from the straight line L11 in the -Z direction. Similarly, the processing circuit 180 sets a straight line L13 parallel to the straight line L11 at a distance of a distance d1 from the straight line L11 in the +Z direction. The distance d1 may be, for example, a distance corresponding to the size of the region of interest set in the breast X-ray image, or may be an arbitrarily set distance.

[0083] Figure 5 This is a first display example in which the marker display process in the first embodiment is executed. Figure 5 The display area 10 includes an ultrasonic image US10. Figure 4 The straight line markers M1 and M2 correspond to the straight line L10 and the straight line L13 in the display area 10. The aforementioned lesion depth markers are equivalent to the straight line markers M1 and M2. The area sandwiched by the straight line markers M1 and M2 shows the depth of the lesion. This makes it easy for the user to find the lesion location. In addition, in the display area 10, a body marker BMr may be displayed instead of the body marker BMr. Figure 3 The body mark BM1 or the body mark BMr is displayed in parallel with the body mark Figure 3 The body is labeled BM1.

[0084] Figure 6 This is a second display example in which the marker display process in the first embodiment is executed. Figure 6 The display area 20 includes the breast X-ray image MG10 and the ultrasonic image US10. Specifically, the display area 20 is configured with the breast X-ray image MG10 on the left side of the dual screen display and the ultrasonic image US10 on the right side. Thus, the user can also find the lesion position by referring to the body mark BM1 on the breast X-ray image MG10.

[0085] As described above, the ultrasonic diagnostic apparatus of the first embodiment calculates the first distance from the region of interest on the breast X-ray image to the body surface based on the region of interest set on the breast X-ray image, and displays a marker indicating the position of the region of interest in the depth direction in the ultrasonic image based on the first distance. Thus, the user can easily confirm the lesion in the ultrasonic image without the trouble of reading the depth position from the breast X-ray image.

[0086] In addition, in the first embodiment, an example of a straight line mark is shown on the ultrasound image, but it is not limited to this. For example, a rectangle or a circle may be used as a mark. For example, a circular mark is Figure 4 A circle with a radius of d1 and an arbitrary point as the center is formed on an arbitrary point on the straight line L11.

[0087] (Second Embodiment)

[0088] In the first embodiment, a case where a lesion depth mark is displayed on an ultrasonic image by a mark display process is described. On the other hand, in the second embodiment, a case where a mark indicating a lesion position (lesion position mark) is displayed on an ultrasonic image by a mark display process is described. In addition, the "lesion position" may be replaced by the "position of the region of interest".

[0089] In the ultrasound diagnostic apparatus of the second embodiment, the mammary X-ray image and the ultrasound image are aligned. Alignment of the mammary X-ray image and the ultrasound image means that the coordinates of the region of interest on the mammary X-ray image correspond to the spatial coordinates of the ultrasound probe.

[0090] Specifically, the ultrasonic diagnostic apparatus of the second embodiment has a magnetic transmitter in the apparatus body and a magnetic sensor in the ultrasonic probe. The magnetic transmitter generates a pulse magnetic field. The magnetic sensor detects the position and angle of the ultrasonic probe based on the predetermined spatial coordinates of the space where the pulse magnetic field is generated. Thus, the breast X-ray image and the ultrasonic image are aligned in the ultrasonic diagnostic apparatus of the second embodiment.

[0091] Figure 7 1 is a flowchart for explaining an example of the operation of the marker display processing in the second embodiment. The marker display processing here refers to the processing of displaying the lesion position marker on the ultrasonic image. Figure 7 The processing shown is started, for example, by receiving an instruction from an operator to execute the mark display processing.

[0092] (Step ST210)

[0093] When the marker display process starts, the processing circuit 180 executes the acquisition function 184. When the acquisition function 184 is executed, the processing circuit 180 acquires the breast X-ray image in the CC direction.

[0094] (Step ST220)

[0095] After obtaining the breast X-ray image in the CC direction, the processing circuit 180 obtains the breast X-ray image in the MLO direction. In addition, step ST210 and step ST220 may be performed simultaneously, or two breast X-ray images may be taken from two different directions.

[0096] (Step ST230)

[0097] After acquiring the breast X-ray image in the MLO direction, the processing circuit 180 executes the calculation function 185. When the calculation function 185 is executed, the processing circuit 180 calculates the first distance from the region of interest on the breast X-ray image to the body surface based on the region of interest set on the acquired breast X-ray image. The processing circuit 180 calculates the first distance using at least one of the breast X-ray image in the CC direction and the breast X-ray image in the MLO direction. In addition, the processing of step ST230 is the same as the processing of the aforementioned step ST120, so the detailed description is omitted.

[0098] (Step ST240)

[0099] After calculating the first distance, the processing circuit 180 calculates a second distance from the body surface to the region of interest on the ultrasound image based on the calculated first distance. Specifically, the processing circuit 180 calculates the second distance based on the first distance using an operation formula that can convert the distance on the breast X-ray image to the distance on the ultrasound image.

[0100] (Step ST250)

[0101] After calculating the second distance, the processing circuit 180 calculates three-dimensional position information on the ultrasonic image based on the calculated second distance, the position information of the region of interest set on the breast X-ray image in the CC direction, and the position information of the region of interest set on the breast X-ray image in the MLO direction.

[0102] Specifically, the processing circuit 180 calculates the position on the two-dimensional plane corresponding to the region of interest on the body mark BM1 based on the position information of the region of interest set on the breast X-ray image in the CC direction and the position information of the region of interest set on the breast X-ray image in the MLO direction. After calculating the position on the two-dimensional plane, the processing circuit 180 calculates the position in the three-dimensional space based on the position on the two-dimensional plane and the calculated second distance.

[0103] Figure 8 FIG. 1 is a diagram for explaining an example of the relationship between the position of the region of interest on the body mark and the three-dimensional marker in the second embodiment. Figure 8 When the body marker BM1 simulates the body surface, the +Z direction corresponds to the depth direction in the body. The above-mentioned position on the two-dimensional plane is, for example, the position (x, y) of the region of interest on the body marker BM1. The above-mentioned position in the three-dimensional space is, for example, the center position (x, y, z) of the three-dimensional marker MC.

[0104] (Step ST260)

[0105] After calculating the three-dimensional position information, the processing circuit 180 executes the display control function 186. If the display control function 186 is executed, the processing circuit 180 displays the lesion position mark corresponding to the probe position information on the ultrasonic image based on the calculated three-dimensional position information. After step S260, the mark display process ends.

[0106] Fig. 9 FIG. 2 is a diagram showing an example of an ultrasonic image in the second embodiment. Fig. 9 A body mark BMr is superimposed on the ultrasonic image US20. A probe position PP is superimposed on the body mark BMr.

[0107] On the ultrasonic image US20, the processing circuit 180 sets a straight line L10 indicating the body surface position. After setting the straight line L10, the processing circuit 180 sets a depth position P at a distance of a second distance D2 from the straight line L10 in the +Z direction. The depth position P corresponds to the position z of the three-dimensional marker MC. After setting the depth position P, the processing circuit 180 sets a circle CL of a radius d3 centered at the depth position P. The circle CL corresponds to the circumference of an arbitrary cross section of the three-dimensional marker MC.

[0108] Fig.10 This is a first display example in which the marker display process in the second embodiment is executed. Fig.10 The display area 30 includes an ultrasonic image US20. Fig. 9 The circular mark Mc corresponds to the circle CL in the display area 30. The aforementioned lesion position mark is equivalent to the circular mark Mc. The area surrounded by the circular mark Mc shows the lesion position. In addition, in the display area 30, a body mark BMr may be displayed instead. Figure 3 The body mark BM1 or the body mark BMr is displayed in parallel with the body mark Figure 3 The body is labeled BM1.

[0109] The size of the circular mark Mc may also be changed, for example, according to the position of the ultrasonic probe. Specifically, the circular mark Mc is located at the probe position PP and the lesion position on the body mark BMr (ie, Figure 3 When the lesion position overlaps with the probe position PP, the size becomes the largest, and the size becomes smaller as the lesion position and the probe position PP overlap and separate. In addition, the largest circular mark can be displayed all the time as a guide.

[0110] In addition, the color of the circular marker Mc may change according to the position of the ultrasonic probe, for example. Specifically, the circular marker Mc becomes green when the probe position PP overlaps the lesion position, and changes color from green to yellow or red as the lesion position and the probe position PP overlap and separate.

[0111] In other words, at least one of the size and color of the circular mark Mc changes according to the position of the ultrasonic probe. This makes it easier for the user to find the lesion position than when only the depth position of the lesion position is shown.

[0112] Fig.11 This is a second display example in which the marker display process in the second embodiment is executed. Fig.11The display area 40 includes the breast X-ray image MG10 and the ultrasonic image US20. Specifically, the display area 40 is configured with the breast X-ray image MG10 on the left side of the dual screen display and the ultrasonic image US20 on the right side. Thus, the user can also find the lesion position by referring to the body mark BM1 on the breast X-ray image MG10.

[0113] As described above, the ultrasonic diagnostic apparatus of the second embodiment acquires two breast X-ray images taken from two different directions, calculates a first distance from the region of interest on the breast X-ray image to the body surface based on the region of interest set on the breast X-ray image, calculates a second distance from the body surface to the region of interest on the ultrasonic image based on the first distance, calculates three-dimensional position information on the ultrasonic image based on the second distance and position information of each region of interest set on the two breast X-ray images, and displays a marker on the ultrasonic image based on the three-dimensional position information. Thus, the user can easily confirm a lesion in the ultrasonic image without the trouble of reading the depth position from the breast X-ray image.

[0114] In the second embodiment, an example in which a circular marker is displayed on an ultrasonic image is shown, but the present invention is not limited thereto. For example, the marker may be rectangular or may be in a shape corresponding to the color Doppler rendering area.

[0115] (Application example)

[0116] In the first embodiment and the second embodiment, the processing of displaying a marker on an ultrasonic image is described. On the other hand, in the application example, the case of controlling the transmission and reception of ultrasonic waves according to the display position of the marker is described.

[0117] The processing circuit 180 controls the ultrasonic transmission and reception using the parameters related to the ultrasonic transmission and reception corresponding to the display position of the marker through the system control function 187. The control of the ultrasonic transmission and reception is, for example, the control of the ultrasonic transmission circuit 110 and the ultrasonic reception circuit 120. Specifically, the processing circuit 180 controls the ultrasonic transmission and reception using the focus position corresponding to the display position of the marker.

[0118] According to this application example, since the transmission and reception of ultrasonic waves are controlled according to the display position of the marker, the user does not need to set parameters related to the transmission and reception of ultrasonic waves, and can perform an inspection using optimal parameters.

[0119] (Third Embodiment)

[0120] In the first and second embodiments, the marker display process executed by the processing circuit included in the ultrasonic diagnostic apparatus is described. On the other hand, in the third embodiment, the marker display process executed by the processing circuit included in the image processing apparatus is described.

[0121] Fig.12 It is a diagram showing a configuration example of an image processing device according to a third embodiment. Fig.13 The image processing device 200 is connected to the input device 201 and the display device 202. In addition, the image processing device 200 is connected to the external device 104 via the network NW. In addition, the input device 201 and the display device 202 are connected to the external device 104 via the network NW. Figure 1 The input device 102 and the display device 103 are substantially the same, and thus description thereof will be omitted.

[0122] The image processing device 200 includes an internal storage circuit 210 , an input interface 220 , an output interface 230 , a communication interface 240 , and a processing circuit 250 .

[0123] The internal storage circuit 210 includes, for example, a storage medium that can be read by a processor, such as a magnetic storage medium, an optical storage medium, or a semiconductor memory, etc. The internal storage circuit 210 stores programs related to the mark display process, etc.

[0124] The input interface 220 receives various instructions from the operator via the input device 201. The input interface 220 is connected to the processing circuit 250 via a bus, for example, and converts the operation instructions input from the operator into electrical signals, and outputs the electrical signals to the processing circuit 250.

[0125] The output interface 230 is, for example, an interface for outputting an electrical signal from the processing circuit 250 to the display device 202. The output interface 230 is connected to the processing circuit 250 via a bus, for example, and outputs the electrical signal from the processing circuit 250 to the display device 202.

[0126] The communication interface 240 is connected to the external device 104 via the network NW, for example, and performs data communication with the external device 104 .

[0127] The processing circuit 250 is, for example, a processor that functions as the core of the image processing device 200. The processing circuit 250 implements a function corresponding to a program for the mark display process stored in the internal storage circuit 210 by executing the program. The processing circuit 250 has, for example, an acquisition function 251, a calculation function 252, a display control function 253, and a system control function 254.

[0128] The system control function 254 is a function for centrally controlling the overall operation of the image processing apparatus 200. The acquisition function 251, the calculation function 252, and the display control function 253 are substantially the same as the acquisition function 184, the calculation function 185, and the display control function 186 described in the first and second embodiments, and thus description thereof is omitted.

[0129] As described above, the image processing device of the third embodiment can be expected to achieve the same effects as those of the first embodiment and the second embodiment.

[0130] The image processing device of this embodiment corresponds to a device obtained by extracting a part of the ultrasonic diagnostic device of the first embodiment and the second embodiment. In other words, the ultrasonic diagnostic device corresponds to a device obtained by adding an ultrasonic diagnostic configuration to the configuration of the image processing device.

[0131] According to at least one of the embodiments described above, the convenience of breast image diagnosis using ultrasonic images can be improved.

[0132] Although several embodiments have been described, these embodiments are shown as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other ways, and various omissions, substitutions, changes, and combinations of embodiments can be performed without departing from the scope of the subject matter of the invention. These embodiments and their variations are included in the scope and subject matter of the invention, and are also included in the invention described in the claims and their equivalents.

Claims

1. An image processing device, wherein: have: a calculation unit that calculates a first distance from the region of interest on the breast X-ray image to the body surface based on the region of interest set on the breast X-ray image; and a display control unit that displays a marker corresponding to a position in a depth direction of the region of interest on the ultrasonic image based on the first distance; The markers include a first straight line marker and a second straight line marker indicating the depth or position of the region of interest on the ultrasound image, The first straight line marker and the second straight line marker are parallel, The display control unit performs display so that a region sandwiched between the first straight line marker and the second straight line marker on the ultrasonic image indicates a position of a lesion.

2. The image processing device according to claim 1, wherein: The display control unit displays a body mark indicating a position of the region of interest on the mammary X-ray image in a plane orthogonal to the depth direction together with the ultrasonic image.

3. The image processing apparatus according to claim 1, wherein: The calculation unit calculates a second distance from the body surface to the region of interest on the ultrasonic image based on the first distance. The display control unit displays the marker in the ultrasonic image using the second distance.

4. The image processing apparatus according to claim 3, wherein: The calculation unit calculates the second distance using an estimation of deformation of the breast based on a difference in body position between an examination of the breast X-ray image and an examination of the ultrasonic image.

5. The image processing apparatus according to claim 3, wherein: The calculation unit calculates three-dimensional position information on the ultrasonic image based on the second distance and position information of each region of interest set on two breast X-ray images taken from two different directions, The display control unit displays the marker on the ultrasonic image based on the three-dimensional position information.

6. The image processing apparatus according to claim 1, wherein: The display control unit displays the breast X-ray image and the ultrasonic image on which the marker is superimposed in parallel.

7. The image processing apparatus according to claim 1, wherein: The display control unit displays the marker according to the size of the region of interest.

8. The image processing apparatus according to any one of claims 1 to 7, wherein: The calculation unit estimates a direction from a region of interest on the breast X-ray image to a body surface using an arm angle associated with the breast X-ray image, and calculates the first distance based on the estimated direction.

9. The image processing apparatus according to claim 1, wherein: the display control unit displays a body mark together with the ultrasonic image, the body mark indicating a position of the region of interest on the mammary X-ray image in a plane orthogonal to the depth direction, Furthermore, the display control unit divides the display area into two to display the breast X-ray image and the ultrasonic image side by side.

10. An ultrasonic diagnostic device, wherein: have: The image processing device according to any one of claims 1 to 9; and An ultrasonic probe for obtaining the ultrasonic image.

11. The ultrasonic diagnostic apparatus according to claim 10, wherein: The device further includes a control unit configured to control the ultrasonic transmission and reception using a parameter related to the ultrasonic transmission and reception according to the display position of the marker.

12. The ultrasonic diagnostic apparatus according to claim 11, wherein: The parameters are the focus position, The control unit controls the transmission and reception of the ultrasonic wave using the focus position corresponding to the display position of the marker.

13. The ultrasonic diagnostic apparatus according to any one of claims 10 to 12, wherein: The display control unit changes at least one of a size and a color of the mark according to a position of the ultrasonic probe.

14. A computer program product comprising an image processing program, wherein: When the image processing program is executed by the processor, the following steps are implemented: The first step is to calculate a first distance from the region of interest on the breast X-ray image to the body surface based on the region of interest set on the breast X-ray image; as well as In a second step, based on the first distance, a mark corresponding to the position of the region of interest in the depth direction is displayed on the ultrasonic image. The markers include a first straight line marker and a second straight line marker indicating the depth or position of the region of interest on the ultrasound image, The first straight line marker and the second straight line marker are parallel, In the second step, display is performed so that the area sandwiched by the first straight line marker and the second straight line marker on the ultrasonic image shows the position of the lesion.

Images