Ultrasound image processing device, ultrasound diagnostic device, and ultrasound image processing program

Abstract

The purpose of this disclosure is to facilitate the observation of a subject using an ultrasound diagnostic device that employs multiple ultrasound probes. [Solution] The processor in the ultrasonic image processing device generates ultrasonic image data corresponding to each ultrasonic probe 14a and 14b based on the received signals output from each ultrasonic probe 14a and 14b, acquires the position and orientation information of each ultrasonic probe 14a and 14b, and generates display image data showing an ultrasonic image from one of the ultrasonic probes 14a and 14b and a probe indicator representing the ultrasonic transmission range of the other ultrasonic probe, based on the ultrasonic image data corresponding to each ultrasonic probe 14a and 14b and the position and orientation information of each ultrasonic probe 14a and 14b.

Description

Technical Field

[0001] The present disclosure relates to an ultrasonic image processing apparatus, an ultrasonic diagnostic apparatus, and an ultrasonic image processing program, and more particularly to an apparatus and a program for generating ultrasonic image data corresponding to each ultrasonic probe using a plurality of ultrasonic probes.

Background Art

[0002] There is an ultrasonic diagnostic apparatus that can observe a subject from different directions using two or more ultrasonic probes. For example, it has been proposed to perform prostate surgery while observing two B-mode images obtained from the rectum and the body surface using a transrectal probe and an intraoperative probe with such an ultrasonic diagnostic apparatus.

[0003] Note that Patent Document 1 below shows an ultrasonic diagnostic apparatus that detects the position of an ultrasonic probe as an apparatus related to the present disclosure described later.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] When observing a subject from different directions, if only two or more ultrasonic images acquired by two or more ultrasonic probes are displayed, there arises a problem that it is difficult to grasp the positional relationship of the cross-sections where B-mode images are observed using each ultrasonic probe.

[0006] An object of the present disclosure is to facilitate the observation of a subject by an ultrasonic diagnostic apparatus using a plurality of ultrasonic probes.

Means for Solving the Problems

[0007] This disclosure relates to an ultrasonic image processing apparatus comprising a processor, wherein the processor generates ultrasonic image data corresponding to each ultrasonic probe based on a received signal output from each of a plurality of ultrasonic probes, acquires position and orientation information for each of the plurality of ultrasonic probes, and generates display image data showing an ultrasonic image from one of the plurality of ultrasonic probes and a probe indicator representing the ultrasonic transmission range of the other ultrasonic probes based on the ultrasonic image data and position and orientation information corresponding to each ultrasonic probe.

[0008] In one embodiment, the probe indicator indicates the region where the scanning range of the ultrasonic beam formed by a first ultrasonic probe, which is one of the plurality of ultrasonic probes, and the scanning range of the ultrasonic beam formed by a second ultrasonic probe, which is another of the ultrasonic probes, intersect.

[0009] In one embodiment, the probe indicator includes an intersection line between the scanning range of the ultrasonic beam formed by the first ultrasonic probe and the scanning range of the ultrasonic beam formed by the second ultrasonic probe, and a splice line extending from the intersection line, wherein the splice line extends in accordance with the projection image of the scanning range of the ultrasonic beam formed by the second ultrasonic probe onto the ultrasonic image by the first ultrasonic probe, in the direction of the ultrasonic beam formed by the second ultrasonic probe.

[0010] In one embodiment, the display image data is image data in which the probe indicator is superimposed on either the ultrasound image from the first ultrasound probe or the ultrasound image from the second ultrasound probe.

[0011] In one embodiment, the displayed image data shows a probe position relationship image that shows the scanning range of the ultrasonic beam formed by a first ultrasonic probe, which is one of the plurality of ultrasonic probes, the scanning range of the ultrasonic beam formed by a second ultrasonic probe, which is another ultrasonic probe, and the positional relationship between the first ultrasonic probe and the second ultrasonic probe.

[0012] In one embodiment, the system comprises an ultrasonic probe and a position and orientation sensor provided on each ultrasonic probe, wherein the processor generates position and orientation information for each ultrasonic probe based on the output value of the position and orientation sensor provided on each ultrasonic probe, and generates display image data based on the position and orientation information of one of the plurality of ultrasonic probes and the position and orientation information of the other ultrasonic probes.

[0013] Furthermore, this disclosure relates to an ultrasonic image processing program loaded into an ultrasonic image processing device, characterized in that it causes a processor in the ultrasonic image processing device to execute a process that generates ultrasonic image data corresponding to each of the plurality of ultrasonic probes based on a received signal output from each of the plurality of ultrasonic probes, acquires position and orientation information for each of the plurality of ultrasonic probes, and generates display image data showing an ultrasonic image from one of the plurality of ultrasonic probes and a probe indicator representing the ultrasonic transmission range of the other ultrasonic probes, based on the ultrasonic image data and position and orientation information corresponding to each of the ultrasonic probes. [Effects of the Invention]

[0014] According to this disclosure, it is possible to facilitate the observation of a subject using an ultrasound diagnostic device that employs multiple ultrasound probes. [Brief explanation of the drawing]

[0015] [Figure 1] This figure shows the configuration of an ultrasound diagnostic device according to an embodiment of the present disclosure. [Figure 2]It is a diagram showing an example of a diagnosis performed using an ultrasonic diagnostic apparatus. [Figure 3] It is a diagram showing a probe position relationship diagram schematically showing examples of the positions and orientations of each ultrasonic probe. [Figure 4] It is a diagram showing an example of an intersection / convex scanning image displayed on a display unit. [Figure 5] It is a diagram of the first observation surface and the second observation surface seen from a direction perpendicular to the normal of the first observation surface and in the direction of looking at the surface on which the second observation surface extends. [Figure 6] It is a diagram in which additional lines are added to the probe position relationship diagram shown in FIG. 3. [Figure 7] It is a diagram showing an example of an intersection / convex scanning image. [Figure 8] It is a diagram showing an example of an intersection / linear scanning image. [Figure 9] It is a diagram showing an example of a three-dimensional probe position relationship image. [Figure 10] It is a diagram showing an intersection / convex scanning image, an intersection / linear scanning image, and a three-dimensional probe position relationship image arranged side by side on a display unit. [Figure 11] It is a diagram showing an intersection / convex scanning image and an intersection / linear scanning image arranged side by side on a display unit, and a three-dimensional probe position relationship image displayed superimposed on the intersection / convex scanning image. [Figure 12] It is a diagram showing an intersection / convex scanning image, a two-dimensional intersection region / linear scanning image, and a three-dimensional probe position relationship image arranged side by side on a display unit.

Mode for Carrying Out the Invention

[0016] Embodiments of the present disclosure will be described with reference to the drawings. The same components shown in multiple drawings are denoted by the same reference numerals to simplify the description. FIG. 1 shows the configuration of an ultrasonic diagnostic apparatus 100 according to an embodiment of the present disclosure. The ultrasonic diagnostic apparatus 100 includes an information processing unit 10, a transmission / reception unit 12, an ultrasonic probe 14a, an ultrasonic probe 14b, an operation device 42, a display unit 44, and a positioning signal generation source 58.

[0017] The storage unit 46 shown in FIG. 1 together with the ultrasonic diagnostic apparatus 100 may be a storage (storage medium) such as a hard disk mounted on the ultrasonic diagnostic apparatus 100. Further, the storage unit 46 may be a memory (storage medium) of a computer on a local area network or a memory of a computer on a telecommunication line such as the Internet.

[0018] Part or all of the information processing unit 10 may be constituted by, for example, one or more computers that execute a program stored in the storage unit 46. By executing the program, the information processing unit 10 constitutes a control unit 30, a signal processing unit 32, an image synthesis unit 34, a display processing unit 36, a probe position and orientation information generation unit 38, a three-dimensional image data acquisition unit 40, and a position and orientation measurement unit 48, and operates as an ultrasonic image processing apparatus.

[0019] The control unit 30 executes overall control of the ultrasonic diagnostic apparatus 100. The control unit 30 may acquire the data obtained by each of the transmission / reception unit 12, the signal processing unit 32, the image synthesis unit 34, the display processing unit 36, the probe position and orientation information generation unit 38, and the three-dimensional image data acquisition unit 40. The control unit 30 executes information processing on the acquired data, and may output the data after the information processing to each of the transmission / reception unit 12, the signal processing unit 32, the image synthesis unit 34, the display processing unit 36, the probe position and orientation information generation unit 38, and the three-dimensional image data acquisition unit 40. Thereby, the control unit 30 may execute information processing in cooperation with each of the transmission / reception unit 12, the signal processing unit 32, the image synthesis unit 34, and the display processing unit 36.

[0020] The operating device 42 may include buttons, levers, a keyboard, a mouse, etc. The operating device 42 may also be a touch panel provided on the display unit 44. The control unit 30 may perform control of the ultrasound diagnostic apparatus 100 based on user operation. The display unit 44 may be a display such as a liquid crystal display or an organic EL display.

[0021] In this embodiment, each process is executed on any computer. Furthermore, any computer may execute these processes using a processor as hardware, a program as software, or a combination thereof. In that case, the processor is configured to work in cooperation with the program to execute the various processes in this embodiment, and can function as a unit or means in this embodiment. Also, the execution order of the processes by the processor is not limited to the order described and may be changed as appropriate. Any computer may be a general-purpose computer, a computer designed for a specific purpose, a workstation, or any other system capable of executing each process.

[0022] A processor may consist of one or more hardware components, and the type of hardware is not limited. For example, a processor may consist of a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a programmable logic device such as an FPGA (Field Programmable Gate Array), a dedicated circuit for executing a specific process such as an ASIC (Application Specific Integrated Circuit), a GPU (Graphic Processing Unit), or an NPU (Neural Processing Unit). Furthermore, the type of hardware may be a combination of different types of hardware. When multiple hardware components are configured to execute one or more processes of a given processor, these multiple hardware components may reside in physically separate devices or in the same device. Also, in any embodiment, the order of each process performed by the processor is not limited to the order described in this specification and may be changed as appropriate. Hardware is composed of electrical circuits, etc., which are combinations of circuit elements such as semiconductor elements.

[0023] Furthermore, the program may be firmware or software such as microcode. Alternatively, the program may be, for example, a set of program modules, each function of which may be implemented by a processor configured to perform its respective function. The program may be program code or multiple code segments stored on one or more non-temporary computer-readable media (e.g., storage media or other storage). The program may be divided and stored on multiple non-temporary computer-readable media located on physically separate devices. Program code or code segments may represent any combination of procedures, functions, subprograms, routines, subroutines, modules, software packages, classes, or instructions, data structures, or program statements. Program code or code segments may be connected to other code segments or hardware circuits by sending and receiving information, data, arguments, parameters, or memory contents.

[0024] Each of the ultrasonic probes 14a and 14b is equipped with multiple ultrasonic transducers. Since the functions of ultrasonic probes 14a and 14b are similar, the operation of ultrasonic probe 14a will be described here.

[0025] The transmitting / receiving unit 12 outputs a transmission signal, which is an electrical signal, to the multiple ultrasonic transducers provided by the ultrasonic probe 14a. Each ultrasonic transducer converts the transmission signal into ultrasound and transmits it toward the subject. By adjusting the delay time of the transmission signal output by the transmitting / receiving unit 12 to each ultrasonic transducer, an ultrasonic beam is formed in a specific direction.

[0026] Each ultrasonic transducer receives ultrasonic waves reflected within the subject, converts them into electrical signals called received signals, and outputs them to the signal processing unit 32. The signal processing unit 32 adjusts the delay time of the received signals output from each ultrasonic transducer so that the received signals from ultrasonic waves coming from the direction the ultrasonic beam is directed reinforce each other, and then adds up the received signals after the delay time adjustment. The signal processing unit 32 outputs the phase-corrected summed signal generated in this way to the image synthesis unit 34.

[0027] The transmitting / receiving unit 12 further changes the delay time of the transmission signal output to each ultrasonic transducer so that the ultrasonic beam is scanned within a specific observation cross-section of the specimen. The signal processing unit 32 also changes the delay time of the received signals output from each ultrasonic transducer to generate a phase-corrected sum signal corresponding to the direction of the ultrasonic beam scanned within the specimen, and then sums up each received signal after adjusting the delay time.

[0028] The image synthesis unit 34 generates B-mode image data as display image data based on the phase summation signals acquired for each direction within the observation cross-section and outputs it to the display processing unit 36. The transmitting / receiving unit 12, the signal processing unit 32, and the image synthesis unit 34 sequentially generate B-mode image data as display image data as time progresses at a predetermined frame rate and output it to the display processing unit 36. Here, the frame rate refers to the number of images generated per unit time.

[0029] The display processing unit 36 ​​converts the display image data generated sequentially over time into a video signal and outputs it to the display unit 44. Based on the video signal, the display unit 44 displays an image based on the display image data generated sequentially over time, i.e., a real-time image of the B-mode image.

[0030] The above describes a process in which an ultrasound beam is scanned within a subject, B-mode image data is generated based on phase-accurate summation signals corresponding to the ultrasound beams in each direction, and the B-mode image is displayed. The ultrasound diagnostic device 100 may also perform Doppler mode operation to determine the blood flow velocity based on the difference (Doppler shift) between the frequency of the phase-accurate summation signal and the frequency of the transmitted signal.

[0031] The image synthesis unit 34, for example, determines the blood flow velocity in a predetermined range defined on the ultrasonic beam in each direction and generates color Doppler data representing the blood flow velocity distribution in the observed cross-section. The color Doppler data shows the blood flow velocity distribution in the observed cross-section by indicating the region where blood flows toward the ultrasonic probe 14a and the region where blood moves away from the ultrasonic probe 14a with different colors. The image synthesis unit 34 generates color Doppler B-mode image data with colors indicating the blood flow velocity distribution added to the B-mode image as display image data and outputs this display image data to the display processing unit 36.

[0032] The display processing unit 36 ​​converts the display image data generated sequentially over time into a video signal and outputs it to the display unit 44. Based on the video signal, the display unit 44 displays a real-time image of the display image data generated sequentially over time, i.e., a color Doppler B-mode image.

[0033] The above describes a process in which the transmitting / receiving unit 12 transmits and receives ultrasound using the ultrasound probe 14a, and the signal processing unit 32 and image synthesis unit 34 generate ultrasound image data such as B-mode image data, color Doppler data, and color Doppler B-mode image data. Similarly, the transmitting / receiving unit 12 may transmit and receive ultrasound using the ultrasound probe 14b, and the signal processing unit 32 and image synthesis unit 34 may generate ultrasound image data. Furthermore, the transmitting / receiving unit 12, the signal processing unit 32, and the image synthesis unit 34 may generate ultrasound image data from the ultrasound probe 14a and ultrasound image data from the ultrasound probe 14b alternately, for example, over time, using a time-division multiplexing method.

[0034] In this case, the display processing unit 36 ​​may generate a video signal such that an image based on ultrasonic image data from the ultrasonic probe 14a and an image based on ultrasonic image data from the ultrasonic probe 14b are sequentially displayed on the display unit 44 over time, and output this signal to the display unit 44. The display unit 44 displays the image based on ultrasonic image data from the ultrasonic probe 14a and the image based on ultrasonic image data from the ultrasonic probe 14b in real time based on the video signal.

[0035] The image displayed by the display unit 44 may be a still image. In this case, the image synthesis unit 34 outputs, for example, one frame of display image data to the display processing unit 36. The display processing unit 36 ​​generates a video signal for displaying an image based on one frame of display image data as a still image and outputs it to the display unit 44. The display unit 44 displays the image based on the display image data as a still image based on the video signal.

[0036] In the following description, each image displayed by the display unit 44 may be a still image or a real-time image. When the display unit 44 displays a real-time image, the image synthesis unit 34 generates display image data sequentially over time based on various data acquired sequentially by the signal processing unit 32 over time, and outputs it to the display processing unit 36. The display processing unit 36 ​​converts the display image data generated sequentially over time into a video signal and outputs it to the display unit 44. The display unit 44 displays a real-time image based on the display image data generated sequentially over time, based on the video signal.

[0037] Figure 2 shows an example of a diagnosis performed using an ultrasound diagnostic device 100. Figure 2 shows the bladder 62, prostate 64, and rectum 66 of a subject 60. In this example, the ultrasound probe 14a is a transrectal probe and is inserted into the rectum 66. The ultrasound probe 14b is a linear probe and is positioned so that the tip surface on which ultrasound is transmitted and received is in contact with the body surface of the subject 60.

[0038] In the example shown in Figure 2, the ultrasound beam is convexly scanned by the ultrasound probe 14a in an observation cross-section crossing the prostate 64. Here, convex scanning refers to scanning the ultrasound beam emitted from a curved surface on which ultrasound is transmitted and received along the curved surface. In addition, the ultrasound beam is linearly scanned by the ultrasound probe 14b in an observation cross-section crossing the prostate 64. Here, linear scanning refers to scanning the ultrasound beam emitted from a surface on which ultrasound is transmitted and received in a straight line while maintaining a constant direction. In the example shown in Figure 2, the observation cross-section of ultrasound probe 14a and the observation cross-section of ultrasound probe 14b intersect.

[0039] Based on the ultrasound images in the observation cross-section of the ultrasound probe 14a and the ultrasound images in the observation cross-section of the ultrasound probe 14b, the practitioner may perform a procedure such as removing the prostate gland 64 while preserving the neurovascular bundles.

[0040] Figure 3 shows a schematic probe position relationship diagram illustrating examples of the positions and orientations of the ultrasonic probes 14a and 14b. It also shows the first observation surface 70a, where the ultrasonic beam is convexly scanned by the ultrasonic probe 14a, and the second observation surface 70b, where the ultrasonic beam is linearly scanned by the ultrasonic probe 14b. For the first and second observation surfaces 70a and 70b, the maximum observable depths are indicated by depth boundary edges 72a and 72b, respectively. One corner of the deepest part of the rectangular second observation surface 70b penetrates the first observation surface 70a. In Figure 3, the intersection line 74 of the first observation surface 70a and the second observation surface 70b is shown as a solid line.

[0041] The practitioner performs the procedure on the prostate gland 64 while referring to the B-mode image displayed by the ultrasound probe 14a, which is a transrectal probe, and the B-mode image displayed by the ultrasound probe 14b, which is a linear probe.

[0042] In the ultrasound diagnostic apparatus 100 according to this embodiment, an intersection-convex scan image, in which intersection lines 74 are drawn on the B-mode image displayed by the ultrasound probe 14a, is displayed on the display unit 44. Referring again to Figure 1, the process of displaying the intersection-convex scan image will be explained.

[0043] A positioning signal source 58 is connected to the position and attitude measurement unit 48, causing the positioning signal source 58 to transmit a positioning signal. The positioning signal transmitted from the positioning signal source 58 may be a radio wave, ultrasound, or a magnetic field. The ultrasonic probes 14a and 14b are equipped with position and attitude sensors 20a and 20b, respectively. The position and attitude sensor 20a receives the positioning signal, generates a detection signal for measuring the position and attitude of the ultrasonic probe 14a (hereinafter referred to as position and attitude), and outputs the detection signal to the position and attitude measurement unit 48. Similarly, the position and attitude sensor 20b receives the positioning signal, generates a detection signal for measuring the position and attitude of the ultrasonic probe 14b, and outputs the detection signal to the position and attitude measurement unit 48.

[0044] The position of the ultrasound probe 14a is the position of a reference point defined on the ultrasound probe 14a. The orientation of the ultrasound probe 14a may be defined by the angle that the reference line defined on the ultrasound probe 14a makes with the xy plane of the xyz coordinate system fixed to the ultrasound diagnostic device 100, the angle that it makes with the z axis, etc. The reference line may be defined, for example, based on the direction in which the multiple ultrasound transducers are arranged. The position and orientation of the ultrasound probe 14b may also be defined by a reference point and a reference line, similar to the ultrasound probe 14a.

[0045] The position and attitude measurement unit 48 generates position and attitude information indicating the position and attitude of the ultrasonic probe 14a based on the detection signal output as an output value from the position and attitude sensor 20a and the positioning signal output to the positioning signal source 58, and outputs this information to the probe position and attitude information generation unit 38. Similarly, the position and attitude measurement unit 48 generates position and attitude information indicating the position and attitude of the ultrasonic probe 14b based on the detection signal output as an output value from the position and attitude sensor 20b and the positioning signal output to the positioning signal source 58, and outputs this information to the probe position and attitude information generation unit 38.

[0046] The probe position and orientation information generation unit 38 obtains scanning range information from the control unit 30 for the ultrasonic probes 14a and 14b, indicating the scanning range of the ultrasonic beam. The scanning range information for each ultrasonic probe 14a and 14b indicates the scanning range based on a coordinate system fixed to the ultrasonic probes 14a and 14b, respectively. Based on the scanning range information for each ultrasonic probe 14a and 14b, and the position and orientation information for each ultrasonic probe 14a and 14b, the probe position and orientation information generation unit 38 obtains intersection line information indicating the range that the intersection line 74 exemplified in Figure 3 extends in the xyz 3D space. The intersection line information includes, for example, a constant that defines the function z=f(x,y) defined in the xyz coordinate system. The probe position and orientation information generation unit 38 outputs the intersection line information and the position and orientation information for the ultrasonic probes 14a and 14b to the control unit 30.

[0047] The control unit 30 outputs intersection information, scanning range information for ultrasonic probes 14a and 14b, and position and orientation information for ultrasonic probes 14a and 14b to the image synthesis unit 34. The image synthesis unit 34 generates intersection-convex scanning image data based on the B-mode image data from ultrasonic probe 14a, the intersection information, and the scanning range information and position and orientation information for ultrasonic probes 14a and 14b.

[0048] As described above, in the ultrasound diagnostic apparatus 100 according to this embodiment, the information processing unit 10 generates position and orientation information for the ultrasound probes 14a and 14b based on the output values ​​of the position and orientation sensors 20a and 20b provided on the ultrasound probes 14a and 14b, respectively, and generates cross-line / convex scanning image data as display image data based on the ultrasound image data from each of the ultrasound probes 14a and 14b, the position and orientation information of one of the ultrasound probes 14a and 14b, and the position and orientation information of the other ultrasound probe.

[0049] Figure 4 shows an example of an intersection / convex scan image 90 displayed on the display unit 44. In this example, the intersection lines 74 are shown as solid lines on the B-mode image 80 obtained by the ultrasonic probe 14a. In addition to the intersection lines 74, extension lines 76 (extension lines) that are extended from the intersection lines 74 are shown as dashed lines.

[0050] The cross lines 74 and extension lines 76 displayed on the display unit 44 represent the image (hereinafter sometimes referred to as the ultrasound image) shown by the ultrasound image data corresponding to one of the ultrasound probes 14a and 14b, and probe indicators that represent the ultrasound transmission range of the other ultrasound probe.

[0051] Figure 5 shows a view of the first observation surface 70a and the second observation surface 70b from a direction perpendicular to the normal of the first observation surface 70a and overlooking the plane to which the second observation surface 70b extends. The corner formed at the left end of the depth boundary edge 72b of the second observation surface 70b penetrates the first observation surface 70a and is located in a region deeper than the first observation surface 70a. From the starting end of the depth direction edge on the penetrating side of the second observation surface 70b, an extension line 76 extends toward the ending end of the depth boundary edge 72b, with an extension line 76 extending from the ending end. The penetrating depth direction edge refers to one of the pair of depth direction edges of the first observation surface 70a that penetrates the first observation surface 70a.

[0052] The extension line 76 starts from the end end of the intersection line 74 and ends at the projection point P where the straight line extending the non-penetrating depth-direction side in the depth direction intersects with the first observation surface 70a. In other words, the extension line 76 is a straight line connecting the end end of the intersection line 74 to the projection point P.

[0053] The probe indicator described above may be an intersection line 74 between the scanning range of the ultrasonic beam formed by the ultrasonic probe 14a (first ultrasonic probe) and the scanning range of the ultrasonic beam formed by the ultrasonic probe 14b (second ultrasonic probe), and an extension line 76 obtained by extending the intersection line 74. The extension line 76 is a projection of the scanning range of the ultrasonic beam formed by the ultrasonic probe 14b onto the ultrasonic image produced by the ultrasonic probe 14a. This projection is a projection of the scanning range of the ultrasonic beam formed by the ultrasonic probe 14b projected onto the ultrasonic image produced by the ultrasonic probe 14a in the direction of the ultrasonic beam formed by the second ultrasonic probe.

[0054] Figure 6 shows the probe position relationship diagram shown in Figure 3 with the addition of extension line 76. Extension line 76 extends from the end of intersection line 74 in the direction in which intersection line 74 extends, and ends at projection point P where the straight line obtained by extending the non-penetrating depth direction line in the depth direction intersects with the first observation surface 70a.

[0055] The image synthesis unit 34 may generate intersection-convex scan image data such that the intersection line 74 and the extension line 76 are drawn in various ways. For example, as shown in Figure 7, the image synthesis unit 34 may generate intersection-convex scan image data such that the section of the intersection line 74 on the end end side is drawn with a solid line that is thinner than the section on the start end side, and the extension line 76 on the side of the extension line 74 that is further away from the intersection line 74 than the section drawn with the solid line is drawn with a dashed line. The intersection line 74 may also be drawn so that the line thickness decreases as it moves from the start end to the end end. The intersection line 74 may also be drawn so that its color changes as it moves from the start end to the end end. The extension line 76 may also be drawn so that its color changes as it moves away from the end end of the intersection line 74.

[0056] Furthermore, the image synthesis unit 34 may generate intersection-linear scanning image data based on B-mode image data from the ultrasonic probe 14b, intersection information, and scanning range information and position / orientation information for the ultrasonic probes 14a and 14b. The intersection-linear scanning image data shows an image (intersection-linear scanning image) in which intersection lines 74 are drawn on the B-mode image displayed by the ultrasonic probe 14b.

[0057] Figure 8 shows an example of an intersection line linear scan image 92 displayed on the display unit 44. In the intersection line linear scan image 92, intersection lines 74 are drawn on the B-mode image 82 obtained by the ultrasonic probe 14b. In this figure, the upward direction corresponds to the depth direction. The left vertical edge of the intersection line linear scan image 92 corresponds to the depth direction edge on the penetrating side of the second observation surface 70b, and the right vertical edge corresponds to the depth direction edge on the non-penetrating side of the second observation surface 70b. The upper horizontal edge corresponds to the depth boundary edge 72b of the second observation surface 70b. The intersection line 74 extends from the starting end on the left vertical edge to the ending end on the upper horizontal edge.

[0058] According to the ultrasound diagnostic apparatus 100 of this embodiment, the intersection-convex scanning image 90 shown in Figures 4 and 7, or the intersection-linear scanning image 92 shown in Figure 8, is displayed on the display unit 44. That is, an image in which intersection lines 74 are drawn on the B-mode image is displayed on the display unit 44. For the intersection-convex scanning image 90, extension lines 76 are drawn on the B-mode image. Therefore, when the subject can be observed from different directions using two ultrasound probes 14a and 14b, the positional relationship between the first observation surface 70a and the second observation surface 70b can be easily grasped by the intersection lines 74 and extension lines 76.

[0059] The image synthesis unit 34 may generate display image data showing an image in which the intersection-convex scan image 90 and the intersection-linear scan image 92 are placed side by side, and output it to the display processing unit 36. That is, the image synthesis unit 34 may generate display image data showing an image in which the intersection-convex scan image 90 and the intersection-linear scan image 92 are placed side by side, based on B-mode image data from the ultrasonic probe 14a, B-mode image data from the ultrasonic probe 14b, intersection information, and scanning range information and position / orientation information for the ultrasonic probes 14a and 14b, and output it to the display processing unit 36. The display processing unit 36 ​​converts the display image data into a video signal and outputs it to the display unit 44. Based on the video signal, the display unit 44 displays an image based on the display image data, i.e., an image in which the intersection-convex scan image 90 and the intersection-linear scan image 92 are placed side by side.

[0060] The image synthesis unit 34 may generate probe position relationship image data showing a probe position relationship diagram as shown in Figure 3 or Figure 6. In this case, the image synthesis unit 34 generates probe position relationship image data as display image data based on intersection information and scanning range information and position and orientation information for the ultrasonic probes 14a and 14b.

[0061] The display processing unit 36 ​​converts the display image data into a video signal and outputs it to the display unit 44. The display unit 44 displays an image based on the display image data, i.e., a probe position relationship image, based on the video signal. The probe position relationship image includes a probe position relationship diagram.

[0062] The display processing unit 36 ​​may generate display image data that shows the probe position relationship image alongside at least one of the intersection-convex scanning image 90 and the intersection-linear scanning image 92, or an image superimposed on one of the intersection-convex scanning image 90 and the intersection-linear scanning image 92. The image synthesis unit 34 generates such display image data based on the intersection information and the scanning range information and position and orientation information for the ultrasonic probes 14a and 14b, and outputs it to the display processing unit 36. The display processing unit 36 ​​converts the display image data into a video signal and outputs it to the display unit 44. The display unit 44 displays an image based on the display image data, i.e., the probe position relationship image, based on the video signal.

[0063] The display processing unit 36 ​​may display a 3D probe position relationship image on the display unit 44, which is obtained by overlaying the probe position relationship diagram onto a 3D image obtained from an external device such as a CT or MRI. The 3D image data acquisition unit 40 shown in Figure 1 reads 3D image data from an external device and outputs it to the control unit 30. The control unit 30 outputs the 3D image data to the image synthesis unit 34.

[0064] The image synthesis unit 34 generates 3D probe position relationship image data as display image data based on intersection information, scanning range information for ultrasound probes 14a and 14b, and 3D image data, and outputs it to the display processing unit 36. The display processing unit 36 ​​converts the display image data into a video signal and outputs it to the display unit 44. The display unit 44 displays an image based on the display image data, i.e., the 3D probe position relationship image, based on the video signal. Figure 9 shows an example of the 3D probe position relationship image 94.

[0065] The display processing unit 36 ​​may generate display image data showing the 3D probe position relationship image 94 alongside at least one of the intersection-convex scan image 90 and the intersection-linear scan image 92, or superimposed on one of the intersection-convex scan image 90 and the intersection-linear scan image 92. The image synthesis unit 34 generates such display image data based on the 3D probe position relationship image data and at least one of the intersection-convex scan image data and the intersection-linear scan image data, and outputs it to the display processing unit 36. The display processing unit 36 ​​converts the display image data into a video signal and outputs it to the display unit 44. The display unit 44 displays an image based on the display image data based on the video signal.

[0066] Figure 10 shows the intersection-convex scan image 90, intersection-linear scan image 92, and 3D probe position relationship image 94 displayed side by side on the display unit 44. The intersection-convex scan image 90 is displayed on the left side of the upper row, and the intersection-linear scan image 92 is displayed on the right side of the upper row. The 3D probe position relationship image 94 is displayed on the lower row. The 3D probe position relationship image 94 displayed on the display unit 44 may be replaced with a probe position relationship image showing the probe position relationship diagram shown in Figure 3 or Figure 6.

[0067] Figure 11 shows the intersection-convex scan image 90 and the intersection-linear scan image 92 displayed side-by-side on the display unit 44, and the 3D probe position relationship image 94 superimposed on the intersection-convex scan image 90. The intersection-convex scan image 90 is displayed on the left, and the intersection-linear scan image 92 is displayed on the right. The 3D probe position relationship image 94 is superimposed on the upper right of the intersection-convex scan image 90. The 3D probe position relationship image 94 displayed on the display unit 44 may be replaced with a probe position relationship image showing the probe position relationship diagram shown in Figure 3 or Figure 6.

[0068] The above describes the process by which the image synthesis unit 34 generates ultrasound image data representing a two-dimensional image. The image synthesis unit 34 may also generate volume data representing a collection of voxels (pixels).

[0069] In this case, the ultrasonic probe 14a is one having a two-dimensional array structure in which ultrasonic transducers are arranged in two orthogonal axes (long axis and short axis). The control unit 30, the transmitting / receiving unit 12, and the ultrasonic probe 14a oscillate the observation surface in the direction of the azimuth angle that intersects the observation surface on which the transmitting and receiving beams are scanned. The oscillation of the observation surface may be performed around the axis in the long axis direction of the two-dimensional array structure.

[0070] The control unit 30, the transmitting / receiving unit 12, and the ultrasonic probe 14a, for example, oscillate the observation surface by a predetermined step size in the azimuth direction each time B-mode image data for one observation surface is generated. The image synthesis unit 34 generates B-mode image data for one observation surface each time the observation surface oscillates by the step size. The image synthesis unit 34 generates volume data composed of B-mode image data from multiple frames acquired for different azimuth angles. This volume data represents the pixel values ​​of multiple voxels arranged in three-dimensional space.

[0071] The image synthesis unit 34 may generate volume rendering image data that represents multiple voxels arranged in three-dimensional space as a two-dimensional image and output it to the display processing unit 36. The display processing unit 36 ​​generates a video signal for displaying the volume rendering image and outputs it to the display unit 44. The display unit 44 displays the volume rendering image based on the video signal.

[0072] In the above embodiment, volume data is acquired by the control unit 30, the transmitting / receiving unit 12, and the ultrasonic probe 14a oscillating across the observation surface. In addition to this process, a process to acquire volume data by transporting the ultrasonic probe 14a in a linear or curved manner may also be performed. The transport of the ultrasonic probe 14a may be performed by the operator's hand or by a mechanism for transporting the ultrasonic probe 14a.

[0073] Here, an example of generating volume data using the ultrasonic probe 14a is shown, but volume data can also be generated for the ultrasonic probe 14b using a similar configuration and processing.

[0074] When volume data is acquired by one of the ultrasound probes 14a and 14b, and two-dimensional ultrasound image data is acquired by the other, the display unit 44 displays a two-dimensional intersection region where the three-dimensional object corresponding to the volume data intersects with the observation plane, instead of the intersection line 74. The two-dimensional intersection region is a planar region. Furthermore, when volume data is acquired by both the ultrasound probes 14a and 14b, the display unit 44 displays a three-dimensional intersection region where the two three-dimensional objects corresponding to the two volume data intersect, instead of the intersection line 74. The three-dimensional intersection region is a three-dimensional region.

[0075] Figure 12 shows an example of an image displayed on the display unit 44 when volume data is acquired by the transrectal ultrasound probe 14a and two-dimensional ultrasound image data is acquired by the linear ultrasound probe 14b. Specifically, Figure 12 shows the intersection / convex scan image 90, the two-dimensional intersection region / linear scan image 98, and the three-dimensional probe position relationship image 94 displayed side by side on the display unit 44. The intersection / convex scan image 90 is displayed on the left side of the upper panel, and the two-dimensional intersection region / linear scan image 98 is displayed on the right side of the upper panel. The three-dimensional probe position relationship image 94 is displayed on the lower panel. The two-dimensional intersection region / linear scan image 98 is an image in which a figure indicating the two-dimensional intersection region is superimposed on the B-mode image.

[0076] The probe indicator displayed by the ultrasound diagnostic apparatus 100 according to this embodiment shows the region where the scanning range of the ultrasound beam formed by ultrasound probe 14a (the first ultrasound probe, which is one of the plurality of ultrasound probes) and the scanning range of the ultrasound beam formed by ultrasound probe 14b (the second ultrasound probe, which is another of the ultrasound probes) intersect. The intersection line 74 and extension line 76 shown in Figures 4 and 7, and the intersection line 74 shown in Figure 8, are probe indicators. The two-dimensional intersection region 96 shown in the two-dimensional intersection region linear scanning image 98 is also a probe indicator. The three-dimensional intersection region displayed when volume data is acquired by both ultrasound probes 14a and 14b is also a probe indicator.

[0077] This disclosure is also applicable to programs and program products. The ultrasonic image processing program according to the above embodiment of this disclosure causes a processor to perform the following processes: generate ultrasonic image data corresponding to each ultrasonic probe based on the received signals output from each of a plurality of ultrasonic probes; acquire position and orientation information for each of the plurality of ultrasonic probes; and generate display image data showing an ultrasonic image from one of the plurality of ultrasonic probes and a probe indicator representing the ultrasonic transmission range of the other ultrasonic probes, based on the ultrasonic image data and position and orientation information corresponding to each ultrasonic probe. The ultrasonic image processing program may be provided stored on a storage medium such as a memory card, USB memory, or CD-ROM.

[0078] The above describes an embodiment in which convex scanning is performed with ultrasonic probe 14a and linear scanning is performed with ultrasonic probe 14b. The scanning of the ultrasonic beam in ultrasonic probes 14a and 14b may be performed by other methods. For example, radial scanning, in which the ultrasonic beam is scanned in a donut shape around the ultrasonic probe, or sector scanning, in which the ultrasonic beam is scanned in a fan shape around a reference point on the ultrasonic probe, may be performed.

[0079] The above describes an embodiment used in a procedure on the prostate. The present invention may be used in other procedures. For example, esophageal surgery may be performed while observing two B-mode images obtained from the esophagus and the body surface, with ultrasound probe 14a used as an esophageal probe and ultrasound probe 14b used as an intraoperative probe such as a linear probe. [Explanation of Symbols]

[0080] 10 Information processing unit, 12 Transceiver unit, 14a, 14b Ultrasonic probe, 20a, 20b Position and orientation sensor, 30 Control unit, 32 Signal processing unit, 34 Image synthesis unit, 36 Display processing unit, 38 Probe position and orientation information generation unit, 40 3D image data acquisition unit, 42 Operating device, 44 Display unit, 46 Storage unit, 48 Position and orientation measurement unit, 58 Positioning signal source, 60 Subject, 62 Bladder, 64 Prostate, 66 Rectum, 70a First observation plane, 70b Second observation plane, 72a, 72b Depth boundary edge, 74 Intersection line, 76 Extension line, 80, 82 B-mode image, 90 Intersection line / convex scan image, 92 Intersection line / linear scan image, 94 3D / probe position relationship image, 96 2D intersection region, 98 2D intersection region / linear scan image.

Claims

1. An ultrasonic imaging apparatus equipped with a processor, The aforementioned processor, Based on the received signals output from each of the multiple ultrasound probes, ultrasound image data corresponding to each ultrasound probe is generated. The position and orientation information of each of the multiple ultrasonic probes is acquired, An ultrasonic image processing apparatus characterized by generating display image data showing an ultrasonic image from one of a plurality of ultrasonic probes and a probe indicator representing the ultrasonic transmission range of the other ultrasonic probes, based on the ultrasonic image data and position and orientation information corresponding to each of the ultrasonic probes.

2. An ultrasonic image processing apparatus according to claim 1, The aforementioned probe indicator is An ultrasonic image processing apparatus characterized by showing a region where the scanning range of an ultrasonic beam formed by a first ultrasonic probe, which is one of the plurality of ultrasonic probes, and the scanning range of an ultrasonic beam formed by a second ultrasonic probe, which is another of the ultrasonic probes, intersect.

3. An ultrasonic image processing apparatus according to claim 2, The aforementioned probe indicator is The scanning range of the ultrasonic beam formed by the first ultrasonic probe and the scanning range of the ultrasonic beam formed by the second ultrasonic probe are included, and the extension of the scanning range of the scanning range of the ultrasonic beam is included, The aforementioned extension wire is, An ultrasonic image processing apparatus characterized in that it provides a projection image of the scanning range of the ultrasonic beam formed by the second ultrasonic probe onto an ultrasonic image formed by the first ultrasonic probe, and the projection image extends in accordance with the direction of the ultrasonic beam formed by the second ultrasonic probe.

4. An ultrasonic image processing apparatus according to claim 2, The aforementioned display image data is An ultrasonic image processing apparatus characterized by displaying the probe indicator superimposed on either the ultrasonic image obtained by the first ultrasonic probe or the ultrasonic image obtained by the second ultrasonic probe.

5. An ultrasonic image processing apparatus according to claim 1, The aforementioned display image data is An ultrasonic image processing apparatus characterized by displaying a probe position relationship image showing the scanning range of an ultrasonic beam formed by a first ultrasonic probe, which is one of a plurality of ultrasonic probes, the scanning range of an ultrasonic beam formed by a second ultrasonic probe, which is another of the ultrasonic probes, and the positional relationship between the first ultrasonic probe and the second ultrasonic probe.

6. An ultrasonic image processing apparatus according to any one of claims 1 to 5, Each of the aforementioned ultrasonic probes, Each ultrasonic probe is provided with a position and orientation sensor, The aforementioned processor, Based on the output values ​​of the position and orientation sensors provided on each ultrasonic probe, position and orientation information for each ultrasonic probe is generated. An ultrasound diagnostic apparatus characterized by generating display image data based on the position and orientation information of one of the plurality of ultrasound probes and the position and orientation information of the other ultrasound probes.

7. An ultrasound image processing program loaded into an ultrasound image processing device, Based on the received signals output from each of the multiple ultrasound probes, ultrasound image data corresponding to each ultrasound probe is generated. The position and orientation information of each of the multiple ultrasonic probes is acquired, Based on the ultrasonic image data and position / orientation information corresponding to each ultrasonic probe, a process is performed to generate display image data showing an ultrasonic image from one of the plurality of ultrasonic probes and a probe indicator representing the ultrasonic transmission range of the other ultrasonic probes. An ultrasonic image processing program characterized by being executed by a processor provided in the aforementioned ultrasonic image processing device.

Images