Ultrasonic volume data forming apparatus and ultrasonic volume data forming program

Abstract

To improve the accuracy of ultrasonic volume data formed by arranging a plurality of frame data.SOLUTION: While the operator moves the ultrasonic probe 14 in the sweep direction, a frame data row 62 composed of a plurality of frame data 64 corresponding to each scan plane SP is acquired. The probe position / posture information acquisition unit 48 acquires position / posture information indicating the position and posture of the ultrasonic probe 14 when each frame data 64 for forming the ultrasonic volume data 60 is acquired, and associates the position / posture information with each frame data 64. The volume data forming unit 50 forms the ultrasonic volume data 60 by arranging each frame data 64 in the arrangement direction after changing at least one of the vertical plane position, which is the position in the plane perpendicular to the arrangement direction of the frame data 64 in the data space, and the posture of each frame data 64 according to the position / posture information associated with the frame data 64.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] This specification discloses an improved ultrasound volume data generation device and ultrasound volume data generation program. [Background technology]

[0002] Conventionally, medical volume data has been generated and used for various purposes. Volume data is data in which data elements called voxels are arranged three-dimensionally. Each data element included in the medical volume data is a parameter representing an area within a subject (e.g., a certain area including tissue, etc.). An arbitrary cross section is set in the medical volume data, and a two-dimensional image representing the cross section can be generated by extracting and reconstructing the medical volume data at that cross section. Such two-dimensional images are used for diagnosing subjects, etc.

[0003] Ultrasound volume data is a type of medical volume data that has been conventionally known. Ultrasound volume data is generated based on received signals obtained by transmitting and receiving ultrasonic waves to and from a subject. Each data element included in the ultrasound volume data is, for example, a parameter indicating the intensity of the reflected wave from the subject.

[0004] Ultrasound volume data can be formed by transmitting and receiving ultrasound waves to and from a subject using a two-dimensional array ultrasonic probe in which ultrasonic vibration elements that generate ultrasound waves are arranged two-dimensionally. However, a method has been proposed in the past in which ultrasound volume data is formed using an ultrasonic probe in which ultrasonic vibration elements are arranged in one direction and which scans an ultrasonic beam on a flat scanning surface.

[0005] Specifically, while moving the ultrasound probe in a direction perpendicular to the scanning plane (referred to as a "sweep" in this specification), received signals corresponding to each scanning plane (referred to as "frame data" in this specification) are acquired. Frame data is data in which data elements indicating the intensity of reflected waves from the subject are arranged two-dimensionally corresponding to the scanning plane. Then, ultrasound volume data is formed by arranging and combining each frame data in a direction perpendicular to the two-dimensional arrangement direction of the data elements (referred to as the "arrangement direction" in this specification) in data space.

[0006] For example, Patent Document 1 discloses an ultrasound diagnostic device that generates volume data based on a plurality of frame data acquired by transmitting and receiving ultrasound waves while sweeping an ultrasound probe, the ultrasound diagnostic device including: a position sensor that acquires position information indicating the position of the ultrasound probe; a data acquisition function that associates position information indicating the position of the ultrasound probe when the frame data was acquired with each frame data; a smoothing processing function that smooths positional deviations of each frame data based on the position information associated with each frame data; and a volume generation circuit that generates ultrasound volume data based on the smoothed plurality of frame data. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Patent No. 7280711 Specification Summary of the Invention [Problem to be solved by the invention]

[0008] When ultrasound volume data is formed by arranging multiple frame data in the alignment direction, it is desirable that the position and orientation of the ultrasound probe be constant when each frame data is acquired. The position of the ultrasound probe here does not mean the position in the sweep direction (referred to as the "sweep position" in this specification), but the position on a plane perpendicular to the sweep direction (referred to as the "vertical plane position" in this specification). Hereinafter, when the term "position" is used simply for the ultrasound probe 14, it is intended to encompass the sweep position and the vertical plane position.

[0009] Since the vertical position and orientation of the ultrasound probe represent the vertical position and orientation of the scanning plane corresponding to the frame data, if the vertical position or orientation of the ultrasound probe varies when each frame data is acquired, the vertical positions and orientations of the multiple scanning planes corresponding to the multiple frame data for forming the ultrasound volume data will also vary, and if the ultrasound volume data is formed by arranging the multiple frame data in the same direction, distortion will occur in the ultrasound volume data, i.e., the accuracy of the ultrasound volume data will be reduced.

[0010] Because the ultrasound probe is swept by the hand of an operator (i.e., a human being) such as a doctor, the body surface of the subject over which the ultrasound probe is swept is not flat, and the body surface of the subject over which the ultrasound probe is swept may be soft (e.g., a breast), it can be difficult to maintain a constant vertical position and orientation of the ultrasound probe when sweeping the ultrasound probe to acquire multiple frame data, which can result in a decrease in the accuracy of the ultrasound volume data.

[0011] The purpose of the ultrasound volume data forming device disclosed in this specification is to improve the accuracy of ultrasound volume data formed by arranging a plurality of frame data. [Means for solving the problem]

[0012] The ultrasound volume data forming device disclosed in the present specification is characterized by comprising: a frame data sequence acquiring unit that acquires a frame data sequence in which an ultrasound probe that scans an ultrasound beam on a subject sweeps the subject in a direction perpendicular to a scanning plane of the ultrasound beam, and each frame data has a data element sequence that indicates signal intensity of a reflected wave from the subject and is arranged two-dimensionally corresponding to the scanning plane, and in which position and orientation information indicating a vertical plane position, which is the position of the ultrasound probe on a plane perpendicular to the sweep direction, when the frame data was acquired, and an orientation of the ultrasound probe when the frame data was acquired is associated with each frame data included in the frame data sequence; and a volume data forming unit that forms ultrasound volume data by arranging the frame data sequence in an arrangement direction that is a direction perpendicular to the two-dimensional arrangement direction of the data element sequence in data space, and that changes at least one of the vertical plane position, which is the position of each frame data included in the frame data sequence on a plane perpendicular to the arrangement direction, and the orientation, according to the position and orientation information associated with the frame data, and then forms ultrasound volume data by arranging each frame data in the arrangement direction.

[0013] The volume data forming unit may extract a first feature point in one frame data and a second feature point corresponding to the first feature point in the other frame data in adjacent frame data in the frame data sequence, and change at least one of the vertical position and orientation of each frame data so that the first feature point and the second feature point are aligned in the alignment direction.

[0014] The volume data forming unit may form the ultrasound volume data by excluding frame data in which at least one of the vertical plane position and orientation has an outlier compared to the other multiple frame data, based on the position and orientation information associated with each frame data.

[0015] The volume data forming unit may generate intermediate frame data to be placed between adjacent frame data in the frame data sequence, based on the adjacent frame data, and form the ultrasound volume data using the intermediate frame data.

[0016] The position and orientation information may indicate the relative vertical position and orientation of the ultrasound probe with respect to the body surface of the subject, based on an image captured by a camera photographing a probe detection marker attached to the ultrasound probe and a body surface detection marker attached to the body surface of the subject.

[0017] Further, an ultrasound volume data formation program disclosed in the present specification causes a computer to function as: a frame data sequence acquisition unit that acquires a frame data sequence in which an ultrasound probe that scans an ultrasound beam on a subject sweeps the subject in a direction perpendicular to a scanning plane of the ultrasound beam, each frame data having a data element sequence that indicates the signal intensity of a reflected wave from the subject and that is two-dimensionally arranged corresponding to the scanning plane, and in which position and orientation information indicating the vertical plane position, which is the position of the ultrasound probe on a plane perpendicular to the sweep direction, when the frame data was acquired, and the orientation of the ultrasound probe when the frame data was acquired is associated with each frame data included in the frame data sequence; and a volume data formation unit that forms ultrasound volume data by arranging the frame data sequence in an arrangement direction that is a direction perpendicular to the two-dimensional arrangement direction of the data element sequence in data space, and that changes at least one of the vertical plane position, which is the position of each frame data included in the frame data sequence on a plane perpendicular to the arrangement direction, and the orientation, according to the position and orientation information associated with the frame data, and then forms ultrasound volume data by arranging each frame data in the arrangement direction. [Effects of the Invention]

[0018] According to the ultrasound volume data forming device disclosed in this specification, it is possible to improve the accuracy of ultrasound volume data formed by arranging a plurality of frame data. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a schematic diagram illustrating the configuration of an ultrasound volume data generation system according to an embodiment of the present invention. [Figure 2] FIG. 10 is a diagram showing an example of an image captured by a camera. [Figure 3] FIG. 2 is a diagram showing a plurality of scanning planes corresponding to a plurality of frame data for forming ultrasound volume data. [Figure 4] 1 is a schematic diagram illustrating the configuration of an ultrasound diagnostic apparatus according to the present embodiment. [Figure 5] FIG. 2 is a conceptual diagram showing the concept of ultrasound volume data formation processing. [Figure 6] FIG. 2 is a diagram showing an ultrasonic probe at a first sweep position and a second sweep position. [Figure 7] FIG. 2 is a first diagram showing first and second frame data arranged in a data space. [Figure 8] FIG. 2 is a second diagram showing the first frame data and the second frame data arranged in the data space. [Figure 9] FIG. 4 is a diagram showing first feature points in first frame data and second feature points in second frame data. [Figure 10] FIG. 10 is a diagram showing frame data associated with outlier probe position and orientation information. [Figure 11] FIG. 10 is a diagram showing intermediate frame data. [Figure 12] 4 is a flowchart showing the flow of processing performed by the ultrasound diagnostic apparatus according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0020] 1 is a schematic diagram of the configuration of an ultrasound volume data formation system 10 according to this embodiment. The ultrasound volume data formation system 10 includes a camera 12 and an ultrasound diagnostic device 16 as an ultrasound volume data formation device including an ultrasound probe 14. The camera 12 and the ultrasound diagnostic device 16 are connected to each other so that they can communicate with each other.

[0021] In this embodiment, a probe detection marker 20 is attached to the ultrasonic probe 14. The probe detection marker 20 is a marker for detecting the position and posture of the ultrasonic probe 14. In addition, a body surface detection marker 22 is attached to the body surface of the subject E. The body surface detection marker 22 has a pattern different from that of the probe detection marker 20, and is a marker for detecting the position and posture of the body surface of the subject E. An example of the probe detection marker 20 and the body surface detection marker 22 is an AR (Argumented Reality) marker.

[0022] The camera 12 includes a lens, an image sensor, a processor including a CPU (Central Processing Unit), etc., and a communication interface including a network adapter, etc. The camera 12 captures images of the ultrasound probe 14 (specifically, the probe detection marker 20) and the subject E (specifically, the body surface detection marker 22). A captured image is formed by the image sensor of the camera 12, and the captured image is transmitted to the ultrasound diagnostic device 16 via the communication interface of the camera 12.

[0023] 2 is a diagram showing an example of an image 24 captured by the camera 12. As described above, the captured image 24 includes images of the probe detection marker 20 and the body surface detection marker 22. The ultrasound diagnostic device 16 can detect the position and posture of the ultrasound probe 14 by analyzing the image of the probe detection marker 20 captured in the captured image 24. Furthermore, the ultrasound diagnostic device 16 can detect the position and posture of the body surface of the subject E by analyzing the image of the body surface detection marker 22 captured in the captured image 24. Details of the detection process of the position and posture of the ultrasound probe 14 and the subject E will be described later.

[0024] The ultrasound diagnostic device 16 as an ultrasound volume data forming device forms ultrasound volume data. Fig. 3 is a diagram showing a plurality of scanning planes SP corresponding to a plurality of frame data for forming ultrasound volume data. In the drawings referred to in this specification, the horizontal scanning direction of the ultrasound beam emitted from the ultrasound probe 14 (particularly, the scanning direction when the first frame data of a plurality of frames for forming ultrasound volume data is acquired) is designated as X. R axis, and the depth direction of the specimen E is Y R axis, X R axis and Y R The horizontal direction perpendicular to the axis is Z R The axis is X. R Axis, Y R axis and Z R The axes represent real space.

[0025] The operator of the ultrasound diagnostic device 16, such as a doctor, places the ultrasound probe 14 on the subject E while scanning the scanning plane SP (i.e., X R Y R The sweep direction (i.e., Z) is perpendicular to the plane. R While moving the ultrasonic probe 14 in the X-axis direction, a plurality of frame data corresponding to each scanning plane SP arranged in the sweep direction is formed. As shown in FIG. 3, when the ultrasonic probe 14 is swept to form a plurality of frame data, the ultrasonic probe 14 moves in the X-axis direction. R Y RIn some cases, the vertical position or orientation of the scanning plane SP may fluctuate. The ultrasound diagnostic device 16 generates ultrasound volume data by combining multiple frame data corresponding to multiple scanning planes SP. In particular, even if the vertical positions or orientations of the multiple scanning planes SP vary, the ultrasound diagnostic device 16 generates ultrasound volume data so as to suppress a decrease in accuracy. Details of the ultrasound volume data generation process will be described later.

[0026] 4 is a schematic diagram of the configuration of the ultrasonic diagnostic device 16. The ultrasonic diagnostic device 16 is a medical device installed in a medical institution such as a hospital.

[0027] The ultrasonic probe 14 is a device that transmits and receives ultrasonic waves to and from the subject E. The ultrasonic probe 14 has a transducer element array consisting of a plurality of transducer elements that transmit and receive ultrasonic waves to and from the subject E. The transducer element array is formed of a plurality of transducer elements arranged in one direction (array direction). When a transmission signal is supplied to each transducer element from the transceiver unit 30, which will be described later, each transducer element generates ultrasonic waves. Specifically, the ultrasonic probe 14 scans an ultrasonic beam on a plane (scanning plane SP) parallel to the array direction.

[0028] As described above, the ultrasonic probe 14 is provided with a probe detection marker 20 .

[0029] The transmitting / receiving unit 30 transmits a transmission signal to the ultrasonic probe 14 (more specifically, to each transducer element of the transducer element array) under the control of the control unit 46, which will be described later. As a result, an ultrasonic beam is scanned from the ultrasonic probe 14 across the scanning plane SP. The transmitting / receiving unit 30 also receives a reception signal from each transducer element that has received a reflected wave from the subject E. The transmitting / receiving unit 30 has an adder and a plurality of delay elements corresponding to each transducer element, and performs a phased addition process in which the adder and the plurality of delay elements align and add the phases of the reception signals from each transducer element. As a result, a reception beam signal is formed in which information indicating the signal intensity of the reflected wave from the subject E is aligned in the depth direction of the subject E. A plurality of reception beam signals corresponding to one scanning plane SP constitute frame data.

[0030] As the operator moves the ultrasonic probe 14 in the sweep direction, a frame data sequence consisting of a plurality of frame data corresponding to each scanning plane SP (see FIG. 3) is acquired. The path of movement of the scanning plane SP along which the ultrasonic probe 14 moves as it is swept may include, for example, the target tissue of the subject E to be examined or treated. Each frame data sequence has a data element sequence that indicates the signal intensity of the reflected wave from the subject E, which is arranged two-dimensionally corresponding to the scanning plane SP. In this way, in this embodiment, the transmitting / receiving unit 30 functions as a frame data sequence acquiring unit.

[0031] The signal processing unit 32 performs various signal processing on the received beam signal from the transmitting / receiving unit 30, including filtering using a band-pass filter and detection processing.

[0032] The image forming unit 34 forms an ultrasonic tomographic image (B-mode image) representing a cross section of the subject E (particularly, a scanning plane SP of the ultrasonic beam) based on the received beam signal that has been signal-processed in the signal processing unit 32. Furthermore, the image forming unit 34 forms a reconstructed ultrasonic image by extracting and reconstructing the formed ultrasonic volume data at an arbitrary cross section, as will be described later.

[0033] The display control unit 36 ​​controls the display of various images on the display 38, including the ultrasonic tomographic image or reconstructed ultrasonic image formed by the image forming unit 34.

[0034] The display 38 serving as a display unit is a display device configured, for example, by a liquid crystal display or an organic EL (Electro Luminescence) display.

[0035] The transmitter / receiver 30, signal processor 32, image generator 34, and display controller 36 of the ultrasound diagnostic device 16 are configured by a processor. The processor includes at least one of a general-purpose processing device (e.g., a CPU) and a dedicated processing device (e.g., a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or a programmable logic device). The processor may not be a single processing device, but may be configured by the cooperation of multiple processing devices located at physically separate locations. Furthermore, each of the above units may be realized by the cooperation of hardware such as a processor and software.

[0036] The communication interface 40 is configured by, for example, a network adapter, etc. The communication interface 40 performs the function of communicating with other devices (particularly, the camera 12). In particular, the communication interface 40 receives the captured image 24 from the camera 12.

[0037] The input interface 42 is configured by, for example, a button, a trackball, a touch panel, etc. The input interface 42 is used to input commands from the operator who uses the ultrasound diagnostic apparatus 16 to the ultrasound diagnostic apparatus 16.

[0038] The memory 44 includes a hard disk drive (HDD), a solid state drive (SSD), an embedded multi-media card (eMMC), a read-only memory (ROM), or a random access memory (RAM). The memory 44 stores an ultrasound volume data formation program for operating each unit of the ultrasound diagnostic device 16. The ultrasound volume data formation program may also be stored in a computer-readable non-transitory storage medium such as a universal serial bus (USB) memory or a CD-ROM. The ultrasound diagnostic device 16 can read and execute the ultrasound volume data formation program from such a storage medium. The ultrasound diagnostic device 16 performs the functions described below by reading the ultrasound volume data formation program, and therefore, the ultrasound diagnostic device 16 can be considered a computer program product.

[0039] The control unit 46 includes at least one of a general-purpose processor (e.g., a CPU) and a dedicated processor (e.g., a GPU, an ASIC, an FPGA, or a programmable logic device). The control unit 46 may not be a single processing unit, but may be configured by the cooperation of multiple processing units located at physically separate locations. The control unit 46 controls each unit of the ultrasound diagnostic apparatus 16. As shown in FIG. 4 , the control unit 46 functions as a probe position and orientation information acquisition unit 48 and a volume data formation unit 50 in accordance with an ultrasound volume data formation program stored in the memory 44.

[0040] The probe position and orientation information acquisition unit 48 acquires position and orientation information indicating the position (vertical plane position and sweep position) and orientation of the ultrasonic probe 14. In particular, the probe position and orientation information acquisition unit 48 acquires position and orientation information indicating the position and orientation of the ultrasonic probe 14 when each frame data for forming the ultrasonic volume data is acquired.

[0041] In this embodiment, the probe position and orientation information acquisition unit 48 acquires position and orientation information by analyzing the captured image 24 acquired by the camera 12 to detect the position and orientation of the ultrasound probe 14. As described above, the captured image 24 includes an image of the probe detection marker 20 for detecting the position and orientation of the ultrasound probe 14 (see FIG. 2). The probe position and orientation information acquisition unit 48 acquires position and orientation information by analyzing the image of the probe detection marker 20 in the captured image 24. The position of the ultrasound probe 14 may be expressed, for example, by three-dimensional coordinates in a camera coordinate system. The orientation of the ultrasound probe 14 may be expressed, for example, by a rotation angle about each of three predetermined orthogonal axes in the camera coordinate system. Note that a known method can be used to detect the position and orientation of the ultrasound probe 14 in the camera coordinate system from the image of the probe detection marker 20 included in the captured image 24, and therefore a detailed description thereof will be omitted here.

[0042] As described above, the captured image 24 also includes images of the body surface detection markers 22 for detecting the position and posture of the body surface of the subject E (see FIG. 2). The probe position and posture information acquisition unit 48 may detect the position and posture of the body surface of the subject E by analyzing the images of the body surface detection markers 22 in the captured image 24. Then, the probe position and posture information acquisition unit 48 may detect the position and posture of the ultrasound probe 14 relative to the position and posture of the body surface of the subject E. This makes it possible to obtain the position and posture of the ultrasound probe 14 with respect to the subject E, taking into account fluctuations in the position or posture of the subject E.

[0043] The probe position and orientation information acquisition unit 48 may detect the position and orientation of the ultrasonic probe 14 by a method other than analyzing the captured image 24. For example, the ultrasonic probe 14 may be provided with a position and orientation sensor such as a magnetic sensor or an acceleration sensor, and the position and orientation of the ultrasonic probe 14 may be detected based on the detection value of the position and orientation sensor.

[0044] The position and orientation of the scan plane SP (see FIG. 3) are determined by the position and orientation of the ultrasound probe 14. Therefore, it can be said that the position and orientation information acquired by the probe position and orientation information acquisition unit 48 indicates the position and orientation of the scan plane SP corresponding to each frame data for forming ultrasound volume data.

[0045] The probe position and orientation information acquisition unit 48 associates the frame data acquired by the transmission / reception unit 30 with position and orientation information indicating the position and orientation of the ultrasound probe 14 at the time the frame data was acquired, and stores the associated frame data in the memory 44. As a result, a frame data string made up of a plurality of frame data, each of which is associated with position and orientation information, is stored in the memory 44.

[0046] The volume data forming unit 50 forms ultrasound volume data based on the above-mentioned frame data sequence. Fig. 5 is a conceptual diagram showing the concept of the processing for forming ultrasound volume data 60. In Fig. 5, mutually orthogonal X coordinates in the data space in which the ultrasound volume data 60 is defined are D Axial direction, Y D Axial and Z directions D The volume data forming unit 50 determines the two-dimensional array direction (X in the example of FIG. 5) of the data element strings that each frame data 64 has. D axis and Y D The direction perpendicular to the axis (Z in the example in Figure 5) D The ultrasound volume data 60 is formed by arranging the frame data 64 in the axial direction. The position of each frame data 64 in the arranging direction may be determined based on the sweep position of the ultrasound probe 14 included in the position and orientation information associated with each frame data 64.

[0047] The frame data 64 may be a plurality of received beam signals (or a plurality of received beam signals processed by the signal processing unit 32) corresponding to one scanning plane SP formed by the transmitting / receiving unit 30, or may be a single ultrasonic tomographic image formed by the image forming unit 34 based on the plurality of received beam signals.

[0048] It should be noted that a known method can be used to generate the ultrasound volume data 60 from the frame data sequence 62, and therefore a detailed description thereof will be omitted here.

[0049] As described above, since the target tissue of the subject E is included in the movement path of the scan plane SP, the ultrasound volume data 60 becomes data including the target tissue of the subject E. When the ultrasound volume data 60 is formed, an operator or the like sets an arbitrary cross section in the ultrasound volume data 60, and the image forming unit 34 reconstructs the ultrasound volume data 60 by cutting out the set cross section, thereby becoming able to form a reconstructed ultrasound image showing the target tissue.

[0050] Here, consider a case where at least one of the vertical position and the orientation of the ultrasonic probe 14 changes while the ultrasonic probe 14 is sweeping to acquire the frame data sequence 62 (see FIG. 3). In such a case, the volume data forming unit 50 changes the arrangement direction (Z D A plane perpendicular to the X axis D Y D At least one of the vertical plane position and orientation, which are positions on the vertical plane (plane), is changed according to the position and orientation information associated with the frame data 64, and then the frame data 64 are arranged in the arrangement direction to form ultrasound volume data 60.

[0051] Specific explanations will be given below with reference to FIG. 6 and subsequent figures. FIG. 6 is a diagram showing the ultrasonic probe 14 at the first sweep position 14a and the second sweep position 14b. It is assumed that at least one of the vertical plane position and orientation of the ultrasonic probe 14 at the second sweep position 14b (referred to herein as the "second position and orientation") varies relative to the vertical plane position and orientation of the ultrasonic probe 14 at the first sweep position 14a. In other words, it is assumed that at least one of the vertical plane position and orientation of the scan plane SPb at the second sweep position 14b varies relative to the vertical plane position and orientation of the scan plane SPa at the first sweep position 14a. In the following explanations, attention will be focused on the first frame data 64a corresponding to the scan plane SPa and the second frame data 64b corresponding to the scan plane SPb (see FIGS. 7 and 8). However, the volume data forming unit 50 also performs similar processing between adjacent frame data 64 in the frame data sequence 62.

[0052] 7 is a first diagram showing the first frame data 64a and the second frame data 64b arranged in a data space. Since the first position and orientation are different from each other, the vertical plane positions and orientations between the scan planes SPa and SPb are also different from each other. If the first frame data 64a and the second frame data 64b are arranged at the same vertical plane positions and orientations as in FIG. 7, the relationship between the vertical plane positions and orientations between the first frame data 64a and the second frame data 64b will be different from the relationship between the vertical plane positions and orientations between the scan planes SPa and SPb. Therefore, distortion occurs in the generated ultrasound volume data 60.

[0053] 8, the volume data forming unit 50 changes at least one of the vertical plane position and orientation of the first frame data 64a or the second frame data 64b so that the relationship of the vertical plane position and orientation between the first frame data 64a and the second frame data 64b becomes the same as the relationship between the first position and orientation and the second position and orientation (in other words, the relationship of the vertical plane position and orientation between the scan plane SPa and the scan plane SPb).Then, the first frame data 64a and the second frame data 64b (and further other frame data) are combined to form ultrasound volume data 60.

[0054] In this way, by changing at least one of the vertical plane position and orientation of the first frame data 64a or the second frame data 64b so that the relationship is the same as that between the first position and orientation and the second position and orientation, the relationship between the vertical plane position and orientation between the first frame data 64a and the second frame data 64b can be made the same as the relationship between the vertical plane position and orientation between the scanning plane SPa and the scanning plane SPb, so that the amount of distortion in the formed ultrasound volume data 60 can be reduced, i.e., the accuracy of the ultrasound volume data 60 can be improved.

[0055] As described above, after changing at least one of the vertical plane position and orientation of the frame data, the volume data forming unit 50 may further extract a first feature point in one frame data 64 and a second feature point corresponding to the first feature point in the other frame data 64 in adjacent frame data 64 in the frame data sequence 62, and change (make further fine adjustments) at least one of the vertical plane position and orientation of each frame data so that the first feature point and the second feature point are aligned in the alignment direction.

[0056] 9 is a diagram showing a first feature point Fa in the first frame data 64a and a second feature point Fb in the second frame data 64b. Here, the description will be focused on the first frame data 64a and the second frame data 64b, but the volume data forming unit 50 can also perform similar processing between adjacent frame data 64 in the frame data sequence 62.

[0057] First, the volume data forming unit 50 detects first feature points Fa from the first frame data 64a. This detection process can be performed, for example, using techniques such as SIFT (Scale-Invariant Feature Transform) or SURF (Speed-Up Robust Features) on the ultrasound tomographic image, which is the first frame data 64a. The volume data forming unit 50 may detect multiple first feature points Fa from the first frame data 64a. Similarly, the volume data forming unit 50 may detect second feature points Fb from the second frame data 64b. The volume data forming unit 50 may also detect multiple second feature points Fb from the second frame data 64b.

[0058] Next, when a plurality of first feature points Fa and second feature points Fb are detected, the volume data forming unit 50 specifies the second feature point Fb corresponding to the first feature point Fa. Prior to the detection of the first feature point Fa and the second feature point Fb, the relationship between the vertical plane position and orientation between the first frame data 64a and the second frame data 64b based on the position and orientation information of the ultrasound probe 14 is changed, and therefore, the X of the second feature point Fb corresponding to a certain first feature point Fa is determined. D Y D Coordinates on the plane (x d2 ,y d2 ) is the X of the first feature point Fa D Y D Coordinates on the plane (x d1 ,y d1 ) in most cases. Therefore, the volume data forming unit 50 D Y DA first feature point Fa and a second feature point Fb whose coordinates on a plane are close to each other can be identified as corresponding. Alternatively, the volume data forming unit 50 may identify the second feature point Fb corresponding to the first feature point Fa by, for example, comparing the pixel values ​​of pixels surrounding the first feature point Fa with the pixel values ​​of pixels surrounding the second feature point Fb. In the example of Fig. 9, a second feature point Fb1 corresponding to the first feature point Fa1 is identified, a second feature point Fb2 corresponding to the first feature point Fa2 is identified, and a second feature point Fb3 corresponding to the first feature point Fa3 is identified.

[0059] Then, the volume data forming unit 50 determines whether the corresponding first feature point Fa and second feature point Fb are aligned in the direction (Z D At least one of the vertical plane position and orientation of the first frame data 64a or the second frame data 64b is changed so that the first feature point Fa is aligned in the X direction. D Y D Coordinates on the plane (x d1 ,y d1 ) and X of the second feature point Fb D Y D Coordinates on the plane (x d2 ,y d2 ) error between (X D Y D 9, when a plurality of first feature points Fa and a plurality of second feature points Fb are detected, the volume data forming unit 50 changes at least one of the vertical plane position and orientation of the first frame data 64a or the second frame data 64b so that the difference between the first feature point Fa and the second feature point Fb is less than a predetermined error threshold (ideally so that they match). When a plurality of first feature points Fa and a plurality of second feature points Fb are detected as shown in Fig. 9, the volume data forming unit 50 changes at least one of the vertical plane position and orientation of the first frame data 64a or the second frame data 64b so that corresponding feature points, i.e., the first feature point Fa1 and the second feature point Fb1, the first feature point Fa2 and the second feature point Fb2, and the first feature point Fa3 and the second feature point Fb3, are aligned in the alignment direction, respectively.

[0060] In this way, by performing fine adjustment based on the first feature point Fa and the second feature point Fb, the amount of distortion in the ultrasound volume data 60 to be formed can be further reduced, i.e., the accuracy of the ultrasound volume data 60 can be further improved.

[0061] FIG. 10 is a diagram showing frame data 64c associated with outlier probe position and orientation information. When sweeping the ultrasonic probe 14 to generate a plurality of frame data, the vertical plane position or orientation of the ultrasonic probe 14 may fluctuate significantly due to some factor. In such a case, if at least one of the vertical plane position and orientation of each frame data 64 is changed according to the position and orientation information associated with the frame data 64, as described above, frame data 64, such as frame data 64c shown in FIG. 10, will be generated, in which at least one of the vertical plane position and orientation is significantly different from other frame data 64a, 64b, and 64d (i.e., an outlier). From the viewpoint of suppressing a decrease in the accuracy of the ultrasound volume data 60, it is better to generate the ultrasound volume data 60 by excluding such frame data 64 whose at least one of the vertical plane position and orientation is an outlier compared to the other plurality of frame data 64.

[0062] Therefore, the volume data forming unit 50 may form the ultrasound volume data 60 by excluding frame data 64 (frame data 64c in the example of Figure 10) whose vertical plane position and / or orientation are outliers compared to the other multiple frame data 64, based on the position and orientation information associated with each frame data 64.

[0063] FIG. 11 is a diagram showing intermediate frame data 64m. When sweeping the ultrasonic probe 14 to form a plurality of frame data, if the operator quickly moves the ultrasonic probe 14 in the sweep direction, the first sweep position 14a and the second sweep position 14b (see FIG. 6) may become separated. That is, the distance in the sweep direction between the scan plane SPa and the scan plane SPb becomes long. In such a case, if the position in the arrangement direction of each frame data 64 is determined based on the sweep position of the ultrasonic probe 14 contained in the position and orientation information associated with each frame data 64, the distance d in the arrangement direction between the first frame data 64a corresponding to the scan plane SPa and the second frame data 64b corresponding to the scan plane SPb in the data space becomes long. If the ultrasound volume data 60 is formed in this state, a missing portion of a data element occurs between the first frame data 64a and the second frame data 64b.

[0064] Therefore, the volume data forming unit 50 may generate intermediate frame data 64m to be placed between adjacent frame data 64 (first frame data 64a and second frame data 64b in the example of FIG. 11) in the frame data sequence 62. The volume data forming unit 50 may then use the intermediate frame data 64m to form the ultrasound volume data 60. The volume data forming unit 50 may form the intermediate frame data 64m when the distance in the sweep direction between the first sweep position 14a and the second sweep position 14b is equal to or greater than a threshold distance.

[0065] Known techniques can be used to generate the intermediate frame data 64m based on the first frame data 64a and the second frame data 64b. For example, the intermediate frame data 64m can be generated by applying a spatiotemporal filter to the first frame data 64a and the second frame data 64b. Filter types that can be used include smoothing filters, anisotropic filters, adaptive filters, and combinations of these. Alternatively, local changes in the second frame data 64b relative to the first frame data 64a can be detected and the intermediate frame data 64m can be subjected to nonlinear correction.

[0066] When the intermediate frame data 64m is generated, the volume data forming unit 50 arranges the intermediate frame data 64m between the first frame data 64a and the second frame data 64b in the arrangement direction, and then forms the ultrasound volume data 60. At this time, it is preferable that the intermediate frame data 64m is arranged in the middle between the first frame data 64a and the second frame data 64b in the arrangement direction.

[0067] The outline of the configuration of the ultrasound volume data formation system 10 according to this embodiment has been described above. The flow of processing by the ultrasound diagnostic device 16 will now be described with reference to the flowchart shown in FIG.

[0068] In step S10, a frame data sequence 62 consisting of a plurality of frame data 64 corresponding to each scan plane SP is acquired while the operator moves the ultrasonic probe 14 in the sweep direction. The probe position and orientation information acquisition unit 48 acquires position and orientation information indicating the position and orientation of the ultrasonic probe 14 when each frame data 64 for forming the ultrasonic volume data 60 is acquired. The probe position and orientation information acquisition unit 48 associates each frame data 64 with the position and orientation information indicating the position and orientation of the ultrasonic probe 14 when the frame data 64 is acquired, and stores the associated information in the memory 44.

[0069] In step S12, the volume data formation unit 50 detects displacement of the vertical plane position and orientation of the latest frame data (e.g., second frame data 64b), which is the frame data 64 adjacent to and acquired next to the immediately previous frame data in the frame data sequence 62, relative to the immediately previous frame data (e.g., first frame data 64a), which is the frame data 64 that was last synthesized among the frame data 64 that have already been synthesized (if there is no frame data 64 that has been synthesized, the frame data 64 at the beginning of the frame data sequence 62). The detection of this displacement is performed based on position and orientation information associated with the immediately previous frame data and position and orientation information associated with the latest frame data.

[0070] In step S14, the volume data formation unit 50 performs affine transformation on the latest frame data based on the displacement detected in step S12 so that the relationship between the vertical plane position and orientation between the immediately preceding frame data and the latest frame data becomes the same as the relationship between the position and orientation information associated with the immediately preceding frame data and the position and orientation information associated with the latest frame data. In other words, at least one of the vertical plane position and orientation of the latest frame data is changed.

[0071] In step S16, the volume data forming unit 50 extracts feature points from each of the immediately preceding frame data and the latest frame data.

[0072] In step S18, the volume data forming unit 50 calculates the X of the feature points detected from the immediately preceding frame data. D Y D The coordinates on the plane (see Figure 9) and the X coordinate of the feature point detected from the latest frame data corresponding to the feature point D Y D Error with respect to the coordinates on the plane (X D Y D It is determined whether the error (distance in the plane) is less than a predetermined error threshold. If the error is equal to or greater than the error threshold, the process proceeds to step S20.

[0073] In step S20, the volume data forming unit 50 fine-tunes the vertical plane position or orientation of the latest frame data so as to reduce the error detected in step S18. After the fine-tuning, the volume data forming unit 50 again determines whether the error is less than the error threshold. The volume data forming unit 50 repeats the fine-tuning of the latest frame data until the error becomes less than the error threshold. If it is determined that the error is less than the error threshold, the process proceeds to step S22.

[0074] In step S22, the volume data forming unit 50 combines (stacks) the immediately preceding frame data and the latest frame data.

[0075] In step S24, the volume data forming unit 50 determines whether the formation process of the ultrasound volume data 60 has been completed, in other words, whether the synthesis process of all frame data 64 for forming the ultrasound volume data 60 has been completed. If the formation process of the ultrasound volume data 60 has not been completed, the process returns to step S12, and in step S12 again, the immediately preceding frame data and the latest frame data are changed, and the processes of steps S12 to S24 are repeated. If it is determined in step S24 that the formation process of the ultrasound volume data 60 has been completed, the process ends.

[0076] The ultrasound volume data forming device according to the present disclosure has been described above, but the ultrasound volume data forming device according to the present disclosure is not limited to the above embodiment, and various modifications are possible as long as they do not deviate from the spirit of the invention.

[0077] For example, in each of the above embodiments, the ultrasound volume data forming device is the ultrasound diagnostic device 16, and the functions of the frame data sequence acquisition unit and the volume data forming unit 50 are possessed by the ultrasound diagnostic device 16. However, these functions do not necessarily have to be performed by the ultrasound diagnostic device 16. For example, these functions may be performed by a server computer or the like communicably connected to the ultrasound diagnostic device 16. In this case, a processor of the server computer or the like serving as the ultrasound volume data forming device acquires a frame data sequence 62 in which position and orientation information of the ultrasound probe 14 is associated with each frame data 64 (performing the function of the frame data sequence acquisition unit), and forms ultrasound volume data 60 based on the frame data sequence 62 (performing the function of the volume data forming unit 50). Furthermore, each of the above functions may not all be performed by a single device, but may be performed by cooperation of multiple devices. [Explanation of symbols]

[0078] 10 ultrasound volume data formation system, 12 camera, 14 ultrasound probe, 14a first sweep position, 14b second sweep position, 16 ultrasound diagnostic device, 20 probe detection marker, 22 body surface detection marker, 24 captured image, 30 transmitter / receiver unit, 32 signal processing unit, 34 image formation unit, 36 display control unit, 38 display, 40 communication interface, 42 input interface, 44 memory, 46 control unit, 48 probe position and orientation information acquisition unit, 50 volume data formation unit, 60 ultrasound volume data, 62 frame data string, 64 frame data, 64a first frame data, 64b second frame data, 64m intermediate frame data.

Claims

1. a frame data sequence acquiring unit that acquires a frame data sequence obtained by an ultrasonic probe that scans an ultrasonic beam on a subject while sweeping the ultrasonic beam in a direction perpendicular to a scanning plane of the ultrasonic beam, wherein each frame data has a data element sequence that indicates signal intensity of a reflected wave from the subject and is two-dimensionally arranged corresponding to the scanning plane, and each frame data included in the frame data sequence is associated with position and orientation information that indicates a vertical plane position, which is the position of the ultrasonic probe on a plane perpendicular to the sweep direction, when the frame data was acquired, and an orientation of the ultrasonic probe when the frame data was acquired; a volume data forming unit that forms ultrasound volume data by arranging the frame data sequence in an arrangement direction that is a direction perpendicular to the two-dimensional array direction of the data element sequence in data space, and that changes at least one of a vertical plane position and an orientation of each frame data included in the frame data sequence, which is a position on a plane perpendicular to the arrangement direction, in accordance with the position and orientation information associated with the frame data, and then forms ultrasound volume data by arranging each frame data in the arrangement direction; An ultrasound volume data forming apparatus comprising:

2. the volume data forming unit extracts a first feature point in one frame data and a second feature point corresponding to the first feature point in the other frame data from adjacent frame data in the frame data sequence, and changes at least one of a vertical plane position and an orientation of each frame data so that the first feature point and the second feature point are aligned in the alignment direction.

2. The ultrasound volume data generating device according to claim 1.

3. the volume data forming unit forms the ultrasound volume data by excluding frame data in which at least one of a vertical plane position and an orientation has an outlier compared to other multiple frame data, based on the position and orientation information associated with each frame data.

2. The ultrasound volume data generating device according to claim 1.

4. the volume data forming unit generates intermediate frame data to be placed between two adjacent frame data in the frame data sequence, based on the two adjacent frame data, and forms the ultrasound volume data using the intermediate frame data.

2. The ultrasound volume data generating device according to claim 1.

5. The position and orientation information indicates a relative vertical position and orientation of the ultrasound probe with respect to the body surface of the subject, based on a captured image obtained by a camera capturing an image of a probe detection marker attached to the ultrasound probe and a body surface detection marker attached to the body surface of the subject.

2. The ultrasound volume data generating device according to claim 1.

6. Computer, a frame data sequence acquiring unit that acquires a frame data sequence obtained by an ultrasonic probe that scans an ultrasonic beam on a subject while sweeping the ultrasonic beam in a direction perpendicular to a scanning plane of the ultrasonic beam, wherein each frame data has a data element sequence that indicates signal intensity of a reflected wave from the subject and is two-dimensionally arranged corresponding to the scanning plane, and each frame data included in the frame data sequence is associated with position and orientation information that indicates a vertical plane position, which is the position of the ultrasonic probe on a plane perpendicular to the sweep direction, when the frame data was acquired, and an orientation of the ultrasonic probe when the frame data was acquired; a volume data forming unit that forms ultrasound volume data by arranging the frame data sequence in an arrangement direction that is a direction perpendicular to the two-dimensional array direction of the data element sequence in data space, and that changes at least one of a vertical plane position and an orientation of each frame data included in the frame data sequence, which is a position on a plane perpendicular to the arrangement direction, in accordance with the position and orientation information associated with the frame data, and then forms ultrasound volume data by arranging each frame data in the arrangement direction; 4. An ultrasound volume data generation program, comprising:

Images