Ultrasonic object volume calculation method and related ultrasonic system

Abstract

The present invention provides an ultrasonic object volume calculation method and an ultrasonic system, which include: collecting a plurality of continuous frames of an object at a plurality of time points by an ultrasonic reflection method; detecting the object in each of the plurality of frames; when the first frame of the plurality of frames is detected to have the object, determining the predicted contour, the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral of the object in the first frame; determining the contour of the object according to the predicted contour, the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral of the object in the first frame; and converting the object into a three-dimensional space according to the contour of the object to calculate the volume of the object; wherein the plurality of frames are two-dimensional images. The ultrasonic system and the ultrasonic object volume calculation method of the present invention can segment the object in real time to display it on a display, and quickly calculate the volume of the object.

Description

Technical Field

[0001] The present invention relates to a method for calculating the volume of an ultrasonic object and a related ultrasonic system, and in particular to a method for calculating the volume of an ultrasonic object that can effectively collect data and quickly calculate and a related ultrasonic system. Background Art

[0002] Existing imaging technologies used in the medical field, such as magnetic resonance imaging (MRI), computed tomography (CT) imaging and ultrasonic three-dimensional imaging, can obtain images clearly and quickly, and are therefore widely used in medical examinations. Among them, the architecture of existing ultrasonic three-dimensional imaging technology is usually composed of a one-dimensional probe and a positioning device. When the one-dimensional probe collects two-dimensional images, it also collects related positioning information, and then combines a sequence of two-dimensional images into a three-dimensional image, thereby estimating the volume of a target object in the three-dimensional image. However, compared with magnetic resonance imaging and CT, the data collection and processing of ultrasonic three-dimensional imaging technology are more complicated and difficult, that is, the amount of calculation of ultrasonic three-dimensional imaging technology during imaging is too large, so the existing technology needs to be improved. Summary of the invention

[0003] Therefore, the present invention provides a method for calculating the volume of an ultrasonic object and a related ultrasonic system, which combines artificial intelligence technology to utilize information in the data collection stage to accelerate the calculation of the ultrasonic object.

[0004] The present invention provides a method for calculating the volume of an ultrasonic object, which comprises: collecting a plurality of continuous frames of an object at a plurality of time points by an ultrasonic reflection method; detecting the object in each of the plurality of frames; when the first frame of the plurality of frames is detected to have the object, determining the predicted contour, the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral of the object in the first frame; determining the contour of the object according to the predicted contour, the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral of the object in the first frame; and converting the object into a three-dimensional space according to the contour of the object to calculate the volume of the object; wherein the plurality of frames are two-dimensional images.

[0005] Preferably, the method further comprises: segmenting the object from the first frame to display the object on a display.

[0006] Preferably, the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral corresponding to the object in the first frame are determined according to the predicted contour.

[0007] Preferably, determining the contour of the object based on the predicted contour, the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral of the object in the first frame also includes: calculating an intermediate feature layer between the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral corresponding to the object; and determining the contour of the object based on the predicted contour and the intermediate feature layer.

[0008] Preferably, the method further includes: determining a third frame that is temporally between the first frame and the second frame based on the first frame and the second frame among the multiple frames; predicting the position and size of the object in the third frame using the maximum circumscribed quadrilateral corresponding to the object in the first frame and the maximum circumscribed quadrilateral corresponding to the object in the second frame; and determining whether the object in the third frame is the same object as the objects in the first frame and the second frame.

[0009] Preferably, the method further comprises: when the object in the third frame is different from the object in the first frame and the second frame, re-detecting the object in each of the multiple frames; and when the object in the third frame is the same as the object in the first frame and the second frame, determining the outline of the object using the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral corresponding to the object in the first frame, and the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral corresponding to the object in the second frame.

[0010] Preferably, the intermediate feature layer is obtained by performing convolution operation under a convolutional neural network (CNN) or UNet-Like neural network architecture.

[0011] Preferably, the method of converting the object into a three-dimensional space to calculate the volume of the object based on the contour of the object further includes: determining a plurality of positioning feature points corresponding to the contour of the object; converting the plurality of positioning feature points into the three-dimensional space based on a neural network; and reconstructing and converting the object into a shape in the three-dimensional space to calculate the volume of the object.

[0012] Preferably, the plurality of positioning feature points are at least one of a tilt angle, an acceleration and a displacement of a probe of the ultrasonic device.

[0013] The present invention also provides an ultrasound system, which includes: a probe, including a positioning device, used to collect multiple continuous frames of an object at multiple time points using an ultrasound reflection method; and a processor, used to detect the object in each of the multiple frames; when the first frame of the multiple frames is detected to have the object, determine the predicted contour, the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral of the object in the first frame; determine the contour of the object according to the predicted contour, the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral of the object in the first frame; and convert the object into a three-dimensional space according to the contour of the object to calculate the volume of the object; wherein the multiple frames are two-dimensional images.

[0014] Preferably, the processor is used to segment the object from the first frame to display the object on a display.

[0015] Preferably, the processor is used for determining the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral corresponding to the object in the first frame according to the predicted contour.

[0016] Preferably, the processor is used to calculate an intermediate feature layer between the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral corresponding to the object; and to determine the contour of the object according to the predicted contour and the intermediate feature layer.

[0017] Preferably, the processor is used to determine a third frame that is temporally between the first frame and the second frame based on the first frame and the second frame among the multiple frames; predict the position and size of the object in the third frame using the maximum circumscribed quadrilateral corresponding to the object in the first frame and the maximum circumscribed quadrilateral corresponding to the object in the second frame; and determine whether the object in the third frame is the same object as the objects in the first frame and the second frame.

[0018] Preferably, when the object in the third frame is different from the object in the first frame and the second frame, the processor re-detects the object in each of the multiple frames; and when the object in the third frame is the same as the object in the first frame and the second frame, the processor determines the outline of the object using the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral corresponding to the object in the first frame, and the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral corresponding to the object in the second frame.

[0019] Preferably, the processor performs convolution operation under a convolutional neural network (CNN) or UNet-Like neural network architecture to calculate the intermediate feature layer between the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral corresponding to the object.

[0020] Preferably, the processor is used to determine a plurality of positioning feature points corresponding to the contour of the object; transform the plurality of positioning feature points into the three-dimensional space according to a neural network; and reconstruct and transform the object into a shape of the three-dimensional space to calculate the volume of the object.

[0021] Preferably, the plurality of positioning feature points are at least one of a tilt angle, an acceleration and a displacement of the probe.

[0022] Compared with the prior art, the ultrasonic system and the method for calculating the volume of an ultrasonic object of the present invention combine artificial intelligence technology, which is beneficial to the information in the data collection stage, can segment objects in real time to display them on a display, and quickly calculate the volume of objects. In addition, according to different needs, the ultrasonic system of the present invention can be applied to medical or other fields to perform ultrasonic imaging and ultrasonic object volume calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 FIG. 4 is a schematic diagram of an ultrasonic system according to an embodiment of the present invention.

[0024] Figure 2 FIG. 4 is a schematic diagram of a method for calculating the volume of an ultrasonic object according to an embodiment of the present invention.

[0025] Figure 3 A schematic diagram of determining the contour of an object according to an embodiment of the present invention.

[0026] Figure 4 A schematic diagram of predicting the contour of an object in an uncaptured frame according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to provide a further understanding of the purpose, structure, features, and functions of the present invention, the following detailed description is given in conjunction with the embodiments.

[0028] Please refer to Figure 1 , Figure 1Schematic diagram of an ultrasound system 10 according to an embodiment of the present invention. The ultrasound system 10 includes a probe 102 and a processor 104. The probe 102 includes a positioning device 106, which collects a plurality of continuous frames of an object at multiple time points by an ultrasound reflection method. In one embodiment, the probe 102 may be a one-dimensional probe, which is used to collect a sequence of two-dimensional images (i.e., a sequence of frames), and when collecting the two-dimensional images of the object, the corresponding positioning information is collected at the same time. For example, the positioning device 106 in the probe 102 may be a mechanical device (e.g., a motor) or a three-axis sensor, so as to measure frames of multiple planes by translation, swinging, or rotation when performing scanning, and collect corresponding positioning information. The processor 104 is used to detect the object in each of the continuous frames, and when the first frame is detected to have an object, the predicted contour, the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral of the object are determined, so that the processor 104 can determine the contour of the object according to the predicted contour, the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral of the object to calculate the volume of the object. In this way, the ultrasound system 10 of the embodiment of the present invention can use the collected partial frames to calculate the volume of the object, so as to display a single frame in real time during data collection.

[0029] In order to enable the ultrasound system 10 to segment the object in a single frame for display on the display when collecting data, and then calculate the volume of the ultrasound object. The ultrasound system 10 of the embodiment of the present invention can execute the ultrasound object volume calculation method 20, such as Figure 2 As shown, the ultrasonic object volume calculation method 20 uses the currently collected data to estimate the object volume, thereby reducing the subsequent calculation time. The ultrasonic object volume calculation method 20 includes the following steps:

[0030] Step 202: Start.

[0031] Step 204: Collect a plurality of continuous frames of the object at a plurality of time points using an ultrasonic reflection method.

[0032] Step 206: Detect objects in each of the plurality of frames.

[0033] Step 208: When it is detected that the first frame has an object, determine a predicted outline, a minimum circumscribed quadrilateral, and a maximum inscribed quadrilateral of the object in the first frame.

[0034] Step 210: Determine the outline of the object according to the predicted outline, the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral of the object in the first frame.

[0035] Step 212: According to the outline of the object in the first frame, transform the object into a three-dimensional space to calculate the volume of the object.

[0036] Step 214: End.

[0037] First, the ultrasound system 10 collects information about continuous frames. In step 204, the probe 102 collects continuous frames at multiple time points using an ultrasound reflection method. Since each frame may or may not contain an object, in step 206, the processor 104 may detect whether there is an object in all frames, specific frames, or sampled frames.

[0038] In step 208, when the processor 104 detects that the first frame has an object, the predicted contour, the minimum circumscribed quadrilateral, and the maximum inscribed quadrilateral of the object in the first frame are determined. In one embodiment, the ultrasound system 10 of the present invention uses artificial intelligence, such as a convolutional neural network (CNN) or a UNet-Like neural network, to determine the contour of the object. For details, please refer to Figure 3 , Figure 3 Schematic diagram of determining the contour of an object according to an embodiment of the present invention. Figure 3 When there is an object O_1 in the frame, the ultrasound system 10 first uses the first layer of the neural network to find out the approximate position, contour (ie, predicted contour), minimum circumscribed quadrilateral CR_1 and maximum inscribed quadrilateral IR_1 of the object.

[0039] Next, in step 210, the processor 104 determines the outline of the object according to the predicted outline, the minimum circumscribed quadrilateral, and the maximum inscribed quadrilateral of the object in the first frame. Figure 3 In the example, the processor 104 uses the second layer of the neural network to detect the position and contour of the object O_1 found by the first layer, that is, to calculate the intermediate feature layer CA_1 between the minimum circumscribed quadrilateral CR_1 and the maximum inscribed quadrilateral IR_1, and then determines the contour of the object based on the predicted contour and the intermediate feature layer. Therefore, in step 212, the ultrasonic system 10 of the present invention can convert the object into a three-dimensional space based on the contour of the object in the first frame to calculate the volume of the object. In this way, the ultrasonic system 10 can segment the object from a specific frame and display it on the display, so as to achieve real-time display of the object during data acquisition.

[0040] It is worth noting that in Figure 3 In the example, when the ultrasonic system 10 performs the second-layer detection in the neural network, the convolution operation is performed on the intermediate feature layer CA_1 between the minimum circumscribed quadrilateral CR_1 and the maximum inscribed quadrilateral IR_1 to reduce the amount of calculation, and the intermediate feature layer CA_1 can enhance the features of the neural network through the object contour determined by the first layer.

[0041] On the other hand, after the ultrasound system 10 obtains multiple frames with objects, in order to ensure that the objects in the frames are the same and predict the object contours of the frames without collecting data, the ultrasound system 10 can further find a third frame between the first frame and the second frame in the same sequence of frames for the frames in which the objects and the object contours have been detected.

[0042] Please also refer to Figure 4 , Figure 4 Schematic diagram of predicting the contour of an object in an uncollected frame according to an embodiment of the present invention. Figure 4 In the example, it is assumed that the ultrasound system 10 has found the object O_1 in the first frame and the object O_2 in the second frame and determined their object contours. The ultrasound system 10 predicts the contour of the object O_3 in the third frame (i.e., the position and size of the object in the third frame) based on the minimum circumscribed quadrilaterals CR_1 and CR_2 of the object contours in the first frame and the second frame. In other words, the ultrasound system 10 predicts the position and size of the object O_3 corresponding to the third frame based on the minimum circumscribed quadrilateral CR_1 corresponding to the object contour in the first frame and the minimum circumscribed quadrilateral CR_2 corresponding to the object contour in the second frame to determine whether the object in the third frame is the same object as the objects in the first frame and the second frame.

[0043] The ultrasonic system 10 can use different conditions to confirm whether the object O_3 in the third frame is the same object as the objects in the first frame and the second frame. In one embodiment, when the centroid of the object O_3 in the third frame is less than the distance d1 from the centroid of the object in the first frame or the centroid of the object in the second frame, the object O_3 in the third frame is determined to be the same object as the objects in the first frame and the second frame; on the contrary, when the centroid of the object O_3 in the third frame is greater than the distance d1 from the centroid of the object in the first frame or the centroid of the object in the second frame, the object O_3 in the third frame is determined to be different from the objects in the first frame and the second frame. Alternatively, it is also possible to determine whether the object O_3 in the third frame is the same object as the objects in the first frame and the second frame based on whether the area or position of the minimum circumscribed quadrilateral CR_3 in the third frame is between the minimum circumscribed quadrilaterals CR_1 and CR_2 in the first frame and the second frame. It should be noted that the condition for determining whether the object O_3 in the third frame is the same object as the objects in the first frame and the second frame is not limited to the above exemplary conditions.

[0044] When the processor 104 determines that the object O_3 in the third frame is different from the objects in the first frame and the second frame, the object in each captured frame is re-detected; conversely, when the object O_3 in the third frame is the same as the objects in the first frame and the second frame, the outline of the object is determined by the minimum circumscribed quadrilateral CR_1 and the maximum inscribed quadrilateral IR_1 corresponding to the object O_1 in the first frame, and the minimum circumscribed quadrilateral CR_2 and the maximum inscribed quadrilateral IR_2 corresponding to the object in the second frame. In the above embodiment, the ultrasonic system 10 of the present invention can use a convolutional neural network or a UNet-Like neural network to determine the minimum circumscribed quadrilateral CR_3 and the maximum inscribed quadrilateral IR_3 of the object in the third frame according to the first frame and the second frame, and then perform convolution operations on the intermediate feature layer CA_1 between the minimum circumscribed quadrilateral CR_1 and the maximum inscribed quadrilateral IR_1 of the first frame and the intermediate feature layer CA_2 between the minimum circumscribed quadrilateral CR_2 and the maximum inscribed quadrilateral IR_2 of the second frame to reduce the amount of calculation, and then fuse the intermediate feature layers CA_1 and CA_2 to strengthen the characteristics of the neural network.

[0045] After determining the contour of the object, the processor 104 further determines a plurality of positioning feature points corresponding to the contour of the object (e.g., the tilt angle, acceleration, and displacement of the probe), and converts the positioning feature points into three-dimensional space by a convolutional neural network or a UNet-Like neural network. In one embodiment, the ultrasound system 10 of the present invention can convert the positioning feature points into three-dimensional space according to different scanning modes (e.g., translation, swing, rotation) of the probe 102.

[0046] Next, the object is reconstructed and converted into a three-dimensional shape based on the positioning feature points of the object's contour to calculate the object's volume. In one embodiment, since there may not be enough frames to reconstruct the three-dimensional shape of the object during ultrasound scanning, the ultrasound system 10 can perform point cloud completion on the positioning feature points converted to the three-dimensional space through a point cloud completion network (PCN) to restore the three-dimensional contour of the object. In addition, since the scanning speed of the probe 102 of the ultrasound system 10 is not fixed when it is handheld, the number of sampled frames is not fixed, so more interpolated frames are required during reconstruction, which increases the error. Therefore, the point cloud completion network can complete the non-existent parts of the known frames, thereby reducing the error in converting to the three-dimensional shape.

[0047] Finally, the processor 104 performs equidistant slicing processing in a specific direction of the point cloud of the three-dimensional shape of the object to obtain discrete point cloud slices corresponding to the point cloud of the three-dimensional shape of the object, and searches for polygons of the outer contour of the point cloud slices one by one in the cutting order to calculate the area of ​​the point cloud slices. In this way, the block volume of the point cloud of the three-dimensional shape of the object can be obtained by using the slice area and the spacing between adjacent slices, and the sum is calculated to obtain the volume of the entire point cloud body corresponding to the point cloud of the three-dimensional shape of the object.

[0048] The above embodiments can illustrate the ultrasonic system and the method for calculating the volume of an ultrasonic object of the present invention, which can segment the object in real time to display it on the display, so as to quickly calculate the volume of the object. In addition, according to different needs, the ultrasonic system of the present invention can be applied to medical or other fields to perform ultrasonic imaging and volume calculation of ultrasonic objects. In addition, the neural network used by the present invention to detect objects is not limited to the above-mentioned convolutional neural network or UNet-Like neural network architecture, and the point cloud completion step is not limited to execution by the point cloud completion network. Other methods that can be used to achieve the same effect are also applicable to the present invention, but are not limited to this.

[0049] In summary, the embodiments of the present invention provide a method for calculating the volume of an ultrasonic object and a related ultrasonic system, which are combined with artificial intelligence technology to utilize information in the data collection stage to accelerate the calculation of the ultrasonic object.

[0050] The present invention has been described by the above-mentioned relevant embodiments, however, the above-mentioned embodiments are only examples for implementing the present invention. It must be pointed out that the disclosed embodiments do not limit the scope of the present invention. On the contrary, changes and modifications made without departing from the spirit and scope of the present invention are all within the scope of patent protection of the present invention.

Claims

1. A method for calculating the volume of an ultrasonic object, characterized in that: Include: Collecting a plurality of continuous frames of an object at a plurality of time points by an ultrasonic reflection method; detecting the object in each of the plurality of frames; When a first frame among the plurality of frames is detected to have the object, a predicted outline, a minimum circumscribed quadrilateral, and a maximum inscribed quadrilateral of the object in the first frame are determined based on a neural network; Performing a convolution operation on an intermediate feature layer between the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral of the object, and determining a contour of the object according to the predicted contour and the intermediate feature layer; as well as According to the outline of the object, transform the object into a three-dimensional space to calculate the volume of the object; The multiple frames are two-dimensional images.

2. The method for calculating the volume of an ultrasonic object as claimed in claim 1, wherein: Also includes: The object is segmented from the first frame to display the object on a display.

3. The method for calculating the volume of an ultrasonic object according to claim 1, wherein: The minimum circumscribed quadrilateral and the maximum inscribed quadrilateral corresponding to the object in the first frame are determined according to the predicted contour.

4. The method for calculating the volume of an ultrasonic object according to claim 1, wherein: Also includes: Determine a third frame that is between the first frame and the second frame in terms of timing according to the first frame and the second frame among the plurality of frames; Predicting the position and size of the object in the third frame using the largest circumscribed quadrilateral of the object in the first frame and the largest circumscribed quadrilateral of the object in the second frame; as well as Determine whether the object in the third frame is the same object as the objects in the first frame and the second frame.

5. The method for calculating the volume of an ultrasonic object as claimed in claim 4, wherein: Also includes: When the object in the third frame is different from the objects in the first frame and the second frame, re-detecting the object in each of the plurality of frames; as well as When the object in the third frame is the same as the objects in the first frame and the second frame, the outline of the object is determined by using the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral corresponding to the object in the first frame and the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral corresponding to the object in the second frame.

6. The method for calculating the volume of an ultrasonic object as claimed in claim 1, wherein: The convolution operation is performed under the convolutional neural network or UNet-Like neural network architecture.

7. The method for calculating the volume of an ultrasonic object as claimed in claim 1, wherein: The step of converting the object into a three-dimensional space to calculate the volume of the object according to the outline of the object further includes: Determining a plurality of positioning feature points corresponding to the contour of the object; According to the neural network, the plurality of positioning feature points are converted into the three-dimensional space; as well as The object is reconstructed and transformed into a shape in the three-dimensional space to calculate the volume of the object.

8. The method for calculating the volume of an ultrasonic object as claimed in claim 7, wherein: The plurality of positioning feature points are at least one of the tilt angle, acceleration and displacement of the probe of the ultrasonic device.

9. An ultrasonic system, characterized in that: Contains: A probe, including a positioning device, is used to collect a plurality of continuous frames of an object at a plurality of time points by an ultrasonic reflection method; as well as A processor is used to detect the object in each of the multiple frames; when a first frame of the multiple frames is detected to have the object, determine a predicted outline, a minimum circumscribed quadrilateral, and a maximum inscribed quadrilateral of the object in the first frame based on a neural network; perform a convolution operation on an intermediate feature layer between the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral of the object, and determine a contour of the object according to the predicted outline and the intermediate feature layer; and convert the object into a three-dimensional space according to the outline of the object to calculate the volume of the object; The multiple frames are two-dimensional images.

10. The ultrasonic system according to claim 9, characterized in that The processor is used for segmenting the object from the first frame to display the object on a display.

11. The ultrasonic system according to claim 9, wherein: The processor is used for determining the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral corresponding to the object in the first frame according to the predicted contour.

12. The ultrasonic system according to claim 9, wherein: The processor is used to determine a third frame that is between the first frame and the second frame in terms of timing according to the first frame and the second frame among the plurality of frames; Predicting the position and size of the object in the third frame using the largest circumscribed quadrilateral of the object in the first frame and the largest circumscribed quadrilateral of the object in the second frame; And determining whether the object in the third frame is the same object as the objects in the first frame and the second frame.

13. The ultrasonic system according to claim 12, wherein: When the object in the third frame is different from the objects in the first frame and the second frame, the processor re-detects the object in each of the plurality of frames; And when the object in the third frame is the same as the objects in the first frame and the second frame, the processor determines the outline of the object using the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral corresponding to the object in the first frame, and the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral corresponding to the object in the second frame.

14. The ultrasonic system according to claim 9, wherein: The processor performs convolution operation under a convolutional neural network or a UNet-Like neural network architecture to calculate the intermediate feature layer between the minimum circumscribed quadrilateral and the maximum inscribed quadrilateral corresponding to the object.

15. The ultrasonic system according to claim 9, wherein: The processor is used to determine a plurality of positioning feature points corresponding to the contour of the object; and transform the plurality of positioning feature points into the three-dimensional space according to a neural network; and reconstructing and transforming the object into a shape of the three-dimensional space to calculate the volume of the object.

16. The ultrasonic system according to claim 15, wherein: The plurality of positioning feature points are at least one of the tilt angle, acceleration and displacement of the probe.

Images