Three-dimensional reconstruction method of ultrasonic image, computer equipment and storage medium

Abstract

The present application is applicable to the field of image processing technology and provides a method, computer device, and storage medium for three-dimensional reconstruction of ultrasonic images. The method includes: performing global intermittent sampling on the current ultrasonic image frame based on a preset intermittent sampling template for each ultrasonic image frame in turn, and calculating the three-dimensional reconstruction data of the sampling points; determining the target bounding box in the ultrasonic image frame of the current frame according to the template bounding box set corresponding to the scanning part of the ultrasonic image sequence, and calculating the three-dimensional reconstruction data of the key area defined by the target bounding box; determining the three-dimensional reconstruction data of the ultrasonic image frame of the current frame according to the three-dimensional reconstruction data of all sampling points and the three-dimensional reconstruction data of the key area; fusing the three-dimensional reconstruction data of the ultrasonic image frame of the current frame with the three-dimensional fusion data corresponding to the previous ultrasonic image frame to obtain the three-dimensional fusion data corresponding to the ultrasonic image frame of the current frame; while ensuring the accuracy of three-dimensional reconstruction and improving the speed of three-dimensional reconstruction, it can reduce the requirements for the computing power of the hardware equipment.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular relates to a three-dimensional reconstruction method of an ultrasound image, a computer device, and a storage medium. Background Art

[0002] In the medical field, real-time, non-invasive visualization of the human body's internal tissues is crucial for clinical work. Ultrasound imaging technology, as a key means of achieving this goal, generates tomographic images (i.e., two-dimensional ultrasound images) of internal tissues by emitting high-frequency sound waves and processing the echo signals reflected by human tissues. This provides medical personnel with intuitive information about the physiological state of internal tissues.

[0003] Traditional two-dimensional ultrasound images often have difficulty clearly displaying tiny tissue abnormalities due to insufficient resolution or overlapping tissues within the human body. Furthermore, it is difficult to discern tissues with complex spatial relationships from two-dimensional ultrasound images. Three-dimensional reconstruction of two-dimensional ultrasound images is usually required to accurately understand the spatial relationships of tissues. However, existing three-dimensional reconstruction methods for ultrasound images have high computational complexity and demanding hardware requirements, making them difficult to popularize in clinical practice. Furthermore, as the size and resolution of ultrasound scanning equipment continue to increase, the higher computational complexity can easily lead to slower three-dimensional reconstruction speeds and higher latency. Summary of the Invention

[0004] In view of this, an embodiment of the present application provides a three-dimensional reconstruction method of an ultrasound image, a computer device and a storage medium to solve the technical problems of the existing three-dimensional reconstruction method of an ultrasound image, such as high computational complexity, strict requirements on hardware equipment, slow three-dimensional reconstruction speed and high latency.

[0005] In a first aspect, an embodiment of the present application provides a method for three-dimensional reconstruction of an ultrasound image, comprising:

[0006] For each frame of ultrasound image in the ultrasound image sequence, global intermittent sampling is performed on the current frame of ultrasound image based on a preset intermittent sampling template, and three-dimensional reconstruction data of each sampling point is calculated;

[0007] Determining a target bounding box in the current frame of the ultrasound image based on a set of template bounding boxes corresponding to the scanned parts of the ultrasound image sequence, and calculating three-dimensional reconstruction data of a key area defined by the target bounding box;

[0008] Determining the three-dimensional reconstruction data of the current frame ultrasound image according to the three-dimensional reconstruction data of all the sampling points and the three-dimensional reconstruction data of the key area;

[0009] The three-dimensional reconstructed data of the current frame ultrasound image is fused with the three-dimensional fusion data corresponding to the previous frame ultrasound image to obtain the three-dimensional fusion data corresponding to the current frame ultrasound image; the three-dimensional fusion data corresponding to the last frame ultrasound image in the ultrasound image sequence is the three-dimensional ultrasound model corresponding to the ultrasound image sequence.

[0010] In an optional implementation of the first aspect, before determining the target bounding box in the current frame of the ultrasound image based on the template bounding box set corresponding to the scanned part of the ultrasound image sequence, the method further includes:

[0011] Acquire a sample image set corresponding to each scanned part; the sample image set is composed of a plurality of sample ultrasound images marked with target pixels;

[0012] Determining a bounding box corresponding to a target pixel in each of the sample ultrasound images;

[0013] For each of the scanned parts, clustering is performed on the bounding boxes corresponding to all target pixels in the sample image set corresponding to the scanned part to obtain a set of template bounding boxes corresponding to the scanned part.

[0014] In an optional implementation of the first aspect, determining a target bounding box in the current frame of the ultrasound image according to a set of template bounding boxes corresponding to a scanned portion of the current frame of the ultrasound image includes:

[0015] For the first frame of ultrasound image, a preset target detection model is used to perform target detection on the first frame of ultrasound image to obtain a candidate bounding box in the first frame of ultrasound image, and a template bounding box that is most similar to the candidate bounding box in the template bounding box set is determined as the target bounding box in the first frame of ultrasound image;

[0016] For the i Frame ultrasound image, according to i -1 frame of the target bounding box in the ultrasound image to determine the i The target bounding box in the frame ultrasound image; i is an integer greater than 1.

[0017] In an optional implementation of the first aspect, for i Frame ultrasound image, according to i -1 frame of the target bounding box in the ultrasound image to determine the i The target bounding box in the frame ultrasound image includes:

[0018] For the i Frame ultrasound image, in the i When there is a target pixel in a preset area outside the initial bounding box in the frame ultrasound image, the first pixel is determined based on the initial bounding box and the preset area. iThe initial bounding box is the observation bounding box in the first frame ultrasound image; wherein the initial bounding box is the i The target bounding box in the -1 frame ultrasound image is mapped to the i a bounding box in a frame ultrasound image;

[0019] According to i The observation bounding box and the initial bounding box in the frame ultrasound image are determined i a first state vector of an observation bounding box in the frame ultrasound image;

[0020] The Kalman filter algorithm is used to process the first state vector to obtain the i Target bounding box in the frame ultrasound image.

[0021] In an optional implementation of the first aspect, calculating the three-dimensional reconstruction data of each sampling point includes:

[0022] Determining the three-dimensional coordinates of each sampling point in the world coordinate system according to the two-dimensional pixel coordinates of each sampling point through the space transformation matrix corresponding to the current frame ultrasound image;

[0023] Determining the 3D voxel index corresponding to each sampling point according to the 3D coordinates of the starting point of the preset 3D voxel grid and the preset voxel resolution, and determining the pixel value of each sampling point as the voxel value corresponding to each sampling point;

[0024] For each of the sampling points, a vector consisting of the three-dimensional voxel index and the voxel value of the sampling point is determined as the three-dimensional reconstruction data of the sampling point.

[0025] In an optional implementation of the first aspect, calculating three-dimensional reconstruction data of a key area defined by the target bounding box includes:

[0026] Determining the three-dimensional coordinates of each pixel in the key area in the world coordinate system according to the two-dimensional pixel coordinates of each pixel in the key area by using the space transformation matrix corresponding to the current frame ultrasound image;

[0027] Determining, based on the three-dimensional coordinates of a starting point of a preset three-dimensional voxel grid and a preset voxel resolution, a three-dimensional voxel index corresponding to each pixel in the key area, and determining a pixel value of each pixel in the key area as a voxel value corresponding to each pixel in the key area;

[0028] For each pixel in the key area, determining a vector consisting of a 3D voxel index and a voxel value corresponding to the pixel as 3D reconstruction data of the pixel;

[0029] A combination of the three-dimensional reconstruction data of all pixels in the key area is determined as the three-dimensional reconstruction data of the key area.

[0030] In an optional implementation of the first aspect, the method further includes:

[0031] For each frame of ultrasound image in the ultrasound image sequence, while one thread is used to determine the 3D reconstructed data of the current frame of ultrasound image, another thread is used to interpolate the grids in the preset 3D voxel grid except for the 3D fusion data corresponding to the previous frame of ultrasound image;

[0032] Correspondingly, the 3D reconstructed data of the current frame ultrasound image is fused with the 3D fused data corresponding to the previous frame ultrasound image to obtain the 3D fused data corresponding to the current frame ultrasound image, including:

[0033] Based on the principle of using 3D reconstructed data to cover interpolated data under the same voxel index, the 3D reconstructed data of the current frame ultrasound image is fused with the 3D fused data corresponding to the previous frame ultrasound image after interpolation to obtain the 3D fused data corresponding to the current frame ultrasound image.

[0034] In a second aspect, an embodiment of the present application provides a computer device, including:

[0035] a first calculation unit, configured to perform global intermittent sampling on each frame of ultrasound image in the ultrasound image sequence based on a preset intermittent sampling template, and calculate three-dimensional reconstruction data of each sampling point;

[0036] a second calculation unit, configured to determine a target bounding box in the current frame of the ultrasound image according to a set of template bounding boxes corresponding to the scanning parts of the ultrasound image sequence, and calculate three-dimensional reconstruction data of a key area defined by the target bounding box;

[0037] a first determining unit, configured to determine the three-dimensional reconstruction data of the current frame ultrasound image based on the three-dimensional reconstruction data of all the sampling points and the three-dimensional reconstruction data of the key area;

[0038] The data fusion unit is used to fuse the three-dimensional reconstructed data of the current frame ultrasound image with the three-dimensional fusion data corresponding to the previous frame ultrasound image to obtain the three-dimensional fusion data corresponding to the current frame ultrasound image; the three-dimensional fusion data corresponding to the last frame ultrasound image in the ultrasound image sequence is the three-dimensional ultrasound model corresponding to the ultrasound image sequence.

[0039] In a third aspect, an embodiment of the present application provides another computer device, comprising a memory and a computer program stored in the memory and executable on a processor, wherein when the processor executes the computer program, the method described in any optional implementation of the first aspect is implemented.

[0040] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the three-dimensional reconstruction method of the ultrasound image as described in any optional implementation of the first aspect above.

[0041] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a computer device, the computer device implements the three-dimensional reconstruction method of the ultrasound image described in any optional implementation manner of the first aspect.

[0042] The implementation of the three-dimensional reconstruction method of ultrasound images, computer equipment, computer-readable storage medium, and computer program product provided in the embodiments of the present application has the following beneficial effects:

[0043] The three-dimensional reconstruction method of ultrasound images provided in the embodiment of the present application can achieve global coarse three-dimensional reconstruction of the ultrasound image by performing global intermittent sampling of the ultrasound image based on a preset intermittent sampling template for each frame of the ultrasound image sequence and calculating the three-dimensional reconstruction data of each sampling point; and can achieve fine three-dimensional reconstruction of the key area by determining the target bounding box in the ultrasound image and calculating the three-dimensional reconstruction data of the key area defined by the target bounding box. Compared with the existing technology that reconstructs all pixels in the ultrasound image, the embodiment of the present application adopts a three-dimensional reconstruction method that combines global coarse reconstruction and local fine phase reconstruction. It can not only ensure the accuracy of the three-dimensional reconstruction, but also reduce the computational complexity of the three-dimensional reconstruction, thereby reducing the requirements for the computing power of the hardware equipment and improving the feasibility of clinical application. At the same time, it can also increase the speed of three-dimensional reconstruction, which is conducive to real-time three-dimensional reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0045] Figure 1 A schematic flowchart of a three-dimensional reconstruction method of an ultrasound image provided in an embodiment of the present application;

[0046] Figure 2 A schematic diagram of a process for determining three-dimensional reconstruction data provided in an embodiment of the present application;

[0047] Figure 3A schematic diagram of a calculation process for three-dimensional reconstruction data provided in another embodiment of the present application;

[0048] Figure 4 A schematic diagram of a process for determining a template bounding box set provided in an embodiment of the present application;

[0049] Figure 5 A schematic diagram of a target bounding box determination process provided in an embodiment of the present application;

[0050] Figure 6 A schematic diagram of a process for determining a target bounding box provided in another embodiment of the present application;

[0051] Figure 7 A schematic diagram of the structure of a computer device provided in an embodiment of the present application;

[0052] Figure 8 A schematic structural diagram of a computer device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0053] The following embodiments are only used to more clearly illustrate the technical solutions of the present application and are therefore only used as examples and are not intended to limit the scope of protection of the present application.

[0054] In the description of the embodiments of the present application, the technical terms "include", "comprise", "have" and any variations thereof mean "including but not limited to", unless otherwise specifically emphasized. In the description of the embodiments of the present application, unless otherwise specified, the technical term "multiple" refers to two or more than two, and the technical terms "at least one" and "one or more" refer to one, two or more. The technical terms "first" and "second" are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. The technical term "and / or" is merely a description of the association relationship of associated objects, indicating that there can be three relationships, such as A and / or B, which can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the previous and subsequent associated objects are in an "or" relationship.

[0055] Related art methods for 3D reconstruction of ultrasound images typically reconstruct all pixels in each frame, resulting in high computational complexity and requiring high computing power from the hardware. Furthermore, as the size and resolution of ultrasound scanners (and, consequently, ultrasound images) continue to increase, existing 3D reconstruction methods require processing more pixels for each frame, resulting in slower reconstruction speeds and higher latency, making real-time reconstruction of ultrasound images impossible.

[0056] In view of this, the present embodiment first provides a method for 3D reconstruction of ultrasound images. The execution entity of this 3D reconstruction method may be a computer device. For example, the computer device may include, but is not limited to, electronic devices such as a desktop computer, a laptop computer, or a tablet computer. The present embodiment does not specifically limit the specific type of computer device.

[0057] Figure 1 This is a schematic flow chart of a method for three-dimensional reconstruction of an ultrasound image provided in an embodiment of the present application. Figure 1 As shown, the three-dimensional reconstruction method of the ultrasound image may include S101 to S104, which are described in detail as follows:

[0058] S101 , for each frame of ultrasound image in the ultrasound image sequence, global intermittent sampling is performed on the current frame of ultrasound image based on a preset intermittent sampling template, and three-dimensional reconstruction data of each sampling point is calculated.

[0059] The ultrasound image sequence may include multiple frames of two-dimensional ultrasound images arranged in chronological order. In practical applications, each frame of the ultrasound image sequence may be transmitted to a computer device in real time, frame by frame, during scanning of a target area (e.g., the abdomen) by an ultrasound scanner.

[0060] Based on this, in some application scenarios, the computer device can perform three-dimensional reconstruction of the current frame of ultrasound image every time it receives a frame of ultrasound image, so that real-time three-dimensional reconstruction of the ultrasound image can be achieved while the ultrasound scanning device scans the target area.

[0061] In other application scenarios, the computer device may also store the received ultrasound image sequence so that each frame of the ultrasound image in the pre-stored ultrasound image sequence can be reconstructed in three dimensions when needed. The three-dimensional reconstruction process of the ultrasound image is as described in S101 to S103.

[0062] The preset gap sampling template is a two-dimensional regularized sparse sampling template used for global sampling of ultrasound images. It can be formed by evenly setting sampling points at preset intervals in the horizontal and vertical directions of the ultrasound image, forming a two-dimensional regularized sparse sampling network covering the ultrasound scanning area in the image. The ultrasound scanning area in the image can refer to the area mapped by the scanning probe of the ultrasound scanning device in the image.

[0063] The horizontal and vertical spacings between adjacent sampling points in the preset gap sampling template are both preset intervals. The preset intervals can be set according to actual needs. For example, the preset interval can be m pixels, that is, there is a sampling point every m pixels in the horizontal and vertical directions of the image.

[0064] For example, Figure 2 This is a schematic diagram of a three-dimensional reconstruction process of an ultrasound image provided in an embodiment of the present application. Figure 2 As shown, assuming Figure 2 21 in the ultrasound image sequence is any frame of ultrasound image. Then the trapezoidal region in the ultrasound image 21 is the ultrasound scanning region. After performing global intermittent sampling on the ultrasound image using the preset intermittent sampling template, the following can be obtained: Figure 2 The multiple sampling points shown in 22 may each include one or more pixels.

[0065] Optionally, after the computer device obtains multiple sampling points in the ultrasound image, it can calculate the three-dimensional reconstruction data of each sampling point by following steps 1.1 to 1.3, as detailed below:

[0066] Step 1.1: According to the two-dimensional pixel coordinates of each sampling point, the three-dimensional coordinates of each sampling point in the world coordinate system are determined by using the space transformation matrix corresponding to the current frame ultrasound image.

[0067] The two-dimensional pixel coordinates of the sampling point may refer to the coordinates of the sampling point in the image coordinate system.

[0068] The spatial transformation matrix corresponding to the current frame of the ultrasound image can be determined based on the spatial position of the ultrasound scanning device corresponding to the current frame of the ultrasound image. It is understandable that since the spatial position of the ultrasound scanning device changes in real time, the spatial transformation matrices corresponding to different frames of ultrasound images are generally different.

[0069] For example, the spatial transformation matrix can be a 4×4 homogeneous transformation matrix, which can be expressed as M=[R|T]. Here, R is a 3×3 matrix used to describe the spatial posture of the ultrasound scanning device, and T is a 3×1 matrix used to describe the three-dimensional coordinates of the origin of the ultrasound image in the world coordinate system.

[0070] Optionally, the computer device can first determine the three-dimensional coordinates of each sampling point in the spatial coordinate system corresponding to the ultrasonic scanning device based on the two-dimensional pixel coordinates of each sampling point and the probe parameters of the ultrasonic scanning device; and then determine the three-dimensional coordinates of each sampling point in the world coordinate system based on the three-dimensional coordinates of each sampling point in the spatial coordinate system corresponding to the ultrasonic scanning device and the spatial transformation matrix corresponding to the current frame ultrasound image.

[0071] Exemplarily, the three-dimensional coordinates of each sampling point in the world coordinate system may be the matrix product of the space transformation matrix and the three-dimensional coordinates of the sampling point in the space coordinate system corresponding to the ultrasonic scanning device.

[0072] Step 1.2: Determine the 3D voxel index corresponding to each sampling point based on the 3D coordinates of the starting point of the preset 3D voxel grid and the preset voxel resolution, and determine the pixel value of each sampling point as the voxel value corresponding to each sampling point.

[0073] The three-dimensional coordinates of the starting point of the preset three-dimensional voxel grid may refer to the three-dimensional coordinates of the starting point in the world coordinate system. The preset voxel resolution may be used to describe the distance between each two adjacent voxels.

[0074] Optionally, for each sampling point, the computer device can calculate the three-dimensional difference between the three-dimensional coordinates of the sampling point in the world coordinate system and the three-dimensional coordinates of the starting point of the three-dimensional voxel network, and determine the ratio of the three-dimensional difference to the preset voxel resolution as the three-dimensional voxel index corresponding to the sampling point, and determine the pixel value of the sampling point in the ultrasound image as the voxel value corresponding to the sampling point.

[0075] Step 1.3: For each sampling point, a vector consisting of the 3D voxel index and voxel value corresponding to the sampling point is determined as the 3D reconstruction data of the sampling point.

[0076] For example, assuming that the 3D voxel index corresponding to any sampling point is ( d , j , k ), the corresponding voxel value is v , then the 3D reconstruction data of the sampling point can be [ d , j , k , v ].

[0077] The calculation process of the three-dimensional reconstruction data corresponding to the above steps 1.1 to 1.3 can be as follows: Figure 3 shown.

[0078] S102 : determining a target bounding box in the current frame ultrasound image according to a set of template bounding boxes corresponding to the scanning parts of the ultrasound image sequence, and calculating 3D reconstruction data of a key area defined by the target bounding box.

[0079] Each scan region may correspond to a template bounding box set. For example, the scan regions may include, but are not limited to, the abdomen, chest, neck, and axilla. The template bounding box set corresponding to each scan region may consist of one or more representative bounding boxes under the scan region. These bounding boxes are used to describe the area where lesions most commonly appear in the ultrasound image of the scan region.

[0080] Based on this, before determining the target bounding box in each frame of ultrasound image, the computer device can determine the template bounding box set corresponding to each scanned part through steps 2.1 to 2.3, as detailed below:

[0081] Step 2.1: Obtain a sample image set corresponding to each scanned part.

[0082] The sample image set may be composed of several sample ultrasound images with target pixels marked. For example, the target pixels may be pixels in the area where the lesions (such as nodules, calcifications or cysts) are located in the ultrasound image. For example, Figure 4 As shown, the sample image set 41 may include a plurality of sample ultrasound images 411 with target pixels marked (ie, pixels marked with red lines in the image).

[0083] Optionally, the target pixels in the sample ultrasound image may be manually annotated or automatically annotated by a computer device. The embodiment of the present application does not limit the specific method for annotating the target pixels.

[0084] Exemplarily, for each scanning part, the computer device can randomly obtain several sample ultrasound images from all historical ultrasound images corresponding to the scanning part, mark the target pixels in each sample ultrasound image, and determine the image set consisting of all sample ultrasound images marked with target pixels as the sample image set corresponding to the scanning part.

[0085] Step 2.2: Determine the bounding box corresponding to the target pixel in each sample ultrasound image.

[0086] The bounding box corresponding to the target pixel refers to the smallest rectangular box that can completely surround all target pixels and is parallel to the image coordinate axis. That is, one pair of parallel sides of the bounding box corresponding to the target pixel can be parallel to the horizontal side of the image, and the other pair of parallel sides can be parallel to the vertical side of the image. For example, please continue to refer to Figure 4 The bounding box of the target pixel can be a rectangular box in each frame of the sample ultrasound image 411.

[0087] In step 2.3, for each scanned part, cluster the bounding boxes corresponding to all target pixels in the sample image set corresponding to the scanned part to obtain a set of template bounding boxes corresponding to the scanned part.

[0088] Optionally, after obtaining the bounding box corresponding to the target pixel in each ultrasound image, the computer device can use a preset clustering algorithm to cluster the bounding boxes corresponding to all target pixels in the sample image set corresponding to the scanned part for each scanned part, thereby obtaining one or more representative template bounding boxes. The one or more representative template bounding boxes constitute the template bounding box set corresponding to the scanned part. For example, please continue to refer to Figure 4 , the template bounding box set corresponding to each scan part can be Figure 4 42 of them.

[0089] Exemplarily, the preset clustering algorithm can be a K-Means clustering algorithm or another clustering algorithm. The specific type of the preset clustering algorithm is not limited in this embodiment of the application. The K in the K-Means clustering algorithm can refer to the desired number of clustering results. Based on this, for each scanned area, when the bounding boxes corresponding to all target pixels in the sample image set corresponding to that scanned area are clustered using the K-Means clustering algorithm, a maximum of K representative template bounding boxes can be obtained. K can be set based on actual conditions, and this embodiment of the application does not limit its specific value.

[0090] After obtaining the template bounding box sets corresponding to the respective scanned parts, the computer device may store the template bounding box sets corresponding to the respective scanned parts in a local memory or a cloud memory so as to be subsequently applied in a three-dimensional reconstruction process of the ultrasound image.

[0091] Based on this, when the computer device needs to determine the target bounding box in the ultrasound image, it can obtain the template bounding box set corresponding to the scanning part of the ultrasound image from the local memory or cloud memory, and then determine the target bounding box in the ultrasound image based on the template bounding box set.

[0092] In an optional implementation, the computer device may determine the target bounding box in the ultrasound image through steps 3.1 and 3.2, as detailed below:

[0093] Step 3.1: For the first frame of ultrasound image, a preset target detection model is used to perform target detection on the first frame of ultrasound image to obtain a candidate bounding box in the first frame of ultrasound image, and the template bounding box that is most similar to the candidate bounding box is set in the template bounding box set and determined as the target bounding box in the first frame of ultrasound image.

[0094] Exemplarily, the preset target detection model can adopt an architecture such as convolutional neural networks (CNN), feature pyramid networks (FPN), YOLO (you only look once) network or retina network (RetinaNet). The embodiment of the present application does not limit the specific architecture of the preset target detection model.

[0095] After performing target detection on the first frame of ultrasound image using a preset target detection model, a candidate bounding box in the first frame of ultrasound image can be obtained. For example, the candidate bounding box can be represented by the two-dimensional pixel coordinates, width, and height of the center point of the candidate bounding box. The two-dimensional pixel coordinates of the center point can refer to the coordinates of the center point in the image coordinate system. The image coordinate system can be a vertex with the upper left corner of the image as the origin, and the two edges of the image intersecting at the vertex are respectively x Axis and y The plane rectangular coordinate system of the axis.

[0096] Based on this, the computer device can match the candidate bounding box in the first frame of the ultrasound image with each template bounding box in the corresponding template bounding box set to calculate the similarity between the candidate bounding box and each template bounding box, and determine the template bounding box with the greatest similarity to the candidate bounding box as the template bounding box of the first frame of the image. Exemplarily, the above similarity can be represented by an intersection-and-union ratio (the ratio of the intersection area to the union area), that is, the computer device can calculate the intersection-and-union ratio of the candidate bounding box with each template bounding box, and determine the template bounding box with the greatest intersection-and-union ratio with the candidate bounding box as the template bounding box of the first frame of the image. Optionally, the above similarity can also be represented by other parameters, and the embodiment of the present application does not limit the specific calculation method of the above similarity.

[0097] For example, Figure 5 As shown, it is assumed that the computer device uses a preset target detection model to perform target detection on the first frame of ultrasound image 51, and obtains the following Figure 5 The computer device may match the candidate bounding box with the corresponding template bounding box set, and determine the bounding box in the template bounding box set that is most similar to the candidate bounding box as the target bounding box in the first frame of the ultrasound image (e.g., Figure 5 53 in FIG).

[0098] Step 3.2, for i Frame ultrasound image, according to i -1 frame of the target bounding box in the ultrasound image to determine the i The target bounding box in the frame ultrasound image; i is an integer greater than 1.

[0099] Since the computer device has determined the template bounding box of the first frame of ultrasound image based on the corresponding template bounding box set, in order to reduce the amount of calculation and increase the speed of determining the target bounding box, for each frame of ultrasound image after the first frame of ultrasound image, the computer device can adjust the target bounding box in the previous frame of ultrasound image to obtain the target bounding box in the current frame of ultrasound image.

[0100] In an optional implementation, the computer device can determine the first frame of ultrasound image after the first frame through steps 3.21 to 3.23. i The target bounding box in the frame ultrasound image is detailed as follows:

[0101] Step 3.21, for i Frame ultrasound image, in the i When there is a target pixel in the preset area outside the initial bounding box in the frame ultrasound image, the first pixel is determined based on the initial bounding box and the preset area. i The observation bounding box in the frame ultrasound image; wherein the initial bounding box is the first i The target bounding box in the -1 frame ultrasound image is mapped to the i Bounding box in the frame ultrasound image.

[0102] For the i Frame ultrasound image, the computer device can first i Frame ultrasound image to determine a i -1 frame of ultrasound image, the target bounding box is exactly the same as the bounding box, which is used as the i An initial bounding box of the frame ultrasound image is formed, and then it is detected whether there is a target pixel in a preset area outside the initial bounding box.

[0103] The preset area outside the initial bounding box may be obtained by expanding each side of the initial bounding box by a preset number of pixels in both directions along a direction perpendicular to the side.

[0104] For example, Figure 6 As shown, assuming Figure 6 The white rectangle in the middle is i For the initial bounding box in the frame ultrasound image, the red rectangular box area obtained by expanding the upper side of the initial bounding box by a preset number of pixels to both sides along the direction perpendicular to the upper side can be the first preset area corresponding to the upper side; similarly, the red rectangular box area obtained by expanding the lower side of the initial bounding box by a preset number of pixels to both sides along the direction perpendicular to the lower side can be the second preset area corresponding to the lower side; the red rectangular box area obtained by expanding the left side of the initial bounding box by a preset number of pixels to both sides along the direction perpendicular to the left side can be the third preset area corresponding to the left side; the red rectangular box area obtained by expanding the right side of the initial bounding box by a preset number of pixels to both sides along the direction perpendicular to the right side can be the fourth preset area corresponding to the right side. The area set consisting of the first preset area, the second preset area, the third preset area and the fourth preset area is the first preset area. i A preset area outside the initial bounding box in the frame ultrasound image.

[0105] The computer device can use a preset target detection model to detect the iWhether there is a target pixel in a preset area outside the initial bounding box in the frame ultrasound image. i If there is a target pixel in any preset area outside the initial bounding box in the frame ultrasound image, it indicates that the i The target bounding box in the frame ultrasound image is relative to the i -1 frame image has changed, the computer device can determine a rectangular frame that can completely surround the initial bounding box and the preset area where the target pixel exists, and use the rectangular frame as the first bounding box. i Frame ultrasound image in the observation bounding box. i If there is no target pixel in all the preset areas outside the initial bounding box in the frame ultrasound image, it means that the i The target bounding box in the frame ultrasound image is relative to the i The target bounding box in the -1 frame image has not changed. At this time, the computer device can determine the initial bounding box as the first i Observation bounding box in the frame ultrasound image.

[0106] Please continue reading Figure 6 , assuming that the target pixel exists in the first preset area, the second preset area, the third preset area, and the fourth preset area, the computer device may use the smallest rectangular frame ( Figure 6 The green rectangle in the middle is identified as i Observation bounding box in the frame ultrasound image.

[0107] Step 3.22, according to i The observation bounding box and the initial bounding box in the frame ultrasound image are used to determine the i The first state vector of the observation bounding box in the frame ultrasound image.

[0108] For example, the first state vector can be obtained by i The geometric information and state variables of the observation bounding box in the frame ultrasound image are represented. The geometric information of the observation bounding box may include the two-dimensional pixel coordinates, width and height of the center point of the observation bounding box. The state variables of the observation bounding box can be used to describe the i The observation bounding box in the frame ultrasound image is relative to the initial bounding box (i.e. i -1 frame ultrasound image) of the target bounding box. The state variables of the observation bounding box may include the coordinate change rate of the center point of the observation bounding box, the width change rate, and the height change rate. For example, assuming that i The observation bounding box in the frame ultrasound image is obtained by ( x 1, y 1,w 1, h 1) indicates that the initial bounding box is obtained by ( x 2, y 2, w 2, h 2) indicates that the i The state variable of the observation bounding box in the frame ultrasound image can be expressed as ( dx , dy , dw , dh ). Based on this, i The first state vector of the observation bounding box in the frame ultrasound image can be expressed as s r =[ x 1, y 1, w 1, h 1, dx , dy , dw , dh ].

[0109] in, x 1 and y 1 respectively i The horizontal and vertical coordinates of the center point of the observation bounding box in the frame ultrasound image in the image coordinate system, w 1 is the i The width of the observation bounding box in the frame ultrasound image, h 1 is the i The height of the observation bounding box in the frame ultrasound image; x 2 and y 2 are respectively i The horizontal and vertical coordinates of the center point of the initial bounding box in the frame ultrasound image in the image coordinate system, w 2nd is the i The width of the initial bounding box in the frame ultrasound image, h 2nd is the i The height of the initial bounding box in the frame ultrasound image.

[0110] Step 3.23, use Kalman filter algorithm to process the first state vector to obtain the i Target bounding box in the frame ultrasound image.

[0111] Optionally, the computer device may first i -1 frame of ultrasound image, the target state vector and the preset state transfer matrix of the target bounding box, predict the i The second state vector of the predicted bounding box in the frame ultrasound image is then iThe first state vector of the observation bounding box in the frame image, the second state vector of the prediction bounding box and the preset Kalman gain coefficient are used to determine the i The target state vector of the target bounding box in the frame ultrasound image. i The second state vector of the predicted bounding box in the frame image can be obtained by s p =[ x p , y p , w p , h p , dx p , dy p , dw p , dh p ] represents. Optionally, the computer device can calculate the first i The target state vector of the target bounding box in the frame ultrasound image:

[0112] s m = s p + K_z *( s r - H * s p );

[0113] in, s m For the i The target state vector of the target bounding box in the frame ultrasound image, s p For the i The second state vector of the predicted bounding box in the frame image, K_z is the preset Kalman gain coefficient, s r For the i The first state vector of the observation bounding box in the frame image, H is the observation matrix in the Kalman filter algorithm.

[0114] For example, the preset Kalman gain coefficient can be the predicted noise of the prediction bounding box according to the computer device. P and the observation noise of the observation bounding box R Automatically determined. Among them, prediction noise can be used to represent the uncertainty of prediction, and observation noise can be used to represent the uncertainty of observation. RThe smaller it is, the more accurate the observation is, and the larger the preset Kalman gain coefficient is; the prediction noise P The smaller it is, the more accurate the prediction is and the smaller the preset Kalman gain coefficient is.

[0115] For example, i The target state vector of the target bounding box in the frame ultrasound image can be expressed as s m =[ x m , y m , w m , h m , dx m , dy m , dw m , dh m ], where ( x m , y m , w m , h m ) is used to indicate the i The target bounding box in the frame ultrasound image. Among them, ( x m , y m ) is the i The two-dimensional pixel coordinates of the center point of the target bounding box in the frame ultrasound image, w m For the i The width of the target bounding box in the frame ultrasound image, h m For the i The height of the target bounding box in the ultrasound image frame.

[0116] Optionally, the computer device may calculate the three-dimensional reconstruction data of the key area defined by the target bounding box by following steps 4.1 to 4.4, as detailed below:

[0117] Step 4.1: According to the two-dimensional pixel coordinates of each pixel in the key area, the three-dimensional coordinates of each pixel in the key area in the world coordinate system are determined by the space transformation matrix corresponding to the current frame ultrasound image.

[0118] Step 4.2, based on the three-dimensional coordinates of the starting point of the preset three-dimensional voxel grid and the preset voxel resolution, determine the three-dimensional voxel index corresponding to each pixel in the key area, and determine the pixel value of each pixel in the key area as the voxel value corresponding to each pixel in the key area.

[0119] Step 4.3: For each pixel in the key area, a vector consisting of the 3D voxel index and voxel value corresponding to the pixel is determined as the 3D reconstruction data of the pixel.

[0120] In step 4.4, the combination of the three-dimensional reconstruction data of all pixels in the key area is determined as the three-dimensional reconstruction data of the key area.

[0121] It should be noted that the calculation process of the three-dimensional reconstruction data of each pixel in the key area is exactly the same as the calculation process of the three-dimensional reconstruction data of each sampling point mentioned above. Therefore, the specific implementation process of steps 4.1 to 4.4 can refer to the description in steps 1.1 to 1.3 above, and will not be repeated here.

[0122] S103 , determining the three-dimensional reconstruction data of the current frame ultrasound image according to the three-dimensional reconstruction data of all sampling points and the three-dimensional reconstruction data of the key area.

[0123] It can be understood that since some sampling points in the ultrasound image will fall into the key area defined by the target bounding box, the computer device can combine the three-dimensional reconstruction data of the key area with the three-dimensional reconstruction data of all sampling points in the non-key area to determine the three-dimensional reconstruction data of the ultrasound image.

[0124] The non-critical area may be an area outside the critical area in the ultrasound image.

[0125] S104, fusing the three-dimensional reconstructed data of the current frame ultrasound image with the three-dimensional fusion data corresponding to the previous frame ultrasound image to obtain the three-dimensional fusion data corresponding to the current frame ultrasound image; the three-dimensional fusion data corresponding to the last frame ultrasound image in the ultrasound image sequence is the three-dimensional ultrasound model corresponding to the ultrasound image sequence.

[0126] It is understandable that the i The three-dimensional fusion data corresponding to the frame ultrasound image fuses the first frame to the i Frame ultrasound image 3D reconstruction data. i -1 frame of ultrasound image corresponding to the 3D fusion data fused 1 to 3 frames i-1 frame of ultrasound image, and the 3D fusion data corresponding to the last frame of ultrasound image fuses the 3D reconstruction data of the ultrasound images from the 1st frame to the last frame. Therefore, the 3D fusion data corresponding to the last frame of ultrasound image in the ultrasound image sequence is the 3D ultrasound model corresponding to the entire ultrasound image sequence.

[0127] In other embodiments, to address the issue of missing volumetric images in non-scanned areas of a 3D ultrasound model, the computer device may interpolate the grids in a preset 3D voxel grid, excluding the 3D fused data corresponding to the previous ultrasound frame, when calculating the 3D reconstruction data for each ultrasound frame. For example, for each ultrasound frame in an ultrasound image sequence, the computer device may use one thread to determine the 3D reconstruction data for the current ultrasound frame while using another thread to interpolate the grids in the preset 3D voxel grid, excluding the 3D fused data corresponding to the previous ultrasound frame. Because the interpolation process and the 3D reconstruction process of the ultrasound image are performed synchronously through different threads, the 3D reconstruction speed is not affected.

[0128] Based on this, in S104, the 3D reconstructed data of the current frame ultrasound image is fused with the 3D fused data corresponding to the previous frame ultrasound image to obtain the 3D fused data corresponding to the current frame ultrasound image, which may include:

[0129] Based on the principle of using 3D reconstructed data to cover interpolated data under the same voxel index, the 3D reconstructed data of the current frame ultrasound image is fused with the 3D fused data corresponding to the previous frame ultrasound image after interpolation to obtain the 3D fused data corresponding to the current frame ultrasound image.

[0130] Among them, the principle of using three-dimensional reconstruction data to cover interpolation data under the same voxel index means that when there are both interpolation data and three-dimensional reconstruction data under the same voxel index, the voxel value in the three-dimensional reconstruction data corresponding to the voxel index is determined as the actual voxel value of the voxel index.

[0131] As can be seen from the above, the three-dimensional reconstruction method of ultrasound images provided in the embodiment of the present application can achieve global coarse three-dimensional reconstruction of the ultrasound image by performing global intermittent sampling of the ultrasound image based on a preset intermittent sampling template for each frame of the ultrasound image sequence and calculating the three-dimensional reconstruction data of each sampling point; by determining the target bounding box in the ultrasound image and calculating the three-dimensional reconstruction data of the key area defined by the target bounding box, it can achieve fine three-dimensional reconstruction of the key area. Compared with the existing technology that reconstructs all pixels in the ultrasound image, the embodiment of the present application adopts a three-dimensional reconstruction method that combines global coarse reconstruction and local fine phase reconstruction. It can not only ensure the accuracy of the three-dimensional reconstruction; but also reduce the computational complexity of the three-dimensional reconstruction, thereby reducing the requirements for the computing power of the hardware equipment and improving the feasibility of clinical application. At the same time, it can also increase the speed of three-dimensional reconstruction, which is conducive to real-time three-dimensional reconstruction.

[0132] In addition, during the three-dimensional reconstruction process of ultrasound images, when determining the target bounding box in each frame of ultrasound image after the first frame of ultrasound image, the target bounding box in the ultrasound image is updated in real time by introducing the Kalman filter algorithm, which can improve the stability of the three-dimensional reconstruction of the key area defined by the target bounding box.

[0133] By interpolating the 3D space corresponding to the non-scanned areas of the ultrasound image, we can mitigate the issue of missing volumetric images in non-scanned areas of the 3D ultrasound model. Because the interpolation process and the 3D reconstruction of the ultrasound image are performed simultaneously in separate threads, the 3D reconstruction speed is not affected.

[0134] It can be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0135] Based on the three-dimensional reconstruction method of ultrasound images provided in the above embodiment, the present application further provides an embodiment of a computer device that implements the above method embodiment. Figure 7 , is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. For ease of explanation, only the parts related to this embodiment are shown. Figure 7 As shown, the computer device 70 may include: a first calculation unit 701, a second calculation unit 702, a first determination unit 703 and a data fusion unit 704.

[0136] The first calculation unit 701 is configured to perform global intermittent sampling on each frame of ultrasound image in the ultrasound image sequence based on a preset intermittent sampling template, and calculate 3D reconstruction data of each sampling point.

[0137] The second calculation unit 702 is configured to determine a target bounding box in the current frame of the ultrasound image according to a set of template bounding boxes corresponding to the scanning parts of the ultrasound image sequence, and calculate 3D reconstruction data of a key area defined by the target bounding box.

[0138] The first determining unit 703 is configured to determine the three-dimensional reconstruction data of the current frame of the ultrasound image according to the three-dimensional reconstruction data of all the sampling points and the three-dimensional reconstruction data of the key area.

[0139] The data fusion unit 704 is used to fuse the three-dimensional reconstructed data of the current frame ultrasound image with the three-dimensional fusion data corresponding to the previous frame ultrasound image to obtain the three-dimensional fusion data corresponding to the current frame ultrasound image; the three-dimensional fusion data corresponding to the last frame ultrasound image in the ultrasound image sequence is the three-dimensional ultrasound model corresponding to the ultrasound image sequence.

[0140] Optionally, the method further includes a first acquisition unit, a second determination unit, and a clustering unit.

[0141] The first acquisition unit is used to acquire a sample image set corresponding to each scanning part; the sample image set is composed of a plurality of sample ultrasound images marked with target pixels.

[0142] The second determining unit is configured to determine a bounding box corresponding to a target pixel in each of the sample ultrasound images.

[0143] The clustering unit is used to cluster the bounding boxes corresponding to all target pixels in the sample image set corresponding to each scanning part, so as to obtain a template bounding box set corresponding to the scanning part.

[0144] Optionally, the first determining unit 703 is specifically configured to:

[0145] For the first frame of ultrasound image, a preset target detection model is used to perform target detection on the first frame of ultrasound image to obtain a candidate bounding box in the first frame of ultrasound image, and a template bounding box that is most similar to the candidate bounding box in the template bounding box set is determined as the target bounding box in the first frame of ultrasound image;

[0146] For the i Frame ultrasound image, according to i -1 frame of the target bounding box in the ultrasound image to determine the i The target bounding box in the frame ultrasound image; i is an integer greater than 1.

[0147] Optionally, the first determining unit 703 is further configured to:

[0148] For the i Frame ultrasound image, in thei When there is a target pixel in a preset area outside the initial bounding box in the frame ultrasound image, the first pixel is determined based on the initial bounding box and the preset area. i The initial bounding box is the observation bounding box in the first frame ultrasound image; wherein the initial bounding box is the i The target bounding box in the -1 frame ultrasound image is mapped to the i a bounding box in a frame ultrasound image;

[0149] According to i The observation bounding box and the initial bounding box in the frame ultrasound image are determined i a first state vector of an observation bounding box in the frame ultrasound image;

[0150] The Kalman filter algorithm is used to process the first state vector to obtain the i Target bounding box in the frame ultrasound image.

[0151] Optionally, the first calculation unit 701 is specifically configured to:

[0152] Determining the three-dimensional coordinates of each sampling point in the world coordinate system according to the two-dimensional pixel coordinates of each sampling point through the space transformation matrix corresponding to the current frame ultrasound image;

[0153] Determining the 3D voxel index corresponding to each sampling point according to the 3D coordinates of the starting point of the preset 3D voxel grid and the preset voxel resolution, and determining the pixel value of each sampling point as the voxel value corresponding to each sampling point;

[0154] For each of the sampling points, a vector consisting of the three-dimensional voxel index and the voxel value of the sampling point is determined as the three-dimensional reconstruction data of the sampling point.

[0155] Optionally, the second calculating unit 702 is specifically configured to:

[0156] Determining the three-dimensional coordinates of each pixel in the key area in the world coordinate system according to the two-dimensional pixel coordinates of each pixel in the key area by using the space transformation matrix corresponding to the current frame ultrasound image;

[0157] Determining, based on the three-dimensional coordinates of a starting point of a preset three-dimensional voxel grid and a preset voxel resolution, a three-dimensional voxel index corresponding to each pixel in the key area, and determining a pixel value of each pixel in the key area as a voxel value corresponding to each pixel in the key area;

[0158] For each pixel in the key area, determining a vector consisting of a 3D voxel index and a voxel value corresponding to the pixel as 3D reconstruction data of the pixel;

[0159] A combination of the three-dimensional reconstruction data of all pixels in the key area is determined as the three-dimensional reconstruction data of the key area.

[0160] Optionally, the computer device may further include an interpolation unit.

[0161] The interpolation unit is used to determine the three-dimensional reconstruction data of the current frame of ultrasound image for each frame of ultrasound image in the ultrasound image sequence using one thread, and to interpolate the grids in the preset three-dimensional voxel grid except the three-dimensional fusion data corresponding to the previous frame of ultrasound image using another thread.

[0162] Correspondingly, the data fusion unit 704 is specifically configured to:

[0163] Based on the principle of using 3D reconstructed data to cover interpolated data under the same voxel index, the 3D reconstructed data of the current frame ultrasound image is fused with the 3D fused data corresponding to the previous frame ultrasound image after interpolation to obtain the 3D fused data corresponding to the current frame ultrasound image.

[0164] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units as needed, that is, the internal structure of the computer device can be divided into different functional units to complete all or part of the functions described above. The functional units in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of each unit in the above-mentioned computer device can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0165] See also Figure 8 , Figure 8 This is a schematic diagram of the structure of a computer device provided in another embodiment of the present application. Figure 8 As shown, the computer device 8 provided in this embodiment may include: a processor 80, a memory 81, and a computer program 82 stored in the memory 81 and executable on the processor 80, such as a program corresponding to the method for three-dimensional reconstruction of ultrasound images. When the processor 80 executes the computer program 82, the steps of the embodiment of the method for three-dimensional reconstruction of ultrasound images are implemented, such as Figure 1 Alternatively, the processor 80 executes the computer program 82 to implement the functions of each unit in the above-mentioned computer device embodiment.

[0166] For example, the computer program 82 may be divided into one or more modules / units, one or more modules / units being stored in the memory 81 and executed by the processor 80 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 82 in the computer device 8. For example, the computer program 82 may be divided into a first calculation unit, a second calculation unit, a first determination unit, and a data fusion unit. The specific functions of each unit can be found in Figure 7 The relevant descriptions in the corresponding embodiments are not repeated here.

[0167] Those skilled in the art will understand that Figure 8 This is merely an example of the computer device 8 and does not constitute a limitation of the computer device 8 . The computer device 8 may include more or fewer components than shown in the figure, or may combine certain components or have different components.

[0168] The processor 80 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0169] The memory 81 can be an internal storage unit of the computer device 8, such as the hard disk or memory of the computer device 8. The memory 81 can also be an external storage device of the computer device 8, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, or flash card equipped on the computer device 8. Furthermore, the memory 81 can include both the internal storage unit of the computer device 8 and an external storage device. The memory 81 is used to store computer programs and other programs and data required by the computer device. The memory 81 can also be used to temporarily store data that has been output or is about to be output.

[0170] An embodiment of the present application further provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, each step of the three-dimensional reconstruction method of the ultrasound image in the above method embodiment is implemented.

[0171] An embodiment of the present application provides a computer program product. When the computer program product is run on a computer device, the computer device implements the steps in the above-mentioned various method embodiments.

[0172] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0173] It should be noted that, unless otherwise specified, all technical terms used in the embodiments of this application have the same meanings as those commonly understood by those skilled in the art in the technical field of this application. The technical terms used in the embodiments of this application are only used to explain the specific embodiments of this application and are not intended to limit this application.

[0174] The phrase "embodiment" mentioned in the description of the embodiments of the present application means that the specific features, structures, or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive with other embodiments. It is understood explicitly and implicitly by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0175] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0176] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A three-dimensional reconstruction method of an ultrasound image, characterized in that: include: For each frame of ultrasound image in the ultrasound image sequence, global intermittent sampling is performed on the current frame of ultrasound image based on a preset intermittent sampling template, and three-dimensional reconstruction data of each sampling point is calculated; Determining a target bounding box in the current frame of the ultrasound image based on a set of template bounding boxes corresponding to the scanned parts of the ultrasound image sequence, and calculating three-dimensional reconstruction data of a key area defined by the target bounding box; Determining the three-dimensional reconstruction data of the current frame ultrasound image according to the three-dimensional reconstruction data of all the sampling points and the three-dimensional reconstruction data of the key area; fusing the three-dimensional reconstructed data of the current frame of ultrasound image with the three-dimensional fusion data corresponding to the previous frame of ultrasound image to obtain the three-dimensional fusion data corresponding to the current frame of ultrasound image; the three-dimensional fusion data corresponding to the last frame of ultrasound image in the ultrasound image sequence is the three-dimensional ultrasound model corresponding to the ultrasound image sequence; Determining a target bounding box in the current frame of ultrasound image according to a set of template bounding boxes corresponding to a scanning portion of the current frame of ultrasound image includes: For the first frame of ultrasound image, a preset target detection model is used to perform target detection on the first frame of ultrasound image to obtain a candidate bounding box in the first frame of ultrasound image, and a template bounding box that is most similar to the candidate bounding box in the template bounding box set is determined as the target bounding box in the first frame of ultrasound image; For the i Frame ultrasound image, in the i When there is a target pixel in a preset area outside the initial bounding box in the frame ultrasound image, the first pixel is determined based on the initial bounding box and the preset area. i The initial bounding box is the observation bounding box in the first frame ultrasound image; wherein the initial bounding box is the i The target bounding box in the -1 frame ultrasound image is mapped to the i a bounding box in a frame ultrasound image; According to i The observation bounding box and the initial bounding box in the frame ultrasound image are determined i a first state vector of an observation bounding box in the frame ultrasound image; The Kalman filter algorithm is used to process the first state vector to obtain the i Target bounding box in the frame ultrasound image.

2. The method according to claim 1, characterized in that Before determining the target bounding box in the current frame of the ultrasound image according to the template bounding box set corresponding to the scanning part of the ultrasound image sequence, the method further includes: Acquire a sample image set corresponding to each scanned part; the sample image set is composed of a plurality of sample ultrasound images marked with target pixels; Determining a bounding box corresponding to a target pixel in each of the sample ultrasound images; For each of the scanned parts, clustering is performed on the bounding boxes corresponding to all target pixels in the sample image set corresponding to the scanned part to obtain a set of template bounding boxes corresponding to the scanned part.

3. The method according to claim 1, characterized in that Calculate the 3D reconstruction data of each sampling point, including: Determining the three-dimensional coordinates of each sampling point in the world coordinate system according to the two-dimensional pixel coordinates of each sampling point through the space transformation matrix corresponding to the current frame ultrasound image; Determining the 3D voxel index corresponding to each sampling point according to the 3D coordinates of the starting point of the preset 3D voxel grid and the preset voxel resolution, and determining the pixel value of each sampling point as the voxel value corresponding to each sampling point; For each of the sampling points, a vector consisting of the three-dimensional voxel index and the voxel value of the sampling point is determined as the three-dimensional reconstruction data of the sampling point.

4. The method according to claim 1, wherein Calculating three-dimensional reconstruction data of a key area defined by the target bounding box includes: Determining the three-dimensional coordinates of each pixel in the key area in the world coordinate system according to the two-dimensional pixel coordinates of each pixel in the key area by using the space transformation matrix corresponding to the current frame ultrasound image; Determining, based on the three-dimensional coordinates of a starting point of a preset three-dimensional voxel grid and a preset voxel resolution, a three-dimensional voxel index corresponding to each pixel in the key area, and determining a pixel value of each pixel in the key area as a voxel value corresponding to each pixel in the key area; For each pixel in the key area, determining a vector consisting of a 3D voxel index and a voxel value corresponding to the pixel as 3D reconstruction data of the pixel; A combination of the three-dimensional reconstruction data of all pixels in the key area is determined as the three-dimensional reconstruction data of the key area.

5. The method according to any one of claims 1 to 4, characterized in that Also includes: For each frame of ultrasound image in the ultrasound image sequence, while one thread is used to determine the 3D reconstructed data of the current frame of ultrasound image, another thread is used to interpolate the grids in the preset 3D voxel grid except for the 3D fusion data corresponding to the previous frame of ultrasound image; Correspondingly, the 3D reconstructed data of the current frame ultrasound image is fused with the 3D fused data corresponding to the previous frame ultrasound image to obtain the 3D fused data corresponding to the current frame ultrasound image, including: Based on the principle of using 3D reconstructed data to cover interpolated data under the same voxel index, the 3D reconstructed data of the current frame ultrasound image is fused with the 3D fused data corresponding to the previous frame ultrasound image after interpolation to obtain the 3D fused data corresponding to the current frame ultrasound image.

6. A computer device, characterized in that: include: a first calculation unit, configured to perform global intermittent sampling on each frame of ultrasound image in the ultrasound image sequence based on a preset intermittent sampling template, and calculate three-dimensional reconstruction data of each sampling point; a second calculation unit, configured to determine a target bounding box in the current frame of the ultrasound image according to a set of template bounding boxes corresponding to the scanning parts of the ultrasound image sequence, and calculate three-dimensional reconstruction data of a key area defined by the target bounding box; a first determining unit, configured to determine the three-dimensional reconstruction data of the current frame ultrasound image based on the three-dimensional reconstruction data of all the sampling points and the three-dimensional reconstruction data of the key area; a data fusion unit, configured to fuse the three-dimensional reconstructed data of the current ultrasonic image frame with the three-dimensional fusion data corresponding to the previous ultrasonic image frame to obtain the three-dimensional fusion data corresponding to the current ultrasonic image frame; the three-dimensional fusion data corresponding to the last ultrasonic image frame in the ultrasonic image sequence being the three-dimensional ultrasonic model corresponding to the ultrasonic image sequence; The second computing unit is specifically configured to: For the first frame of ultrasound image, a preset target detection model is used to perform target detection on the first frame of ultrasound image to obtain a candidate bounding box in the first frame of ultrasound image, and a template bounding box that is most similar to the candidate bounding box in the template bounding box set is determined as the target bounding box in the first frame of ultrasound image; For the i Frame ultrasound image, in the i When there is a target pixel in a preset area outside the initial bounding box in the frame ultrasound image, the first pixel is determined based on the initial bounding box and the preset area. i The initial bounding box is the observation bounding box in the first frame ultrasound image; wherein the initial bounding box is the i The target bounding box in the -1 frame ultrasound image is mapped to the i a bounding box in a frame ultrasound image; According to i The observation bounding box and the initial bounding box in the frame ultrasound image are determined i a first state vector of an observation bounding box in the frame ultrasound image; The Kalman filter algorithm is used to process the first state vector to obtain the i Target bounding box in the frame ultrasound image.

7. A computer device, characterized in that: The method comprises a memory and a computer program stored in the memory and executable on a processor, wherein the processor implements the method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images