Three-dimensional reconstruction method of ultrasonic image, computer equipment and storage medium

Abstract

The invention is suitable for the technical field of image processing, and provides a three-dimensional reconstruction method of an ultrasonic image, computer equipment and a storage medium. The method comprises the following steps: sequentially aiming at each frame of ultrasonic image, carrying out global gap sampling on the current frame of ultrasonic image based on a preset gap sampling template, and calculating three-dimensional reconstruction data of sampling points; determining a target bounding box in the current frame of ultrasonic image according to a template bounding box set corresponding to the scanning part of the ultrasonic image sequence, and calculating three-dimensional reconstruction data of a key area limited by the target bounding box; determining the three-dimensional reconstruction data of the current frame of ultrasonic image according to the three-dimensional reconstruction data of all the sampling points and the three-dimensional reconstruction data of the key area; fusing the three-dimensional reconstruction data of the current frame of ultrasonic image into the three-dimensional fusion data corresponding to the previous frame of ultrasonic image to obtain three-dimensional fusion data corresponding to the current frame of ultrasonic image; the three-dimensional reconstruction precision can be guaranteed, the three-dimensional reconstruction speed is improved, and meanwhile the requirement for the computing power of hardware equipment is lowered.

Description

Technical Field

[0001] This application belongs to the technical field of image processing, and particularly relates to a three-dimensional reconstruction method for ultrasonic images, a computer device, and a storage medium. Background Art

[0002] In the medical field, real-time and non-invasive visualization of internal human tissues is crucial for clinical work. As an important means to achieve this goal, ultrasonic imaging technology can generate tomographic images of internal human tissues (i.e., two-dimensional ultrasonic images) by emitting high-frequency acoustic wave signals and processing the echo signals reflected by human tissues, thereby providing intuitive information about the physiological state of internal human tissues for medical staff.

[0003] Traditional two-dimensional ultrasonic images are often difficult to clearly display small tissue abnormalities due to insufficient resolution or overlapping of internal human tissues, and it is difficult to distinguish tissues with complex spatial relationships from two-dimensional ultrasonic images. Usually, three-dimensional reconstruction of two-dimensional ultrasonic images is required to accurately understand the spatial relationships of tissues. However, the existing three-dimensional reconstruction methods for ultrasonic images have a high computational complexity and require relatively demanding hardware devices, making it difficult to popularize in clinics. In addition, with the continuous increase in the size and resolution of ultrasonic scanning devices, the high computational complexity easily leads to a slow three-dimensional reconstruction speed and a high latency. Summary of the Invention

[0004] In view of this, the embodiments of this application provide a three-dimensional reconstruction method for ultrasonic images, a computer device, and a storage medium to solve the technical problems of high computational complexity, relatively demanding hardware devices, slow three-dimensional reconstruction speed, and high latency of the existing three-dimensional reconstruction methods for ultrasonic images.

[0005] In a first aspect, the embodiments of this application provide a three-dimensional reconstruction method for ultrasonic images, including: Successively for each frame of ultrasonic image in the ultrasonic image sequence, globally and intermittently sample the current frame of ultrasonic image based on a preset gap sampling template, and calculate the three-dimensional reconstruction data of each sampling point; Determine the target bounding box in the current frame of ultrasonic image according to the template bounding box set corresponding to the scanned part of the ultrasonic image sequence, and calculate the three-dimensional reconstruction data of the key area defined by the target bounding box; Determine the three-dimensional reconstruction data of the current frame of ultrasonic image according to the three-dimensional reconstruction data of all the sampling points and the three-dimensional reconstruction data of the key area; Fuse the three-dimensional reconstruction data of the current-frame ultrasound image into the three-dimensional fusion data corresponding to the previous-frame ultrasound image to obtain the three-dimensional fusion data corresponding to the current-frame ultrasound image; the three-dimensional fusion data corresponding to the last ultrasound image in the ultrasound image sequence is the three-dimensional ultrasound model corresponding to the ultrasound image sequence.

[0006] In an optional implementation manner of the first aspect, before determining the target bounding box in the current-frame ultrasound image according to the set of template bounding boxes corresponding to the scanned part of the ultrasound image sequence, it further includes: Obtain a set of sample images corresponding to each scanned part; the set of sample images consists of several sample ultrasound images with target pixels marked. Determine the bounding box corresponding to the target pixel in each of the sample ultrasound images. For each scanned part, cluster the bounding boxes corresponding to all target pixels in the set of sample images corresponding to the scanned part to obtain the set of template bounding boxes corresponding to the scanned part.

[0007] In an optional implementation manner of the first aspect, determining the target bounding box in the current-frame ultrasound image according to the set of template bounding boxes corresponding to the scanned part of the current-frame ultrasound image includes: For the first-frame ultrasound image, use a preset target detection model to perform target detection on the first-frame ultrasound image to obtain candidate bounding boxes in the first-frame ultrasound image, and determine the template bounding box in the set of template bounding boxes that is most similar to the candidate bounding box as the target bounding box in the first-frame ultrasound image; For the i -th frame ultrasound image, determine the target bounding box in the i -th frame ultrasound image according to the target bounding box in the i -1-th frame ultrasound image; i is an integer greater than 1.

[0008] In an optional implementation manner of the first aspect, for the i -th frame ultrasound image, determining the target bounding box in the i -th frame ultrasound image according to the target bounding box in the i -1-th frame ultrasound image includes: For the i -th frame ultrasound image, when there are target pixels in the preset area outside the initial bounding box in the i -th frame ultrasound image, determine the observed bounding box in the i -th frame ultrasound image according to the initial bounding box and the preset area; wherein, the initial bounding box is the target bounding box in the i -1-th frame ultrasound image mapped to the iBounding box in the frame ultrasound image; According to the i observed bounding box and the initial bounding box in the frame ultrasound image, determine the i first state vector of the observed bounding box in the frame ultrasound image; Process the first state vector using the Kalman filtering algorithm to obtain the i target bounding box in the frame ultrasound image.

[0009] In an alternative implementation of the first aspect, calculating the three-dimensional reconstruction data of each sampling point includes: According to the two-dimensional pixel coordinates of each sampling point, determine the three-dimensional coordinates of each sampling point in the world coordinate system through the spatial transformation matrix corresponding to the current frame ultrasound image; According to the three-dimensional coordinates of the starting point of the preset three-dimensional voxel grid and the preset voxel resolution, determine the three-dimensional voxel indices corresponding to each sampling point respectively, and determine the pixel values of each sampling point as the voxel values corresponding to each sampling point; For each sampling point, determine the vector composed of the three-dimensional voxel index and the voxel value of the sampling point as the three-dimensional reconstruction data of the sampling point.

[0010] In an alternative implementation of the first aspect, calculating the three-dimensional reconstruction data of the key area defined by the target bounding box includes: According to the two-dimensional pixel coordinates of each pixel in the key area, determine the three-dimensional coordinates of each pixel in the key area in the world coordinate system through the spatial transformation matrix corresponding to the current frame ultrasound image; According to the three-dimensional coordinates of the starting point of the preset three-dimensional voxel grid and the preset voxel resolution, determine the three-dimensional voxel indices corresponding to each pixel in the key area respectively, and determine the pixel values of each pixel in the key area as the voxel values corresponding to each pixel in the key area; For each pixel in the key area, determine the vector composed of the three-dimensional voxel index corresponding to the pixel and the voxel value as the three-dimensional reconstruction data of the pixel; Determine the combination of the three-dimensional reconstruction data of all pixels in the key area as the three-dimensional reconstruction data of the key area.

[0011] In an alternative implementation of the first aspect, it further includes: For each frame ultrasound image in the ultrasound image sequence, while using one thread to determine the three-dimensional reconstruction data of the current frame ultrasound image, use another thread to interpolate the grid in the preset three-dimensional voxel grid except for the three-dimensional fusion data corresponding to the previous frame ultrasound image; Correspondingly, fusing the three-dimensional reconstruction data of the current-frame ultrasound image into the three-dimensional fusion data corresponding to the previous-frame ultrasound image to obtain the three-dimensional fusion data corresponding to the current-frame ultrasound image, including: Based on the principle of using the three-dimensional reconstruction data to cover the interpolation data at the same voxel index, fusing the three-dimensional reconstruction data of the current-frame ultrasound image into the three-dimensional fusion data corresponding to the interpolated previous-frame ultrasound image to obtain the three-dimensional fusion data corresponding to the current-frame ultrasound image.

[0012] In a second aspect, an embodiment of the present application provides a computer device, including: A first calculation unit, configured to sequentially perform global intermittent sampling on each ultrasound image in an ultrasound image sequence based on a preset gap sampling template, and calculate the three-dimensional reconstruction data of each sampling point; A second calculation unit, configured to determine a target bounding box in the current-frame ultrasound image according to a template bounding box set corresponding to the scanned part of the ultrasound image sequence, and calculate the three-dimensional reconstruction data of a key area defined by the target bounding box; A first determination unit, configured to determine the three-dimensional reconstruction data of the current-frame ultrasound image according to the three-dimensional reconstruction data of all the sampling points and the three-dimensional reconstruction data of the key area; A data fusion unit, configured to fuse the three-dimensional reconstruction data of the current-frame ultrasound image into the three-dimensional fusion data corresponding to the previous-frame ultrasound image to obtain the three-dimensional fusion data corresponding to the current-frame ultrasound image; the three-dimensional fusion data corresponding to the last ultrasound image in the ultrasound image sequence is the three-dimensional ultrasound model corresponding to the ultrasound image sequence.

[0013] In a third aspect, an embodiment of the present application provides another computer device, including a memory and a computer program stored in the memory and executable on a processor. When the processor executes the computer program, the method described in any optional implementation manner of the first aspect above is implemented.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the three-dimensional reconstruction method of the ultrasound image described in any optional implementation manner of the first aspect above is implemented.

[0015] In a fifth aspect, an embodiment of the present application provides a computer program product, and when the computer program product runs on a computer device, the computer device is enabled to implement the three-dimensional reconstruction method of the ultrasound image described in any optional implementation manner of the first aspect.

[0016] Implementing the three-dimensional reconstruction method of the ultrasound image, computer device, computer-readable storage medium, and computer program product provided by the embodiments of the present application has the following beneficial effects: The 3D reconstruction method for ultrasonic images provided by the embodiments of the present application, for each frame of ultrasonic image in the ultrasonic image sequence, globally samples the ultrasonic image intermittently based on a preset gap sampling template, and calculates the 3D reconstruction data of each sampling point, which can achieve the global rough 3D reconstruction of the ultrasonic image; by determining the target bounding box in the ultrasonic image and calculating the 3D reconstruction data of the key area defined by the target bounding box, the refined 3D reconstruction of the key area can be achieved. Compared with the prior art that reconstructs all pixels in the ultrasonic image, the 3D reconstruction method provided by the embodiments of the present application combines global rough reconstruction and local refined reconstruction, which can not only ensure the accuracy of 3D reconstruction, but also reduce the computational complexity of 3D reconstruction, thereby reducing the requirement for the computing power of hardware devices and improving the feasibility of clinical applications. At the same time, it can also improve the speed of 3D reconstruction, which is conducive to real-time 3D reconstruction. Description of the Drawings

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0018] Figure 1 It is a schematic flowchart of a 3D reconstruction method for ultrasonic images provided by the embodiments of the present application; Figure 2 It is a schematic diagram of the determination process of 3D reconstruction data provided by the embodiments of the present application; Figure 3 It is a schematic diagram of the calculation process of 3D reconstruction data provided by another embodiment of the present application; Figure 4 It is a schematic diagram of the determination process of a template bounding box set provided by the embodiments of the present application; Figure 5 It is a schematic diagram of the determination process of a target bounding box provided by the embodiments of the present application; Figure 6 It is a schematic diagram of the determination process of a target bounding box provided by another embodiment of the present application; Figure 7 It is a schematic diagram of the structure of a computer device provided by the embodiments of the present application; Figure 8 It is a schematic diagram of the structure of a computer device provided by another embodiment of the present application. Detailed Embodiments

[0019] The following embodiments are only used to more clearly illustrate the technical solutions of the present application, so they are only examples and cannot be used to limit the protection scope of the present application.

[0020] In the description of the embodiments of the present application, technical terms such as "include", "comprise", "have" and any variations thereof all mean "including but not limited to", unless otherwise specifically emphasized in other ways. In the description of the embodiments of the present application, unless otherwise specified, the technical term "a plurality" means two or more, and the technical terms "at least one", "one or more" mean one, two or more than two. Technical terms such as "first" and "second" are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity, specific order or primary-secondary relationship of the indicated technical features. The technical term "and / or" is only a description of the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the front and rear associated objects.

[0021] The three-dimensional reconstruction method of ultrasonic images in the related art usually performs three-dimensional reconstruction on all pixels in each frame of ultrasonic image, so the computational complexity is relatively high, and it requires the hardware device to have a relatively high computing power. Moreover, with the continuous increase in the size and resolution of the ultrasonic scanning device (i.e., the ultrasonic image), the existing three-dimensional reconstruction methods need to process more pixels when performing three-dimensional reconstruction on each frame of ultrasonic image, resulting in a slower three-dimensional reconstruction speed, higher latency, and inability to achieve real-time reconstruction of ultrasonic images.

[0022] In view of this, the embodiments of the present application first provide a three-dimensional reconstruction method for ultrasonic images. The execution subject of the three-dimensional reconstruction method of ultrasonic images can be a computer device. Exemplarily, the computer device may include, but is not limited to, electronic devices such as desktop computers, laptop computers, or tablet computers. The specific type of the computer device in the embodiments of the present application is not particularly limited.

[0023] Figure 1 It is a schematic flowchart of a three-dimensional reconstruction method for ultrasonic images provided by the embodiments of the present application. As Figure 1 shown, the three-dimensional reconstruction method of ultrasonic images may include S101 to S104, which are described in detail as follows: S101, for each frame of ultrasonic image in the ultrasonic image sequence in turn, perform global intermittent sampling on the current frame of ultrasonic image based on a preset gap sampling template, and calculate the three-dimensional reconstruction data of each sampling point.

[0024] Among them, the ultrasonic image sequence may include multiple two-dimensional ultrasonic images arranged in the order of scanning time. In practical applications, each ultrasonic image in the ultrasonic image sequence may be transmitted frame by frame in real time to the computer device by the ultrasonic scanning device during the process of scanning the target part (such as the abdomen).

[0025] Based on this, in some application scenarios, the computer device may perform three-dimensional reconstruction on each received ultrasonic image. In this way, real-time three-dimensional reconstruction of the ultrasonic image can be achieved while the ultrasonic scanning device scans the target part.

[0026] In other application scenarios, the computer device may also store the received ultrasonic image sequence so that the three-dimensional reconstruction can be performed on each ultrasonic image in the pre-stored ultrasonic image sequence when needed later. Among them, the three-dimensional reconstruction process of the ultrasonic image is as described in S101 - S103.

[0027] The preset gap sampling template is a two-dimensional regularized sparse sampling template for global sampling of ultrasonic images. It can be a two-dimensional regularized sparse sampling network formed by uniformly setting sampling points at preset intervals in the horizontal and vertical directions of the ultrasonic image, covering the ultrasonic scanning area in the image. The ultrasonic scanning area in the image may refer to the mapping area of the scanning probe of the ultrasonic scanning device in the image.

[0028] Among them, the distances between adjacent sampling points in the preset gap sampling template are both preset intervals in the horizontal and vertical directions. The preset interval can be set according to actual needs. For example, the preset interval can be m pixels, that is, there is a sampling point every m pixels in the horizontal and vertical directions of the image.

[0029] Exemplarily, Figure 2 is a schematic diagram of the three-dimensional reconstruction process of an ultrasonic image provided by an embodiment of the present application. As Figure 2 shown, assume that Figure 2 21 in is any frame of the ultrasonic image sequence. Then the trapezoid-like area in the ultrasonic image 21 is the ultrasonic scanning area. After performing global gap sampling on the ultrasonic image using the preset gap sampling template, multiple sampling points as shown in Figure 2 22 in can be obtained, and each sampling point may include one or more pixels.

[0030] Optionally, after the computer device obtains multiple sampling points in the ultrasonic image, the three-dimensional reconstruction data of each sampling point can be calculated through the following steps 1.1 - 1.3, which are described in detail as follows: Step 1.1, according to the two-dimensional pixel coordinates of each sampling point, determine the three-dimensional coordinates of each sampling point in the world coordinate system through the spatial transformation matrix corresponding to the current frame of ultrasonic image.

[0031] Among them, the two-dimensional pixel coordinates of the sampling point can refer to the coordinates of the sampling point in the image coordinate system.

[0032] The spatial transformation matrix corresponding to the current frame of ultrasound image can be determined according to the spatial pose of the ultrasound scanning device corresponding to the current frame of ultrasound image. It can be understood that since the spatial pose of the ultrasound scanning device changes in real time, the spatial transformation matrices corresponding to different frames of ultrasound images are usually different.

[0033] Exemplarily, the spatial transformation matrix can be a 4×4 homogeneous transformation matrix, which can be expressed as M = [R|T]. Among them, R is a 3×3 matrix used to describe the spatial pose of the ultrasound scanning device; T is a 3×1 matrix used to describe the three-dimensional coordinates of the origin of the ultrasound image in the world coordinate system.

[0034] Optionally, the computer device can first determine the three-dimensional coordinates of each sampling point in the spatial coordinate system corresponding to the ultrasound scanning device according to the two-dimensional pixel coordinates of each sampling point and the probe parameters of the ultrasound scanning device; then determine the three-dimensional coordinates of each sampling point in the world coordinate system according to the three-dimensional coordinates of each sampling point in the spatial coordinate system corresponding to the ultrasound scanning device and the spatial transformation matrix corresponding to the current frame of ultrasound image.

[0035] Exemplarily, the three-dimensional coordinates of each sampling point in the world coordinate system can be the matrix product of the spatial transformation matrix and the three-dimensional coordinates of the sampling point in the spatial coordinate system corresponding to the ultrasound scanning device.

[0036] Step 1.2: According to the three-dimensional coordinates of the starting point of the preset three-dimensional voxel grid and the preset voxel resolution, determine the three-dimensional voxel indices corresponding to each sampling point respectively, and determine the pixel values of each sampling point as the voxel values corresponding to each sampling point.

[0037] Among them, the three-dimensional coordinates of the starting point of the preset three-dimensional voxel grid can refer to the three-dimensional coordinates of the starting point in the world coordinate system. The preset voxel resolution can be used to describe the distance between every two adjacent voxels.

[0038] Optionally, for each sampling point, the computer device can calculate the three-dimensional difference between the three-dimensional coordinates of the sampling point in the world coordinate system and the three-dimensional coordinates of the starting point of the three-dimensional voxel network, and determine the ratio of the three-dimensional difference to the preset voxel resolution as the three-dimensional voxel index corresponding to the sampling point, and determine the pixel value of the sampling point in the ultrasound image as the voxel value corresponding to the sampling point.

[0039] Step 1.3: For each sampling point, determine the vector composed of the three-dimensional voxel index and the voxel value corresponding to the sampling point as the three-dimensional reconstruction data of the sampling point.

[0040] Exemplarily, assume that the three-dimensional voxel index corresponding to any sampling point is ( d , j , k ), and the corresponding voxel value is v . Then, the three-dimensional reconstruction data of this sampling point can be d , j , k , v .

[0041] The calculation process of the three-dimensional reconstruction data corresponding to the above steps 1.1 to 1.3 can be as Figure 3 shown.

[0042] S102. According to the set of template bounding boxes corresponding to the scanning part of the ultrasonic image sequence, determine the target bounding box in the current frame of ultrasonic image, and calculate the three-dimensional reconstruction data of the key area defined by the target bounding box.

[0043] Among them, each scanning part can correspond to a set of template bounding boxes. Exemplarily, the scanning parts can include but are not limited to the abdomen, chest, neck, armpit, etc. The set of template bounding boxes corresponding to each scanning part can be composed of one or more representative bounding boxes under this scanning part, and these bounding boxes are used to describe the area where the lesions most frequently appear in the ultrasonic images of this scanning part.

[0044] Based on this, before determining the target bounding box in each frame of ultrasonic image, the computer device can determine the set of template bounding boxes corresponding to each scanning part through steps 2.1 to 2.3, which are described in detail as follows: Step 2.1, obtain the set of sample images corresponding to each scanning part respectively.

[0045] Among them, the set of sample images can be composed of several sample ultrasonic images with target pixels marked. Exemplarily, the target pixels can be the pixels in the area where the lesions (such as nodules, calcification spots or cysts, etc.) are located in the ultrasonic image. For example, as Figure 4 shown, the set of sample images 41 can include multiple sample ultrasonic images 411 with target pixels (i.e., the pixels marked by the red lines in the image) marked.

[0046] Optionally, the target pixels in the sample ultrasonic images can be manually marked by humans or automatically marked by the computer device. The present application embodiment does not limit the specific marking method of the target pixels.

[0047] Exemplarily, for each scanned part, the computer device may randomly obtain several sample ultrasound images from all the historical ultrasound images corresponding to the scanned part, label the target pixels in each sample ultrasound image, and determine the image set composed of all the sample ultrasound images with labeled target pixels as the sample image set corresponding to the scanned part.

[0048] Step 2.2: Determine the bounding box corresponding to the target pixels in each sample ultrasound image.

[0049] Among them, the bounding box corresponding to the target pixels refers to the smallest rectangular box that can completely enclose all the target pixels and is parallel to the image coordinate axes. That is, one pair of parallel sides of the bounding box corresponding to the target pixels can be parallel to the horizontal side of the image, and the other pair of parallel sides can be parallel to the vertical side of the image. For example, please continue to refer to Figure 4 , the bounding box of the target pixels can be the rectangular box in each frame of the sample ultrasound image 411.

[0050] Step 2.3: For each scanned part, cluster the bounding boxes corresponding to all the target pixels in the sample image set corresponding to the scanned part to obtain the template bounding box set corresponding to the scanned part.

[0051] Optionally, after obtaining the bounding box corresponding to the target pixels in each ultrasound image, for each scanned part, the computer device may use a preset clustering algorithm to cluster the bounding boxes corresponding to all the target pixels in the sample image set corresponding to the scanned part, so as to obtain one or more representative template bounding boxes, and the one or more representative template bounding boxes constitute the template bounding box set corresponding to the scanned part. Exemplarily, please continue to refer to Figure 4 , the template bounding box set corresponding to each scanned part may be Figure 4 42 in

[0052] Exemplarily, the preset clustering algorithm may be the K-Means clustering algorithm or other clustering algorithms. The embodiments of the present application do not limit the specific type of the preset clustering algorithm. Among them, K in the K-Means clustering algorithm may refer to the number of expected clustering results. Based on this, for each scanned part, when using the K-Means clustering algorithm to cluster the bounding boxes corresponding to all the target pixels in the sample image set corresponding to the scanned part, at most K representative template bounding boxes can be obtained. K can be set according to the actual situation, and the embodiments of the present application do not limit its specific value.

[0053] After obtaining the template bounding box sets corresponding to each scanned part, the computer device can store the template bounding box sets corresponding to each scanned part in a local memory or a cloud memory, so as to be subsequently applied to the 3D reconstruction process of the ultrasound image.

[0054] Based on this, when the computer device needs to determine the target bounding box in the ultrasound image, it can obtain the template bounding box set corresponding to the scanned part of the ultrasound image from the local memory or the cloud memory, and then determine the target bounding box in the ultrasound image according to the template bounding box set.

[0055] In an optional implementation manner, the computer device can determine the target bounding box in the ultrasound image through steps 3.1 to 3.2, which are described in detail as follows: Step 3.1, for the first frame of ultrasound image, use a preset target detection model to perform target detection on the first frame of ultrasound image, obtain the candidate bounding boxes in the first frame of ultrasound image, and determine the template bounding box in the template bounding box set that is most similar to the candidate bounding box as the target bounding box in the first frame of ultrasound image.

[0056] Exemplarily, the preset target detection model can adopt architectures such as convolutional neural networks (CNN), feature pyramid networks (FPN), YOLO (you only look once) network, or retina network (RetinaNet). The embodiments of the present application do not limit the specific architecture of the preset target detection model.

[0057] After performing target detection on the first frame of ultrasound image using the preset target detection model, the candidate bounding boxes in the first frame of ultrasound image can be obtained. Exemplarily, the candidate bounding box can be represented by the two-dimensional pixel coordinates, width, and height of the center point of the candidate bounding box. Among them, the two-dimensional pixel coordinates of the center point can refer to the coordinates of the center point in the image coordinate system. The image coordinate system can be a plane rectangular coordinate system with the upper left vertex of the image as the origin and the two sides of the image that intersect at this vertex as the x axis and y axis.

[0058] Based on this, the computer device can match the candidate bounding box in the first-frame ultrasound image with each template bounding box in the corresponding template bounding box set to calculate the similarity between the candidate bounding box and each template bounding box, and determine the template bounding box with the maximum similarity to the candidate bounding box as the template bounding box of the first-frame image. Exemplarily, the above similarity can be represented by the intersection over union (the ratio of the intersection area to the union area), that is, the computer device can calculate the intersection over union of the candidate bounding box and each template bounding box, and determine the template bounding box with the maximum intersection over union with the candidate bounding box as the template bounding box of the first-frame image. Optionally, the above similarity can also be represented by other parameters, and the embodiments of the present application do not limit the specific calculation method of the above similarity.

[0059] Exemplarily, as Figure 5 shown, assuming that after the computer device performs object detection on the first-frame ultrasound image 51 using a preset object detection model, the candidate bounding box shown in 52 as Figure 5 is obtained, then the computer device can match the candidate bounding box with the corresponding template bounding box set, and determine the bounding box in the template bounding box set that is most similar to the candidate bounding box as the target bounding box in the first-frame ultrasound image (as Figure 5 shown in 53).

[0060] Step 3.2, for the i -th frame ultrasound image, determine the target bounding box in the i -th frame ultrasound image according to the target bounding box in the i -1-th frame ultrasound image; i is an integer greater than 1.

[0061] Since the computer device has determined the template bounding box of the first-frame ultrasound image based on the corresponding template bounding box set, in order to reduce the computational complexity and improve the determination speed of the target bounding box, for each ultrasound image after the first-frame ultrasound image, the computer device can adjust the target bounding box in its previous 1-frame ultrasound image to obtain the target bounding box in the current-frame ultrasound image.

[0062] In an optional implementation manner, the computer device can determine the target bounding box in the i -th ultrasound image after the first-frame ultrasound image through steps 3.21 to 3.23, which are described in detail as follows: Step 3.21, for the i -th ultrasound image, when there are target pixels in the preset area outside the initial bounding box in the i -th ultrasound image, determine the observation bounding box in the i -th ultrasound image according to the initial bounding box and the preset area; where the initial bounding box is the i- The bounding box of the target in the -1-frame ultrasound image is mapped to the bounding box in the i -frame ultrasound image.

[0063] For the i -frame ultrasound image, the computer device can first determine a bounding box in the i -frame ultrasound image that is exactly the same as the bounding box of the target in the i -1-frame ultrasound image, and use it as the initial bounding box of the i -frame ultrasound image, and then detect whether there are target pixels in the preset area outside the initial bounding box.

[0064] Among them, the preset area outside the initial bounding box can be obtained by expanding each side of the initial bounding box by a preset number of pixels in the direction perpendicular to that side.

[0065] Exemplarily, as Figure 6 shown, assume that the white rectangular box in Figure 6 is the initial bounding box in the i -frame ultrasound image. Then, the red rectangular box area obtained by expanding the upper side of the initial bounding box by a preset number of pixels in the direction perpendicular to the upper side can be the first preset area corresponding to the upper side. Similarly, the red rectangular box area obtained by expanding the lower side of the initial bounding box by a preset number of pixels in the direction perpendicular to the lower side can be the second preset area corresponding to the lower side. The red rectangular box area obtained by expanding the left side of the initial bounding box by a preset number of pixels in the direction perpendicular to the left side can be the third preset area corresponding to the left side. The red rectangular box area obtained by expanding the right side of the initial bounding box by a preset number of pixels in the direction perpendicular to the right side can be the fourth preset area corresponding to the right side. The area set composed of the first preset area, the second preset area, the third preset area, and the fourth preset area is the preset area outside the initial bounding box in the i -frame ultrasound image.

[0066] The computer device can use a preset target detection model to detect whether there are target pixels in the preset area outside the initial bounding box in the i -frame ultrasound image. Optionally, in the case where there are target pixels in any one of the preset areas outside the initial bounding box in the i -frame ultrasound image, it means that the bounding box of the target in the i -frame ultrasound image has changed relative to the bounding box of the target in the i -1-frame image. At this time, the computer device can determine a rectangular box that can completely enclose the initial bounding box and the preset area with target pixels, and use this rectangular box as the observed bounding box in the i -frame ultrasound image. Optionally, in the iIf there are no target pixels in all preset regions outside the initial bounding box in the frame ultrasound image, it indicates that the target bounding box in the i frame ultrasound image has not changed relative to the target bounding box in the i -1 frame image. At this time, the computer device can determine the initial bounding box as the i observation bounding box in the frame ultrasound image.

[0067] Please continue to refer to Figure 6 . Assuming that there are target pixels in the first preset region, the second preset region, the third preset region, and the fourth preset region, the computer device can use the smallest rectangular box that can completely enclose the initial bounding box, the first preset region, the second preset region, the third preset region, and the fourth preset region ( Figure 6 the green rectangular box in i ) to determine the observation bounding box in the

[0068] frame ultrasound image. i Step 3.22, based on the observation bounding box and the initial bounding box in the i frame ultrasound image, determine the first state vector of the observation bounding box in the

[0069] Exemplarily, the first state vector can be represented by the geometric information and state variables of the observation bounding box in the i frame ultrasound image. The geometric information of the observation bounding box can include the two-dimensional pixel coordinates, width, and height of the center point of the observation bounding box. The state variables of the observation bounding box can be used to describe the i change rate of the observation bounding box in the frame ultrasound image relative to the initial bounding box (i.e., the target bounding box in the i -1 frame ultrasound image). The state variables of the observation bounding box can include the coordinate change rate, width change rate, and height change rate of the center point of the observation bounding box. Exemplarily, assuming that the observation bounding box in the i frame ultrasound image is represented by ([[]] x 1, y 1, w 1, h 1) and the initial bounding box is represented by ([[]] x 2, y 2, w 2, h 2), then the state variables of the observation bounding box in the i frame ultrasound image can be represented by ([[]] dx , dy , dw , dh ). Based on this, the first state vector of the observation bounding box in the i frame ultrasound image can be represented ass r = x 1, y 1, w 1, h 1, dx , dy , dw , dh 。

[0070] Among them, x 1 and y 1 are respectively the abscissa and ordinate of the center point of the observed bounding box in the i th frame of ultrasound image in the image coordinate system, w 1 is the width of the observed bounding box in the i th frame of ultrasound image, h 1 is the height of the observed bounding box in the i th frame of ultrasound image; x 2 and y 2 are respectively the abscissa and ordinate of the center point of the initial bounding box in the i th frame of ultrasound image in the image coordinate system, w 2 is the width of the initial bounding box in the i th frame of ultrasound image, h 2 is the height of the initial bounding box in the i th frame of ultrasound image.

[0071] Step 3.23, process the first state vector using the Kalman filter algorithm to obtain the target bounding box in the i th frame of ultrasound image.

[0072] Optionally, the computer device can first predict the second state vector of the predicted bounding box in the i th frame of ultrasound image according to the target state vector of the target bounding box in the i th frame of ultrasound image and the preset state transition matrix, and then determine the target state vector of the target bounding box in the i th frame of ultrasound image according to the first state vector of the observed bounding box in the i th frame of ultrasound image, the second state vector of the predicted bounding box, and the preset Kalman gain coefficient. Exemplarily, the second state vector of the predicted bounding box in the i th frame of ultrasound image can be obtained through s p = x p , y p , w p , h p , dxp , dy p , dw p , dh p indicates. Optionally, the computer device can calculate the target state vector of the target bounding box in the i th frame of ultrasound image through the following formula: s m = s p + K_z * ( s r - H * s p ); Wherein, s m is the target state vector of the target bounding box in the i th frame of ultrasound image, s p is the second state vector of the predicted bounding box in the i th frame of image, K_z is the preset Kalman gain coefficient, s r is the first state vector of the observed bounding box in the i th frame of image, H is the observation matrix in the Kalman filter algorithm.

[0073] Exemplarily, the preset Kalman gain coefficient can be automatically determined by the computer device according to the prediction noise of the predicted bounding box P and the observation noise of the observed bounding box R . Among them, the prediction noise can be used to represent the uncertainty of the prediction, and the observation noise can be used to represent the uncertainty of the observation. The smaller the observation noise R , the more accurate the observation, and the larger the preset Kalman gain coefficient; the smaller the prediction noise P , the more accurate the prediction, and the smaller the preset Kalman gain coefficient.

[0074] Exemplarily, the target state vector of the target bounding box in the i th frame of ultrasound image can be expressed as s m = x m , y m , w m , h m , dx m, dy m , dw m , dh m , where the ( x m , y m , w m , h m ) is used to represent the target bounding box in the i -th frame of ultrasound image. Among them, the ( x m , y m ) is the two-dimensional pixel coordinates of the center point of the target bounding box in the i -th frame of ultrasound image, w m is the width of the target bounding box in the i -th frame of ultrasound image, h m is the height of the target bounding box in the i -th frame of ultrasound image.

[0075] Optionally, the computer device can calculate the three-dimensional reconstruction data of the key area defined by the target bounding box through the following steps 4.1 to 4.4, which are described in detail as follows: Step 4.1, according to the two-dimensional pixel coordinates of each pixel in the key area, determine the three-dimensional coordinates of each pixel in the key area in the world coordinate system through the space conversion matrix corresponding to the current frame of ultrasound image.

[0076] Step 4.2, according to the three-dimensional coordinates of the starting point of the preset three-dimensional voxel grid and the preset voxel resolution, determine the three-dimensional voxel indices corresponding to each pixel in the key area, and determine the pixel values of each pixel in the key area as the voxel values corresponding to each pixel in the key area.

[0077] Step 4.3, for each pixel in the key area, determine the vector composed of the three-dimensional voxel index and voxel value corresponding to the pixel as the three-dimensional reconstruction data of the pixel.

[0078] Step 4.4, determine the combination of the three-dimensional reconstruction data of all pixels in the key area as the three-dimensional reconstruction data of the key area.

[0079] It should be noted that the calculation process of the 3D reconstruction data of each pixel in the key region is exactly the same as that of the 3D reconstruction data of each sampling point above. Therefore, for the specific implementation process of steps 4.1 to 4.4, reference can be made to the description in steps 1.1 to 1.3 above, and it will not be elaborated here.

[0080] S103. Determine the 3D reconstruction data of the current frame of ultrasound image according to the 3D reconstruction data of all sampling points and the 3D reconstruction data of the key region.

[0081] It can be understood that since some sampling points in the ultrasound image will fall into the key region defined by the target bounding box, the computer device can determine the combination of the 3D reconstruction data of the key region and the 3D reconstruction data of all sampling points in the non-key region as the 3D reconstruction data of the ultrasound image.

[0082] Among them, the non-key region can be the region outside the key region in the ultrasound image.

[0083] S104. Integrate the 3D reconstruction data of the current frame of ultrasound image into the 3D fusion data corresponding to the previous frame of ultrasound image to obtain the 3D fusion data corresponding to the current frame of ultrasound image; the 3D fusion data corresponding to the last frame of ultrasound image in the ultrasound image sequence is the 3D ultrasound model corresponding to the ultrasound image sequence.

[0084] It can be understood that the 3D fusion data corresponding to the i th frame of ultrasound image integrates the 3D reconstruction data of the 1st frame to the i th frame of ultrasound image. And so on, the 3D fusion data corresponding to the i -1th frame of ultrasound image integrates the 3D reconstruction data of the 1st frame to the i -1th frame of ultrasound image, and the 3D fusion data corresponding to the last frame of ultrasound image integrates the 3D reconstruction data of the 1st frame to the last frame of ultrasound image. Therefore, the 3D fusion data corresponding to the last frame of ultrasound image in the ultrasound image sequence is the 3D ultrasound model corresponding to the entire ultrasound image sequence.

[0085] In some other embodiments, in order to improve the problem of missing volume images in non-scanned areas in the three-dimensional ultrasound model, the computer device may interpolate the grids in the preset three-dimensional voxel grid except for the three-dimensional fusion data corresponding to the previous frame of ultrasound image when calculating the three-dimensional reconstruction data of each frame of ultrasound image. For example, for each frame of ultrasound image in the ultrasound image sequence, the computer device may, while using one thread to determine the three-dimensional reconstruction data of the current frame of ultrasound image, use another thread to interpolate the grids in the preset three-dimensional voxel grid except for the three-dimensional fusion data corresponding to the previous frame of ultrasound image. Since the interpolation process and the three-dimensional reconstruction process of the ultrasound image are synchronized through different threads, the three-dimensional reconstruction speed will not be affected.

[0086] Based on this, fusing the three-dimensional reconstruction data of the current frame of ultrasound image into the three-dimensional fusion data corresponding to the previous frame of ultrasound image in S104 to obtain the three-dimensional fusion data corresponding to the current frame of ultrasound image may include: Based on the principle of covering the interpolation data with the three-dimensional reconstruction data under the same voxel index, fusing the three-dimensional reconstruction data of the current frame of ultrasound image into the three-dimensional fusion data corresponding to the interpolated previous frame of ultrasound image to obtain the three-dimensional fusion data corresponding to the current frame of ultrasound image.

[0087] Among them, the principle of covering the interpolation data with the three-dimensional reconstruction data under the same voxel index means that when there is both interpolation data and three-dimensional reconstruction data under the same voxel index, the voxel value in the three-dimensional reconstruction data corresponding to the voxel index is determined as the actual voxel value of the voxel index.

[0088] As can be seen from the above, for each frame of ultrasound image in the ultrasound image sequence, the three-dimensional reconstruction method of the ultrasound image provided by the embodiments of the present application can achieve global rough three-dimensional reconstruction of the ultrasound image by globally and intermittently sampling the ultrasound image based on a preset gap sampling template and calculating the three-dimensional reconstruction data of each sampling point; by determining the target bounding box in the ultrasound image and calculating the three-dimensional reconstruction data of the key area defined by the target bounding box, it can achieve refined three-dimensional reconstruction of the key area. Compared with the prior art that reconstructs all pixels in the ultrasound image, the three-dimensional reconstruction method of the embodiments of the present application adopts a three-dimensional reconstruction method combining global rough reconstruction and local refinement reconstruction, which can not only ensure the accuracy of three-dimensional reconstruction; but also reduce the computational complexity of three-dimensional reconstruction, thereby reducing the requirement for the computing power of hardware devices and improving the feasibility of clinical applications; at the same time, it can also improve the speed of three-dimensional reconstruction, which is beneficial for real-time three-dimensional reconstruction.

[0089] In addition, during the three-dimensional reconstruction of the ultrasound image, when determining the target bounding box in each ultrasound image after the first frame of the ultrasound image, by introducing the Kalman filtering algorithm to update the target bounding box in the ultrasound image in real time, the stability of the three-dimensional reconstruction of the key area defined by the target bounding box can be improved.

[0090] By interpolating the three-dimensional space corresponding to the non-ultrasound scanning area in the ultrasound image, the problem of missing volume images in the non-scanning area of the three-dimensional ultrasound model can be improved. Since the interpolation process and the three-dimensional reconstruction process of the ultrasound image are synchronized through different threads, it will not affect the three-dimensional reconstruction speed.

[0091] It can be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0092] Based on the three-dimensional reconstruction method of the ultrasound image provided in the above embodiments, the embodiments of the present application further provide an embodiment of a computer device for implementing the above method embodiments. Please refer to Figure 7 , which is a schematic structural diagram of a computer device provided by the embodiments of the present application. For the convenience of description, only the parts related to this embodiment are shown. As Figure 7 shown, the computer device 70 may include: a first calculation unit 701, a second calculation unit 702, a first determination unit 703, and a data fusion unit 704. Among them: The first calculation unit 701 is configured to sequentially perform global intermittent sampling on the current frame of the ultrasound image based on a preset gap sampling template for each frame of the ultrasound image sequence, and calculate the three-dimensional reconstruction data of each sampling point.

[0093] The second calculation unit 702 is configured to determine the target bounding box in the current frame of the ultrasound image according to the template bounding box set corresponding to the scanning part of the ultrasound image sequence, and calculate the three-dimensional reconstruction data of the key area defined by the target bounding box.

[0094] The first determination unit 703 is configured to determine the three-dimensional reconstruction data of the current frame of the ultrasound image according to the three-dimensional reconstruction data of all the sampling points and the three-dimensional reconstruction data of the key area.

[0095] The data fusion unit 704 is configured to fuse the three-dimensional reconstruction data of the current frame of the ultrasound image into the three-dimensional fusion data corresponding to the previous frame of the ultrasound image to obtain the three-dimensional fusion data corresponding to the current frame of the ultrasound image; the three-dimensional fusion data corresponding to the last frame of the ultrasound image sequence is the three-dimensional ultrasound model corresponding to the ultrasound image sequence.

[0096] Optionally, it further includes a first acquisition unit, a second determination unit, and a clustering unit. Specifically: The first acquisition unit is used to acquire a sample image set corresponding to each scanned part; the sample image set is composed of a number of sample ultrasound images marked with target pixels.

[0097] The second determination unit is used to determine the bounding box corresponding to the target pixels in each of the sample ultrasound images.

[0098] The clustering unit is used to cluster the bounding boxes corresponding to all the target pixels in the sample image set corresponding to each scanned part, to obtain a template bounding box set corresponding to the scanned part.

[0099] Optionally, the first determination unit 703 is specifically used for: For the first frame of ultrasound image, use a preset target detection model to perform target detection on the first frame of ultrasound image, obtain the candidate bounding box in the first frame of ultrasound image, and determine the template bounding box in the template bounding box set that is most similar to the candidate bounding box as the target bounding box in the first frame of ultrasound image; For the i th frame of ultrasound image, determine the target bounding box in the i th frame of ultrasound image according to the target bounding box in the i th - 1 frame of ultrasound image; i is an integer greater than 1.

[0100] Optionally, the first determination unit 703 is further specifically used for: For the i th frame of ultrasound image, in the case that there are target pixels in the preset area outside the initial bounding box in the i th frame of ultrasound image, determine the observed bounding box in the i th frame of ultrasound image according to the initial bounding box and the preset area; wherein, the initial bounding box is the bounding box obtained by mapping the target bounding box in the i th - 1 frame of ultrasound image to the i th frame of ultrasound image; According to the observed bounding box and the initial bounding box in the i th frame of ultrasound image, determine the first state vector of the observed bounding box in the i th frame of ultrasound image; Use the Kalman filter algorithm to process the first state vector to obtain the target bounding box in the i th frame of ultrasound image.

[0101] Optionally, the first calculation unit 701 is specifically used for: Determine the three-dimensional coordinates of each of the sampling points in the world coordinate system through the spatial transformation matrix corresponding to the current frame of ultrasonic image according to the two-dimensional pixel coordinates of each of the sampling points; Determine the three-dimensional voxel indices corresponding to each of the sampling points respectively according to the three-dimensional coordinates of the starting point of the preset three-dimensional voxel grid and the preset voxel resolution, and determine the pixel values of each of the sampling points as the voxel values corresponding to each of the sampling points; For each of the sampling points, determine the vector composed of the three-dimensional voxel index and the voxel value of the sampling point as the three-dimensional reconstruction data of the sampling point.

[0102] Optionally, the second calculation unit 702 is specifically configured to: Determine the three-dimensional coordinates of each pixel in the key area in the world coordinate system through the spatial transformation matrix corresponding to the current frame of ultrasonic image according to the two-dimensional pixel coordinates of each pixel in the key area; Determine the three-dimensional voxel indices corresponding to each pixel in the key area respectively according to the three-dimensional coordinates of the starting point of the preset three-dimensional voxel grid and the preset voxel resolution, and determine the pixel values of each pixel in the key area as the voxel values corresponding to each pixel in the key area; For each pixel in the key area, determine the vector composed of the three-dimensional voxel index corresponding to the pixel and the voxel value as the three-dimensional reconstruction data of the pixel; Determine the combination of the three-dimensional reconstruction data of all pixels in the key area as the three-dimensional reconstruction data of the key area.

[0103] Optionally, the computer device may further include an interpolation unit.

[0104] The interpolation unit is used to, for each frame of ultrasonic image in the ultrasonic image sequence, while using one thread to determine the three-dimensional reconstruction data of the current frame of ultrasonic image, use another thread to perform interpolation on the grid in the preset three-dimensional voxel grid except for the three-dimensional fusion data corresponding to the previous frame of ultrasonic image.

[0105] Correspondingly, the data fusion unit 704 is specifically configured to: Based on the principle of covering the interpolation data with the three-dimensional reconstruction data under the same voxel index, fuse the three-dimensional reconstruction data of the current frame of ultrasonic image into the three-dimensional fusion data corresponding to the interpolated previous frame of ultrasonic image to obtain the three-dimensional fusion data corresponding to the current frame of ultrasonic image.

[0106] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of each functional unit is used as an example for illustration. In actual applications, the above functions can be allocated to different functional units as needed, that is, the internal structure of the computer device is divided into different functional units to complete all or part of the functions described above. Each functional unit in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working processes of each unit in the above computer device can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0107] Please refer to Figure 8 , Figure 8 which is a schematic structural diagram of a computer device provided by another embodiment of this application. As Figure 8 shown, the computer device 8 provided in this embodiment may include: a processor 80, a memory 81, and a computer program 82 stored in the memory 81 and executable on the processor 80, such as a program corresponding to the three-dimensional reconstruction method of ultrasonic images. When the processor 80 executes the computer program 82, the steps in the above embodiment of the three-dimensional reconstruction method of ultrasonic images are implemented, such as Figure 1 the S101~S104 shown. Or when the processor 80 executes the computer program 82, the functions of each unit in the above embodiment of the computer device are implemented.

[0108] Exemplarily, the computer program 82 can be divided into one or more modules / units. One or more modules / units are stored in the memory 81 and executed by the processor 80 to complete this application. One or more modules / units can be a series of computer program instruction segments capable of completing specific functions, and these instruction segments are used to describe the execution process of the computer program 82 in the computer device 8. For example, the computer program 82 can be divided into a first calculation unit, a second calculation unit, a first determination unit, and a data fusion unit. For the specific functions of each unit, please refer to Figure 7 the relevant descriptions in the corresponding embodiments and will not be elaborated here.

[0109] Those skilled in the art can understand that Figure 8 is only an example of the computer device 8 and does not constitute a limitation on the computer device 8. It may include more or fewer components than shown in the figure, or combine some components, or different components.

[0110] The processor 80 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or the processor may also be any conventional processor, etc.

[0111] The memory 81 may be an internal storage unit of the computer device 8, such as the hard disk or memory of the computer device 8. The memory 81 may also be an external storage device of the computer device 8, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, or a flash card equipped on the computer device 8, etc. Further, the memory 81 may also include both the internal storage unit and the external storage device of the computer device 8. The memory 81 is used to store computer programs and other programs and data required by the computer device. The memory 81 may also be used to temporarily store data that has been output or is to be output.

[0112] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, each step in the three-dimensional reconstruction method of the ultrasonic image in the above method embodiment is implemented.

[0113] The embodiment of the present application provides a computer program product. When the computer program product runs on a computer device, the computer device is enabled to implement the steps in the above-mentioned various method embodiments.

[0114] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0115] It should be noted that unless otherwise specified, all technical terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the technical field to which the present application belongs. The technical terms used in the embodiments of the present application are only used to explain the specific embodiments of the present application, rather than being intended to limit the present application.

[0116] As used in the description of the embodiments of the present application, the term "embodiment" means that the specific features, structures, or characteristics described in connection with the embodiments may be included in at least one embodiment of the present application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein may be combined with other embodiments.

[0117] Those of ordinary skill in the art will realize that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. A professional technician can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0118] The above-described embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A three-dimensional reconstruction method for ultrasonic images, characterized in that, Including: For each frame of ultrasound image in the ultrasound image sequence in turn, perform global intermittent sampling on the current frame of ultrasound image based on a preset gap sampling template, and calculate the three-dimensional reconstruction data of each sampling point; According to the set of template bounding boxes corresponding to the scanned part of the ultrasound image sequence, determine the target bounding box in the current frame of ultrasound image, and calculate the three-dimensional reconstruction data of the key area defined by the target bounding box; According to the three-dimensional reconstruction data of all the sampling points and the three-dimensional reconstruction data of the key area, determine the three-dimensional reconstruction data of the current frame of ultrasound image; Fuse the three-dimensional reconstruction data of the current frame of ultrasound image into the three-dimensional fusion data corresponding to the previous frame of ultrasound image to obtain the three-dimensional fusion data corresponding to the current frame of ultrasound image; the three-dimensional fusion data corresponding to the last frame of ultrasound image in the ultrasound image sequence is the three-dimensional ultrasound model corresponding to the ultrasound image sequence.

2. The method according to claim 1, characterized in that Before determining the target bounding box in the current frame of ultrasound image according to the set of template bounding boxes corresponding to the scanned part of the ultrasound image sequence, it further includes: Obtain the set of sample images corresponding to each scanned part; the set of sample images consists of several sample ultrasound images marked with target pixels; Determine the bounding box corresponding to the target pixel in each of the sample ultrasound images; For each scanned part, cluster the bounding boxes corresponding to all the target pixels in the set of sample images corresponding to the scanned part to obtain the set of template bounding boxes corresponding to the scanned part.

3. The method according to claim 1, characterized in that, Determining the target bounding box in the current frame of ultrasound image according to the set of template bounding boxes corresponding to the scanned part of the current frame of ultrasound image includes: For the first frame of ultrasound image, use a preset target detection model to perform target detection on the first frame of ultrasound image to obtain candidate bounding boxes in the first frame of ultrasound image, and determine the template bounding box in the set of template bounding boxes that is most similar to the candidate bounding box as the target bounding box in the first frame of ultrasound image; For the i -th frame of ultrasound image, determine the target bounding box in the i -th frame of ultrasound image based on the target bounding box in the i -th frame of ultrasound image; i is an integer greater than 1.

4. The method according to claim 3, wherein For the i -th frame ultrasound image, determine the target bounding box in the i -1-th frame ultrasound image, including: i the target bounding box in the -th frame ultrasound image, For the i frame ultrasound image, when there are target pixels in a preset area outside the initial bounding box in the i frame ultrasound image, an observation bounding box in the i frame ultrasound image is determined according to the initial bounding box and the preset area; wherein, the initial bounding box is the bounding box in the i -1 frame ultrasound image that the target bounding box in the i frame ultrasound image is mapped to; According to the observed bounding box and the initial bounding box in the i frame of ultrasound image, determine the first state vector of the observed bounding box in the i frame of ultrasound image; The Kalman filter algorithm is used to process the first state vector to obtain the i target bounding box in the nth frame of ultrasound image.

5. The method according to claim 1, wherein Calculating the three-dimensional reconstruction data of each sampling point includes: According to the two-dimensional pixel coordinates of each sampling point, determine the three-dimensional coordinates of each sampling point in the world coordinate system through the space transformation matrix corresponding to the current frame of ultrasound image; According to the three-dimensional coordinates of the starting point of the preset three-dimensional voxel grid and the preset voxel resolution, determine the three-dimensional voxel index corresponding to each sampling point, and determine the pixel value of each sampling point as the voxel value corresponding to each sampling point; For each sampling point, determine the vector composed of the three-dimensional voxel index and the voxel value of the sampling point as the three-dimensional reconstruction data of the sampling point.

6. The method according to claim 1, wherein Calculating the three-dimensional reconstruction data of the key area defined by the target bounding box includes: According to the two-dimensional pixel coordinates of each pixel in the key area, determine the three-dimensional coordinates of each pixel in the key area in the world coordinate system through the space transformation matrix corresponding to the current frame of ultrasound image; Based on the three-dimensional coordinates of the starting point of the preset three-dimensional voxel grid and the preset voxel resolution, determine the three-dimensional voxel indices corresponding to each pixel in the key area, and determine the voxel values corresponding to each pixel in the key area as the pixel values of each pixel in the key area; For each pixel in the key area, determine the vector composed of the three-dimensional voxel index and the voxel value corresponding to the pixel as the three-dimensional reconstruction data of the pixel; Determine the combination of the three-dimensional reconstruction data of all pixels in the key area as the three-dimensional reconstruction data of the key area.

7. The method according to any one of claims 1-6, characterized in that, It further includes: For each frame of ultrasound image in the ultrasound image sequence, while using one thread to determine the three-dimensional reconstruction data of the current frame of ultrasound image, use another thread to interpolate the grid in the preset three-dimensional voxel grid except for the three-dimensional fusion data corresponding to the previous frame of ultrasound image; Correspondingly, fusing the three-dimensional reconstruction data of the current frame of ultrasound image into the three-dimensional fusion data corresponding to the previous frame of ultrasound image to obtain the three-dimensional fusion data corresponding to the current frame of ultrasound image, including: Based on the principle of covering the interpolation data with the three-dimensional reconstruction data under the same voxel index, fuse the three-dimensional reconstruction data of the current frame of ultrasound image into the three-dimensional fusion data corresponding to the interpolated previous frame of ultrasound image to obtain the three-dimensional fusion data corresponding to the current frame of ultrasound image.

8. A computer device, characterized in that, It includes: A first calculation unit, configured to sequentially perform global intermittent sampling on each frame of ultrasound image in the ultrasound image sequence based on a preset gap sampling template, and calculate the three-dimensional reconstruction data of each sampling point; A second calculation unit, configured to determine the target bounding box in the current frame of ultrasound image according to the template bounding box set corresponding to the scanning part of the ultrasound image sequence, and calculate the three-dimensional reconstruction data of the key area defined by the target bounding box; A first determination unit, configured to determine the three-dimensional reconstruction data of the current frame of ultrasound image according to the three-dimensional reconstruction data of all the sampling points and the three-dimensional reconstruction data of the key area; A data fusion unit, configured to fuse the three-dimensional reconstruction data of the current frame of ultrasound image into the three-dimensional fusion data corresponding to the previous frame of ultrasound image to obtain the three-dimensional fusion data corresponding to the current frame of ultrasound image; the three-dimensional fusion data corresponding to the last frame of ultrasound image in the ultrasound image sequence is the three-dimensional ultrasound model corresponding to the ultrasound image sequence.

9. A computer device, characterized in that, It includes a memory and a computer program stored in the memory and executable on a processor, and when the processor executes the computer program, it implements the method according to any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the method according to any one of claims 1-7.

Images