Three-dimensional reconstruction method, device and equipment of multi-view two-dimensional ultrasound image and medium

Abstract

The application relates to a three-dimensional reconstruction method, device, equipment and medium of multi-view two-dimensional ultrasound images, and particularly relates to the technical field of medical image processing. The method comprises the following steps: acquiring two-dimensional ultrasound data collected by an ultrasound probe under a main view angle and an auxiliary view angle to obtain a main view angle image set and an auxiliary view angle image set; converting the main view angle image set and the auxiliary view angle image set from an ultrasound coordinate system to a view coordinate system to obtain a main voxel matrix and an auxiliary voxel matrix; performing weighted calculation on the main voxel matrix and the auxiliary voxel matrix to obtain a main auxiliary voxel matrix; and performing three-dimensional reconstruction on the main auxiliary voxel matrix to obtain a three-dimensional reconstruction result. Based on the technical scheme provided in the application, the range and depth of ultrasound reconstruction are increased by providing two-dimensional ultrasound data of multiple view angles, so that the accuracy of three-dimensional reconstruction is improved.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, specifically to a method, apparatus, device, and medium for three-dimensional reconstruction of multi-view two-dimensional ultrasound images. Background Technology

[0002] The technique of reconstructing three-dimensional ultrasound images from two-dimensional images is widely used in auxiliary diagnostic scenarios.

[0003] Existing methods for three-dimensional reconstruction of two-dimensional ultrasound images can use a handheld two-dimensional ultrasound probe to acquire image information, and then collect the spatiotemporal information of the image through other positioning methods to achieve three-dimensional restoration.

[0004] Based on the above technical solutions, due to factors such as the frequency of ultrasonic acquisition, the angle of manual scanning, and the speed of deviation, uneven distribution of the original data layers after reconstruction is likely to occur, affecting the three-dimensional reconstruction effect. Summary of the Invention

[0005] This application provides a method, apparatus, device, and medium for three-dimensional reconstruction of multi-view two-dimensional ultrasound images, and the technical solution is as follows.

[0006] On the one hand, a method for three-dimensional reconstruction of multi-view two-dimensional ultrasound images is provided, the method comprising:

[0007] Two-dimensional ultrasound data acquired by the ultrasound probe under the main view and the secondary view are obtained to obtain the main view image set and the secondary view image set;

[0008] Transform the main view image set and the auxiliary view image set from the ultrasonic coordinate system to the view coordinate system to obtain the main voxel matrix and the auxiliary voxel matrix;

[0009] The main voxel matrix and the auxiliary voxel matrix are weighted to obtain the main and auxiliary voxel matrices;

[0010] The principal and secondary voxel matrices are reconstructed in three dimensions to obtain the three-dimensional reconstruction results.

[0011] In another aspect, a three-dimensional reconstruction device for multi-view two-dimensional ultrasound images is provided, the device comprising:

[0012] The two-dimensional ultrasound data acquisition module is used to acquire two-dimensional ultrasound data collected by the ultrasound probe in the main view and the auxiliary view, and to obtain the main view image set and the auxiliary view image set.

[0013] The voxel matrix acquisition module is used to transform the main view image set and the auxiliary view image set from the ultrasound coordinate system to the view coordinate system to obtain the main voxel matrix and the auxiliary voxel matrix.

[0014] The matrix weighting calculation module is used to perform weighted calculations on the main voxel matrix and the auxiliary voxel matrix to obtain the main and auxiliary voxel matrices;

[0015] The three-dimensional reconstruction module is used to perform three-dimensional reconstruction on the principal and secondary voxel matrices to obtain the three-dimensional reconstruction results.

[0016] In another aspect, a computer device is provided, characterized in that the computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, at least one program, code set or instruction set being loaded and executed by the processor to realize the above-described method for three-dimensional reconstruction of multi-view two-dimensional ultrasound images.

[0017] In another aspect, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored therein, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the above-described method for three-dimensional reconstruction of multi-view two-dimensional ultrasound images.

[0018] In another aspect, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned method for three-dimensional reconstruction of multi-view two-dimensional ultrasound images.

[0019] The technical solution provided in this application may include the following beneficial effects:

[0020] Two-dimensional ultrasound data acquired by an ultrasound probe from both primary and secondary viewpoints are obtained. The resulting primary and secondary viewpoint image sets are transformed from the ultrasound coordinate system to the view coordinate system to obtain the main voxel matrix and the secondary voxel matrix. The main and secondary voxel matrices are then weighted to obtain the primary and secondary voxel matrices. Three-dimensional reconstruction is performed on the primary and secondary voxel matrices that have been fused from different viewpoints to obtain the three-dimensional reconstruction result. By providing two-dimensional ultrasound data from multiple viewpoints, the range and depth of ultrasound reconstruction are increased. This compensates for the defects of uneven distribution of the original data layers after reconstruction, or even large angular offsets, caused by deviations in ultrasound acquisition frequency, manual scanning angle, and speed, thereby improving the accuracy of three-dimensional reconstruction. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of a three-dimensional reconstruction apparatus according to an exemplary embodiment.

[0023] Figure 2 This is a flowchart illustrating a method for three-dimensional reconstruction of multi-view two-dimensional ultrasound images according to an exemplary embodiment.

[0024] Figure 3 This is a flowchart illustrating a method for three-dimensional reconstruction of multi-view two-dimensional ultrasound images according to an exemplary embodiment.

[0025] Figure 4 This is a schematic diagram illustrating two-dimensional ultrasound data acquisition according to an exemplary embodiment.

[0026] Figure 5 This is a flowchart illustrating a method for three-dimensional reconstruction of multi-view two-dimensional ultrasound images according to an exemplary embodiment.

[0027] Figure 6 This is a schematic diagram illustrating bicubic interpolation according to an exemplary embodiment.

[0028] Figure 7 This is a structural block diagram of a three-dimensional reconstruction device for multi-view two-dimensional ultrasound images according to an exemplary embodiment.

[0029] Figure 8 This is a schematic diagram of a computer device provided according to an exemplary embodiment. Detailed Implementation

[0030] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] It should be understood that the term "instruction" mentioned in the embodiments of this application can be a direct instruction, an indirect instruction, or an indication of a relationship. For example, A instructing B can mean that A directly instructs B, such as B being able to obtain information through A; it can also mean that A indirectly instructs B, such as A instructing C, so B can obtain information through C; or it can mean that there is a relationship between A and B.

[0032] In the description of the embodiments of this application, the term "correspondence" may indicate that there is a direct or indirect correspondence between two things, or that there is an association between two things, or that there is a relationship of instruction and being instructed, configuration and being configured, etc.

[0033] In the embodiments of this application, "predefined" can be achieved by pre-storing corresponding codes, tables or other means that can be used to indicate relevant information in the device (e.g., including terminal devices and network devices). This application does not limit the specific implementation method.

[0034] The three-dimensional reconstruction method shown in this application can be applied to, for example... Figure 1 The illustrated 3D reconstruction device includes: a portable ultrasound diagnostic instrument, a magnetic positioning clamp, a magnetic positioning device, and a digital acquisition and reconstruction system. The process of performing 3D reconstruction from a 2D ultrasound image based on this device is as follows:

[0035] (1) Assemble a portable ultrasound diagnostic instrument (including an ultrasound probe and an ultrasound system) and install a magnetic positioning clamp on the ultrasound probe.

[0036] (2) Assemble a digital acquisition and reconstruction system and establish its communication with the ultrasonic system and magnetic positioning equipment.

[0037] (3) Assemble a magnetic positioning device (including a magnetic positioning receiver and a magnetic positioning transmitter), wherein the magnetic positioning transmitter is placed in a fixed position in space, and the magnetic positioning receiver is installed on the magnetic positioning fixture.

[0038] (4) Use an ultrasonic probe to acquire two-dimensional ultrasonic data at high frequency, and use a magnetic locator to collect magnetic positioning data. Transmit the two-dimensional ultrasonic data and magnetic positioning data together to the digital acquisition and analysis system.

[0039] (5) Digital acquisition and reconstruction system, which can be used to receive the two-dimensional ultrasound data set transmitted by the ultrasound system and the magnetic positioning data set of the magnetic positioning instrument, for interaction, processing and display of three-dimensional reconstructed images.

[0040] This application provides a method for three-dimensional reconstruction of multi-view two-dimensional ultrasound images, which comprehensively considers images from multiple acquisition directions, enabling rapid and accurate three-dimensional reconstruction and interactive display. The technical solution provided in this application will be further described below.

[0041] Figure 2 This is a flowchart illustrating a method for three-dimensional reconstruction of multi-view two-dimensional ultrasound images according to an exemplary embodiment. This method is applied in computer equipment. Figure 2 As shown, the three-dimensional reconstruction method of this multi-view two-dimensional ultrasound image may include the following steps:

[0042] Step 210: Acquire two-dimensional ultrasound data collected by the ultrasound probe under the main view and the secondary view to obtain the main view image set and the secondary view image set.

[0043] In this embodiment of the application, an ultrasound probe is used to perform an ultrasound scan on the object to be detected (such as limbs). The ultrasound probe is placed at different positions around the object to be detected, thereby forming two different perspectives: a primary perspective and a secondary perspective. The two-dimensional ultrasound data acquired under the primary perspective is recorded as the primary perspective image set, and the two-dimensional ultrasound data acquired under the secondary perspective is recorded as the secondary perspective image set.

[0044] Step 220: Transform the main view image set and the auxiliary view image set from the ultrasound coordinate system to the view coordinate system to obtain the main voxel matrix and the auxiliary voxel matrix.

[0045] The ultrasonic coordinate system is the coordinate system corresponding to the data acquired by the ultrasonic probe. The ultrasonic coordinate system is related to the structure of the ultrasonic probe. As the ultrasonic probe moves, its coordinate system in the real physical space also changes. In order to uniformly map the two-dimensional ultrasonic data to a coordinate system, a fixed coordinate system needs to be set up. This coordinate system is the actual coordinate system for integrating the two-dimensional ultrasonic data, and it is denoted as the view coordinate system.

[0046] In this embodiment of the application, the primary view image set and the secondary view image set obtained by the ultrasonic probe through ultrasonic scanning are transformed into coordinates, specifically from the ultrasonic coordinate system to the view coordinate system. The primary view image set is transformed into a main voxel matrix, and the secondary view image set is transformed into a secondary voxel matrix.

[0047] Step 230: Perform weighted calculations on the main voxel matrix and the auxiliary voxel matrix to obtain the main and auxiliary voxel matrices.

[0048] In this embodiment, the main voxel matrix and the auxiliary voxel matrix are weighted according to their respective weights to obtain the main and auxiliary voxel matrices.

[0049] Step 240: Perform three-dimensional reconstruction on the principal and sub-voxel matrices to obtain the three-dimensional reconstruction results.

[0050] In the embodiments of this application, three-dimensional reconstruction is performed using data including principal and sub-voxel matrices to obtain three-dimensional reconstruction results.

[0051] In summary, the three-dimensional reconstruction method for multi-view two-dimensional ultrasound images provided in this embodiment acquires two-dimensional ultrasound data collected by the ultrasound probe from the primary and secondary viewpoints. The obtained primary and secondary viewpoint image sets are transformed from the ultrasound coordinate system to the view coordinate system to obtain the main voxel matrix and the auxiliary voxel matrix. Then, the main voxel matrix and the auxiliary voxel matrix are weighted to obtain the primary and secondary voxel matrices. Three-dimensional reconstruction is performed on the primary and secondary voxel matrices that have been fused from different viewpoints to obtain the three-dimensional reconstruction result. By providing two-dimensional ultrasound data from multiple viewpoints, the range and depth of ultrasound reconstruction are increased, which makes up for the defects of uneven distribution of the original data layers after reconstruction or even large angular offsets caused by the offset of ultrasound acquisition frequency, manual scanning angle, and speed, thereby improving the accuracy of three-dimensional reconstruction.

[0052] In the illustrative embodiment, the primary view and the secondary view are two opposite viewpoints on the same coordinate axis, and each viewpoint image set includes data collected by the ultrasound probe translating along different coordinate axes. The data collected by translating along different coordinate axes are then used to fill the matrix to obtain the voxel matrix under the corresponding viewpoint.

[0053] Figure 3 This is a flowchart illustrating a method for three-dimensional reconstruction of multi-view two-dimensional ultrasound images according to an exemplary embodiment. This method is applied in computer equipment. Figure 3 As shown, the three-dimensional reconstruction method of this multi-view two-dimensional ultrasound image may include the following steps:

[0054] Step 310: Place the ultrasound probe in the negative direction of the first coordinate axis so that the ultrasound probe is in the main view; translate the ultrasound probe along the second coordinate axis and the third coordinate axis respectively to acquire two-dimensional ultrasound data and obtain the first main view image set and the second main view image set.

[0055] Step 320: Place the ultrasound probe in the positive direction of the first coordinate axis so that the ultrasound probe is in the auxiliary viewpoint; translate the ultrasound probe along the second coordinate axis and the third coordinate axis respectively to perform two-dimensional ultrasound data acquisition, and obtain the first auxiliary viewpoint image set and the second auxiliary viewpoint image set.

[0056] For example, in conjunction with reference Figure 4 Taking the arm as an example, with the back of the hand facing up, place the ultrasound probe vertically on the arm, with the Z-axis perpendicular to the ultrasound image plane of the limbs. In the primary view, position the probe in the negative Y-axis direction, perpendicular to the XZ plane. The images acquired by translating along the Z-axis and X-axis directions constitute the first primary view image set and the second primary view image set, respectively. In the secondary view, position the ultrasound probe in the positive Y-axis direction, perpendicular to the XZ plane. The images acquired by translating along the Z-axis and X-axis directions constitute the first secondary view image set and the second secondary view image set.

[0057] Step 330: Transform the first main view image set from the ultrasound coordinate system to the view coordinate system, and store the obtained first set of pixels in the subject matrix; transform the second main view image set from the ultrasound coordinate system to the view coordinate system, and use the obtained second set of pixels to update the data in the subject matrix to obtain the final subject matrix.

[0058] In this embodiment, the first main view image set is traversed first, and the pixels are transferred to the view coordinate system through transformation relationship. This set of pixels is stored in the subject matrix as the original pixels. Then, the pixels of the second main view image set are also transferred to the view coordinate system through transformation relationship. This set of pixels is used to update the data in the subject matrix to obtain the final subject matrix.

[0059] In one possible implementation, the data in the main pixel matrix is ​​updated using the obtained second set of pixels to obtain the final main pixel matrix. This includes: traversing the pixels in the second set of pixels; if the current pixel does not exist in the main pixel matrix, then storing the current pixel in the main pixel matrix; if the current pixel already exists in the main pixel matrix, then performing a weighted average calculation between the current pixel and the existing pixels, and replacing the existing pixels in the main pixel matrix with the obtained pixel.

[0060] For example, for the second group of pixels, if the mapped point does not exist in the main pixel matrix, it is added to the main pixel matrix; if it already exists, the original value is replaced by a weighted average of w1 and w2 (the initial preset values ​​are 0.5 and 0.5).

[0061] Step 340: Transform the second auxiliary viewpoint image set from the ultrasound coordinate system to the view coordinate system, and store the obtained third set of pixels in the auxiliary voxel matrix; transform the second auxiliary viewpoint image set from the ultrasound coordinate system to the view coordinate system, and use the obtained fourth set of pixels to update the data in the auxiliary voxel matrix to obtain the final auxiliary voxel matrix.

[0062] In this embodiment, the first auxiliary view image set is traversed first, and the pixels are transferred to the view coordinate system through transformation relationship. This set of pixels is stored in the auxiliary voxel matrix as the original pixels. Then, the pixels of the second auxiliary view image set are also transferred to the view coordinate system through transformation relationship. This set of pixels is used to update the data in the auxiliary voxel matrix to obtain the final auxiliary voxel matrix.

[0063] In one possible implementation, the data in the auxiliary voxel matrix is ​​updated using the obtained fourth group of pixels to obtain the final auxiliary voxel matrix. This includes: traversing the pixels in the fourth group of pixels; if the current pixel does not exist in the auxiliary voxel matrix, then storing the current pixel in the auxiliary voxel matrix; if the current pixel already exists in the auxiliary voxel matrix, then performing a weighted average calculation between the current pixel and the existing pixels, and replacing the existing pixels in the auxiliary voxel matrix with the resulting pixel.

[0064] For example, for the fourth group of pixels, if the mapped point does not exist in the auxiliary voxel matrix, it is added to the auxiliary voxel matrix; if it already exists, the original value is replaced by a weighted average of w1 and w2 (the initial preset values ​​are 0.5 and 0.5).

[0065] Step 350: Traverse all voxels in the main voxel matrix and auxiliary voxel matrix in the negative direction of the first coordinate axis, and perform weighted calculations on the same voxels in the main voxel matrix and auxiliary voxel matrix according to the first weight strategy to obtain the first part of the value in the main and auxiliary voxel matrices.

[0066] Step 360: Traverse all voxels in the main voxel matrix and auxiliary voxel matrix in the positive direction of the first coordinate axis, and perform weighted calculations on the same voxels in the main voxel matrix and auxiliary voxel matrix according to the second weighting strategy to obtain the second part of the value in the main and auxiliary voxel matrices.

[0067] In the first weighting strategy, the weight of a voxel in the main voxel matrix is ​​greater than the weight of a voxel in the auxiliary voxel matrix; in the second weighting strategy, the weight of a voxel in the main voxel matrix is ​​less than the weight of a voxel in the auxiliary voxel matrix.

[0068] For example, firstly, all voxels in the negative direction of the first coordinate axis of the main voxel matrix and the auxiliary voxel matrix are traversed, and the first weighting strategy is adopted, with the weight values ​​of the main voxel matrix and the auxiliary voxel matrix being w3 and w4 respectively (initial preset values ​​of 0.8 and 0.2), and the weights are calculated accordingly. Then, all voxels in the positive direction of the first coordinate axis of the main voxel matrix and the auxiliary voxel matrix are traversed, and the second weighting strategy is adopted, with the weight values ​​of the main voxel matrix and the auxiliary voxel matrix being w4 and w3 respectively (initial preset values ​​of 0.2 and 0.8), and the weights are calculated accordingly.

[0069] Step 370: Perform three-dimensional reconstruction on the principal and secondary voxel matrices to obtain the three-dimensional reconstruction results.

[0070] In summary, the three-dimensional reconstruction method for multi-view two-dimensional ultrasound images provided in this embodiment uses two opposing viewpoints along the first coordinate axis as the primary and secondary viewpoints. Each viewpoint image set includes data acquired by translating the ultrasound probe along the second and third coordinate axes. The data acquired by translating along different coordinate axes are then used to fill the matrix to obtain the voxel matrix under the corresponding viewpoint. Finally, different weighting strategies are used to perform weighted calculations on the voxels in the corresponding directions of the first coordinate axis of the two voxel matrices, thereby effectively integrating the two-dimensional ultrasound data acquired from the two viewpoints to obtain an accurate primary and secondary voxel matrix.

[0071] In an illustrative embodiment, the principal and secondary voxel matrices obtained from multi-view ultrasound are reconstructed and filled in three dimensions.

[0072] Figure 5 This is a flowchart illustrating a method for three-dimensional reconstruction of multi-view two-dimensional ultrasound images according to an exemplary embodiment. This method is applied in computer equipment. Figure 5 As shown, step 240 (or step 370) above can be replaced by the following steps:

[0073] Step 510: Scale the principal and sub-voxel matrices proportionally from the view coordinate system to the reconstructed coordinate system to obtain the reconstructed principal and sub-voxel matrices.

[0074] In this embodiment of the application, the reconstructed coordinate system R is assumed to be a proportional mapping of the view coordinate system V, and P R = RV P V P R To reconstruct the coordinates in the coordinate system, P V T represents the coordinates in the view coordinate system. RV This can be understood as the scaling ratio. Through the above coordinate transformation, the principal and sub-voxel matrices are converted into reconstructed principal and sub-voxel matrices.

[0075] It is understandable that scaling the data proportionally can change the computational load and accuracy of the reconstruction, such as increasing or decreasing the amount of data computation in the interpolation process, thereby speeding up the calculation or increasing the reconstruction accuracy.

[0076] Step 520: Use bicubic interpolation to perform three-dimensional reconstruction of the reconstructed principal and secondary voxel matrices to obtain the three-dimensional reconstruction result.

[0077] In the embodiments of this application, a bicubic interpolation strategy is adopted, such as... Figure 6 As shown, in the reconstructed coordinate system R, 64 points with the closest pixel values ​​are searched along eight directions. Equations are calculated along the x, y, and z directions respectively to solve for the pixel values ​​of the points to be calculated in the reconstructed coordinate system.

[0078]

[0079] in, a = -0.5.

[0080] In one possible implementation, step 520 includes: dividing the reconstructed principal and secondary voxel matrix into multiple sub-regions; for each sub-region, performing bicubic interpolation 3D reconstruction in parallel using a separate GPU to obtain the sub-region 3D reconstruction result; and fusing all the sub-region 3D reconstruction results to obtain the final 3D reconstruction result.

[0081] In this implementation, since the calculations between each voxel are independent, a multi-threaded GPU acceleration approach can be used to divide the entire voxel into multiple sub-regions, which are then assigned to different threads for simultaneous interpolation calculations, thereby reducing reconstruction time.

[0082] In summary, the three-dimensional reconstruction method for multi-view two-dimensional ultrasound images provided in this embodiment performs three-dimensional reconstruction and filling of the principal and secondary voxel matrices obtained from multi-view ultrasound. Furthermore, the reconstruction calculation process can be accelerated by multi-threaded GPUs to reduce reconstruction time and improve reconstruction accuracy.

[0083] In the illustrative embodiment, the coordinate transformation method from the ultrasonic coordinate system to the view coordinate system can refer to the following process:

[0084] (1) Determine the coordinate transformation between the magnetic positioning receiver and the magnetic positioning transmitter.

[0085] Let the coordinate system of the magnetic positioning transmitter be W, which remains fixed during a single experiment. The coordinate system of the magnetic positioning receiver is S, which changes as the receiver moves or rotates. Let P be the coordinate matrix formed by these coordinates in the S coordinate system. S Each column represents a coordinate point. At any given time, let P be the matrix of coordinates in the W coordinate system. W Then P W =T WS P S T WS The data is obtained from the six degrees of freedom data of the magnetic positioning receiver in the physical space W. The six degrees of freedom data includes three translational components and three angular components, representing the offset and attitude of the magnetic positioning receiver relative to the origin of the physical space coordinate system.

[0086] (2) Determine the coordinate transformation between the ultrasonic probe and the magnetic positioning receiver.

[0087] Let the ultrasonic coordinate system be U, which changes as the ultrasonic probe moves, and the magnetic positioning receiver coordinate system be S. Let P be the coordinate matrix formed by the coordinates in the S coordinate system. S Each column represents a coordinate point. At any given time, let P be the matrix of coordinates in the U coordinate system.U Then P S =T SU P U T SU The fixed matrix, which can be calculated from the ultrasonic probe fixture, includes three translational components and three angular components, representing the offset and orientation of the ultrasonic probe origin relative to the origin of the sensor coordinate system.

[0088] (3) Define the view coordinate system

[0089] Let the view coordinate system be V, and let P be the coordinate matrix formed by the coordinates in the V coordinate system. V Each column represents a coordinate point. To make the displayed view range as close as possible to the actual required imaging range, while minimizing the computational load of 3D imaging, the ultrasonic coordinate system of the first or intermediate frames can be selected as the view coordinate system. Therefore, the view coordinate system for a single experiment is fixed, and the transformation from the magnetic positioning transmitter to the view coordinates is a fixed matrix, i.e., P. W =T WS0 P S =T WS0 T SU P U0 =T WS0 T Su P V T WS0 It is the T corresponding to the selected frame. WS P U0 It is the selected frame corresponding to P u The x and y coordinates of the view are respectively based on the imaging size I of the ultrasound image. W (Image width) and I H (Imaging height) is the maximum value, and the value in the z direction can be expanded according to actual needs.

[0090] (4) Transformation relationship from ultrasonic coordinate system to view coordinate system

[0091] By P W =T WS0 T SU P V =T WS P S =T WS T SU P U The transformation matrix T from the ultrasonic coordinate system to the view coordinate system can be calculated. VU =(T WS0 T SU ) -1 T WS T SU .

[0092] It should be noted that the above method embodiments can be implemented individually or in combination, and this application does not limit them in this regard.

[0093] Figure 7 This is a structural block diagram illustrating a three-dimensional reconstruction device for multi-view two-dimensional ultrasound images according to an exemplary embodiment. The device includes:

[0094] The two-dimensional ultrasound data acquisition module 701 is used to acquire two-dimensional ultrasound data acquired by the ultrasound probe under the main view and the auxiliary view, and to obtain the main view image set and the auxiliary view image set.

[0095] The voxel matrix acquisition module 702 is used to transform the main view image set and the auxiliary view image set from the ultrasound coordinate system to the view coordinate system to obtain the main voxel matrix and the auxiliary voxel matrix.

[0096] The matrix weighting calculation module 703 is used to perform weighted calculations on the main voxel matrix and the auxiliary voxel matrix to obtain the main and auxiliary voxel matrices.

[0097] The three-dimensional reconstruction module 704 is used to perform three-dimensional reconstruction on the principal and secondary voxel matrices to obtain the three-dimensional reconstruction result.

[0098] In one possible implementation, the two-dimensional ultrasound data acquisition module 701 is used for:

[0099] The ultrasound probe is placed in the negative direction of the first coordinate axis so that the ultrasound probe is under the main viewpoint; the ultrasound probe is translated along the second coordinate axis and the third coordinate axis respectively to perform two-dimensional ultrasound data acquisition, and obtain the first main viewpoint image set and the second main viewpoint image set;

[0100] The ultrasound probe is placed in the positive direction of the first coordinate axis so that the ultrasound probe is under the auxiliary viewpoint; the ultrasound probe is translated along the second coordinate axis and the third coordinate axis respectively to perform two-dimensional ultrasound data acquisition, and obtain the first auxiliary viewpoint image set and the second auxiliary viewpoint image set.

[0101] In one possible implementation, the voxel matrix acquisition module 702 is used for:

[0102] The first main-view image set is transformed from the ultrasound coordinate system to the view coordinate system, and the resulting first set of pixels is stored in the subject matrix; the second main-view image set is transformed from the ultrasound coordinate system to the view coordinate system, and the resulting second set of pixels is used to update the data in the subject matrix to obtain the final subject matrix.

[0103] The second auxiliary viewpoint image set is transformed from the ultrasound coordinate system to the view coordinate system, and the resulting third set of pixels is stored in the auxiliary voxel matrix; the second auxiliary viewpoint image set is transformed from the ultrasound coordinate system to the view coordinate system, and the data in the auxiliary voxel matrix is ​​updated using the resulting fourth set of pixels to obtain the final auxiliary voxel matrix.

[0104] In one possible implementation, the voxel matrix acquisition module 702 is used for:

[0105] Traverse the pixels in the second group of pixels. If the current pixel does not exist in the main pixel matrix, store the current pixel in the main pixel matrix. If the current pixel already exists in the main pixel matrix, calculate the weighted average of the current pixel and the existing pixels, and replace the existing pixels in the main pixel matrix with the obtained pixel.

[0106] Traverse the pixels in the fourth group of pixels. If the current pixel does not exist in the auxiliary voxel matrix, store the current pixel in the auxiliary voxel matrix. If the current pixel already exists in the auxiliary voxel matrix, calculate the weighted average of the current pixel and the existing pixels, and replace the existing pixels in the auxiliary voxel matrix with the resulting pixel.

[0107] In one possible implementation, the matrix weighting calculation module 703 is used for:

[0108] Traverse all voxels in the main voxel matrix and the auxiliary voxel matrix in the negative direction of the first coordinate axis, and perform weighted calculations on the same voxels in the main voxel matrix and the auxiliary voxel matrix according to the first weighting strategy to obtain the first part of the values ​​in the main and auxiliary voxel matrices;

[0109] Traverse all voxels in the main voxel matrix and the auxiliary voxel matrix in the positive direction of the first coordinate axis, and perform weighted calculations on the same voxels in the main voxel matrix and the auxiliary voxel matrix according to the second weighting strategy to obtain the second part of the value in the main and auxiliary voxel matrices;

[0110] In the first weighting strategy, the weight of a voxel in the main voxel matrix is ​​greater than the weight of a voxel in the auxiliary voxel matrix; in the second weighting strategy, the weight of a voxel in the main voxel matrix is ​​less than the weight of a voxel in the auxiliary voxel matrix.

[0111] In one possible implementation, the 3D reconstruction module 704 is used for:

[0112] The principal and sub-voxel matrices are scaled proportionally from the view coordinate system to the reconstructed coordinate system to obtain the reconstructed principal and sub-voxel matrices.

[0113] The reconstructed principal and secondary voxel matrices are reconstructed in three dimensions using bicubic interpolation to obtain the three-dimensional reconstruction result.

[0114] In one possible implementation, the 3D reconstruction module 704 is used for:

[0115] The reconstructed principal and secondary voxel matrix is ​​divided into multiple sub-regions;

[0116] For each sub-region, a bicubic interpolation method is used to perform 3D reconstruction in parallel on a separate GPU to obtain the 3D reconstruction result of the sub-region.

[0117] The 3D reconstruction results of all sub-regions are fused together to obtain the final 3D reconstruction result.

[0118] It should be noted that the three-dimensional reconstruction device for multi-view two-dimensional ultrasound images provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0119] Please see Figure 8 This is a schematic diagram of a computer device provided according to an exemplary embodiment of the present application. The computer device includes a memory and a processor. The memory is used to store a computer program. When the computer program is executed by the processor, it implements the above-described method for three-dimensional reconstruction of multi-view two-dimensional ultrasound images.

[0120] The processor can be a central processing unit (CPU). It can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations thereof.

[0121] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the methods in the embodiments of this invention. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby implementing the methods described in the above embodiments.

[0122] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0123] In one exemplary embodiment, a computer-readable storage medium is also provided for storing at least one computer program, which is loaded and executed by a processor to implement all or part of the steps in the above-described method. For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, or optical data storage device, etc.

[0124] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0125] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for three-dimensional reconstruction of multi-view two-dimensional ultrasound images, characterized in that, The method includes: Two-dimensional ultrasound data acquired by an ultrasound probe under a primary view and a secondary view are obtained to obtain a primary view image set and a secondary view image set. The primary view and the secondary view are two opposite viewpoints on the same coordinate axis, and each view image set includes data acquired by the ultrasound probe translating along different coordinate axes. The acquisition of two-dimensional ultrasound data from the ultrasound probe under primary and secondary viewpoints, resulting in primary viewpoint image sets and secondary viewpoint image sets, includes: The ultrasound probe is placed in the negative direction of the first coordinate axis so that the ultrasound probe is under the main viewpoint; the ultrasound probe is translated along the second coordinate axis and the third coordinate axis respectively to perform two-dimensional ultrasound data acquisition, and obtain the first main viewpoint image set and the second main viewpoint image set; The ultrasound probe is placed in the positive direction of the first coordinate axis so that the ultrasound probe is under the auxiliary viewpoint; the ultrasound probe is translated along the second coordinate axis and the third coordinate axis respectively to perform two-dimensional ultrasound data acquisition, and obtain the first auxiliary viewpoint image set and the second auxiliary viewpoint image set. Transform the main view image set and the auxiliary view image set from the ultrasonic coordinate system to the view coordinate system to obtain the main voxel matrix and the auxiliary voxel matrix; The main voxel matrix and the auxiliary voxel matrix are weighted to obtain the main and auxiliary voxel matrices; The principal and secondary voxel matrices are reconstructed in three dimensions to obtain the three-dimensional reconstruction result; The weighted calculation of the main voxel matrix and the auxiliary voxel matrix to obtain the main and auxiliary voxel matrices includes: Traverse all voxels in the main voxel matrix and the auxiliary voxel matrix in the negative direction of the first coordinate axis, and perform weighted calculations on the same voxels in the main voxel matrix and the auxiliary voxel matrix according to the first weighting strategy to obtain the first part of the values ​​in the main and auxiliary voxel matrices; Traverse all voxels in the main voxel matrix and the auxiliary voxel matrix in the positive direction of the first coordinate axis, and perform weighted calculations on the same voxels in the main voxel matrix and the auxiliary voxel matrix according to the second weighting strategy to obtain the second part of the value in the main and auxiliary voxel matrices; In the first weighting strategy, the weight of a voxel in the main voxel matrix is ​​greater than the weight of a voxel in the auxiliary voxel matrix; in the second weighting strategy, the weight of a voxel in the main voxel matrix is ​​less than the weight of a voxel in the auxiliary voxel matrix.

2. The method according to claim 1, characterized in that, The process of transforming the primary viewpoint image set and the secondary viewpoint image set from the ultrasonic coordinate system to the view coordinate system to obtain the main voxel matrix and the secondary voxel matrix includes: The first main-view image set is transformed from the ultrasound coordinate system to the view coordinate system, and the resulting first set of pixels is stored in the subject matrix; the second main-view image set is transformed from the ultrasound coordinate system to the view coordinate system, and the resulting second set of pixels is used to update the data in the subject matrix to obtain the final subject matrix. The second auxiliary viewpoint image set is transformed from the ultrasound coordinate system to the view coordinate system, and the resulting third set of pixels is stored in the auxiliary voxel matrix; the second auxiliary viewpoint image set is transformed from the ultrasound coordinate system to the view coordinate system, and the data in the auxiliary voxel matrix is ​​updated using the resulting fourth set of pixels to obtain the final auxiliary voxel matrix.

3. The method according to claim 2, characterized in that, The step of updating the data in the subject matrix using the obtained second set of pixels to obtain the final subject matrix includes: Traverse the pixels in the second group of pixels. If the current pixel does not exist in the main pixel matrix, store the current pixel in the main pixel matrix. If the current pixel already exists in the main pixel matrix, calculate the weighted average of the current pixel and the existing pixels, and replace the existing pixels in the main pixel matrix with the obtained pixel. The step of updating the data in the auxiliary voxel matrix using the obtained fourth group of pixels to obtain the final auxiliary voxel matrix includes: Traverse the pixels in the fourth group of pixels. If the current pixel does not exist in the auxiliary voxel matrix, store the current pixel in the auxiliary voxel matrix. If the current pixel already exists in the auxiliary voxel matrix, calculate the weighted average of the current pixel and the existing pixels, and replace the existing pixels in the auxiliary voxel matrix with the resulting pixel.

4. The method according to claim 1, characterized in that, The three-dimensional reconstruction of the principal and secondary voxel matrices to obtain the three-dimensional reconstruction result includes: The principal and sub-voxel matrices are scaled proportionally from the view coordinate system to the reconstructed coordinate system to obtain the reconstructed principal and sub-voxel matrices. The reconstructed principal and secondary voxel matrices are reconstructed in three dimensions using bicubic interpolation to obtain the three-dimensional reconstruction result.

5. The method according to claim 4, characterized in that, The method employs bicubic interpolation to perform three-dimensional reconstruction of the reconstructed principal and secondary voxel matrices, obtaining the three-dimensional reconstruction result, including: The reconstructed principal and secondary voxel matrix is ​​divided into multiple sub-regions; For each sub-region, a bicubic interpolation method is used to perform 3D reconstruction in parallel on a separate GPU to obtain the 3D reconstruction result of the sub-region. The 3D reconstruction results of all sub-regions are fused together to obtain the final 3D reconstruction result.

6. A three-dimensional reconstruction device for multi-view two-dimensional ultrasound images, characterized in that, The apparatus for performing a three-dimensional reconstruction method of multi-view two-dimensional ultrasound images as described in any one of claims 1 to 5, the apparatus comprising: The two-dimensional ultrasound data acquisition module is used to acquire two-dimensional ultrasound data collected by the ultrasound probe in the main view and the auxiliary view, and to obtain the main view image set and the auxiliary view image set. The voxel matrix acquisition module is used to transform the main view image set and the auxiliary view image set from the ultrasound coordinate system to the view coordinate system to obtain the main voxel matrix and the auxiliary voxel matrix. The matrix weighting calculation module is used to perform weighted calculations on the main voxel matrix and the auxiliary voxel matrix to obtain the main and auxiliary voxel matrices; The three-dimensional reconstruction module is used to perform three-dimensional reconstruction on the principal and secondary voxel matrices to obtain the three-dimensional reconstruction results.

7. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, code set, or instruction set, the at least one instruction, at least one program, code set, or instruction set being loaded and executed by the processor to implement the three-dimensional reconstruction method of multi-view two-dimensional ultrasound images as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the three-dimensional reconstruction method of multi-view two-dimensional ultrasound images as described in any one of claims 1 to 5.

Images