Ultrasonic three-dimensional wide-scene imaging method and ultrasonic equipment

Abstract

The invention discloses an ultrasonic three-dimensional wide-scene imaging method and ultrasonic equipment. The ultrasonic three-dimensional wide-scene imaging method comprises the following steps: firstly, acquiring a plurality of initial three-dimensional data volumes on tissues and organs through a volume probe; then, for each initial three-dimensional data volume, polar coordinate information is converted into Cartesian coordinate information, and a target three-dimensional data volume is obtained; extracting feature points and feature point descriptors of each target three-dimensional data volume based on an ORB algorithm, and matching the target three-dimensional data volumes to obtain a plurality of spatial transformation matrixes; and finally, obtaining three-dimensional wide-scene image data by adopting a weighted stitching method. According to the embodiment of the invention, the target three-dimensional data volumes are registered based on the ORB algorithm, and the three-dimensional wide-scene image is finally obtained, so that the fusion effect of the scanned image with a certain rotation angle is better, and the finally obtained three-dimensional wide-scene image has better continuity, smoothness and stability.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to an ultrasonic three-dimensional panoramic imaging method and an ultrasonic device. Background Art

[0002] In the traditional ultrasonic two-dimensional panoramic imaging technology, a series of two-dimensional images of the surface of the observed tissue or organ are collected by a probe, and then stitched into a continuous large-field-of-view image. However, since the ultrasonic two-dimensional panoramic imaging technology can only display the sectional view of the tissue or organ at a certain angle and cannot display the change of the surface curvature of the tissue or organ, it is impossible to completely present the three-dimensional structure of the tissue or organ, and there are limitations. The traditional ultrasonic three-dimensional imaging technology can break through the limitations of the ultrasonic two-dimensional panoramic imaging technology and can present the three-dimensional structure of the observed tissue or organ. However, due to the limitation of the width and scanning angle of the ultrasonic volume probe, it is impossible to completely present the three-dimensional structure of a relatively large tissue or organ.

[0003] To solve the above problems, the three-dimensional panoramic imaging technology has emerged. However, the three-dimensional panoramic imaging technology in the prior art adopts a manual scanning method, and a three-dimensional panoramic image with a better fusion effect can only be obtained when two three-dimensional images are parallel. The fusion effect of scanned images with a certain rotation angle is not good. Summary of the Invention

[0004] The present invention provides an ultrasonic three-dimensional panoramic imaging method and an ultrasonic device to solve the problem that the three-dimensional panoramic imaging technology in the prior art has a poor fusion effect on scanned images with a certain rotation angle.

[0005] In a first aspect, an embodiment of the present application provides an ultrasonic three-dimensional panoramic imaging method, and the method includes:

[0006] Collecting a plurality of initial three-dimensional data volumes on a tissue or organ in a polar coordinate system through a volume probe;

[0007] For each initial three-dimensional data volume, based on a scan conversion algorithm, after converting the polar coordinate information corresponding to each initial three-dimensional data into Cartesian coordinate information, performing interpolation calculation on each Cartesian coordinate information to obtain a target three-dimensional data volume;

[0008] After extracting feature points and feature point descriptors of each target three-dimensional data volume based on the ORB algorithm, based on the extracted feature points and feature point descriptors, matching a fixed three-dimensional data volume with each to-be-registered three-dimensional data volume respectively to obtain a plurality of spatial transformation matrices, where the fixed three-dimensional data volume is any one of the obtained target three-dimensional data volumes, and the to-be-registered three-dimensional data volume is other target three-dimensional data volumes except the fixed three-dimensional data volume;

[0009] The multiple spatial transformation matrices are calculated by using a weighted splicing method. After obtaining the three-dimensional panoramic image data, the three-dimensional panoramic image is displayed based on the three-dimensional panoramic image data.

[0010] In a possible implementation manner, the interpolating and calculating each piece of Cartesian coordinate information to obtain a target three-dimensional data volume includes:

[0011] For each piece of Cartesian coordinate information, three-axis interpolation is performed on the Cartesian coordinate information based on a preset number of pixel points adjacent to each other on the scan line in the polar coordinate system, and the pixel values of the preset number of pixel points are obtained respectively;

[0012] For each pixel value among the pixel values of the preset number of pixel points, three-axis horizontal interpolation is performed on the pixel value along the horizontal axis in the polar coordinate system;

[0013] The data obtained after the three-axis horizontal interpolation is used as the target three-dimensional data volume.

[0014] In a possible implementation manner, after the interpolating and calculating each piece of Cartesian coordinate information to obtain a target three-dimensional data volume, it further includes:

[0015] Performing Gaussian filtering processing and / or histogram equalization processing on the target three-dimensional data volume.

[0016] In a possible implementation manner, the extracting the feature points and feature point descriptors of each target three-dimensional data volume based on the ORB algorithm includes:

[0017] For each target three-dimensional data volume, the feature points of the target three-dimensional data volume are extracted by using the Oriented FAST algorithm;

[0018] The feature point descriptors of the target three-dimensional data volume are extracted by using the Rotated BRIEF algorithm based on the feature points of the target three-dimensional data volume.

[0019] In a possible implementation manner, the matching the fixed three-dimensional data volume with each to-be-registered three-dimensional data volume respectively based on the extracted feature points and feature point descriptors to obtain multiple spatial transformation matrices includes:

[0020] For each group of the fixed three-dimensional data volume and the to-be-registered three-dimensional data volume, based on the BFMatcher algorithm, according to the feature point descriptors of the fixed three-dimensional data volume and the feature point descriptors of the to-be-registered three-dimensional data volume, the feature points of the fixed three-dimensional data volume and the feature points of the to-be-registered three-dimensional data volume are matched to obtain multiple groups of feature points;

[0021] Based on the RANSAC method, screen the multiple sets of feature points, and use the screened feature points as the spatial transformation matrix.

[0022] In a possible implementation manner, after matching the fixed three-dimensional data volume with each three-dimensional data volume to be registered respectively based on the extracted feature points and feature point descriptors to obtain multiple spatial transformation matrices, it further includes:

[0023] For each spatial transformation matrix, solve the spatial transformation matrix and the global transformation matrix based on the SVD method to obtain the first rotation angle and the first translation distance corresponding to the spatial transformation matrix, and obtain the second rotation angle and the second translation distance corresponding to the global transformation matrix, where the global transformation matrix is calculated based on the three-dimensional panoramic image data obtained by the weighted stitching method last time;

[0024] According to the first rotation angle, the second rotation angle, the first translation distance, the second translation distance, a preset threshold, and the number of successfully registered times, determine whether the spatial transformation matrix is successfully registered;

[0025] If the registration is successful, use the weighted stitching method with the successfully registered spatial transformation matrix to obtain the three-dimensional panoramic image data of this time; if the registration is unsuccessful, discard the three-dimensional data volume to be registered corresponding to the unsuccessfully registered spatial transformation matrix.

[0026] In a possible implementation manner, the determining whether the spatial transformation matrix is successfully registered according to the first rotation angle, the second rotation angle, the first translation distance, the second translation distance, a preset threshold, and the number of successfully registered times includes:

[0027] If the following preset conditions are met, determine that the spatial transformation matrix is successfully registered, otherwise, determine that the spatial transformation matrix is unsuccessfully registered, where the preset conditions include all of the following:

[0028] The first translation distance is less than or equal to the first preset threshold;

[0029] The first translation distance is less than or equal to the second translation distance divided by the number of successfully registered times;

[0030] The first rotation angle is less than or equal to the second preset threshold;

[0031] The first rotation angle is less than or equal to the second rotation angle divided by the number of successfully registered times.

[0032] In a second aspect, an embodiment of the present application provides an ultrasonic device, including a volume probe, a processing unit, and a display unit;

[0033] The volume probe is used to collect multiple initial three-dimensional data volumes on a tissue organ in a polar coordinate system;

[0034] The processing unit is configured to, for each initial three-dimensional data volume, based on a scan conversion algorithm, convert the polar coordinate information corresponding to each initial three-dimensional data volume into Cartesian coordinate information, and then perform interpolation calculation on each Cartesian coordinate information to obtain a target three-dimensional data volume; after extracting feature points and feature point descriptors corresponding to each target three-dimensional data volume based on the ORB algorithm, based on the extracted feature points and feature point descriptors, match the fixed three-dimensional data volume with each to-be-configured three-dimensional data volume respectively to obtain multiple spatial transformation matrices, where the fixed three-dimensional data volume is any one of the obtained target three-dimensional data volumes, and the to-be-registered three-dimensional data volume is other target three-dimensional data volumes except the fixed three-dimensional data volume; calculate the multiple spatial transformation matrices by using a weighted stitching method to obtain three-dimensional panoramic image data;

[0035] The display unit is used to display a three-dimensional panoramic image corresponding to the three-dimensional panoramic image data.

[0036] In a possible implementation manner, the processing unit is specifically configured to:

[0037] For each Cartesian coordinate information, perform cubic axial interpolation on the Cartesian coordinate information based on a preset number of pixel points adjacent to the scan line in the polar coordinate system to respectively obtain the pixel values of the preset number of pixel points;

[0038] For each pixel value among the pixel values of the preset number of pixel points, perform cubic lateral interpolation on the pixel value along the transverse axis in the polar coordinate system;

[0039] Use the data obtained after the cubic lateral interpolation as the target three-dimensional data volume.

[0040] In a third aspect, an embodiment of the present application provides an ultrasonic device, and the device includes:

[0041] An acquisition module, which acquires multiple initial three-dimensional data volumes on a tissue organ in a polar coordinate system through a volume probe;

[0042] A reconstruction module, which, for each initial three-dimensional data volume, based on a scan conversion algorithm, converts the polar coordinate information corresponding to each initial three-dimensional data volume into Cartesian coordinate information, and then performs interpolation calculation on each Cartesian coordinate information to obtain a target three-dimensional data volume;

[0043] The registration module, after extracting the feature points and feature point descriptors of each target three-dimensional data volume based on the ORB algorithm, based on the extracted feature points and feature point descriptors, matches the fixed three-dimensional data volume with each three-dimensional data volume to be registered respectively, and obtains a plurality of spatial transformation matrices, wherein the fixed three-dimensional data volume is any one of the obtained target three-dimensional data volumes, and the three-dimensional data volume to be registered is other target three-dimensional data volumes except the fixed three-dimensional data volume;

[0044] The fusion module calculates the plurality of spatial transformation matrices by using the weighted stitching method to obtain three-dimensional panoramic image data;

[0045] The display module is used to display the three-dimensional panoramic image corresponding to the three-dimensional panoramic image data.

[0046] The beneficial effects of the present invention are as follows:

[0047] An ultrasonic three-dimensional panoramic imaging method and an ultrasonic device provided by an embodiment of the present application first collect a plurality of initial three-dimensional data volumes on a tissue organ located in a polar coordinate system through a volume probe; then, for each initial three-dimensional data volume, based on the scan conversion algorithm, after converting the polar coordinate information corresponding to each initial three-dimensional data volume into Cartesian coordinate information, interpolation calculation is performed on each Cartesian coordinate information to obtain a target three-dimensional data volume; after extracting the feature points and feature point descriptors of each target three-dimensional data volume based on the ORB algorithm, based on the extracted feature points and feature point descriptors, the fixed three-dimensional data volume is respectively matched with each three-dimensional data volume to be registered to obtain a plurality of spatial transformation matrices; finally, the plurality of spatial transformation matrices are calculated by using the weighted stitching method to obtain three-dimensional panoramic image data, and then the three-dimensional panoramic image is displayed based on the three-dimensional panoramic image data. In the embodiment of the present application, since the ORB algorithm is used to register between the target three-dimensional data volumes, and finally a three-dimensional panoramic image is obtained, the fusion effect of the scanned images with a certain rotation angle is better, so that the finally obtained three-dimensional panoramic image has better continuity, smoothness and stability. Description of the Drawings

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can be obtained according to these drawings without creative efforts.

[0049] Figure 1 It is a schematic structural diagram of an ultrasonic device provided by an embodiment of the present application;

[0050] Figure 2 It is a flowchart of the ultrasonic three-dimensional panoramic imaging method provided by an embodiment of the present application;

[0051] Figure 3 Schematic diagram for converting polar coordinate information into Cartesian coordinate information provided by an embodiment of the present application;

[0052] Figure 4 Flowchart for performing interpolation calculation on each piece of Cartesian coordinate information provided by an embodiment of the present application;

[0053] Figure 5(a) is a schematic diagram of cubic axial interpolation provided by an embodiment of the present application;

[0054] Figure 5(b) is a schematic diagram of cubic lateral interpolation provided by an embodiment of the present application;

[0055] Figure 6 Flowchart for extracting feature points and feature point descriptors of each target three-dimensional data volume based on the ORB algorithm provided by an embodiment of the present application;

[0056] Figure 7 Flowchart for extracting feature points of a target three-dimensional data volume through the Oriented FAST algorithm provided by an embodiment of the present application;

[0057] Figure 8 Flowchart for extracting feature point descriptors of a target three-dimensional data volume provided by an embodiment of the present application;

[0058] Figure 9 Flowchart for matching a fixed three-dimensional data volume with each three-dimensional data volume to be registered respectively provided by an embodiment of the present application;

[0059] Figure 10 Flowchart for making a rationality decision judgment on multiple spatial transformation matrices provided by an embodiment of the present application;

[0060] Figure 11 Schematic diagram of three-dimensional panoramic image display provided by an embodiment of the present application;

[0061] Figure 12 Schematic diagram of the structure of an ultrasonic device provided by an embodiment of the present application. Detailed implementation manners

[0062] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0063] The application scenarios described in the embodiments of the present disclosure are for more clearly illustrating the technical solutions of the embodiments of the present disclosure, and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. Those of ordinary skill in the art will know that with the emergence of new application scenarios, the technical solutions provided by the embodiments of the present disclosure are equally applicable to similar technical problems.

[0064] Moreover, in the description of the embodiments of the present application, unless otherwise specified, " / " means "or". For example, A / B may represent A or B; "and / or" in the text is only a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone.

[0065] Hereinafter, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as implying or suggesting relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality" is two or more.

[0066] Traditional two-dimensional ultrasound panoramic imaging technology can only display the sectional view of a tissue or organ at a certain angle and cannot display the change of the surface curvature of the tissue or organ. Traditional three-dimensional ultrasound imaging technology is limited by the width and scanning angle of the ultrasound volume probe and cannot completely present the three-dimensional structure of a relatively large tissue or organ. To solve the above problems, three-dimensional panoramic imaging technology has emerged. However, in the existing three-dimensional panoramic imaging technology, if the manual scanning method is adopted, a good fusion effect can only be obtained when two three-dimensional images are parallel, and the fusion effect of scanned images with a certain rotation angle is not good.

[0067] In view of the above problems, the present application provides an ultrasound three-dimensional panoramic imaging method and an ultrasound device. Hereinafter, the application scenario of an ultrasound three-dimensional panoramic imaging method provided by the embodiments of the present application will be introduced with reference to the accompanying drawings. As Figure 1 shown, it is a schematic structural diagram of an ultrasound device. This ultrasound device can perform three-dimensional panoramic imaging. The ultrasound device includes a volume probe 101, a processing unit 102, and a display screen 103, where:

[0068] The volume probe 101 is configured to collect a plurality of initial three-dimensional data volumes located in a polar coordinate system on a tissue or organ;

[0069] The processing unit 102 is configured to, for each initial three-dimensional data volume, based on a scan conversion algorithm, convert each polar coordinate information into Cartesian coordinate information, and then perform interpolation calculation on each Cartesian coordinate information to obtain a target three-dimensional data volume; after extracting feature points and feature point descriptors of each target three-dimensional data volume based on the ORB algorithm, based on the extracted feature points and feature point descriptors, perform matching between the fixed three-dimensional data volume and each three-dimensional data volume to be registered respectively, to obtain a plurality of spatial transformation matrices, where the fixed three-dimensional data volume is any one of the obtained target three-dimensional data volumes, and the three-dimensional data volumes to be registered are other target three-dimensional data volumes except the fixed three-dimensional data volume; calculate the plurality of spatial transformation matrices by using a weighted stitching method to obtain three-dimensional panoramic image data;

[0070] The display screen 103 is configured to display a three-dimensional panoramic image corresponding to the three-dimensional panoramic image data.

[0071] Figure 1 Among them, a plurality of initial three-dimensional data volumes on the tissue organ are collected by the volume probe 101, and then through the ultrasonic device processor 102, for each initial three-dimensional data volume, based on the scan conversion algorithm, a target three-dimensional data volume is obtained, the target three-dimensional data volume is registered to obtain a plurality of spatial transformation matrices, the plurality of spatial transformation matrices are calculated by using a weighted stitching method to obtain three-dimensional panoramic image data, and finally through the display screen 103, a three-dimensional panoramic image corresponding to the three-dimensional panoramic image data is displayed.

[0072] The method provided by the embodiments of the present application is not limited to Figure 1 the application scenarios shown, and can also be used in other possible application scenarios, which are not limited in the embodiments of the present application.

[0073] As Figure 2 shown, it is a flowchart of the ultrasonic three-dimensional panoramic imaging method according to the embodiments of the present application, and the specific steps are as follows:

[0074] S201. Collect a plurality of initial three-dimensional data volumes on the tissue organ in the polar coordinate system through the volume probe;

[0075] S202. For each initial three-dimensional data volume, based on the scan conversion algorithm, convert the polar coordinate information corresponding to each initial three-dimensional data volume into Cartesian coordinate information, and then perform interpolation calculation on each Cartesian coordinate information to obtain a target three-dimensional data volume;

[0076] S203. After extracting the feature points and feature point descriptors of each target three-dimensional data volume based on the ORB algorithm, based on the extracted feature points and feature point descriptors, the fixed three-dimensional data volume is respectively matched with each three-dimensional data volume to be registered to obtain a plurality of spatial transformation matrices, where the fixed three-dimensional data volume is any one of the obtained target three-dimensional data volumes, and the three-dimensional data volume to be registered is other target three-dimensional data volumes except the fixed three-dimensional data volume;

[0077] S204. Calculate the plurality of spatial transformation matrices by using the weighted stitching method. After obtaining the three-dimensional panoramic image data, display the three-dimensional panoramic image based on the three-dimensional panoramic image data.

[0078] An ultrasonic three-dimensional panoramic imaging method provided by an embodiment of the present application first collects a plurality of initial three-dimensional data volumes located in a polar coordinate system on a tissue organ through a volume probe; then, for each initial three-dimensional data volume, based on the scan conversion algorithm, after converting the polar coordinate information corresponding to each initial three-dimensional data volume into Cartesian coordinate information, perform interpolation calculation on each Cartesian coordinate information to obtain a target three-dimensional data volume; after extracting the feature points and feature point descriptors of each target three-dimensional data volume based on the ORB algorithm, based on the extracted feature points and feature point descriptors, the fixed three-dimensional data volume is respectively matched with each three-dimensional data volume to be registered to obtain a plurality of spatial transformation matrices; finally, calculate the plurality of spatial transformation matrices by using the weighted stitching method. After obtaining the three-dimensional panoramic image data, display the three-dimensional panoramic image based on the three-dimensional panoramic image data. In the embodiment of the present application, since the ORB algorithm is used to register the target three-dimensional data volumes, a three-dimensional panoramic image is finally obtained, so that the fusion effect of the scanned images with a certain rotation angle is better, and the finally obtained three-dimensional panoramic image has better continuity, smoothness and stability.

[0079] It should be noted that the working principle of the volume probe is to emit ultrasonic waves through the swing of the motor inside it and receive the echoes carrying the two-dimensional section information of the tissue organ. During the process of collecting the tissue organ multiple times through the volume probe in a manual scanning manner, first let the volume probe stop at a certain position of the tissue organ to collect a plurality of two-dimensional sections, and the plurality of collected two-dimensional sections form an initial three-dimensional data volume. Then move the volume probe to another position to collect a plurality of two-dimensional sections, and move the volume probe multiple times in sequence to obtain a plurality of initial three-dimensional data volumes, where there needs to be an overlapping area between the first collected position and the subsequent collected positions.

[0080] In a specific embodiment, what is collected from the volume probe is a plurality of initial three-dimensional data volumes located in a polar coordinate system on a tissue organ, and what is obtained is a plurality of polar coordinate information located in each initial three-dimensional data volume. Therefore, it is necessary to perform three-dimensional data reconstruction on the polar coordinate information.

[0081] In the embodiments of the present application, the three-dimensional data in the polar coordinate system are the included angle θ and the depth R. For example, the initial three-dimensional data of point a1 is (θ1, R1), the initial three-dimensional data of point a2 is (θ2, R2), the initial three-dimensional data of point a3 is (θ3, R3), and the initial three-dimensional data of point a4 is (θ4, R4). Then the initial three-dimensional data volume is {(θ1, R1), (θ2, R2), (θ3, R3), (θ4, R4)}.

[0082] In a feasible implementation manner, a scan conversion algorithm is adopted and a GPU (Graphics Processing Unit) is used for acceleration to improve performance. The following gives a method for three-dimensional data reconstruction based on the scan conversion algorithm.

[0083] 1) Convert the polar coordinate information into Cartesian coordinate information

[0084] As a feasible implementation manner, as Figure 3 shown, it is a schematic diagram for converting polar coordinate information into Cartesian coordinate information provided by the embodiments of the present application. For example, taking a certain pixel point A in the Cartesian coordinate system as an example, if the Cartesian coordinate information of A is (X a , Y a ), the coordinates (X a , Y a) ) of A in the Cartesian coordinate information corresponding to the polar coordinate information can be obtained through the included angle θ and the depth R in the known polar coordinate information:

[0085]

[0086] Y a = (R - probeRadius) + sampleNum × deadCoe

[0087] where startAngle is the scanning angle of the volume probe, 2α is the scanning radian of the volume probe, σ is the radian interval between every two scanning lines, probeRadius is the radius of the volume probe, sampleNum is the number of pixel points on the scanning line with depth R, and deadCoe is the dead zone coefficient.

[0088] 2) Perform interpolation calculation on each Cartesian coordinate information to obtain the target three-dimensional data volume

[0089] As a feasible implementation manner, as Figure 4 shown, it is the steps for performing interpolation calculation on each Cartesian coordinate information provided by the embodiments of the present application.

[0090] S401. For each piece of Cartesian coordinate information, perform cubic axial interpolation on the Cartesian coordinate information based on a preset number of adjacent pixel points on the scan line in the polar coordinate system to obtain the pixel values of the preset number of pixel points respectively.

[0091] In a specific embodiment, as shown in Fig. 5(a), it is a schematic diagram of cubic axial interpolation provided by an embodiment of the present application. For example, taking the point (X, Y) as an example, perform cubic axial interpolation based on 4 adjacent pixel points on the scan line to obtain the pixel values Pixel of four pixel points (X_1, Y), (X0, Y), (X1, Y), and (X2, Y) respectively. The axial difference calculation formula is as follows:

[0092] Pixel(X_1, Y) = h(1 + YoffSet) × Pixel(X_1, Y_1) + h(YoffSet) × Pixel(X_1, Y0) + h(1 - YoffSet) × Pixel(X_1, Y1) + h(2 - YoffSet) × Pixel(X_1, Y2)

[0093] Pixel(X0, Y) = h(1 + YoffSet) × Pixel(X0, Y_1) + h(YoffSet) × Pixel(X0, Y0) + h(1 - YoffSet) × Pixel(X0, Y1) + h(2 - YoffSet) × Pixel(X0, Y2)

[0094] Pixel(X1, Y) = h(1 + YoffSet) × Pixel(X1, Y_1) + h(YoffSet) × Pixel(X1, Y0) + h(1 - YoffSet) × Pixel(X1, Y1) + h(2 - YoffSet) × Pixel(X1, Y2)

[0095] Pixel(X2, Y) = h(1 + YoffSet) × Pixel(X2, Y_1) + h(YoffSet) × Pixel(X2, Y0) + h(1 - YoffSet) × Pixel(X2, Y1) + h(2 - YoffSet) × Pixel(X2, Y2)

[0096] Among them, the calculation formula of h(x) is as follows:

[0097]

[0098] It should be noted that α is a constant, and its value is -1 or -0.75 or -0.5.

[0099] S402. For each pixel value among the pixel values of a preset number of pixel points, perform three times of horizontal interpolation on the pixel value along the horizontal axis in the polar coordinate system.

[0100] In a specific embodiment, as shown in FIG. 5(b), it is a schematic diagram of three - time horizontal interpolation provided by an embodiment of the present application. Based on the pixel values of four pixel points (X_1, Y), (X0, Y), (X1, Y), and (X2, Y) obtained by axial interpolation, perform three - time horizontal interpolation to obtain the pixel value of (X, Y). The horizontal difference calculation formula is as follows:

[0101] Pixel(X, Y) = h(1 + XoffSet)×Pixel(X_1, Y)+h(XoffSet)×Pixel(X_1, Y)+h(1 - XoffSet)×Pixel(X_1, Y1)+h(2 - XoffSet)×Pixel(X_1, Y)

[0102] Wherein, XoffSet is the offset in the X - axis direction, and YoffSet is the offset in the Y - axis direction.

[0103] S403. Take the data obtained after three - time horizontal interpolation as the target three - dimensional data volume.

[0104] In the embodiment of the present application, the data to be reconstructed can be subjected to three - time axial interpolation calculation through CUDA (Compute Unified Device Architecture), and then three - time horizontal interpolation calculation, and finally obtain the target three - dimensional data volume after three - dimensional reconstruction. For example, there are 1 million data to be reconstructed. During the target three - dimensional data reconstruction process, each data needs to be subjected to a three - time axial interpolation calculation and then a three - time horizontal interpolation calculation, and GPU acceleration processing is adopted to improve performance and ensure timeliness.

[0105] It should be noted that the commonly used interpolation methods in the scan - conversion algorithm include linear interpolation method, quadratic interpolation method, and cubic interpolation method. Among them, the linear interpolation method uses two adjacent pixel points near the scan line for interpolation; the quadratic interpolation method uses three adjacent pixel points near the scan line for interpolation; and the cubic interpolation method used in the present application uses four adjacent pixel points near the scan line for interpolation, and the image quality is better.

[0106] In addition, after performing interpolation calculation on each Cartesian coordinate information to obtain the target three - dimensional data volume, it further includes: performing Gaussian filtering processing and / or histogram equalization processing on the target three - dimensional data volume.

[0107] In a feasible implementation, Gaussian filtering is used to process the noise generated during the 3D reconstruction of the target 3D data volume, and histogram equalization is used to enhance the image contrast of the target 3D data volume.

[0108] In a specific embodiment, before fusing the target 3D data volume, it is also necessary to register the target 3D data volume. For example, taking the mapping transformation of two target 3D data volumes in spatial coordinate positions and voxel values as an example, fix the 3D data volume I 1 (x, y, z) and the 3D data volume I to be registered 2 (x, y, z). The mapping transformation of the two target 3D data volumes in voxel values and spatial coordinate positions is as follows:

[0109] I 1 (x, y, z) = g(I 2 (T(x, y, z)))

[0110] Among them, g is the voxel value transformation matrix, and T is the 3D space transformation matrix. Since there is a large overlapping area between two adjacent target 3D data volumes and the voxel values basically do not change, it is necessary to register the target 3D data volume by obtaining the space transformation matrix between the target 3D data volumes.

[0111] In a feasible implementation, after extracting the feature points and feature point descriptors of each target 3D data volume based on the ORB algorithm, based on the extracted feature points and feature point descriptors, the fixed 3D data volume is respectively matched with each 3D data volume to be registered to obtain multiple space transformation matrices. The following gives the method of extracting feature points and feature point descriptors and matching the fixed 3D data volume with each 3D data volume to be registered during the image registration process.

[0112] 1) Extract the feature points and feature point descriptors of each target 3D data volume based on the ORB algorithm

[0113] As a feasible implementation, as Figure 6 shown, the steps of extracting the feature points and feature point descriptors of each target 3D data volume based on the ORB algorithm provided by the embodiments of the present application are shown. It should be noted that the ORB algorithm includes the Oriented FAST algorithm and the Rotated BRIEF algorithm. Among them, the Oriented FAST algorithm is used to extract the feature points of the target 3D data volume, and the Rotated BRIEF algorithm is used to extract the feature point descriptors of the target 3D data volume.

[0114] S601. For each target 3D data volume, extract the feature points of the target 3D data volume through the Oriented FAST algorithm; as Figure 7 shown, taking a pixel point p as an example, Ip is the pixel value of pixel point p, and the specific steps are as follows:

[0115] S701: With pixel point p as the center and a radius of 3 pixel points, determine a circle. The 16 pixel points on the circle are p 1 , p 2 ……p 16 ;

[0116] S702: For the pixel values of the 16 pixel points on the circle, calculate the differences between the pixel values I p1 , I p2 ……I p16 of the 16 pixel points on the circle and the pixel value I p of pixel point p;

[0117] S703: If there are n pixel points on the circle that satisfy the condition that the difference is greater than the preset value t or the difference is less than the preset value -t, then determine pixel point p as a candidate point, where n can be 12;

[0118] S704: For the obtained multiple candidate points, based on the non-maximum suppression algorithm, perform candidate point screening to obtain multiple feature points.

[0119] It should be noted that in a specific embodiment, the Oriented FAST algorithm is based on the processing of Gaussian blurring, downsampling, and feature point detection of the target three-dimensional data volume, so that the feature points of the obtained target three-dimensional data volume have scale invariance; define and connect the center O and the centroid C of the three-dimensional data volume to obtain the direction δ of the feature point, so that the feature points of the obtained target three-dimensional data volume have rotational invariance.

[0120] S602: Based on the feature points of the target three-dimensional data volume through the Rotated BRIEF algorithm, extract the feature point descriptors of the target three-dimensional data volume. As Figure 8 shown, taking a feature point B of the target three-dimensional data volume as an example, the steps are as follows:

[0121] S801: For the feature point B of the target three-dimensional data volume, take S×S as the neighborhood window with feature point B as the center, where S is preset;

[0122] S802: Select N pairs of random points within the neighborhood window. For the N pairs of random points, repeat binary assignment to form a binary code, and the obtained binary code is the description of feature point B, that is, the feature point descriptor, where N can be 256.

[0123] It should be noted that the Rotated BRIEF algorithm can also calculate S δ based on the direction δ of the feature point, and through S δInstead of S, the resulting feature point descriptors are made rotation invariant.

[0124] Among them, let S δ = R δ × S.

[0125] 2) Based on the extracted feature points and feature point descriptors, match the fixed 3D data volume with each 3D data volume to be registered respectively

[0126] As a feasible implementation, as Figure 9 shown, the steps of matching the fixed 3D data volume with each 3D data volume to be registered provided by the embodiment of the present application.

[0127] S901. For each group of fixed 3D data volume and 3D data volume to be registered, based on the BFMatcher algorithm, according to the feature point descriptors of the fixed 3D data volume and the feature point descriptors of the 3D data volume to be registered, match the feature points of the fixed 3D data volume and the feature points of the 3D data volume to be registered to obtain multiple groups of feature points;

[0128] S902. Based on the RANSAC method, screen the multiple groups of feature points, and use the screened feature points as the spatial transformation matrix.

[0129] It should be noted that in a specific embodiment, during the process of matching the fixed 3D data volume with each 3D data volume to be registered, GPU can also be used for acceleration to shorten the calculation time and ensure the timeliness in clinical applications.

[0130] In addition, as a feasible implementation, in a specific embodiment, based on the extracted feature points and feature point descriptors, after matching the fixed 3D data volume with each 3D data volume to be registered respectively to obtain multiple spatial transformation matrices, in order to ensure the continuity and stability of the subsequent fusion of the target 3D data volume, it is also necessary to judge the rationality of the obtained multiple spatial transformation matrices. As Figure 10 shown, the steps of judging the rationality of multiple spatial transformation matrices provided by the embodiment of the present application.

[0131] S1001. For each spatial transformation matrix, based on the SVD method, solve the spatial transformation matrix and the global transformation matrix to obtain the first rotation angles (θ x1 , θ y1 , θ z1 ) corresponding to the spatial transformation matrix and the first translation distance d 1 , and obtain the second rotation angles (θ x2 , θ y2 , θ z2) and the second translation distance d 2 , where the global transformation matrix is calculated based on the three-dimensional panoramic image data obtained by the weighted stitching method last time;

[0132] For example, taking a spatial transformation matrix C as an example:

[0133]

[0134] Among them, the upper left 3×3 matrix in the spatial transformation matrix C is the rotation matrix D,

[0135]

[0136] Based on the SVD method, the rotation matrix D can be decomposed into a product of an orthogonal matrix U, a diagonal matrix ∑, and the transpose V of an orthogonal matrix, T i.e., D = U∑V T . From this, the rotation angles θ x , θ y , θ z of the X, Y, and Z axes in the Cartesian coordinate system corresponding to the spatial transformation matrix C can be obtained.

[0137] It should be noted that the translation distance

[0138] S1002. According to the first rotation angle (θ x1 , θ y1 , θ z1 ), the second rotation angle (θ x2 , θ y2 , θ z2 ), the first translation distance d 1 , the second translation distance d 2 , the first preset threshold D, the second preset threshold (α, β, ω), and the number of successful registrations t, determine whether the spatial transformation matrix is successfully registered;

[0139] In a specific embodiment, if the following preset conditions are met, it is determined that the spatial transformation matrix is successfully registered; otherwise, it is determined that the spatial transformation matrix is not successfully registered, where the preset conditions include all of the following:

[0140] The first translation distance d 1 is less than or equal to the first preset threshold D;

[0141] The first translation distance d 1 is less than or equal to the second translation distance d 2 divided by the number of successful registrations t;

[0142] The first rotation angle (θ x1 , θ y1 , θz1 ) less than or equal to a second preset threshold (α, β, ω);

[0143] The first rotation angle (θ x1 , θ y1 , θ z1 ) is less than or equal to the second rotation angle (θ x2 , θ y2 , θ z2 ) divided by the number of successful registrations t.

[0144] S1003. If the registration is successful, the spatially transformed matrix of the successful registration is used to obtain the current three-dimensional panoramic image data by weighted stitching; if the registration is unsuccessful, the three-dimensional data volume to be registered corresponding to the spatially transformed matrix of the unsuccessful registration is discarded.

[0145] It should be noted that discarding the three-dimensional data volume to be registered corresponding to the spatially transformed matrix of the unsuccessful registration does not affect the final display effect.

[0146] In addition, after the spatially transformed matrix of the successful registration is used to obtain the current three-dimensional panoramic image data by weighted stitching, it is necessary to solve for the global transformation matrix and feedback the global transformation matrix to the step of making a rationality decision on the spatially transformed matrix in the next time.

[0147] For example, in a feasible implementation, take the acquisition of 5 initial three-dimensional data volumes on a tissue organ by a volume probe as an example.

[0148] Step 1. For the obtained 5 initial three-dimensional data volumes, based on the scan conversion algorithm, convert the polar coordinate information into Cartesian coordinate information, and after interpolation calculation, obtain 5 target three-dimensional data volumes F1, F2, F3, F4, F5.

[0149] Step 2. Based on the ORB algorithm, the target three-dimensional data volume F1 can be used as the fixed three-dimensional data volume, and the target three-dimensional data volumes F2, F3, F4, F5 can be used as the three-dimensional data volumes to be registered. According to the fixed three-dimensional data volume F1 and the three-dimensional data volumes to be registered F2, F3, two spatial transformation matrices E1, E2 are obtained, and the spatial transformation matrices E1, E2 are calculated by weighted stitching to obtain the three-dimensional panoramic image data. According to the obtained three-dimensional panoramic image data, the current global transformation matrix G1 is calculated.

[0150] Step 3: For the three-dimensional data volume F4 to be registered, continue to generate a spatial transformation matrix E3 from the fixed three-dimensional data volume F1 and the three-dimensional data volume F4 to be registered, and solve for the global transformation matrix G1 by calculating the spatial transformation matrix E3 and the three-dimensional panoramic image data obtained by the weighted stitching method last time based on the SVD method, and determine whether the spatial transformation matrix is successfully registered. If the registration is successful, use the weighted stitching method with the successfully registered spatial transformation matrix E3 to obtain the three-dimensional panoramic image data this time, and calculate the global transformation matrix G2 this time; if the registration is unsuccessful, discard the three-dimensional data volume F4 corresponding to the unsuccessfully registered spatial transformation matrix.

[0151] Step 4: For the three-dimensional data volume F5 to be registered, continue to generate a spatial transformation matrix E4 from the fixed three-dimensional data volume F1 and the three-dimensional data volume F5 to be registered, and solve for the global transformation matrix G2 by calculating the spatial transformation matrix E4 and the three-dimensional panoramic image data obtained by the weighted stitching method last time based on the SVD method, and determine whether the spatial transformation matrix is successfully registered. If the registration is successful, use the weighted stitching method with the successfully registered spatial transformation matrix E4 to obtain the three-dimensional panoramic image data this time; if the registration is unsuccessful, discard the three-dimensional data volume F5 corresponding to the unsuccessfully registered spatial transformation matrix.

[0152] Step 5: Display the three-dimensional panoramic image based on the finally obtained three-dimensional panoramic image data.

[0153] The method provided by the embodiments of the present application is not limited to the above embodiments and can also be used in other possible embodiments, which are not limited in the embodiments of the present application.

[0154] Based on the above description, in a specific embodiment, first collect a plurality of initial three-dimensional data volumes on a tissue organ using a volume probe, then for each initial three-dimensional data volume, obtain a target three-dimensional data volume, then register the target three-dimensional data volume and obtain a plurality of spatial transformation matrices, and finally use the weighted stitching method to calculate the plurality of spatial transformation matrices. After obtaining the three-dimensional panoramic image data, display the three-dimensional panoramic image based on the three-dimensional panoramic image data, as Figure 11 shown, which is a schematic diagram of the three-dimensional panoramic image display provided by the embodiments of the present application.

[0155] In a feasible implementation, the GPU is used to render the obtained three-dimensional panoramic image data to obtain a three-dimensional panoramic image. In clinical operations, the obtained three-dimensional panoramic image can be rotated, translated, and scaled, that is, the surface morphology of the entire tissue or organ can be presented 360 degrees; it is also possible to customize the cutting of the three-dimensional panoramic image at any position and any three-dimensional angle to display its sectional image, that is, the internal morphology of the entire tissue or organ can be completely presented; it is also possible to customize an arbitrarily irregular three-dimensional ROI (Region of Interest) to crop the three-dimensional panoramic data volume, that is, the arbitrary local three-dimensional morphology of the entire tissue or organ can be presented; it is also possible to perform distance measurement, angle measurement, etc. on the three-dimensional panoramic data volume, that is, the expression of any size and angle of the entire tissue or organ can be presented. Thus, the accuracy and convenience of clinical diagnosis are improved.

[0156] Based on the same inventive concept, an ultrasonic three-dimensional panoramic imaging method as described above in the present application can also be implemented by an ultrasonic device. The effect of this ultrasonic device is similar to that of the foregoing method and will not be elaborated here.

[0157] The following will be described in detail with Figure 1 the ultrasonic device shown.

[0158] In a possible implementation manner, the processing unit 102 is specifically configured to:

[0159] For each Cartesian coordinate information, perform three axial interpolations on the Cartesian coordinate information based on a preset number of pixel points adjacent to the scanning line in the polar coordinate system to respectively obtain the pixel values of the preset number of pixel points;

[0160] For each pixel value among the pixel values of the preset number of pixel points, perform three transverse interpolations on the pixel value along the transverse axis in the polar coordinate system;

[0161] Use the data obtained after the three transverse interpolations as the target three-dimensional data volume.

[0162] In a possible implementation manner, the processing unit 102 is specifically configured to:

[0163] Perform Gaussian filtering processing and / or histogram equalization processing on the target three-dimensional data volume.

[0164] In a possible implementation manner, the processing unit 102 is specifically configured to:

[0165] For each target three-dimensional data volume, extract the feature points of the target three-dimensional data volume through the Oriented FAST algorithm;

[0166] Extract the feature point descriptors of the target 3D data volume based on the feature points of the target 3D data volume through the Rotated BRIEF algorithm.

[0167] In a possible implementation, the processing unit 102 is specifically configured to:

[0168] For each set of fixed 3D data volume and the 3D data volume to be registered, based on the BFMatcher algorithm, match the feature points of the fixed 3D data volume and the feature points of the 3D data volume to be registered according to the feature point descriptors of the fixed 3D data volume and the feature point descriptors of the 3D data volume to be registered, and obtain multiple sets of feature points;

[0169] Based on the RANSAC method, screen the multiple sets of feature points, and use the screened feature points as the spatial transformation matrix.

[0170] In a possible implementation, the processing unit 102 is specifically configured to:

[0171] For each spatial transformation matrix, solve the spatial transformation matrix and the global transformation matrix based on the SVD method to obtain the first rotation angle and the first translation distance corresponding to the spatial transformation matrix, and obtain the second rotation angle and the second translation distance corresponding to the global transformation matrix, where the global transformation matrix is calculated based on the 3D panoramic image data obtained by the weighted stitching method last time;

[0172] Judge whether the spatial transformation matrix is successfully registered according to the first rotation angle, the second rotation angle, the first translation distance, the second translation distance, the preset threshold, and the number of successful registration times;

[0173] If the registration is successful, use the weighted stitching method for the successfully registered spatial transformation matrix to obtain the 3D panoramic image data of this time; if the registration is unsuccessful, discard the 3D data volume to be registered corresponding to the unsuccessfully registered spatial transformation matrix.

[0174] In a possible implementation, the processing unit 102 is specifically configured to:

[0175] If the following preset conditions are met, determine that the spatial transformation matrix is successfully registered, otherwise, determine that the spatial transformation matrix is unsuccessfully registered, where the preset conditions include all of the following:

[0176] The first translation distance is less than or equal to the first preset threshold;

[0177] The first translation distance is less than or equal to the second translation distance divided by the number of successful registration times;

[0178] The first rotation angle is less than or equal to the second preset threshold;

[0179] The first rotation angle is less than or equal to the second rotation angle divided by the number of successful registrations.

[0180] Based on the same general concept, the above-described ultrasonic three-dimensional panoramic imaging method of the present application can also be implemented by an ultrasonic device. The effects of this ultrasonic device are similar to those of the foregoing method and device, and will not be elaborated herein.

[0181] As Figure 12 shown, a structural diagram of an ultrasonic device provided by an embodiment of the present application is as follows. The device includes:

[0182] An acquisition module 1301 that acquires a plurality of initial three-dimensional data volumes located in a polar coordinate system on a tissue organ through a volume probe;

[0183] A reconstruction module 1302 that, for each initial three-dimensional data volume, based on a scan conversion algorithm, converts each polar coordinate information into Cartesian coordinate information and then performs interpolation calculation on each Cartesian coordinate information to obtain a target three-dimensional data volume;

[0184] A registration module 1303 that, after extracting feature points and feature point descriptors of each target three-dimensional data volume based on the ORB algorithm, matches the fixed three-dimensional data volume with each to-be-registered three-dimensional data volume respectively based on the extracted feature points and feature point descriptors to obtain a plurality of spatial transformation matrices, where the fixed three-dimensional data volume is any one of the obtained target three-dimensional data volumes, and the to-be-registered three-dimensional data volume is other target three-dimensional data volumes except the fixed three-dimensional data volume;

[0185] A fusion module 1304 that calculates a plurality of spatial transformation matrices using a weighted stitching method to obtain three-dimensional panoramic image data;

[0186] A display module 1305 for displaying a three-dimensional panoramic image corresponding to the three-dimensional panoramic image data.

[0187] In a possible implementation manner, the reconstruction module 1302 is specifically configured to:

[0188] For each Cartesian coordinate information, perform three axial interpolations on the Cartesian coordinate information based on a preset number of pixel points adjacent to the scan line in the polar coordinate system to respectively obtain the pixel values of the preset number of pixel points;

[0189] For each pixel value among the pixel values of the preset number of pixel points, perform three transverse interpolations on the pixel value along the transverse axis in the polar coordinate system;

[0190] Use the data obtained after the three transverse interpolations as the target three-dimensional data volume.

[0191] In a possible implementation manner, it further includes:

[0192] The preprocessing module 1306 is specifically configured to:

[0193] Perform Gaussian filtering processing and / or histogram equalization processing on the target three-dimensional data volume.

[0194] In a possible implementation manner, the registration module 1303 is specifically configured to:

[0195] For each target three-dimensional data volume, extract feature points of the target three-dimensional data volume through the Oriented FAST algorithm;

[0196] Based on the feature points of the target three-dimensional data volume, extract feature point descriptors of the target three-dimensional data volume through the Rotated BRIEF algorithm.

[0197] In a possible implementation manner, the registration module 1303 is specifically configured to:

[0198] For each set of fixed three-dimensional data volume and the three-dimensional data volume to be registered, based on the BFMatcher algorithm, according to the feature point descriptors of the fixed three-dimensional data volume and the feature point descriptors of the three-dimensional data volume to be registered, match the feature points of the fixed three-dimensional data volume and the feature points of the three-dimensional data volume to be registered to obtain multiple sets of feature points;

[0199] Based on the RANSAC method, screen the multiple sets of feature points, and use the screened feature points as the spatial transformation matrix.

[0200] In a possible implementation manner, it further includes:

[0201] The decision module 1307 is specifically configured to:

[0202] For each spatial transformation matrix, solve the spatial transformation matrix and the global transformation matrix based on the SVD method to obtain the first rotation angle and the first translation distance corresponding to the spatial transformation matrix, and obtain the second rotation angle and the second translation distance corresponding to the global transformation matrix, where the global transformation matrix is calculated based on the three-dimensional panoramic image data obtained by the weighted stitching method last time;

[0203] According to the first rotation angle, the second rotation angle, the first translation distance, the second translation distance, the preset threshold, and the number of successful registration times, determine whether the spatial transformation matrix is successfully registered;

[0204] If the registration is successful, use the successfully registered spatial transformation matrix to obtain the three-dimensional panoramic image data of this time by the weighted stitching method; if the registration is unsuccessful, discard the three-dimensional data volume to be registered corresponding to the unsuccessfully registered spatial transformation matrix.

[0205] In a possible implementation manner, the decision module 1307 is specifically configured to:

[0206] If the following preset conditions are met, it is determined that the spatial transformation matrix registration is successful; otherwise, it is determined that the spatial transformation matrix registration is unsuccessful. The preset conditions include all of the following:

[0207] The first translation distance is less than or equal to the first preset threshold;

[0208] The first translation distance is less than or equal to the second translation distance divided by the number of successfully registered times;

[0209] The first rotation angle is less than or equal to the second preset threshold;

[0210] The first rotation angle is less than or equal to the second rotation angle divided by the number of successfully registered times.

[0211] An ultrasonic three-dimensional panoramic imaging method and an ultrasonic device provided by an embodiment of the present application first collect a plurality of initial three-dimensional data volumes located in a polar coordinate system on a tissue organ through a volume probe; then, for each initial three-dimensional data volume, based on a scan conversion algorithm, after converting the polar coordinate information corresponding to each initial three-dimensional data volume into Cartesian coordinate information, interpolation calculation is performed on each piece of Cartesian coordinate information to obtain a target three-dimensional data volume; after extracting feature points and feature point descriptors of each target three-dimensional data volume based on the ORB algorithm, based on the extracted feature points and feature point descriptors, the fixed three-dimensional data volume is respectively matched with each three-dimensional data volume to be registered to obtain a plurality of spatial transformation matrices; finally, a weighted stitching method is used to calculate the plurality of spatial transformation matrices, and after obtaining three-dimensional panoramic image data, a three-dimensional panoramic image is displayed based on the three-dimensional panoramic image data. In the embodiment of the present application, since the registration is performed between target three-dimensional data volumes based on the ORB algorithm, a three-dimensional panoramic image is finally obtained, so that the fusion effect of scanned images with a certain rotation angle is better, and the finally obtained three-dimensional panoramic image has better continuity, smoothness and stability.

[0212] The present application is described above with reference to the block diagrams and / or flowcharts showing methods, apparatuses (systems) and / or computer program products according to embodiments of the present application. It should be understood that one block of the block diagrams and / or flowcharts and combinations of blocks in the block diagrams and / or flowcharts can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, and / or other programmable data processing devices to generate a machine, so that the instructions executed via the computer processor and / or other programmable data processing devices create a method for implementing the functions / actions specified in the blocks of the block diagrams and / or flowcharts.

[0213] Accordingly, the present application may also be implemented by hardware and / or software, including firmware, resident software, microcode, etc. Further, the present application may take the form of a computer program product on a computer-usable or computer-readable storage medium having computer-usable or computer-readable program code embodied in the medium for use by or in connection with an instruction execution system. In the context of the present application, a computer-usable or computer-readable medium may be any medium that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

[0214] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims

1. An ultrasonic three-dimensional panoramic imaging method, characterized in that, it includes: Collecting a plurality of initial three-dimensional data volumes on a tissue organ in a polar coordinate system through a volume probe; For each initial three-dimensional data volume, based on a scan conversion algorithm, after converting the polar coordinate information corresponding to each initial three-dimensional data into Cartesian coordinate information, performing interpolation calculation on each Cartesian coordinate information to obtain a target three-dimensional data volume; After extracting the feature points and feature point descriptors of each target three-dimensional data volume based on the ORB algorithm, based on the extracted feature points and feature point descriptors, respectively matching the fixed three-dimensional data volume with each to-be-registered three-dimensional data volume to obtain a plurality of spatial transformation matrices, wherein, the fixed three-dimensional data volume is any one of the obtained target three-dimensional data volumes, and the to-be-registered three-dimensional data volume is other target three-dimensional data volumes except the fixed three-dimensional data volume; Calculating the plurality of spatial transformation matrices by using a weighted stitching method to obtain three-dimensional panoramic image data, and then displaying a three-dimensional panoramic image based on the three-dimensional panoramic image data.

2. The method according to claim 1, characterized in that, The performing interpolation calculation on each Cartesian coordinate information to obtain a target three-dimensional data volume includes: For each Cartesian coordinate information, performing cubic axial interpolation on the Cartesian coordinate information based on a preset number of pixel points adjacent to the scan line in the polar coordinate system to respectively obtain the pixel values of the preset number of pixel points; For each pixel value among the pixel values of the preset number of pixel points, performing cubic transverse interpolation on the pixel value along the transverse axis in the polar coordinate system; Taking the data obtained after the cubic transverse interpolation as the target three-dimensional data volume.

3. The method according to claim 1, characterized in that, After performing interpolation calculation on each Cartesian coordinate information to obtain a target three-dimensional data volume, it further includes: Performing Gaussian filtering processing and / or histogram equalization processing on the target three-dimensional data volume.

4. The method according to claim 1, characterized in that, The extracting the feature points and feature point descriptors of each target three-dimensional data volume based on the ORB algorithm includes: For each target three-dimensional data volume, extracting the feature points of the target three-dimensional data volume through the Oriented FAST algorithm; Extracting the feature point descriptors of the target three-dimensional data volume through the Rotated BRIEF algorithm based on the feature points of the target three-dimensional data volume.

5. The method according to claim 1, characterized in that, The respectively matching the fixed three-dimensional data volume with each to-be-registered three-dimensional data volume based on the extracted feature points and feature point descriptors to obtain a plurality of spatial transformation matrices includes: For each group of fixed three-dimensional data volume and to-be-registered three-dimensional data volume, based on the BFMatcher algorithm, according to the feature point descriptors of the fixed three-dimensional data volume and the feature point descriptors of the to-be-registered three-dimensional data volume, matching the feature points of the fixed three-dimensional data volume and the feature points of the to-be-registered three-dimensional data volume to obtain multiple groups of feature points; Based on the RANSAC method, the multiple sets of feature points are screened, and the screened feature points are used as the spatial transformation matrix.

6. The method according to claim 1, wherein, after matching the fixed three-dimensional data volume with each three-dimensional data volume to be registered respectively based on the extracted feature points and feature point descriptors to obtain multiple spatial transformation matrices, it further includes: For each spatial transformation matrix, based on the SVD method, solve the spatial transformation matrix and the global transformation matrix to obtain the first rotation angle and the first translation distance corresponding to the spatial transformation matrix, and obtain the second rotation angle and the second translation distance corresponding to the global transformation matrix, wherein the global transformation matrix is calculated according to the three-dimensional panoramic image data obtained by the weighted stitching method last time; Judge whether the spatial transformation matrix is successfully registered according to the first rotation angle, the second rotation angle, the first translation distance, the second translation distance, a preset threshold, and the number of successful registrations; If the registration is successful, use the weighted stitching method for the successfully registered spatial transformation matrix to obtain the three-dimensional panoramic image data of this time; if the registration is unsuccessful, discard the three-dimensional data volume to be registered corresponding to the unsuccessfully registered spatial transformation matrix.

7. The method according to claim 6, wherein, the judging whether the spatial transformation matrix is successfully registered according to the first rotation angle, the second rotation angle, the first translation distance, the second translation distance, a preset threshold, and the number of successful registrations includes: If the following preset conditions are met, it is determined that the spatial transformation matrix is successfully registered, otherwise, it is determined that the spatial transformation matrix is unsuccessfully registered, wherein the preset conditions include all of the following: The first translation distance is less than or equal to the first preset threshold; The first translation distance is less than or equal to the second translation distance divided by the number of successful registrations; The first rotation angle is less than or equal to the second preset threshold; The first rotation angle is less than or equal to the second rotation angle divided by the number of successful registrations.

8. An ultrasonic device, wherein, it includes a volume probe, a processing unit, and a display unit; The volume probe is used to collect multiple initial three-dimensional data volumes of a tissue organ in a polar coordinate system; The processing unit is used for each initial three-dimensional data volume, based on the scan conversion algorithm, convert the polar coordinate information corresponding to each initial three-dimensional data volume into Cartesian coordinate information, and then perform interpolation calculation on each Cartesian coordinate information to obtain a target three-dimensional data volume; after extracting the feature points and feature point descriptors corresponding to each target three-dimensional data volume based on the ORB algorithm, based on the extracted feature points and feature point descriptors, match the fixed three-dimensional data volume with each three-dimensional data volume to be configured respectively to obtain multiple spatial transformation matrices, wherein the fixed three-dimensional data volume is any one of the obtained target three-dimensional data volumes, and the three-dimensional data volume to be registered is other target three-dimensional data volumes except the fixed three-dimensional data volume; calculate the multiple spatial transformation matrices by using the weighted stitching method to obtain the three-dimensional panoramic image data; The display unit is configured to display a three-dimensional panoramic image corresponding to the three-dimensional panoramic image data.

9. The ultrasonic device according to claim 8, wherein, the processing unit is specifically configured to: for each Cartesian coordinate information, perform cubic axial interpolation on the Cartesian coordinate information based on a preset number of pixel points adjacent to each other on a scan line in a polar coordinate system to respectively obtain the pixel values of the preset number of pixel points; for each pixel value among the pixel values of the preset number of pixel points, perform cubic lateral interpolation on the pixel value along a transverse axis in the polar coordinate system; use the data obtained after the cubic lateral interpolation as the target three-dimensional data volume.

10. An ultrasonic device, wherein, the device includes: an acquisition module, configured to acquire a plurality of initial three-dimensional data volumes located on a tissue organ in a polar coordinate system through a volume probe; a reconstruction module, for each initial three-dimensional data volume, based on a scan conversion algorithm, convert the polar coordinate information corresponding to each initial three-dimensional data volume into Cartesian coordinate information, and then perform interpolation calculation on each Cartesian coordinate information to obtain a target three-dimensional data volume; a registration module, after extracting feature points and feature point descriptors of each target three-dimensional data volume based on the ORB algorithm, based on the extracted feature points and feature point descriptors, perform matching between a fixed three-dimensional data volume and each to-be-registered three-dimensional data volume respectively to obtain a plurality of spatial transformation matrices, wherein, the fixed three-dimensional data volume is any one of the obtained target three-dimensional data volumes, and the to-be-registered three-dimensional data volume is any other target three-dimensional data volume except the fixed three-dimensional data volume; a fusion module, configured to calculate the plurality of spatial transformation matrices by using a weighted stitching method to obtain three-dimensional panoramic image data; a display module, configured to display a three-dimensional panoramic image corresponding to the three-dimensional panoramic image data.

Images