A method and system for visualizing three-dimensional reconstruction of ultrasonic medical images

By matching and reconstructing the overlapping areas of the two-dimensional image in ultrasonic medical images, combining texture clarity and position weights, the problem of poor three-dimensional reconstruction effect in the prior art is solved, and clearer and more accurate lesion visualization is achieved.

CN118657894BActive Publication Date: 2024-11-19SHAANXI NUCLEAR IND 215 HOSPITAL
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411147406.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-21
Publication Date
2024-11-19
Estimated Expiration
2044-08-21

AI Technical Summary

Technical Problem

In the prior art, the three-dimensional reconstruction effect of ultrasound medical images is poor, mainly because the two-dimensional image is prone to fall into local optimality when matching and fusion, resulting in artifact phenomena and poor visualization effects.

Method used

By acquiring the lesion area and overlapping area of ​​two-dimensional ultrasound images at different acquisition angles, matching pixel points, and calculating the fusion reference weight based on the local texture clarity and position distribution relationship, the overlapping area is reconstructed to improve the visualization effect of the three-dimensional model.

Benefits of technology

It improves the three-dimensional reconstruction effect of ultrasound medical images, reduces artifacts, and enhances the visual clarity and accuracy of the lesions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118657894B_ABST
    Figure CN118657894B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of image analysis and processing technology, and specifically to a method and system for visual three-dimensional reconstruction of ultrasonic medical images. The present invention first obtains the lesion area of ​​a two-dimensional ultrasonic image, and the overlapping area between two-dimensional ultrasonic images at adjacent acquisition angles; further, all pixel points in the overlapping area of ​​each two-dimensional ultrasonic image are matched with pixel points in another two-dimensional ultrasonic image; according to the position distribution of the lesion area and the overlapping area, combined with the local texture clarity, the fusion reference weight of each pixel point in the overlapping area is obtained, and then the overlapping area is reconstructed to obtain a three-dimensional model. The present invention analyzes the lesion position and image texture clarity in adjacent two-dimensional ultrasonic images, respectively determines the reference value of each two-dimensional ultrasonic image for reconstructing the overlapping area, and then reconstructs the overlapping area, thereby improving the fusion matching effect, and then improving the visual three-dimensional reconstruction effect of the lesion in the ultrasonic medical image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image analysis and processing, and in particular to a method and system for visualizing three-dimensional reconstruction of ultrasonic medical images. Background Art

[0002] Three-dimensional ultrasound imaging technology can intuitively display the three-dimensional morphology and structural details of human organs, which helps to observe the details of lesions more comprehensively and plays an important role in clinical diagnosis and treatment, patient communication and medical education. Currently, three-dimensional images can be constructed using two-dimensional images from multiple angles, but due to the shooting angle problem, the matching two-dimensional images often overlap to a certain extent, resulting in artifacts in the constructed three-dimensional images.

[0003] In the existing technology, the two-dimensional images of adjacent angles are usually fused based on the three-dimensional space transformation parameters based on the ICP (Iterative Closest Point) registration algorithm to reduce the artifact error caused by the overlapping area. However, this method is prone to fall into local optimality, resulting in poor matching and fusion effects, and then resulting in poor visualization three-dimensional reconstruction of ultrasound medical images. Summary of the invention

[0004] In order to solve the technical problem that the existing technology has poor visual 3D reconstruction effect on ultrasonic medical images, the purpose of the present invention is to provide a method and system for visual 3D reconstruction of ultrasonic medical images. The technical solution adopted is as follows:

[0005] The present invention provides a method for visualizing three-dimensional reconstruction of ultrasonic medical images, the method comprising:

[0006] Acquire the lesion area of ​​the two-dimensional ultrasound images of the scanning part at different acquisition angles, and the overlapping area between the two-dimensional ultrasound images at adjacent acquisition angles;

[0007] Between the two-dimensional ultrasonic images at adjacent acquisition angles, all pixel points in the overlapping area of ​​each of the two-dimensional ultrasonic images are matched with pixel points in another two-dimensional ultrasonic image to obtain matching points of each pixel point in the overlapping area of ​​each of the two-dimensional ultrasonic images in another two-dimensional ultrasonic image;

[0008] In each of the two-dimensional ultrasound images, according to the position distribution relationship between the lesion area and the overlapping area, combined with the local texture clarity of each pixel in the overlapping area, a fusion reference weight of each pixel in the overlapping area is obtained;

[0009] According to the grayscale value of each pixel point in the overlapping area and the fusion reference weight, combined with the corresponding matching points, the overlapping area between the two-dimensional ultrasound images at adjacent acquisition angles is reconstructed; based on all adjacent two-dimensional ultrasound images and all corresponding reconstructed overlapping areas, a three-dimensional model of the scanned part is obtained.

[0010] Furthermore, the method for obtaining the matching points includes:

[0011] Between the two-dimensional ultrasonic images at adjacent acquisition angles, any pixel point in the overlapping area of ​​each of the two-dimensional ultrasonic images is used as a target pixel point, and each pixel point in another two-dimensional ultrasonic image is used as a reference pixel point;

[0012] Obtain a first matching effect according to a difference in position distribution between the target pixel point and each of the reference pixel points in the corresponding two-dimensional ultrasound image; obtain a second matching effect according to a difference in local texture between the target pixel point and each of the reference pixel points;

[0013] In combination with the first matching effect and the second matching effect, the matching confidence between the target pixel point and each of the reference pixel points is obtained; the first matching effect and the second matching effect are both positively correlated with the matching confidence; the reference pixel point with the largest matching confidence with the target pixel point is used as the matching point of the target pixel point.

[0014] Furthermore, the method for obtaining the first matching effect includes:

[0015] The Euclidean distance between the position coordinates of the target pixel point and each of the reference pixel points in the corresponding two-dimensional ultrasound image is obtained, and after negative correlation mapping of the Euclidean distance, the mapping result is used as the first matching effect between the target pixel point and each of the reference pixel points.

[0016] Furthermore, the method for obtaining the local texture difference includes:

[0017] Constructing corresponding preset neighborhoods with the target pixel and the reference pixel as centers respectively;

[0018] According to the grayscale difference of all pixels at the same position in the corresponding preset neighborhood of the target pixel and the reference pixel, the local texture difference between the target pixel and the reference pixel is obtained.

[0019] Furthermore, the method for obtaining the fusion reference weight includes:

[0020] In each of the two-dimensional ultrasound images, determine whether the lesion area is completely located in the overlapping area in the two-dimensional ultrasound image, and obtain a position reference weight of each pixel point in the overlapping area in combination with a distance between each pixel point in the overlapping area and a region boundary of the lesion area;

[0021] According to the position reference weight and the local texture clarity, a fusion reference weight of each pixel in the overlapping area is obtained; the position reference weight and the local texture clarity are both positively correlated with the fusion reference weight.

[0022] Furthermore, the method for obtaining the location reference weight includes:

[0023] When the lesion area is completely located in or completely not located in the overlapping area, the initial position weight of each pixel point in the overlapping area is set to 1; otherwise, the boundary line between the lesion area and the overlapping area is obtained, and the initial position weight of each pixel point in the overlapping area is obtained according to the area difference between the area of ​​the lesion area in the overlapping area and the total area of ​​the lesion area, and the spatial distance between each pixel point in the overlapping area and the boundary line, wherein both the area difference and the spatial distance are negatively correlated with the initial position weight;

[0024] In each of the two-dimensional ultrasound images, obtaining the minimum spatial distance between each pixel point in the overlapping area and the area boundary of the lesion area;

[0025] A position reference weight is acquired according to the initial position weight and the minimum spatial distance, wherein the initial position weight is positively correlated with the position reference weight, and the minimum spatial distance is negatively correlated with the position reference weight.

[0026] Furthermore, the method for acquiring the local texture clarity includes:

[0027] Taking the gradient mean of all pixels in each of the two-dimensional ultrasound images as a first texture clarity parameter;

[0028] Constructing a preset neighborhood with each pixel point as the center in the overlapping area of ​​each of the two-dimensional ultrasonic images, and acquiring a second texture clarity parameter according to a grayscale difference between the neighborhood pixel point and the corresponding center pixel point;

[0029] The local texture clarity of each pixel in the overlapping area is obtained according to the first texture clarity parameter and the second texture clarity parameter, and both the first texture clarity parameter and the second texture clarity parameter are positively correlated with the local texture clarity.

[0030] Furthermore, the overlapping area reconstruction method includes:

[0031] Obtaining, for each pixel point in the overlapping area in each of the two-dimensional ultrasonic images, a matching fusion reference weight of the matching point in another two-dimensional ultrasonic image at an adjacent acquisition angle, wherein the sum of the matching fusion reference weight of the matching point and the fusion reference weight of the corresponding pixel point is 1;

[0032] The grayscale values ​​of corresponding pixels are weighted and summed using the matching fusion reference weight and the fusion reference weight, and the weighted sum is used as the grayscale value of the pixel at the corresponding position in the overlapping area to obtain a reconstructed overlapping area.

[0033] Furthermore, the method for obtaining the lesion area includes:

[0034] The lesion area in each of the two-dimensional ultrasound images is extracted based on a convolutional neural network model.

[0035] The present invention also proposes a visualized three-dimensional reconstruction system for ultrasonic medical images, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of a visualized three-dimensional reconstruction method for ultrasonic medical images when executing the computer program.

[0036] The present invention has the following beneficial effects:

[0037] The present invention first obtains the lesion area of ​​the two-dimensional ultrasonic images of the scanning part at different acquisition angles, and the overlapping area between the two-dimensional ultrasonic images at adjacent acquisition angles; between the two-dimensional ultrasonic images at adjacent acquisition angles, all the pixel points in the overlapping area of ​​each two-dimensional ultrasonic image are matched with the pixel points in another two-dimensional ultrasonic image, and the matching points of each pixel point in the overlapping area of ​​each two-dimensional ultrasonic image in the other two-dimensional ultrasonic image are obtained in combination with the matching effect, so as to subsequently analyze the reference value of each pair of pixel points for reconstructing the overlapping area; in each two-dimensional ultrasonic image, according to the position distribution relationship between the lesion area and the overlapping area, combined with the local texture clarity of each pixel point in the overlapping area, the fusion reference weight of each pixel point in the overlapping area is obtained, and the fusion reference weight not only pays attention to the image clarity, but also pays attention to the importance of the lesion area, thereby determining the reference value for reconstructing the overlapping area; according to the gray value and the fusion reference weight of each pixel point, combined with the corresponding matching point, the overlapping area between the two-dimensional ultrasonic images at adjacent acquisition angles is reconstructed; according to all adjacent two-dimensional ultrasonic images and all corresponding reconstructed overlapping areas, a three-dimensional model of the scanning part is obtained. The present invention analyzes the lesion location and image texture clarity in adjacent two-dimensional ultrasonic images, determines the reference value of each two-dimensional ultrasonic image for reconstructing the overlapping area, and then reconstructs the overlapping area, thereby improving the fusion matching effect, and further improving the visual three-dimensional reconstruction effect of the lesion in the ultrasonic medical image. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 A flowchart of a method for visual three-dimensional reconstruction of ultrasonic medical images provided by an embodiment of the present invention;

[0040] Figure 2 A flowchart of a method for obtaining matching points provided by an embodiment of the present invention;

[0041] Figure 3 A flowchart of a method for obtaining a fusion reference weight is provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the method and system for visual three-dimensional reconstruction of ultrasonic medical images proposed by the present invention, its specific implementation, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0043] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0044] The specific scheme of the method and system for visual three-dimensional reconstruction of ultrasonic medical images provided by the present invention is described in detail below with reference to the accompanying drawings.

[0045] See also Figure 1 , which shows a flowchart of a method for visual three-dimensional reconstruction of ultrasonic medical images provided by an embodiment of the present invention, specifically comprising:

[0046] Step S1, acquiring the lesion area of ​​the two-dimensional ultrasound images of the scanning part at different acquisition angles, and the overlapping area between the two-dimensional ultrasound images at adjacent acquisition angles.

[0047] In one embodiment of the present invention, the breast is taken as an example of the scanning part to illustrate; the breast is scanned using an ultrasonic probe to obtain two-dimensional ultrasonic images at different acquisition angles; specifically, the breast area is scanned at radial acquisition angles with the nipple as the center, and 12 two-dimensional ultrasonic images at different acquisition angles are evenly obtained in the entire breast area, wherein these 12 acquisition angles can completely cover the entire breast area; further, the lesion area in the two-dimensional ultrasonic image at each acquisition angle is obtained, wherein the lesion area refers to a nodule, a mass or a tumor, etc.; all the two-dimensional ultrasonic images are sorted in the order of the acquisition angles, and the overlapping part between the two-dimensional ultrasonic images at two adjacent acquisition angles is used as the overlapping area, so that the overlapping area can be subsequently reconstructed for three-dimensional reconstruction.

[0048] Preferably, in one embodiment of the present invention, the method for obtaining the lesion area includes: extracting the lesion area in each two-dimensional ultrasound image based on a convolutional neural network model. The implementer can annotate the lesion area of ​​a large number of two-dimensional ultrasound images, construct a training sample set, and then train the convolutional neural network, and then input the two-dimensional ultrasound image into the trained convolutional neural network to automatically identify the lesion area; the training and application of the convolutional neural network are already well known to those skilled in the art, and will not be described in detail here.

[0049] In other embodiments of the present invention, the implementer may also perform feature analysis on the two-dimensional ultrasound image based on the texture features between the lesion area and other normal breast tissue areas, and use a threshold segmentation algorithm to obtain the lesion area; or use an edge detection algorithm to obtain the boundary of the lesion area, and then obtain the lesion area, which are all well-known technologies and will not be elaborated here.

[0050] It should be noted that since the two-dimensional ultrasound image is nearly fan-shaped, the implementer can further perform pre-processing such as scaling, cropping, and noise reduction on the two-dimensional ultrasound image to obtain a regular, high-quality two-dimensional ultrasound image that is easy to process. These are all well-known technologies and will not be described in detail here.

[0051] It should be noted that the implementer can also scan other parts such as the kidneys and uterus to reconstruct a three-dimensional model; the implementer can also adjust the acquisition method of the two-dimensional ultrasound image by himself, which is related to the imaging range of the probe. For example, only 4 or 8 scans are needed to completely cover the entire breast area; at the same time, it should be noted that there can be overlap between two-dimensional ultrasound images under adjacent acquisition angles, but overlap between two-dimensional ultrasound images under non-adjacent acquisition angles should be avoided as much as possible. For example, there is overlap between the first and second images, but the first image should not overlap with the third image, so as to reduce the difficulty of reconstructing the overlapping area.

[0052] Step S2, between two-dimensional ultrasonic images at adjacent acquisition angles, all pixel points in the overlapping area of ​​each two-dimensional ultrasonic image are matched with pixel points in another two-dimensional ultrasonic image, and matching points of each pixel point in the overlapping area of ​​each two-dimensional ultrasonic image in the other two-dimensional ultrasonic image are obtained.

[0053] Considering that the overlapping area between two-dimensional ultrasound images at adjacent acquisition angles may be the overlap of different local parts of the scanned part, it will affect the local analysis of the overlapping area, and further affect the subsequent three-dimensional model reconstruction. Therefore, the embodiment of the present invention first matches the pixel points in the overlapping area, so as to facilitate the subsequent analysis and reconstruction of the overlapping area between two-dimensional ultrasound images at adjacent acquisition angles.

[0054] Preferably, in one embodiment of the present invention, the method for obtaining matching points includes:

[0055] See also Figure 2 , which shows a flow chart of a method for obtaining matching points provided by an embodiment of the present invention, including:

[0056] Step S201 , between two-dimensional ultrasound images at adjacent acquisition angles, any pixel point in an overlapping area of ​​each two-dimensional ultrasound image is used as a target pixel point, and each pixel point in another two-dimensional ultrasound image is used as a reference pixel point.

[0057] Since the overlapping area is formed by the overlap of two-dimensional ultrasonic images at two adjacent acquisition angles, there is an overlapping area in each two-dimensional ultrasonic image between each acquisition angle and the two-dimensional ultrasonic image corresponding to the next adjacent acquisition angle, and the overlapping area is the same as the overlapping area of ​​the two-dimensional ultrasonic image corresponding to the next adjacent acquisition angle.

[0058] It should be noted that the embodiments of the present invention are specifically described from the perspective of matching each pixel point in the overlapping area of ​​each two-dimensional ultrasonic image with the pixel point in the two-dimensional ultrasonic image of the next adjacent acquisition angle, that is, each pixel point in the two-dimensional ultrasonic image of the next adjacent acquisition angle is always matched as a reference pixel point.

[0059] In order to facilitate the subsequent analysis and description of the matching process of each pixel point in the overlapping area, an embodiment of the present invention first determines the target pixel point and all reference pixel points subsequently matched therewith, so as to facilitate subsequent pairing.

[0060] Step S202, obtaining a first matching effect according to the position distribution difference between the target pixel point and each reference pixel point in the corresponding two-dimensional ultrasound image; obtaining a second matching effect according to the local texture difference between the target pixel point and each reference pixel point.

[0061] Taking into account that there is a certain offset in the acquisition angle between each acquisition angle and the corresponding two-dimensional ultrasound image of the next adjacent acquisition angle, the characteristic expression and position distribution of each local part of the scanned part should be similar or consistent in the corresponding two-dimensional ultrasound image; therefore, the matching effect between the target pixel point and each reference pixel point can be evaluated based on this, and then the matching is completed; the first matching effect evaluates the matching effect from the perspective of position distribution, and the second matching effect evaluates the matching effect from the perspective of local texture features. The more similar the position distribution is and the smaller the local texture difference is, the better the matching effect is.

[0062] In a preferred embodiment of the present invention, considering that the position coordinates can more intuitively reflect the position distribution, the method for obtaining the first matching effect includes:

[0063] The Euclidean distance between the position coordinates of the target pixel and each reference pixel in the corresponding two-dimensional ultrasound image is obtained, and after negative correlation mapping of the Euclidean distance, the mapping result is used as the first matching effect parameter between the target pixel and each reference pixel.

[0064] Since there is a certain viewing angle deviation in the two-dimensional ultrasound images at adjacent acquisition angles, in order to analyze the position of the target pixel point and each reference pixel point in the corresponding two-dimensional ultrasound image, it is necessary to perform coordinate transformation on the two-dimensional ultrasound images at adjacent acquisition angles. As an example, SIFT feature matching is performed between the two-dimensional ultrasound images corresponding to each acquisition angle and the next adjacent acquisition angle, and a two-dimensional coordinate system is constructed with any feature point in the first two-dimensional ultrasound image as the origin. Another two-dimensional coordinate system is constructed with the corresponding matching feature point in the two-dimensional ultrasound image corresponding to the next adjacent acquisition angle as the coordinate origin to obtain the position coordinates of each pixel point in the two two-dimensional ultrasound images, where the main viewing direction of all two-dimensional ultrasound images is consistent; the Euclidean distance between the position coordinates corresponding to the target pixel point and each reference pixel point is taken as the natural constant. Exponential function with base In , to perform negative correlation mapping and normalization to obtain the first matching effect parameter. In other examples, other negative correlation mapping means such as reciprocal operation may also be used, which will not be described in detail here.

[0065] In a preferred embodiment of the present invention, considering that the local grayscale difference of a pixel point can reflect its local texture feature, the method for obtaining the local texture difference includes:

[0066] The corresponding preset neighborhoods are constructed with the target pixel and the reference pixel as the center respectively; the local texture difference between the target pixel and the reference pixel is obtained according to the grayscale difference of all pixels at the same position in the corresponding preset neighborhood.

[0067] As an example, take the target pixel or reference pixel as the center and construct a The preset neighborhood is used to analyze the absolute value of the grayscale difference between pixels at the same position; the absolute values ​​of the grayscale differences of all pixels at the same position are multiplied, and the multiplication result is the local texture difference; the larger the absolute value of the grayscale difference, the smaller the similarity of the local texture features, and the worse the corresponding matching effect. Therefore, the logic is further adjusted by negative correlation mapping, and the inverse operation can be performed. Before the inverse operation, the anti-zero parameter 1 needs to be added to the multiplication result to avoid meaninglessness, thereby obtaining the second matching effect.

[0068] In other examples, preset neighborhoods of other sizes can also be constructed, and the absolute values ​​of the grayscale differences of all pixels at the same position can be further accumulated, and the negative correlation can be further mapped to the exponential function. I will not go into details here.

[0069] In another embodiment of the present invention, when obtaining the local texture difference, the gradient difference may be used instead of the grayscale difference, or the grayscale difference and the gradient difference may be combined to evaluate the local texture difference, which will not be described in detail here.

[0070] Step S203, combining the first matching effect and the second matching effect, obtaining the matching confidence between the target pixel and each reference pixel; the first matching effect and the second matching effect are both positively correlated with the matching confidence; the reference pixel with the largest matching confidence with the target pixel is used as the matching point of the target pixel.

[0071] As an example, the first matching effect and the second matching effect are added and combined to obtain the matching confidence, and then the reference pixel point corresponding to the maximum matching confidence is selected as the matching point of the target pixel point; in other examples, the implementer can also combine the two through basic mathematical operations such as multiplication or weighted summation or related mapping methods, which will not be repeated here.

[0072] In another embodiment of the present invention, the implementer may also use an image registration method to match each two-dimensional ultrasonic image with the two-dimensional ultrasonic image corresponding to the next adjacent acquisition angle, thereby obtaining a matching point for each pixel point in the overlapping area of ​​each two-dimensional ultrasonic image in the two-dimensional ultrasonic image corresponding to the next adjacent acquisition angle; this is a well-known technology and will not be described in detail here.

[0073] In another embodiment of the present invention, the implementer may also use the Scale-Invariant Feature Transform (SIFT) algorithm to first obtain feature points in each two-dimensional ultrasound image respectively, and then perform feature matching on each two-dimensional ultrasound image with the two-dimensional ultrasound image corresponding to the next adjacent acquisition angle based on the feature points, and further obtain the matching effect between each pair of feature points to obtain the matching confidence, thereby screening out the confidence matching points; specifically, the matching feature points with matching confidence greater than a preset threshold of 0.7 are used as confidence matching points; then, each confidence matching point in the overlapping area is used as the cluster center, and the metric distance is constructed by the distance of the pixel point relative to the cluster center and the sum of the local texture difference, and K-means clustering is performed to obtain each confidence matching point cluster, and further each confidence matching point in the overlapping area is matched with the pixel points in the two clusters corresponding to its feature matching point, thereby completing the matching of the pixel points between each two-dimensional ultrasound image and the next adjacent two-dimensional ultrasound image.

[0074] It should be noted that the SIFT algorithm and the K-means clustering algorithm are already well-known technologies and will not be described in detail here. In other embodiments of the present invention, other feature matching algorithms and clustering algorithms may also be used, or other preset thresholds may be set to obtain confidence matching points. The matching process for each pair of confidence matching points corresponding to the pixel points in the two clusters is consistent with the matching method described in steps S201-S203 and will not be described in detail here.

[0075] Step S3, in each two-dimensional ultrasound image, according to the position distribution relationship between the lesion area and the overlapping area, combined with the local texture clarity of each pixel in the overlapping area, obtain the fusion reference weight of each pixel in the overlapping area.

[0076] Taking into account that the overlapping area is the object of analysis and reconstruction, it is necessary to analyze the reference value of the pixel points at each position in the overlapping area of ​​each two-dimensional ultrasound image during reconstruction; and taking into account that the lesion area is the key observation object for constructing a three-dimensional model to assist in analysis and diagnosis, in order to ensure the performance effect of the lesion area after three-dimensional reconstruction, when the lesion area is in the overlapping area, more emphasis should be placed on analyzing the clarity and boundary information of the lesion area; therefore, the embodiment of the present invention obtains the fusion reference weight of each pixel point in the overlapping area based on the positional distribution relationship between the lesion area and the overlapping area, combined with the local texture clarity of each pixel point in the overlapping area, and the fusion reference weight reflects the reference value of the corresponding pixel point in the two-dimensional ultrasound image to the reconstruction of the overlapping area.

[0077] Preferably, in one embodiment of the present invention, the method for obtaining the fusion reference weight includes:

[0078] See also Figure 3, which shows a flow chart of a method for obtaining a fusion reference weight provided by an embodiment of the present invention, including:

[0079] Step S301, in each two-dimensional ultrasound image, determine whether the lesion area is completely located in the overlapping area of ​​the two-dimensional ultrasound image, and obtain the position reference weight of each pixel point in the overlapping area based on the distance between each pixel point in the overlapping area and the area boundary of the lesion area.

[0080] Considering that when the lesion area is completely located in or not completely located in the overlapping area, it is only necessary to pay attention to the texture clarity of the overlapping area and whether the boundary of the lesion area is clear, so as to ensure good image quality and clear three-dimensional lesion structure; but considering that if the lesion area is not completely located in the overlapping area, that is, part of it is located in the overlapping area and part of it is located in the non-overlapping area, it is necessary to further pay attention to the transition effect between the overlapping area and the non-overlapping area, so as to facilitate image smoothing between the overlapping areas.

[0081] Therefore, in a preferred embodiment of the present invention, the method for obtaining the location reference weight includes:

[0082] When the lesion area is completely located in or completely not located in the overlapping area, the initial position weight of each pixel in the overlapping area is set to 1; otherwise, the boundary line between the lesion area and the overlapping area is obtained, and the initial position weight of each pixel in the overlapping area is obtained according to the area difference between the area of ​​the lesion area in the overlapping area and the total area of ​​the lesion area, and the spatial distance between each pixel in the overlapping area and the boundary line. The area difference and the spatial distance are negatively correlated with the initial position weight;

[0083] In each two-dimensional ultrasound image, the minimum spatial distance between each pixel point in the overlapping area and the boundary of the lesion area is obtained;

[0084] The position reference weight is obtained according to the initial position weight and the minimum spatial distance. The initial position weight is positively correlated with the position reference weight, and the minimum spatial distance is negatively correlated with the position reference weight.

[0085] The position reference weight combines the location distribution and boundary information of the lesion area to evaluate the reference value of each pixel during reconstruction. It and the local texture clarity of the pixel point provide important reference value for the subsequent reconstruction of the overlapping area.

[0086] As an example, the initial position weight is first obtained. When the lesion area is partially located in the overlapping area, the initial position weight of each pixel in the overlapping area is obtained as follows:

[0087] Get the boundary line between the overlapping area and the non-overlapping area of ​​the lesion area, that is, the boundary line formed by the pixels located in the overlapping area but adjacent to the non-overlapping area; get the area ratio of the lesion area in the overlapping area to the total area of ​​the lesion area in the two-dimensional ultrasound image. The larger the area ratio, the smaller the area difference; get the minimum Euclidean distance between each pixel point in the overlapping area and all the pixels on the boundary line. The minimum Euclidean distance reflects the degree to which each pixel point is close to the boundary line. The closer to the boundary line, the more important the pixel point information is and the greater the reference value is, so as to facilitate subsequent smoothing; after negative correlation mapping of the minimum Euclidean distance, use multiplication to combine it with the area ratio to obtain the initial position weight. Negative correlation mapping can be regarded as mapping to an exponential function. In the example above, you can also add the anti-zero parameter and do the reciprocal operation, which will not be described in detail here.

[0088] In this example, after obtaining the initial position weight, the minimum spatial distance between each pixel in the overlapping area and the boundary of the lesion area is further obtained. The minimum spatial distance reflects the proximity of each pixel to the boundary of the lesion area. The closer the pixel is to the boundary, the more important it is and the greater its reference value is, so as to facilitate the subsequent construction of a clear three-dimensional lesion structure. The initial position weight is further multiplied and combined with the minimum spatial distance to obtain the position reference weight. In other examples, the two can also be combined through basic mathematical operations such as multiplication or weighted summation or related mapping methods, which will not be repeated here.

[0089] Step S302, obtaining a fusion reference weight for each pixel in the overlapping area according to the position reference weight and the local texture clarity; the position reference weight and the local texture clarity are both positively correlated with the fusion reference weight.

[0090] As an example, the position reference weight and the local texture clarity are multiplied and combined, and then normalized to the maximum and minimum values ​​to obtain the fusion reference weight; in other examples, the two can also be combined through basic mathematical operations such as multiplication or weighted summation or related mapping methods, and other normalization methods can also be used, which will not be repeated here.

[0091] In a preferred embodiment of the present invention, considering that the texture in the two-dimensional ultrasound image of the breast area is relatively complex, the gradient of the pixel point has a high-frequency change, and the texture clarity of the local image is relatively high; and considering that the greater the grayscale difference between the pixel point and its surrounding adjacent pixels, the higher the local texture clarity of the image; therefore, the method for obtaining the local texture clarity includes:

[0092] The gradient mean of all pixels in each two-dimensional ultrasound image is used as the first texture clarity parameter;

[0093] In a preset neighborhood centered on each pixel point in the overlapping area of ​​each two-dimensional ultrasonic image, a second texture clarity parameter is obtained according to a grayscale difference between the neighborhood pixel point and the center corresponding pixel point;

[0094] According to the first texture clarity parameter and the second texture clarity parameter, the local texture clarity of each pixel in the overlapping area is obtained, and the first texture clarity parameter and the second texture clarity parameter are both positively correlated with the local texture clarity.

[0095] As an example, the Sobel operator is specifically used to obtain the gradient value of each pixel point, and the gradient values ​​of all pixel points in the two-dimensional ultrasound image are averaged to obtain a first texture clarity parameter; the larger the gradient mean value, the clearer the texture; at the same time, a preset neighborhood is constructed with each pixel point as the center, and the construction method of the preset neighborhood is consistent with the preset neighborhood method and size constructed when analyzing the local texture difference in step S202, and will not be repeated here; in the preset neighborhood, the absolute value of the grayscale difference between each neighborhood pixel point and the central pixel point is accumulated to obtain a second texture clarity parameter; the first texture clarity parameter is further multiplied and combined with the second texture clarity parameter to obtain local texture clarity.

[0096] In other examples, the implementer may also analyze the entropy of the grayscale values ​​between each pixel and its neighboring pixels to evaluate the local texture clarity. The larger the entropy, the greater the local texture clarity. The implementer may also further obtain the contrast in the image. The higher the contrast, the greater the local texture clarity. These are all existing technologies well known to those skilled in the art and will not be elaborated here.

[0097] Step S4, reconstructing the overlapping area between the two-dimensional ultrasound images at adjacent acquisition angles according to the grayscale value and fusion reference weight of each pixel in the overlapping area and combining the corresponding matching points; obtaining the three-dimensional model of the scanned part according to all adjacent two-dimensional ultrasound images and all corresponding reconstructed overlapping areas.

[0098] In steps S2-S3, the fusion reference weight of each pixel point in the overlapping area of ​​each two-dimensional ultrasonic image and the two-dimensional ultrasonic image of the next adjacent acquisition angle is obtained respectively. The larger the fusion reference weight, the greater the reference value of the pixel point in the two-dimensional ultrasonic image for reconstructing the overlapping area. However, it is still necessary to combine the reference information of the matching point in the two-dimensional ultrasonic image of the next adjacent acquisition angle to fuse and reconstruct the overlapping area.

[0099] Therefore, in a preferred embodiment of the present invention, the method for reconstructing the overlapping area includes:

[0100] Obtaining, for each pixel point in the overlapping area of ​​each two-dimensional ultrasound image, a matching fusion reference weight of a matching point in another two-dimensional ultrasound image at an adjacent acquisition angle, wherein the sum of the matching fusion reference weight of the matching point and the fusion reference weight of the corresponding pixel point is 1;

[0101] The grayscale values ​​of corresponding pixels are weighted and summed using the matching fusion reference weight and the fusion reference weight, and the weighted sum is used as the grayscale value of the pixel at the corresponding position in the overlapping area to obtain the reconstructed overlapping area.

[0102] In an embodiment of the present invention, the grayscale value of the pixel point at each position in the overlapping area of ​​each two-dimensional ultrasound image is weightedly summed with the grayscale value of the corresponding matching point in the next adjacent two-dimensional ultrasound image using corresponding weights to obtain a reconstructed grayscale value of the pixel point at each position in the overlapping area. The reconstructed grayscale value integrates the reference information of each pair of matching points, thereby obtaining a reconstructed overlapping area.

[0103] After obtaining the reconstructed overlapping area under every two adjacent acquisition angles, all the two-dimensional images sorted according to the acquisition angle, i.e., the scanning orientation, can be three-dimensionally mapped. Specifically, the Ray Casting algorithm is used for volume reconstruction. After obtaining the initial three-dimensional volume data, the ISO surface extraction method can be further used to generate a three-dimensional model with clear boundaries. At this point, the three-dimensional ultrasound model of the scanned area can be obtained, which facilitates the understanding and observation of the lesion structure of the scanned area.

[0104] It should be noted that constructing a three-dimensional model using multi-angle two-dimensional plane images is a well-known technology. Implementers can also use other algorithms or methods besides the Ray Casting algorithm and the ISO surface extraction method, which will not be described here.

[0105] In summary, the embodiment of the present invention first obtains the lesion area of ​​the two-dimensional ultrasound image at different acquisition angles, and the overlapping area between the two-dimensional ultrasound images at adjacent acquisition angles; further matches all the pixel points in the overlapping area of ​​each two-dimensional ultrasound image with the pixel points in another two-dimensional ultrasound image; further, according to the position distribution relationship between the lesion area and the overlapping area, combined with the local texture clarity, obtains the fusion reference weight of each pixel point in the overlapping area, and then reconstructs the overlapping area between the two-dimensional ultrasound images at adjacent acquisition angles to obtain a three-dimensional model of the scanned part. The present invention analyzes the lesion position and image texture clarity in adjacent two-dimensional ultrasound images, respectively determines the reference value of each two-dimensional ultrasound image for reconstructing the overlapping area, and then reconstructs the overlapping area, thereby improving the fusion matching effect, and then improving the visualization three-dimensional reconstruction effect of the lesion in the ultrasound medical image.

[0106] The present invention also proposes a visualized three-dimensional reconstruction system for ultrasonic medical images. The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of a visualized three-dimensional reconstruction method for ultrasonic medical images are implemented.

[0107] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0108] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A method for visualizing three-dimensional reconstruction of ultrasonic medical images, characterized in that: The method comprises: Acquire the lesion area of ​​the two-dimensional ultrasound images of the scanning part at different acquisition angles, and the overlapping area between the two-dimensional ultrasound images at adjacent acquisition angles; Between the two-dimensional ultrasonic images at adjacent acquisition angles, all pixel points in the overlapping area of ​​each of the two-dimensional ultrasonic images are matched with pixel points in another two-dimensional ultrasonic image to obtain matching points of each pixel point in the overlapping area of ​​each of the two-dimensional ultrasonic images in another two-dimensional ultrasonic image; In each of the two-dimensional ultrasound images, according to the position distribution relationship between the lesion area and the overlapping area, combined with the local texture clarity of each pixel in the overlapping area, a fusion reference weight of each pixel in the overlapping area is obtained; the fusion reference weight reflects the reference value of the corresponding pixel in the two-dimensional ultrasound image for reconstructing the overlapping area; Reconstruct the overlapping area between the two-dimensional ultrasound images at adjacent acquisition angles according to the gray value of each pixel point in the overlapping area and the fusion reference weight, combined with the corresponding matching point; obtain the three-dimensional model of the scanned part according to all adjacent two-dimensional ultrasound images and all corresponding reconstructed overlapping areas; The method for obtaining the fusion reference weight includes: In each of the two-dimensional ultrasound images, determine whether the lesion area is completely located in the overlapping area in the two-dimensional ultrasound image, and obtain a position reference weight of each pixel point in the overlapping area in combination with a distance between each pixel point in the overlapping area and a region boundary of the lesion area; According to the position reference weight and the local texture clarity, obtaining a fusion reference weight for each pixel in the overlapping area; the position reference weight and the local texture clarity are both positively correlated with the fusion reference weight; The method for obtaining the location reference weight includes: When the lesion area is completely located in or completely not located in the overlapping area, the initial position weight of each pixel point in the overlapping area is set to 1; otherwise, the boundary line between the lesion area and the overlapping area is obtained, and the initial position weight of each pixel point in the overlapping area is obtained according to the area difference between the area of ​​the lesion area in the overlapping area and the total area of ​​the lesion area, and the spatial distance between each pixel point in the overlapping area and the boundary line, wherein both the area difference and the spatial distance are negatively correlated with the initial position weight; In each of the two-dimensional ultrasound images, obtaining the minimum spatial distance between each pixel point in the overlapping area and the area boundary of the lesion area; Acquire a position reference weight according to the initial position weight and the minimum spatial distance, wherein the initial position weight is positively correlated with the position reference weight, and the minimum spatial distance is negatively correlated with the position reference weight; The method for obtaining the local texture clarity includes: Taking the gradient mean of all pixels in each of the two-dimensional ultrasound images as a first texture clarity parameter; Constructing a preset neighborhood with each pixel point as the center in the overlapping area of ​​each of the two-dimensional ultrasonic images, and acquiring a second texture clarity parameter according to a grayscale difference between the neighborhood pixel point and the corresponding center pixel point; According to the first texture clarity parameter and the second texture clarity parameter, obtaining the local texture clarity of each pixel in the overlapping area, wherein the first texture clarity parameter and the second texture clarity parameter are both positively correlated with the local texture clarity; The overlapping area reconstruction method comprises: Obtaining, for each pixel point in the overlapping area in each of the two-dimensional ultrasonic images, a matching fusion reference weight of the matching point in another two-dimensional ultrasonic image at an adjacent acquisition angle, wherein the sum of the matching fusion reference weight of the matching point and the fusion reference weight of the corresponding pixel point is 1; The grayscale values ​​of corresponding pixels are weighted and summed using the matching fusion reference weight and the fusion reference weight, and the weighted sum is used as the grayscale value of the pixel at the corresponding position in the overlapping area to obtain a reconstructed overlapping area.

2. A method for visualizing three-dimensional reconstruction of ultrasonic medical images according to claim 1, characterized in that: The method for obtaining the matching points includes: Between the two-dimensional ultrasonic images at adjacent acquisition angles, any pixel point in the overlapping area of ​​each of the two-dimensional ultrasonic images is used as a target pixel point, and each pixel point in another two-dimensional ultrasonic image is used as a reference pixel point; Obtain a first matching effect according to a difference in position distribution between the target pixel point and each of the reference pixel points in the corresponding two-dimensional ultrasound image; obtain a second matching effect according to a difference in local texture between the target pixel point and each of the reference pixel points; In combination with the first matching effect and the second matching effect, a matching confidence between the target pixel and each of the reference pixels is obtained; the first matching effect and the second matching effect are both positively correlated with the matching confidence; the reference pixel with the largest matching confidence with the target pixel is used as the matching point of the target pixel; The method for obtaining the local texture difference includes: Constructing corresponding preset neighborhoods with the target pixel and the reference pixel as centers respectively; According to the grayscale difference of all pixels at the same position in the corresponding preset neighborhood of the target pixel and the reference pixel, the local texture difference between the target pixel and the reference pixel is obtained.

3. A method for visualizing three-dimensional reconstruction of ultrasonic medical images according to claim 2, characterized in that: The method for obtaining the first matching effect includes: The Euclidean distance between the position coordinates of the target pixel point and each of the reference pixel points in the corresponding two-dimensional ultrasound image is obtained, and after negative correlation mapping of the Euclidean distance, the mapping result is used as the first matching effect between the target pixel point and each of the reference pixel points.

4. The method for visualizing three-dimensional reconstruction of ultrasonic medical images according to claim 1, characterized in that: The method for obtaining the lesion area comprises: The lesion area in each of the two-dimensional ultrasound images is extracted based on a convolutional neural network model.

5. A visualization three-dimensional reconstruction system for ultrasonic medical images, characterized in that: The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of a method for visual three-dimensional reconstruction of ultrasonic medical images as described in any one of claims 1 to 4 are implemented.