High-definition vascular structure reconstruction and blood flow function evaluation method and system

Abstract

The invention provides a high-definition vascular structure reconstruction and blood flow function evaluation method and system, and the method comprises the steps: constructing a high-dimensional feature manifold space based on an ultrasound contrast image, carrying out the recognition to obtain a vascular subgroup with different blood flow function modes, and finally obtaining a high-definition rainbow vascular structure video and a vascular heterogeneity index in an ultrasound scanning region. A powerful tool can be provided for evaluating tumor microvascular structures, blood flow functions and time-space heterogeneous characteristics, abundant parameters for quantitatively describing tumor complex vascular systems are provided for early evaluation of drug curative effects, and a target blood flow perfusion area can be specifically recognized according to functional characteristics; and a target prompt and feedback intervention basis is provided for drug targeted release.

Description

Technical Field

[0001] The present application relates to the field of ultrasonic imaging technology, and in particular to a high-definition vascular structure reconstruction and blood flow function evaluation method, a high-definition vascular structure reconstruction and blood flow function evaluation device, an electronic device, and a computer-readable storage medium. Background Art

[0002] Angiogenesis is one of the key biomarkers of tumors. Since tumors rely on the generation of microvascular systems to grow and metastasize, dynamic and clear observation of microvessels is of great significance for clinical diagnosis and treatment evaluation. More and more clinical studies have shown that almost all malignant tumors show significant temporal and spatial heterogeneity in imaging, and greater heterogeneity often reflects the treatment resistance produced by drug-resistant cell subpopulations, suggesting negative drug treatment effects. In fact, changes in microenvironmental conditions caused by abnormal microblood flow changes, such as hypoxia and acidosis, can stimulate the production of cell subpopulations with greater metastatic potential and drug resistance in local tumor areas, which can greatly weaken the effectiveness of traditional global cancer treatment strategies.

[0003] In recent years, targeted precision therapy strategies that can be controlled to release under endogenous environmental characteristics (hypoxia and / or enzyme activity) or external triggers (light / acoustic / magnetic) have gradually attracted attention. They improve efficacy and reduce toxicity by maximizing the accumulation of free drugs at predetermined action sites. However, determining the specific location of drug release in lesions (such as tumors) that evolve dynamically in time and space is a major challenge for targeted therapy. Therefore, the development of spatially explicit imaging modes and tumor targeted therapy strategies under guidance is expected to achieve a synergistic effect of 1+1>2, improve the feasibility of tumor cure and prevent metastasis and recurrence, and bring huge opportunities for the development of new tumor treatment methods in the future. However, the spatial resolution of conventional magnetic resonance imaging, computed tomography or ultrasound Doppler imaging is insufficient, which not only makes it difficult to clearly image actively changing tumor microvessels, but also fails to specifically identify highly malignant tumor areas.

[0004] In recent years, ultrasound imaging technology has developed rapidly. In particular, ultrasound contrast imaging has achieved high signal-to-background ratio enhancement of tiny blood flow signals with the help of microbubble contrast agents, and ultrasound super-resolution imaging technology based on ultrasound contrast imaging can non-invasively reconstruct the morphology and blood flow function of tumor microvessels. So far, this technology has not been directly applied to target area identification for drug release. Summary of the invention

[0005] The embodiments of the present application provide a high-definition vascular structure reconstruction and blood flow function evaluation method and system, a high-definition vascular structure reconstruction and blood flow function evaluation device, an electronic device and a computer-readable storage medium, which can not only reliably evaluate the tumor microvascular structure, blood flow function and temporal and spatial heterogeneity, and provide rich parameters for quantitatively characterizing the complex vascular system of the tumor for the early evaluation of drug efficacy, but also can specifically identify the target blood perfusion area according to the functional characteristics, and provide target prompts and feedback intervention basis for the targeted release of drugs.

[0006] The present application provides a high-definition vascular structure reconstruction and blood flow function evaluation method, that is, constructing a high-dimensional feature manifold space based on ultrasound contrast images, automatically identifying vascular subgroups with different blood flow function patterns, and finally obtaining a high-definition rainbow vascular structure video and vascular heterogeneity indicators in the ultrasound scanning area. The specific steps include: splicing the original video Z0 containing M frames of ultrasound contrast images in chronological order to obtain a video space Ω1, further, by identifying the pixels belonging to microbubbles, a total of R microbubble spatiotemporal trajectory pixel points are obtained; in the video space Ω1, the instantaneous movement rate and direction of the microbubble at each microbubble spatiotemporal trajectory pixel point, that is, the instantaneous blood flow function characteristics, and then, based on the R The instantaneous blood flow function features corresponding to the pixels of the microbubble space-time trajectory are constructed to obtain a high-dimensional feature manifold space Ω2. In the high-dimensional feature manifold space Ω2, the instantaneous blood flow function features corresponding to the R pixels of the microbubble space-time trajectory are divided into K categories, i.e., K blood flow function modes, by using the pattern recognition method, wherein the blood flow function mode to which the instantaneous blood flow function features corresponding to each pixel of the microbubble space-time trajectory belong is unique. The instantaneous blood flow function features corresponding to the R pixels of the microbubble space-time trajectory divided into K categories in the high-dimensional feature manifold space Ω2 are traced back to the R pixels of the microbubble space-time trajectory in the video space Ω1, and each pixel of the microbubble space-time trajectory is given a unique color code C according to the category. i (i=1, 2, 3, ....K); then, K rainbow ultrasound contrast imaging videos Z with the same size as the original video Z0 are formed. i (i=1,2,3,....K); for rainbow ultrasound contrast video Z i Perform super-resolution reconstruction to obtain high-definition vascular structure video Z * ; The number of categories K and the difference between categories are used as indicators of vascular heterogeneity within the scanning field of view.

[0007] In one embodiment, an original video Z0 including M frames of ultrasound contrast images is spliced ​​in time sequence to obtain a video space Ω1. Further, a total of R microbubble spatiotemporal trajectory pixel points are obtained by identifying the pixels to which the microbubbles belong, including: for an original video Z0 including M frames of two-dimensional ultrasound contrast images (image size A×L, unit: pixel), the two-dimensional ultrasound contrast images are spliced ​​frame by frame in time sequence in the form of a numerical matrix to construct a video space Ω1 (three-dimensional, A×L×M, unit: pixel); R microbubble spatiotemporal trajectory pixel points P (x r , z r , t r )(r=1,2,3,...,R,1≤x r ≤A,1≤y r ≤L,1≤z r ≤M) are identified as pixels belonging to the microbubble space-time trajectory, wherein the R microbubble space-time trajectory pixel points P(x r , z r , t r ) can present the three-dimensional spatiotemporal trajectory of moving microbubbles in the video space Ω1.

[0008] In one embodiment, in the video space Ω1, estimating the instantaneous motion speed and direction of the microbubble at each microbubble spatiotemporal trajectory pixel point, that is, the instantaneous blood flow function characteristics, includes: in the video space Ω1, based on R microbubble spatiotemporal trajectory pixel points P(x r , z r , t r )(r=1, 2, 3, ..., R), calculate the local structure tensor; and / or extract the trajectory skeleton and perform linear fitting to obtain R microbubble spatiotemporal trajectory pixel points P(x r , z r , t r The instantaneous trajectory direction vector T at )(r=1, 2, 3, ..., R) r (x r , z r , t r )(r=1,2,3,...,R); Calculate each instantaneous trajectory direction vector T r (x r , z r , t r )(r=1,2,3,...,R) corresponds to the azimuth angle θ r (x r , z r , t r )(r=1,2,3,...,R) and polar angle Get the R microbubble spatiotemporal trajectory pixel points P(x r , z r , t r The instantaneous movement direction of microbubbles at (r=1, 2, 3, ..., R) is D r (r = 1, 2, 3, ..., R) and instantaneous motion rate V r (r=1, 2, 3, ..., R);

[0009] In one embodiment, a high-dimensional feature manifold space Ω2 is constructed based on the instantaneous blood flow function characteristics corresponding to the R microbubble spatiotemporal trajectory pixel points, including: taking the instantaneous motion direction and instantaneous motion rate as the basis, the R microbubble spatiotemporal trajectory pixel points P(x r , z r , t r The instantaneous movement direction of microbubbles at (r=1, 2, 3, ..., R) is D r (r = 1, 2, 3, ..., R) and instantaneous motion rate V r (r=1, 2, 3, ..., R) are combined in pairs to form the instantaneous blood flow function feature vector {(D r , V r )|r=1,2,3,…,R} and mapped into the high-dimensional feature manifold space Ω2.

[0010] In one embodiment, in the high-dimensional feature manifold space Ω2, the instantaneous blood flow function features corresponding to the R microbubble spatiotemporal trajectory pixel points are divided into K categories using a pattern recognition method, including: the instantaneous blood flow function feature vectors {(D r , V r )|r=1,2,3,...,R} to perform unsupervised clustering or weakly supervised clustering to obtain K clusters, i.e., K blood flow function patterns, wherein each instantaneous blood flow function feature vector belongs to a unique blood flow function pattern, i.e., each microbubble spatiotemporal trajectory pixel point belongs to a unique blood flow function pattern.

[0011] In one embodiment, the instantaneous blood flow function features corresponding to the R microbubble spatiotemporal trajectory pixels divided into K categories in the high-dimensional feature manifold space Ω2 are traced back to the R microbubble spatiotemporal trajectory pixels in the video space Ω1, and each microbubble spatiotemporal trajectory pixel is given a unique color code C according to the category. i (i=1, 2, 3, ....K); then, K rainbow ultrasound contrast imaging videos Z with the same size as the original video Z0 are formed. i (i=1, 2, 3, ....K) includes: tracing back the R microbubble spatiotemporal trajectory pixel points from the high-dimensional feature manifold space Ω2 to the video space Ω1 one by one, and according to the R microbubble spatiotemporal trajectory pixel points P(xr , z r , t r )'s instantaneous blood flow function characteristic vector {(D r , V r )|r=1,2,3,...,R} uniquely belongs to the blood flow function mode, giving the corresponding microbubble spatiotemporal trajectory pixel in the video space Ω1 a unique color code C i (i=1, 2, 3, ....K), and finally form K microbubble spatiotemporal trajectory pixel subsets; according to the color coding order, the pixels in the original video Z0 belonging to the color coding C i The i-th microbubble spatiotemporal trajectory pixel subsets (i=1, 2, 3, ....K) are separated in sequence to construct K rainbow ultrasound contrast imaging videos Z with the same size as the original video Z0. i (i=1, 2, 3, ....K).

[0012] In one embodiment, the rainbow ultrasound contrast video Z i Perform super-resolution reconstruction to obtain high-definition vascular structure video Z * Including, for K rainbow ultrasound contrast videos Z i (i=1,2,3,....K) are reconstructed by ultrasound localization microscope, and K color codes are obtained correspondingly as C i (i=1,2,3,....K) super-resolution microvascular subset image V i (i=1, 2, 3, ....K), further, by performing all super-resolution microvascular subset images V i Weighted superposition is performed to obtain a rainbow super-resolution microvascular image with K colors, i.e., a high-definition vascular structure video Z * .

[0013] In one embodiment, using the number of categories K and the difference between categories as indicators of vascular heterogeneity within the scanning field of view includes calculating the maximum / minimum distance between the centers of K clusters based on the number of blood flow function patterns K and the spatial distribution of the K blood flow function patterns in the high-dimensional feature manifold space Ω2; and / or the uniformity of the overall proportion of the K clusters.

[0014] This application provides a high-definition vascular structure reconstruction and blood flow function evaluation system, which consists of the following modules:

[0015] The video space construction module splices the original video Z0 containing M frames of ultrasound contrast imaging images in time sequence to obtain the video space Ω1. Furthermore, a total of R microbubble spatiotemporal trajectory pixel points are obtained by identifying the pixels to which the microbubbles belong.

[0016] The high-dimensional feature manifold space construction module estimates the instantaneous motion rate and direction of the microbubble at each pixel point of the microbubble spatiotemporal trajectory in the video space Ω1, that is, the instantaneous blood flow function characteristics. Then, the high-dimensional feature manifold space Ω2 is constructed based on the instantaneous blood flow function characteristics corresponding to the R pixel points of the microbubble spatiotemporal trajectory.

[0017] The blood flow function pattern recognition module uses the pattern recognition method to classify the instantaneous blood flow function features corresponding to the R microbubble spatiotemporal trajectory pixels into K categories, i.e., K blood flow function patterns, in which the blood flow function pattern to which the instantaneous blood flow function features corresponding to each microbubble spatiotemporal trajectory pixel belong is unique;

[0018] The Rainbow Ultrasound Contrast Imaging Video Reconstruction Module traces back the instantaneous blood flow function features corresponding to the R microbubble spatiotemporal trajectory pixels divided into K categories in the high-dimensional feature manifold space Ω2 to the R microbubble spatiotemporal trajectory pixels in the video space Ω1, and assigns each microbubble spatiotemporal trajectory pixel a unique color code C according to the category. i (i=1, 2, 3, ....K); then, K rainbow ultrasound contrast imaging videos Z with the same size as the original video Z0 are formed. i (i=1, 2, 3, ....K);

[0019] High-definition vascular structure video reconstruction module: convert rainbow ultrasound contrast video Z i The high-definition vascular structure video Z is obtained through super-resolution reconstruction algorithm * ;

[0020] Vascular heterogeneity evaluation module: The number of categories K in the scanning field of view and the difference between categories are used as indicators of vascular heterogeneity.

[0021] Accordingly, the present application provides an electronic device, comprising: a processor; and a memory, wherein computer program instructions are stored in the memory, and when the computer program instructions are executed by the processor, the processor executes the high-definition vascular structure reconstruction and blood flow function evaluation method and system as described in any of the above.

[0022] Accordingly, the present application provides a computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the processor executes the high-definition vascular structure reconstruction and blood flow function evaluation method and system as described above.

[0023] Benefits

[0024] The embodiment of the present application provides a high-definition vascular structure reconstruction and blood flow function evaluation method, a high-definition vascular structure reconstruction and blood flow function evaluation device, an electronic device, and a computer-readable storage medium. Based on the high-dimensional feature manifold space constructed by ultrasound contrast imaging video, multiple blood flow function patterns and high-definition rainbow vascular structure videos are identified, and tumor microvascular structure, blood flow function, and temporal and spatial heterogeneous characteristics are evaluated at the same time, which can not only provide a powerful tool for early clinical treatment evaluation, but also provide target prompts and feedback intervention basis for targeted drug release. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 The figure is a flow chart of a high-definition vascular structure reconstruction and blood flow function evaluation method provided in one embodiment of the present application.

[0026] Figure 2A Shown is a schematic diagram of a video space including multiple microbubbles provided by an embodiment of the present application.

[0027] Figure 2B Shown is a schematic diagram of a video space including multiple microbubble spatiotemporal trajectories provided by an embodiment of the present application.

[0028] Figure 3 Shown is a schematic diagram of calculating the instantaneous trajectory direction vector and instantaneous blood flow function characteristics at a pixel point of a single microbubble spatiotemporal trajectory provided by an embodiment of the present application.

[0029] Figure 4 Shown is a schematic diagram of the instantaneous blood flow function feature vector of a single microbubble spatiotemporal trajectory pixel point in a high-dimensional feature manifold space provided by an embodiment of the present application.

[0030] Figure 5A Shown is a schematic diagram of instantaneous blood flow function feature vectors of 48642 microbubble spatiotemporal trajectory pixels in a high-dimensional feature manifold space provided by an embodiment of the present application.

[0031] Figure 5B Shown is a schematic diagram of blood flow function pattern recognition results of instantaneous blood flow function feature vectors of 48642 microbubble spatiotemporal trajectory pixels in a high-dimensional feature manifold space provided by an embodiment of the present application.

[0032] Figure 6 Shown is a schematic diagram of blood flow function pattern recognition results of 48642 microbubble spatiotemporal trajectory pixels in a video space provided by an embodiment of the present application.

[0033] Fig. 7A Shown is a schematic diagram of the spatiotemporal trajectory of microbubbles corresponding to the first blood flow functional mode in the video space provided in an embodiment of the present application.

[0034] Figure 7BShown is a schematic diagram of the spatiotemporal trajectory of microbubbles corresponding to the second blood flow functional mode in the video space provided in an embodiment of the present application.

[0035] Figure 7C Shown is a schematic diagram of the spatiotemporal trajectory of microbubbles corresponding to the third blood flow function mode in the video space provided in an embodiment of the present application.

[0036] Fig.7D Shown is a schematic diagram of the spatiotemporal trajectory of microbubbles corresponding to the fourth blood flow function mode in the video space provided in an embodiment of the present application.

[0037] Fig. 8A Shown is a schematic diagram of a super-resolution microvessel subset image corresponding to a first blood flow functional mode in a video space provided by an embodiment of the present application.

[0038] Figure 8B Shown is a schematic diagram of a super-resolution microvessel subset image corresponding to a second blood flow functional mode in a video space provided by an embodiment of the present application.

[0039] Figure 8C Shown is a schematic diagram of a super-resolution microvessel subset image corresponding to the third blood flow function mode in the video space provided by an embodiment of the present application.

[0040] Fig.8D Shown is a schematic diagram of a super-resolution microvessel subset image corresponding to the fourth blood flow function mode in the video space provided by an embodiment of the present application.

[0041] Fig. 9 Shown is a schematic diagram of a method for identifying drug action target areas and focusing drug release for high-definition vascular structure reconstruction and blood flow function evaluation provided in one embodiment of the present application.

[0042] Fig.10 Shown is a schematic structural diagram of a high-definition vascular structure reconstruction and blood flow function evaluation device provided in one embodiment of the present application.

[0043] Fig.11 Shown is a schematic structural diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present application are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0045] As mentioned above, changes in tumor microenvironment conditions caused by abnormal microblood flow changes, such as hypoxia and acidosis, will stimulate the local tumor area to produce cell subpopulations with greater metastatic potential and drug resistance, which will greatly weaken the effectiveness of traditional global cancer treatment strategies. Although controlled release targeted precision treatment strategies can improve efficacy and reduce toxicity by maximizing the accumulation of free drugs at predetermined action sites, determining the specific location of drug release in tumors that dynamically evolve in time and space is a major challenge for targeted therapy. Therefore, the development of spatially explicit imaging modes and tumor targeted treatment strategies guided by them are crucial for clinical precision diagnosis and treatment. However, traditional clinical imaging strategies are not only unable to clearly image complex tumor microvasculature, but also unable to specifically identify tumor subregions with high potential for malignant development.

[0046] In response to the above technical problems, the present application proposes a high-definition vascular structure reconstruction and blood flow function evaluation method and system. The basic idea is: only multiple frames of ultrasound contrast images containing flowing microbubble images need to be taken, and then a high-dimensional feature manifold space is constructed based on multiple frames of ultrasound contrast images including microbubble images, so that vascular subpopulations with different blood flow function patterns can be identified, and finally a high-definition rainbow vascular structure video and vascular heterogeneity indicators within the ultrasound scanning area are obtained. This not only enables it to reliably evaluate the tumor microvascular structure, blood flow function and temporal and spatial heterogeneity characteristics, and provide rich parameters for quantitatively characterizing the complex vascular system of the tumor for the early evaluation of drug efficacy, but also can specifically identify the target blood perfusion sub-area according to the functional characteristics, thereby providing target prompts and feedback intervention basis for targeted drug release.

[0047] It should be noted that the present application does not specifically limit the application scenarios of high-definition vascular structure reconstruction and blood flow function evaluation. After introducing the basic principles of the present application, various non-limiting embodiments of the present application will be specifically introduced below.

[0048] Example 1

[0049] Figure 1 FIG. 1 is a flow chart of a method for high-definition vascular structure reconstruction and blood flow function evaluation provided by an embodiment of the present application. Figure 1 As shown, the high-definition vascular structure reconstruction and blood flow function evaluation method includes the following steps:

[0050] Step 101: splice the original video Z0 containing M frames of ultrasound contrast imaging images in time sequence to obtain the video space Ω1, and further obtain a total of R microbubble spatiotemporal trajectory pixel points by identifying the pixels belonging to the microbubbles.

[0051] Specifically, the original video Z0 containing M frames of ultrasound contrast images may be an image captured by an ultrasound imaging system after an ultrasound contrast agent is injected into the blood. The image of the microbubble in the ultrasound contrast image may be a point spread function image with a certain radius centered on the microbubble. Due to the diffraction effect of the ultrasound waves emitted by the ultrasound imaging system, the microbubble appears in the contrast image as an image with a certain radius centered on the microbubble, that is, an image of the point spread function of the microbubble.

[0052] In each frame of ultrasound contrast imaging, each microbubble will have a corresponding point spread function image. When the microbubble flows in the blood vessels, M frames of ultrasound contrast imaging including multiple microbubble images of size A×L are spliced ​​frame by frame in time sequence in the form of numerical matrices to form a three-dimensional video space Ω1 of size A×L×M, including two spatial dimensions and one time dimension information. Since the microbubble images of the flowing microbubbles can be partially connected and overlapped in the time dimension (dimension size M) direction of the video space Ω1, each microbubble can form a coherent three-dimensional columnar structure in the video space Ω1.

[0053] Figure 2A FIG. 2 is a schematic diagram of obtaining a video space Ω1 by splicing the original video Z0 in time sequence provided by an embodiment of the present application. Figure 2A As shown, the video space 220 is obtained by splicing the original video containing 300 frames of 100×100 pixel ultrasound contrast images in chronological order, with a size of 100×100×300, including 8 three-dimensional columnar structures formed by 8 microbubbles, namely columnar structure 211, columnar structure 212, columnar structure 213, columnar structure 214, columnar structure 215, columnar structure 216, columnar structure 217, and columnar structure 218.

[0054] It should be understood that the number of microbubbles in each frame of ultrasound contrast imaging can be selected according to actual needs, such as reconstruction accuracy, blood flow velocity in the imaging area, and vascular complexity. This application does not specifically limit the number of microbubbles in each frame of ultrasound contrast imaging.

[0055] In one embodiment, step 101 may also be to splice 200, 300 or 400 consecutive frames of ultrasound contrast images within a time interval in chronological order. Specifically, the number of frames spliced ​​in chronological order can be selected according to actual conditions. For example, when the blood flow velocity in the imaging area is slow, the acquisition time can be extended and the number of acquisition frames can be increased. At the same time, the ultrasound imaging frame rate can be selected according to actual conditions. For example, when the blood flow velocity in the imaging area is fast, that is, the microbubble movement velocity is fast, the ultrasound imaging frame rate can be increased, so as to control the microbubbles to form a three-dimensional connected columnar structure in the video space.

[0056] It should be understood that due to the interference of noise and background signals in ultrasound contrast images, it is necessary to distinguish the pixels belonging to microbubbles from the noise and background pixels in the video space. Specifically, Figure 2A The columnar structure 211 to the columnar structure 218 shown in the figure are used to identify the pixel points of the microbubble spatiotemporal trajectory, and the obtained Figure 2B Shown are microbubble space-time trajectory 221 , microbubble space-time trajectory 222 , microbubble space-time trajectory 223 , microbubble space-time trajectory 224 , microbubble space-time trajectory 225 , microbubble space-time trajectory 226 , microbubble space-time trajectory 227 , and microbubble space-time trajectory 228 .

[0057] In one embodiment, 48642 microbubble spatiotemporal trajectory pixels P (x r , z r , t r )(r=1, 2, 3,..., 48642, 1≤x r ≤100, 1≤y r ≤100, 1≤z r ≤300) were identified as pixels belonging to the microbubble spatiotemporal trajectory, and the remaining 2951358 pixels were identified as non-microbubble spatiotemporal trajectory pixels, such as Figure 2B As shown, the 48642 microbubble spatiotemporal trajectory pixels P(x r , z r , t r ) can present the three-dimensional spatiotemporal trajectory of moving microbubbles in the video space Ω1.

[0058] It should be understood that the grayscale threshold can be set according to actual needs, such as the signal-to-noise ratio of the ultrasound contrast imaging image, the degree of respiratory movement interference, and other factors. The present application does not make any specific limitation on the setting of the grayscale threshold.

[0059] Step 102: in the video space Ω1, the instantaneous movement rate and direction of the microbubble at each pixel point of the microbubble spatiotemporal trajectory, i.e., the instantaneous blood flow function characteristics, are estimated, and then, a high-dimensional feature manifold space Ω2 is constructed based on the instantaneous blood flow function characteristics corresponding to the R pixel points of the microbubble spatiotemporal trajectory;

[0060] Specifically, since the spatiotemporal trajectory of the microbubble is obtained by splicing multiple frames of contrast images in a time interval in a time sequence, the spatiotemporal trajectory of the microbubble can be the motion trajectory of the microbubble in the time interval where the multiple frames of contrast images are located. Therefore, the direction and speed of the movement of the microbubble at any position on the spatiotemporal trajectory are the instantaneous motion direction and speed of the microbubble, and the instantaneous motion direction and speed of the microbubble corresponding to the pixel coordinates of each microbubble spatiotemporal trajectory are equal to the instantaneous motion direction and speed of the corresponding microbubble in the corresponding time point and the corresponding frame.

[0061] For example, in Figure 2B The 48642 microbubble space-time trajectory pixels P (x r , z r , t r )(r=1, 2, 3,..., 48642, 1≤x r ≤100, 1≤y r ≤100, 1≤z r ≤300) to calculate the structure tensor; and / or extract the trajectory skeleton and perform linear fitting to obtain 48642 microbubble spatiotemporal trajectory pixel points P(x r , z r , t r The instantaneous trajectory direction vector T at )(r=1,2,3,...,48642) r (x r , z r , t r )(r=1, 2, 3,..., 48642);

[0062] In one embodiment, the instantaneous trajectory direction vector T is obtained by extracting the trajectory skeleton and performing straight line fitting. r Specifically, the trajectory skeleton can be extracted by using an iterative erosion boundary algorithm. The iterative erosion boundary algorithm obtains the skeleton of the corresponding microbubble spatiotemporal trajectory by performing refinement operations multiple times. The algorithm is simple and has a fast calculation speed. The straight line fitting can use the Hough transform straight line detection method. The Hough transform straight line detection method has strong anti-noise and anti-deformation capabilities and can accurately extract the trajectory straight line corresponding to the trajectory.

[0063] In one embodiment, the structure tensor can be obtained by using the main direction of the gradient in the neighborhood of the pixel point, and the instantaneous trajectory direction vector T perpendicular to the local main grayscale gradient direction indicated by the structure tensor is inferred. r .

[0064] Furthermore, the 48642 microbubble spatiotemporal trajectory pixels P(x r , z r , t r )(r=1, 2, 3, ..., 48642 corresponding to the instantaneous movement direction of 48642 microbubbles D r (r = 1, 2, 3, ..., 48642) and 48642 instantaneous motion rates V r (r = 1, 2, 3, ..., 48642), after pairwise combination, 48642 instantaneous blood flow function feature vectors {(D r , V r)|r=1,2,3,...,48642}, and map 48642 instantaneous blood flow function feature vectors into the high-dimensional feature manifold space Ω2.

[0065] Specifically, the spherical coordinates of the instantaneous trajectory direction vector at the pixel point of the microbubble space-time trajectory can be calculated first, and then the instantaneous movement direction and instantaneous movement rate of the microbubble at the pixel point of the microbubble space-time trajectory can be calculated, and finally the instantaneous blood flow function feature vector can be formed. Figure 3 As shown, the instantaneous trajectory direction vector 31 at the pixel point 30 of the single microbubble space-time trajectory is expressed in the spherical coordinate system to obtain the polar angle 33 and the azimuth angle 32. In one embodiment, since the instantaneous axial motion direction of the microbubble at the pixel point 30 of the microbubble space-time trajectory is along the positive direction of the y-axis, the instantaneous axial motion direction of the microbubble is set to -1, otherwise it is set to 1; the γ value is used as the instantaneous lateral motion direction of the microbubble, and the instantaneous axial motion direction and the instantaneous lateral motion direction of the microbubble are combined as the instantaneous motion direction of the microbubble, that is, D 30 =[-1,γ]; the absolute value of 1 / tan(θ) is taken as the instantaneous motion rate of the microbubble at pixel point 30 of the microbubble spatiotemporal trajectory, that is, V r =|1 / tan(θ)|. Based on the above, the instantaneous blood flow function characteristic vector [-1, γ, |1 / tan(θ)|] corresponding to the pixel point 30 of the microbubble spatiotemporal trajectory is obtained.

[0066] Repeat the above steps to obtain 48642 microbubble spatiotemporal trajectory pixel points P(x r , z r , t r )(r=1,2,3,...,48642) of 48642 instantaneous blood flow function feature vectors {(D r , V r )|r=1, 2, 3,..., 48642}.

[0067] by Figure 3 Taking the microbubble spatiotemporal trajectory pixel point 30 as an example, Figure 4 As shown, in the high-dimensional characteristic manifold space 40 composed of the microbubble instantaneous motion rate axis 41, the microbubble instantaneous axial motion direction axis 42 and the microbubble instantaneous lateral motion direction axis 43, Figure 3 The microbubble spatiotemporal trajectory pixel point 30 in the projection is projected to the position of the feature point 44, and the corresponding high-dimensional feature manifold space Ω2 coordinate is [-1, γ, |1 / tan(θ)|]. Repeat the above steps to obtain 48642 microbubble spatiotemporal trajectory pixel points P(x r , z r , t r )(r=1,2,3,...,48642) 48642 instantaneous blood flow function feature points, such as Figure 5AAs shown, the coordinates correspond to {(D r , V r )|r=1, 2, 3,..., 48642}.

[0068] Step 103: In the high-dimensional feature manifold space Ω2, the instantaneous blood flow function features corresponding to the R microbubble spatiotemporal trajectory pixels are divided into K categories, namely, K blood flow function patterns, by using a pattern recognition method.

[0069] Specifically, from step 101 to step 102, 48642 microbubble spatiotemporal trajectory pixels P (x r , z r , t r )(r=1,2,3,...,48642) 48642 instantaneous blood flow function feature vectors {(D r , V r )|r=1,2,3,...,48642}, and represented in the high-dimensional feature manifold space, forming 48642 instantaneous blood flow function feature points, such as Figure 5A As shown, in the high-dimensional feature manifold space 510 composed of the microbubble instantaneous motion rate axis 521, the microbubble instantaneous axial motion direction axis 522 and the microbubble instantaneous lateral motion direction axis 523, pattern recognition is performed on 48642 instantaneous blood flow function feature points 514, and the following is obtained: Figure 5B The blood flow function pattern recognition results shown in Figure 1 are as follows: Figure 5A The 48642 instantaneous blood flow function feature points 514 shown are divided into four categories of blood flow function modes: a first blood flow function mode 525, a second blood flow function mode 526, a third blood flow function mode 527, and a fourth blood flow function mode 528, which respectively contain 11017, 7453, 10219, and 19953 instantaneous blood flow function feature points. The blood flow function mode to which each instantaneous blood flow function feature vector belongs is unique, that is, the blood flow function mode to which each microbubble spatiotemporal trajectory pixel point belongs is unique.

[0070] In one embodiment, preferably, the pattern recognition adopts unsupervised Kmeans clustering. Kmeans clustering has fast convergence speed and high algorithm efficiency, and can adaptively identify blood flow function patterns with similar functional characteristics.

[0071] In one embodiment, preferably, pattern recognition uses a support vector machine, which can achieve binary classification of high-dimensional data in a nonlinear form and can accurately separate blood flow function patterns with different functional characteristics.

[0072] Step 104: The instantaneous blood flow function features corresponding to the R microbubble spatiotemporal trajectory pixels divided into K categories in the high-dimensional feature manifold space Ω2 are traced back to the R microbubble spatiotemporal trajectory pixels in the video space Ω1, and each microbubble spatiotemporal trajectory pixel is given a unique color code C according to the category. i (i=1, 2, 3, ....K); then, K rainbow ultrasound contrast imaging videos Z with the same size as the original video Z0 are formed. i (i=1, 2, 3, ....K);

[0073] Specifically, Figure 5B The 48642 microbubble spatiotemporal trajectory pixels in the high-dimensional feature manifold space 520 shown in FIG are traced back one by one to Figure 6 The video space 60 shown in FIG. 5 is a graph showing a blood flow function pattern to which the instantaneous blood flow function feature vectors of the 48642 microbubble spatiotemporal trajectory pixels in the high-dimensional feature manifold space 520 uniquely belong, and a unique color code is assigned to the 48642 microbubble spatiotemporal trajectory pixels in the video space 60, thereby forming four microbubble spatiotemporal trajectory pixel subsets, as shown in FIG. Figure 6 As shown, 11017 microbubble spatiotemporal trajectory pixels in the first blood flow function pattern 525 correspond to microbubble spatiotemporal trajectory 64, 7453 pixels in the second blood flow function pattern 526 correspond to microbubble spatiotemporal trajectory 63, 10219 pixels in the third blood flow function pattern 527 correspond to microbubble spatiotemporal trajectories 66-68, and 19953 pixels in the fourth blood flow function pattern 528 correspond to microbubble spatiotemporal trajectories 61, 62 and 65. Further, according to the color coding order, the 11017 microbubble spatiotemporal trajectory pixels belonging to color coding 1, i.e., the first blood flow function pattern 525, are separated, as shown in FIG. Fig. 7A As shown, an ultrasound contrast video Z1 corresponding to the first blood flow function mode 525 with a size of 100×100×300 and color coding of 1 is constructed, in which 11017 microbubble spatiotemporal trajectory pixels form a microbubble spatiotemporal trajectory 711; the microbubble spatiotemporal trajectory pixels corresponding to the second, third, and fourth blood flow function modes 526, 527, and 528 are sequentially subjected to the same operation. In summary, a total of 4 ultrasound contrast videos Z1, Z2, Z3, and Z4 with a size of 100×100×300 are obtained, corresponding to the formation of Fig. 7A Microbubble space-time trajectory in 711, Figure 7B Microbubble space-time trajectory in 721, Figure 7C Microbubble spatiotemporal trajectories in 731-733, Fig.7D The microbubble spatiotemporal trajectories 741-743 in the image show that the microbubble spatiotemporal trajectories are effectively separated, which lays the foundation for more accurate reconstruction and quantification of microvascular structure and function.

[0074] Step 105: For K rainbow ultrasound contrast videos Z i(i=1,2,3,....K) are super-resolved and reconstructed respectively, and K corresponding colors are coded as C i (i=1,2,3,....K) super-resolution microvascular subset image V i (i=1, 2, 3, ....K), further, by performing all super-resolution microvascular subset images V i Perform weighted superposition to obtain a rainbow super-resolution microvascular image with K colors;

[0075] Specifically, the corresponding Fig. 7A , 7B , 7C, 7D, the four ultrasound contrast imaging videos Z1, Z2, Z3, and Z4 with a size of 100×100×300 are super-reconstructed to obtain the corresponding Fig. 8A , 8B , 8C, and 8D, namely, the four super-resolution microvessel subset images V1, V2, V3, and V4, have corresponding color codes of 1, 2, 3, and 4, respectively.

[0076] In one embodiment, preferably, the super-resolution reconstruction strategy uses ultrasound localization microscopy technology and nearest neighbor tracking technology. Ultrasound localization microscopy technology and nearest neighbor tracking technology have fast operation speed and simple algorithm design, and can reconstruct vascular structure and function with extremely high spatial resolution.

[0077] In one embodiment, preferably, the super-resolution reconstruction strategy adopts ultrasound diffraction attenuation microscopy technology. The ultrasound diffraction attenuation microscopy technology has high algorithm efficiency and fast calculation speed, and can reconstruct continuous vascular structures with extremely high temporal resolution and spatial resolution.

[0078] Step 106: Based on the number K of blood flow function patterns in the high-dimensional feature manifold space Ω2 and the spatial distribution characteristics of the K blood flow function patterns, establish a blood flow function quantitative evaluation index, including: the maximum / minimum distance between the centers of the K clusters; and / or the uniformity of the overall proportion of the K clusters.

[0079] Specifically, in Figure 5BIn the high-dimensional feature manifold space 520 shown, the average values ​​of 11017, 7453, 10219, and 19953 instantaneous blood flow function feature points corresponding to the first blood flow function pattern 525, the second blood flow function pattern 526, the third blood flow function pattern 527, and the fourth blood flow function pattern 528, respectively, can be calculated to obtain the center point positions of the four clusters (X1, Y1, Z1), (X2, Y2, Z2), (X3, Y3, Z3), and (X4, Y4, Z4); the proportion of instantaneous blood flow function feature points belonging to the first blood flow function pattern 525, the second blood flow function pattern 526, the third blood flow function pattern 527, and the fourth blood flow function pattern 528 to all instantaneous blood flow function feature points can be calculated respectively.

[0080] In one embodiment, the maximum distance between the center points of the four clusters (X1, Y1, Z1), (X2, Y2, Z2), (X3, Y3, Z3), and (X4, Y4, Z4) can be calculated. Figure 5B , the positions of the four cluster center points corresponding to the first blood flow function pattern 525, the second blood flow function pattern 526, the third blood flow function pattern 527, and the fourth blood flow function pattern 528 are (0.25, 0.43, 0.36), (0.33, 0.17, 0.29), (0.81, -0.27, 0.29), and (0.74, -0.19, 0.52), respectively. The Euclidean distances between every two cluster center points are 0.28, 0.90, 0.81, 0.65, 0.59, and 0.25, respectively, and the maximum distance is 0.90.

[0081] In one embodiment, the variance of the proportion of each cluster in the four clusters can be calculated. Figure 5B In the figure, the proportions of the corresponding instantaneous blood flow function feature points in the four clusters to all instantaneous blood flow function feature points are 0.2, 0.1, 0.6, and 0.1, respectively, and the variance is 0.24.

[0082] Example 2

[0083] Fig. 9 FIG. 1 is a flow chart of a method for high-definition vascular structure reconstruction and blood flow function evaluation target area identification and ultrasound focused drug release provided by an embodiment of the present application. Fig. 9 As shown, the target area identification and ultrasound focused drug release method for high-definition vascular structure reconstruction and blood flow function evaluation includes the following steps:

[0084] Step 901: After drug-loaded microbubbles are prepared and injected into the body, a low mechanical index plane wave emitted by an ultrasound array transducer is used to collect an ultrasound contrast imaging sequence.

[0085] Specifically, according to the type of disease to be treated, a therapeutic drug is selected, and microbubbles loaded with the therapeutic drug are prepared. Furthermore, after the ultrasound contrast microbubbles carrying the drug are injected into the biological body according to the set drug dose, an ultrasound array transducer corresponding to the imaging center frequency is selected based on the estimated depth of the imaging area, and the position of the ultrasound probe is fixed at the selected imaging area, and the ultrasound array transducer is started to emit a plane wave with a low mechanical index, and plane wave ultrasound data is collected to obtain M frames of ultrasound contrast images.

[0086] It should be noted that drug-loaded microbubbles refer to microbubbles that can carry drugs, and the diameter is usually less than 5 microns, wherein the drug carried can be any compound with therapeutic or preventive effect, can be gene drugs, chemotherapeutics or targeted antibody drugs, can be loaded on the surface of microbubbles or wrapped in microbubbles, and the application is not specifically limited. In addition, drug-loaded microbubbles can produce special acoustic responses under ultrasonic stimulation, such as generating special acoustic signals such as harmonics and subharmonics, so that they can be developed in ultrasonic images. The injection mode of drug-loaded microbubbles can be intravenous injection, or it can be lymphatic duct or ureteral injection, and the application is not specifically limited.

[0087] In one embodiment of the present application, the array transducer is an array composed of multiple transducers arranged in a certain form, such as a linear array arrangement, a phased array arrangement, etc., to complete the transmission of ultrasound, the focusing of ultrasound, and the reception of ultrasound signals. During the acquisition of ultrasound contrast imaging images, dynamic video images of microbubble flow can be displayed in real time on an ultrasound imaging display.

[0088] In one embodiment of the present application, the mechanical index is set to 0.1, and the imaging frame rate is set to 500 Hz.

[0089] In one embodiment of the present application, in a shallower imaging depth range, such as lymph, thyroid, etc., a high-frequency probe with an imaging center frequency of 15 MHz is selected; in a deeper imaging depth range, such as kidney, liver, etc., a low-frequency probe with an imaging center frequency of 7 MHz is selected.

[0090] Step 902: Identify drug release target areas.

[0091] Specifically, by collecting M frames of ultrasound contrast-enhanced images after the release of drug-loaded microbubbles, the blood flow structure and functional characteristics of the imaging area are analyzed to identify the range of the vascular area targeted by the drug, wherein the target vascular area for drug release can be selected through human interaction based on the high-definition vascular reconstruction results; or it can be automatically selected based on the characteristics of the lesion. This application does not specifically limit the number of ultrasound contrast-enhanced image sequences and target vascular areas to be collected.

[0092] The analysis of the blood flow structure and functional characteristics of the imaging area may include the following processing steps: splicing the original video Z0 containing M frames of ultrasound contrast imaging images in chronological order to obtain a video space Ω1, further, by identifying the pixels to which the microbubbles belong, a total of R microbubble spatiotemporal trajectory pixel points are obtained; in the video space Ω1, the instantaneous movement rate and direction of the microbubble at each microbubble spatiotemporal trajectory pixel point are estimated, that is, the instantaneous blood flow functional characteristics, and then, based on the instantaneous blood flow functional characteristics corresponding to the R microbubble spatiotemporal trajectory pixel points, a high-dimensional feature manifold space Ω2 is constructed; in the high In the high-dimensional feature manifold space Ω2, the instantaneous blood flow function features corresponding to the R microbubble spatiotemporal trajectory pixels are divided into K categories, i.e., K blood flow function modes, by using the pattern recognition method. Among them, the blood flow function mode to which the instantaneous blood flow function features corresponding to each microbubble spatiotemporal trajectory pixel belongs is unique; the instantaneous blood flow function features corresponding to the R microbubble spatiotemporal trajectory pixels divided into K categories in the high-dimensional feature manifold space Ω2 are traced back to the R microbubble spatiotemporal trajectory pixels in the video space Ω1, and each microbubble spatiotemporal trajectory pixel is given a unique color code C according to the category. i (i=1, 2, 3, ....K); then, K rainbow ultrasound contrast imaging videos Z with the same size as the original video Z0 and reflecting different blood flow function characteristics are formed. i (i=1, 2, 3, ....K). The number of categories K and the difference between categories are used as indicators of vascular heterogeneity within the scanning field of view.

[0093] Specifically, the human interactive selection based on the high-definition vascular reconstruction results includes the following processing steps: i (i=1, 2, 3, ....K) respectively perform high-definition microvascular reconstruction to form a rainbow super-resolution microvascular image with K colors, that is, a high-definition vascular structure video Z * , and displayed on the display screen, the location and size of the target vascular area for drug release can be manually judged and interactively selected.

[0094] Among them, the high-definition microvascular reconstruction strategy preferably uses ultrasound positioning microscopy technology. Ultrasound positioning microscopy technology has fast computing speed and simple algorithm, and can reconstruct vascular structure and function with extremely high spatial resolution, which is not specifically limited in this application.

[0095] It should be noted that the interactive area selection can be input through a touchpad or a mouse, the shape of the selection area can be marked with a square or a circle, and the number can be one or more, and this application does not make any specific limitations.

[0096] In one embodiment of the present application, the user can manually select an interactive region based on a single super-resolution microvascular image, or select a region from K color-coded C images. i(i=1,2,3,....K) super-resolution microvascular subset image V i (i=1, 2, 3, ....K) Select part of the image for weighted superposition, and then perform manual interactive area selection based on the superimposed image.

[0097] In another embodiment of the present application, the user can first manually and interactively select multiple vascular regions in multiple super-resolution microvascular images. Secondly, the computer performs quantitative analysis of the vascular structure and functional parameters of the selected vascular regions, and feeds back the quantitative values ​​to the user. Finally, the user screens the vascular regions according to the feedback values ​​to determine the target vascular regions.

[0098] It should be noted that the vascular structure and function parameters can be regional vascular density, regional vascular curvature average, regional vascular curvature distribution, regional vascular average diameter, regional vascular diameter distribution, regional blood flow average velocity, regional blood flow velocity distribution, and regional blood flow direction entropy, and this application does not make specific limitations.

[0099] Specifically, the automatic selection process based on lesion characteristics includes the following processing steps: automatically selecting the two-dimensional imaging section with the largest heterogeneity index, and automatically extracting the rainbow ultrasound contrast video Z with the highest microbubble flow rate and the corresponding color code c c (c<K), and perform maximum density projection operation on it, and select the coordinate set of pixels with gray value greater than 0 after projection as the target blood vessel area. In addition, automatically extract the top P rainbow ultrasound contrast imaging videos Z1, Z2, ..., Z P (P<K), and perform maximum density projection operation on them respectively, and obtain P rainbow projection images MIP1, MIP2, ..., MIP P (P<K) Perform a secondary maximum projection operation, and select the coordinate set of pixels whose gray value is greater than 0 after the secondary maximum projection as the target blood vessel area. This application does not make specific limitations.

[0100] Step 903: Start the ultrasonic array transducer to emit ultrasonic waves with a high mechanical index focused on the targeted release area to break the drug-loaded microbubbles.

[0101] The focus control modes include single focus, multi-focus and variable focus, etc. Specifically, the size, position and number of basic focus units to be synthesized are determined according to the size and position of the lesion area. Considering that the target vascular area is an area with a certain range, the present application selects a multi-focus form to achieve the destruction of drug-loaded microbubbles in the target vascular area.

[0102] It should be noted that the device that transmits focused ultrasound waves can be an ultrasonic array transducer for ultrasound contrast imaging, or a second ultrasonic array transducer specifically used for focused imaging, which is not specifically limited in this application. The user can change the focus size by changing the number of array elements involved in the emission focus. By selecting an ultrasonic array transducer equipped with an appropriate number of array elements, the purpose of drug release in a larger or smaller local area can be achieved by switching between large and small focal points. The user can select a suitable focus size according to the shape and size of the target area, which is not specifically limited in this application.

[0103] In one embodiment of the present application, the sound pressure of the focused ultrasound is set to 0.3 MPa, the number of cycles is 5000, the ultrasonic treatment time is 4 minutes, and the mechanical index is 0.3.

[0104] Step 904: Start plane wave acquisition under low mechanical index to obtain ultrasound contrast imaging of the target area after drug treatment and evaluate the drug effect.

[0105] After the drug release operation is completed, the position of the ultrasound probe is fixed at the targeted release area, and the ultrasound array transducer is started to emit a plane wave with a low mechanical index to collect plane wave ultrasound data, obtain M frames of ultrasound contrast-enhanced images, and quantitatively evaluate the drug effect based on the ultrasound contrast-enhanced images.

[0106] Specifically, the quantitative evaluation of the drug effect includes: for M frames of ultrasound contrast imaging images of the targeted release area, taking the mean or median of the image grayscale value, obtaining a function of echo intensity variation over time, i.e., a time-intensity curve, and curve fitting the time-intensity curve to obtain vascular function evaluation parameters, such as peak time and area under the curve.

[0107] In another embodiment of the present application, quantitative evaluation of the drug effect includes splicing the original video Z0 containing M frames of ultrasound contrast imaging images in chronological order to obtain a video space Ω1, and further, obtaining a total of R microbubble spatiotemporal trajectory pixel points by identifying the pixels belonging to the microbubbles; in the video space Ω1, estimating the instantaneous movement rate and direction of the microbubble at each microbubble spatiotemporal trajectory pixel point, that is, the instantaneous blood flow function characteristics, and then, constructing a high-dimensional feature manifold space Ω2 based on the instantaneous blood flow function characteristics corresponding to the R microbubble spatiotemporal trajectory pixel points; in the high-dimensional feature manifold space Ω2, using a pattern recognition method to divide the instantaneous blood flow function characteristics corresponding to the R microbubble spatiotemporal trajectory pixel points into K categories, that is, K blood flow function patterns, and further, based on the number of blood flow function patterns K in the high-dimensional feature manifold space Ω2 and the spatial distribution of the K blood flow function patterns, calculating the maximum / minimum distance between the centers of the K clusters; and / or the uniformity of the overall proportion of the K clusters as drug effect evaluation parameters.

[0108] Example 3

[0109] Fig.10 FIG. 1 is a schematic diagram of a high-definition vascular structure reconstruction and blood flow function evaluation device provided in an embodiment of the present application. Fig.10 As shown, the high-definition vascular structure reconstruction and blood flow function evaluation device 1000 includes:

[0110] The video space construction module 1010 is configured to splice the original video Z0 containing M frames of ultrasound contrast imaging images in time sequence to obtain the video space Ω1, and further obtain a total of R microbubble spatiotemporal trajectory pixel points by identifying the pixels belonging to the microbubbles;

[0111] The high-dimensional feature manifold space construction module 1020 estimates the instantaneous motion rate and direction of the microbubble at each pixel point of the microbubble spatiotemporal trajectory in the video space Ω1, that is, the instantaneous blood flow function characteristics, and then constructs the high-dimensional feature manifold space Ω2 based on the instantaneous blood flow function characteristics corresponding to the R pixel points of the microbubble spatiotemporal trajectory;

[0112] The blood flow function pattern recognition module 1030 uses a pattern recognition method to classify the instantaneous blood flow function features corresponding to the R microbubble spatiotemporal trajectory pixel points into K categories, i.e., K blood flow function patterns, in the high-dimensional feature manifold space Ω2, wherein the blood flow function pattern to which the instantaneous blood flow function features corresponding to each microbubble spatiotemporal trajectory pixel point belong is unique;

[0113] The rainbow ultrasound contrast video reconstruction module 1040 traces back the instantaneous blood flow function features corresponding to the R microbubble spatiotemporal trajectory pixels divided into K categories in the high-dimensional feature manifold space Ω2 to the R microbubble spatiotemporal trajectory pixels in the video space Ω1, and assigns each microbubble spatiotemporal trajectory pixel a unique color code C according to the category. i (i=1, 2, 3, ....K); then, K rainbow ultrasound contrast imaging videos Z with the same size as the original video Z0 are formed. i (i=1, 2, 3, ....K);

[0114] High-definition vascular structure video reconstruction module 1050: converting rainbow ultrasound contrast video Z i The high-definition vascular structure video Z is obtained through super-resolution reconstruction algorithm * ;

[0115] The vascular heterogeneity evaluation module 1060 uses the number of categories K in the scanning field of view and the difference between categories as vascular heterogeneity indicators.

[0116] In one embodiment, the video space construction module 1010 can be further configured to: for an original video Z0 (image size A×L, unit: pixel) containing M frames of two-dimensional ultrasound contrast images, splice the two-dimensional ultrasound contrast images frame by frame in the form of a numerical matrix and in time sequence to construct a video space Ω1 (three-dimensional, A×L×M, unit: pixel); and select R microbubble spatiotemporal trajectory pixel points P (x r , z r , t r )(r=1,2,3,...,R,1≤x r ≤A,1≤y r ≤L,1≤z r ≤M) are identified as pixels belonging to the microbubble space-time trajectory, wherein the R microbubble space-time trajectory pixel points P(x r , z r , t r ) can present the three-dimensional spatiotemporal trajectory of moving microbubbles in the video space Ω1.

[0117] In one embodiment, the high-dimensional feature manifold space construction module 1020 can be further configured as follows: in the video space Ω1, based on the R microbubble spatiotemporal trajectory pixel points P(x r , z r , t r )(r=1, 2, 3, ..., R), calculate the local structure tensor; and / or extract the trajectory skeleton and perform linear fitting to obtain R microbubble spatiotemporal trajectory pixel points P(x r , z r , t r The instantaneous trajectory direction vector T at )(r=1, 2, 3, ..., R) r (x r , z r , t r )(r=1,2,3,...,R); Calculate each instantaneous trajectory direction vector T r (x r , z r , t r )(r=1,2,3,...,R) corresponds to the azimuth angle θ r (x r , z r , t r )(r=1,2,3,...,R) and polar angle Get the R microbubble spatiotemporal trajectory pixel points P(x r , z r , t r The instantaneous movement direction of microbubbles at (r=1, 2, 3, ..., R) is Dr (r = 1, 2, 3, ..., R) and instantaneous motion rate V r (r = 1, 2, 3, ..., R); R microbubble spatiotemporal trajectory pixel points P (x r , z r , t r The instantaneous movement direction of microbubbles at (r=1, 2, 3, ..., R) is D r (r = 1, 2, 3, ..., R) and instantaneous motion rate V r (r=1, 2, 3, ..., R) are combined in pairs to form the instantaneous blood flow function feature vector {(D r , V r )|r=1,2,3,…,R} and mapped into the high-dimensional feature manifold space Ω2.

[0118] In one embodiment, the blood flow function pattern recognition module 1030 may be further configured to: in the high-dimensional feature manifold space Ω2, for the instantaneous blood flow function feature vector {(D r , V r )|r=1,2,3,...,R} to perform unsupervised clustering or weakly supervised clustering to obtain K clusters, i.e., K blood flow function patterns, wherein each instantaneous blood flow function feature vector belongs to a unique blood flow function pattern, i.e., each microbubble spatiotemporal trajectory pixel point belongs to a unique blood flow function pattern.

[0119] In one embodiment, the rainbow ultrasound contrast imaging video reconstruction module 1040 can be further configured to: trace back the R microbubble spatiotemporal trajectory pixel points from the high-dimensional feature manifold space Ω2 to the video space Ω1 one by one, and according to the R microbubble spatiotemporal trajectory pixel points P(x r , z r , t r )'s instantaneous blood flow function characteristic vector {(D r , V r )|r=1,2,3,...,R} uniquely belongs to the blood flow function mode, giving the corresponding microbubble spatiotemporal trajectory pixel in the video space Ω1 a unique color code C i (i=1, 2, 3, ....K), and finally form K microbubble spatiotemporal trajectory pixel subsets; according to the color coding order, the pixels in the original video Z0 belonging to the color coding C i The i-th microbubble spatiotemporal trajectory pixel subsets (i=1, 2, 3, ....K) are separated in sequence to construct K rainbow ultrasound contrast imaging videos Z with the same size as the original video Z0. i (i=1, 2, 3, ....K).

[0120] In one embodiment, the high-definition vascular structure video reconstruction module 1050 may be further configured to: i (i=1,2,3,....K) are reconstructed by ultrasound localization microscope, and K color codes are obtained correspondingly as C i (i=1,2,3,....K) super-resolution microvascular subset image V i (i=1, 2, 3, ....K), further, by performing all super-resolution microvascular subset images V i Weighted superposition is performed to obtain a rainbow super-resolution microvascular image with K colors, i.e., a high-definition vascular structure video Z * .

[0121] In one embodiment, the vascular heterogeneity evaluation module 1060 can be further configured to: calculate the maximum / minimum distance between the centers of the K clusters based on the number K of blood flow function patterns in the high-dimensional feature manifold space Ω2 and the spatial distribution of the K blood flow function patterns; and / or the uniformity of the overall proportion of the K clusters.

[0122] It should be noted that the high-definition vascular structure reconstruction and blood flow function evaluation device 1000 according to the embodiment of the present application is integrated into the electronic device 1100 as a software module and / or a hardware module. In other words, the electronic device 1100 includes the high-definition vascular structure reconstruction and blood flow function evaluation device 1000. For example, the high-definition vascular structure reconstruction and blood flow function evaluation device 1000 is a software module in the operating system of the electronic device 1100, or an application developed for the operating system.

[0123] In another embodiment of the present application, the high-definition vascular structure reconstruction and blood flow function evaluation device 1000 and the electronic device 1100 may also be separate devices (for example, a server), and the high-definition vascular structure reconstruction and blood flow function evaluation device 1000 may be connected to the electronic device 1100 via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0124] Example 4

[0125] Fig.11 FIG. 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Fig.11 As shown, the electronic device 1100 includes: one or more processors 1101 and a memory 1102; and computer program instructions stored in the memory 1102. When the computer program instructions are executed by the processor 1101, the processor 1101 executes a high-definition vascular structure reconstruction and blood flow function evaluation method as in any of the above-mentioned embodiments.

[0126] The processor 1101 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0127] The memory 1102 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on a computer-readable storage medium, and the processor 1101 may run the program instructions to implement the steps and / or other desired functions in the high-definition vascular structure reconstruction and blood flow function evaluation method of each embodiment of the present application above. Information such as microbubble angiography images and microbubble velocities may also be stored in the computer-readable storage medium.

[0128] In one example, the electronic device 1100 includes: an input device 1103 and an output device 1104 , and these components are interconnected via a bus system and / or other forms of connection mechanisms.

[0129] For example, when the electronic device is an ultrasound imaging system, the input device 1103 may be an ultrasound probe for detecting the location of microbubbles. When the electronic device is a stand-alone device, the input device 1103 may be a communication network connector for receiving collected input signals from an external removable device. In addition, the input device 1103 may also include, for example, a keyboard, a mouse, a microphone, and the like.

[0130] The output device 1104 can output various information to the outside, and may include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0131] Of course, to simplify, Fig.11 Only some of the components related to the present application in the electronic device 1100 are shown, and components such as a bus, an input device / output interface, etc. are omitted. In addition, according to specific application conditions, the electronic device 1100 may also include any other appropriate components.

[0132] Example 5

[0133] In addition to the above methods and devices, the embodiments of the present application may also be a computer program product, including computer program instructions, which, when executed by a processor, enable the processor to execute the steps in the high-definition vascular structure reconstruction and blood flow function evaluation method of any of the above embodiments.

[0134] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, etc., and may also include conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0135] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the high-definition vascular structure reconstruction and blood flow function evaluation method according to various embodiments of the present application described in the above-mentioned "High-definition vascular structure reconstruction and blood flow function evaluation method" section of this specification.

[0136] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0137] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0138] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used in this application refer to the words "and / or", and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used in this application refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0139] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0140] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0141] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

[0142] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for high-definition vascular structure reconstruction and blood flow function evaluation, characterized in that: Based on ultrasound contrast images, a high-dimensional feature manifold space is constructed to identify vascular subgroups with different blood flow function patterns, and finally a high-definition rainbow vascular structure video and vascular heterogeneity indicators within the ultrasound scanning area are obtained. The specific steps include: The original video Z0 containing M frames of ultrasound contrast images is spliced ​​in time sequence to obtain the video space Ω1. Furthermore, a total of R microbubble spatiotemporal trajectory pixel points are obtained by identifying the pixels to which the microbubbles belong; In the video space Ω1, the instantaneous movement rate and direction of the microbubble at each pixel point of the microbubble spatiotemporal trajectory, that is, the instantaneous blood flow function characteristics, are estimated. Then, based on the instantaneous blood flow function characteristics corresponding to the R pixel points of the microbubble spatiotemporal trajectory, a high-dimensional feature manifold space Ω2 is constructed; In the high-dimensional feature manifold space Ω2, the instantaneous blood flow function features corresponding to the R microbubble spatiotemporal trajectory pixels are divided into K categories, i.e., K blood flow function modes, using pattern recognition methods. Among them, the blood flow function mode to which the instantaneous blood flow function features corresponding to each microbubble spatiotemporal trajectory pixel belong is unique. The instantaneous blood flow function features corresponding to the R microbubble spatiotemporal trajectory pixels divided into K categories in the high-dimensional feature manifold space Ω2 are traced back to the R microbubble spatiotemporal trajectory pixels in the video space Ω1, and each microbubble spatiotemporal trajectory pixel is given a unique color code C according to the category. i (i=1, 2, 3, ....K); then, K rainbow ultrasound contrast imaging videos Z with the same size as the original video Z0 and reflecting different blood flow function characteristics are formed. i (i=1, 2, 3, ....K); Rainbow Ultrasound Contrast Imaging Video Z i Perform super-resolution reconstruction to obtain high-definition vascular structure video Z * ; The number of categories K and the difference between categories are used as indicators of vascular heterogeneity within the scanning field of view.

2. The video space Ω1 according to claim 1, characterized in that: Constructed by the following steps: For the original video Z0 (image size A×L, unit: pixel) containing M frames of two-dimensional ultrasound contrast-enhanced images, the two-dimensional ultrasound contrast-enhanced images are spliced ​​frame by frame in the form of a numerical matrix and in time sequence to construct a video space Ω1 (three-dimensional, A×L×M, unit: pixel); The R microbubble spatiotemporal trajectory pixels P(x r , z r , t r )(r=1,2,3,...,R,1≤x r ≤A,1≤y r ≤L,1≤z r ≤M) are identified as pixels belonging to the microbubble space-time trajectory, wherein the R microbubble space-time trajectory pixel points P(x r , z r , t r ) can present the three-dimensional spatiotemporal trajectory of moving microbubbles in the video space Ω1.

3. The high-dimensional feature manifold space Ω2 according to claim 1, characterized in that: Based on the instantaneous motion direction and instantaneous motion rate, the R microbubble spatiotemporal trajectory pixels in the video space Ω1 are mapped and represented in the high-dimensional feature manifold space Ω2 through the following steps, including: In the video space Ω1, based on the R microbubble spatiotemporal trajectory pixels P(x r , z r , t r )(r=1, 2, 3, ..., R), calculate the local structure tensor; and / or extract the trajectory skeleton and perform linear fitting to obtain R microbubble spatiotemporal trajectory pixel points P(x r , z r , t r The instantaneous trajectory direction vector T at )(r=1, 2, 3, ..., R) r (x r , z r , t r )(r=1,2,3,...,R); Calculate each instantaneous trajectory direction vector T r (x r , z r , t r )(r=1,2,3,...,R) corresponds to the azimuth angle θ r (x r , z r , t r )(r=1,2,3,...,R) and polar angle Get the R microbubble spatiotemporal trajectory pixel points P(x r , z r , t r The instantaneous movement direction of microbubbles at (r=1, 2, 3, ..., R) is D r (r = 1, 2, 3, ..., R) and instantaneous motion rate V r (r=1, 2, 3, ..., R); The R microbubble spatiotemporal trajectory pixels P(x r , z r , t r The instantaneous movement direction of microbubbles at (r=1, 2, 3, ..., R) is D r (r = 1, 2, 3, ..., R) and instantaneous motion rate V r (r=1, 2, 3, ..., R) are combined in pairs to form the instantaneous blood flow function feature vector {(D r , V r )|r=1,2,3,…,R} and mapped into the high-dimensional feature manifold space Ω2.

4. The K blood flow function modes according to claim 1, characterized in that: The following steps are performed to obtain: in the high-dimensional feature manifold space Ω2, the instantaneous blood flow function feature vector {(D r , V r )|r=1,2,3,...,R} to perform unsupervised clustering or weakly supervised clustering to obtain K clusters, i.e., K blood flow function patterns, wherein each instantaneous blood flow function feature vector belongs to a unique blood flow function pattern, i.e., each microbubble spatiotemporal trajectory pixel point belongs to a unique blood flow function pattern.

5. The rainbow ultrasound contrast imaging video according to claim 1, characterized in that: Obtained by the following steps: The R microbubble spatiotemporal trajectory pixels are traced back one by one from the high-dimensional feature manifold space Ω2 to the video space Ω1, and the R microbubble spatiotemporal trajectory pixels P(x r , z r , t r )'s instantaneous blood flow function characteristic vector {(D r , V r )|r=1,2,3,...,R} uniquely belongs to the blood flow function mode, giving the corresponding microbubble spatiotemporal trajectory pixel in the video space Ω1 a unique color code C i (i=1, 2, 3, ....K), finally forming K microbubble spatiotemporal trajectory pixel subsets; According to the color coding order, the original video Z0 that belongs to color code C i The i-th microbubble spatiotemporal trajectory pixel subsets (i=1, 2, 3, ....K) are separated in sequence to construct K rainbow ultrasound contrast imaging videos Z that are consistent with the original video Z0 and reflect different blood flow function characteristics. i (i=1, 2, 3, ....K).

6. A high-definition vascular structure reconstruction and blood flow function evaluation system, characterized in that: The method according to claim 1 is adopted, and the system is composed of the following modules: The video space construction module splices the original video Z0 containing M frames of ultrasound contrast imaging images in time sequence to obtain the video space Ω1. Furthermore, a total of R microbubble spatiotemporal trajectory pixel points are obtained by identifying the pixels to which the microbubbles belong. The high-dimensional feature manifold space construction module estimates the instantaneous motion rate and direction of the microbubble at each pixel point of the microbubble spatiotemporal trajectory in the video space Ω1, that is, the instantaneous blood flow function characteristics. Then, the high-dimensional feature manifold space Ω2 is constructed based on the instantaneous blood flow function characteristics corresponding to the R pixel points of the microbubble spatiotemporal trajectory. The blood flow function pattern recognition module uses the pattern recognition method to classify the instantaneous blood flow function features corresponding to the R microbubble spatiotemporal trajectory pixels into K categories, i.e., K blood flow function patterns, in which the blood flow function pattern to which the instantaneous blood flow function features corresponding to each microbubble spatiotemporal trajectory pixel belong is unique; The Rainbow Ultrasound Contrast Imaging Video Reconstruction Module traces back the instantaneous blood flow function features corresponding to the R microbubble spatiotemporal trajectory pixels divided into K categories in the high-dimensional feature manifold space Ω2 to the R microbubble spatiotemporal trajectory pixels in the video space Ω1, and assigns each microbubble spatiotemporal trajectory pixel a unique color code C according to the category. i (i=1, 2, 3, ....K); then, K rainbow ultrasound contrast imaging videos Z with the same size as the original video Z0 and reflecting different blood flow function characteristics are formed. i (i=1, 2, 3, ....K); High-definition vascular structure video reconstruction module: convert rainbow ultrasound contrast video Z i The high-definition vascular structure video Z is obtained through super-resolution reconstruction algorithm * ; Vascular heterogeneity evaluation module: The number of categories K in the scanning field of view and the difference between categories are used as indicators of vascular heterogeneity.

7. An electronic device comprising: processor; as well as A memory, wherein computer program instructions are stored in the memory, and when the computer program instructions are executed by the processor, the processor executes the high-definition vascular structure reconstruction and blood flow function evaluation method as described in any one of claims 1 to 5. A computer-readable storage medium having computer program instructions stored thereon, wherein when the computer program instructions are executed by a processor, the processor executes the high-definition vascular structure reconstruction and blood flow function evaluation method as claimed in any one of claims 1 to 5.

Images