A high-definition ultrasound blood flow imaging method and system based on spatiotemporal multi-scale flexible registration

Abstract

The application discloses a high-definition ultrasonic blood flow imaging method and system based on space-time multi-scale flexible registration, which is used for removing the adverse effects of motion artifacts and realizing high-definition ultrasonic blood flow imaging. The registration method based on a multi-scale space-time similarity matrix is used to significantly improve the ultrasonic blood flow imaging quality of a moving organ. The specific method comprises the following steps: a preprocessing module M1 is responsible for mapping the collected N frames of ultrasonic echo IQ complex signals X1 frame by frame from the complex domain to the real domain, and a matrix Y1 is constructed; a similarity calculation module M2 is responsible for calculating the multi-scale space-time similarity matrix of Y1 according to different sliding window sizes C (i=1, 2, …, W-C+1, j=1, 2, …, H-C+1, k=1, 2, …, N-1); a clustering module M3 is responsible for performing multi-scale adaptive local space-time clustering and labeling the IQ complex signal at each position in X1 based on the clustering result (p=1, 2, 3, …, P); a registration module M4 is responsible for registering the IQ* complex signal corresponding to each label according to the multi-scale time direction similarity distribution characteristics and the multi-scale intra-class spatial structure characteristics; and a result output module M5 is responsible for wall filtering the multi-scale registered IQ complex signal and obtaining a high-definition ultrasonic blood flow imaging result W.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of ultrasonic blood flow imaging, in particular to a high-definition ultrasonic blood flow imaging method based on spatiotemporal multi-scale flexible registration, an electronic device, a device, a system and a computer storage medium. BACKGROUND

[0002] The vascular system is essential for the normal function of human organs, and vascular dysfunction directly or indirectly leads to various pathological conditions, causing significant morbidity and mortality. As a safe, non-invasive, portable and real-time imaging system that can provide hemodynamic parameters, ultrasonic imaging is one of the most commonly used imaging methods in clinical practice. However, in actual clinical ultrasonic imaging, the imaging area is often affected by strong motion artifacts, resulting in a significant decrease in the sensitivity of small blood vessels and serious interference with clinical judgment. In recent years, the ultrasonic localization microscope (ULM) has changed the previous impression of poor ultrasonic image quality and low blood flow sensitivity. By positioning and tracking microbubbles in the contrast mode, high-definition super-resolution blood flow images can be constructed, which can provide hemodynamic parameters of the imaging area while giving more detailed blood vessel structures. However, unfortunately, the ULM method requires the accumulation of tens of thousands of microbubble trajectory images, and has higher requirements for long-time quasi-static acquisition. The adverse effects of motion artifacts are also amplified in ULM. Therefore, how to remove or correct motion artifacts is an important and challenging challenge in the field of ultrasonic imaging.

[0003] Currently, the motion artifacts have a serious impact, and in addition to requiring patients to hold their breath for a long time during ultrasonic acquisition, they are often weakened by registration methods based on rigid affine transformation of images. However, for complex deformation of abdominal organs, these rigid registration methods often cannot correct the fine blood vessel structure, and it is also difficult to improve the quality of ultrasonic blood flow imaging. SUMMARY

[0004] To solve the above technical problems, the application discloses a high-definition ultrasonic blood flow imaging method based on spatiotemporal multi-scale flexible registration, an electronic device, a device, a system and a computer storage medium.

[0005] Specifically, the high-definition ultrasonic blood flow imaging method based on spatiotemporal multi-scale flexible registration comprises the following steps:

[0006] Step 10: N frames of ultrasonic echo IQ complex signals X1 of the imaging area affected by motion are collected; wherein the size of each image is WxH, and the size of the IQ complex signal X1 is WxHxN; preferably, N is not less than 300 frames;

[0007] Step 20: responsible for mapping the collected N frames of ultrasonic echo IQ complex signals X1 frame by frame from the complex domain to the real domain, and constructing a matrix Y1;

[0008] Step 30 is responsible for calculating the multi-scale spatio-temporal similarity matrix Y1 of Y1 according to different sliding window sizes C (i = 1, 2, …, W – C + 1, j = 1, 2, …, H - C + 1, k = 1, 2, …, N - 1); where C is the power of 2;

[0009] Step 40 is responsible for multi-scale adaptive local spatio-temporal clustering of and labeling the IQ complex signal at each position in X1 as (p = 1, 2, 3, …, P); where P is the number of classes after adaptive clustering;

[0010] Step 50 is responsible for registering the IQ complex signal corresponding to each label according to the multi-scale temporal direction similarity distribution characteristics and the multi-scale intra-class spatial structure characteristics.

[0011] Step 60 is responsible for wall filtering the multi-scale registered IQ complex signal and obtaining the high-definition ultrasound blood flow imaging result W.

[0012] The electronic device for high-definition ultrasound blood flow imaging based on spatio-temporal multi-scale flexible registration includes a processor, a memory storing executable instructions, and a storage medium; wherein the processor is used to execute the computer program corresponding to the high-definition ultrasound blood flow imaging method based on spatio-temporal multi-scale flexible registration; and the storage medium is responsible for storing the computer program corresponding to the high-definition ultrasound blood flow imaging method based on spatio-temporal multi-scale flexible registration and imaging data.

[0013] The high-definition ultrasound blood flow imaging device based on spatio-temporal multi-scale flexible registration includes: an ultrasound imaging acquisition module S1 for acquiring ultrasound echo IQ complex signals of an imaging area affected by motion; a preprocessing module M1 for constructing a three-dimensional real matrix Y1; a similarity calculation module M2 for calculating a multi-scale spatio-temporal similarity matrix (i = 1, 2, …, W – C + 1, j = 1, 2, …, H - C + 1, k = 1, 2, …, N - 1); a clustering module M3 for multi-scale adaptive local spatio-temporal clustering to determine the label corresponding to each position in X1; a registration module M4 for merging frames with the same label and registering according to multi-scale spatio-temporal characteristics; and a result output module M5 responsible for wall filtering the multi-scale registered IQ complex signal and obtaining the high-definition ultrasound blood flow imaging result W.

[0014] The high-definition ultrasound blood flow imaging system based on spatio-temporal multi-scale flexible registration comprises: an acquisition device configured to execute the first method; and an electronic device or an imaging device.

[0015] The high-definition ultrasound blood flow imaging storage medium based on spatio-temporal multi-scale flexible registration comprises: a computer program and imaging data corresponding to the high-definition ultrasound blood flow imaging based on spatio-temporal multi-scale flexible registration, and is configured to enable the electronic device to execute the high-definition ultrasound blood flow imaging method based on spatio-temporal multi-scale flexible registration. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 A schematic diagram of the high-definition ultrasound blood flow imaging method based on spatio-temporal multi-scale flexible registration is provided for an exemplary embodiment of the present application.

[0017] Figure 2 A module schematic diagram of the high-definition ultrasound blood flow imaging method based on spatio-temporal multi-scale flexible registration is provided for an exemplary embodiment of the present application.

[0018] Figure 3 An electronic device of an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present application will be described clearly and completely in the embodiments of the present application in combination with the drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0020] Figure 1 A schematic diagram of the high-definition ultrasound blood flow imaging method based on spatio-temporal multi-scale flexible registration is provided for an exemplary embodiment of the present application. The method comprises the following acquisition and calculation steps:

[0021] In step 10, N frames of ultrasound echo IQ complex signals X1 of an imaging region affected by motion are acquired; wherein each frame has a size of WxH, and the IQ complex signal X1 has a size of WxHxN.

[0022] In an embodiment of the present application, the N frames of IQ complex signals of the liver moving with the rhythm of respiration can be acquired by a multi-angle plane wave coherent compounding imaging technology and demodulated, can be acquired based on a spherical wave coherent compounding ultrasound ultrafast imaging technology and demodulated, or can be acquired based on other ultrafast ultrasound acquisition modes and demodulated. In the exemplary embodiment, 500 frames of IQ complex signals are acquired by a 7-angle plane wave coherent compounding imaging technology and demodulated; N is 500 frames.

[0023] Step 20 is responsible for mapping the collected N frames of ultrasonic echo IQ complex signals X1 frame by frame from the complex domain to the real domain to construct matrix Y1.

[0024] In the present exemplary embodiment, the complex modulus operation is performed on the 500 frames of ultrasonic echo IQ complex signals X1 frame by frame, and the real matrix Y1, i.e., the ultrasonic B-mode image sequence, is constructed.

[0025] Step 30 is responsible for calculating the multi-scale spatiotemporal similarity matrix of Y1 according to different sliding window sizes C. (i = 1, 2, …, W - C + 1, j = 1, 2, …, H - C + 1, k = 1, 2, …, N - 1); where C is a power of 2.

[0026] In the present exemplary embodiment, C is taken as 32, i.e., a 32 * 32 sliding window is first taken to calculate the cosine similarity-time sequence and form the spatiotemporal similarity matrix ; further, C is set as 16 and the same operation is performed to obtain matrix ; each time C is halved until C is 1 and the spatiotemporal similarity matrix is calculated, and the multi-scale spatiotemporal similarity matrix is obtained.

[0027] Step 40 is responsible for performing multi-scale adaptive local spatiotemporal clustering on and labeling each position of the IQ complex signal in X1 based on the clustering results as (p = 1, 2, 3, …, P); where P is the number of classes after adaptive clustering.

[0028] In the present exemplary embodiment, the spatiotemporal similarity matrix obtained under the condition that C is 32, 16, 8, 4, 2, and 1 is respectively clustered at each scale, and each position in X1 obtained at each scale is labeled as (p = 1, 2, 3, …, P).

[0029] Step 50 is responsible for registering the IQ complex signal corresponding to each label according to the multi-scale temporal direction similarity distribution characteristics and the multi-scale intra-class spatial structure characteristics.

[0030] In the present exemplary embodiment, the clustering label In each class of (p = 1, 2, 3, …, P), the displacement size and direction of each point in the region of the class are calculated frame by frame and point by point according to the local peak value distribution of the similarity in the region of the class, and the displacement and direction required for registration of each pixel at the scale are taken as the displacement and direction. After the calculation and registration of the large scale are completed, the smaller scale is subjected to more fine-grained registration, so as to complete the complex flexible registration by using the space-time features step by step and scale by scale.

[0031] Step 60, a result output module M5 is responsible for filtering the IQ complex signal after the multi-scale registration and obtaining a high-definition ultrasonic blood flow imaging result W.

[0032] In the present exemplary embodiment, the IQ complex signal after the multi-scale registration is subjected to singular value decomposition, the first 30% of the characteristic values are discarded to remove the tissue influence, and the complex modulus operation is performed on the reconstructed complex space-time matrix to obtain the corresponding high-definition ultrasonic blood flow imaging result W.

[0033] Figure 2 A module schematic diagram of the high-definition ultrasonic blood flow imaging method based on the space-time multi-scale flexible registration provided for the present exemplary embodiment is provided. The module schematic diagram comprises:

[0034] 1) An ultrasonic imaging acquisition module S1 is responsible for acquiring an imaging region affected by motion to obtain an ultrasonic echo IQ complex signal;

[0035] 2) A preprocessing module M1 is responsible for performing complex domain to real domain mapping on the N frames of ultrasonic echo IQ complex signals X1 frame by frame to construct a matrix Y1;

[0036] 3) A similarity calculation module M2 is responsible for calculating a multi-scale space-time similarity matrix of Y1 according to different sliding window sizes C (i = 1, 2, …, W – C + 1, j = 1, 2, …, H – C + 1, k = 1, 2, …, N – 1);

[0037] 4) A clustering module M3 is responsible for performing multi-scale adaptive local space-time clustering on and labeling the IQ complex signal at each position in X1 as (p = 1, 2, 3, …, P); wherein P is the number of classes after adaptive clustering;

[0038] 5) A registration module M4 is responsible for registering the IQ* complex signal corresponding to each label according to the multi-scale time direction similarity distribution characteristics and the multi-scale intra-class spatial structure characteristics;

[0039] 6) a result output module M5 responsible for wall filtering the multi-scale registered IQ complex signal and obtaining a high-definition ultrasound blood flow imaging result W;

[0040] In the following, reference will be made to Figure 3 An electronic device of an embodiment of the present application is described.

[0041] As shown, the electronic device 70 includes one or more processors 701 and a memory 702.

[0042] The processor 701 can be a central processing unit (CPU) or other form of processing unit having data processing and / or instruction executing capabilities, and can control other components in the electronic device 70 to perform desired functions.

[0043] The memory 702 can include one or more computer program products that can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory, for example, can include random access memory (RAM), cache memory, and / or the like. The non-volatile memory, for example, can include read-only memory (ROM), hard disk, flash memory, and / or the like. One or more computer program instructions can be stored on the computer-readable storage media, and the processor 701 can run the program instructions to implement the high-definition ultrasound blood flow imaging method based on spatio-temporal multi-scale flexible registration of various embodiments of the present application described above and / or other desired functions. Various contents such as the ultrasound echo IQ complex signal X1 including the motion-affected imaging region, the constructed matrix Y1, the multi-scale spatio-temporal similarity matrix , the high-definition ultrasound blood flow imaging result, and the like can also be stored in the computer-readable storage media.

[0044] In one example, the electronic device 70 can also include an input device 703 and an output device 704, which are interconnected through a bus system and / or other form of connection mechanism (not shown).

[0045] The input device 703 can include, for example, a keyboard, a mouse, and / or the like.

[0046] The output device 704 can output various information to the outside, including various contents such as the ultrasound echo IQ complex signal X1 including the motion-affected imaging region, the constructed matrix Y1, the multi-scale spatio-temporal similarity matrix , the high-definition ultrasound blood flow imaging result, and the like. The output device 704 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and / or the like.

[0047] Of course, in order to simplify, Figure 3Only some of the components of the electronic device 70 that are relevant to the present application are shown. Components such as buses, input / output interfaces, and the like are omitted. In addition, the electronic device 70 can include any other suitable components according to the specific application.

[0048] In addition to the methods and devices described above, embodiments of the present application can also be a computer program product including computer program instructions that, when run by a processor, cause the processor to perform the steps of the method of high-definition ultrasound blood flow imaging based on spatio-temporal multi-scale flexible registration according to various embodiments of the present application described above in the specification.

[0049] The computer program product can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++, etc., and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server.

[0050] In addition, embodiments of the present application can also be a computer readable storage medium having stored thereon computer program instructions that, when run by a processor, cause the processor to perform the steps of the method of high-definition ultrasound blood flow imaging based on spatio-temporal multi-scale flexible registration according to various embodiments of the present application described above in the specification.

[0051] The computer readable storage medium can be any combination of one or more non-transitory media. The non-transitory medium can be a non-transitory signal medium or a non-transitory storage medium. The non-transitory storage medium can include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above. More specific examples (a non-exhaustive list) of the non-transitory storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0052] The above describes the basic principles of the present application in combination with specific embodiments, but it needs to be pointed out that the advantages, benefits, effects and the like mentioned in the present application are only examples and are not limiting, and these advantages, benefits, effects and the like cannot be considered as necessary for each embodiment of the present application. In addition, the above specific details disclosed are only for the purpose of example and understanding, and are not limiting, and the above details do not limit the present application to be necessarily implemented with the above specific details.

[0053] The block diagrams of the devices, apparatuses, equipment, systems involved in the present application are only illustrative examples and are not intended to require or imply the connection, arrangement, configuration shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, systems can be connected, arranged, configured in any manner. Words such as "include", "contain", "have" and the like are open-ended words, which mean "including but not limited to", and can be used interchangeably, and the word "or" and "and" used herein means the word "and / or", and can be used interchangeably, unless the context clearly indicates otherwise. The word "such as" used herein means the phrase "such as but not limited to", and can be used interchangeably.

[0054] It also needs to be pointed out that in the devices, apparatuses and methods of the present application, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions of the present application.

[0055] The above description of the disclosed aspects is provided so that any person skilled in the art can make or use the present application. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but is intended to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0056] The above description has been given for the purpose of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A high-definition ultrasound blood flow imaging method based on spatio-temporal multi-scale flexible registration, characterized in that, The multiscale spatiotemporal similarity matrix of the collected ultrasonic echo original IQ complex signal is taken as a feature, and then a flexible deformation trend in the original IQ complex signal is revealed and registration is completed by using a spatiotemporal clustering strategy, and then wall filtering is performed on the registered IQ* complex signal, and finally a high-definition ultrasonic blood flow imaging result is obtained; comprising the following steps: 1) N frames of ultrasonic echo IQ complex signals X1 of an imaging region affected by motion are collected; wherein each frame of image has a size of WxH, and the IQ complex signal X1 has a size of WxHxN; 2) for X1, a complex domain to real domain mapping is performed frame by frame to construct a matrix Y1; 3) for each adjacent two frames of images in Y1, a similarity-time sequence in a preset sliding window size parameter C is obtained; wherein C is a power of 2; 4) Slide the window pixel by pixel and repeat step 3) to get the spatio-temporal similarity matrix i = 1,2,..., W - C + 1, j = 1,2,..., H - C + 1, k = 1,2,..., N - 1; 5) According to the time direction gradient feature of , adaptive local spatio-temporal clustering is performed on , and based on the clustering results, the IQ complex signal of each position in X1 is labeled as p = 1, 2, 3, …, P; P is the number of classes after adaptive local spatio-temporal clustering; 6) the sliding window size parameter C is adjusted to C / 2, and steps 3-5 are repeated until C=1, to obtain a multiscale spatiotemporal similarity matrix and a spatiotemporal clustering result; 7) the IQ* complex signal corresponding to each label is registered according to the multiscale temporal direction similarity distribution feature and the multiscale intra-class spatial structure feature; 8) the multiscale registered IQ complex signal is wall filtered to obtain a high-definition ultrasonic blood flow imaging result W.

2. The method of claim 1, wherein, For X1, a complex domain to real domain mapping is performed frame by frame to construct a matrix Y1; the complex domain to real domain mapping includes complex modulus operation, complex phase angle operation, complex real part operation or complex imaginary part operation.

3. The method of claim 1, wherein, For each adjacent two frames of images in Y1, a similarity-time sequence in a preset sliding window size parameter C is obtained; the similarity calculation method includes mean square error, cosine similarity, structural similarity, Manhattan distance, Pearson correlation coefficient, Jaccard similarity coefficient, divergence, feature similarity based on pattern recognition, and feature similarity based on deep learning.

4. The method of claim 1, wherein, According to the time direction gradient feature of , adaptive local spatio-temporal clustering is performed, and based on the clustering result, the IQ complex signal of each position in X1 is labeled as Z p , p = 1, 2, 3, …, P; the clustering method includes K-means clustering, hierarchical clustering, mean shift clustering, density-based clustering, maximum likelihood clustering, and deep learning-based clustering.

5. The method of claim 1, wherein, The IQ* complex signal corresponding to each label is registered according to the multiscale temporal direction similarity distribution feature and the multiscale intra-class spatial structure feature; The similarity distribution feature includes similarity peak value, similarity fluctuation, similarity gradient histogram, and similarity gradient texture; The spatial structure feature includes Euclidean distance, structural similarity, and deep learning-based hidden space feature.

6. The method of claim 1, wherein, The IQ* complex signal corresponding to each label is registered according to the multiscale temporal direction similarity distribution feature and the multiscale intra-class spatial structure feature; the registration method includes affine transformation, two-stage registration, multi-modal registration, optimization-based registration, pattern recognition-based registration, and deep learning-based registration.

7. The method of claim 1, wherein, The multiscale registered IQ complex signal is wall filtered to obtain a high-definition ultrasonic blood flow imaging result W; the wall filtering method includes SVD matrix decomposition, non-negative matrix decomposition, TT tensor decomposition, DMD dynamic mode decomposition, Butterworth filter, Chebyshev filter, Bessel filter, or neural operator filter; the high-definition ultrasonic blood flow imaging result can be combined with an ultrasonic super-resolution strategy to further improve the imaging result resolution; the ultrasonic super-resolution strategy includes ultrasonic positioning microscope, ultrasonic diffraction attenuation microscope, and entropy-based radial super-resolution strategy.

8. An electronic device comprising a processor and a computer storage medium storing processor-executable instructions for execution by the processor to implement the method of any one of claims 1 to 7.

9. An image forming apparatus characterized by comprising: The imaging apparatus for implementing any one of the imaging methods of claims 1 to 7 comprises: 1) a preprocessing module M1 responsible for mapping the N frames of acquired ultrasonic echo IQ complex signals X1 frame by frame from the complex domain to the real domain to construct a matrix Y1; 2) a similarity calculation module M2 responsible for calculating the multi-scale spatio-temporal similarity matrix of Y1 according to different sliding window sizes C i = 1, 2, …, W - C + 1, j = 1, 2, …, H - C + 1, k = 1, 2, …, N - 1; 3) a clustering module M3 in charge of performing a multi-scale adaptive local spatio-temporal clustering on and based on the clustering results, labelling the IQ complex signal for each position in X1 as p = 1, 2, 3,..., P; where P is the number of classes after adaptive clustering; 4) a registration module M4 responsible for registering the IQ* complex signal corresponding to each label according to the multi-scale time direction similarity distribution characteristics and the multi-scale intra-class spatial structure characteristics; 5) a result output module M5 responsible for wall filtering the multi-scale registered IQ complex signal and obtaining a high-definition ultrasonic blood flow imaging result W.

10. An imaging system characterized by, Comprise: 1) an acquisition device S1 responsible for acquiring an imaging region affected by motion to obtain ultrasonic echo IQ complex signals; 2) the electronic device of claim 8 or the imaging apparatus of claim 9.

11. A computer storage medium, characterized in that The computer storage medium is used to store processor-executable instructions so that the processor executes the imaging method of any one of the above claims 1 to 7.

Images