High-definition ultrasonic blood flow imaging method and system based on space-time multi-scale flexible registration

Abstract

The invention discloses a high-definition ultrasonic blood flow imaging method and a high-definition ultrasonic blood flow imaging system based on space-time multi-scale flexible registration, which are used for removing adverse effects of motion artifacts and realizing high-definition ultrasonic blood flow imaging, and remarkably improving the ultrasonic blood flow imaging quality of moving organs by utilizing a registration method based on a multi-scale space-time similarity matrix. The method specifically comprises the following steps: a preprocessing module M1 is responsible for mapping acquired N frames of ultrasonic echo IQ complex signals X1 frame by frame from a complex field to a real field to construct a matrix Y1; the similarity calculation module M2 is responsible for calculating a multi-scale space-time similarity matrix # imgabs0 # (i = 1, 2,..., W-C + 1, j = 1, 2,..., H-C + 1, k = 1, 2,..., N-1) of Y1 according to different sliding window sizes C; the clustering module M3 is responsible for performing multi-scale adaptive local space-time clustering on the # imgabs 1 # and labeling the IQ complex signal at each position in the X1 as # imgabs 2 # (p = 1, 2, 3,..., P) based on a clustering result; the registration module M4 is responsible for registering the IQ * complex signal corresponding to each label according to a multi-scale time direction similarity distribution feature and a multi-scale intra-class spatial structure feature; and the result output module M5 is responsible for carrying out wall filtering on the IQ complex signal after multi-scale registration and obtaining a high-definition ultrasonic blood flow imaging result W.

Description

Technical Field

[0001] The present application relates to the technical field of ultrasonic blood flow imaging, and in particular to a high-definition ultrasonic blood flow imaging method, electronic equipment, device, system and computer storage medium based on spatiotemporal multi-scale flexible registration. Background Art

[0002] The vascular system is essential for the normal function of human organs. Vascular dysfunction directly or indirectly leads to a variety of pathological conditions, causing significant morbidity and mortality. Ultrasound imaging, as a safe, non-invasive, portable real-time imaging system that can provide hemodynamic parameters, is one of the most commonly used imaging methods in clinical practice. However, in actual clinical ultrasound imaging, the imaging area is often affected by strong motion artifacts, so that the sensitivity of small blood vessels is seriously reduced, which seriously interferes with clinical judgment. In recent years, ultrasound localization microscopy (ULM) has changed people's previous impression of poor ultrasound image quality and low blood flow sensitivity. By locating and tracking microbubbles in the contrast mode to construct high-definition super-resolution blood flow images, it can provide more detailed vascular structures while providing hemodynamic parameters in the imaging area. Unfortunately, however, ULM-type methods require the accumulation of tens of thousands of frames of microbubble trajectory maps, which have higher requirements for long-term quasi-static acquisition, and the adverse effects of motion artifacts are also focused and amplified in ULM. Therefore, how to remove or correct motion artifacts is an important and thorny challenge to be solved in the field of ultrasound imaging.

[0003] At present, motion artifacts have a serious impact. In addition to requiring patients to hold their breath for a long time during ultrasound acquisition, they are often reduced through registration methods based on rigid affine transformation of images. However, for the complex deformation of abdominal organs, these rigid registration methods are often unable to correct fine vascular structures and are difficult to improve the quality of ultrasound blood flow imaging. Summary of the invention

[0004] In order to solve the above technical problems, the present invention discloses a high-definition ultrasonic blood flow imaging method, electronic equipment, device, system and computer storage medium based on spatiotemporal multi-scale flexible registration.

[0005] Specifically, the high-definition ultrasonic blood flow imaging method based on spatiotemporal multi-scale flexible registration includes the following steps:

[0006] Step 10, collecting and obtaining N frames of ultrasonic echo IQ complex signals X1 of the imaging area affected by motion; wherein the size of each frame image is W×H, and the size of the IQ complex signal X1 is W×H×N; preferably, N is not less than 300 frames;

[0007] Step 20 is responsible for mapping the collected N frames of ultrasonic echo IQ complex signals X1 from the complex domain to the real domain frame by frame to construct a matrix Y1;

[0008] Step 30 is responsible for calculating the multi-scale spatiotemporal similarity matrix of Y1 according to different sliding window sizes C (i = 1, 2, …, W – C + 1, j = 1, 2, …, H - C + 1, k = 1, 2, …, N - 1); where C is a power of 2;

[0009] Step 40, responsible for Perform multi-scale adaptive local spatiotemporal clustering and label the IQ complex signal at each position in X1 based on the clustering results (p = 1, 2, 3, …, P); where P is the number of categories after adaptive clustering;

[0010] Step 50 is responsible for registering the IQ* complex signal corresponding to each tag according to the multi-scale time-direction similarity distribution characteristics and the multi-scale intra-class spatial structure characteristics;

[0011] Step 60 is responsible for performing wall filtering on the IQ complex signal after multi-scale registration and obtaining a high-definition ultrasonic blood flow imaging result W.

[0012] The electronic device for high-definition ultrasonic blood flow imaging based on spatiotemporal multi-scale flexible registration includes a processor, a memory storing executable instructions, and a storage medium; wherein the processor is used to execute a computer program corresponding to the high-definition ultrasonic blood flow imaging method based on spatiotemporal multi-scale flexible registration; and the storage medium is responsible for storing the computer program and imaging data corresponding to the high-definition ultrasonic blood flow imaging method based on spatiotemporal multi-scale flexible registration.

[0013] The high-definition ultrasonic blood flow imaging device based on spatiotemporal multi-scale flexible registration includes: an ultrasonic imaging acquisition module S1, which is used to acquire ultrasonic echo IQ complex signals of the imaging area affected by motion; a preprocessing module M1, which is used to construct a three-dimensional real matrix Y1; and a similarity calculation module M2, which is used to calculate a multi-scale spatiotemporal similarity matrix. (i = 1, 2, …, W – C +1, j = 1, 2, …, H - C + 1, k = 1, 2, …, N - 1); clustering module M3, used for multi-scale adaptive local spatiotemporal clustering to determine the label corresponding to each position in X1; registration module M4, used to merge frames with the same label and perform registration based on multi-scale spatiotemporal features; result output module M5, responsible for wall filtering the IQ complex signal after multi-scale registration and obtaining the high-definition ultrasound blood flow imaging result W.

[0014] The high-definition ultrasonic blood flow imaging system based on spatiotemporal multi-scale flexible registration includes: an acquisition device for executing the first method; and the electronic device or imaging device.

[0015] The high-definition ultrasonic blood flow imaging storage medium based on spatiotemporal multi-scale flexible registration includes: a computer program and imaging data corresponding to the high-definition ultrasonic blood flow imaging based on spatiotemporal multi-scale flexible registration, which is used for the electronic device to execute the high-definition ultrasonic blood flow imaging method based on spatiotemporal multi-scale flexible registration. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of a high-definition ultrasound blood flow imaging method based on spatiotemporal multi-scale flexible registration provided by an exemplary embodiment of the present application;

[0017] Figure 2 A schematic diagram of a module of a high-definition ultrasound blood flow imaging method based on spatiotemporal multi-scale flexible registration provided by an exemplary embodiment of the present application;

[0018] Figure 3 This is an electronic device according to an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0020] Figure 1 The figure shows a schematic diagram of a high-definition ultrasound blood flow imaging method based on spatiotemporal multi-scale flexible registration provided by an exemplary embodiment of the present application, which includes the following acquisition and calculation steps:

[0021] Step 10, collect and obtain N frames of ultrasonic echo IQ complex signals X1 of the imaging area affected by motion; wherein the size of each frame image is W×H, and the size of the IQ complex signal X1 is W×H×N.

[0022] In one embodiment of the present application, the N frames of IQ complex signals of the liver moving with the respiratory rhythm can be acquired by multi-angle plane wave coherent composite imaging technology and demodulated; can also be acquired based on spherical wave coherent composite ultrasound ultra-fast imaging technology and demodulated; can also be acquired based on other ultra-fast ultrasound acquisition methods and demodulated. In this exemplary embodiment, 7-angle plane wave coherent composite imaging technology is used to acquire and demodulate 500 frames of IQ complex signals; N is 500 frames.

[0023] Step 20 is responsible for mapping the collected N frames of ultrasonic echo IQ complex signals X1 from the complex domain to the real domain frame by frame to construct a matrix Y1.

[0024] In this exemplary embodiment, complex analog operation is performed frame by frame on 500 frames of ultrasound echo IQ complex signal X1 to construct a real matrix Y1, ie, an ultrasound B-mode image sequence.

[0025] Step 30 is responsible for calculating the multi-scale spatiotemporal similarity matrix of Y1 according to different sliding window sizes C (i = 1, 2, …, W – C + 1, j = 1, 2, …, H - C + 1, k = 1, 2, …, N - 1); where C is a power of 2.

[0026] In this exemplary embodiment, C is 32, that is, first a sliding window of 32*32 is taken with a step size of 1 to calculate the cosine similarity-time series window by window and form a spatiotemporal similarity matrix ; Then, set C to 16 and do the same operation to get the matrix ; Each time C is halved until C is 1 and the spatiotemporal similarity matrix is ​​calculated to obtain the multi-scale spatiotemporal similarity matrix .

[0027] Step 40, responsible for Perform multi-scale adaptive local spatiotemporal clustering and label the IQ complex signal at each position in X1 based on the clustering results (p = 1, 2, 3, …, P); where P is the number of categories after adaptive clustering;

[0028] In this exemplary embodiment, adaptive clustering is performed scale by scale for the spatiotemporal similarity matrices obtained when C is 32, 16, 8, 4, 2, and 1, and each position in X1 obtained at each scale is labeled as (p = 1, 2,3, …, P).

[0029] Step 50 is responsible for registering the IQ* complex signal corresponding to each tag according to the multi-scale time-direction similarity distribution characteristics and the multi-scale intra-class spatial structure characteristics;

[0030] In this exemplary embodiment, the cluster labels corresponding to each scale are In each class (p = 1, 2, 3, …, P), according to the local peak distribution of similarity in the class area, the displacement size and direction of each point in the class area and the peak are calculated frame by frame, and this is used as the displacement and direction required for registration of each pixel at this scale. After completing the calculation and registration of the large scale, a more fine-grained registration is performed on the smaller scale, so as to complete complex flexible registration by using spatiotemporal features level by level and scale by scale.

[0031] Step 60, the result output module M5 is responsible for performing wall filtering on the IQ complex signal after multi-scale registration and obtaining a high-definition ultrasonic blood flow imaging result W.

[0032] In this exemplary embodiment, singular value decomposition is performed on the IQ complex signal after multi-scale registration, the first 30% of eigenvalues ​​are discarded to remove tissue influence, and complex modulus operation is performed on the reconstructed complex space-time matrix to obtain the corresponding high-definition ultrasonic blood flow imaging result W.

[0033] Figure 2 A schematic diagram of a module of a high-definition ultrasound blood flow imaging method based on spatiotemporal multi-scale flexible registration provided by an exemplary embodiment of the present application. It includes:

[0034] 1) Ultrasonic imaging acquisition module S1, responsible for acquiring the imaging area affected by motion and obtaining ultrasonic echo IQ complex signals;

[0035] 2) The preprocessing module M1 is responsible for mapping the collected N frames of ultrasonic echo IQ complex signals X1 from the complex domain to the real domain frame by frame to construct the matrix Y1;

[0036] 3) Similarity calculation module M2 is responsible for calculating the multi-scale spatiotemporal similarity matrix of Y1 according to different sliding window sizes C (i = 1, 2, …, W – C + 1, j = 1, 2, …, H – C + 1, k = 1, 2, …, N -1);

[0037] 4) Clustering module M3, responsible for Perform multi-scale adaptive local spatiotemporal clustering and label the IQ complex signal at each position in X1 based on the clustering results (p = 1, 2, 3, …, P); where P is the number of categories after adaptive clustering;

[0038] 5) Registration module M4, responsible for registering the IQ* complex signal corresponding to each label according to the multi-scale time-direction similarity distribution characteristics and the multi-scale intra-class spatial structure characteristics;

[0039] 6) Result output module M5, responsible for performing wall filtering on the IQ complex signal after multi-scale registration and obtaining the high-definition ultrasonic blood flow imaging result W;

[0040] Below, reference Figure 3 An electronic device according to an embodiment of the present application is described.

[0041] As shown, the electronic device 70 includes one or more processors 701 and a memory 702.

[0042] The processor 701 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 70 to perform desired functions.

[0043] The memory 702 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or highly buffered memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 701 may run the program instructions to implement the high-definition ultrasonic blood flow imaging method based on spatiotemporal multi-scale flexible registration of the various embodiments of the present application described above and / or other desired functions. The computer storable medium may also store information such as an ultrasonic echo IQ complex signal X1 including an imaging area affected by motion, a constructed matrix Y1, a multi-scale spatiotemporal similarity matrix , high-definition ultrasound blood flow imaging results, and other contents.

[0044] In one example, the electronic device 70 may further include: an input device 703 and an output device 704 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0045] The input device 703 may include, for example, a keyboard, a mouse, etc.

[0046] The output device 704 can output various information to the outside, including the ultrasound echo IQ complex signal X1 of the imaging area affected by the motion, the constructed matrix Y1, the multi-scale spatiotemporal similarity matrix , high-definition ultrasound blood flow imaging results, etc. The output device 704 may include, for example, a display, a speaker, a printer, a communication network and its connected remote output device, etc.

[0047] Of course, to simplify, Figure 3Only some of the components related to the present application in the electronic device 70 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application conditions, the electronic device 70 may also include any other appropriate components.

[0048] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the high-definition ultrasound blood flow imaging method based on spatiotemporal multi-scale flexible registration according to various embodiments of the present application described above in this specification.

[0049] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0050] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the high-definition ultrasound blood flow imaging method based on spatiotemporal multi-scale flexible registration according to various embodiments of the present application described above in this specification.

[0051] The computer-readable storage medium may adopt any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM, a read-only memory (ROM, an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0052] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0053] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably therewith, and the words "or" and "and" used here refer to the words "and / or", and can be used interchangeably therewith, unless the context clearly indicates otherwise, and the words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably therewith.

[0054] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0055] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0056] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A high-definition ultrasound blood flow imaging method and system based on spatiotemporal multi-scale flexible registration, characterized in that: The multi-scale spatiotemporal similarity matrix of the collected original IQ complex signal of the ultrasound echo is used as a feature, and then the spatiotemporal clustering strategy is used to reveal the flexible deformation trend in the original IQ complex signal and complete the registration, and then the registered IQ* complex signal is wall filtered to finally obtain the high-definition ultrasound blood flow imaging result; the following steps are included: 1) Acquire N frames of ultrasonic echo IQ complex signals X1 of the imaging area affected by motion; wherein the size of each frame image is W×H, and the size of the IQ complex signal X1 is W×H×N; 2) For X1, the matrix Y1 is constructed by mapping the complex domain to the real domain frame by frame; 3) For every two adjacent frames of images in Y1, obtain the similarity-time series within the window according to the preset sliding window size parameter C, where C is a power of 2; 4) Slide the window pixel by pixel and repeat step 3) to obtain the spatiotemporal similarity matrix (i = 1, 2, …, W –C + 1, j = 1, 2, …, H – C + 1, k = 1, 2, …, N – 1); 5) According to The temporal gradient characteristics of Perform adaptive local spatiotemporal clustering, and label the IQ complex signal at each position in X1 based on the clustering results as (p = 1, 2, 3, …, P); P is the number of categories after adaptive local spatiotemporal clustering; 6) Adjust the sliding window size parameter C to C / 2, and repeat steps 3)-5) until C = 1, and obtain the multi-scale spatiotemporal similarity matrix and spatiotemporal clustering results; 7) Align the IQ* complex signals corresponding to each label according to the multi-scale time-direction similarity distribution characteristics and the multi-scale intra-class spatial structure characteristics; 8) Perform wall filtering on the IQ complex signal after multi-scale registration and obtain the high-definition ultrasound blood flow imaging result W.

2. A high-definition ultrasonic blood flow imaging method according to claim 1, characterized in that: For X1, a matrix Y1 is constructed by mapping from the complex domain to the real domain frame by frame; the mapping from the complex domain to the real domain includes complex modulus operation, complex phase angle operation, complex real part operation or complex imaginary part operation.

3. The high-definition ultrasonic blood flow imaging method according to claim 1, characterized in that: For every two adjacent frames of images in Y1, the similarity-time series within the window is obtained according to the preset sliding window size parameter C; the similarity calculation methods include mean square error, cosine similarity, structural similarity, Manhattan distance, Pearson correlation coefficient, Jaccard similarity coefficient, divergence, feature similarity based on pattern recognition, and feature similarity based on deep learning.

4. The high-definition ultrasonic blood flow imaging method according to claim 1, characterized in that: according to The temporal gradient characteristics of Perform adaptive local spatiotemporal clustering and label the IQ complex signal at each position in X1 as Z based on the clustering results p (p = 1,2,3,…, P); the clustering methods include K-means clustering, hierarchical clustering, mean shift clustering, density-based clustering, maximum expectation clustering, and deep learning-based clustering.

5. The high-definition ultrasonic blood flow imaging method according to claim 1, characterized in that: The IQ* complex signal corresponding to each label is registered according to the multi-scale time-direction similarity distribution characteristics and the multi-scale intra-class spatial structure characteristics; The similarity distribution features include similarity peak, similarity fluctuation, similarity gradient histogram, and similarity gradient texture; The spatial structural features include Euclidean distance, structural similarity, and latent space features based on deep learning.

6. The high-definition ultrasonic blood flow imaging method according to claim 1, characterized in that: The IQ* complex signal corresponding to each label is registered according to the multi-scale time-direction similarity distribution characteristics and the multi-scale intra-class spatial structure characteristics; the registration methods include affine transformation, two-stage registration, multimodal registration, optimization-based registration, pattern recognition-based registration, and deep learning-based registration.

7. The high-definition ultrasonic blood flow imaging method according to claim 1, characterized in that: The IQ complex signal after multi-scale registration is wall filtered to obtain a high-definition ultrasonic blood flow imaging result W; the wall filtering method includes SVD matrix decomposition, non-negative matrix decomposition, TT tensor decomposition, DMD dynamic mode decomposition, Butterworth filter, Chebyshev filter, Bessel filter or neural operator filter; the high-definition ultrasonic blood flow imaging result can be combined with an ultrasonic super-resolution strategy to further improve the resolution of the imaging result; the so-called ultrasonic super-resolution strategy includes an ultrasonic localization microscope, an ultrasonic diffraction attenuation microscope, and an entropy-based radial super-resolution strategy.

8. An electronic device, comprising a processor and a computer storage medium, wherein the computer storage medium stores instructions executable by the processor, and the processor is configured to execute the instructions to implement the method according to any one of claims 1 to 7.

9. An imaging device, characterized in that: The imaging device is used to implement the imaging method of claims 1 to 8, comprising: 1) The preprocessing module M1 is responsible for mapping the collected N frames of ultrasonic echo IQ complex signals X1 from the complex domain to the real domain frame by frame to construct the matrix Y1; 2) Similarity calculation module M2 is responsible for calculating the multi-scale spatiotemporal similarity matrix of Y1 according to different sliding window sizes C (i = 1, 2, …, W – C + 1, j = 1, 2, …, H – C + 1, k = 1, 2, …, N – 1); 3) Clustering module M3, responsible for Perform multi-scale adaptive local spatiotemporal clustering and label the IQ complex signal at each position in X1 based on the clustering results (p = 1, 2, 3, …, P); where P is the number of categories after adaptive clustering; 4) Registration module M4, responsible for registering the IQ* complex signal corresponding to each label according to the multi-scale time-direction similarity distribution characteristics and the multi-scale intra-class spatial structure characteristics; 5) The result output module M5 is responsible for performing wall filtering on the IQ complex signal after multi-scale registration and obtaining the high-definition ultrasonic blood flow imaging result W.

10. An imaging system, characterized in that: include: 1) The acquisition device S1 is responsible for acquiring the imaging area affected by the motion and obtaining the ultrasonic echo IQ complex signal; 2) The electronic device as claimed in claim 8 or the imaging device as claimed in claim 9.

11. A computer storage medium, characterized in that: The computer storage medium is used to store processor-executable instructions so that the processor executes the imaging method according to any one of claims 1 to 7.

Images