Cross-reference method and system for two-dimensional (2D) ultrasound scanning of tissue volumes

By calculating the corresponding transformation matrix of the target anatomical structure, cross-reference of two-dimensional ultrasound images is achieved, solving the problem that two-dimensional ultrasound imaging is difficult to obtain high-resolution three-dimensional anatomical structure in the prior art, and improving diagnostic efficiency and accuracy.

CN118985018APending Publication Date: 2024-11-19EXO IMAGING INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380030446.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-03-25
Filing Date
2023-03-02
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

In the prior art, two-dimensional ultrasound imaging is difficult to effectively obtain high-resolution three-dimensional anatomy from orthogonal views, resulting in time-consuming manual registration and diagnostic variability and inconsistency.

Method used

By generating multiple 2D ultrasound images associated with the orthogonal scanning direction, the corresponding transformation matrix of the target anatomy is calculated, and the matching and correspondence relationship determination between images is achieved, thereby automatically crossing the points in different views.

Benefits of technology

The cross-reference efficiency of two-dimensional ultrasound scanning is improved, the variability of diagnosis and manual operation time is reduced, and the matching accuracy of three-dimensional anatomy is enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118985018A_ABST
    Figure CN118985018A_ABST
Patent Text Reader

Abstract

A cross-referencing method of 2D ultrasound scanning of a tissue volume includes generating first and second 2D representations of a target anatomical structure from first and second series of 2D ultrasound images of the generated tissue volume, respectively. The method includes generating first and second simulated 2D representations of the target anatomical structure from the second and first series of 2D ultrasound images, respectively. The 2D and simulated 2D representations are processed to at least substantially match pixel locations in the 2D and simulated 2D representations associated with the target anatomical structure. The method includes determining first and second corresponding transformation matrices from the processed 2D and simulated 2D representations, and determining corresponding locations in the first series of 2D ultrasound images for locations in the second series of 2D ultrasound images associated with the target anatomical structure using the first and second corresponding transformation matrices.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Related Applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 323,515, filed on March 25, 2022, entitled “Method and System for Cross-Referencing of Two-Dimensional (2D) Ultrasound Scans of a Tissue Volume,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present invention generally relates to a method and system for cross-referencing a two-dimensional (2D) ultrasound scan of a tissue volume, and more particularly, to a method and system for cross-referencing two 2D ultrasound image scans of a tissue volume from orthogonal or substantially orthogonal views or planes. Background Art

[0004] Two-dimensional (2D) ultrasound (US) imaging is safe, inexpensive, and widely used in medical practice, and has real-time and high-resolution capabilities. Conventional 2D ultrasound imaging techniques can be used to extract 2D ultrasound images (which may also be called cross-sectional images, image planes / frames, or B-mode / B-scan images) of anatomical structures or tissue volumes scanned by an ultrasound probe.

[0005] However, conventional 2D ultrasound imaging technology has the inherent limitation of relying on 2D images to describe three-dimensional (3D) anatomical structures. The operator can usually manually operate (move) the transducer to obtain a 2D image (or a series of 2D images, also called a 2D scan or freehand scan) of the body organs to cover the 3D anatomical structure.

[0006] In 3D imaging modalities such as magnetic resonance imaging (MRI) or computed tomography (CT), images can be acquired with a known 3D pixel spacing that enables multiplanar reconstruction imaging: the scan data can be visualized transversely (or axially), coronally, or sagittally, depending on the diagnostic task. However, in freehand ultrasound, only the pixel spacing in the imaging plane (xy) is available, while the pixel spacing in the out-of-plane direction (z-direction) may not be available. Therefore, if an ultrasound scan is acquired in the transverse plane, a simple projection of the image in a sagittal view may result in a poor quality and low-resolution sagittal scan that may not be useful for diagnostic imaging.

[0007] An inherent limitation of ultrasound imaging is that it may have high resolution in the imaging plane (high axial and lateral resolution), but not out of plane. Therefore, in standard clinical assessments, two 2D ultrasound image scans are usually acquired from orthogonal directions to better capture the 3D anatomy at high resolution. Although it may help to acquire the image, matching the two views and finding corresponding points in the other view is a challenging problem because freehand scanning is done manually and there are no reference points for matching the images. This problem is also known as the two view registration problem in the field of computer vision and medical imaging.

[0008] In a clinical setting, images from both views are typically cross-referenced (or registered) in the mind of a professional radiologist / sonographer to form a subjective impression of 3D anatomy and / or pathology. This practice is time-consuming, inefficient, and places a high cognitive load on the radiologist, and often results in variability and inconsistency in measurements and diagnoses.

[0009] Without loss of generality, the following will refer to Figure 4 The content on thyroid ultrasound examination further illustrates the above issue.

[0010] Figure 4 Schematic diagrams of the (human) thyroid gland (labeled "T") are shown in Figures (a) and (b). The arrows in Figures (a) and (b) indicate the scanning directions of the ultrasound probe in the transverse (TRX) and sagittal (SAG) directions, respectively. Figures (c) and (d) are sample TRX and SAG thyroid ultrasound scans with manually marked thyroid lobe borders superimposed (contours labeled "C1" and "C2").

[0011] As part of routine thyroid ultrasound workflow, clinicians can scan the left and right lobes of the thyroid gland sequentially. Figure 4 The left lobe (shown) scan may include a transverse (TRX) plane scan (such as Figure 4 (a)) and sagittal (SAG) plane scan (as shown Figure 4 (b)). For example, each side can be scanned first in the transverse (TRX) plane and then in the sagittal (SAG) plane.

[0012] The clinician can then review the patient's scan, which contains both a transverse (TRX) scan and a sagittal (SAG) scan. The transverse image (e.g. Figure 4 (c) to see if a nodule or suspected lesion is present. When a nodule is found, it is often necessary to manually locate each nodule in the corresponding image in the sagittal view (e.g. Figure 4(d) to examine the nodule in both views. To do this, the clinician may have to scroll through all frames of the SAG scan to locate the same nodule. Regions of interest (ROIs) or landmarks can then be annotated and measured on both views and recorded for later review. This task is cognitively demanding and time consuming.

[0013] Therefore, there is a need to provide a two-dimensional (2D) ultrasound scanning cross-referencing method and system for tissue volumes that seeks to overcome or at least ameliorate one or more deficiencies associated with conventional methods and systems, particularly with respect to manual ultrasound scanning of tissue volumes. The present invention was developed in this context. Summary of the invention

[0014] In a first aspect of the present invention, there is provided a cross-referencing method for two-dimensional (2D) ultrasound scanning of a tissue volume, the method comprising:

[0015] generating a first series of 2D ultrasound images of the tissue volume associated with a first plurality of locations, respectively, along a first scanning direction of the tissue volume, the tissue volume including the target anatomical structure;

[0016] generating a second series of 2D ultrasound images of the tissue volume respectively associated with the second plurality of locations along a second scanning direction of the tissue volume, wherein the second scanning direction is at least substantially orthogonal to the first scanning direction;

[0017] generating a first 2D representation of the target anatomical structure from the first series of 2D ultrasound images, the first 2D representation being associated with one of the images within the first series of 2D ultrasound images;

[0018] generating a first simulated 2D representation of the target anatomical structure from the second series of 2D ultrasound images, the first simulated 2D representation of the target anatomical structure and the first 2D representation corresponding to each other in a direction in which the target anatomical structure is viewed;

[0019] processing the first 2D representation and the first simulated 2D representation of the target anatomical structure so that pixel locations associated with the target anatomical structure in the first simulated 2D representation at least substantially match pixel locations associated with the target anatomical structure in the first 2D representation;

[0020] determining a first corresponding transformation matrix based on the processed first 2D representation of the target anatomical structure and the processed first simulated 2D representation;

[0021] generating a second 2D representation of the target anatomical structure from the second series of 2D ultrasound images, the second 2D representation being associated with one of the images within the second series of 2D ultrasound images;

[0022] generating a second simulated 2D representation of the target anatomical structure from the first series of 2D ultrasound images, the second simulated 2D representation of the target anatomical structure and the second 2D representation corresponding to each other in a direction in which the target anatomical structure is viewed;

[0023] processing the second 2D representation and the second simulated 2D representation of the target anatomical structure so that pixel locations associated with the target anatomical structure in the second simulated 2D representation at least substantially match pixel locations associated with the target anatomical structure in the second 2D representation;

[0024] determining a second corresponding transformation matrix based on the processed second 2D representation of the target anatomical structure and the processed second simulated 2D representation; and

[0025] Corresponding second locations associated with the target anatomical structure in the first series of 2D ultrasound images are determined for the first locations associated with the target anatomical structure in the second series of 2D ultrasound images using the first corresponding transformation matrix and the second corresponding transformation matrix.

[0026] In various embodiments, determining the second position using the first corresponding transformation matrix and the second corresponding transformation matrix includes:

[0027] A first function of the first corresponding transformation matrix and the second corresponding transformation matrix is ​​applied to the pixel coordinates corresponding to the first position to obtain the pixel coordinates corresponding to the second position.

[0028] In various embodiments, the method further comprises:

[0029] A corresponding fourth position associated with the target anatomical structure in the second series of 2D ultrasound images is determined for the third position associated with the target anatomical structure in the first series of 2D ultrasound images using the first corresponding transformation matrix and the second corresponding transformation matrix.

[0030] In various embodiments, determining the fourth position using the first corresponding transformation matrix and the second corresponding transformation matrix includes:

[0031] A second function of the first corresponding transformation matrix and the second corresponding transformation matrix is ​​applied to the pixel coordinates corresponding to the third position to obtain pixel coordinates corresponding to the fourth position.

[0032] In various embodiments, the method further comprises:

[0033] generating a first plurality of binary segmentation masks from the first series of 2D ultrasound images, each binary segmentation mask of the first plurality of binary segmentation masks corresponding to a respective one of the 2D ultrasound images within the first series of 2D ultrasound images; and

[0034] generating a second plurality of binary segmentation masks from the second series of 2D ultrasound images, each binary segmentation mask in the second plurality of binary segmentation masks corresponding to a respective one of the 2D ultrasound images within the second series of 2D ultrasound images,

[0035] wherein a first 2D representation of the target anatomical structure is generated from one of the binary segmentation masks within the first plurality of binary segmentation masks, and

[0036] Therein, a second 2D representation of the target anatomical structure is generated from one of the binary segmentation masks within the second plurality of binary segmentation masks.

[0037] In various embodiments, the first 2D representation of the target anatomical structure corresponds to a maximum binary segmentation mask of the first plurality of binary segmentation masks, and the second 2D representation of the target anatomical structure corresponds to a maximum binary segmentation mask of the second plurality of binary segmentation masks.

[0038] In various embodiments, generating a first simulated 2D representation of the target anatomical structure from the second series of 2D ultrasound images includes generating the first simulated 2D representation of the target anatomical structure from a second plurality of binary segmentation masks, and generating a second simulated 2D representation of the target anatomical structure from the first series of 2D ultrasound images includes generating the second simulated 2D representation of the target anatomical structure from the first plurality of binary segmentation masks.

[0039] In various embodiments, processing the first 2D representation includes generating a first signed distance map from the first 2D representation, and processing the second 2D representation includes generating a second signed distance map from the second 2D representation.

[0040] In various embodiments, processing the first simulated 2D representation includes scaling a width of the first simulated 2D representation to match a width of the first 2D representation, and processing the second simulated 2D representation includes scaling a width of the second simulated 2D representation to match a width of the second 2D representation.

[0041] In various embodiments, processing the first simulated 2D representation also includes determining and sampling contour pixels associated with the contour of the target anatomical structure through the scaled first simulated 2D representation, and processing the second simulated 2D representation also includes determining and sampling contour pixels associated with the contour of the target anatomical structure through the scaled second simulated 2D representation.

[0042] In various embodiments, processing the first simulated 2D representation further includes sampling a first subset of pixels from the scaled first simulated 2D representation, and processing the second simulated 2D representation further includes sampling a second subset of pixels from the scaled second simulated 2D representation.

[0043] In various embodiments, processing the first analog 2D representation further includes estimating a coarse offset between the scaled first analog 2D representation and the first 2D representation via a first signed distance map and a sampled first subset of pixels, and processing the second analog 2D representation further includes estimating a coarse offset between the scaled second analog 2D representation and the second 2D representation via a second signed distance map and a sampled second subset of pixels.

[0044] In various embodiments, determining a first corresponding transformation matrix includes estimating the scale and translation between the scaled first simulated 2D representation and the first 2D representation by minimizing a first alignment cost defined as a first cost function with respect to the first corresponding transformation matrix; and determining a second corresponding transformation matrix includes estimating the scale and translation between the scaled second simulated 2D representation and the second 2D representation by minimizing a second alignment cost defined as a second cost function with respect to the second corresponding transformation matrix.

[0045] In various embodiments, the first scanning direction corresponds to a transverse scan of the tissue volume and the second scanning direction corresponds to a sagittal scan of the tissue volume.

[0046] In a second aspect of the present invention, there is provided a cross-referencing system for 2D ultrasound scanning of a tissue volume, the system comprising:

[0047] Memory; and

[0048] at least one processor communicatively coupled to the memory and configured to:

[0049] generating a first series of 2D ultrasound images of the tissue volume associated with a first plurality of locations, respectively, along a first scanning direction of the tissue volume, the tissue volume including the target anatomical structure;

[0050] generating a second series of 2D ultrasound images of the tissue volume respectively associated with the second plurality of locations along a second scanning direction of the tissue volume, wherein the second scanning direction is at least substantially orthogonal to the first scanning direction;

[0051] generating a first 2D representation of the target anatomical structure from the first series of 2D ultrasound images, the first 2D representation being associated with one of the images within the first series of 2D ultrasound images;

[0052] generating a first simulated 2D representation of the target anatomical structure from the second series of 2D ultrasound images, the first simulated 2D representation of the target anatomical structure and the first 2D representation corresponding to each other in a direction in which the target anatomical structure is viewed;

[0053] processing the first 2D representation and the first simulated 2D representation of the target anatomical structure so that pixel locations associated with the target anatomical structure in the first simulated 2D representation at least substantially match pixel locations associated with the target anatomical structure in the first 2D representation;

[0054] determining a first corresponding transformation matrix based on the processed first 2D representation of the target anatomical structure and the processed first simulated 2D representation;

[0055] generating a second 2D representation of the target anatomical structure from the second series of 2D ultrasound images, the second 2D representation being associated with one of the images within the second series of 2D ultrasound images;

[0056] generating a second simulated 2D representation of the target anatomical structure from the first series of 2D ultrasound images, the second simulated 2D representation of the target anatomical structure and the second 2D representation corresponding to each other in a direction in which the target anatomical structure is viewed;

[0057] processing the second 2D representation and the second simulated 2D representation of the target anatomical structure so that pixel locations associated with the target anatomical structure in the second simulated 2D representation at least substantially match pixel locations associated with the target anatomical structure in the second 2D representation;

[0058] determining a second corresponding transformation matrix based on the processed second 2D representation of the target anatomical structure and the processed second simulated 2D representation; and

[0059] Corresponding second locations associated with the target anatomical structure in the first series of 2D ultrasound images are determined for the first locations associated with the target anatomical structure in the second series of 2D ultrasound images using the first corresponding transformation matrix and the second corresponding transformation matrix.

[0060] In various embodiments, when the at least one processor determines the second position using the first corresponding transformation matrix and the second corresponding transformation matrix, the at least one processor is configured to:

[0061] A first function of the first corresponding transformation matrix and the second corresponding transformation matrix is ​​applied to the pixel coordinates corresponding to the first position to obtain the pixel coordinates corresponding to the second position.

[0062] In various embodiments, the at least one processor is further configured to:

[0063] A corresponding fourth position associated with the target anatomical structure in the second series of 2D ultrasound images is determined for the third position associated with the target anatomical structure in the first series of 2D ultrasound images using the first corresponding transformation matrix and the second corresponding transformation matrix.

[0064] In various embodiments, when the at least one processor determines the fourth position using the first corresponding transformation matrix and the second corresponding transformation matrix, the at least one processor is configured to:

[0065] A second function of the first corresponding transformation matrix and the second corresponding transformation matrix is ​​applied to the pixel coordinates corresponding to the third position to obtain pixel coordinates corresponding to the fourth position.

[0066] In various embodiments, at least one processor is further configured to:

[0067] generating a first plurality of binary segmentation masks from the first series of 2D ultrasound images, each binary segmentation mask of the first plurality of binary segmentation masks corresponding to a respective one of the 2D ultrasound images within the first series of 2D ultrasound images; and

[0068] generating a second plurality of binary segmentation masks from the second series of 2D ultrasound images, each binary segmentation mask in the second plurality of binary segmentation masks corresponding to a respective one of the 2D ultrasound images within the second series of 2D ultrasound images;

[0069] wherein a first 2D representation of the target anatomical structure is generated from one of the binary segmentation masks within the first plurality of binary segmentation masks, and

[0070] Therein, a second 2D representation of the target anatomical structure is generated from one of the binary segmentation masks within the second plurality of binary segmentation masks.

[0071] In various embodiments, the first 2D representation of the target anatomical structure corresponds to a maximum binary segmentation mask of the first plurality of binary segmentation masks, and the second 2D representation of the target anatomical structure corresponds to a maximum binary segmentation mask of the second plurality of binary segmentation masks.

[0072] In various embodiments, the first scanning direction corresponds to a transverse scan of the tissue volume and the second scanning direction corresponds to a sagittal scan of the tissue volume.

[0073] In various embodiments, the system also includes an ultrasound transducer that is communicatively coupled to the memory and at least one processor, wherein the at least one processor is configured to: generate a first series of 2D ultrasound images based on a first series of ultrasound waves acquired by the ultrasound transducer positioned at a first plurality of positions with respect to a first plurality of time instances; and generate a second series of 2D ultrasound images based on a second series of ultrasound waves acquired by the ultrasound transducer positioned at a second plurality of positions with respect to a second plurality of time instances.

[0074] In a third aspect of the present invention, there is provided a computer program product, contained in one or more non-transitory computer-readable storage media, comprising instructions executable by at least one processor to implement a cross-referencing method for 2D ultrasound scanning of a tissue volume, the method comprising:

[0075] generating a first series of 2D ultrasound images of the tissue volume associated with a first plurality of locations, respectively, along a first scanning direction of the tissue volume, the tissue volume including the target anatomical structure;

[0076] generating a second series of 2D ultrasound images of the tissue volume respectively associated with the second plurality of locations along a second scanning direction of the tissue volume, wherein the second scanning direction is at least substantially orthogonal to the first scanning direction;

[0077] generating a first 2D representation of the target anatomical structure from the first series of 2D ultrasound images, the first 2D representation being associated with one of the images within the first series of 2D ultrasound images;

[0078] generating a first simulated 2D representation of the target anatomical structure from the second series of 2D ultrasound images, the first simulated 2D representation of the target anatomical structure and the first 2D representation corresponding to each other in a direction in which the target anatomical structure is viewed;

[0079] processing the first 2D representation and the first simulated 2D representation of the target anatomical structure so that pixel locations associated with the target anatomical structure in the first simulated 2D representation at least substantially match pixel locations associated with the target anatomical structure in the first 2D representation;

[0080] determining a first corresponding transformation matrix based on the processed first 2D representation of the target anatomical structure and the processed first simulated 2D representation;

[0081] generating a second 2D representation of the target anatomical structure from the second series of 2D ultrasound images, the second 2D representation being associated with one of the images within the second series of 2D ultrasound images;

[0082] generating a second simulated 2D representation of the target anatomical structure from the first series of 2D ultrasound images, the second simulated 2D representation of the target anatomical structure and the second 2D representation corresponding to each other in a direction in which the target anatomical structure is viewed;

[0083] processing the second 2D representation and the second simulated 2D representation of the target anatomical structure so that pixel locations associated with the target anatomical structure in the second simulated 2D representation at least substantially match pixel locations associated with the target anatomical structure in the second 2D representation;

[0084] determining a second corresponding transformation matrix based on the processed second 2D representation of the target anatomical structure and the processed second simulated 2D representation; and

[0085] Corresponding second locations associated with the target anatomical structure in the first series of 2D ultrasound images are determined for the first locations associated with the target anatomical structure in the second series of 2D ultrasound images using the first corresponding transformation matrix and the second corresponding transformation matrix.

[0086] Note that the various embodiments described above can be combined with any other embodiments described herein. The features and advantages described in the specification are not all inclusive, and in particular, many additional features and advantages will be apparent to those of ordinary skill in the art in view of the drawings, the specification, and the claims. In addition, it should be noted that the language used in the specification is primarily selected for readability and teaching purposes, and is not intended to describe or limit the subject matter of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Embodiments of the present invention will be better understood and apparent to those of ordinary skill in the art through the following written description which is given by way of example only in conjunction with the accompanying drawings.

[0088] Figure 1 Depicted is a schematic flow chart of a cross-referencing method for 2D ultrasound scanning of a tissue volume according to various embodiments of the present invention.

[0089] Figure 2 A schematic block diagram of a cross-referencing system for 2D ultrasound scanning of a tissue volume according to various embodiments of the present invention is depicted, for example corresponding to Figure 1 The method described in .

[0090] Figure 3 Depicted is a schematic block diagram of an exemplary computer system that may be used to implement or practice a cross-referencing system for 2D ultrasound scanning of a tissue volume according to various embodiments of the present invention, such as Figure 2 The system depicted in .

[0091] Figure 4 Depicted is a schematic diagram of the human thyroid gland, along with sample transverse and sagittal ultrasound scan images of the thyroid gland.

[0092] Figure 5 Depicted are flow charts illustrating aspects of a cross-referencing method for 2D ultrasound scanning of a tissue volume according to various embodiments of the present invention.

[0093] Figure 6 Another flow chart illustrating aspects of a cross-referencing method of 2D ultrasound scanning of a tissue volume according to various embodiments of the present invention is depicted.

[0094] Figure 7 Shown in more detail Figure 6 A portion of the flowchart is shown.

[0095] Figure 8 Shown in more detail Figure 6 Another part of the flowchart shown.

[0096] Fig. 9 Screen shots depicting a cross-referencing tool or system from a first view to a second view according to various embodiments of the present invention.

[0097] Fig.10 Screen shots depicting a cross-referencing system from a second view to a first view according to various embodiments of the invention. DETAILED DESCRIPTION

[0098] Various embodiments of the present invention provide a cross-referencing method (computer-implemented method) and system (including a memory and at least one processor communicatively coupled to the memory) of a 2D ultrasound scan of a tissue volume, and more specifically, provide a cross-referencing method and system of two 2D ultrasound image scans of a tissue volume from orthogonal or substantially orthogonal views or planes.

[0099] As described in the background, two-dimensional (2D) ultrasound imaging is safe, inexpensive, and widely used in medical practice. However, in traditional ultrasound imaging, the registration (or cross-referencing) of images obtained from two different angles is done in the head by the radiologist / sonographer to form a subjective impression of the 3D anatomical structure and / or pathology, which is time-consuming and inefficient, and imposes a high cognitive load on the radiologist, and often leads to variable and erroneous diagnoses.

[0100] Thus, various embodiments of the present invention provide for more efficient and / or reliable cross-referencing (registration) of two 2D ultrasound scans of a tissue volume from two different views. Cross-referencing can find corresponding points from one plane to another. In other words, cross-referencing can map a given point on one plane or view (e.g., a transverse (TRX) view) to a corresponding point on another plane or view (e.g., a sagittal (SAG) view).

[0101] Thus, various embodiments of the present invention can bridge the gap between 2D ultrasound and 3D imaging modalities (e.g., computed tomography (CT) or magnetic resonance imaging (MRI)) by providing correspondence between points in different views. Examples of clinical applications may include, but are not limited to: cardiology, vascular imaging, prostate, bladder, and thyroid imaging.

[0102] Various embodiments of the present invention may benefit clinical workflow by simplifying the measurement, screening and / or monitoring of organs in terms of time because they make matching of anatomical structures easier and more reliable.

[0103] Figure 1 A schematic flow chart of a method 100 (computer-implemented method) for cross-referencing a two-dimensional (2D) ultrasound scan of a tissue volume (including a target anatomical structure) using at least one processor is described in accordance with various embodiments of the present invention.

[0104] Method 100 includes step 102, which generates a first series (or sequence or set) of 2D ultrasound images (which may also be interchangeably referred to as cross-sectional images, image planes, image frames / slices or B-mode / B-scan images) of a tissue volume associated with a first plurality of positions along a first scanning direction of the tissue volume, respectively, the tissue volume including the target anatomical structure.

[0105] The method 100 also includes step 104 of generating a second series (or sequence or set) of 2D ultrasound images of the tissue volume associated with the second plurality of locations respectively along a second scanning direction of the tissue volume, wherein the second scanning direction is at least substantially orthogonal to the first scanning direction. In various embodiments, the term "at least substantially orthogonal" in relation to two directions (e.g., "first scanning direction" and "second scanning direction") herein may be understood as the two directions being orthogonal to each other, or nearly orthogonal to each other, for example, deviating from orthogonality by less than or equal to 10 degrees, for example, less than or equal to 5 degrees, for example, less than or equal to 2 degrees, for example, less than or equal to 1 degree.

[0106] With regard to step 102, a first series of 2D ultrasound images may be generated based on a first series of ultrasound waves acquired by an ultrasound transducer positioned at a first plurality of positions with respect to a first plurality of time instances, respectively. Alternatively, a first series of 2D ultrasound images may be generated based on first image data stored in a memory, the first image data being associated with a first series of ultrasound waves acquired by an ultrasound transducer positioned at a first plurality of positions with respect to a first plurality of time instances, respectively.

[0107] Similarly, with respect to step 104, a second series of 2D ultrasound images may be generated based on the first series of ultrasound waves acquired by the ultrasound transducer positioned at the second plurality of positions with respect to the second plurality of time instances, respectively. Alternatively, a second series of 2D ultrasound images may be generated based on second image data stored in a memory, the second image data being associated with the second series of ultrasound waves acquired by the ultrasound transducer positioned at the second plurality of positions with respect to the first plurality of time instances, respectively.

[0108] In various embodiments, with respect to steps 102 and 104, the ultrasound transducer is configured to transmit ultrasound waves relative to a tissue volume plane (e.g., a cross-sectional plane perpendicular to the scanning direction) and acquire ultrasound waves reflected from the tissue volume plane, and the ultrasound transducer can be used to acquire a first series of ultrasound waves and a second series of ultrasound waves (in time sequence) at a first plurality of positions along a first scanning direction and a second plurality of positions along a second scanning direction of the tissue volume. Such an ultrasound transducer can be referred to as a 2D ultrasound transducer.

[0109] Various embodiments of the present invention are directed to cross-referenced 2D ultrasound scans or scans obtained by freehand ultrasound scanning of a tissue volume. In this regard, an operator may move a 2D ultrasound transducer (or a portable handheld ultrasound probe containing a 2D ultrasound transducer) along a scanning direction of the tissue volume (e.g., along an axis through the length of the tissue volume) to perform an ultrasound scan of the tissue volume, whereby the 2D ultrasound transducer acquires a series of ultrasound waves at multiple locations along the scanning direction with respect to multiple time instances. The ultrasound waves received at each time instance (at the corresponding location) may then be processed in a manner known in the art to generate a 2D ultrasound image associated with the corresponding location, and therefore need not be described in detail herein. Thus, a series of 2D ultrasound images of the tissue volume may be acquired, each 2D ultrasound image having an associated location (e.g., marked or labeled with associated location information), for example, corresponding to the location of the 2D ultrasound transducer that acquired the ultrasound wave (based on which the 2D ultrasound image is generated) or corresponding to the location / position along the tissue volume of the ultrasound wave acquired by the reflected 2D ultrasound transducer.

[0110] The 2D ultrasonic transducer may be any conventional 2D ultrasonic transducer configured to transmit and acquire ultrasonic waves relative to the plane of the tissue volume, and therefore need not be described in detail herein. For example, but not limited to, a conventional 2D ultrasonic transducer may include an array of transducer elements configured to transmit and acquire ultrasonic waves relative to the plane of the tissue volume. Therefore, those skilled in the art will appreciate that the present invention is not limited to any particular type of 2D ultrasonic transducer.

[0111] The method 100 further comprises step 106 of generating a first 2D representation of the target anatomical structure from the first series of 2D ultrasound images, the first 2D representation being associated with one of the images within the first series of 2D ultrasound images.

[0112] The method 100 further comprises step 108 of generating a first simulated 2D representation of the target anatomical structure through the second series of 2D ultrasound images, wherein the first simulated 2D representation of the target anatomical structure and the first 2D representation correspond to each other in a direction of viewing the target anatomical structure.

[0113] Method 100 also includes step 110 of processing the first 2D representation and the first simulated 2D representation of the target anatomy so that pixel locations associated with the target anatomy in the first simulated 2D representation at least substantially match pixel locations associated with the target anatomy in the first 2D representation. In various embodiments, the term “at least substantially matches” as used herein in relation to pixel positions in two different 2D entities (e.g., the processed first 2D representation and the first simulated 2D representation) may be understood to mean that the position (e.g., 2D coordinate) of a pixel (or multiple pixels, e.g., all pixels) in one of the two 2D entities (e.g., the processed first simulated 2D representation) matches the position (e.g., 2D coordinate) of a corresponding pixel (or multiple corresponding pixels) in the other of the two 2D entities (e.g., the processed first 2D representation), or nearly matches the position of the corresponding pixel (or multiple corresponding pixels), e.g., with a position deviation (e.g., between x-coordinates and / or between y-coordinates) that is less than or equal to a given number of pixel pitches (e.g., less than or equal to 10 pixel pitches, e.g., less than or equal to 5 pixel pitches, e.g., less than or equal to 2 pixel pitches, e.g., less than or equal to 1 pixel pitch).

[0114] The method 100 further comprises a step 112 of determining a first corresponding transformation matrix by means of the processed first 2D representation of the target anatomical structure and the processed first simulated 2D representation.

[0115] The method 100 further comprises step 114 of generating a second 2D representation of the target anatomical structure from the second series of 2D ultrasound images, the second 2D representation being associated with one of the images within the second series of 2D ultrasound images.

[0116] The method 100 further comprises step 116 of generating a second simulated 2D representation of the target anatomical structure from the first series of 2D ultrasound images, the second simulated 2D representation of the target anatomical structure and the second 2D representation corresponding to each other in a direction in which the target anatomical structure is viewed.

[0117] The method 100 further includes step 118 of processing the second 2D representation and the second simulated 2D representation of the target anatomical structure so that pixel locations associated with the target anatomical structure in the second simulated 2D representation at least substantially match pixel locations associated with the target anatomical structure in the second 2D representation.

[0118] The method 100 further comprises a step 120 of determining a second corresponding transformation matrix from the processed second 2D representation of the target anatomical structure and the processed second simulated 2D representation.

[0119] The method 100 further comprises determining, for a first position associated with the target anatomical structure in the second series of 2D ultrasound images, a corresponding second position associated with the target anatomical structure in the first series of 2D ultrasound images using the first corresponding transformation matrix and the second corresponding transformation matrix, step 122. In various embodiments, the first position may be a point selected by a user on one of the images (image frames) within the second series of 2D ultrasound images, and the second position may be a (cross-referenced) point on one of the images (image frames) within the first series of 2D ultrasound images obtained with the aid of the first transformation matrix and the second transformation matrix.

[0120] In various embodiments, with respect to step 122, determining the second position using the first corresponding transformation matrix and the second corresponding transformation matrix includes applying a first function of the first corresponding transformation matrix and the second corresponding transformation matrix to the pixel coordinates corresponding to the first position, thereby obtaining the pixel coordinates corresponding to the second position.

[0121] In various embodiments, method 100 may further include determining, for a third position associated with the target anatomical structure in the first series of 2D ultrasound images, a corresponding fourth position associated with the target anatomical structure in the second series of 2D ultrasound images using the first corresponding transformation matrix and the second corresponding transformation matrix. In various embodiments, the third position may be a point selected by a user on one of the images (image frames) within the first series of 2D ultrasound images, and the fourth position may be a (cross-referenced) point on one of the images (image frames) within the second series of 2D ultrasound images obtained with the aid of the first transformation matrix and the second transformation matrix.

[0122] In various embodiments, determining the fourth position using the first corresponding transformation matrix and the second corresponding transformation matrix includes: applying a second function of the first corresponding transformation matrix and the second corresponding transformation matrix to the pixel coordinates corresponding to the third position to obtain the pixel coordinates corresponding to the fourth position.

[0123] In various embodiments, the method 100 may further include generating a first plurality of binary segmentation masks from the first series of 2D ultrasound images, each binary segmentation mask in the first plurality of binary segmentation masks corresponding to a respective one of the 2D ultrasound images within the first series of 2D ultrasound images. In various embodiments, the first 2D representation of the target anatomical structure may be generated from one of the binary segmentation masks generated from within the first plurality of binary segmentation masks.

[0124] In various embodiments, the method 100 may further include generating a second plurality of binary segmentation masks from the second series of 2D ultrasound images, each binary segmentation mask in the second plurality of binary segmentation masks corresponding to a respective one of the 2D ultrasound images of the second series of 2D ultrasound images. In various embodiments, a second 2D representation of the target anatomical structure may be generated from one of the binary segmentation masks within the second plurality of binary segmentation masks.

[0125] In various embodiments, as is known in the art of digital image processing, the first plurality of binary segmentation masks and / or the second plurality of binary segmentation masks may be obtained by using a segmentation method or process. An example of a suitable segmentation method used by the various embodiments is called U-Net, which is a convolutional neural network developed for biomedical image segmentation. In digital image processing and computer vision, image segmentation refers to the process of segmenting a digital image into multiple fragments (collections of pixels, also called image objects). More specifically, image segmentation refers to the process of assigning a label (or value) to each pixel in an image so that pixels with the same label (or value) share certain features.

[0126] In various embodiments, in each binary segmentation mask, the values ​​of pixels associated with the target anatomical structure can be assumed to be a first value (e.g., "1"), while the values ​​of pixels not associated with the target anatomical structure can be assumed to be a second value different from the first value (e.g., "0").

[0127] In various embodiments, the first 2D representation of the target anatomical structure may correspond to a maximum binary segmentation mask of the first plurality of binary segmentation masks.

[0128] In various embodiments, the second 2D representation of the target anatomical structure may correspond to a maximum binary segmentation mask of the second plurality of binary segmentation masks.

[0129] In various embodiments, the term "maximum binary segmentation mask" used herein may be understood as a binary segmentation mask having the maximum number of pixels associated with the target anatomical structure among multiple binary segmentation masks. For example, the maximum binary segmentation mask may be a binary segmentation mask having the largest number of pixels having a first value (e.g., "1") among multiple binary segmentation masks.

[0130] In various embodiments, with respect to step 106, generating the first 2D representation includes removing a region in the largest binary segmentation mask in the first plurality of binary segmentation masks that is incorrectly labeled as corresponding to the target anatomical structure. For example, a maximum continuous component search algorithm may be applied to determine the largest continuous component (largest cluster) in the largest binary segmentation mask, and regions that are not connected to the largest continuous component may be removed from the mask.

[0131] In various embodiments, with respect to step 114, generating a second 2D representation includes removing a region in the largest binary segmentation mask in the second plurality of binary segmentation masks that is incorrectly labeled as corresponding to the target anatomical structure. For example, a maximum continuous component search algorithm may be applied to determine the largest continuous component (largest cluster) in the largest binary segmentation mask, and regions that are not connected to the largest continuous component may be removed from the mask.

[0132] In various embodiments, with respect to step 108, generating a first simulated 2D representation of the target anatomical structure from the second series of 2D ultrasound images includes generating the first simulated 2D representation of the target anatomical structure from the second plurality of binary segmentation masks. In various embodiments, regions in the first simulated 2D representation that are incorrectly labeled as corresponding to the target anatomical structure may be removed. For example, a maximum continuous component search algorithm may be applied to determine the maximum continuous component (maximum cluster), and regions that are not connected to the maximum continuous component may be removed.

[0133] In various embodiments, with respect to step 116, generating a second simulated 2D representation of the target anatomical structure from the first series of 2D ultrasound images includes generating a second simulated 2D representation of the target anatomical structure from the first plurality of binary segmentation masks. In various embodiments, regions in the second simulated 2D representation that are incorrectly labeled as corresponding to the target anatomical structure may be removed. For example, a maximum continuous component search algorithm may be applied to determine the maximum continuous component (maximum cluster), and regions that are not connected to the maximum continuous component may be removed.

[0134] In various embodiments, with respect to step 110, processing the first 2D representation includes generating a first signed distance map from the first 2D representation. In various embodiments, generating the first signed distance map includes determining a boundary contour of the first 2D representation (e.g., a maximum binary segmentation mask in a first plurality of binary segmentation masks), generating a first distance map in which the value of each pixel is the distance of the pixel to its closest contour pixel, converting the first 2D representation (e.g., the maximum binary segmentation mask) into a first signed map; generating the first signed distance map by element-wise multiplication of the first signed map and the first distance map.

[0135] In various embodiments, with respect to step 118, processing the second 2D representation includes generating a second signed distance map from the second 2D representation. In various embodiments, generating the second signed distance map includes determining a boundary contour of the 2D representation (e.g., a maximum binary segmentation mask in the second plurality of binary segmentation masks), generating a second distance map in which the value of each pixel is the distance between the pixel and its nearest contour pixel, converting the 2D representation (e.g., the maximum binary segmentation mask) to a second signed map; and generating the second signed distance map by element-wise multiplication of the second signed map and the second distance map.

[0136] In various embodiments, with respect to step 110 , processing the first simulated 2D representation includes scaling a width of the first simulated 2D representation to match a width of the first 2D representation.

[0137] In various embodiments, with respect to step 118 , processing the second simulated 2D representation includes scaling a width of the second simulated 2D representation to match a width of the second 2D representation.

[0138] In various embodiments, with respect to step 110, processing the first simulated 2D representation further includes determining and sampling contour pixels associated with a contour of the target anatomical structure from the scaled first simulated 2D representation.

[0139] In various embodiments, with respect to step 118 , processing the second simulated 2D representation further includes determining and sampling contour pixels associated with a contour of the target anatomical structure from the scaled second simulated 2D representation.

[0140] In various embodiments, with respect to step 110, processing the first simulated 2D representation further includes sampling a first subset of pixels from the scaled first simulated 2D representation.

[0141] In various embodiments, with respect to step 118, processing the second simulated 2D representation further includes sampling a second subset of pixels from the scaled second simulated 2D representation.

[0142] In various embodiments, with respect to step 110, processing the first analog 2D representation further includes estimating a coarse offset between the scaled first analog 2D representation and the first 2D representation using the first signed distance map and the sampled first subset of pixels.

[0143] In various embodiments, with respect to step 118, processing the second simulated 2D representation further includes estimating a coarse offset between the scaled second simulated 2D representation and the second 2D representation using the second signed distance map and the sampled second subset of pixels.

[0144] In various embodiments, with respect to step 112, determining a first corresponding transformation matrix includes minimizing a first alignment cost defined as a first cost function with respect to the first corresponding transformation matrix, estimating a scale and a translation between the scaled first simulated 2D representation and the first 2D representation. In various embodiments, the first alignment cost is defined as a sum of squared distances with respect to the first corresponding transformation matrix.

[0145] In various embodiments, with respect to step 120, determining a second corresponding transformation matrix includes minimizing a second alignment cost defined as a second cost function with respect to the second corresponding transformation matrix, estimating a scale and a translation between the scaled second simulated 2D representation and the second 2D representation. In various embodiments, the second alignment cost is defined as a sum of squared distances with respect to the second corresponding transformation matrix.

[0146] In various embodiments, minimizing the first alignment cost and / or minimizing the second alignment cost includes applying an optimization algorithm, such as a Gauss-Newton algorithm.

[0147] In various embodiments, the first scanning direction corresponds to a transverse scan of the tissue volume and the second scanning direction corresponds to a sagittal scan of the tissue volume.

[0148] In various embodiments, the target anatomical structure includes at least one of: a body organ, a portion of a body organ (eg, a thyroid lobe), a portion of a vascular system.

[0149] In various embodiments, the body organ includes at least one of the following: thyroid, prostate, bladder, heart, lung, stomach, liver, kidney.

[0150] Figure 2 A schematic block diagram of a cross-referencing system 200 for two-dimensional (2D) ultrasound scanning of a tissue volume according to various embodiments of the present invention is described, such as corresponding to a cross-referencing method 100 for two-dimensional (2D) ultrasound scanning of a tissue volume using at least one processor as described above according to various embodiments of the present invention.

[0151] The system 200 includes a memory 204 and at least one processor 206 communicatively coupled to the memory 204, and the processor 206 is configured to: generate a first series of 2D ultrasound images of the tissue volume associated with a first plurality of locations along a first scanning direction of the tissue volume, the tissue volume including a target anatomical structure; generate a second series of 2D ultrasound images of the tissue volume associated with a second plurality of locations along a second scanning direction of the tissue volume, wherein the second scanning direction is at least substantially orthogonal to the first scanning direction; generate a first 2D representation of the target anatomical structure from the first series of 2D ultrasound images, the first 2D representation being associated with one of the images within the first series of 2D ultrasound images; generate a first simulated 2D representation of the target anatomical structure from the second series of 2D ultrasound images, the first simulated 2D representation of the target anatomical structure and the first 2D representation corresponding to each other in a direction in which the target anatomical structure is observed; and process the first 2D representation of the target anatomical structure and the first simulated 2D representation so that a pixel position associated with the target anatomical structure in the first simulated 2D representation is at least substantially orthogonal to a pixel position associated with the target anatomical structure in the first 2D representation. matching; determining a first corresponding transformation matrix based on the processed first 2D representation and the processed first simulated 2D representation of the target anatomical structure; generating a second 2D representation of the target anatomical structure from the second series of 2D ultrasound images, the second 2D representation being associated with one of the images within the second series of 2D ultrasound images; generating a second simulated 2D representation of the target anatomical structure from the first series of 2D ultrasound images, the second simulated 2D representation and the second 2D representation of the target anatomical structure corresponding to each other in a direction in which the target anatomical structure is viewed; processing the second 2D representation and the second simulated 2D representation of the target anatomical structure so that pixel positions associated with the target anatomical structure in the second simulated 2D representation at least substantially match pixel positions associated with the target anatomical structure in the second 2D representation; determining a second corresponding transformation matrix based on the processed second 2D representation and the processed second simulated 2D representation of the target anatomical structure; and determining, using the first corresponding transformation matrix and the second corresponding transformation matrix, a corresponding second position associated with the target anatomical structure in the first series of 2D ultrasound images for the first position associated with the target anatomical structure in the second series of 2D ultrasound images.

[0152] Those skilled in the art will appreciate that the at least one processor 206 may be configured to perform desired functions or operations through a set of instructions (eg, software modules) executable by the at least one processor 206 to perform the desired functions or operations.

[0153] Therefore, if Figure 2As shown, the system 200 may further include a 2D ultrasound image generator 208, a 2D representation generator 210, a simulated 2D representation generator 212, a 2D and simulated 2D representation processor 214, a corresponding transformation matrix determiner 216, and a corresponding position determiner 218. The 2D ultrasound image generator 208 is configured to generate a first series of 2D ultrasound images and a second series of 2D ultrasound images; the 2D representation generator 210 is configured to generate a first 2D representation of a target anatomical structure based on the first series of 2D ultrasound images, and to generate a second 2D representation of the target anatomical structure based on the second series of 2D ultrasound images; the simulated 2D representation generator 212 is configured to generate a first simulated 2D representation of the target anatomical structure based on the second series of 2D ultrasound images, and to generate a first simulated 2D representation of the target anatomical structure based on the second series of 2D ultrasound images. Second 2D representation ; The 2D and simulated 2D representation processor 214 is configured to process a first 2D representation and a first simulated 2D representation of the target anatomical structure, and to process a second 2D representation and a second simulated 2D representation of the target anatomical structure; the corresponding transformation matrix determiner 216 is configured to determine a first corresponding transformation matrix for the processed first 2D representation and the first simulated 2D representation, and to determine a second corresponding transformation matrix for the processed second 2D representation and the second simulated 2D representation; and the corresponding position determiner 218 is configured to determine a second position associated with the target anatomical structure in the first series of 2D ultrasound images for the first position associated with the target anatomical structure in the second series of 2D ultrasound images by using the first corresponding transformation matrix and the second corresponding transformation matrix.

[0154] In some embodiments, the system 200 may also optionally include an ultrasound transducer 202 communicatively coupled to a memory 204 and to at least one processor 206. In some embodiments, the ultrasound transducer 202 may be mounted on a freehand ultrasound probe.

[0155] In some embodiments, the ultrasound image generator 208 can be configured to generate a first series of 2D ultrasound images based on a first series of ultrasound waves acquired by the ultrasound transducer 202 positioned at a first plurality of positions with respect to a first plurality of time instances; and to generate a second series of 2D ultrasound images based on a second series of ultrasound waves acquired by the ultrasound transducer 202 positioned at a second plurality of positions with respect to a second plurality of time instances.

[0156] Alternatively or additionally, in some embodiments, the memory 204 may be configured to store first image data associated with a first series of ultrasonic waves acquired by an ultrasonic transducer (e.g., ultrasonic transducer 202) positioned at a first plurality of positions with respect to a first plurality of time instances; and to store second image data associated with a second series of ultrasonic waves acquired by an ultrasonic transducer (e.g., ultrasonic transducer 202) positioned at a second plurality of positions with respect to a second plurality of time instances; and at least one processor 206 (e.g., 2D ultrasonic image generator 208) may be configured to: generate a first series of 2D ultrasonic images based on the first image data stored in the memory 204; and generate a second series of 2D ultrasonic images based on the second image data stored in the memory 204.

[0157] Those skilled in the art will appreciate that the above modules are not necessarily separate modules, and one or more modules may be implemented by or as a functional module (e.g., a circuit or software program) as needed or appropriate without departing from the scope of the present invention. For example, the 2D ultrasound image generator 208, the 2D representation generator 210, the simulated 2D representation generator 212, the 2D and simulated 2D representation processors 214, the corresponding transformation matrix determiner 216, and / or the corresponding position determiner 218 may be implemented (e.g., compiled together) as an executable software program (e.g., a software application or simply referred to as an "application"), such as may be stored in the memory 204, and may be executed by at least one processor 206 to perform the functions / operations described herein according to various embodiments.

[0158] In various embodiments, the system 200 corresponds to the system described above. Figure 1 The method 100 shown, therefore, according to different embodiments, various functions or operations configured to be performed by at least one processor 206 may correspond to the various steps of the method 100 described above, so for the sake of clarity and brevity, there is no need to repeat these steps for the system 200. In other words, various embodiments described in this document in the context of the method are also applicable to the corresponding system or device, and vice versa.

[0159] For example, in various embodiments, the memory 204 may store a 2D ultrasound image generator 208, a 2D representation generator 210, a simulated 2D representation generator 212, a 2D and simulated 2D representation processor 214, a corresponding transformation matrix determiner 216, and / or a corresponding position determiner 218, which correspond to the various steps of the method 100 described above, respectively. They may be executed by at least one processor 206 to perform the corresponding functions / operations described herein.

[0160] According to various embodiments of the present disclosure, a computer system, a controller, a microcontroller or any other system providing processing capability may be provided. Such a system may be considered to include one or more processors and one or more computer-readable storage media. For example, the system 200 described above may include a processor (or controller) 206 and a computer-readable storage medium (or memory) 204, which are used, for example, in various processes described herein. The memory or computer-readable storage medium used in various embodiments may be a volatile memory such as a dynamic random access memory (DRAM) or a non-volatile memory such as a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory (e.g., a floating gate memory, a charge trap memory, a magnetoresistive random access memory (MRAM), or a phase change random access memory (PCRAM)).

[0161] In various embodiments, a "circuit" may be understood as any type of logic implementation entity, which may be a dedicated circuit or a processor that executes software stored in a memory, firmware, or any combination thereof. Thus, in an embodiment, a "circuit" may be a hardwired logic circuit or a programmable logic circuit such as a programmable processor such as a microprocessor (e.g., a Complex Instruction Set Computer (CISC) processor or a Reduced Instruction Set Computer (RISC) processor). A "circuit" may also be a processor that executes software such as any type of computer program (e.g., a computer program using a virtual machine code such as Java). According to various alternative embodiments, any other kind of implementation of the various functions described in more detail below may also be understood as a "circuit". Similarly, a "module" may be part of a system according to various embodiments of the present invention and may include a "circuit" as described above, or may be understood thereby as any type of logic implementation entity.

[0162] Certain portions of the present disclosure are presented explicitly or implicitly in the form of algorithms and functional or symbolic representations of operations on data within a computer memory. These algorithmic descriptions and functional or symbolic representations are means used by those skilled in the art of data processing to most effectively communicate the essence of their work to other persons skilled in the art. Algorithms are generally considered here to be self-consistent sequences of steps that lead to a desired result. These steps are steps that require physical operations on physical quantities, such as electrical, magnetic or optical signals that can be stored, transmitted, combined, compared and otherwise manipulated.

[0163] Unless otherwise specifically stated, and as will become apparent from the following text, in this specification, detailed descriptions using the terms "generate," "estimate," "modify," "present," and the like refer to the actions and processes of a computer system or similar electronic device that manipulates and transforms data represented as physical quantities within a computer system into other data similarly represented as physical quantities within a computer system or other information storage, transmission, or display device.

[0164] This specification also discloses a system, device or apparatus for performing the operation / function of the method described herein. Such a system, device or apparatus can be specially constructed for the desired purpose, or can include a general-purpose computer or other equipment selectively activated or reconfigured by a computer program stored in a computer. The algorithm proposed herein has no inherent relationship with any specific computer or other equipment. According to the teachings of this article, various general-purpose machines can be used together with a computer program. Alternatively, it may be appropriate to construct a more specialized device to perform the required method steps.

[0165] In addition, the present specification also discloses at least implicitly a computer program or software / functional module, because it is obvious to those skilled in the art that the various steps of the method described herein can be implemented by computer code. The computer program is not intended to be limited to any particular programming language and its implementation. It should be understood that a variety of programming languages ​​and their encodings can be used to implement the teachings of the disclosure contained herein. In addition, the computer program is not intended to be limited to any particular control flow. Without departing from the scope of the present invention, there are many other variations of the computer program, which can use different control flows. It should be understood by those skilled in the art that the various modules described herein (e.g., 2D ultrasound image generator 208, 2D representation generator 210, simulated 2D representation generator 212, 2D and simulated 2D representation processor 214, corresponding transformation matrix determiner 216, and / or corresponding position determiner 218) can be software modules implemented by a computer program or instruction set executable by a computer processor to perform the desired function, or can be hardware modules designed to be functional hardware units that perform the desired function. It should also be understood that a combination of hardware and software modules can be implemented.

[0166] In addition, one or more steps of the computer program / module or method described herein can be executed in parallel rather than sequentially. Such a computer program can be stored on any computer-readable medium. The computer-readable medium may include storage devices such as disks or optical disks, memory chips or other storage devices suitable for engaging with general-purpose computers. When the computer program is loaded and executed on such a general-purpose computer, a device for implementing the method steps described herein can be effectively generated.

[0167] In various embodiments, a computer program product is provided, which is contained in one or more computer-readable storage media (non-transitory computer-readable storage media), including instructions (e.g., 2D ultrasound image generator 208, 2D representation generator 210, simulated 2D representation generator 212, 2D and simulated 2D representation processor 214, corresponding transformation matrix determiner 216 and / or corresponding position determiner 218) executable by one or more computer processors to perform the above-referenced Figure 1 The cross-referencing method 100 for two-dimensional (2D) ultrasound scanning of a tissue volume is described. Thus, the various computer programs or modules described herein may be stored in a computer program product that can be received by a system (e.g., a computer system or an electronic device), such as Figure 2 The system 200 is shown so as to be executed by at least one processor 206 of the system 200 to perform required or desired functions.

[0168] The software or functional modules described herein may also be implemented as hardware modules. More specifically, in a hardware sense, a module is a functional hardware unit designed to be used in conjunction with other components or modules. For example, a module may be implemented using discrete electronic components, or it may form part of an entire electronic circuit, such as an Application Specific Integrated Circuit (ASIC). There are many other possibilities. Those skilled in the art will appreciate that the software or functional modules described herein may also be implemented as a combination of hardware and software modules.

[0169] Those skilled in the art will appreciate that the system 200 may be comprised of separate units or as an integrated unit. For example, in various embodiments, the system 200 may include a computer system including at least one processor 206, a memory 204, a 2D ultrasound image generator 208, a 2D representation generator 210, a simulated 2D representation generator 212, a 2D and simulated 2D representation processor 214, a corresponding transformation matrix determiner 216, and a corresponding position determiner 218, and an independent ultrasound probe including an ultrasound transducer 202 communicatively coupled to the computer system. In other words, the independent ultrasound probe may acquire a series of ultrasound waves of a tissue volume, and may then transmit the ultrasound wave series (e.g., based on wireless or wired communication) to a computer system located at a different location to perform the above-referenced method. Figure 1 In various other embodiments, the system 200 may correspond to or include an ultrasound probe, including an ultrasound transducer 202, at least one processor 206, a memory 204, a 2D ultrasound image generator 208, a 2D representation generator 210, a simulated 2D representation generator 212, a 2D and simulated 2D representation processor 214, a corresponding transformation matrix determiner 216, and a corresponding position determiner 218.

[0170] In various embodiments, the above-described computer system may be implemented by any computer system (e.g., a portable or desktop computer system), such as Figure 3The computer system 300 schematically shown in the figure is for example only and not for limitation. Various methods / steps or functional modules (e.g., 2D ultrasound image generator 208, 2D representation generator 210, simulated 2D representation generator 212, 2D and simulated 2D representation processor 214, corresponding transformation matrix determiner 216 and / or corresponding position determiner 218) can be implemented as software, such as a computer program executed in the computer system 300, and instructing the computer system 300 (especially, one or more processors therein) to perform the methods / functions of various embodiments described herein. The computer system 300 may include a computer module 302, an input module (such as a keyboard 304 and a mouse 306), and a plurality of output devices (such as a display 308 and a printer 310). The computer module 302 may be connected to a computer network 312 via a suitable transceiver device 314 to enable access to, for example, the Internet or other network systems such as a local area network (LAN) or a wide area network (WAN). The computer module 302 in the example may include a processor 318 for executing various instructions, a random access memory (RAM) 320, and a read-only memory (ROM) 322. The computer module 302 may also include several input / output (I / O) interfaces, such as an I / O interface 324 to the display 308, and an I / O interface 326 connected to the keyboard 304. The components of the computer module 302 typically communicate through an interconnect bus 328 and in a manner known to those skilled in the relevant art.

[0171] Those skilled in the art will appreciate that the terms used herein are only used for the purpose of describing various embodiments and are not intended to limit the present invention. As used herein, the singular forms "a", "an" and "the" are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that when the terms "comprise" and / or "comprising" are used in this specification, the presence of the features, integers, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof is not excluded.

[0172] For illustrative purposes only and not limiting, Figure 4An example scan (or series, sequence, or plurality) 402 of 2D ultrasound images (e.g., 404a, 404b, 404c, and 404d) of a tissue volume associated with a plurality of locations along a scan direction 408 of the tissue volume is depicted. In this regard, the series of 2D ultrasound images 402 may be generated based on a series of ultrasound waves acquired by an ultrasound transducer (e.g., mounted in an ultrasound probe 410) at a plurality of locations (e.g., 406a, 406b, 406c, 406d, 406e) along the scan direction 408 of the ultrasound probe.

[0173] In order to make the present invention easy to understand and put into practice, various exemplary embodiments of the present invention will be described below only by way of example and not limitation. However, it should be understood by those skilled in the art that the present invention may include various different forms or configurations and should not be construed as being limited to the exemplary embodiments described below. On the contrary, providing these exemplary embodiments will make this disclosure full and complete, and fully convey the scope of the present invention to those skilled in the art.

[0174] Various embodiments of the present invention disclose a method and system that can assist a user by automatically cross-referencing points in two different views (e.g., TRX and SAG views) using information common to two perpendicular ultrasound scans. More specifically, when a user selects a region of interest on a first view (e.g., a transverse view), various embodiments of the present invention can assist the user in locating the corresponding region of interest on a second view (e.g., a sagittal view).

[0175] According to various embodiments, for cross-referencing, two 2D scans of the 3D anatomical structure from two (almost) orthogonal views may be provided. Without losing generality and for simplicity, the following will mainly assume that the first scan is performed on a transverse (TRX) view and the second scan is performed on a sagittal (SAG) view.

[0176] According to various embodiments, a segmentation method may be applied to segment the boundaries of a target anatomical structure (e.g., thyroid gland) in each 2D frame of the two views. In various embodiments, the term "target anatomical structure" as used herein may refer to an anatomical structure (or internal anatomical structure) of interest, such as a body organ (e.g., thyroid gland, prostate gland, bladder, heart, lung, stomach, liver, kidney, or other body organ) or a portion of a vascular system (e.g., carotid artery).

[0177] Since both views contain the same 3D anatomy, there is common information in the two perpendicular ultrasound scan sets. According to various embodiments, this common information is used to calculate a transformation matrix that transforms any given point [idx trx ,y trx , x trx] is mapped to the corresponding point [idx] on the SAG view sag ,y sag , x sag Here, idx represents the frame index, and x and y represent the coordinates of the corresponding point. The frame index represents the specific image frame in the series of 2D ultrasound images (scans) in which the corresponding point is located. For example, [idx trx ,y trx , x trx ] indicates the location at index idx trx The TRX frame (the frame of the TRX scan) has a coordinate y in the frame. trx , x trx point.

[0178] According to various embodiments of the present invention, a (computer-implemented) method or algorithm for calculating a transformation of a cross-referencing tool or system may be provided. When a region of interest or landmark (e.g., nodule) has been previously identified (e.g., by a user) on a first image plane (e.g., a transverse plane), the cross-referencing tool or system may help the user locate an approximate region of interest (ROI) or landmark (e.g., nodule) on a second image plane (e.g., a sagittal plane).

[0179] According to various embodiments, a cross-reference tool or system may use a linear transformation (matrix) to select specific frame indices and coordinates within a sagittal (longitudinal) image based on frame indices and coordinates provided by a user on a transverse image. The mapping obtained from the transformation may be used to place the transverse and sagittal images in pairs into the same spatial coordinate system, allowing transverse viewpoints to be mapped to sagittal viewpoints.

[0180] When the user calls the method or algorithm, the method or algorithm can receive images of two 2D ultrasound scans (e.g., a 2D thyroid image series) taken along orthogonal scanning directions, such as a transverse (TRX) scan (including a plurality of TRX images (image frames)) and a sagittal (SAG) scan (including a plurality of SAG images (image frames)) as input, and can output a pair of transformation matrices (also referred to herein as corresponding transformation matrices) T trx and T sag .

[0181] In the following, as an illustrative example, a pair of transformation matrices T will be described in the context of 2D ultrasound imaging of the thyroid lobe including transverse (TRX) scanning and sagittal (SAG) scanning. trx and T sag The thyroid lobe may be an example of a target anatomical structure.

[0182] According to some embodiments, the calculation of the transformation matrix may include the following steps:

[0183] 1. Segment the target anatomical structure (thyroid lobe) in each frame of the TRX scan and the SAG scan to obtain a binary mask (also called a binary segmentation mask) of the TRX scan and the SAG scan image frames.

[0184] In the binary mask, the pixel value is assumed to be a first value (e.g., "1") representing the thyroid region, or a second value (e.g., "0") representing the non-thyroid region. Segmentation can be performed by any general segmentation method known in the art (e.g., a classical or neural network-based segmentation method).

[0185] 2. Find the best binary mask from the TRX scan and the best binary mask from the SAG scan.

[0186] The best binary mask for a TRX scan may be the maximum binary mask among the binary masks obtained from the TRX scans, and the best binary mask for a SAG scan may be the maximum binary mask among the binary masks obtained from the SAG scans. The best (or maximum) binary mask for a TRX scan may be referred to as a first 2D representation of the thyroid lobe (target anatomical structure), and the best (or maximum) binary mask for a SAG scan may be referred to as a second 2D representation of the thyroid lobe (target anatomical structure).

[0187] 3. Stack all the masks scanned by SAG from (H sag ×W sag ×N sag ) is reconstructed (projected) into (H sag ×N sag ×W sag ), where N sag is the number of image frames scanned by SAG, H sag and W sag are the number of rows and columns for each SAG image frame, respectively. By doing this, a simulated stack of TRX masks is created from the stack of original SAGs. Then, the maximum simulated TRX mask is selected in a similar manner to step 2.

[0188] 4. Find the initial (rough) aspect ratio of the simulated TRX frame. This ratio matches the simulated TRX and actual TRX images and can be optimized later to refine the ratio. In various embodiments, the aspect ratio can be used to scale the width of the simulated TRX mask to obtain a scaled version of the simulated TRX mask, which can be referred to as a scaled first simulated 2D representation of the thyroid lobe (target anatomy).

[0189] 5. Create a distance map for the selected TRX mask (i.e., create a distance map for the best binary mask of the TRX scan obtained in step 2). In the distance map (also called TRX distance map), the value of a pixel is the distance from that pixel to the nearest pixel at the edge of the thyroid lobe (target anatomical structure) in the TRX mask. Points within the thyroid lobe in the simulated TRX mask can be sampled for the next step, as using all pixels within the thyroid lobe (target anatomical structure) can be computationally intensive.

[0190] 6. Use a vectorized heuristic search method to solve the optimization problem, which aligns the previously generated images (TRX distance map and points sampled from the simulated TRX mask).

[0191] 7. Sample points from the edge of the thyroid lobe (target anatomy) mask in the simulated TRX mask and use the alignment parameters found in the previous step and an optimization algorithm, such as the Gauss-Newton algorithm, to minimize the alignment cost and find the transformation matrix T trx .

[0192] 8. Repeat steps (3) to (7) above from the TRX scanned mask and calculate the transformation matrix T sag .

[0193] 9. Optional: Find the in-plane offset for each frame, this is similar to step 6 and leads to finding the matrix RC trx , to compensate for hand movements (of the user, e.g., clinician) that occur during scanning (using a freehand ultrasound probe).

[0194] 10. With T trx and T sag (and optionally RC trx ), any point in the TRX scan (with frame index idx trx and coordinates (x trx ,y trx )) can be mapped to the corresponding point of the SAG scan (with frame index idx sag and coordinates (x sag ,y sag )). Therefore, cross-reference between TRX and SAG scan images can be achieved. That is, for any point in the transverse view, i.e., frame index idx trx Where (x trx ,y trx ), we can get the cross reference point in the sagittal view, i.e. the frame index idx sag Where (x sag ,y sag ). The frame index indicates a specific frame (2D ultrasound image frame) in a series of 2D ultrasound images (image frames) of a TRX scan or a SAG scan.

[0195] The following will refer to Figure 5 Description of various aspects of a cross-referencing method for 2D ultrasound scanning according to various embodiments (eg, Figure 1 Flowchart 500 of method 100 is shown.

[0196] exist Figure 5 In , a pair of freehand ultrasound scans of TRX and SAG views can be provided as input. Figure 5 Left side 501 and 502 show: (A) ("2D scan TRX", i.e. transverse scan) and (B) ("2D scan SAG", i.e. sagittal scan). This can include Figure 1 In the method of steps 102 and 104. In an illustrative example, the thyroid image can be captured once by the ultrasound imaging device from a first view (TRX plane) and then from a second view (SAG plane), for example similar to Figure 4 (a) and (b) are shown and described.

[0197] In addition, two transformation matrices T can be provided sag and T trx As output (see Figure 5 503 and 504 on the right) are used to scan any point (x trx ,y trx ) and frame index idx trx Mapped to the corresponding point (x sag ,y sag ) and frame index idx sag , and vice versa. This can include Figure 1 In steps 112, 120 and 122 of the method.

[0198] The mapping (cross-reference) can be expressed by the following formula (1) and formula (2):

[0199] (idx sag ,y sag ,x sag ) = f sag (T trx ,T sag ,ws sag ,idx trx ,y trx ,x trx ,) (1)

[0200] (idx trx ,y trx ,x trx ) = f trx (T trx ,T sag ,wstrx ,idx sag ,y sag ,x sag ) (2)

[0201] Here (idx trx ,y trx ,x trx ) represents a point in the transverse (TRX) view, (idx sag ,y sag ,x sag ) represents the corresponding (i.e., cross-reference) point in the sagittal (SAG) view, T sag and T trx is the corresponding transformation matrix, f sag and f trx is the transformation matrix T sag and T trx and the scaling factor ws trx 、ws sag The mapping function maps positions from the TRX view to the SAG view and vice versa.

[0202] method

[0203] The basic task of various embodiments of the present invention can be seen as matching points of a 3D anatomical structure from two orthogonal (or substantially orthogonal) views, in particular, Figure 5 TRX and SAG views are shown in diagram 500. According to various embodiments, the 3D pixel-to-pixel cross-referencing problem may be decomposed into two parallel co-registration problems that may be solved simultaneously.

[0204] From TRX to SAG: Points of the TRX view scan are cross-referenced with corresponding points on the SAG view scan (upper half of diagram 500).

[0205] To use common (shared) information from two views, you can Figure 2 , hereinafter referred to as SAG) scan to reconstruct the first view (view Figure 1 The simulated view is a “simulated view” of the target anatomical structure in the first view (hereinafter referred to as TRX), and vice versa. Figure 1 ) in the second view (view Figure 2 ) is obtained by the projection in .

[0206] After some post-processing of the simulated views, the simulated TRX can be co-registered with the original TRX to calculate the corresponding transformation matrix T trx .

[0207] Similarly, the 2D scan in TRX can generate a simulated SAG and calculate the corresponding transformation matrix T sag .

[0208] Refer again Figure 5 A more detailed explanation of various aspects of the method will now be given with reference to the flowchart 500 shown. In the flowchart 500, each step is represented by an arrow and an associated number, such as ①, ②, ③, ④, ..., etc. It should be understood that the numbering does not necessarily mean a temporal order, because, for example, some steps (such as steps ① and ④) can be performed sequentially or simultaneously, and / or some steps marked with "higher" numbers (such as step ①) can also be performed before or simultaneously with steps marked with "lower" numbers (such as step ②). In addition, the arrows (steps) in the flowchart 500 can generally represent the transition between different stages of the process, and each stage is represented by an image marked with capital letters, such as "A", "B", "C", etc.

[0209] Figure 5 The various steps shown in the flowchart 500 can be described as follows:

[0210] Step ①: Segment the target anatomical structure in the first view (TRX view): Segment each frame of the TRX view of the 2D scan of the target anatomical structure (see (A) with an image size of (height H trx , width W trx ), the number of TRX frames is N trx ) is fed into a segmentation method (e.g., a U-Net segmentation network) to obtain a corresponding binary segmentation mask (C). The pixel value in the segmentation mask is 1 (corresponding to the thyroid region) or 0 (corresponding to the non-thyroid region). For illustration, the pixels corresponding to the target anatomical structure (thyroid region) in (C) are white, while the pixels corresponding to the non-thyroid region are black. Step ① may include Figure 1 In step 106 of method 100 shown.

[0211] Step ②: Find the maximum (optimal) TRX segmentation mask (G) in the generated mask (C) representing the target anatomical structure. The term "optimal" here refers to the maximum segmentation mask in the segmentation mask of the target anatomical structure. Step ② may include Figure 1 In step 106 of method 100 shown.

[0212] Step ③: Generate a signed distance map (K) based on the segmentation mask (G). In order to obtain the signed distance map (K), first, the boundary contour of the segmentation mask (G) (that is, the boundary contour of the target anatomical structure (thyroid region) in the segmentation mask (G)) can be determined. The value of each pixel in the distance map is the distance from the pixel to its nearest contour pixel. The pixel value corresponding to the contour pixel in the distance map is zero. In order to obtain a signed distance map, the thyroid segmentation mask (G) can be converted into a signed distance map (hereinafter referred to as "signed (G)"), in which the values ​​of thyroid pixels and non-thyroid pixels are -1 and 1, instead of 0 and 1. The signed distance map (K) is then generated by element-by-element multiplication of the sign (G) and the distance map. Step ③ may include Figure 1 In step 110 of method 100 shown.

[0213] Step ④ Segmentation of target anatomical structure in the second view (SAG view): sag ,W sag ,N sag ) is input into the SAG segmentation method to obtain the corresponding binary segmentation mask in the SAG view (D). Similar to (C), the pixels corresponding to the target anatomical structure (thyroid region) in (D) are white, and the pixels corresponding to the non-thyroid region are black. Step ④ can include Figure 1 In step 114 of method 100 shown.

[0214] Step ⑤: Generate simulated TRX segmentation and extract the shape (H) from the generated SAG mask (D) sag ,N sag ) of the maximum simulated TRX segmentation mask (H). Step ⑤ may include Figure 1 In step 108 of method 100 shown.

[0215] Step ⑥: Estimate the width (N) of the simulated TRX mask (H) sag ) is a rough scaling factor ws trx The rough estimate is based on the fact that the width-to-height ratio (WoH) of the thyroid gland in the real TRX frame (G) and the simulated TRX frame (I) should be the same.

[0216] In ultrasound 2D scanning, the in-plane pixel spacing (width and height) within a scan frame is known, while the out-of-plane pixel spacing between two consecutive frames is unknown, which mainly depends on the moving direction and speed of the ultrasound probe. This means that the height (H) of the simulated TRX mask is sag ) is probably almost correct, and the width (N sag ) may need to be scaled by a factor (ws trx ) is scaled. The goal of this step is to estimate the width (N) of the simulated TRX mask (H)sag ) is a rough scaling factor ws trx This rough estimate is based on the fact that the width / height ratio (WoH) of the thyroid gland in the real TRX frame (G) and the simulated TRX frame (I) should be the same. Step ⑥ may include Figure 1 In step 110 of method 100 shown.

[0217] Step ⑦: Extract and sample contour pixels (M) from the simulated TRX segmentation mask (I). Given a mask (I), first extract its contours and then retain the longest contour. The longest contour obtained is a pixel-wide chain whose pixel coordinates are sorted and stored in a matrix of shape (2, num_pixels), where num_pixels represents the number of pixels in the longest contour. Then, a subset of the obtained num_pixels contour pixels (for example, 50) can be sampled for further edge-based alignment in step ⑩. In (M), the sampled contour pixels are displayed as white dots. Step ⑦ may include Figure 1 In step 110 of method 100 shown.

[0218] Step ⑧: Sample the anchor pixel (L) in region (I). Since there are a large number of foreground pixels in (I), a subset of them is sampled. For example, foreground pixels whose row index and column index can be divided by a given number n (for example, but not limited to, n=25) can be retained. In (L), the sampled pixels are displayed as white dots (L). Step ④ can include Figure 1 In step 110 of method 100 shown.

[0219] Step ⑨: Estimate the rough offset (δR) between the thyroid masks in (I) and (G) by maximizing the alignment response calculated for (L) and (K) using the following formula (3): trx , δC trx ):

[0220] arg max δR,δC ∑K((r+δR trx, c+δC trx )|L(r,c)>0) (3)

[0221] Where (r,c) is the row and column number of the foreground pixel in (L). Step ⑨ can be included in Figure 1 In step 110 of method 100 shown.

[0222] Step ⑩: Using formula (4), define the relative transformation matrix T by minimizing trx The alignment cost is the sum of the squared distances, estimating the scale and translation between the segmentation masks in (I) and (G):

[0223]

[0224] Where J is the distance map (such as Figure 6 ), W is a point on the boundary of the target anatomical structure (i.e. Figure 5 The white point on the image (M) is transformed by T trx After the distortion.

[0225] Therefore, the transformation matrix T can be obtained trx Step ⑩ may include Figure 1 In step 112 of method 100 shown.

[0226] The optimization problem (ie, minimizing the alignment cost) may be solved, for example, by a Gauss-Newton algorithm, for example in the same or similar manner as shown in the pseudo code given below ("Algorithm 1: Optimization by Gauss-Newton method").

[0227] In pseudocode, deltaR trx Corresponding to the above δR trx , deltaC trx Corresponding to the above δC trx , W represents the transformation T trx Distorted boundary points.

[0228]

[0229] Algorithm 1:

[0230] From SAG to TRX: (lower half of the 500 chart)

[0231] These steps are similar to the steps in the upper portion of Figure 500, but are completed from SAG to TRX.

[0232] step From having (H sag , W sag , N sag ) of the (overall) shape of the SAG segmentation mask (D) and extract the (H sag , W sag ) of the shape of the best ground-truth SAG view segmentation mask (Q). This step is similar to step ②. Can be included in Figure 1 In step 114 of method 100 shown.

[0233] step Generate a signed distance map (T) based on the mask boundary of (Q). The distance map generation algorithm is the same as the algorithm used in ③. Can be included in Figure 1 In step 118 of method 100 shown.

[0234] step From having (H trx ,W trx ,N trx ) of the (overall) shape of the TRX segmentation mask (D) and extract the (H trx ,N trx The best simulated SAG view segmentation mask (P) of the shape of ). The best frame extraction algorithm is the same as step ⑤. Step Can be included in Figure 1 In step 116 of method 100 shown.

[0235] step Estimate the rough scale factor ws of the simulated SAG mask (P) along its width sag Then according to ws sag Adjust (P) to (R). This rough estimate is based on the fact that the aspect ratio (WoH) of the thyroid gland in the real SAG frame (Q) and the simulated SAG frame (P) should be the same. Step Can be included in Figure 1 In step 118 of method 100 shown.

[0236] step Similar to step ⑦, the boundary pixels (contour pixels) (V) are extracted from the selected simulated SAG thyroid segmentation mask (R) and sampled. In (V), the sampled pixels are shown as white dots. Can be included in Figure 1 In step 118 of method 100 shown.

[0237] step Sample the regional anchor pixel (U) of (R). The sampling strategy is the same as that used in step ⑧. In (U), the sampled pixels are displayed as white dots. Step Can be included in Figure 1 In step 118 of method 100 shown.

[0238] step The rough offset (δR) between the thyroid masks in (T) and (Q) is estimated by maximizing the alignment response calculated for (U) and (T) sag , δC sag ). This step is the same as the one used in step ⑨. Can be included in Figure 1 In step 118 of method 100 shown.

[0239] step Using the following formula (5), by minimizing the relative transformation matrix T defined as sagThe alignment cost is the sum of the squared distances, estimating the scale and translation between the segmentation masks of (R) and (Q):

[0240]

[0241] Where S is the distance map (such as Figure 6 As shown), W is the point on the boundary of the target anatomical structure (i.e. Figure 5 The white point on the image (V) is transformed by T sag of distortion.

[0242] Solve the optimization problem by the same algorithm as used in step ⑩. Thus, the transformation matrix T can be obtained sag .step Can be included in Figure 1 In step 120 of method 100 shown.

[0243] Final mapping equation

[0244] With the calculation of T trx and T sag , any point in the TRX scan (frame index idx trx and coordinates (x trx ,y trx )) can be mapped to the corresponding point (frame index idx) of the SAG scan using the following formulas (6) to (7) sag and coordinates (x sag ,y sag )):

[0245] From TRX(idx trx ,y trx , x trx ) to SAG(idx sag ,y sag ,x sag )’s mapping:

[0246] idx sag = int((x trx -T trx (2,3)) / (T trx (1, 1)*ws trx )+0.5)

[0247] y sag = int((y trx -T trx (2,3)) / T trx (2, 2) + 0.5)

[0248] x sag = int(idx trx *ws sag *Tsag (1, 1) + T sag (2, 3) + 0.5) (6)

[0249] From SAG(idx sag ,y sag , x sag ) to TRX(idx trx ,y trx , x trx )’s mapping:

[0250] idx trx = int((x sag -T sag (2,3)) / (T sag (1,1)*ws sag )+0.5)

[0251] y trx = int(y sag *T trx (2, 2)+T trx (2,3)+0.5)

[0252] x trx = int(idx sag *ws trx *T trx (1, 1) + T trx (2,3)+0.5) (7)

[0253] This can include Figure 1 In step 122 of the method 100 shown in FIG. 1 , a cross-reference between two views can be achieved. Exemplarily, formula (6) corresponds to an embodiment of the above formula (1), and formula (7) corresponds to an embodiment of the above formula (2).

[0254] Figure 6 Another flowchart 600 is shown that illustrates various aspects of a cross-referencing method for 2D ultrasound scanning according to various embodiments (eg, Figure 1 100), and Figure 7 and Figure 8 Shown in more detail Figure 6 The portion of the flowchart shown in .

[0255] Flowchart 600 is somewhat similar to Figure 5 500 shown in FIG. 5 , and in particular, the steps and stages in flowchart 600 are similar to those in FIG. Figure 5 The same reference numerals or labels as those in the flowchart 500 in FIG. 5 are used, and therefore, will not be described in detail here. Instead, the following will mainly describe Figure 5 The differences compared to flow chart 500 are shown.

[0256] refer to Figure 6 and Figure 7 In flowchart 600, step ⑤ (flowchart 500) may include steps (5a) and (5b), so that the transition from stage (D) to stage (H) is performed through the intermediate stage (F). Figure 7 It is shown that the intermediate stage (F) obtained by step (5a) corresponds to the maximum simulated TRX segmentation mask in the sagittal (SAG) segmentation scan, including a (small) false positive area (indicated by reference numeral 701), i.e., an area in the segmentation mask (F) that is incorrectly marked as corresponding to the target anatomical structure (thyroid lobe). This false positive area 701 can be removed by post-processing.

[0257] Due to the structural complexity of the thyroid region in the selected frame and the relatively low signal-to-noise ratio of the ultrasound image, there may sometimes be false positive thyroid segmentation areas in the selected frame (such as Figure 7 In various embodiments, as shown in step (5b), a maximum continuous component search algorithm can be used to exclude the selected thyroid segmentation mask that is not connected to the main thyroid segmentation ( Figure 7 702 in the region 704). That is, in step (5b), the maximum continuous component (also called the maximum continuous component, the maximum connected component or the maximum cluster) of the mask can be obtained, so that noise can be removed. The maximum continuous component search algorithm is known in the field of image processing and is therefore not described in detail here. As a non-limiting example of such an algorithm, the so-called OpenCV cv2.connectedComponentsWithStats algorithm can be used.

[0258] In a similar manner to step ⑤ above, step Can include steps (13a) and (13b) in flowchart 600, wherein step (13a) produces a maximum simulated SAG segmentation mask from a lateral (TRX) segmentation scan (stage (N) in flowchart 600), and step (13b) produces a maximum component of the mask (stage (P) in flowchart 600).

[0259] Similarly, in a manner similar to step ⑤ above, step ② may include steps (2a) and (2b) in flowchart 600, wherein step (2a) produces a maximum TRX segmentation mask from a transverse (TRX) segmentation scan (stage (E) in flowchart 600), and step (2b) produces a maximum component of the mask (stage (G) in flowchart 600).

[0260] Finally, in a similar manner to step ⑤ above, step It can include steps (11a) and (11b) in flowchart 600, wherein step (11a) produces a maximum sagittal (SAG) segmentation mask from a SAG segmentation scan (stage (O) in flowchart 600), and step (11b) produces a maximum component of the mask (stage (Q) in flowchart 600).

[0261] Thus, the method shown in flowchart 600 may include further refining the binary segmentation masks (E), (F), (N), and (O) obtained from the TRX and SAG scans.

[0262] Reference Figure 6 and Figure 8 In flowchart 600, step ③ (flowchart 500) may include steps (3a) and (3b), so that the transition from stage (G) to stage (K) is performed through the intermediate stage (J).

[0263] In step (3a), a distance map (J) can be generated based on the mask boundary of the binary segmentation mask (G). Given the binary segmentation mask (G), the contour of the mask can be searched by a suitable contour search algorithm known in the art. The longest contour can then be retained. A distance map (J) can be generated based on this longest contour. The value of each pixel in the distance map (J) is the distance from the pixel to its nearest contour pixel. Points on the contour boundary have a value of zero in the distance map.

[0264] In step (3b), the distance map (J) may be converted into a signed distance map (K). To obtain the signed distance map (K), the binary segmentation mask (G) may first be converted into a signed map, where the values ​​of thyroid and non-thyroid pixels may be -1 and 1 instead of 0 and 1. Then, the signed distance map (K) may be generated by element-wise multiplication of the sign (G) and the distance map (J).

[0265] In a similar manner to step ③ above, step The method may include steps (12a) and (12b) in flowchart 600, wherein step (12a) generates a distance map (S) based on the mask boundary of the binary segmentation mask (Q), and step (12b) converts the distance map (S) into a signed distance map (T).

[0266] According to various embodiments, one or more of the following steps may optionally be included in flowchart 500 or flowchart 600 to further enhance cross-referencing:

[0267] In the optional step (not shown), the initial transformation matrix T can be used sag and T trx The in-plane offset between the real and simulated images is further refined.

[0268] This step may include finding the per-frame in-plane offset (RC trx , the shape is (2, N trx )). Ideally, the scanning trajectory of both TRX and SAG scans should be straight. However, it may be difficult for sonographers and radiologists to strictly follow a straight line when scanning TRX scans. Therefore, a module for estimating the per-frame in-plane offset of TRX frames can be added to refine the registration of each pair of real TRX frames and simulated TRX frames.

[0269] In particular, for each TRX segmentation mask ((E'), the shape is (H trx , W trx )) model, and its corresponding simulated TRX mask ((F') with a shape of (H sag , N sag )) can be retrieved by the last part of formula (6).

[0270] Then (G') and (L') can be obtained according to step (2b) and step ⑧, respectively. Then, by performing an alignment based on a heuristic search algorithm for (G') and (L'), the offset of the TRX frame (F') relative to its corresponding real TRX frame (E') can be obtained, which is similar to the algorithm used in step ⑨ (here, the signed distance map (K) is replaced by the binary mask (G')).

[0271] Based on the in-plane offset (RC trx ), we can get the RC of the real SAG frame relative to its corresponding simulated SAG frame sag , used for inverse transformation.

[0272] In another optional step (not shown), step The T obtained in sag The refinement can be performed by performing an intensity-based Lucas-Kanade algorithm. This step is to reduce possible aperture issues caused by scanning when the target anatomy is not fully scanned (e.g., the thyroid gland is too long in the vertical direction to be covered by one frame in the SAG view). To reduce the impact of this problem, the corresponding images ((N') and (O')) of the simulated SAG mask (N) and the true SAG mask (O) can be first extracted from (A) and (B). Then, the scale factor ws can be used sag Resize (N') to (R') along its width. T can be refined by optimizing the sum of the squared differences between the two images (N') and (O') using the following formula: sag :

[0273]

[0274] If you perform the optional steps and steps The refined cross-reference equation is as follows:

[0275] The final mapping equation (including additional steps and ):

[0276] By Step and Get(ws trx , Ttrx, RCtrx) and (ws sag , T sag , R.C. sag ). Then, any point in the TRX scan (frame index idx) can be converted to trx and coordinates (x trx ,y trx )) is mapped to the corresponding point on the SAG scan (frame index idx sag and (x sag ,y sag )):

[0277] From TRX(idx trx ,y trx , x trx ) to SAG(idx sag ,y sag ,x sag )’s mapping:

[0278] idx sag = int((x trx -T trx (2,3)-RC trx [1, idx trx ]) / (T ttx (1, 1)*ws trx )+0.5)

[0279] y sag = int((y trx -T trx (2,3)-RC trx [0, idx trx ]) / T trx (2, 2) + 0.5)

[0280] x sag = int(idx trx *ws sag *T sag (1, 1) + T sag (2, 3) + 0.5) (9)

[0281] From SAG(idx sag ,y sag , x sag ) to TRX(idx trx ,y trx , x trx )’s mapping:

[0282] idx trx = int((x sag -T sag (2,3)) / (T sag (1, 1)*ws sag )+0.5)

[0283] y trx = int(y sag *T trx (2, 2)+T trx (2,3)+RC sag [0, x sag ]+0.5)

[0284] x trx = int(idx sag *ws trx *T trx (1, 1)+T trx (2,3)+RC sag [1, x sag ]+0.5) (10)

[0285] For example, equation (9) corresponds to another embodiment of equation (1) above, and equation (10) corresponds to another embodiment of equation (2) above. Therefore, cross-reference between two views (TRX and SAG) can be achieved.

[0286] Fig. 9 Screen shots depicting a cross-referencing tool or system from a first view to a second view according to various embodiments of the present invention: by selecting any point on the transverse scan (e.g., the cursor pointer on a thyroid nodule in the lower lobe of the thyroid gland on the transverse view (point P1) in the left panel), the cross-referencing tool or system locates the corresponding nodule on the sagittal scan (i.e., point P2 on the right panel with corresponding x, y position and frame number).

[0287] Fig.10 Screen shots depicting the cross-referencing system from the second view to the first view according to various embodiments of the present invention: by selecting another thyroid nodule on the superior lobe of the thyroid gland on the sagittal scan (point P3 on the right panel), the cross-referencing tool locates the nodule (corresponding x, y positions and frame numbers) in the transverse scan (point P4 on the left panel).

[0288] According to one aspect of the present invention, a method and system for cross-referencing ultrasound scans from two orthogonal planes based on a segmentation mask is provided.

[0289] According to another aspect of the present invention, a non-transitory computer-readable storage medium is provided having instructions for execution by a processor to: receive an ultrasound scan pair; apply the ultrasound scan pair to a series of transformations and post-processing; and generate a transformation map from the paired two scans to form a final probabilistic segmentation.

[0290] According to various embodiments, a (computer-implemented) method or algorithm may be provided that may compute a transformation that outputs a specific frame index and coordinate within a second view (e.g., a sagittal view) based on the frame index and coordinates provided by a user on a first view (e.g., a transverse view). The algorithm may use a linear transformation to place the transverse and sagittal image pairs into the same spatial coordinate system. This may allow a user to locate a region of interest (e.g., a nodule) in a sagittal image using a region of interest defined in a transverse image.

[0291] According to various embodiments, the redundancy of two perpendicular freehand ultrasound scan acquisitions can be exploited: out-of-plane distances in one scan can be accurately recovered from the other scan since they appear in-plane. To do this, various embodiments provide a method to register the two acquisitions.

[0292] According to various embodiments, a cross-referencing module may be provided that computes two transformation matrices and then maps any frame index and coordinates provided by the user on the transverse image to frame index and coordinates within the sagittal (longitudinal) image.

[0293] The mapping is based on two transformation matrices that can be used to place transverse and sagittal image pairs into the same spatial coordinate system, allowing transverse viewpoints to be mapped to sagittal viewpoints and vice versa.

[0294] Although the embodiments of the present invention have been specifically shown and described with reference to specific embodiments, it should be understood by those skilled in the art that various changes may be made in form and detail without departing from the spirit and scope of the present invention as defined by the appended claims. Therefore, the scope of the present invention is indicated by the appended claims, and therefore, all changes within the meaning and scope of the equivalents of the claims should be accepted.

Claims

1. A method for cross-referencing a two-dimensional (2D) ultrasound scan of a tissue volume, the method comprising: generating a first series of 2D ultrasound images of a tissue volume associated with a first plurality of locations, respectively, along a first scanning direction of the tissue volume, the tissue volume including a target anatomical structure; generating a second series of 2D ultrasound images of the tissue volume respectively associated with a second plurality of locations along a second scanning direction of the tissue volume, wherein the second scanning direction is at least substantially orthogonal to the first scanning direction; generating a first 2D representation of the target anatomical structure from the first series of 2D ultrasound images, the first 2D representation being associated with one of the images within the first series of 2D ultrasound images; generating a first simulated 2D representation of the target anatomical structure from the second series of 2D ultrasound images, the first simulated 2D representation of the target anatomical structure and the first 2D representation corresponding to each other in a direction in which the target anatomical structure is viewed; processing the first 2D representation and the first simulated 2D representation of the target anatomical structure so that pixel locations associated with the target anatomical structure in the first simulated 2D representation at least substantially match pixel locations associated with the target anatomical structure in the first 2D representation; determining a first corresponding transformation matrix based on the processed first 2D representation and the processed first simulated 2D representation of the target anatomical structure; generating a second 2D representation of the target anatomical structure from a second series of 2D ultrasound images, the second 2D representation being associated with one of the images within the second series of 2D ultrasound images; generating a second simulated 2D representation of the target anatomical structure from the first series of 2D ultrasound images, the second simulated 2D representation and the second 2D representation of the target anatomical structure corresponding to each other in a direction in which the target anatomical structure is viewed; processing a second 2D representation and a second simulated 2D representation of the target anatomical structure so that pixel locations associated with the target anatomical structure in the second simulated 2D representation at least substantially match pixel locations associated with the target anatomical structure in the second 2D representation; determining a second corresponding transformation matrix based on the processed second 2D representation of the target anatomical structure and the processed second simulated 2D representation; and Using the first corresponding transformation matrix and the second corresponding transformation matrix, corresponding second positions associated with the target anatomical structure in the first series of 2D ultrasound images are determined for the first positions associated with the target anatomical structure in the second series of 2D ultrasound images.

2. The method according to claim 1, wherein: Determining the second position using the first corresponding transformation matrix and the second corresponding transformation matrix includes: A first function of the first corresponding transformation matrix and the second corresponding transformation matrix is ​​applied to the pixel coordinates corresponding to the first position to obtain the pixel coordinates corresponding to the second position.

3. The method according to claim 1 or 2, further comprising: A corresponding fourth position associated with the target anatomical structure in the second series of 2D ultrasound images is determined for a third position associated with the target anatomical structure in the first series of 2D ultrasound images using the first corresponding transformation matrix and the second corresponding transformation matrix.

4. The method according to claim 3, wherein: Determining the fourth position using the first corresponding transformation matrix and the second corresponding transformation matrix includes: A second function of the first corresponding transformation matrix and the second corresponding transformation matrix is ​​applied to the pixel coordinates corresponding to the third position to obtain pixel coordinates corresponding to a fourth position.

5. The method according to any one of claims 1 to 4, further comprising: generating a first plurality of binary segmentation masks from the first series of 2D ultrasound images, each binary segmentation mask of the first plurality of binary segmentation masks corresponding to a respective one of the 2D ultrasound images within the first series of 2D ultrasound images; as well as generating a second plurality of binary segmentation masks from the second series of 2D ultrasound images, each binary segmentation mask of the second plurality of binary segmentation masks corresponding to a respective one of the 2D ultrasound images within the second series of 2D ultrasound images, wherein a first 2D representation of the target anatomical structure is generated from one of the binary segmentation masks within the first plurality of binary segmentation masks, and Wherein a second 2D representation of the target anatomical structure is generated from one of the binary segmentation masks within the second plurality of binary segmentation masks.

6. The method according to claim 5, wherein: The first 2D representation of the target anatomical structure corresponds to a maximum binary segmentation mask among the first plurality of binary segmentation masks; as well as The second 2D representation of the target anatomical structure corresponds to a maximum binary segmentation mask among the second plurality of binary segmentation masks.

7. The method according to claim 5 or 6, wherein: generating a first simulated 2D representation of the target anatomical structure from the second series of 2D ultrasound images, comprising generating a first simulated 2D representation of the target anatomical structure from the second plurality of binary segmentation masks; and Generating a second simulated 2D representation of the target anatomical structure from the first series of 2D ultrasound images includes generating a second simulated 2D representation of the target anatomical structure from the first plurality of binary segmentation masks.

8. The method according to any one of claims 5 to 7, wherein: Processing the first 2D representation includes generating a first signed distance map from the first 2D representation; and Processing the second 2D representation includes generating a second signed distance map from the second 2D representation.

9. The method according to claim 8, wherein: processing the first simulated 2D representation, including scaling a width of the first simulated 2D representation to match a width of the first 2D representation; and Processing the second simulated 2D representation includes scaling a width of the second simulated 2D representation to match a width of the second 2D representation.

10. The method according to claim 9, wherein: processing the first simulated 2D representation, further comprising determining and sampling contour pixels associated with a contour of the target anatomical structure from the scaled first simulated 2D representation, and Processing the second simulated 2D representation further includes determining and sampling contour pixels associated with a contour of the target anatomical structure from the scaled second simulated 2D representation.

11. The method according to claim 10, wherein: Processing the first simulated 2D representation further comprises sampling a first subset of pixels from the scaled first simulated 2D representation; and Processing the second simulated 2D representation further includes sampling a second subset of pixels from the scaled second simulated 2D representation.

12. The method according to claim 11, wherein: processing the first analog 2D representation, further comprising estimating a coarse offset between the scaled first analog 2D representation and the first 2D representation using the first signed distance map and the sampled first subset of pixels; as well as Processing the second analog 2D representation further includes estimating a coarse offset between the scaled second analog 2D representation and the second 2D representation using the second signed distance map and the sampled second subset of pixels.

13. The method according to claim 12, wherein: determining the first corresponding transformation matrix comprises estimating a scale and a translation between the scaled first simulated 2D representation and the first 2D representation by minimizing a first alignment cost, the first alignment cost being defined as a first cost function with respect to the first corresponding transformation matrix; as well as Determining the second corresponding transformation matrix includes estimating a scale and a translation between the scaled second simulated 2D representation and the second 2D representation by minimizing a second alignment cost, the second alignment cost being defined as a second cost function with respect to the second corresponding transformation matrix.

14. The method according to any one of claims 1 to 13, wherein: The first scanning direction corresponds to a transverse scan of the tissue volume, and the second scanning direction corresponds to a sagittal scan of the tissue volume.

15. A cross-referencing system for two-dimensional (2D) ultrasound scanning of a tissue volume, the system comprising: Memory; as well as at least one processor, communicatively coupled to the memory, and configured to: generating a first series of 2D ultrasound images of a tissue volume associated with a first plurality of locations, respectively, along a first scanning direction of the tissue volume, the tissue volume including a target anatomical structure; generating a second series of 2D ultrasound images of the tissue volume respectively associated with a second plurality of locations along a second scanning direction of the tissue volume, wherein the second scanning direction is at least substantially orthogonal to the first scanning direction; generating a first 2D representation of the target anatomical structure from the first series of 2D ultrasound images, the first 2D representation being associated with one of the images within the first series of 2D ultrasound images; generating a first simulated 2D representation of the target anatomical structure from the second series of 2D ultrasound images, the first simulated 2D representation of the target anatomical structure and the first 2D representation corresponding to each other in a direction in which the target anatomical structure is viewed; processing the first 2D representation and the first simulated 2D representation of the target anatomical structure so that pixel locations associated with the target anatomical structure in the first simulated 2D representation at least substantially match pixel locations associated with the target anatomical structure in the first 2D representation; determining a first corresponding transformation matrix based on the processed first 2D representation and the processed first simulated 2D representation of the target anatomical structure; generating a second 2D representation of the target anatomical structure from a second series of 2D ultrasound images, the second 2D representation being associated with one of the images within the second series of 2D ultrasound images; generating a second simulated 2D representation of the target anatomical structure from the first series of 2D ultrasound images, the second simulated 2D representation and the second 2D representation of the target anatomical structure corresponding to each other in a direction in which the target anatomical structure is viewed; processing a second 2D representation and a second simulated 2D representation of the target anatomical structure so that pixel locations associated with the target anatomical structure in the second simulated 2D representation at least substantially match pixel locations associated with the target anatomical structure in the second 2D representation; determining a second corresponding transformation matrix based on the processed second 2D representation of the target anatomical structure and the processed second simulated 2D representation; and Using the first corresponding transformation matrix and the second corresponding transformation matrix, corresponding second positions associated with the target anatomical structure in the first series of 2D ultrasound images are determined for the first positions associated with the target anatomical structure in the second series of 2D ultrasound images.

16. The system of claim 15, wherein: When the at least one processor determines the second position using the first corresponding transformation matrix and the second corresponding transformation matrix, the at least one processor is configured to: A first function of the first corresponding transformation matrix and the second corresponding transformation matrix is ​​applied to the pixel coordinates corresponding to the first position to obtain the pixel coordinates corresponding to the second position.

17. The system according to claim 15 or 16, wherein: The at least one processor is further configured to: A corresponding fourth position associated with the target anatomical structure in the second series of 2D ultrasound images is determined for a third position associated with the target anatomical structure in the first series of 2D ultrasound images using the first corresponding transformation matrix and the second corresponding transformation matrix.

18. The system of claim 17, wherein: When the at least one processor determines the fourth position using the first corresponding transformation matrix and the second corresponding transformation matrix, the at least one processor is configured to: A second function of the first corresponding transformation matrix and the second corresponding transformation matrix is ​​applied to the pixel coordinates corresponding to the third position to obtain pixel coordinates corresponding to a fourth position.

19. A system according to any one of claims 15 to 18, wherein: The at least one processor is further configured to: generating a first plurality of binary segmentation masks from the first series of 2D ultrasound images, each binary segmentation mask of the first plurality of binary segmentation masks corresponding to a respective one of the 2D ultrasound images within the first series of 2D ultrasound images; as well as generating a second plurality of binary segmentation masks from the second series of 2D ultrasound images, each binary segmentation mask of the second plurality of binary segmentation masks corresponding to a respective one of the 2D ultrasound images within the second series of 2D ultrasound images, wherein a first 2D representation of the target anatomical structure is generated from one of the binary segmentation masks within the first plurality of binary segmentation masks, and Wherein a second 2D representation of the target anatomical structure is generated from one of the binary segmentation masks within the second plurality of binary segmentation masks.

20. The system of claim 19, wherein: The first 2D representation of the target anatomical structure corresponds to a maximum binary segmentation mask among the first plurality of binary segmentation masks; And the second 2D representation of the target anatomical structure corresponds to a maximum binary segmentation mask among the second plurality of binary segmentation masks.

21. A system according to any one of claims 15 to 20, wherein: The first scanning direction corresponds to a transverse scan of the tissue volume and the second scanning direction corresponds to a sagittal scan of the tissue volume.

22. The system of any one of claims 15 to 21, further comprising an ultrasound transducer communicatively coupled to the memory and the at least one processor, wherein: The at least one processor is configured to: generating the first series of 2D ultrasound images based on a first series of ultrasound waves acquired by the ultrasound transducer positioned at a first plurality of locations for a first plurality of time instances; as well as The second series of 2D ultrasound images is generated based on a second series of ultrasound waves acquired by the ultrasound transducer positioned at a second plurality of locations for a second plurality of time instances.

23. A computer program product, embodied in one or more non-transitory computer-readable storage media, comprising instructions executable by at least one processor to implement a method of cross-referencing a two-dimensional (2D) ultrasound scan of a tissue volume, the method comprising: generating a first series of 2D ultrasound images of a tissue volume associated with a first plurality of locations, respectively, along a first scanning direction of the tissue volume, the tissue volume including a target anatomical structure; generating a second series of 2D ultrasound images of the tissue volume respectively associated with a second plurality of locations along a second scanning direction of the tissue volume, wherein the second scanning direction is at least substantially orthogonal to the first scanning direction; generating a first 2D representation of the target anatomical structure from the first series of 2D ultrasound images, the first 2D representation being associated with one of the images within the first series of 2D ultrasound images; generating a first simulated 2D representation of the target anatomical structure from the second series of 2D ultrasound images, the first simulated 2D representation of the target anatomical structure and the first 2D representation corresponding to each other in a direction in which the target anatomical structure is viewed; processing the first 2D representation and the first simulated 2D representation of the target anatomical structure so that pixel locations associated with the target anatomical structure in the first simulated 2D representation at least substantially match pixel locations associated with the target anatomical structure in the first 2D representation; determining a first corresponding transformation matrix based on the processed first 2D representation and the processed first simulated 2D representation of the target anatomical structure; generating a second 2D representation of the target anatomical structure from a second series of 2D ultrasound images, the second 2D representation being associated with one of the images within the second series of 2D ultrasound images; generating a second simulated 2D representation of the target anatomical structure from the first series of 2D ultrasound images, the second simulated 2D representation and the second 2D representation of the target anatomical structure corresponding to each other in a direction in which the target anatomical structure is viewed; processing a second 2D representation and a second simulated 2D representation of the target anatomical structure so that pixel locations associated with the target anatomical structure in the second simulated 2D representation at least substantially match pixel locations associated with the target anatomical structure in the second 2D representation; determining a second corresponding transformation matrix based on the processed second 2D representation of the target anatomical structure and the processed second simulated 2D representation; and Using the first corresponding transformation matrix and the second corresponding transformation matrix, corresponding second positions associated with the target anatomical structure in the first series of 2D ultrasound images are determined for the first positions associated with the target anatomical structure in the second series of 2D ultrasound images.