Computer-implemented method for visualizing an elongated anatomical structure

Abstract

A computer implemented method for visualizing using ultrasound, for example, an elongated anatomical structure (20) of a fetal spine is provided. The method comprises the steps of: receiving a plurality of 3D ultrasound image volumes, each image volume depicting at least a portion of the elongated anatomical structure (20); on each 3D ultrasound image volume, automatically or semi-automatically fitting a parametric curve (30) to the portion of the depicted elongated anatomical structure, the parametric curve being defined by curve parameters; reconstructing each 3D ultrasound image volume by applying a transformation that straightens the parametric curve along at least one axis, thereby generating a plurality of reconstructed image volumes and a reconstructed parametric curve (32, 34); registering the reconstructed image volumes with each other by determining connection points of their respective parametric curves; and, fusing the reconstructed image volumes with each other to produce a fused image that depicts the entire elongated anatomical structure or a larger portion thereof than the 3D ultrasound image volumes.

Description

Technical Field

[0001] The present invention relates to a computer-implemented method for visualizing slender anatomical structures, specifically the fetal spine, using ultrasound, a computer program, an image evaluation device configured to perform the method of the present invention, and an ultrasound system. Background Technology

[0002] Ultrasound (US) is the most widely used diagnostic imaging modality for visualizing the fetus. The ISUOG (International Society of Ultrasound in Obstetrics & Gynecology) and AIUM (American Institute of Ultrasound in Medicine) guidelines recommend at least three ultrasound examinations during the first, second, and third trimesters of pregnancy, with additional scans typically performed for high-risk pregnancies.

[0003] In this context, early detection of fetal spinal abnormalities allows for parental counseling and appropriate obstetric management. Typically, the fetal spine consists of the following segments: cervical (7 vertebrae), thoracic (12 vertebrae), lumbar (5 vertebrae), sacral (5 vertebrae), and terminates at the coccyx. A thorough evaluation of the fetal spine and all its vertebrae is necessary during any obstetric ultrasound examination, as the discovery of spinal abnormalities can significantly impact obstetric management.

[0004] A preliminary screening test for fetal structural abnormalities, such as vertebral deformities (hemievertebrae, sphenoid vertebrae, etc.), neural tube defects (spina bifida and other myelomeningoceles), and fetal scoliosis, is an anatomical ultrasound with detailed fetal spinal imaging and an assessment performed in the second trimester. With the right tools, some abnormalities can be detected in early pregnancy, but not all abnormalities are due to poor early spinal ossification.

[0005] Fetal scoliosis is a complex congenital malformation associated with many congenital abnormalities. The prognosis for a fetus with abnormal spinal curvature is generally determined by the prognosis of the underlying malformation and is usually poor. Due to the poor prognosis of many associated abnormalities, up to 60% choose to terminate the pregnancy. The middle of the second trimester to the early third trimester is the ideal time to identify abnormal spinal curvature because there is sufficient ossification of the spine, unlike the crowded fetus in late pregnancy that makes imaging of the spine difficult. Detection requires careful longitudinal scanning back and forth in the coronal and sagittal planes. However, early detection of scoliosis in the second and third trimesters is often challenging because a longitudinal image of the entire spine cannot be obtained with the limited field of view of the ultrasound probe.

[0006] As mentioned above, early detection of spinal abnormalities is a challenge. Ultrasound examination of the fetal spine of interest in early pregnancy is not part of the ISUOG or AIUM guidelines, primarily because the ossification and size of the fetal spine at this gestational age are poor, hindering ultrasound examination. Furthermore, poor visualization of the fetal spine in early pregnancy has been reported in approximately 15% of cases due to maternal habit and unfavorable fetal position. Spulveda et al. reviewed the ultrasound features of fetal spinal abnormalities in early pregnancy in “Fetal spinal anomalies in a first-trimester sonographic screeing program for aneuploidy” (Prenat Diagn, January 2011; 31(1):107-14) and found that the types of spinal abnormalities detected differed when comparing early and mid-pregnancy. Spina bifida was the most frequently detected abnormality in mid-pregnancy to date, while severe kyphosis was the most frequently detected abnormality in early pregnancy studies. Their early pregnancy abnormalities were associated with poorer perinatal outcomes. Almost all cases of body stalk abnormalities, but less than half of spina bifida cases, are detected in early pregnancy. Spinal length has been found to be a good indicator of fetal growth and is highly correlated with CRL and biparietal diameter in early pregnancy. Therefore, ultrasound examination of the spine in mid- or late-pregnancy appears crucial. However, the limited field of view of the ultrasound probe makes it difficult to calculate fetal spinal length and identify and analyze different vertebral segments in mid- or late-pregnancy.

[0007] Therefore, it is necessary to use ultrasound in late pregnancy to effectively generate images of the entire fetal spine or most of the spine, which can be used for further detailed evaluation of the fetal spine.

[0008] US 2008 / 0287796 A1 discloses a method and system for visualizing the spine in 3D medical images. The spinal cord centerline is automatically determined within a 3D medical image volume, such as a CT volume. A reconstructed image volume is then generated based on the spinal cord centerline. The reconstructed image volume can be a straightened spine volume or a volume based on multiplanar reconstruction (MPR) that follows the natural curves of the spine. The reconstructed volume can be displayed as a 2D slice or a 3D volume rendering.

[0009] WO 2015 / 063632 A2 discloses a method comprising obtaining first image data including voxels representing a structure of interest. The structure of interest includes multiple distinct substructures. The method further includes segmenting a volume comprising only the first image data for each of the multiple distinct substructures. The method further includes creating a distinct local coordinate system for each of the distinct substructures for each of the volumes. The method further includes independent visual rendering of a reconstructed image set for each of the multiple distinct substructures to visualize the structure of interest. The reconstructed image set for the substructures includes corresponding segmented volumes from the segmented volumes and distinct cutting planes generated from the local coordinate systems for the substructures.

[0010] WO 2005 048198A1 relates to a method and corresponding apparatus for visualizing the tubular structure of an object using a 3D image dataset of the object. To provide more efficient and illustrative visualization, a method is proposed comprising the steps of: - generating and visualizing a curved plane reconstruction view from a symbolic path view of the tubular structure, the symbolic path view representing the tubular structure, and the path points of the symbolic paths being assigned their 3D spatial location data; and - generating and visualizing at least one planar view of the object by selecting an observation point of the tubular structure in the curved plane reconstruction view or the symbolic path view. However, the aforementioned problem remains, namely that the entire fetal spine or a large portion of the spine cannot be adequately displayed in a single image for analyzing various spinal deformities.

[0011] Purpose of the invention

[0012] Therefore, one object of the present invention is to provide a computer-implemented method for efficiently generating visualizations of elongated anatomical structures that cannot be visualized in a single ultrasound field of view, such as the fetal spine captured using ultrasound in the early stages of mid-to-late pregnancy and capable of being used for detailed malformation hypotheses. Another object of the present invention is to provide corresponding computer programs, image evaluation devices configured to perform embodiments of the method of the present invention, and ultrasound systems. Summary of the Invention

[0013] To better address one or more of the above-mentioned problems, in a first aspect of the present invention, a computer-implemented method for visualizing a fetal spine using ultrasound is provided, the method comprising the following steps:

[0014] - Receive multiple 3D ultrasound image volumes, each image volume depicting at least a portion of an elongated anatomical structure with curved longitudinal extension;

[0015] - On each 3D ultrasound image volume, a parametric curve is automatically or semi-automatically fitted to a portion of the elongated anatomical structure depicted along its longitudinal direction, the parametric curve being defined by curve parameters.

[0016] - Each 3D ultrasound image volume is reconstructed by applying a transformation that straightens the parametric curve along at least one axis, thereby generating multiple reconstructed image volumes and reconstructed parametric curves;

[0017] - By determining the connection points of their respective parametric curves, the reconstructed image volumes are mutually registered; and

[0018] - The reconstructed image volumes are fused together to produce a fused image depicting the entire elongated anatomical structure or a portion thereof that is larger than the volume of the 3D ultrasound image.

[0019] In other words, the present invention can provide a method for visualizing an entire elongated anatomical structure or a large portion of an entire elongated anatomical structure from multiple ultrasound volumes via (e.g., based on elongated structures) background fusion, so as to enable immediate assessment of the complete anatomical structure. The three-dimensional (3D) ultrasound image volume can be a 3D medical image of a part of a patient's body. After an image volume has been acquired from a patient (e.g., a pregnant woman), the 3D ultrasound image volume can be received in real time from an image acquisition device. Alternatively, the 3D ultrasound image volume can be received by inputting or loading a previously acquired image volume stored, for example, in the memory or storage of a computer system. Furthermore, the 3D ultrasound image volume can be acquired, for example, during a hospital examination, and then, for example, sent to another location at a data processing service center to perform the method of the present invention.

[0020] Elongated anatomical structures have a tortuous longitudinal extension, so the parametric curve fitted to them is typically not straight. Anatomical structures can be any elongated structure within a human or animal body; specifically, they can be larger structures that cannot be captured within the field of view (FoV) of a single ultrasound image, such as the spine of a fetus in mid- or late-pregnancy, the spine of a child or adult, skeletal structures, limbs or portions of limbs, blood vessels, or portions of the intestine. Preferably, the elongated anatomical structure has a longitudinal extension, which is the direction in which it is elongated. In the case of the spine, the longitudinal extension may, for example, run along the central neural canal. In the case of blood vessels, the longitudinal extension may be in the direction of blood flow.

[0021] In this paper, the invention will be explained with reference to the example of the fetal spine, an elongated anatomical structure. However, the invention is also applicable to other elongated anatomical structures.

[0022] In one embodiment, the method includes the step of acquiring multiple 3D ultrasound image volumes from the subject (specifically, the pregnant woman), each image volume depicting at least a portion of the fetal spine. Preferably, the fetus will be in the second or third trimester to allow sufficient ossification of the spine.

[0023] Various ultrasound devices can be used to acquire 3D ultrasound image volumes. For example, a 3D image volume can be collected by acquiring a series of two-dimensional (2D) ultrasound images, recording the transducer orientation for each image / slice, and then generating a 3D ultrasound image volume based on the 2D images and transducer orientation. Alternatively, a 3D ultrasound image volume can be captured directly using a matrix array transducer that samples points throughout the entire pyramidal volume (i.e., the 3D ultrasound image volume) using beam control. Multiple 3D ultrasound image volumes can be captured during the same examination, specifically, substantially at the same time, preferably on the same day. Furthermore, each 3D ultrasound image volume can include different portions of the same fetal spine. Each 3D ultrasound image volume can include at least one vertebra of the fetal spine. Preferably, the fields of view (FoV) of the 3D ultrasound image volumes can at least partially overlap each other, i.e., some portions of the fetal spine are depicted on several 3D ultrasound image volumes. The amount of overlap can be in the range of 10% to 50% of the volume of the 3D ultrasound image, preferably in the range of 20% to 40%, and more preferably about 30%.

[0024] In each 3D ultrasound image volume, a parametric curve is automatically or semi-automatically fitted to a portion of the depicted elongated anatomical structure along its longitudinal extension or direction. In the case of the spine, the parametric curve can be fitted to the centerline of the depicted spinal segment. The parametric curve can define the elongated anatomical structure as a function of one or more independent variables called curve parameters, and thus can express the coordinates of points defining the geometric extension of the elongated anatomical structure in each 3D ultrasound image volume. The parametric curve can be referred to as a parametric representation or parameterization of the elongated anatomical structure. The advantage of using a parametric curve as an approximation is that it is a simple definition of longitudinal extension. Furthermore, by using curve parameters, it is possible to parametrically fuse several image volumes, as described below. The parametric curve preferably defines the elongated anatomical structure in three dimensions, for example, in the coordinates of the 3D image volume.

[0025] Parametric curves are typically continuous and preferably (but not necessarily) differentiable, i.e., smooth. In embodiments, the parametric curve has a predetermined format, for example, it is predetermined as a polynomial, exponential, or trigonometric function, including variations or linear combinations thereof. For example, it can be an nth-degree polynomial, or more preferably, it is predetermined as a spline function, preferably cubic (a cubic spline function). A spline function is a function defined by piecewise polynomials. A spline is called nth-degree if each polynomial segment has a maximum degree of n. The places where segments meet are called "nodes".

[0026] Curve parameters are the parameters that define the curve. For spline functions, parameters include, for example, the positions of the nodes and parameters of the polynomial segments.

[0027] In the case of a slender anatomical structure, the fetal spine, the parametric curve can be an approximation of the spine's central line, that is, a line extending along the central neural canal of the spine.

[0028] Parametric curves can be automatically fitted to portions of the depicted elongated anatomical structures, eliminating the need for user interaction. For example, segments of blood vessels, neural canals, or the spine can be identified using appropriate segmentation techniques and fitted parametric curves. For instance, neural canals can be identified automatically or through user interaction, and pre-formatted parametric curves can be fitted to them to determine curve parameters. In another embodiment, the centerline of the spine can be identified using keypoint estimation employing artificial intelligence.

[0029] In another embodiment, the parametric curves are generated semi-automatically, meaning some user interaction is required. For example, a user can view the automatically generated parametric curves, such as by viewing the parametric curves overlaid on a representation of the 3D image volume on the screen, and correct the parametric curves if necessary.

[0030] Control points on elongated anatomical structures, such as those on the fetal spine (or within the central neural canal), can be selected semi-automatically or automatically, and curve parameters can be determined by fitting a (pre-defined) parametric curve to the selected control points. Preferably, the selected points are defined at portions of elongated anatomical structures that are easily visible within the volume of a 3D ultrasound image, for example, at each vertebra of the fetal spine.

[0031] According to the present invention, parametric curves are used to reconstruct each 3D ultrasound image volume by applying a transformation that straightens the parametric curves along at least one axis to generate multiple reconstructed image volumes (including reconstructed parametric curves). Alternatively, the parametric curves are straightened in at least one direction. In specific embodiments, the reconstructed parametric curves may be completely straight (linear), while in other embodiments, the parametric curves are only partially straightened, for example, straightened in only one direction, as explained in more detail below. In the reconstructed image volumes, elongated anatomical structures are at least to some extent "uncurved." Therefore, the evaluation of elongated anatomical structures is significantly simplified.

[0032] Reconstruction can be performed as follows: First, determine the desired transformation of the parametric curve. This can be an operation / transformation that straightens the curve in at least one direction, as illustrated below. The transformation can be defined based solely on the parametric curve, but in some embodiments, the intensity value of the 3D image volume is also considered, for example, by analyzing the 3D image volume to determine the direction in which the parametric curve is straightened. The transformation can then be performed on the parametric curve to produce a reconstructed parametric curve, which is preferably also defined by the curve parameters. Furthermore, the transformation itself can be defined algebraically or via a transformation matrix.

[0033] The size (dimension) of the reconstructed image volume (also called the output image or output volume) can then be defined, where, generally, the z-axis corresponds to the longitudinal direction of the anatomical structure, i.e., along the parametric curve, and the x-axis and y-axis are (locally) perpendicular to it. In one embodiment, the y-axis is the horizontal (left-right) axis of the fetus. The third axis (x-axis) can be in the depth direction, perpendicular to the other two axes, and can roughly correspond to the anterior-posterior axis of the fetus. The size of the reconstructed image in the z-direction can be calculated from the parametric curve, while the extensions in the x- and y-directions can be freely chosen according to visualization needs. In the example of the fetal spine, the size in the y-direction can be 2-3 cm to cover at least a portion of the fetal spine, including the spinous processes and the ribs extending from them. The size in the x-direction (depth) can be chosen to be only one pixel, in which case the output volume is a two-dimensional (2D) output image. If the size of the output image in the depth direction is only one or a few (e.g., up to 10) pixels, it corresponds to a “plate” transformation. Plate reconstruction allows for detailed examination of structures. Alternatively, the dimension in the x-direction can be a few pixels / voxels, for example, 10-256 voxels, so the output image is an output volume, where the dimension can be, for example, 0.5cm-2cm, and it corresponds to a "thick plate" transformation. Thick plate reconstruction allows people to view slender anatomical structures, preferably complete anatomical structures, such as the spine, and the image is not occluded by other anatomical structures.

[0034] In addition, you can freely choose the resolution of the desired output image (corresponding to the grid size, i.e. how many pixels / voxels in each direction).

[0035] The transformation can then be performed in reverse, i.e., from the reconstructed image volume to the original or input image volume. Therefore, for each pixel in the output image, its relative coordinates with respect to the reconstruction parametric curve can be determined, such as its distance to the curve and its position along the curve's length. These relative coordinates can then be transformed to the (original) parametric curve, thereby finding the coordinates of the corresponding point within the image space of the original 3D ultrasound image volume. Since this point may not lie directly on the grid of the original image volume, known re-meshization techniques such as first-order interpolation can be used to compute the voxel values ​​in the reconstructed image. In other words, once the transformation is defined, it is applied directly to the grid of the output image. Each voxel pixel in the output image volume is mapped to a position in the original 3D ultrasound image volume. Starting from this position, the intensity values ​​are interpolated using an interpolation kernel, which can be of order 0 (nearest difference), but is typically first-order (linear difference), or second-order (corresponding to cubic interpolation).

[0036] In another embodiment, the volume of the (original) 3D ultrasound image can be calculated to transform into the reconstructed image volume; that is, a forward transformation is performed. In this case, re-meshing can be performed on the grid of the output image because the voxels of the input image may not directly map to the pixels or voxels of the output image.

[0037] Reconstruction is performed on each of the multiple 3D ultrasound image volumes, resulting in multiple reconstructed image volumes. Once these multiple reconstructed image volumes are merged together, a single overview is presented, allowing for immediate evaluation of the spine, understanding of spine length, presence of each segment, and evidence of any spinal compression or spina bifida.

[0038] Furthermore, the reconstruction of 3D ultrasound image volumes simplifies the registration and fusion of reconstructed image volumes with each other. Reconstructed image volumes are registered with each other by determining the connection points of their respective reconstruction parameter curves. This simplifies registration because candidate points for registering reconstructed image volumes lie on the corresponding parameter curves ("original" parameter curves or preferably reconstructed parameter curves). In an embodiment, connection points are determined by taking a point on the parameter curve of the first reconstructed image volume (e.g., at a predetermined distance from one end of the image volume, or at a predetermined vertebra, or simply at the endpoint of the parameter curve) and determining the corresponding point on the parameter curve of the second adjacent but overlapping reconstructed image volume. Therefore, if two adjacent reconstructed image volumes are fused by overlapping these two connection points, the overlapping portions of the adjacent image volumes should match, such that the two image volumes are registered with each other. In other words, two adjacent reconstructed image volumes are connected at the connection points.

[0039] Connection points on a second reconstructed image volume corresponding to connection points on a first reconstructed image volume can be determined in various ways. In one embodiment, one can simply find the corresponding vertebra on the parametric curve of the second reconstructed image volume. In another embodiment, the parametric curves of the first and second image volumes (preferably before reconstruction) are compared to each other, and the best-fit connection point is found. In a further preferred embodiment, an optimization algorithm can be used to determine the connection point. Such an optimization algorithm can, for example, change the second connection point along its respective parametric curve and calculate a similarity metric between the overlapping portions of adjacent (reconstructed) image volumes for each location. This similarity metric is preferably a measure of how well the two image volumes are registered. The optimization algorithm can then optimize the similarity metric, for example, by minimizing a (cost) function that depends on the location of the connection point on the parametric curve.

[0040] Since the reconstructed parametric curves are also parameterized, the registration between reconstructed image volumes can be simplified to parameterized registration along the arc length of the spine (i.e., along the parametric curves). Therefore, the search for connection points of the parametric curves is performed in a reduced-dimensional space, thereby reducing the complexity of non-rigid registration by reducing the number of variables.

[0041] Therefore, the proposed invention provides a novel method for fully visualizing the entire anatomy from multiple ultrasound volumes via background (e.g., the spine) fusion and presentation, such as in a reconstructed view for immediate assessment of spinal defects. This further allows for the automatic or semi-automatic extraction of parameters and anatomical objects (e.g., segments) from the reconstructed image volume for quantitative assessment. The reconstructed image volume can be presented from two orthogonal directions in thin-plate and thick-plate views. Thus, the invention allows for, for example, immediate assessment of the presence of each spinal segment, their labeling, intervertebral distances, and the presence of fetal spinal abnormalities.

[0042] In embodiments of the invention, parametric curves can be generated by automatically or manually identifying control points on slender anatomical structures and specifically by fitting spline functions to these control points. For example, control points can be located at the center of each or some segments of the spine.

[0043] The spinal segment can be a single vertebra. The control point can be located at the center of the vertebral body. Alternatively, the control point can be located at the vertebral foramen of the vertebra. This is advantageous when using semi-automatic identification of control points, as the vertebral foramen can be easily detected within the volume of a 3D ultrasound image, for example, by thresholding. Control points can be identified at each vertebra or only at each second, third, or fourth vertebra.

[0044] Once the control points are identified, parametric curves can be fitted to them to obtain smooth curves. Advantageously, splines can be used as parametric curves because they are simple to construct, easy and accurate to evaluate, and capable of approximate complex shapes through curve fitting and interactive curve design. Curve fitting may require an exact or approximate fit to the control points. Preferably, the parametric curve is continuous and differentiable, but it can also be a piecewise linear function connecting the control points.

[0045] For example, control points can be manually placed at the center of each spinal segment (C1-7, T1-12, L1-5, S1-5) using a locating device. This can be done by the user while viewing a cross-sectional view through image volume (e.g., a multi-plane reconstruction (MPR) view displayed on the screen). Alternatively, control points can be automatically generated at the centerline of the spine, for example, by using AI-based control point estimation (e.g., control point regression). In the latter case, the user can check the control points, such as whether they are correctly placed at the center of each segment, and can manually change their positions, where this process (automatic selection of control points and user control) is an example of semi-automatic selection.

[0046] Once the parametric curve (e.g., spline curve) is generated, reconstruction can be performed using the parametric curve.

[0047] In embodiments, the reconstructed image volume, or the fused reconstructed image volume, can be visualized by aggregating its intensity values ​​along the viewing direction using a synthesis function. In useful embodiments, the viewing direction is a third axis or depth axis (x-axis), orthogonal to the first two axes, i.e., orthogonal to the longitudinal direction of the elongated anatomical structure. In the example of the spine, the viewing direction can preferably be perpendicular to the length of the spine and perpendicular to the ribs. However, it can also be along a second axis or y-axis, producing a sagittal view. The synthesis function can be maximum intensity, producing maximum intensity projection (MIP), average intensity, or a standard volumetric rendering cumulative value, such as volumetric ray tracing (VRT). The aggregated intensity values ​​can be displayed to the user on a screen or other display device, which may be part of a medical ultrasound device.

[0048] Arc length reconstruction

[0049] According to a first embodiment of reconstructing each 3D ultrasound image volume, an arc-length reconstruction of so-called elongated anatomical structures is performed. In this type of reconstruction, anatomical structures (e.g., the spine) are flattened and stretched. Therefore, in the reconstruction coordinate system, the fetus appears straightened along its left-right axis and along its front-back axis. Specifically, in this arc-length reconstruction embodiment, the step of reconstructing each 3D ultrasound image includes applying a transformation that expands the parametric curve so that it is straightened using a local coordinate system comprising, or consisting of, two axes orthogonal to the tangent of the curve, so that the reconstructed image volume includes the arc-length reconstruction of the spine. Using this reconstruction, the arc-length of the spine is preserved, and the reconstructed parametric curve can be a straight line. In other words, the parametric curve is straightened in both directions.

[0050] A local coordinate system is a coordinate system applied to a point along the parametric curve. Since the parametric curve is not straight before reconstruction, the orientation of the axes is different in each local coordinate system, as explained herein. However, after reconstruction, for each local coordinate system, at least one axis is aligned, i.e., the parametric curve has been straightened along that axis. In arc length reconstruction, preferably, the two axes orthogonal to the tangent of the curve (principal axes) are aligned.

[0051] In arc-length reconstruction, the principal axis (z-axis) of the reconstructed image volume follows the parametric curve of the central neural canal of the straightened and taut spine. Therefore, the principal axis corresponds to the tangent of the parametric curve. The second axis (y-axis) can be orthogonal to this curve in each local coordinate system and parallel to the ribs, i.e., approximately in the left-right direction of the fetus. Preferably, arc-length reconstruction also takes into account the fact that the spine may have torsion, i.e., the orientation of the two axes orthogonal to the tangent of the parametric curve may differ in each local coordinate system. In other words, the auxiliary axis along which it is reconstructed is a "free axis," i.e., it rotates about the principal axis. The aim is to orient the second axis to visualize the widest portion of the spinous process. That is, the auxiliary axis is aligned so that it is in a plane containing the widest portion of the vertebrae. In other words, the auxiliary axis is preferably parallel to the thoracic cavity, i.e., in the left-right direction.

[0052] In an embodiment, the orientation of the second axis can be determined by first finding the fetal thoracic cavity through thresholding. If control points along the central axis of the spine are known, local thresholding can be performed around these points. This can identify at least some portions of the ribs extending from the spine in a generally horizontal left-right direction. Thus, a set of points located on the spine and thoracic cavity can be found. This set of points can be analyzed to find the overall left-right direction, or an auxiliary axis (y-axis) for each local coordinate system. For example, a straight line can be fitted to each rib, as determined by thresholding. Alternatively, this can be done by principal component analysis of this set of points, which can be visualized as fitting an ellipse to the set of points—where the principal axis will be along the spine and the second axis will provide the desired y-axis orientation. Thus, the reconstructed image volume and / or fused image can include a principal axis corresponding to the parametric curve, and an auxiliary axis locally orthogonal to the principal axis and parallel to the ribs.

[0053] Therefore, in arc-length reconstruction, the spine is reconstructed along its arc length, thus also orienting the spine along the y-axis. Scrolling through such reconstruction slices allows a person to walk from front to back (the spine flattens and taut). Thick-plate reconstructions using MIP (maximum intensity projection) or VRT (volume ray tracing) can present a single overview, allowing for immediate evaluation of the spine, assessment of spinal length, the presence of each segment, and any evidence of spinal compression or spina bifida. These plates can be presented in coronal or sagittal views. Utilizing manual rewriting to reduce error accumulation, this reconstruction can be automated through end-to-end directional centerline tracing of the spine.

[0054] In the next step, the spine from multiple ultrasound acquisitions is fused. After the arc length of the spine is reconstructed, semi-automatic fusion can be achieved by transforming one volume to another along the principal axis of each reconstructed volume. The connection efficiency is very high because the search for connection points on the parametric curves is performed in a reduced-dimensional space, where the search space can be restricted to one dimension (along the parametric curve). Therefore, the variables can simply be translation vectors along the reconstructed parametric curves. Thus, this embodiment provides highly robust and efficient registration of different 3D volumetric images.

[0055] Curvature Preservation Reconstruction

[0056] According to a second embodiment of reconstructing each 3D ultrasound image volume, a curvature-preserving reconstruction of an elongated anatomical structure is performed, wherein the unfolding (straightening) of the elongated anatomical structure is performed along only one axis, preferably along an axis orthogonal to a reference plane of the anatomical structure. The reference plane can be determined by fitting a plane to the anatomical structure, in the case of a fetus, fitting to a portion of the fetus (e.g., the spine), for example, fitting to anatomical landmarks on the spinous processes, and / or fitting to the thoracic cavity. According to this curvature-preserving reconstruction, the step of reconstructing each 3D ultrasound image volume includes applying a transformation in a local coordinate system that unfolds the parametric curve along an axis orthogonal to the reference plane, such that the reconstructed image volume includes a curvature-preserving isometric reconstruction of the anatomical structure. This reference plane is not warped but planar. The reference plane can be a least-squares fitting plane of the elongated anatomical structure or its larger anatomical structure. In the case of the spine, it can be a plane fitted to anatomical landmarks on the spinous processes. In an embodiment, the reference plane is a typical coronal plane of the fetus. This reconstruction unfolds the spine by flattening the “waves” of the spine perpendicular to its reference plane. The total length of the spine is preserved. This can be likened to the total path a ship must take, including its up-and-down movement on waves, where the total path flattens out in a plan view, thus preserving the sideway curves (which would correspond to the curvature of the spine on the left-right axis). This reconstruction is useful for analyzing conditions such as scoliosis.

[0057] This reconstruction is equivalent to Curved Planar Reformation (CPR), specifically to stretched CPR, as described in the paper "CPR—Curved Planar Reformation" by Armin Kanitsar, Dominik Fleischmann, et al., published at IEEE Visualization 2002. In CPR, longitudinal structures (e.g., blood vessels) are resampled onto a surface defined by the vessel's centerline and an additional vector called the vector of interest. In stretched CPR, the surface defined by the vessel's centerline and the vector of interest is curved in one dimension and planar in another. The dimensions of the stretched curvature result in the plane fully displaying the tubular structure without overlap. In the curvature-preserving reconstruction of this invention, this surface is referred to as a "virtual overlay," upon which the elongated anatomical structure lies. One axis of the reconstructed view of the spine is a line extending along the principal axis of the elongated anatomical structure, specifically along the control points, along the length of the overlay. The other axes of the reconstructed view are orthogonal to it, but no longer free axes, but are identical for every point along the parametric curve and in every local coordinate system. Preferably, the other axes of the reconstructed view are orthogonal to the principal axis and lie within the reference plane. A virtual overlay can be generated given a parametric curve and the volume of the 3D image.

[0058] In the curvature-preserving reconstruction of this embodiment, the measurements within the reconstruction volume are equidistant. Generally, in an isometric view, the distance between two points corresponds to the actual distance between those points. That is, measurements can be taken within the reconstruction volume of this embodiment. Thick-plate reconstructions using MIP or VRT can present a single overview, allowing for immediate evaluation of the spine and its curvature, assessment of scoliosis, evaluation of spinal length, the presence of each segment, and any evidence of spinal compression or spina bifida. Other thin-plate reconstructions allow for detailed examination of the structure.

[0059] In a preferred embodiment of the curvature-preserving reconstruction of elongated anatomical structures, the elongated anatomical structure is flattened onto a reference plane, which is a least-squares fitting plane. The least-squares fitting plane is fitted to the elongated anatomical structure, for example, to control points (e.g., control points at the center of each or some spinal segments) and / or anatomical landmarks on the spinous processes, and / or to segments of the thoracic cavity, preferably the thoracic cavity of a fetus. Thoracic segments are obtained relatively easily using a simple thresholding method. Note that the spine is not projected onto this least-squares reference plane, but this transformation preserves the total length of the spine when the spine is flattened and stretched onto the least-squares fitting plane.

[0060] For example, after determining the parametric curve (e.g., spline curve) as described above, a reference plane or least-squares fitting plane defines the principal and auxiliary axes, i.e., axis vectors. Therefore, it is possible to define a transformation algebraically and apply it to the grid of the output image, where each voxel in the output image volume maps to a position in the original ultrasound image volume.

[0061] Registration and fusion

[0062] Using any of these reconstructions (arc-length reconstruction or curvature-preserving reconstruction), segments of the spine can be automatically or semi-automatically fused by registering and blending overlapping reconstructed image volumes to each other. By performing the above reconstructions, the automatic registration algorithm becomes more robust than existing techniques by reducing the number of variables. In one embodiment, the only variable is a translation vector (connection point) between the reconstruction parametric curves of the pair of reconstructed image volumes to be registered. This allows one reconstructed image volume to be strictly registered with each other. In another embodiment, other variables are the positions of control points used to fit the parametric curves, e.g., control points on the spinal segments. In other embodiments, the variables are curve parameters of the parametric curves (e.g., spline curves) that fit the spine in the two volumes. This embodiment also improves the fitting of the parametric curves to the spine and the registration. For example, this can be performed by an optimization algorithm that includes curve parameters and connection points in parameters that vary in, for example, minimizing the cost function curve. Thus, the registration can be parameterized along with the variables of the spline curves of the spine fitted to the pair of reconstructed image volumes. This allows non-rigid registration of one spinal segment (reconstructed image) with another spinal segment.

[0063] According to an embodiment, the step of registering two (overlapping) reconstructed image volumes may include the following steps:

[0064] Choose the connection points along their respective reconstructed parametric curves.

[0065] When connecting at the connection point, a similarity metric is calculated between the overlapping portions of the two reconstructed image volumes.

[0066] The connection points are translated along the reconstructed parameter curve and the similarity criterion is recalculated.

[0067] The final step (translation and recalculation) can be repeated several times. The same registration and fusion process can be applied to reconstruct the image volume for arc length and preserve the reconstructed image volume for curvature.

[0068] A tie point is a point on each of the two parametric curves belonging to each of two overlapping or adjacent image volumes; that is, the point where the two parametric curves will connect to produce a continuous parametric curve (e.g., a spline curve) through the two reconstructed image volumes. The tie point can be parameterized as a translation along this parametric curve. The tie point can be a location where two (reconstructed) parametric curves can be connected to form a continuous parametric curve fitted to the spine. According to this embodiment, portions of the overlapping reconstructed image volumes are compared to each other, for example, by calculating a similarity metric, to evaluate the goodness of fit of the registration. This is followed by translating the tie point (and optionally changing other variables as described herein), and calculating the similarity metric again. This step can be repeated (preferably iterated) several times until a good similarity metric is found. Preferably, the similarity metric is optimized, for example, using an optimization algorithm or optimizer, i.e., determining the maximum value of the similarity metric. The similarity metric can be optimized by minimizing a cost function, where the cost function defines the effect of registering the two reconstructed volumes.

[0069] The similarity metric used in the registration process can be a mean squared error metric or any similarity metric commonly used in image registration, such as normalized cross-correlation or cross-correlation information. (Preferably, multiple) steps of translation and recalculation of the similarity metric are performed to find the variable that maximizes the similarity metric. The variable can be the connection point of a continuous spline curve through the original and / or reconstructed image volumes, and optionally, curve parameters. For example, a suitable optimizer could be a gradient descent optimizer. The probability of finding a minimum during optimization is proportional to the dimensionality of the search space; therefore, by reducing the dimensionality of the search space (i.e., the number of variables), the registration process becomes more robust.

[0070] During the fusion step, the overlapping portion of a pair of reconstructed image volumes that have been registered with each other can be used to calculate voxels in the fused image by using voxel intensity values ​​from one of the overlapping image volumes, or by combining the intensity values ​​of the two overlapping portions. In an embodiment, this is achieved by “fading” one reconstructed image volume into the other; that is, the weights of the two reconstructed images in the combined intensity values ​​vary along the principal axis, so that one image is gradually “blended” into the other.

[0071] Furthermore, interpolators can be used during registration and fusion. Interpolators can be used to re-mesh the source image (i.e., the first reconstructed image) onto the target image (i.e., the second reconstructed image) mesh. More specifically, a similarity metric can be evaluated across all voxels in the target image mesh. At each voxel location, given the current transformation (connection point), the location in the source image must be identified. These may fall on non-mesh locations, i.e., intermediate voxels in the source image. An interpolator (typically first-order) can be used to obtain the intensity of these intermediate voxels (sub-voxels) from the source image.

[0072] In embodiments of the invention, the step of registering two reconstructed images may include weighting a similarity metric based on the distance to the parametric curve to highlight features near the spine. This allows for high-quality registration of spinal anatomy while ignoring local deformities far from the spine, such as fetal limb deformities or maternal habit deformities. Preferably, weighting may be applied such that values ​​near the centerline are given greater weight in the metric. The distance to the parametric curve can be a distance along a direction orthogonal to the parametric curve. In other words, a radius can be defined, defining a circle around the parametric curve, in which values ​​can be given greater weight in the metric. The radius can be in the range of 1 cm to 10 cm, preferably 1.5 cm to 5 cm. Similarly, the cost function (metric) can also be weighted based on the distance to the parametric curve, highlighting features along the parametric curve.

[0073] In other embodiments of the invention, the step of registering two reconstructed images may include refitting the parametric curve of the reconstructed image volume to, for example, the portion of the fused image or the depiction of elongated anatomical structures on the reconstructed image volume, and wherein the step of reconstructing the 3D ultrasound image volume is performed again using the refitted parametric curve.

[0074] To improve registration between two adjacent image volumes, curve parameters and connection points along the parametric curve can be changed. The reconstructed 3D ultrasound image volume can be regenerated in the next iteration based on the changed curve parameters, and the similarity metric can be recalculated, and so on. This can be accomplished using arc length or curvature preservation of the reconstruction. In an embodiment, during the registration process, the control points to which the parametric curve can be fitted can be displaced, specifically, along a spinal segment (e.g., a vertebra). Subsequently, the parametric curve can be fitted to the control points again. Thus, parametric curve fitting is improved along with registration. In an embodiment, this step can be performed using an optimizer (e.g., a gradient descent optimizer).

[0075] When using an optimizer, the complexity of the optimization process and the probability of finding the maximum of the similarity metric (the minimum of the cost function) are proportional to the dimensionality of the search space. Transformations in nonrigid fusion are generally high-dimensional. In this embodiment, nonrigid fusion can be performed using only a small number of parameters that define the parametric curves fitting the spinal anatomy. Therefore, the process can be efficient because the dimensionality of the problem can be reduced compared to generally known nonrigid fusion problems.

[0076] Furthermore, since the metric can be evaluated only on the reconstructed image being visualized, the number of voxels accessed in each iteration of the metric evaluation can be small, making the process even faster.

[0077] In embodiments, the method may further include, for example, steps of automatically identifying landmarks on elongated anatomical structures to identify each vertebra, and / or quantifying the origin and end points of each vertebra to identify the vertebral body. Preferably, this step is performed on the reconstructed volume or on the fused image.

[0078] In one use case, evaluating the maximum combined strength orthogonal to the parametric curve can output the position of the spinal segment and the intervertebral distance.

[0079] Given arc length reconstructions of the spine from multiple acquisitions, this embodiment can automatically identify landmarks for identifying each vertebra. This can be conveniently done in a straightened view, requiring only a search of segments in a single parameter space (i.e., along the principal axis). These segments can then be displayed in other views. Furthermore, several measurements, such as intervertebral distance and transverse pedicle distance, can be derived. Skin lines can also be automatically detected, and any deviations (skin loss or abnormal curves / undulations) can be highlighted in the sagittal view.

[0080] In yet another embodiment, the method may further include the step of automatically performing quantitative measurements on the fused images, wherein, specifically, the intervertebral distance, transverse pedicle distance, and / or skin line are automatically determined.

[0081] In another aspect of the invention, a computer program is provided, comprising program code instructions that, when executed by a processor, enable the processor to perform the method of the invention.

[0082] The present invention also relates to a computer-readable medium comprising the above-described computer program. The computer-readable medium can be any digital data storage device, such as a USB flash drive, CD-ROM, SD card, SSD card, or hard disk. Naturally, the computer program does not need to be stored on such a computer-readable medium for provision to a customer, but can be downloaded, for example, from a remote server or cloud via the Internet.

[0083] In a third aspect of the invention, an image evaluation apparatus is provided, configured to perform an embodiment of the method of the invention, the evaluation apparatus comprising:

[0084] A data storage device for receiving multiple 3D ultrasound image volumes, each image volume depicting at least a portion of an elongated anatomical structure;

[0085] A computing unit, which performs the above-described method, and

[0086] A screen used to display at least one reconstructed image volume or a fused image.

[0087] The data storage device is configured to store data, such as a hard disk drive. The computing unit can be a processor (e.g., CPU or GPU) capable of performing the methods described above. The screen can be a display capable of showing one or more images generated using the methods described above. Alternatively, the screen can be the screen of an ultrasound device. Additionally, the screen can be a touchscreen, which can serve as an interface for inputting commands. For example, a user can set the control points described above by tapping a specific location on the screen.

[0088] In another aspect of the invention, an ultrasonic system is provided, comprising:

[0089] The probe is configured to obtain 3D ultrasound volume, and

[0090] The aforementioned image evaluation device.

[0091] The probe is preferably a 3D transducer, which can directly obtain 3D ultrasound image volume via beam control and / or beamforming.

[0092] All the features and advantages mentioned in the combined method also apply to computer programs, image evaluation devices, and ultrasound systems, and vice versa. Attached Figure Description

[0093] The present invention will now be described with reference to the accompanying drawings and specific embodiments, wherein:

[0094] Figure 1 A schematic diagram and perspective view of the fetus, its spine, and thoracic cavity are shown;

[0095] Figure 2 A schematic diagram of arc length reconstruction is shown;

[0096] Figure 3 A schematic diagram of curvature-preservation reconstruction is shown;

[0097] Figure 4 This is a flowchart illustrating step e of an embodiment of the method according to the present invention;

[0098] Figure 5 A flowchart illustrating the steps included in the registration and reconstruction of image volume according to an embodiment of the present invention is shown;

[0099] Figure 6 A fused image of the fetal spine obtained through arc length reconstruction is shown;

[0100] Figure 7 Two ultrasound images are shown: the top one was obtained through curvature preservation reconstruction, and the bottom one was obtained through arc length reconstruction.

[0101] Figure 8 A block diagram illustrating the iterative process during the configuration of at least two 3D ultrasound image volumes is shown;

[0102] Figure 9 An ultrasound image is shown in which the automatic identification of spinal segments in each of the cervical, thoracic, lumbar, and sacral vertebrae is performed and displayed.

[0103] Figure 10 This is a schematic diagram of a system according to an embodiment of the present invention. Detailed Implementation

[0104] Throughout the accompanying drawings, the same or corresponding features / elements in different embodiments are indicated by the same reference numerals.

[0105] Digital images (e.g., ultrasound images) consist of digital representations of one or more objects (e.g., the spine). The digital representation of an object is often described herein in terms of identifying and manipulating the object. This manipulation is a virtual manipulation performed in the memory or other circuitry / hardware of a computer system. Therefore, it should be understood that embodiments of the invention can be performed within a computer system using data stored within the system. For example, according to various embodiments of the invention, electronic data representing the volume of a 3D ultrasound image is manipulated within a computer system to reconstruct the image, thereby visualizing the spine.

[0106] Figure 1 This is a schematic perspective view of the fetus 10 as seen from the front. The spine 20 and thoracic cavity 22 of the embryo 10 (i.e., the fetus) are highlighted. Furthermore, a coordinate system is shown, where the z-direction extends along the length of the spine, corresponding to the longitudinal axis, and the y-axis corresponds to the left-right or horizontal axis. Specifically, it is the horizontal axis of the least-squares fitted plane of the thoracic cavity. The x-direction corresponds to the anterior-posterior axis.

[0107] Figure 2 The principle of arc length reconstruction is illustrated. The top of the figure shows a virtual cover 40 on which the spine rests. The virtual cover is a warped surface 40 that follows the curvature of a parametric curve 30, shown as a line below the boat. The virtual cover 40 can also be slightly twisted about the parametric curve 30 such that the y-direction in each local coordinate system (i.e., the axis orthogonal to the tangent of the parametric curve and located within the virtual cover 40) is parallel to the spinous process. In arc length reconstruction, the parametric curve is preferably fully straightened and taut to produce a straight reconstructed parametric curve 32.

[0108] Figure 3The principle of curvature-preserving reconstruction is illustrated. The top of the figure again shows a virtual overlay 40 over which the spine lies. The virtual overlay 40 is straight in one direction and follows the curvature of parametric curve 30 in another direction, which is shown as a line below the boat. In other words, the virtual overlay 40 is curved in one dimension and planar in another. In a preferred embodiment, the planar direction of the overlay is parallel to a reference plane, specifically, parallel to the least-squares fitting plane of the spine or thoracic cavity. In curvature-preserving reconstruction, the spine is expanded by flattening the “wave”; that is, the virtual overlay is stretched to produce a resampled surface 42 as shown in the lower part of the figure. The reconstructed parametric curve 34 preserves the total length of parametric curve 30, as does the spine curve in the planar direction of the virtual overlay 40. Expansion occurs only along one axis, which is preferably an axis orthogonal to the least-squares fitting plane.

[0109] Figure 4 This is a flowchart illustrating the steps of an embodiment of a computer-implemented method for visualizing a fetal spine using ultrasound according to the present invention. In step 100, a plurality of 3D ultrasound image volumes are received, each image volume depicting at least a portion of the fetal spine. In this embodiment, receiving means that the 3D ultrasound image volumes are loaded onto the computer performing the method of the present invention. 3D ultrasound image volumes can be acquired in the same ultrasound examination, and consecutive image volumes along the fetal spine can overlap by 10% to 50% of the image volume (or spine length). For example, there can be 2-10, preferably 3-7, 3D ultrasound image volumes for the same fetal spine.

[0110] In step 102, within each 3D ultrasound image volume, a parametric curve is automatically or semi-automatically fitted to the centerline of the depicted spinal segment. The parametric curve is defined by curve parameters. In one embodiment of the invention, the user manually identifies segments of the spine (e.g., the centers of some vertebrae) via a pointing device on the screen (e.g., a mouse-driven cursor) or by tapping on the touchscreen 218. The user defines control points by tapping the spinal segments, which can be curve parameters of the parametric curve. The parametric curve fitted to the control points is displayed in real time, allowing the user to decide whether the curve needs to be corrected or whether additional control points need to be identified to provide a parametric curve that conforms to the geometry of the fetal spine. In another embodiment, spinal segments can be automatically identified, for example, by segmentation.

[0111] In step 104, each 3D ultrasound image volume is reconstructed by applying a transformation that straightens the parametric curves along at least one axis to generate multiple reconstructed image volumes and reconstructed parametric curves. In embodiments where the parametric curve fitted to the spine in step 102 is a continuous and differentiable spline curve, reconstruction along at least one axis can be implicitly performed. For example, in some embodiments, the reconstruction of the image volume can be derived directly in the desired direction by differentiating the parametric curve in that direction.

[0112] In step 106, the reconstructed image volumes are registered with each other by determining the connection points of their respective parametric curves. In one embodiment, adjacent 3D ultrasound image volumes overlap by approximately 10-50% of their size, preferably 20-40%, thereby allowing registration of the overlapping portions of the image volumes. Unlike the original 3D image volumes, the reconstructed image volumes are registered with each other. This reduces the complexity of non-rigid registration by reducing the number of variables. Preferably, the similarity measure is also weighted based on distance to the parametric curves, highlighting features along the spinal anatomy. In a useful embodiment, the parameter space of the cost function minimized during registration step 106 also includes the curve parameters of the parametric curves in the two reconstructed image volumes. Therefore, in step 106, the parametric curves can be modified (refitted), and the reconstruction of step 104 can be performed again to further improve the registration process. This is indicated by the iteration of steps 102, 104, and 106. Finally, once registration 106 has resulted in determining the connection points between the reconstructed parametric curves and optionally refitted curve parameters, the reconstructed image volumes are fused with each other.

[0113] exist Figure 5 The diagram shows a flowchart illustrating the steps included in the registration and reconstruction of image volume according to an embodiment of the present invention.

[0114] In step 402, connection points of the respective reconstruction parameter curves of the reconstructed image volumes along the parameter curves are selected. In step 404, a similarity metric is calculated between the overlapping portions of two reconstructed volumes when they are connected at their respective connection points. In step 406, the similarity metric is calculated again after translating the connection points along the reconstruction parameter curves. The similarity metric can be weighted based on the distance to the parameter curves to highlight features near the spine. In step 410, the parameter curves of the reconstructed volumes are refitted, and the steps of reconstructing the 3D ultrasound volumes are performed again using the modified parameter curves (see also the steps between steps 106 and 102 indicating iteration). Figure 2 (The arrow in the image).

[0115] Figure 6Two reconstructed ultrasound images are shown, in which the spine is unfolded along its arc length to generate a reconstructed volume. A slice of the reconstructed image volume, or thin-plate reconstruction, is shown on the right. A volume rendering of the reconstructed image volume orthogonal to the arc length is shown on the left. That is, the spine is "stretched" along its length. The reconstructed volume is generated along the principal axis, or (in this sense) the horizontal axis, which is the arc length of the spine. Figure 5 The vertical axis in the diagram corresponds to the y-axis described in this paper. Thick-plate visualization via this reconstructed MIP (maximum intensity projection) or VRT (volume ray tracing) shows what the spine would look like if the infant were lying flat on their back, rather than with the spine curved as it is in the womb. This allows for immediate assessment of the presence of each spinal segment (cervical, thoracic, lumbar, and sacral), marking all spinal segments from top to bottom, assessing intervertebral distances, and evaluating the presence of spina bifida.

[0116] Figure 7 Two ultrasound images are shown: the top one was obtained through curvature-preserving reconstruction, and the bottom one through arc-length reconstruction. The curvature-preserving reconstruction of the spine in the top image shows the curvature in the spine. Reconstruction parameter curves are superimposed on the images. Unfolding is performed only along one axis, which is orthogonal to the least-squares fitting plane of the spine points (e.g., control points). In the curvature-preserving reconstruction in the top image, the spine is unfolded by flattening the "waves" of the spine orthogonal to its least-squares fitting plane. The total length of the spine is preserved. All measurements on the displayed view are also preserved. Therefore, the views are isometric.

[0117] During mid-pregnancy, it is often impossible to scan the spine in a single 3D view. At least two views may be required to cover the entire spine. Between the two volumes, the spine can be non-rigid. This could be due to the baby's movement within the uterus. In anterior spinal presentations, it could also be due to probe pressure capable of causing deformation. Probe pressure itself can induce fetal movement. Therefore, registration and fusion of separately captured 3D ultrasound image volumes are necessary.

[0118] Given two acquisitions covering two parts of the spine, non-rigid registration between the two image volumes (typically involving a large parameter space) can be simplified to registration parameterized along the spine (i.e., along a parametric curve) via the reconstructed volumes. In one embodiment generating arc-length reconstruction, the registration can be reconstructed such that the variables to be determined are translation vectors or transformations along the principal axes between the two reconstructed volumes. In another embodiment generating curvature-preserving reconstruction, the registration can be parameterized together with variables fitting a low-dimensional parametric curve of the spine, e.g., by stitching. This together optimizes the parametric curve fitting the spine as well as the transformation fusing the two volumes themselves. Again, because this is parameterized along a parametric curve (i.e., along the spine), the registration is inherently non-rigid. Therefore, the search space for registration is limited relative to the visualization of the spine based on the parameterization for the spine and the clinical problem at hand. Furthermore, if arc-length reconstruction registration is performed first, and thus the translation transformation between the two reconstructed volumes is determined during non-rigid registration, this information can also be used during the two curvature-preserving reconstruction non-rigid registrations. Therefore, the latter approach can present further improvements.

[0119] therefore, Figure 8 The non-rigid registration method of this embodiment is illustrated. Two reconstructed overlapping image volumes 302 and 304 are input into the method, wherein, assuming specific translation vectors between the parametric curves (generating specific connection points), a similarity metric 308 between the overlapping portions of the two reconstructed image volumes 302 and 304 is calculated. This may need to be used on an interpolator 306 because the grid points of the two image volumes may not be consistent with each translation vector. The similarity metric 308 is input into an optimizer 310, which optimizes the similarity metric by changing the translation vectors, and in some embodiments, also changing the curve parameters of the parametric curve 312. Using the new translation vectors, the interpolator 306 calculates new similarity metrics 308 until a maximum value of the similarity metric 308 has been reached, as determined by the optimizer.

[0120] Figure 9 An ultrasound image is shown, in which the automatic identification of spinal segments in each of the cervical, thoracic, lumbar, and sacral vertebrae is performed and displayed. Different groups can be visualized using different indicators. In this embodiment, pentagons and circles are used to indicate different vertebral groups. Note that... Figure 9 Each group was not identified. Automatic identification of the markers, or the identification of each vertebra, is conveniently done on the straightened view, i.e., on the arc-length reconstruction, where only a search of the segmented image in a parameter space is needed (along the principal axis, i.e., the horizontal direction in the displayed image). These can then be displayed on other views, such as in... Figure 9As shown in the diagram. From this, several measurements can be derived, such as intervertebral distance, transverse pedicle distance, etc. It can also automatically detect, for example, deviations in the skin line and any prominent features in the sagittal view, such as skin loss or abnormal curves / undulations.

[0121] Figure 10 This is a schematic diagram of an ultrasound system 200 according to an embodiment of the present invention, configured to perform the method of the invention. The ultrasound system 200 includes a typical ultrasound hardware unit 202, which includes a CPU 204, a GPU 206, and a digital storage medium 208, such as a hard disk or solid-state drive. A computer program can be loaded into the hardware unit from a CD-ROM 210 or via the Internet 212. The hardware unit 202 is connected to a user interface 214, which includes a keyboard 216 and an optional touchpad 218. The touchpad 218 can also serve as a display device for displaying imaging parameters. The hardware unit 202 is connected to an ultrasound probe 220, which includes an array of ultrasound transducers 222 that allow, preferably in real-time, the acquisition of 3D ultrasound image volumes 224, such as B-mode images, from an object or patient (not shown). 3D ultrasound images 224 acquired using ultrasound probe 220 and reconstructed or fused images generated by the method of the present invention executed by CPU 104 and / or GPU are displayed on screen 226, which can be any commercially available display unit, such as a screen, television, flat panel screen, projector, etc. Furthermore, a connection to a remote computer or server 228 may exist, for example, via Internet 112. The method according to the present invention can be executed by the CPU 204 or GPU 206 of hardware unit 202, but can also be executed by the processor of remote server 228.

[0122] The foregoing discussion is intended to be illustrative of the system only and should not be construed as limiting the appended claims to any specific embodiment or group of embodiments. Therefore, while the system has been described in particular detail with reference to exemplary embodiments, it should be understood that those skilled in the art can devise numerous modifications and alternative embodiments without departing from the broader and contemplated spirit and scope of the system as set forth in the following claims. Thus, the specification and drawings are intended to be illustrative and not to limit the scope of the appended claims.

Claims

1. A computer-implemented method for visualizing slender anatomical structures using ultrasound, the method comprising the steps of: - Receive (100) multiple 3D ultrasound image volumes (224), each image volume depicting at least a portion of an elongated anatomical structure with curved longitudinal extension; - On each 3D ultrasound image volume, a parametric curve (30) (102) is automatically or semi-automatically fitted to the portion of the depicted elongated anatomical structure extending longitudinally, the parametric curve being defined by curve parameters; - Reconstruct (104) each 3D ultrasound image volume by applying a transformation that straightens the parameter curve (30) along at least one axis to generate multiple reconstructed image volumes and reconstructed parameter curves (32, 34); - By determining the connection points of the respective reconstruction parameter curves (32, 34) of the reconstructed image volumes, the reconstructed image volumes are registered with each other (106); and - The reconstructed image volumes are fused together (108) to produce a fused image depicting the entire elongated anatomical structure or a portion thereof that is larger than the volume of the 3D ultrasound image.

2. The method according to claim 1, wherein, Reconstructing the volume of each 3D ultrasound image involves the following steps: - Define the size and resolution of the reconstructed image volume; - For each voxel in the reconstructed image volume, the corresponding coordinates of the voxel in the 3D ultrasound image volume are found by using the parameter curve (30) and the reconstruction parameter curve (32, 34); - The intensity value of each voxel in the reconstructed image volume is calculated by interpolating the intensity value of the voxel that is closest to the corresponding coordinate in the 3D ultrasound image volume.

3. The method according to claim 1 or 2, wherein, The elongated anatomical structure is the spine (20) of the fetus (10), and the parametric curve is fitted to the center line of the portion of the spine being depicted.

4. The method according to any one of claims 1 to 2, wherein, The parametric curve is generated by automatically or manually identifying control points on the elongated anatomical structure and fitting the parametric curve to the control points.

5. The method according to claim 4, wherein, The parametric curves are generated by automatically or manually identifying control points at the center of each or some segments of the spine (20) and fitting spline functions to the control points.

6. The method according to any one of claims 1 to 2, wherein, The step of reconstructing each 3D ultrasound image volume includes applying a transformation to expand the parametric curve so as to straighten it using a local coordinate system comprising two axes orthogonal to the tangent of the parametric curve, such that the reconstructed image volume includes the arc length reconstruction of the elongated anatomical structure.

7. The method according to claim 6, wherein, The elongated anatomical structure is the spine (20) of the fetus (10), and the parametric curve is fitted to the center line of the portion of the depicted spine, wherein the reconstructed image volume and / or the fused image includes a principal axis corresponding to the tangent of the parametric curve, and an auxiliary axis orthogonal to the principal axis and parallel to the ribs of the fetus.

8. The method according to any one of claims 1 to 2, wherein, The step of reconstructing each 3D ultrasound image volume includes applying a transformation of the unfolded parametric curve in a local coordinate system along an axis orthogonal to a reference plane of the elongated anatomical structure, such that the reconstructed image volume includes a curvature-preserving isometric reconstruction of the elongated anatomical structure.

9. The method according to claim 8, wherein, The elongated anatomical structure is the spine (20) of the fetus (10), and the parametric curve is fitted to the center line of the portion of the spine depicted, wherein the reference plane of the fetus (10) is determined by fitting a plane to the spine (20) and / or to anatomical landmarks on the spinous process and / or to the thoracic cavity (22) of the fetus.

10. The method according to any one of claims 1-2, wherein, The steps for registering the volumes of the two reconstructed images include: - Select the connection points of each reconstructed parameter curve along the parameter curve (302), - Using the connection point selected by (304) and calculating the similarity measure between the overlapping portions of the two reconstructed image volumes, - Translate the connection point along one of the reconstruction parameter curves and recalculate (306) the similarity metric.

11. The method according to claim 10, wherein, The step of registering two reconstructed image volumes includes weighting the similarity metric based on the distance to the parametric curve in order to highlight image features close to the elongated anatomical structure.

12. The method according to any one of claims 1-2, wherein, The step of registering the reconstructed image volumes to each other includes refitting (310) the reconstruction parameter curves (32, 34) of the reconstructed image volumes.

13. The method according to any one of claims 1-2, the method further comprising the step of automatically performing quantitative measurements on the fused image.

14. The method according to claim 13, wherein, The steps of automatically performing quantitative measurements on the fused images include automatically determining the intervertebral distance, transverse pedicle distance, and / or skin line.

15. A computer program product comprising program code instructions, said program code instructions, when executed by a processor, enabling the processor to perform the method according to any one of claims 1 to 14.

16. An image evaluation apparatus configured to perform the method according to any one of claims 1 to 14, the evaluation apparatus comprising: A memory for receiving multiple 3D ultrasound image volumes, each depicting at least a portion of an elongated anatomical structure. A computing unit, configured to perform the method according to any one of claims 1 to 14, and A screen for displaying the reconstructed image volume or the fused image.

17. An ultrasound system, comprising: The probe is configured to obtain 3D ultrasound volume, and The image evaluation device according to claim 16.

Images