Obtaining a guide for 3D representation of anatomical structures

Abstract

A mechanism has been proposed for determining whether an imaging probe, such as an ultrasound imaging probe, is in a desired orientation and / or position relative to an anatomical structure. Image data from the imaging probe is processed to generate a 3D landmark model having anatomical landmarks of the anatomical structure. The 3D landmark model is then processed to determine whether the imaging probe is in the desired orientation and / or position.

Description

Technical Field

[0001] The present invention relates to the field of 3D imaging, and particularly to a guiding mechanism for a 3D imaging system.

Background Art

[0002] Congenital heart disease (CHD) is the most common type of congenital defect, affecting 1% of newborns. CHD may be asymptomatic during fetal life but causes significant morbidity and mortality after birth. Early diagnosis of CHD leads to better outcomes at birth and treatment options. In particular, effective in-utero therapies for specific CHD lesions such as fetal aortic valvuloplasty for hypoplastic left heart syndrome are becoming available and have been shown to significantly improve the natural history and prognosis of the disease.

[0003] However, these potential benefits depend on accurate fetal ultrasound diagnosis of CHD. The fetal diagnosis rate of CHD in the community is recognized to be in the range of 30 to 50% even in developed countries where fetal ultrasound is widespread. However, theoretically, a thorough screening of the fetal heart should be able to detect 90% of abnormalities.

[0004] The main reason for this diagnostic gap is insufficient / heterogeneous expertise in obtaining the right heart plane and interpreting fetal heart images. This is mainly thought to be due to the diagnostic challenges presented by the small, rapidly beating fetal heart and the relatively low exposure of caregivers to each specific type of congenital heart disease. Signs of heart disease are often subtle and exist under multiple modalities / associations (tetralogy of Fallot) that can contact various structures within the heart such as valves, septa, and myocardium and require carefully targeted examinations.

Summary of the Invention

Problems to be Solved by the Invention

[0005] Therefore, an intuitive, accurate, dynamic, and interactive display of the fetal heart is needed to facilitate the identification of cardiac abnormalities.

[0006] U.S. Patent Application Publication No. 2019 / 200964(A1) discloses a system and method for generating patient-specific organ models based on organ ultrasound images and probe position data.

[0007] US 2006 / 241445 A1 discloses a method for modeling anatomical structures based on the measured positional and directional coordinates of an ultrasonic sensor and the contour of interest in the acquired ultrasonic image. [Means for solving the problem]

[0008] This invention is defined by the claims.

[0009] According to one aspect of the present invention, a computer implementation method is provided for guiding the imaging process of individual anatomical structures.

[0010] A computer implementation method includes: acquiring image data of an anatomical structure from an imaging system having an imaging probe; processing the image data to acquire one or more sets of one or more 2D images, each identified set of one or more 2D images representing a different predetermined view of the anatomical structure; identifying one or more predetermined anatomical landmarks in each set of one or more 2D images; processing one or more predetermined anatomical landmarks to generate a 3D landmark model; and determining, by processing the 3D landmark model, whether the current orientation and / or position of the imaging probe is within a predetermined range with respect to a desired orientation and / or position of the imaging probe.

[0011] This disclosure provides a mechanism for assisting or guiding clinicians when performing an imaging process using an imaging probe. This is achieved by using a 3D landmark model or "pose model" that identifies the relative position of anatomical landmarks (obtained from image data) in 3D space. This facilitates the identification of whether the desired orientation / position of the imaging probe is achieved, among other approaches, for example, by identifying whether the anatomical landmarks are in the expected position and / or whether the field of view of the imaging probe coincides with the desired field of view (e.g., whether the volume / region imaged by the probe coincides with the desired volume / region to be imaged).

[0012] Determining whether the orientation and / or position of the imaging probe is within a predetermined range is advantageous for assisting guidance during the imaging process, for example, in assisting the performance of tasks that involve performing the imaging process to capture a desired angle or image of an anatomical structure.

[0013] For example, when performing imaging using spatiotemporal image correlation (STIC) techniques, it is beneficial to start imaging from a specific location (or within a specific range of locations) to perform effective and accurate imaging. For instance, during fetal cardiac monitoring, it is important to image a sufficient range of the fetus to obtain clinically useful images, and the selection of an appropriate starting position determines whether a sufficient range can be obtained.

[0014] Furthermore, selecting the appropriate irradiation angle (for the ultrasound imaging system) is beneficial in reducing shadow artifacts in ultrasound images of anatomical structures, such as shadow artifacts from the fetal spine and ribs (when imaging the fetal heart).

[0015] Preferably, the step of processing one or more predetermined anatomical landmarks includes mapping one or more predetermined landmarks within each set of one or more 2D images to a 3D coordinate system, thereby generating a 3D landmark model. In this way, the 3D landmark model defines the location of one or more predetermined landmarks in the 3D coordinate system.

[0016] Preferably, one or more predetermined landmarks include multiple predetermined landmarks. In some examples, different sets of one or more predetermined landmarks may be identified in each set of one or more 2D images.

[0017] This method may further include the step of processing a 3D landmark model to determine a desired orientation and / or position for the imaging probe relative to an anatomical structure. The anatomical landmark represents the location of important features of the anatomical structure. The desired orientation and / or position can be determined based on the desired position relative to important features of the anatomical structure. This approach advantageously means that the desired orientation and / or position can be specifically adapted or individualized for each individual receiving the image.

[0018] The method may further include a step of obtaining user input indicating a desired anatomical feature for imaging, and a step of processing a 3D landmark model to determine a desired direction and / or position, which includes a step of processing a 3D landmark model to identify a desired direction and / or position based on the desired anatomical feature for imaging.

[0019] The method may further include, in response to determining that the current orientation and / or position of the imaging probe is not within a predetermined range, generating guide information to move the imaging probe to a desired orientation and / or position relative to an anatomical structure, based on the identified current orientation and / or position of the imaging probe.

[0020] In some embodiments, the method includes providing a user-perceptible output in response to guidance information. For example, the method may include providing a visual indicator of the direction in which the imaging probe should be moved to align the imaging probe in a desired direction and / or position. This approach provides a mechanism that can achieve improved imaging performance by using guided human-machine interaction processing.

[0021] This method may include sending a trigger signal to the imaging system to trigger the acquisition of a second, different image data of an anatomical structure in response to determining that the current orientation and / or position of the imaging probe is within a predetermined range.

[0022] This allows the imaging system to be triggered to acquire image data in response to the orientation and / or direction of the imaging probe within a predetermined range. With this means, the imaging device can automatically begin acquiring imaging data when the imaging probe is in the most appropriate or desired position.

[0023] Preferably, the imaging system is an ultrasonic imaging system, and the second image data is image data acquired using spatiotemporal image correlation techniques. In some examples, the image data may not be acquired using spatiotemporal image correlation (STIC) techniques in order to conserve processing power and energy before the first image data reaches a desired direction / position. The present invention is particularly well suited to controlling when STIC data is used to acquire it, since the accuracy and / or suitability of the STIC data is particularly sensitive to the starting position during the imaging process.

[0024] The method can further include providing a user-perceivable output in response to determining that the current direction and / or position of the imaging probe is within a predetermined range. This user-perceivable output can include audio, visual, or haptic feedback. This information provides clinically useful information to assist the user during the imaging process, for example, by providing information indicating when the imaging probe is in an appropriate position. For example, the user can start or initiate the imaging process in response to the user-perceivable output.

[0025] A computer-implemented method for guiding the imaging process of individual anatomical structures has also been proposed, which includes repeatedly iterating any of the aforementioned methods until the current direction and / or position of the imaging probe is within a predetermined range.

[0026] A computer-implemented method for guiding the image processing of individual anatomical structures has also been proposed. The computer-implemented method includes repeatedly performing the method according to any one of claims 1 to 8 (i.e., any of the aforementioned methods for guiding image processing) until the current direction and / or position of the imaging probe is within a predetermined range, tracking the subsequent movement of the imaging probe using a position and / or direction sensor, and repeatedly predicting whether the direction and / or position of the imaging probe is within a predetermined range with respect to the desired direction and / or position of the imaging probe based on the tracked movement until the predicted direction and / or position of the imaging probe is within a predetermined range.

[0027] In a preferred example, the anatomical structure is the heart. The present disclosure is particularly suitable for the analysis of the heart, especially the fetal heart, for the refinement of techniques for identifying at least the anatomical landmarks of the heart and the benefit of early recognition of fetal heart abnormalities. However, other suitable anatomical structures, such as the brain, lungs, kidneys, or other organs, will be apparent to those skilled in the art.

[0028] In some examples, each identified set of one or more 2D images represents a different one of a four-chamber view, a left ventricular outflow tract view, a right ventricular outflow tract view, a three-vessel view, a three-vessel and trachea view, an abdominal site view, an aortic arch view, and / or a ductal arch view.

[0029] In some examples, the step of processing the image data includes processing the image data using a machine learning method to identify one or more sets of one or more 2D images.

[0030] Also proposed is a computer program product including computer program code means for causing a processing system to execute all of any of the steps described herein when executed on a computing device having the processing system.

[0031] Also proposed is a processing system for guiding an imaging process of an individual anatomical structure.

[0032] The processing system obtains image data of an anatomical structure from an imaging system having an imaging probe, processes the image data to obtain one or more sets of one or more 2D images, each identified set of one or more 2D images represents a different predetermined view of the anatomical structure, identifies one or more predetermined anatomical landmarks within each set of one or more 2D images, processes the one or more predetermined anatomical landmarks to generate a 3D landmark model, and processes the 3D landmark model to determine whether the current direction and / or position of the imaging probe is within a predetermined range relative to the desired direction and / or position of the imaging probe.

[0033] The processing system may be adapted to execute any of the methods described herein, and may be adapted accordingly. Similarly, any of the methods described herein may be adapted, with the necessary changes, to execute any of the steps described in relation to the processing system.

[0034] A person skilled in the art can easily adapt any processing system described herein to carry out any method described herein. Similarly, a person skilled in the art can easily adapt any method described herein to carry out the operation of any processing system described herein.

[0035] These and other aspects of the present invention are evident from the embodiments described below and will be explained with reference thereto.

[0036] To better understand the present invention and to more clearly illustrate how it can be implemented, the accompanying drawings are referenced here, merely as examples. [Brief explanation of the drawing]

[0037] [Figure 1] A method according to one embodiment is shown. [Figure 2] The current and desired imaging volumes for anatomical structures are illustrated. [Figure 3] The current and desired imaging volumes for anatomical structures are illustrated. [Figure 4] A method according to one embodiment is shown. [Figure 5] This shows a system comprising a processing system according to one embodiment. [Figure 6] This image shows a set of 2D images of the fetal heart obtained from ultrasound imaging data throughout the cardiac cycle. [Figure 7] This shows the processing system. [Modes for carrying out the invention]

[0038] The present invention will be described with reference to the drawings.

[0039] Detailed descriptions and specific examples illustrate exemplary embodiments of the apparatus, systems, and methods, but are for illustrative purposes only and are not intended to limit the scope of the invention. These and other features, aspects, and advantages of the apparatus, systems, and methods of the invention will be better understood from the following description, the appended claims, and the appended drawings. The figures are for illustrative purposes only and are not drawn to scale. The same reference numerals are used throughout the drawings to indicate the same or similar parts.

[0040] The present invention provides a mechanism for determining whether an imaging probe, such as an ultrasound probe, is in a desired direction and / or position relative to an anatomical structure. Image data from the imaging probe is processed to generate a 3D landmark model that includes anatomical landmarks of the anatomical structure. The 3D landmark model is then processed to determine whether the imaging probe is in a desired direction and / or position.

[0041] This disclosure is based on the understanding that a 3D landmark model generated from image data obtained by an imaging probe changes as the orientation and / or position of the imaging probe changes. Therefore, by processing the positions of known anatomical landmarks, it is possible to determine whether the imaging probe is in a desired orientation and / or position (i.e., within a predetermined range).

[0042] The embodiment may be used, for example, in ultrasound imaging, such as those performed for fetal cardiac analysis. The embodiment is particularly advantageous for initiating or determining a starting position for performing STIC acquisition of image data.

[0043] Figure 1 schematically illustrates a workflow 10, including a method 100 performed by one embodiment, to illustrate the underlying concepts of this disclosure. Both workflow 10 and method 100 represent embodiments of the concepts of the present invention.

[0044] In step 110, image data 191 of the anatomical structure is acquired. Therefore, the image data is medical image data such as ultrasound image data, magnetic resonance image data, computed tomography image data, and X-ray image data. The image data is any data that, when processed, can generate a 3D image or 3D representation of the anatomical structure.

[0045] Image data is acquired from an imaging system that uses an imaging probe, such as an ultrasound probe, to acquire image data. Other forms of directed imaging systems, such as X-ray scanners, CT scanners, or MRI scanners, may be used. Image data may include, for example, low-resolution 3D data, such as data obtained when performing a rapid sweep of anatomical structures using an imaging probe.

[0046] Image data is obtained while the imaging probe is in the same position or position relative to the anatomical structure, i.e., while the imaging probe is stationary. For example, in the case of ultrasound imaging, the ultrasound probe is held by the clinician at the same point on the surface of the subject. In this scenario, image data 191 may be image data obtained by performing a 3D ultrasound imaging acquisition process using a fixed imaging probe, which acquires a 3D ultrasound image and / or a series of 2D ultrasound images (which may represent a 3D imaging volume). If the image data includes ultrasound data, this can be done by having the imaging probe transmit ultrasound pulses in different directions and monitoring the response to ultrasound pulses in different directions.

[0047] The position and / or orientation of the imaging probe thereby defines the current imaging volume of the imaging probe, i.e., the area / volume currently being imaged by the imaging probe. Thus, the image data represents the image data of the current imaging volume being imaged by the (stationary) imaging probe and may include ultrasound image data.

[0048] Image data may include, for example, coarse 3D scans of anatomical structures, i.e., image data obtained by performing scans that are not performed at the highest possible quality on the imaging system.

[0049] The method also performs step 130, which involves obtaining one or more (2D) image sets 193 from image data 191. Each image set 193 contains one or more 2D images of an anatomical structure from a specific anatomical viewpoint, and as a result, each image set provides a different anatomical viewpoint of the anatomical structure. Images from a specific anatomical viewpoint provide images of a specific view of the anatomical structure. An image providing a four-chamber cardiac view of the heart is an example of an image taken from a specific anatomical viewpoint, but other examples will be readily apparent to those skilled in the art.

[0050] One or more image sets 193 are obtained in step 130 by processing the image data 191. In one example, a classifier or machine learning method (e.g., a (convolutional) neural network or other machine learning method) is used to classify 2D images or 2D slices of the image data and identify images from a specific anatomical viewpoint. In another example, tagged images (e.g., images tagged by a clinician during the image processing process) are processed to identify images from a specific anatomical viewpoint based on the tags of each image.

[0051] Next, in step 140, one or more sets of images 193 are processed to identify anatomical landmarks ("landmark points" or "key points") within each set of one or more 2D images. Anatomical landmarks are points of individual anatomical structures that exhibit homology within individual species but may differ from one individual to another, such as the mitral valve point, the location of the fossa ovale, and the location of the aortic valve.

[0052] The anatomical landmarks identified in each set of one or more 2D images may differ depending on the anatomical viewpoint of the set of 2D images, for example, so that it is known which anatomical landmarks are expected to be present for a particular anatomical viewpoint. Process 140 may include an image segmentation process for identifying one or more given anatomical landmarks within each 2D image. Approaches for (automated) identification of anatomical landmarks in 2D medical images are widely known and understood, and machine learning methods, classifiers, and / or edge detection techniques can be used.

[0053] Next, in step 150, the identified anatomical landmarks are processed to generate a 3D landmark model 194. In particular, the identified landmarks can be mapped to a 3D coordinate space in order to generate a 3D landmark model. This is achievable because the anatomical viewpoint of each image in the image set is already known (i.e., the relationship between the images in the image set and 3D space is already known). In this way, the anatomical landmarks can be directly mapped or registered to 3D coordinates, giving a patient-specific 3D landmark model with major anatomical features / junctions mapped to 3D.

[0054] A 3D landmark model can, for example, include information that defines the coordinates of each anatomical landmark in 3D coordinate space.

[0055] A 3D landmark model represents the location of anatomical landmarks relative to a specific position and / or orientation of the imaging probe. Therefore, the position and / or orientation of the 3D landmark model (and the anatomical landmarks contained within it) in 3D space is essentially linked to the position and / or orientation of the imaging probe. It should be understood that if the imaging probe is moved, and a new 3D landmark model is constructed from the new image data obtained from the moved imaging probe, the position and / or orientation of the 3D landmark model itself will change.

[0056] For example, the 3D coordinate position of each anatomical landmark may change when the imaging probe is moved.

[0057] Therefore, the orientation and / or position of the 3D landmark model changes as the current image volume of the imaging probe, which is linked to the position and / or orientation of the imaging probe, changes.

[0058] Next, method 100 performs step 160, which determines whether the current orientation and / or position of the imaging probe is within a predetermined range relative to a desired orientation and / or position of the imaging probe by processing a 3D landmark model.

[0059] As mentioned above, the spatial relationship between the position and / or orientation of the imaging probe and the 3D landmark model is known. This is because the 3D landmark model is constructed from the imaging data generated by the imaging probe. Conceptually, this spatial relationship is known because the 3D landmark model represents the position of anatomical landmarks in the image data 191 acquired in step 110. This relationship facilitates the identification of whether the position and / or orientation of the imaging probe is within the range of the desired position and / or orientation.

[0060] In some examples, step 160 may include step 165, which identifies a desired orientation and / or position of the imaging probe by processing a 3D landmark model. This can be done, for example, by identifying a desired spatial relationship between anatomical landmarks in the 3D landmark model and then identifying the orientation and / or position of the imaging probe to achieve this desired spatial relationship.

[0061] For example, the desired orientation and / or position of the imaging probe may be a desired orientation and / or position that images a specific predetermined volume of an anatomical structure from a particular viewpoint. This predetermined volume is associated with a predetermined desired position for an anatomical landmark, having a predetermined spatial relationship between different anatomical landmarks. A box or predetermined shape representing the desired imaging volume can be fitted around the anatomical landmark of the 3D landmark model to identify the desired orientation of the 3D landmark model.

[0062] As a mere example, if the anatomical structure is the heart, and the desired orientation and / or position of the imaging probe is the orientation and / or position to achieve a four-ventricular view, then the desired orientation and / or position may be the position in which the anatomical landmarks are positioned in a particular configuration to achieve a four-ventricular view.

[0063] In some examples, step 165 may include identifying the desired imaging volume by processing a 3D landmark model. This can be done by processing the 3D landmark model to identify the imaging volume that positions anatomical landmarks at desired locations and / or in desired spatial relationships to each other. The desired imaging volume is essentially linked to the desired location and / or direction of the imaging probe (which defines the imaging volume imaged by the probe).

[0064] This desired imaging volume can be used to determine whether the current imaging volume is within a predetermined range relative to the desired imaging volume, and therefore whether the current position and / or orientation of the imaging probe is within a predetermined range of the desired position and / or orientation of the imaging probe. In other words, an evaluation can be made regarding whether the current imaging volume is aligned with the desired imaging volume.

[0065] In some embodiments, step 165 includes obtaining user input indicating a desired anatomical feature or desired view (point) for imaging, and the desired direction and / or position includes processing a 3D landmark model to identify the desired direction and / or position based on the desired anatomical feature or viewpoint (point) for imaging (i.e., the desired anatomical feature is acquired when the imaging probe is imaged from a desired direction and / or position, or when the desired view is achieved). Step 165 may include processing user input to determine the desired anatomical feature or viewpoint (point) and identifying the desired position and / or direction of the imaging probe based on the desired anatomical feature or viewpoint (point) (in accordance with best known medical practices, which may be set in a lookup table or other suitable mapping system, for example). The desired anatomical feature may be, for example, a specific anatomical feature for Doppler gating (e.g., a feature to be evaluated in the subsequent Doppler imaging process).

[0066] Depending on the desired imaging process, any appropriate desired anatomical feature may be shown. For example, in the case of cranial imaging, the desired anatomical feature may be, for example, a Willis or a circle of the corpus callosum for the assessment of corpus callosum agenesis. In other examples, for example, for cardiac imaging, the desired anatomical feature may include a desired view of the heart, for example, a four-ventricle view.

[0067] In some cases, the desired orientation and / or position of the imaging probe is predetermined and / or pre-set, for example, according to medical standards or for a particular imaging system, to define the position and / or future imaging. As an example, the imaging system may be specifically designed for fetal cardiac ultrasound, and the desired orientation and / or position may be predetermined to be the most clinically appropriate position (according to known medical studies) for initiating an imaging process such as the STIC acquisition process.

[0068] In some examples, step 160 involves determining the necessary modifications or transformations (e.g., translation and / or rotation) to the 3D landmark model to reposition it to a desired location / rotation. This effectively defines how the imaging probe should be positioned and / or oriented so that, when acquiring new image data and constructing a new 3D landmark model from the new image data, the new 3D landmark model is oriented and / or positioned to a desired location / rotation.

[0069] Therefore, the 3D landmark model can be used to effectively determine the error between the current position and / or orientation of the 3D landmark model (and thus the imaging probe) and the desired position and / or orientation of the 3D landmark model. This information can be used to determine whether the position and / or orientation of the imaging probe is within the range of the desired position and / or orientation. This information can also be used to provide guidance on how to correct the position and / or orientation of the imaging probe to reach the desired position and / or orientation.

[0070] Therefore, step 160 may include step 167, which determines the difference between the desired direction and the current direction. The difference to be determined is a numerical value or measurement that represents the required difference between the current direction and / or position and the desired direction and / or position. This can be done, for example, by determining the required change in direction (e.g., degrees or radians) and / or change in position (e.g., distance measurement).

[0071] For example, step 167 may include determining the transformations that need to be applied to the landmark model to align the landmark model to a desired position and / or orientation, i.e., to position the anatomical landmarks in a desired position and / or orientation relative to one another. This transformation defines the necessary changes in the position and / or orientation of the imaging probe, since the position and / or orientation of the landmark model is linked (e.g., by a predetermined and / or known relationship) to the position and / or orientation of the imaging probe.

[0072] Some combinations of these parameters can be used to define the difference for the purposes of step 167.

[0073] Step 160 may also include step 169, which determines whether the current position and / or orientation of the imaging probe is within a predetermined range with respect to a desired position and / or orientation of the imaging probe. This can be achieved, for example, by determining whether the difference determined (e.g., the difference calculated in step 167) exceeds or exceeds a predetermined threshold.

[0074] In another example, step 160 may include determining whether the 3D landmark model is within a predetermined range of a desired pattern for the 3D landmark model in order to determine, for example, whether the 3D landmark model demonstrates that the imaging probe is in a desired position and / or orientation relative to an anatomical structure.

[0075] In one example where the 3D landmark model includes information defining the coordinates of multiple anatomical landmarks in 3D coordinate space, step 160 may include determining, for each anatomical landmark, the distance between the location of the anatomical landmark and a reference location (representing the location of the anatomical landmark in 3D coordinate space for the desired position and / or orientation of the imaging probe). The average distance may be used as a measure of the error between the current position and / or orientation of the imaging probe and the desired position and / or orientation of the imaging probe. If this measure of error exceeds a predetermined threshold, it indicates that the imaging probe is not within a predetermined range for the desired position and / or orientation of the imaging probe.

[0076] Figures 2 and 3 conceptually illustrate an exemplary process for determining whether the current orientation and / or position of the imaging probe is within a predetermined range relative to a desired orientation and / or position of the imaging probe.

[0077] In particular, Figure 2 shows the 3D landmark model 210 (shown as a 2D projection of the 3D landmark model). The 3D landmark model is constructed from the imaging data as described above. This defines the 3D landmark model, which reflects the current imaging volume 220, i.e., the current position and / or orientation of the imaging probe. In particular, one surface of the 3D landmark model (e.g., the top surface of the frustum representing the imaging volume for ultrasound images) can represent the position and / or orientation of the imaging probe. From the 3D landmark model, for example, by identifying the imaging volume 230 that positions the anatomical landmarks of the 3D landmark model in a desired position and / or spatial relationship, it is possible to identify the desired position and / or direction of the imaging probe.

[0078] More specifically, Figure 2 shows how a desired imaging volume 230 can be determined from a 3D landmark model, and a desired imaging volume can be selected such that, for example, the anatomical landmarks of the 3D landmark model 210 are located in a desired position (e.g., to image a specific anatomical feature and / or to achieve a specific anatomical view).

[0079] The difference between the current imaging volume 220 and the desired imaging volume 230 can be determined, that is, data representing the difference between the current position and / or orientation of the imaging probe and the desired position and / or orientation of the imaging probe can be determined. For example, if the difference is applied to a 3D landmark model, it may be a transformation (e.g., defining a change in orientation and / or position) that will align the 3D landmark model to the desired orientation and / or position.

[0080] Figure 3 illustrates a cross-section of image data suitable for use in generating a 3D landmark model, as shown in Figure 2. The relative positions of anatomical landmarks in the 3D landmark model 210, as well as the relative positions of the current imaging volume 220 and the desired imaging volume 230, are also shown.

[0081] Returning to Figure 1, workflow 10 may further include step 170 of generating guidance for the user, for example, guidance presented to the user by controlling the user interface. The guidance may provide information on how to change the orientation and / or position of the imaging probe to achieve a desired orientation and / or position of the imaging probe. For example, the guidance may provide information on how to change the orientation and / or position of the imaging probe to bring about an imaging volume that aligns with a desired imaging volume. This guidance may include, for example, elevation and azimuth correction angles of the imaging probe for ultrasound images.

[0082] This guidance may be represented visually (e.g., through the use of text output and / or graphic output such as arrows) and / or aurally (e.g., through the use of audible instructions for the user).

[0083] In at least one embodiment, step 170 includes processing a 3D landmark model to generate guidance (information) for the user.

[0084] In one example, this process may involve determining a transformation applied to a 3D landmark model, resulting in the 3D landmark model matching or aligning with a reference 3D landmark model. The relationship between this transformation and the required movement of the imaging probe is known or predetermined (since the 3D landmark model is generated from image data obtained by the imaging probe), and as a result, guidance (information) on how the imaging probe should be moved can be determined.

[0085] In the second example, step 170 includes determining the difference between the current imaging volume and the desired imaging volume (defined by processing a 3D landmark model to identify the shape that places anatomical landmarks in the desired reference position). This difference can be processed to determine the necessary changes to the current imaging volume, e.g., translation and / or direction, so that the current imaging volume can be aligned with the desired imaging volume. This information may be processed to generate guidance (information) to guide the user on how to modify the imaging volume (and therefore the direction and / or position of the imaging probe) to achieve the desired imaging volume.

[0086] In some examples, as shown in the figure, a guide is generated if it is determined that the current orientation and / or position of the imaging probe is not within a predetermined range (in step 169).

[0087] Workflow 10 may further include step 192, which generates an output indicating that the imaging probe is aligned to a desired direction and / or position, in response to the determination that the current direction and / or position of the imaging probe is within a predetermined range. This output may be an electrical signal or a change in an electrical signal.

[0088] In some examples, the output generated in step 192 may be used to control a user interface for communicating this alignment to the user (e.g., triggering a green light, generating text output, and / or generating a beep / vibration to indicate that the current orientation and / or position of the imaging probe is aligned with the desired orientation and / or position). Thus, the output in step 192 may be a user-perceptible output (e.g., visual, audio, or tactile output).

[0089] This user-perceivable output provides the user, for example, a clinician, with useful clinical information to understand when the imaging probe is in a predetermined or desired position. The user may then manually trigger a further imaging process, for example, which is initiated at the desired position and / or orientation of the imaging probe.

[0090] In some cases, the output of step 160 can be used to automatically trigger the acquisition of a second, different image data of an anatomical structure (using the same imaging probe / system).

[0091] Therefore, in some examples, the workflow 10 may include step 195 of sending a trigger signal to the imaging system to trigger the acquisition of a second different image data of an anatomical structure in response to determining that the current orientation and / or position of the imaging probe is within a predetermined range.

[0092] In a particularly preferred embodiment, the resolution and / or quality of the image data is lower than the resolution of the second image data. Therefore, the imaging probe can be guided to a desired / preferred position and / or direction by processing the lower-quality image data before it is later used to acquire high-quality imaging data.

[0093] The use of step 195 is particularly advantageous when the imaging system is an ultrasound imaging system and the second image data is image data acquired using the spatiotemporal image correlation (STIC) technique. This is because STIC images require considerable processing power and benefit significantly from starting from a specific location and / or direction.

[0094] In some examples, when the anatomical structure is the fetal heart, the desired position and / or orientation of the imaging probe may be such that the imaging probe is positioned so as to be above the pre-fetal chest, allowing it to acquire a four-chamber view of the fetal heart. This position has been shown to provide particularly high-quality acquisition of STIC images. Those skilled in the art will understand that such a position may be associated with a specific arrangement or pattern for a 3D landmark model, i.e., a 3D landmark model that is positioned and / or oriented at a particular location.

[0095] In some examples, step 195 may include setting one or more imaging parameters based on a 3D landmark model (for example, for automatically triggered imaging). This process of setting one or more imaging parameters may be performed even if imaging is not automatically triggered.

[0096] For example, the focus of the ultrasound imaging process performed by an ultrasound imaging system may be set based on the location of a given anatomical landmark in a 3D landmark model. As an example, in the case of fetal cardiac ultrasound imaging, the focus of the ultrasound imaging process may be determined based on the location and / or position of an anatomical landmark representing the lower leg (center of the heart).

[0097] As another example, step 195 may include setting one or more Doppler window parameters based on a 3D landmark model (e.g., setting the location of a region of the ultrasound image to be generated using a Doppler-based imaging process). For example, the 3D landmark model may be processed to identify appropriate / desired locations for positioning gates, windows, regions, or volumes of the ultrasound image on which the Doppler shift will be recorded. This is achievable because the 3D landmark model facilitates the identification of desired locations in the ultrasound image, thereby enabling the positioning of gates for Doppler imaging at the desired locations. Desired locations may be indicated by user input or based on standard operational practices (e.g., as described in past data, literature, etc.).

[0098] In some examples, method 100 may be repeated iteratively as the imaging probe moves, for example, to iteratively determine whether the imaging probe is within a desired position and / or direction range of the imaging probe.

[0099] Figure 4 is a flowchart of Method 400 according to one embodiment of the present invention. Method 400 is designed to guide the imaging process of individual anatomical structures, and the computer implementation method includes iteratively performing the imaging until the current orientation and / or position of the imaging probe is within a predetermined range.

[0100] Method 400 includes performing Method 100 or Workflow 10 as described above.

[0101] Method 400 also includes step 410 of generating guidance for a user, for example by controlling a user interface, in response to Method 100 predicting that the imaging probe is not within a predetermined range of a desired direction and / or position of the imaging probe. The guidance may provide information on how to change the direction and / or position of the imaging probe to achieve a desired direction and / or position of the imaging probe.

[0102] Step 410 may be performed in the same manner as or identical to Step 170 described above, or it may be omitted (for example, if Step 170 is performed as part of Method 100).

[0103] Method 400 also includes step 420 of tracking the (subsequent) movement of the imaging probe. Step 420 is performed using position and / or orientation sensors such as accelerometers (e.g., mounted on the imaging probe itself), optical tracking systems, camera-based tracking systems (e.g., monitoring the imaging probe and using image recognition techniques), and magnetic-based tracking systems. Mechanisms for using such sensors to track the position and / or orientation of an object such as an imaging probe are well known and are common in the prior art.

[0104] Method 400 includes, for example, a process 430 that iteratively predicts whether the direction and / or position of the imaging probe is within a predetermined range relative to a desired direction and / or position of the imaging probe, based on tracked movement, until the predicted direction and / or position of the imaging probe is within a predetermined range.

[0105] In other words, it can be used to track changes in the position and / or orientation of the imaging probe and to establish whether the imaging probe is aligned with the desired orientation and / or position. The information regarding the changes in orientation and / or position required for the imaging probe is derived from the 3D landmark model as described above.

[0106] This approach reduces the number of times image data needs to be acquired, avoiding, for example, the need to repeatedly perform method 100, and predicts whether the imaging probe is within a predetermined range of desired directions and / or positions for the imaging probe.

[0107] Effectively, Method 400 proposes using a 3D landmark model to determine any changes to the position and / or orientation of the ultrasound probe to achieve a desired position and / or orientation, and then tracking the movement of the ultrasound probe using position and / or orientation sensors to determine / predict when the desired orientation and / or position of the ultrasound probe is achieved. This provides an approach for highly accurate guidance of the imaging process without requiring repeated acquisition of image data (as would be required if, for example, Method 100 were simply repeated).

[0108] In some examples, method 100 is repeated (i.e., a second instance of method 100 is performed) in response to a process 430 that predicts whether the orientation and / or position of the imaging probe is within a predetermined range. This facilitates the performance of more accurate / precise predictions as to whether the imaging probe is within a predetermined range of the desired orientation and / or position of the imaging probe.

[0109] In response to this second example of Method 100, if it is predicted that the imaging probe is not within a predetermined range of the desired direction and / or position of the imaging probe, the Method may return to step 410 / 420. Otherwise, steps 192 and / or 195 (described above with reference to Figure 1) may be performed.

[0110] Figure 5 shows a system 500 comprising a processing system 510 according to one embodiment of the present invention. System 500 also comprises an imaging system 520 and an output display 530. The processing system 510 is designed to guide the imaging process of individual anatomical structures.

[0111] The processing system 510 is configured to acquire image data of an anatomical structure from an imaging system 520 having an imaging probe, to process the image data to obtain one or more sets of one or more 2D images, each identified set of one or more 2D images representing a different predetermined view of the anatomical structure, to identify one or more predetermined anatomical landmarks in each set of one or more 2D images, to process one or more predetermined anatomical landmarks to generate a 3D landmark model, and to process the 3D landmark model to determine whether the current orientation and / or position of the imaging probe is within a predetermined range relative to a desired orientation and / or position of the imaging probe.

[0112] Therefore, the processing system 510 is configured to perform the method 100 described with reference to Figure 1. The processing system 510 can be appropriately configured to perform any of the aforementioned methods with necessary modifications.

[0113] For example, the processing system 510 may be configured to generate guidance for the user by controlling a user interface such as an output display 530.

[0114] The guidance may provide information on how to modify the orientation and / or position of the imaging probe to achieve a desired orientation and / or position of the imaging probe. For example, the guidance may provide information on how to modify the orientation and / or position of the imaging probe to bring about an imaging volume that aligns with a desired imaging volume. This guidance may include, for example, elevation and azimuth correction angles of the imaging probe for ultrasound imaging.

[0115] This guidance may be represented visually via output display 530 (for example, using text output and / or graphical output such as arrows).

[0116] In this way, the processing system 510 may be configured to control the output display 530 in response to a determination of whether the current orientation and / or position of the imaging probe is within a predetermined range with respect to a desired orientation and / or position of the imaging probe.

[0117] The output display 530 may include, for example, a monitor or screen configured to provide a visual output of guidance, as provided by the processing system 510.

[0118] Figure 6 shows a set of 2D images 600 of the fetal heart obtained from ultrasound image data over the cardiac cycle. Thus, Figure 6 illustrates an exemplary result of step 130 described above with reference to Figure 1.

[0119] This set represents a four-ventricular view of the fetal heart at different phases of the cardiac cycle, with 12 anatomical landmarks traced across the phases. Points 601 and 602 are the midpoint of the mitral and tricuspid valves, point 603 is the leg of the heart, points 604 and 605 are the endpoints of the mitral and tricuspid valves, point 606 is the vertebra, point 607 is the aorta, point 608 is the apex of the heart, points 609 and 610 are the left and right atria, and points 611 and 612 are the left and right ventricles.

[0120] Further sets of 2D images can be obtained from the same ultrasound imaging data, each representing a different predetermined view. For example, further sets of 2D images corresponding to left ventricular outflow tract views, right ventricular outflow tract views, three-vascular views, three-vascular and tracheal views, abdominal region views, aortic arch views, and / or ductal arch views can be obtained from the ultrasound image data. Anatomical landmarks visible in each set of 2D images can be tracked across all phases of the cardiac cycle. The points tracked across all views can be mapped to 3D coordinates to generate a 3D landmark model.

[0121] While the example has been illustrated in the context of appropriately positioning an image to initiate the STIC acquisition process, those skilled in the art will understand how the proposed system can be used in any suitable imaging process that benefits from positioning an image at a specific location or in a particular location. For example, Doppler (ultrasound) imaging processes benefit from positioning the imaging probe at a suitable location to target a Doppler image of a desired anatomical feature (e.g., the heart for transcranial Doppler imaging or the intraventricular septum for imaging Willis's circle).

[0122] Figure 7 is a schematic diagram of a processing system 510 according to an embodiment of the present disclosure. As shown, the processing system 510 may include a (data) processor 760, a memory 764, and a communication module 768. These elements can communicate with each other directly or indirectly, for example, via one or more buses.

[0123] The processor 760 may include a central processing unit (CPU), a digital signal processor (DSP), an ASIC, a controller, an FPGA, another hardware device, a firmware device, or any combination thereof, configured to perform the operations described herein. The processor 760 may also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors working with a DSP core, or any other such configuration. In some embodiments, the processor is a distributed processing system, formed, for example, from a set of distributed processors.

[0124] Memory 764 may include cache memory (e.g., the cache memory of processor 760), random access memory (RAM), magnetoresistive RAM (MRAM), read-only memory (ROM), field-programmable gate array read-only memory (PROM), erasable field-programmable gate array read-only memory (EPROM), electrically erasable field-programmable gate array read-only memory (EEPROM), flash memory, solid-state memory devices, hard disk drives, other forms of volatile and non-volatile memory, or combinations of different types of memory. In one embodiment, memory 764 includes a non-temporary computer-readable medium that can store instructions. For example, memory 764, or a non-temporary computer-readable medium, may have program code recorded thereon, the program code containing instructions for causing a processing system 510, or one or more components of the processing system 510, in particular the processor 760, to perform the operations described herein. For example, the processing system 510 may perform the operation of method 700. Instructions 766 may also be referred to as code or program code. The terms “instruction” and “code” should be interpreted broadly to include any type of computer-readable statement. For example, the terms “instruction” and “code” may refer to one or more programs, routines, subroutines, functions, procedures, etc., and “instruction” and “code” may include a single computer-readable statement or many computer-readable statements. The memory 764 in which the code is recorded is sometimes called the computer program product.

[0125] The communication module 768 may include any electronic and / or logic circuits to facilitate direct or indirect communication of data between the processing system 510, the penetration device, and / or the user interface (or other further devices). In this regard, the communication module 768 may be an input / output (I / O) device. In some cases, the communication module 768 facilitates direct or indirect communication between various elements and / or systems (Figure 5) of the processing circuit 510.

[0126] It will be understood that the disclosed method is preferably a method to be carried out on a computer. Accordingly, the concept of a computer program comprising computer program code for implementing any described method when the program is executed on a processing system such as a computer or a set of distributed processors is also proposed.

[0127] Different parts, lines, or blocks of code in a computer program according to one embodiment may be executed by a processing system or computer to perform the method described herein. In some alternative implementations, functions shown in a block diagram or flowchart may occur in a different order than shown in the diagram. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or in reverse order depending on the function in which the blocks are involved.

[0128] This disclosure proposes a computer program (product) that, when executed by a computer or processing system, includes instructions causing the computer or processing system to perform any of the methods (steps) described herein. The computer program (product) may be stored on a non-temporary computer-readable medium.

[0129] Similarly, computer-readable (storage) media are proposed that, when executed by a computer or processing system, contain instructions causing the computer or processing system to perform any of the methods (steps) described herein. Computer-readable data carriers storing the aforementioned computer programs (products) are also proposed. Data carrier signals for carrying the aforementioned computer programs (products) are also proposed.

[0130] Those skilled in the art can easily develop a processing system to carry out any of the methods described herein. Thus, each step in the flowchart may represent a different action performed by the processing system, which may be performed by each module of the processing system.

[0131] Variations of the disclosed embodiments can be understood and implemented by those skilled in the art in carrying out the claimed invention, based on the study of the drawings, disclosures, and appended claims. In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude the plural. The mere fact that certain means are described in different dependent claims does not imply that combinations of these means cannot be used advantageously.

[0132] A single processor or other unit can fulfill the functions of several items enumerated in the claims. Computer programs may be stored / distributed on suitable media such as optical or solid-state media supplied together with or as part of other hardware, but may also be distributed in other forms such as the internet or other wired or wireless telecommunications systems.

[0133] When the term “or conforms to” is used in the claims or description, it should be noted that the term “or conforms to” is intended to be equivalent to the term “or constitutes.” No reference numeral in the claims should be construed as limiting in scope.

Claims

1. A computer implementation method for guiding the imaging process of individual anatomical structures, wherein the method is: The steps include acquiring image data of the anatomical structure from an imaging system having an imaging probe, A step of processing the aforementioned image data to obtain one or more sets of one or more 2D images, wherein each identified set of one or more 2D images represents a predetermined different view of the anatomical structure, The steps include identifying one or more predetermined anatomical landmarks in each set of one or more 2D images, The steps of generating a 3D landmark model by processing the one or more predetermined anatomical landmarks in each set of the one or more 2D images so as to map the one or more predetermined anatomical landmarks to a 3D coordinate system, The steps include identifying a desired spatial relationship between anatomical landmarks in the 3D landmark model, and identifying the direction and / or position of the imaging probe to achieve the desired spatial relationship, A step of determining whether the current direction and / or position of the imaging probe is within a predetermined range with respect to the direction and / or position of the imaging probe, which achieves the desired spatial relationship by processing the 3D landmark model to determine the necessary modification or transformation of the 3D landmark model to reposition the 3D landmark model to a desired position / rotation. Having, Computer implementation method.

2. The computer implementation method according to claim 1, further comprising the step of processing the 3D landmark model to determine the desired orientation and / or position of the imaging probe relative to the anatomical structure.

3. The computer implementation method according to claim 2, further comprising the step of obtaining user input indicating a desired anatomical feature for imaging, the step of processing the 3D landmark model to determine a desired direction and / or position, the step of processing the 3D landmark model to identify a desired direction and / or position based on the desired anatomical feature for imaging.

4. The computer implementation method according to any one of claims 1 to 3, further comprising the step of generating guide information for moving the imaging probe to a desired direction and / or position relative to the anatomical structure, based on the identified current direction and / or position of the imaging probe, in response to the determination that the current direction and / or position of the imaging probe is not within the predetermined range.

5. The computer implementation method according to any one of claims 1 to 4, further comprising the step of transmitting a trigger signal to the imaging system to trigger the acquisition of a second different image data of the anatomical structure in response to the determination that the current orientation and / or position of the imaging probe is within the predetermined range.

6. The computer implementation method according to claim 5, wherein the imaging system is an ultrasonic imaging system, and the second image data is image data acquired using spatiotemporal image correlation technology.

7. The computer implementation method according to any one of claims 1 to 6, further comprising the step of providing a user-perceptible output in response to determining that the current orientation and / or position of the imaging probe is within the predetermined range.

8. A computer implementation method for guiding the imaging process of individual anatomical structures, The step of repeatedly performing the method according to any one of claims 1 to 7 until the current direction and / or position of the imaging probe is within the predetermined range. A method having

9. A computer implementation method for guiding the imaging process of individual anatomical structures, wherein the computer implementation method is: A step of carrying out the method according to any one of claims 1 to 7, The steps include tracking the subsequent movement of the imaging probe using position and / or direction sensors, The steps include repeatedly predicting, based on the tracked movement, whether the direction and / or position of the imaging probe is within the predetermined range relative to a desired direction and / or position of the imaging probe, until the predicted direction and / or position of the imaging probe is within the predetermined range; A computer implementation method comprising the step of repeatedly performing until the current direction and / or position of the imaging probe is within the predetermined range.

10. The computer implementation method according to any one of claims 1 to 9, wherein the anatomical structure is a heart.

11. The computer implementation method according to claim 10, wherein each identified set of one or more 2D images represents a different one of a four-ventricular view, a left ventricular outflow tract view, a right ventricular outflow tract view, a three-vascular view, a three-vascular and trachea view, an abdominal viscera view, an aortic arch view, and / or a ductal arch view.

12. The computer implementation method according to any one of claims 1 to 11, wherein the step of processing the image data comprises processing the image data using a machine learning method to identify one or more sets of one or more 2D images.

13. A computer program product having computer program code means that, when executed on a computing device having a processing system, causes the processing system to execute all of the steps of the method according to any one of claims 1 to 12.

14. A processing system for guiding the imaging process of individual anatomical structures, wherein the processing system is: The steps include acquiring image data of the anatomical structure from an imaging system having an imaging probe, A step of processing the image data to obtain one or more sets of one or more 2D images, wherein each identified set of the one or more 2D images represents a predetermined different view of the anatomical structure, The steps of identifying one or more predetermined anatomical landmarks in each set of one or more 2D images, The steps of generating a 3D landmark model by processing the one or more predetermined anatomical landmarks in each set of the one or more 2D images so as to map the one or more predetermined anatomical landmarks to a 3D coordinate system, The steps include identifying a desired spatial relationship between anatomical landmarks in the 3D landmark model, and identifying the direction and / or position of the imaging probe to achieve the desired spatial relationship, A step of determining whether the current direction and / or position of the imaging probe is within a predetermined range with respect to the direction and / or position of the imaging probe, which achieves the desired spatial relationship by processing the 3D landmark model to determine the necessary modification or transformation of the 3D landmark model to reposition the 3D landmark model to a desired position / rotation. A processing system configured to perform the following.

Images