DETERMINATION OF AN AREA OF INTEREST TO BE PRESENTED

DE502018015226D1Active Publication Date: 2024-11-07SIEMENS HEALTHINEERS AG
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
DE502018015226
Authority / Receiving Office
DE · DE
Patent Type
Patents
Current Assignee / Owner
Filing Date
2018-07-26
Publication Date
2024-11-07
Estimated Expiration
2038-07-26

AI Technical Summary

Technical Problem

The challenge lies in accurately associating a two-dimensional X-ray projection image with a three-dimensional ultrasound volume data set in breast examinations, which is difficult due to the cone-shaped X-ray path, leading to increased radiation exposure and distorted images that complicate diagnosis.

Method used

A method that determines a region of interest in the ultrasound volume data set using an X-ray 3D ultrasound device, where a partial data set is extracted based on the X-ray projection image, incorporating geometric information to align the X-ray projection image with the ultrasound volume data set, reducing radiation exposure and distortion.

Benefits of technology

This method allows for precise visualization of the area of interest with reduced radiation exposure, enabling quicker and more accurate diagnosis by focusing on specific data sets rather than the entire ultrasound volume, improving diagnostic efficiency.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader
Need to check novelty before this filing date? Find Prior Art

Description

[0001] The invention relates to a method for determining an area of ​​interest to be displayed, a method for displaying an area of ​​interest, an image processing device and an X-ray system.

[0002] Medical examinations of the breast, particularly of a human female breast, are generally performed to detect and diagnose malignant changes in breast tissue accurately and reliably. Mammography is a common method for conducting such an examination.

[0003] During a mammogram, the breast is positioned and compressed between two compression plates in an X-ray machine or mammography unit, and an X-ray image is produced in this state. X-rays are emitted from an X-ray source. As they pass through the breast, their intensity is attenuated depending on the type and structure of the tissue being penetrated, and the resulting image is then captured by a detector as a mammogram.

[0004] Depending on the findings or tissue characteristics of the breast being examined, it can be advantageous to obtain additional or supplementary information about the breast tissue. In practice, a volumetric image of the breast is therefore frequently acquired using tomosynthesis or ultrasound. However, a disadvantage of tomosynthesis is that the patient is exposed to increased radiation.

[0005] During an ultrasound scan, also known as a sonography scan, ultrasound waves are emitted from an ultrasound probe towards the object being examined, such as a patient's breast. These ultrasound waves are scattered, refracted, and reflected by the various structures of the object. The returning ultrasound waves are received by the probe, then analyzed and converted to create an image of the interior of the object. This image can be generated in either two or three dimensions.

[0006] To effectively combine the different techniques of ultrasound and X-ray mammography and to simplify the diagnostic process, combination devices are used in practice. In such a combined X-ray 3D ultrasound device (hereinafter referred to as an XUS device), an ultrasound probe is typically positioned between the X-ray source and the detector of the X-ray machine. It can be located on either the side of the object being examined facing the detector or the side facing the X-ray source. The ultrasound probe, the detector, and the compression plates are generally arranged parallel to each other. This arrangement allows for the generation of images with essentially similar views of the object being examined using the two different imaging techniques.

[0007] However, establishing a spatial correlation between a two-dimensional X-ray image (mammogram) and a position in a three-dimensional ultrasound image or volumetric ultrasound dataset proves difficult. For example, if a radiologist identifies a position in the X-ray image that lies within an area of ​​interest, a clear assignment of that position to a corresponding position in the ultrasound image is not possible. This is because, due to the inherent nature of a projection image, spatial information is lacking. Furthermore, the X-ray beam path is conical in shape. Consequently, a position in the X-ray projection image or a pixel of the X-ray detector cannot simply be assigned the elements in the three-dimensional ultrasound image that correspond to the structures of the object under investigation that lie on a perpendicular line passing through the pixel to the detector surface.

[0008] To determine the precise position, a second X-ray projection image is often taken in practice at a defined angle to the first X-ray projection image. This allows for the precise determination of the position relative to the XUS device and thus also within the three-dimensional ultrasound image. However, this is disadvantageous for the patient due to the increased radiation exposure.

[0009] In principle, a transformation of the coordinate system of the ultrasound volume dataset is also possible. For this, the ultrasound volume dataset is transformed into the conical geometry of the X-ray beam path using standard transformation and interpolation methods. The ultrasound volume dataset then correlates with the X-ray projection image, making visualization readily possible. However, after the transformation, the ultrasound volume dataset is a distorted image, which makes accurate diagnosis impossible or at least extremely difficult.

[0010] From publication EP 3 336 803 A1, an image verification method and a system for carrying out the method are known, which enriches the information displayed in the reference image or in the synthetic views and / or the slabs reviewed by the physician with colored information layers. These layers are derived from an image volume and combined with the reference image, the slabs, and / or the synthetic views. Enriching the reference image, the slabs, and / or the synthetic views with colored layers highlights the colored areas of interest or features of the image volume in the slabs and / or the synthetic views in a way that prevents the features from being lost due to tissue overlay effects or other problems in combining the colored layers and the reference image to form the displayed combined images.

[0011] From the publication BENEDIKT SCHAEFGEN et al.: "Initial results of the FUSION-X-US prototype combining 3D automated breast ultrasound and digital breast tomosynthesis", EUROPE-AN RADIOLOGY, Vol. 28, No. 6, January 4, 2018 (2018-01-04), pages 2499-2506, XP055537415, DE ISSN: 0938-7994, DOI: 10.1007 / s00330-017-5235-8 , A prospective feasibility study using the FUSION-X-US prototype for digital breast tomosynthesis and ABVS in 23 patients is known. These patients had an indication for tomosynthesis according to current guidelines, based on clinical examination and standard imaging. The prototype's ABVS and tomosynthesis images were interpreted separately by two experts. The study compares the detection and BT-RADS(E) scores of breast lesions using only the FUSION-X-US prototype's tomosynthesis and ABVS data with the results of a complete diagnostic workup.

[0012] From publication EP 3 219 261 A1, a method for breast image reconstruction is known, which includes positioning a breast on a carrier plate of an imaging system, compressing the breast with a flexible paddle, obtaining imaging data, and estimating a breast thickness profile by at least one of the following methods: placing markers on the breast, performing an image-based analysis of the obtained data, using an auxiliary system, and performing a model-based calculation. The three-dimensional reconstruction includes the use of a breast surface thickness profile in at least one of the following: an iterative reconstruction, a filtered back-projection reconstruction, and a joint reconstruction performed using information obtained from an ultrasound scan.

[0013] It is therefore an object of the present invention to specify a local mapping between a two-dimensional X-ray projection image and an ultrasound volume data set and to enable good diagnosis at the lowest possible radiation dose.

[0014] This problem is solved by a method for determining an area of ​​interest according to claim 1, a method for displaying an area of ​​interest according to claim 10, an image processing device according to claim 11 and an X-ray system according to claim 12.

[0015] The aforementioned method for determining a region of interest (ROI) to be represented in an ultrasound volume dataset of the interior of an object under investigation, acquired using an X-ray 3D ultrasound device, comprises at least the steps according to claim 1.

[0016] The object of examination can, in principle, be any object that can be examined using X-rays and ultrasound. However, the preferred method is the breast of a human, female patient. The area of ​​interest to be visualized is one that requires special attention during the examination or diagnosis. This could include, for example, lesions, microcalcifications, nodules, or similar findings.

[0017] The data, i.e., the X-ray projection image and the ultrasound volume data set, are acquired before the actual method according to the invention begins, in particular using an X-ray 3D ultrasound device (XUS device). This is a combination device with which both X-ray images and ultrasound images of the object under examination can be generated. It therefore preferably comprises an X-ray source and an X-ray detector and also includes an ultrasound probe designed to emit ultrasound waves and detect their reflections from the object under examination. Thus, the XUS device is preferably a combined ultrasound mammography device, corresponding to the object under examination.

[0018] For example, in a preparatory step before the inventive method, the XUS device can be used to acquire both the two-dimensional X-ray projection image and the three-dimensional ultrasound volume data set.

[0019] The X-ray projection image and / or the ultrasound volume data set can, for example, also be stored in a memory and retrieved from it to carry out the method according to the invention. The X-ray projection image is preferably a mammogram. The projection position is generally understood to be a position within the X-ray projection image, i.e., for example, a pixel, preferably corresponding to a pixel of an X-ray detector. The projection position is preferably, and particularly preferably, as centrally as possible, within the area of ​​interest. Its determination can be carried out by user input and / or by means of a computer algorithm ("computer-aided detection" - CAD), as will be explained in more detail later. The projection position is determined in particular in only one, or exactly one, X-ray projection image, since this advantageously keeps the radiation exposure for the patient low.

[0020] The projection beam is a line connecting the projection position, or a corresponding detector pixel, to the X-ray source, i.e., the X-ray focus. The X-ray source has a conical beam geometry. This results in oblique angles of incidence, deviating from perpendicular incidence, particularly at the detector's edges. Neglecting scattering effects, the pixel, or image point, at the projection position represents, as its pixel value or brightness, a summed or integrated information about the tissue of interest that was penetrated along the projection beam.

[0021] To fully visualize this information within the three-dimensional ultrasound volume dataset without using the entire dataset, a portion of the data is selected and a subset is created, i.e., extracted. This is done based on the projection position. That is, the data selection is based on a mapping or correlation between the projection position and at least a portion of the ultrasound volume dataset that corresponds to the projection position—that is, data containing information about the same areas of the object under investigation as the pixel of the projection position. The subset preferably represents a small portion of the ultrasound volume dataset, for example, less than 20%, preferably less than 10%, so that a radiologist can quickly gain a detailed understanding of the area of ​​interest.

[0022] Furthermore, geometric and positional information relating to the mammography 3D ultrasound device is used during the extraction of the partial dataset. This includes, in particular, the relative position—that is, the position in the translational degrees of freedom and / or the orientation in the rotational degrees of freedom—between the X-ray source and the detector, and thus also between the X-ray source and the pixel of the detector or the image point in the X-ray projection image. This information allows the path of a projection beam from the X-ray source to the detector to be traced. This enables the correlation of the areas of the object examined that were penetrated by the beam with the corresponding data from the ultrasound volume dataset and the projection position. However, geometric information can encompass other data as well, as will be explained in more detail later.

[0023] The method according to the invention can be repeated for each area of ​​interest of the object under investigation, so that all areas of interest of the object under investigation can be viewed or assessed in a simple manner.

[0024] In the aforementioned method for displaying an area of ​​interest, a number of views from a partial data set, which was determined according to a method according to the invention for determining an area to be displayed, are provided and at least one view is displayed.

[0025] The number of views can be one or more views that represent a (pseudo-)three-dimensional, preferably two-dimensional, representation of the subdataset. The number of views can therefore, for example, include a single two-dimensional image or a stack of layered images containing multiple images.

[0026] The provision of views can occur independently of their display, for example, by storing them in memory. Regardless of whether they are generated directly or retrieved from memory, they can be displayed using a standard display device such as a monitor, other display, 3D glasses, a projector, or similar.

[0027] The aforementioned image processing device comprises all components for carrying out one of the methods described above according to the invention.

[0028] In principle, the image processing device can operate independently of the modalities required for data acquisition. Therefore, it can also process data from separate X-ray and ultrasound devices according to the invention. For example, the position of a transducer of the ultrasound device relative to the object under investigation and / or X-ray device can be determined or tracked using known sensory methods. This also allows the relative position of the X-ray projection image and the three-dimensional ultrasound data set to be determined.

[0029] The aforementioned X-ray system comprises an image processing device according to the invention and an X-ray 3D ultrasound device. The latter includes a source-detector arrangement configured for capturing an X-ray projection image of an object under examination. The XUS device also includes an ultrasound arrangement configured for capturing an ultrasound volume data set. The XUS device is preferably a mammography unit that, compared to conventional mammography units, additionally features an ultrasound arrangement.

[0030] The image processing device according to the invention can advantageously be retrofitted into existing X-ray systems. However, it is also possible to equip newly manufactured X-ray systems with an image processing device according to the invention during the manufacturing process.

[0031] The essential components of the image processing device according to the invention can be predominantly implemented as software components. However, these components can also be partially implemented as software-supported hardware, such as FPGAs or the like, particularly when very fast calculations are required. Likewise, the necessary interfaces, for example, when it is only a matter of transferring data from other software components, can be implemented as software interfaces. Alternatively, they can be implemented as hardware interfaces controlled by suitable software.

[0032] In particular, the image processing device according to the invention can be part of a user terminal of an X-ray system.

[0033] A largely software-based implementation has the advantage that even previously used image processing equipment can be easily retrofitted via a software update to operate according to the invention. In this respect, the problem is also solved by a corresponding computer program product with a computer program that can be directly loaded into a memory device of an image processing unit of an X-ray system, containing program sections to execute all steps of the method according to the invention when the program is run in the image processing unit. In addition to the computer program, such a computer program product may optionally include additional components such as documentation and / or additional components, including hardware components such as hardware keys (dongles, etc.) for using the software.

[0034] For transport to and / or storage on or in the image processing device, a computer-readable medium, such as a memory stick, a hard drive, or other portable or permanently installed data carrier, can be used, on which the program sections of the computer program that can be read and executed by a computer unit of the image processing device are stored. The computer unit can, for example, include one or more cooperating microprocessors or similar components.

[0035] Further, particularly advantageous embodiments and developments of the invention result from the dependent claims and the following description, wherein the independent claims of a claim category may also be further developed analogously to the dependent claims or parts of the description of another claim category and, in particular, individual features of different embodiments or variants may be combined to form new embodiments or variants.

[0036] In principle, it is possible to limit the area to be displayed by, for example, excluding peripheral areas located near the chest wall. Preferably, the partial dataset includes, in particular, at least all the partial data assigned to areas of the object examined that were penetrated by the projection beam. The partial dataset thus includes, in particular, all the information accumulated in the pixel of the projection position. That is, it contains at least the areas or parts of the ultrasound volume dataset that are assigned to a connecting line from the corresponding detector pixel to the X-ray source. This ensures that the interpreter finds at least the formation in the partial dataset that they expect based on the determined projection position. Furthermore, the partial dataset can include additional data, particularly from the area of ​​interest, as will be explained in more detail below.

[0037] In a first variant, the partial dataset is preferably represented as a surface. This surface is derived, for example, as the intersection of the ultrasound volume dataset with a plane, which, in a mathematical sense, represents an infinitely extended two-dimensional object. The plane is defined, for example, using three vectors or the three-point form. A first vector, or position vector, can be freely defined as the origin vector and is therefore, for example, located at a point (the origin of the defined coordinate system) on a perpendicular projection of the X-ray source onto the detector. The plane can further be spanned, for example, by the following two position vectors: a first direction vector (U or z) perpendicular to the detector surface, i.e., in the direction from the origin to the X-ray source (or vice versa) and / or a first direction vector (U or z) in a propagation direction, i.e., in a main propagation direction, of the ultrasound waves, a second direction vector (V) in the direction from the origin to the projection position.

[0038] Choosing the first direction vector in the direction of propagation of the ultrasound waves is particularly advantageous, as this makes shadowing effects, such as the sound shadowing caused by a lesion, easily recognizable.

[0039] Due to the geometric construction, the projection beam automatically lies entirely within the previously described plane. By intersecting this plane with the ultrasound volume dataset, a two-dimensional sub-dataset is advantageously defined or extracted, containing all the relevant information regarding the areas of the object under investigation that were penetrated by the projection beam. This area can then be easily provided and displayed as a two-dimensional view or image, for example, using the method described at the beginning.

[0040] In a second variant, the sub-dataset preferably includes data from a defined volume range.

[0041] In a simple way, the volume area can be defined as the area which is at most 2 cm, preferably at most 1 cm, away from the associated projection position in a direction perpendicular to the main propagation direction of the ultrasound waves.

[0042] The volume area preferably comprises a corridor through the object under investigation, extending in a first direction of the projection beam and having a corridor width of at most 2 cm, particularly preferably at most 1 cm. The fact that the corridor extends in a first direction of the projection beam means that its principal direction of extension is in this direction. The first direction of the projection beam is preferably its component perpendicular to a principal direction of propagation of the ultrasound waves. Particularly preferably, this component is also parallel to the detector surface. The corridor width specifies the extent of the corridor in a direction perpendicular to the principal direction of extension, which is preferably also parallel to the detector surface and / or perpendicular to the principal direction of propagation of the ultrasound waves.

[0043] The volume domain preferably comprises an angular domain. The vertex of the angular domain is particularly preferably located at the position of the X-ray source. For example, with a conical geometry of the angular domain or of the beam path emitted by the X-ray source, the intersection between the angular domain and the ultrasound volume data set results in a cone segment.

[0044] A three-dimensional subset of the data is thus generated, which at least also includes the areas associated with the projection beam. The area encompassed by the subset becomes larger at the edges of the detector, i.e., larger the further the projection position is from the X-ray source.

[0045] However, the partial dataset particularly preferably comprises data from a cuboid-shaped area (bounding box) of the ultrasound volume dataset, which encompasses the previously described angular range, in particular precisely. This advantageously simplifies the following representation.

[0046] The partial dataset can be represented, as described above, for example, as serially arranged, i.e., successive, slice images of a slice image stack ordered according to their position. The slice images can each be arranged, for example, parallel to the detector surface or perpendicular to the projection beam. In particular, the position or depth in the object under investigation corresponds to the position in the slice image stack.

[0047] Preferably, the projection ray forms a central axis of the angular range. That is, the angular range is arranged such that the projection ray extends through the center point of a cross-sectional area with the angular range that is perpendicular to it. Such a central arrangement of the projection ray ensures that the assessor can also include areas adjacent to the areas assigned to the projection ray in the assessment.

[0048] The angular range preferably extends at angles of no more than 15°, particularly preferably no more than 10°, and most preferably no more than 5°, to the projection beam. This allows, in particular, the edge regions of the detector, where the radiation intensity and thus the diagnostic quality in the X-ray projection image is lower, to be examined over a larger area in the ultrasound volume dataset or sub-dataset. This ensures good diagnostic accuracy even in these areas.

[0049] The partial dataset preferably comprises partial data corresponding to a volume of at most 3 cm³, particularly preferably at most 2 cm³, and most preferably at most 1 cm³. This limitation of the partial dataset ensures that the examiner does not have to evaluate large stacks of tomographic images. This allows them to advantageously concentrate on the details in the area of ​​interest.

[0050] The partial dataset is preferably analyzed using automatic detection and / or automatic classification of the area of ​​interest. For this purpose, the partial dataset can be transmitted, for example, to an integrated or external CAD system. Using established CAD methods, the exact position of the area of ​​interest is then determined; thus, the area of ​​interest is localized. Alternatively or additionally, classification is performed using established CAD methods. This determines, for example, whether the tissue change is benign or malignant. Depending on the requirements, a more precise classification of the tissue change can also be performed.

[0051] Preferably, the ultrasound volume dataset and the two-dimensional X-ray projection image were acquired during a single compression step of the object under examination. This avoids the cumbersome registration of the acquired data against each other. This also saves time during the examination, reduces patient discomfort, and saves on personnel and equipment costs.

[0052] The geometric information preferably includes at least one of the following: beam geometry, distance between X-ray source and detector, thickness of the object under investigation, distance between X-ray source and projection position.

[0053] Beam geometry describes the geometric properties of the beam path of an X-ray source. Among other things, parallel and conical beam geometries are known, with conical beam geometries being the most frequently used in practice due to their simpler technical implementation. The emission angle or aperture angle is used, among other things, to further characterize a conical beam geometry.

[0054] The distance between the X-ray source and the detector describes the shortest distance between the two components. In practice, X-ray devices are known where this distance, e.g., d, can be variably adjusted using an adjustment mechanism. In these cases, the distance between the X-ray source and the detector must be determined. This can be done, for example, within the control mechanism of the adjustment device itself, i.e., via internal position detection and / or a built-in stepper motor, and / or by means of an additional distance or rangefinder, such as a laser rangefinder.

[0055] The thickness of the object under investigation also corresponds to geometric information relating to the XUS device. It describes the propagation of the object in a direction perpendicular to the detector surface. The thickness of the object can be determined, for example, by measuring the distance between the detector and a compression plate that can be lowered onto it, i.e., moved towards and away from the detector. This distance can be measured using the methods described above. Alternatively or additionally, the thickness of the object can also be determined from the ultrasound volume data set. If the position of the projection beam is known, this allows for the determination of an entry point into and an exit point from the object, and consequently, the areas penetrated by the beam within the object.

[0056] As described above, the distance between the X-ray source and the projection position corresponds to the distance between the X-ray source and the detector pixel assigned to the image point of the projection position. This distance can be easily determined if the distance of the X-ray source to the detector and the dimensions of the detector itself are known. Preferably, the relative position of the projection position to the X-ray source is determined. This allows the path of the projection beam to be reconstructed as a connecting line between the projection position or the responding detector pixel and the X-ray source.

[0057] The projection position is determined automatically. The production position can be entered manually by an operator or examiner, for example, by clicking on a corresponding point in the displayed X-ray projection image, entering point coordinates, or similar methods; however, such input is not required. The projection position is determined automatically using a CAD (computer-aided detection) algorithm. Numerous such algorithms are known from practice, employing methods such as threshold analysis, edge detection, and / or similar techniques.

[0058] In combination, the determination can be carried out, for example, by the operator entering an area within which the automatic determination should take place. Conversely, within the framework of a CAD algorithm, an area can first be determined within which the operator specifies the projection position. Furthermore, it is also possible to suggest a projection position using the CAD algorithm, which can then be corrected by the operator if necessary. However, such a combination is not used.

[0059] Overall, the invention makes it possible to reduce a relatively large ultrasound volume dataset for diagnostic purposes to a small subset by using the X-ray projection image as an overview or for selecting the area of ​​interest or the projection position. This enables faster yet thorough diagnostics, whereby the diagnostician is only confronted with the types of image material already familiar from their daily professional practice. Furthermore, the X-ray dose for the patient is advantageously kept low, since only one X-ray projection image needs to be acquired as an overview.

[0060] The invention is explained in more detail below with reference to the accompanying figures and exemplary embodiments. The same components are designated with identical reference numerals in the various figures. The figures are generally not to scale. They show: FIG 1 a rough schematic representation of an embodiment of an X-ray system according to the invention, FIG 2 a schematic block diagram of an embodiment of an image processing device according to the invention with a display device, FIG 3 a schematic block diagram for the sequence of an embodiment of a method according to the invention for determining and displaying an area of ​​interest, FIG 4 a schematic perspective view of the extraction of a partial data set according to a first embodiment of a method according to the invention for determining an area of ​​interest to be displayed, FIG 5 a schematic perspective view of the extraction of a partial data set according to a second embodiment of a method according to the invention for determining an area of ​​interest to be displayed, and FIG 6 a view.which was determined by means of an embodiment of a method according to the invention for representing an area of ​​interest from an ultrasound volume data set.

[0061] In FIG 1 An exemplary and roughly schematic X-ray system 1 according to the invention, here a mammography system, is shown. Relative directional terms such as "top" and "bottom," etc., refer to a mammography system 1 set up as intended for operation. The mammography system 1 comprises a combined X-ray-ultrasound device 2 (XUS device), here a mammography-ultrasound device, and a computer system 20. The XUS device 2 has a stand 17 and a source-detector assembly 3, which in turn comprises an X-ray source Q and a detector 5 with a detection area D. During operation, the stand 17 rests on the floor. The source-detector assembly 3 is slidably connected to it so that the height of the detector area D, i.e., the distance to the floor, can be adjusted to the breast height of a patient.

[0062] One of the patient's breasts B (shown schematically here) rests on the detector surface D as the examination object B. An ultrasound array 6 is positioned above breast B and detector surface D. This array also serves as a compression plate and is slidably connected to the source-detector array 3. For the examination, breast B is compressed and simultaneously fixed in place by lowering the ultrasound array 6 onto it, thus exerting pressure on breast O between the ultrasound array 6 and detector surface D.

[0063] The X-ray source Q is positioned opposite the detector 5 and is designed such that the detector 5 detects the X-rays emitted by the source Q after at least part of the X-rays has penetrated the patient's breast O. Thus, an X-ray projection image is acquired. The source-detector assembly 3 can be pivoted relative to the support column 17 by means of a rotating arm 18 within a range of, for example, ± 90° around a basic position in which it is perpendicular to the base. By pivoting the source-detector assembly 3, data of the breast B can be acquired from different body directions of the patient, e.g., craniocaudal, mediolateral, mediolateral-oblique, or the like.

[0064] The ultrasound setup 6 includes an ultrasound probe (not shown here) which, during operation, emits ultrasound waves towards breast B. The ultrasound waves are scattered, refracted, and reflected by the different structures of breast B. The returning ultrasound waves are received by the ultrasound probe and subsequently evaluated and converted so that a three-dimensional ultrasound volume dataset (3D-US) of the interior of breast B can be generated.

[0065] Computer system 20 comprises a computer unit 12 and, connected to it, a mouse 13, a keyboard 14, and a monitor 15. The monitor 15 serves as the display unit 15, while the mouse 13 and keyboard 14 each serve as input devices. Computer unit 12 includes an image processing unit 10 and a display unit 11 (shown schematically as blocks here), as well as a drive 16 for reading CDs or DVDs. The image processing unit 10 and the display unit 11 can share components of computer unit 12, such as memory, processors, and the like. Computer system 20 can be located in the same room as XUS device 2, or it can be located in an adjacent control room or at a greater distance.

[0066] In FIG 2 Figure 1 shows an embodiment of an image processing device 10 according to the invention together with an embodiment of a display device 11. The image processing device 10 comprises an input interface 21 and an output interface 22, as well as a detection unit 23 and an extraction unit 24. The input interface 21 is connected to the detection unit 23 and transmits incoming data to the detection unit 23, which includes a three-dimensional ultrasound volume data set 3D-US, an X-ray projection image RB, and geometric information GI.

[0067] For example, a projection position PR1, PR2 is created from the X-ray projection image RB by means of a user input BE (see FIG 4 , FIG5 The determined projection positions PR1, PR2, the 3D-US ultrasound volume data set, and the geometric information GI are transmitted to extraction unit 24. There, based on the projection positions PR1, PR2, and the geometric information GI, a sub-data set CBV1, CBV1', CBV2 is extracted from the 3D-US ultrasound volume data set, as shown by FIG3 bis FIG 5 This will be explained in more detail later. The sub-data set CBV1, CBV1', CBV2 is forwarded to the display unit 11 via the output interface 22.

[0068] The display unit 11 comprises a provisioning unit 25 and a display unit 26. The provisioning unit 25 selects a number of views from the sub-dataset CBV1, CBV1', CBV2 in which the area of ​​interest can be displayed as conveniently as possible for a diagnostician. These views are processed by the display unit 26 so that they can be displayed by a standard display unit such as the screen 15.

[0069] FIG3 The diagram shows a roughly schematic block diagram that, in two steps IV and V, illustrates the process of an embodiment of a method according to the invention for determining an area of ​​interest to be displayed, and in two steps VI and VII, illustrates the process of an embodiment of a method according to the invention for displaying an area of ​​interest.

[0070] In a preparatory first step I, which is not included in the inventive method, the patient's breast B is compressed between the detector 5 and the ultrasound array 6 by lowering the ultrasound array 6 onto the detector 5 until a predefined pressure is exerted on the breast B. In the compressed state, the breast B can be examined more easily using both X-rays and ultrasound and is also fixed in such a way that no movement or repositioning of the breast B occurs between the individual acquisition steps described below.

[0071] In a preparatory second step II, which is also not included in the inventive method, an X-ray projection image RB is acquired. As already described above, X-rays are generated by means of an X-ray source Q, penetrate the breast B, and then, attenuated, strike the detector surface D and are detected depending on their intensity. The attenuation or absorption, i.e., the reduction of the detected intensity, of the X-rays depends on the structures or the density of the irradiated tissue of the breast B. The acquired X-ray projection image RB can, for example, be displayed on the screen 15.

[0072] In a preparatory step III, which is also not included in the inventive method, a three-dimensional ultrasound volume dataset (3D-US) is acquired. For this purpose, ultrasound waves in the form of ultrasound pulses are emitted from different positions relative to the breast B, similar to a scan, according to a method known to those skilled in the art. These waves are scattered, refracted, and reflected by the different structures of the object under investigation. The returning ultrasound waves are received by the ultrasound probe and subsequently evaluated and converted into the three-dimensional ultrasound volume dataset (3D-US).

[0073] Steps II and III can, in principle, be performed independently of each other, as long as the transmission of the X-ray radiation through the ultrasound arrangement 6 is ensured. However, the acquisition of the X-ray projection image RB preferably takes place before the acquisition of the ultrasound volume data set 3D-US, since this allows a subsequent fourth step IV to be performed in parallel.

[0074] In the fourth step IV, the actual embodiment of the inventive method for determining an area of ​​interest to be represented begins, the steps of which are also described in detail using the following: FIG 4 and 5The fourth step, IV, describes the determination of a projection position PR1, PR2. This determination can be performed, for example, by manual input from a user BE. Additionally or alternatively, a CAD algorithm can be executed, which automatically detects the area of ​​interest and, if necessary, the projection position PR1, PR2 using known methods. The projection position PR1, PR2 corresponds to a point on the detector surface D.

[0075] In a fifth step V, a subset of data CBV1, CBV1', CBV1" CBV2 is extracted from the 3D-US ultrasound volume dataset. This is done based on the projection position PR1, PR2 and a set of geometric information GI relating to the XUS device 2, as illustrated below with two examples. FIG 4 and FIG 5 will be explained.

[0076] FIG 4 Figure 1 shows an exemplary schematic perspective representation illustrating the extraction of two partial datasets, CBV1 and CBV2. The X-ray source Q is positioned at a perpendicular distance or height h above the detector surface D. The detector 5 consists of a multitude of pixels arranged in a matrix on the detection surface D. The pixels are organized into mutually perpendicular rows and columns extending in the principal directions of the detector 5 (not shown here for clarity). For illustration, an outline of the breast B is depicted on the detector surface D. Additionally, a right-handed Cartesian coordinate system with an x-direction, a y-direction, and a z-direction is shown for easier orientation. The z-direction corresponds to a direction from the X-ray source Q to a projection point of the X-ray source Q on the detector surface D.The y-direction corresponds to a principal direction of detector 5, which essentially points away from the patient, and the x-direction, which corresponds to the other principal direction of detector 5, results from a combination of the two previous directions to form a right-handed Cartesian coordinate system.

[0077] The first projection position PR1 determined in step IV corresponds to a position or pixel of the detector area D. A first projection beam R1 extends from the X-ray source Q to the first projection position PR1. The length of the first projection beam R1 corresponds to the distance a1 between the X-ray source Q and the projection position PR1. The position of the first projection beam R1 relative to the 3D-US ultrasound volume dataset can be determined using the geometric information GI. This means that the position of the first projection beam R1 relative to the detector area D and the X-ray source Q can be determined, since the relative position of the X-ray source Q to the detector area D is also known or can at least be determined. For this purpose, the positions of the X-ray source Q and the detector area D can be described, for example, in a common coordinate system (x, y, z).

[0078] From the 3D-US ultrasound volume dataset, the first sub-dataset CBV1, corresponding to projection position PR1, is selected as follows. First, a plane is constructed, spanned by a first direction vector U1 and a second direction vector V1. The first direction vector U1 extends in a direction from the projection point PQ of the X-ray source Q to the first projection position PR1. The second direction vector V1 extends here (for clarity and without loss of generality) in a direction opposite to the z-direction (though, for simplicity, the second direction vector V1 could also extend in the z-direction). The first sub-dataset CBV1 (shown here only schematically and in part) is then created from the intersection of the previously described plane and the 3D-US ultrasound volume dataset.Due to its construction, the sub-dataset includes at least the data corresponding to the areas penetrated by the projection beam R1. Furthermore, it is possible to further reduce the size of the first sub-dataset CBV1 by extracting only a defined area of ​​the intersection sufficient for diagnosis, encompassing the projection position, preferably in the center.

[0079] For the second projection position PR2, a construction for extracting the second sub-dataset CBV2 is shown for further illustration, which can be followed analogously based on the explanations just given for the first sub-dataset CBV1.

[0080] FIG 5 shows an exemplary schematic perspective representation to illustrate the extraction of two sub-datasets CBV1', CBV1''. FIG 5 is essentially similar to FIG 4 , which is why only the different extraction of the sub-datasets CBV1', CBVl"' will be explained in more detail below.

[0081] The position of the projection beam R1 in relation to the ultrasound volume data set is analogous to FIG 4 This is known. However, instead of a plane, a truncated cone is constructed to select the third sub-dataset CBV1'. The truncated cone represents the conical beam characteristic or beam geometry of the X-ray source Q. It has an imaginary apex, a central axis, and an opening angle. The imaginary apex is located at the same position as the X-ray source Q. The central axis extends in one direction of the projection beam R1 through the centers of cross-sectional areas of the truncated cone that are perpendicular to it. The opening angle denotes an opening angle of the cone to which the central axis bisects the angle. Here, for example, it is 10°. The projection position P1 is located in the center of the truncated cone. That is, the truncated cone extends in the direction of the projection beam and in the opposite direction to both sides of the projection position PR1 within a defined area.The third sub-dataset CBV1' is formed from an intersection of the truncated cone constructed in this way and the ultrasound volume dataset 3D-US.

[0082] To simplify the representation of the third sub-dataset CBV1', additional data from adjacent areas of the 3D-US ultrasound volume dataset can be included. Accordingly, a fourth sub-dataset CBV1" is extracted, preferably as a cuboid bounding box, which encompasses the third sub-dataset CBV1' within a region of the 3D-US ultrasound volume dataset defined as larger than the truncated cone. The fourth sub-dataset CBVL" is preferably arranged such that its principal directions of extension correspond to the xyz directions of the previously described Cartesian coordinate system.

[0083] The previously described defined areas for the sub-datasets CBV1', CBVL" are preferably defined such that they correspond to a volume of at most 3 cm 3< , preferably at most 2 cm 3< , particularly preferably at most 1 cm 3< .

[0084] The subsets CBV1, CBV1', CBV1", CBV2 are also known as cone-beam views and can be stored separately. They represent a subset of the 3D-US volumetric ultrasound dataset that can be interpreted more quickly.

[0085] To FIG 4 and 5 It should be noted that these figures are by no means drawn to scale. For example, the height h of the X-ray sources Q above the detector area D is usually considerably greater than shown here. The arrangement has also been simplified for clarity. Unlike what is shown, the X-ray source Q is preferably positioned as centrally as possible above the detector area D.

[0086] The extracted sub-datasets CBV1, CBV1', CBV1", CBV2 are now re-analyzed in the following using FIG 3 Steps VI and VII, as described above, illustrate an embodiment of a method according to the invention for displaying an area of ​​interest. For this purpose, a number of views are first provided in a sixth step VI. This provision can be achieved, for example, by receiving the views directly from the image processing unit or retrieving them from memory. For the two-dimensional image subsets CBV1 and CBV2, for example, a display area can be selected. For the three-dimensional volume datasets CBV1' and CBVL', for example, a viewing direction on the three-dimensional volume dataset can be defined, and corresponding views can be generated.

[0087] If, as with the sub-datasets CBV1', CBVL', several views 30 exist, one or more (with simultaneous display) of the views 30 can be selected for display in step VII. For example, the first view 30 can be selected that includes the projection position. Further views can be displayed, for example, as a successive sequence of images ordered according to depth, especially as a film, or controlled by user input such as scrolling or the like.

[0088] The only representation takes place in an eighth step VIII, by transmitting the number of views to be displayed 30 to a display unit, such as the screen 15.

[0089] One such view 30 is exemplary in FIG 6The projection position PR1 was determined in the X-ray projection image RB in an area of ​​interest on the object B. This position is also visualized in view 30 of a partial dataset determined according to the invention by means of a crosshair 33. The position of the projection beam R1 relative to view 30 is also marked. The partial dataset comprises at least the data from the 3D-UV ultrasound volume dataset, which originates from a corridor 31 of the object B. The corridor 31 extends in the plane of view 30 (which is arranged essentially perpendicular to the main propagation direction of the ultrasound waves) and parallel to the projection beam R1. The corridor has a width 32 of 2 cm.

[0090] The present invention thus enables simple and rapid diagnosis, since it is not necessary to consider the entire ultrasound volume data set, but rather a preselection of the area of ​​interest is made in the X-ray projection image. Since only one X-ray projection image needs to be acquired as an overview, the radiation exposure for the patient is advantageously low.

[0091] Finally, it should be noted once again that the devices and methods described in detail above are merely exemplary embodiments, which can be modified in various ways by a person skilled in the art without departing from the scope of the invention. For example, only one mammography system was described above, but the invention can, in principle, apply to any X-ray system with a combined X-ray 3D ultrasound device. Furthermore, the use of the indefinite articles "a" or "an" does not preclude the possibility that the features in question may be present multiple times. Likewise, the terms "device," "unit," "arrangement," and "system" do not preclude the possibility that the component in question consists of several interacting sub-components, which may also be spatially distributed.

Claims

1. Method carried out by a computer (20) for determining a region of interest to be rendered in an ultrasound volume data set (3D-US) of the interior of an object under examination (B), comprising at least the following steps: - automatically determining a projection position (PR1, PR2) in a two dimensional X-ray projection image (RB) of the object under examination (B), wherein a projection ray (R1, R2) assigned to the projection position (PR1, PR2) passes through the region of interest and - extracting a partial data set (CBV1, CBV1', CBV1 ", CBV2) encompassed by the ultrasound volume data set (3D-US) using the projection position (PR1, PR2) and on the basis of geometrical information (GI) relating to the X-ray 3D ultrasound unit (2), wherein the selection of the data takes place according to a correlation between the projection position and at least some of the data of the ultrasound volume data set corresponding to the projection position.

2. Method according to claim 1, wherein the partial data set (CBV1, CBV1', CBV1 ", CBV2) at least comprises partial data which is assigned to regions of the object under examination (B) that have been penetrated by the projection ray (R1, R2).

3. Method according to claim 2, wherein the partial data set (CBV1, CBV2) is implemented as a surface.

4. Method according to claim 1 or 2, wherein the partial data set (CBV1`; CBV1") comprises data from a defined volume range.

5. Method according to claim 4, wherein the volume range encompasses a corridor (31) through the object under examination (B) that extends in a first direction of the projection ray (R1, R2) and has a corridor width (32) of no more than 2 cm, preferably of no more than 1 cm.

6. Method according to one of claims 4 or 5, wherein the partial data set comprises partial data corresponding to a volume of no more than 3 cm3, preferably of no more than 2 cm3, with particular preference of no more than 1 cm3.

7. Method according to one of the preceding claims, wherein the partial data set is analysed by means of automatic detection and / or automatic classification of the region of interest.

8. Method according to one of the preceding claims, wherein the ultrasound volume data set (3D-US) and the two dimensional X-ray projection image (RB) have been acquired during a compression step of the object under examination (B).

9. Method according to one of the preceding claims, wherein the geometrical information (GI) comprises at least one of the following items of information: beam geometry, distance (h) between X-ray source (Q) and detector (D), thickness of the object under examination (B), distance (a1, a2) between X-ray source (Q) and projection position (PR1, PR2).

10. Method for rendering a region of interest, wherein - a number of views (30) from a partial data set (CBV1, CBV1', CBV1", CBV2) that has been determined according to a method according to one of claims 1 to 9 are provided and - at least one view (30) is rendered.

11. Image processing device (10) for determining a region of interest to be rendered in an ultrasound volume data set (3D-US) of the inside of an object under examination (B), said data set having been acquired by means of an X-ray 3D ultrasound unit, comprising - a determination unit which is designed to automatically determine a projection position (PR1, PR2) in a two dimensional X-ray projection image (RB) of the object under examination (B), wherein a projection ray (R1, R2) assigned to the projection position (PR1, PR2) passes through the region of interest (ROI) and - an extraction unit which is designed to extract a partial data set (CBV1, CBV1', CBV1", CBV2) from the ultrasound volume data set (3D-US) using the projection position (PR1, PR2) and on the basis of geometrical information (GI) relating to the mammography 3D ultrasound unit (2), wherein the selection of the data takes place according to a correlation between the projection position and at least some of the data of the ultrasound volume data set corresponding to the projection position.

12. X-ray system (1) having an image processing device (10) according to claim 11 and an X-ray 3D ultrasound unit (2) comprising - a source-detector arrangement (3) which is designed to obtain an X-ray projection image (RB) of an object under examination (B) and - an ultrasound arrangement (6) which is designed to acquire an ultrasound volume data set (3D-US).

13. Computer program product comprising a computer program which can be loaded directly into a storage device of an image processing device (10), having program sections for carrying out all the steps of a method according to one of claims 1 to 10 when the computer program is executed in the image processing device (10).

14. Computer-readable medium on which program sections readable and executable by the processing unit are stored in order to carry out all the steps of a method according to one of claims 1 to 10 when the program sections are executed by the processing unit.