Method for generating training data for an AI-based assistance system and device for supporting X-ray diagnostics

Abstract

Method for generating synthetic image pairs from a 3D X-ray CT dataset for training an AI-based assistance system for X-ray diagnostics, comprising the following steps: Defining the coordinates of a virtual X-ray arrangement in an X-ray room coordinate system (XRS) comprising a virtual X-ray light source, a virtual planar X-ray detector array and an intermediate virtual transmission area, wherein the beam direction of the virtual X-ray light source is perpendicular to the virtual X-ray detector array; Providing a 3D X-ray CT dataset comprising at least three-dimensional voxel coordinates and X-ray attenuation coefficients assigned to the voxels, and selecting the voxels representing the target anatomy of a patient as a CT model of the patient; Arranging the CT model in the virtual radiographic field by varying the pose of the CT model through rotation and / or translation in the virtual radiographic field such that a plurality of image poses compatible with the target anatomy are generated, wherein For each image pose, measured values ​​on the virtual X-ray detector array are calculated as a synthetic X-ray image based on the X-ray attenuation coefficients. characterized by Defining the pose of at least one virtual depth imaging camera in the RKS with a view towards the virtual transmission area and the virtual X-ray detector array; Defining a 3D point mesh enveloping the CT model for each image pose as a native patient skin model; Determining a three-dimensional representation of the virtual X-ray detector array and the native patient skin model in the depth image camera coordinate system (DCS) and Calculating a synthetic two-dimensional depth image using depth information from the three-dimensional representation; Combine one synthetic X-ray image and one synthetic depth image to form a synthetic image pair.

Description

[0001] The invention relates to a method for generating image pairs that can teach an assistance system, by means of a machine learning method, to transfer the evaluations of a set of first images of the image pairs to the set of second images, so that the output of evaluations is made possible solely on the basis of previously unknown second images.

[0002] The invention also relates to a device for supporting medical X-ray imaging. In particular, the invention relates to an assistance system for medical X-ray equipment that, based on artificial intelligence (AI) – here: machine learning (ML) – makes assessments and, if necessary, automatically generates signal outputs and initiates actions based on these assessments.

[0003] With the widespread availability of modern and powerful microprocessors and now efficient machine learning algorithms, interest in AI-based systems for solving complex tasks has increased in recent years. This is particularly true for digital image processing, where patterns in megapixel datasets often need to be detected and evaluated at high speed, e.g., at high video frame rates, which can quickly tire or otherwise overwhelm a human observer.

[0004] Artificial neural networks (ANNs) represent a potentially very useful tool for camera-based environmental analysis, particularly for autonomous vehicles. They are trained using pre-evaluated training data to generalize the evaluation of captured image information and thus arrive at "meaningful" decisions when presented with previously unknown images. A particularly appealing aspect of automated learning using pre-evaluated examples is the possibility of machine-implementing complex functionalities that cannot yet be precisely formulated linguistically or even expressed in mathematical formulas. This leads to the frequently pursued approach of translating human experience into program code that—while not fully comprehensible—can nevertheless be copied as often as desired and is then permanently available to a user, at least in an advisory capacity.

[0005] In medical applications, experience is particularly in demand, and not only the time-consuming process of training medical professionals suggests increasing the level of automation in devices for patient diagnosis and therapy using AI methods.

[0006] Since their introduction, X-ray images have proven indispensable for the in vivo examination of the human skeleton, and even simple X-ray machines remain in use for this purpose in many medical practices. The correct positioning of a patient—or rather, a patient's extremity—between an X-ray light source and a device for capturing the X-ray image is crucial for the diagnostic quality, that is, the diagnostic value for the physician, of a resulting skeletal X-ray image.

[0007] Proper patient positioning on the first attempt requires experience from medical personnel. Multiple exposures of the patient to ionizing radiation should be avoided for health and organizational reasons, meaning that even with today's common digital X-ray detectors offering instant image acquisition, a trial-and-error approach to correct positioning is not feasible. Instead, the physician or medical assistant must still rely on their own expert judgment.

[0008] To facilitate and objectify the positioning task, it has been proposed to optically and electronically capture the patient before taking an X-ray image using conventional camera technology based on the visual (VIS) to near-infrared (NIR) spectrum, and to use the captured data to control an X-ray machine or to estimate the patient's pose. For example, US patent 9,433,395 B2 proposes continuously monitoring the patient with pose-detecting sensors, including a depth camera, and transmitting any changes in the patient's pose to a control device that can reposition the movable X-ray light source and / or the movable X-ray detection device.In this way, a patient positioning once determined by medical personnel can be maintained even if the patient should move unintentionally before the X-ray light source is triggered.

[0009] Patent specification CN 113180709 B does not address the possibility of moving the X-ray device itself, but rather the ability to give the patient instructions on how to change their position so that a high-quality diagnostic X-ray image can be produced immediately. Here, too, the patient is electronically captured by VIS / NIR cameras, and a continuous positional assessment is calculated. This calculation must be based on an assumed ideal patient position in order to identify deviations from it. It can be assumed that such an ideal position is unsuitable for a large number of patients due to anatomical variability. The handling of patients who cannot be positioned according to the system remains unclear.

[0010] As a point to note, a distinction must be made between a patient's pose during an image acquisition and the pose of a 3D image of the patient. In computer vision based on electronic cameras, the (choice of) pose refers to the definition of coordinates in a predetermined coordinate system for the three translational and three rotational degrees of freedom of a 3D object. A patient's pose has additional degrees of freedom due to their internal mobility (e.g., the position of the ankle, knee, etc.), which are also taken into account by medical personnel when positioning the patient for the X-ray. In the following, the term "pose" will only be used in the context of computer vision.

[0011] The inventors are currently unaware of any AI-based solution for predicting the diagnostic quality of an X-ray image to be acquired from a VIS / NIR image of the patient or a target anatomy of the patient. A plausible reason for this is a lack of suitable training datasets. The reasons for this lack are: a) There is currently no X-ray machine on the market that simultaneously captures VIS / NIR camera images of the patient from any predetermined perspective alongside the X-ray images, let alone images with 3D or depth information. Therefore, there are also no ready-made example image pairs from medical practice for training. b) Even if an X-ray machine is experimentally equipped with a VIS / NIR camera, the overall system is still without medical approval and cannot, for example, be easily tested on living patients by medical professionals in a clinic. c) Furthermore, it would be irresponsible to deliberately position living patients “incorrectly” and generate X-ray images of poor diagnostic quality, just to produce negative examples for KNN training. d) X-raying living volunteers without a medical indication solely for the purpose of obtaining image pairs is generally not a good option.

[0012] In this situation, the only option is to synthetically generate image pairs for training an AI.

[0013] For example, training images for KNN systems for use in road traffic can now be synthesized, whereas previously real images were captured and annotated by human employees (see online publication of the Fraunhofer Institute for Physical Measurement Techniques, 2022, https: / / www.ipm.fraunhofer.de / de / gf / objekterfassunglaserscanning / anw / datenvisualisierung / synthetische-trainingsdaten-KNN-strasse-umfeld.html). The main argument for using synthetic images here is cost and time savings.

[0014] The publication US 2021 / 118131 A1 specifically addresses the generation of X-ray images for training an AI-based analysis system, with the goal of quickly recognizing and locating a specific structure within a complex image. For this purpose, multiple augmented X-ray images are synthesized from a 3D X-ray CT dataset containing a container representing an object of interest (e.g., a weapon). A portion of the CT dataset representing the object of interest is isolated, and a series of synthetic X-ray images of this object are initially generated from different illumination directions. The X-ray images of the object are then individually inserted into an X-ray image of the container, which serves as a background with a complex image structure, possibly after translation and scaling of their original sizes.Augmentation thus consists of generating X-ray images on the computer, showing the object inside the container potentially at different locations, sizes, and angles. Although the publication also mentions medical applications, its primary purpose appears to be AI-supported inspections of goods, for example, at borders. VIS / NIR camera images, especially depth images, are not used in this process.

[0015] The document “Booij R, Budde RPJ, Dijkshoorn ML, van Straten M. Accuracy of automated patient positioning in CT using a 3D camera for body contour detection. Eur Radiol. 2019 Apr;29(4):2079-2088. doi: 10.1007 / s00330-018-5745-z. Epub 2018 Oct 10. PMID: 30306328; PMCID: PMC6420476” describes an AI-based assistance system for X-ray diagnostics that was trained using depth images. In the study described, the accuracy of automated patient positioning using a 3D camera was compared with manual positioning by radiologic technologists. The results show that the 3D camera exhibited significantly smaller deviations from the ideal table height than manual positioning.

[0016] Furthermore, “Singh, Vivek et. al.: DARWIN: Deformable Patient Avatar Representation With Deep Image Network. In: Medical Image Computing and Computer-Assisted Intervention - MICCAI 2017: 20 thInternational Conference, Quebec City, QC, Canada, September 11-13, 2027, Proceedings, Part II 20. Springer International Publishing, 2017. pp. 497-504” a technical solution that aims to robustly estimate the detailed body surface area of ​​a patient under clothing using a single snapshot from a rangefinder. For this purpose, deep convolutional networks are trained using real clinical datasets with large variations and augmentations.

[0017] The object of the invention is to propose a method for generating synthetic image pairs for training a k-nearest neighbors (KNN) algorithm for an X-ray diagnostic assistance system. A further object of the invention is thus implicitly to propose a device as an assistance system for X-ray diagnostics based on machine learning, the geometric arrangement of whose components must be known before the generation of the training data and implemented in the same way after training. The generated image pairs define features of the device, and conversely, the device determines the generation of the image pairs.

[0018] The task is solved by a method for generating synthetic image pairs from a 3D X-ray CT dataset for training an AI-based assistance system for X-ray diagnostics, comprising the following steps: Defining the coordinates of a virtual X-ray arrangement in an X-ray room coordinate system (XRS) comprising a virtual X-ray light source, a virtual planar X-ray detector array and an intermediate virtual transmission area, wherein the beam direction of the virtual X-ray light source is perpendicular to the virtual X-ray detector array; Providing a 3D X-ray CT dataset comprising at least three-dimensional voxel coordinates and X-ray attenuation coefficients assigned to the voxels, and selecting the voxels representing the target anatomy of a patient as a CT model of the patient; Arranging the CT model in the virtual radiographic field by varying the pose of the CT model through rotation and / or translation in the virtual radiographic field such that a plurality of image poses compatible with the target anatomy are generated, wherein For each image pose, measured values ​​on the virtual X-ray detector array are calculated as a synthetic X-ray image based on the X-ray attenuation coefficients. characterized by Defining the pose of at least one virtual depth imaging camera in the RKS with a view towards the virtual transmission area and the virtual X-ray detector array; Defining a 3D point mesh enveloping the CT model for each image pose as a native patient skin model; Determining a three-dimensional representation of the virtual X-ray detector array and the native patient skin model in the depth image camera coordinate system (DCS) and Calculating a synthetic two-dimensional depth image using depth information from the three-dimensional representation; combining one synthetic X-ray image and one synthetic depth image to form a synthetic image pair.

[0019] The dependent claims specify advantageous embodiments of the process.

[0020] Experts are well aware that synthetic X-ray images of a patient's target anatomy—e.g., lower leg, knee, hand—can be generated from a 3D X-ray CT dataset encompassing the target anatomy, using any desired radiation direction. In a sense, this is the reverse of obtaining the CT dataset from individual X-ray images. In the simulation, the X-ray equipment is not typically moved around the patient; instead, a virtual 3D patient, represented as a CT model, can be translated and rotated as desired within a predetermined radiation field in a fixed X-ray room coordinate system (RKS). This allows for the definition of an image pose for the target anatomy. However, if realistic X-ray images are to be simulated, these image poses must be compatible with the target anatomy within the RKS.If, for example, the patient is standing with their back to or lying on the flat X-ray detector array, then not all poses for the knee or foot are suitable for imaging. Firstly, a patient lying on their back cannot perform every possible rotation of their knee, and secondly, a target anatomy can at best touch a support surface or detector array, but never penetrate it. It is quite obvious, especially to an expert, which image poses in which X-ray configuration can be considered compatible with the target anatomy.

[0021] The synthetic X-ray images are not directly used for training a k-nearest neighbors (KNN) algorithm. Rather, after generating the synthetic image pairs according to the invention, an evaluation of the diagnostic quality of the synthetic X-ray images is performed. Only these diagnostic quality metrics, in conjunction with the synthetic depth images of each image pair, are used as training data for the KNN algorithm.

[0022] A medical professional can certainly assess the diagnostic quality of simulated X-ray images, even if they don't look entirely realistic. However, it is alternatively and preferably possible to use a separately prepared KNN (Kinky-Nearest Neighbors) algorithm that has already learned to assess diagnostic quality through expert evaluation of real X-ray images. With the help of such a KNN evaluator, the training dataset, which initially only includes the synthetic image pairs, can be automatically supplemented with diagnostic quality metrics. A functional KNN evaluator can be found, for example, in the work by Mairhöfer et al., Proceedings of the Fourth Conference on Medical Imaging with Deep Learning, PMLR 143:484-496, 2021.

[0023] However, a KNN evaluator should be provided with synthetic images that are as realistic as possible for evaluation, since they have been trained on precisely such images. Therefore, when simulating the images, it is advisable to consider the material properties, or at least the X-ray scattering coefficients, from pre-tabulated tissue data. For example, a mapping of Hounsfield Units (HU) from the 3D X-ray CT dataset to the material can be performed. The materials and their physical properties can be found, for example, in PENELOPE 2006 (F. Salvat, JM Fernandez-Varea and J. Sempau, PENELOPE - A code system for Monte Carlo simulation of electron and photon transport, NEA-OECD, Issy-les-Moulineaux, www.nea.fr / html / dbprog).Following the translation of the CT dataset into a 3D material model, individual photons from the X-ray source and their path to the X-ray detector can be simulated using a ray-tracing method to calculate an X-ray image. This is also a known state of the art.

[0024] The invention focuses on the additional generation of at least one synthetic depth image for each synthetic X-ray image, wherein the at least one depth image is defined as a realistic output of the measured values ​​from at least one depth imaging camera. The camera is positioned with its view towards the transmission area and the detector array of the X-ray apparatus. It is readily possible to use multiple depth imaging cameras arranged around the transmission area with different viewing directions simultaneously for training a KNN X-ray assistant; however, even then, synthetic image pairs are always required in the sense that a measure of diagnostic quality is assigned to the first image, and the same measure is assigned to the second image by correspondence with the first image.

[0025] For the sake of simplicity, we will henceforth refer only to a depth imaging camera.

[0026] According to the invention, the information required for simulating the depth images can be directly extracted from the same 3D X-ray CT datasets used to simulate the X-ray images. By selecting an X-ray attenuation value (also known as Hounsfield Units or HU value) in the CT data such that precisely those voxels are selected in which an increase in X-ray attenuation values ​​compared to air is observed, the selected voxels represent a material-containing surface. In a further selection step, the selected voxels are then differentiated according to whether they belong to the target anatomy of the patient or to the surrounding environment, for example, a patient support. Only the voxels representing the target anatomy form the basis for the CT model of the patient, which is to be X-rayed in the virtual X-ray setup. This CT model is also suitable for defining a patient skin model.For example, a marching cube algorithm can be used to surround the voxels of the CT model with a 3D point mesh to highlight the surface shape of the CT model. If a fine-meshed mesh closely surrounds the surface, the patient's skin is effectively traced. In this context, a native patient skin model is understood to be the 3D point mesh that surrounds the voxels of the CT model of the target anatomy as accurately as possible. It thus originates directly from the available raw data.

[0027] The depth imaging camera, which typically captures light in the VIS / NIR spectrum, detects not only the patient's skin but also parts of the surrounding environment, particularly the X-ray detector behind or below the patient. The resulting depth images are typically point clouds in Cartesian coordinates, where a coordinate plane is indexed with the pixels of the light-sensitive camera chip, and distance values ​​to the camera chip are recorded. A depth image representation consisting of pixel coordinates and distance is commonly referred to as 2.5-dimensional. The depth imaging camera coordinate system (DCS) is fixed to the depth imaging camera and is also determined by defining the pose of a virtual depth imaging camera relative to the virtual X-ray setup. In other words, the transformation of the representation of an object in the RCS to a representation in the DCS is predetermined after the camera pose is selected.

[0028] To generate a synthetic image pair according to the invention, the CT model of the target anatomy is first separated from the 3D X-ray CT dataset, positioned in the virtual X-ray setup in an image pose compatible with the target anatomy, and then virtually X-rayed – as is known per se. Furthermore, the native patient skin model is also defined by the CT model and is known in the X-ray system. The virtual depth camera "sees" the patient skin model and the X-ray detector array (or a patient support comprising a detector array) from its defined pose, which is defined in the X-ray system. A coordinate transformation of the objects visible to the camera from the X-ray system is performed to obtain a 3D representation of these objects in the X-ray system.The calculated point cloud still contains numerous points that remain hidden from the depth camera because they are obscured by other points or by surface elements between them—in the case of the native patient skin model, the points represent the nodes of a planar mesh, meaning the meshes are opaque. These obscured points are identified and removed from the dataset to generate a simulated depth image.

[0029] The invention will be explained in more detail below, also with the aid of figures. This will show: Fig. 1. A sketch of the invention problem using the example of X-ray images of an ankle for different image poses of the foot; the stars on the right represent a measure of the diagnostic quality; Fig. 2 a patient skin model (here of the foot) extracted from a 3D X-ray CT dataset as well as a patient dressing visible from the CT data; Fig. 3 a) a real depth image of a prepared human foot, b) a simulated depth image from a measured CT dataset of the foot from a) without softening and c) the simulated depth image from b) after softening; Fig. 4 Two views of a functional prototype of an X-ray apparatus with integrated assistance system in the inventors' laboratory; the depth imaging camera is highlighted;

[0030] The Fig. Figure 1, using the example of an X-ray of an ankle, outlines the object of the invention. Because different foot positions result in different X-ray images with correspondingly different diagnostic quality – marked by stars on the right as a measure – a way should be found to directly assign the foot position to these measures and only generate a real X-ray image after this assignment. For this purpose, the foot position must first be determined from a VIS / NIR depth image that is harmless to the patient.

[0031] The in Fig. The two depicted 3D point grids show both the actual skin contour of the patient whose CT dataset is being evaluated, i.e., the native patient skin model of a foot, and a portion of the patient support on which the foot rested during the CT scan. The points attributable to the patient support can be omitted. According to the invention, the CT model of the foot is first separated from any other irrelevant CT data. Then, the native patient skin model is determined—typically once per CT dataset. The coordinate transformations for the CT model and the native patient skin model are always identical.

[0032] If the patient were thicker or thinner, or had local thickenings, e.g., due to ulcers, the patient skin model can be relatively easily modified by locally or globally shifting the coordinates of the grid points of the 3D point network in the computer. In this way, multiple modified patient skin models can be synthesized from a single native patient skin model, allowing for the simulation of correspondingly modified depth images. This serves to improve the generalization capabilities of the KNN (cognitive neural network) being trained with the depth images.

[0033] In a preferred embodiment of the patient skin model modification, the size is changed by positively or negatively shifting all points along the local normal direction of the 3D grid. The local normal direction can be determined, for example, by simply calculating the tangent at each grid point as a function of the coordinates of neighboring grid points. Such shifts make the patient appear either thicker or thinner than in reality, while their skeleton remains unchanged in all variations. For each selected image pose of a CT model, exactly one X-ray image and a plurality of depth images can then be synthesized, resulting in a corresponding plurality of image pairs per image pose and CT dataset.

[0034] It is considered a preferred embodiment to calculate a synthetic depth image for a modified patient skin model for a predetermined image pose and to combine it with the synthetic X-ray image of the image pose to form a synthetic image pair.

[0035] To simulate synthetic X-ray images, different image poses compatible with the foot are selected. While no impossible poses are used, some very poor-quality poses are employed to generate only low-quality X-ray images. For each of these image poses, a three-dimensional point cloud is generated in the field of view of the virtual depth camera within the RKS (Real-World System). This point cloud is created by fusing the 3D point mesh of the foot and a point representation of the detector array. This point cloud can then be transformed into the TKS (Transverse X-ray System) connected to the depth camera.

[0036] From its predetermined perspective on the patient, the depth imaging camera only sees those points in the fusion point cloud that are closest to it in the field of view. In other words, a simulated depth image can be determined by removing all points from a fusion point cloud, knowing the camera's position, that cannot be captured by the camera due to occlusion caused by their coordinates. It is important to note that the patient's skin model represents a closed surface, meaning that occlusion also occurs between the points closest to the camera.

[0037] The Fig. Figure 3 a) shows a real depth image of a prepared human foot, which is also used for real CT measurements. If one proceeds as described above with a CT model of the foot to simulate a depth image from the same perspective as in the real scan, Fig. 3 a), then the synthetic depth image is initially Fig. 3 b). A comparison of the images reveals some differences that make post-processing advisable.

[0038] Due to the technical characteristics of depth cameras – in this case, time-of-flight (ToF) – object transitions do not produce clear boundaries as in Fig. 3 b), but characteristic soft transitions as in Fig. 3 a). In the synthetic depth images of the Fig. 3 b) These do not initially exist because clear surface boundaries are calculated and therefore precisely known, unlike in a real measurement. To simulate the smooth transitions, relevant object boundaries are automatically determined and subjected to a blurring process. A common method of blurring, which is also used here, Fig. 3 b) is applied, for example, as can be seen on Wikipedia at https: / / en.wikipedia.org / wiki / Kernel_(image_processing). To clarify: The blurring only occurs for selected pixels at the predefined object boundaries in the image. Fig. 3 c) shows the synthetic depth image of the Fig. 3 b) after applying the blurring.

[0039] An additional characteristic of depth images is the occurrence of invalid points where the depth cannot be determined for whatever reason. Such points appear black in the depth image. This occasionally happens at object transitions with a large difference in depth (e.g., between a toe and a table in [reference to a specific example]). Fig. 3a)). Since such measurement errors do not occur in the synthetic images, they can be artificially added. This can be done by creating a gradient image, where each pixel is assigned a measure of the depth difference to its neighboring pixels. The pixels in the gradient image with the highest absolute values ​​correspond to the areas in the image with the greatest depth variation. It is usually sufficient to identify the pixels with the highest values ​​in the gradient image and set them to 0 in the synthetic depth image to simulate invalid points. This post-processing is optional.

[0040] Generally, for a machine learning algorithm to be successful, it is crucial that the training data closely resembles the actual input images captured by the depth camera. Otherwise, there is a risk that the algorithm will search for extraneous features in the images and classify them accordingly. Therefore, blurring the depth images, at least in the area of ​​object transitions, is a preferred precautionary measure. Adding invalid points can also be a helpful training option.

[0041] In Fig.Figure 4 shows a functional prototype of the assistance system according to the invention in a structural unit with a commercially available X-ray device, shown in the upper image from the patient's support surface and in the lower image in a side view. The permanently installed depth camera, which is arranged in additional 3D-printed housing parts, is marked by a border in both images. The X-ray detector array is not shown in the images; it is located here in the plane of the patient's support surface, which is a very common configuration.

[0042] Due to the technical modification, the functional prototype is no longer approved for use on patients, but can still, for example, measure phantoms and / or specimens of human cadavers. However, the typically very limited number of available phantoms / specimens does not provide a good basis for creating image pairs for machine learning. The far larger quantity of real 3D X-ray CT datasets from medical practice is made accessible for use in KNN training purposes with the help of the image pair simulation according to the invention.

[0043] According to the invention, an AI-based assistance system for X-ray diagnostics, trained with synthetic depth images and diagnostic parameters from the aforementioned process steps, comprises at least one processor for executing the trained artificial neural network (ANN), at least one depth imaging camera communicating with the processor, and an interface for data exchange with a medical X-ray apparatus, wherein the X-ray apparatus comprises at least one planar X-ray detector array, an X-ray light source shining perpendicularly onto the X-ray detector array, and an intermediate transmission area. The system is characterized in that the at least one depth imaging camera is arranged in the same pose, with a viewing direction towards the transmission area and the detector array, as was also defined as the pose of the virtual depth camera for generating the synthetic image pairs.

[0044] Preferably, the assistance system can be designed such that the depth camera continuously captures a depth image of the patient on the patient support, with the processor being configured to determine a metric for the diagnostic value of the captured depth image in real time, keeping pace with any change in the patient's pose within the radiographic field. In other words, a real-time assessment of the patient's pose, and thus simultaneously of the diagnostic quality of an X-ray image in that pose, takes place.

[0045] The assistance system according to the invention can be a separate device that is attached to an existing X-ray machine and communicates with it via the aforementioned interface. The interface serves at least for the electronic transmission of the assistance system's real-time assessment. Preferably, the assistance system is configured such that the processor generates an output and communicates at least with the X-ray machine, the output preventing the X-ray light source from being triggered if the depth measurement determined for the acquired image does not meet a predetermined minimum criterion for diagnostic quality. In this way, unnecessary X-ray exposures of the patient are reliably avoided.

[0046] An advantageous design of the assistance system can consist of the processor calculating the distance between the depth measurement determined for the acquired image and the fulfillment of the minimum criterion for diagnostic quality, and outputting this as an acoustic and / or visual proximity signal. The assistance system can be equipped with an additional sound generator or a signal lamp for this purpose, and, for example, a series of sound or light pulses with a variable repetition rate can indicate, analogous to a collision warning system in a car, how the patient can move into a position suitable for the X-ray image.

[0047] In view of medical-technical certification requirements, it is preferable to provide the assistance system with the X-ray apparatus in a single structural unit, whereby the pose of the depth imaging camera is precisely predetermined by design through its arrangement in a housing.

[0048] In summary, the inventive method for synthesizing training images for each image pose of a CT model of a target anatomy in a virtual X-ray setup allows the calculation of a synthetic image pair consisting of an X-ray image and a depth image for each—optionally modified—patient skin model. Of particular importance is the fact that this yields a large number of simulated X-ray images with low to very low diagnostic quality, which could not have been acquired from a living patient, while simultaneously generating the appropriate, realistic depth images. Only with this training data is it currently possible to establish an assistance system for predicting the diagnostic quality of depth images using an artificial neural network. The device's features must be appropriately configured with respect to the synthetic training data.

Claims

[1] Method for generating synthetic image pairs from a 3D X-ray CT dataset for training an AI-based assistance system for X-ray diagnostics comprising the steps: Defining the coordinates of a virtual X-ray arrangement in an X-ray room coordinate system (XRS) comprising a virtual X-ray light source, a virtual planar X-ray detector array and an intermediate virtual transmission area, wherein the beam direction of the virtual X-ray light source is perpendicular to the virtual X-ray detector array; Providing a 3D X-ray CT dataset comprising at least three-dimensional voxel coordinates and X-ray attenuation coefficients assigned to the voxels, and selecting the voxels representing the target anatomy of a patient as a CT model of the patient; Arranging the CT model in the virtual radiographic field by varying the pose of the CT model through rotation and / or translation in the virtual radiographic field such that a plurality of image poses compatible with the target anatomy are generated, wherein For each image pose, measured values ​​on the virtual X-ray detector array are calculated as a synthetic X-ray image based on the X-ray attenuation coefficients. characterized by Defining the pose of at least one virtual depth imaging camera in the RKS with a view towards the virtual transmission area and the virtual X-ray detector array; Defining a 3D point mesh enveloping the CT model for each image pose as a native patient skin model; Determining a three-dimensional representation of the virtual X-ray detector array and the native patient skin model in the depth image camera coordinate system (DCS) and Calculating a synthetic two-dimensional depth image using depth information from the three-dimensional representation; Combine one synthetic X-ray image and one synthetic depth image to form a synthetic image pair. [2] Method according to claim 1, characterized by , that the native patient skin model is modified by moving nodes of the 3D mesh along the local normal direction. [3] Method according to claim 2, characterized by that all nodes are shifted in the positive or negative normal direction. [4] Method according to one of claims 2 or 3, characterized by , that a synthetic depth image is calculated for a modified patient skin model for a predetermined image pose and combined with the synthetic X-ray image of the image pose to form a synthetic image pair. [5] Method according to any of the preceding claims, characterized by, that a digital blurring of the depth information of the synthetic depth images is carried out at least in the area of ​​object transitions. [6] Method according to any of the preceding claims, characterized by , that each synthetic depth image of an image pair is assigned a measure of diagnostic quality based on the synthetic X-ray image of the image pair.

Images