Method and apparatus for testing algorithms or systems for processing, evaluating or reconstructing x-ray-based images

Abstract

The invention relates to a method for testing algorithms or systems for processing, evaluating or reconstructing x-ray-based images, comprising the steps: - Providing at least one working image (A) in the form of a CT image, - Generating a number of simulation images (S) from the at least one working image (A), wherein for image points (V) of the simulation images (S) image values ​​of the working image (A) are summed along a line (X), - Output of the simulation images (S), especially for testing a module for processing, evaluating or reconstructing images or for training a machine learning network. Furthermore, the invention comprises a device, a control unit and an imaging X-ray system.

Description

[0001] The invention relates to a method and a device for testing algorithms or systems for processing, evaluating or reconstructing x-ray-based images, a control device for an imaging x-ray system and an imaging x-ray system.

[0002] In X-ray imaging, it is common practice to test image reconstruction systems, image evaluation systems, or image control systems for an imaging system. For this purpose, the data from the imaging system is transferred to the respective system under test, and tests are then performed. For example, raw data is fed to an image reconstruction system so that images can be reconstructed and these images can then be reviewed by an expert for image defects.

[0003] Typically, customer workflows in computed tomography (CT) include a topographic scan and a main scan. Rotational, static, and sequence scans are also frequently performed, which differ in their acquisition procedure and, consequently, in their reconstruction. Topographic scans contain different content than main scans. This means that two different sets of raw data exist for each type of scan. For realistic testing, both sets of raw data must originate from the same object, as the areas selected in the topographic scan must also be present in the main scan. If the areas do not exist or do not match, the tests cannot be performed, or incorrect body regions will be displayed. This prevents proper testing of the entire workflow or verification of the algorithms, for example, if the topographic image shows the head region while the main scan only shows lung images.

[0004] If a detector does not (yet) actually exist, or if patients should not be exposed to unnecessary radiation to generate test data, the corresponding test data can be simulated. However, this data usually does not accurately reflect reality and generally does not include the correct detector behavior.

[0005] Furthermore, test data must be generated for each scan protocol. This can result in an extremely large amount of data. Consider the testing of an image reconstruction system for a new CT product type. CT systems typically have more than 160 scan protocols, and each of these protocols can be performed in multiple variations. Additionally, depending on the scan, rotation time, pitch, X-ray voltage, beam current, and other parameters can be changed. Therefore, several thousand files could easily be required for the product type being tested.

[0006] For example, if the image reconstruction system is only to be tested for a single topographic scan, the following raw data sets are required: For each patient position (face up / down, left and right lateral decubitus, and head or feet first for these positions), two data sets are needed for the table orientation (a total of 16 possibilities), and for each table orientation, two data sets are needed for different tensions, for a total of 32 data sets. If further variations are added, such as different rotation speeds or different integration times, the number of variations must be multiplied by the 32 data sets.

[0007] If the topographic protocols need to be changed, corresponding new test data must be generated. The patient or phantom positions must match the previous topographic positions; otherwise, the main scan positions will no longer be correct.

[0008] However, this is only the minimum set of test data. Typically, additional parameter variations are added, depending on the scan range, such as pitch (0.35 - 1.5 in 0.05 increments), rotation (0.25, 0.30, 0.33, 0.50, 1.0), voltage (70kV - 140kV), current (any value), and many more.

[0009] For example, if tests are to be conducted based on images showing an entire human body, a main scan covering an area of ​​two meters would typically take 50-80 seconds. An image file can be up to 200GB per scan (i.e., a single data set). For the number of test data sets mentioned in the example above, the required storage capacity could easily reach several hundred terabytes if many variations are to be tested.

[0010] In addition to image reconstruction, system test procedures must also be tested with real detector raw data. However, no images can be generated without real detector hardware.

[0011] Real-world raw data is dependent on tuning tables. If the detector hardware is changed, the images can only be used to a limited extent, as artifacts may appear in the images. This renders older raw data unusable.

[0012] Previously, a fixed position was defined for a full-body phantom. The head was positioned at a specific location, e.g., 500 mm. Topographic scans were then performed from 400 mm to 2455 mm to create a 2 m topographic scan. Main scans were performed using the same positions.

[0013] Each time the scan protocol is changed, the scans with the full-body phantom are repeated in the same positions and the raw data is recorded. This can easily take several hours for a large number of required datasets.

[0014] For non-existent detector hardware or tests irrelevant to customer workflows, simulated raw data was used. If the detector hardware is changed, all protocols must be updated, as the tuning tables will no longer be valid and artifacts will appear in the images.

[0015] It is an object of the present invention to provide a method and a device for testing algorithms or systems for processing, evaluating, or reconstructing X-ray-based images, a control device for an X-ray imaging system, and an X-ray imaging system itself, with which the disadvantages described above are avoided. In particular, it is an object of the invention to generate as many test data sets as possible in a short time using a minimal "basic data set".

[0016] This problem is solved by a method according to claim 1, a device according to claim 11, a control device according to claim 12 and an imaging X-ray system according to claim 13.

[0017] A method according to the invention serves to test algorithms or systems for processing, evaluating, or reconstructing X-ray-based images. It comprises the following steps: - Providing at least one working image in the form of a CT image, - Generating a number of simulation images from the at least one working image, whereby for each pixel of the simulation images, image values ​​of the working image are summed along a line, - Outputting the simulation images, especially for testing a module for processing, evaluating or reconstructing images or for training a machine learning network.

[0018] This method is used to test algorithms or systems for processing, evaluating, or reconstructing X-ray-based images. These can be, for example, image reconstruction systems, evaluation systems, or image processing systems (in hardware or software implementation). The method is also suitable for generating training data to train a machine learning model intended for use in a corresponding system. For example, the method is suitable for generating raw CT data that can be used to test an image reconstruction system or that can be reconstructed into CT images to test evaluation or image processing systems.

[0019] First, at least one working image in the form of a CT scan is provided. The working image is therefore a 3D image of an object, in particular a patient, which is preferably reconstructed from projection images and is preferably available as a stack of tomographic images. The working image should have been acquired with an imaging system for which the relevant systems are to be tested, but this is not strictly necessary. If the relevant imaging system is not accessible or does not yet exist, a working image can also be acquired with a different imaging system. It is particularly preferred that the base materials of the object captured in the working image are known, e.g., bone or soft tissue.It is particularly preferred that the working image is available as a dual- or multi-energy image, or that several working images of the same subject are available, which have been recorded with different X-ray energies.

[0020] A working image preferably shows at least part of the body of a patient, a phantom, or an object. Ideally, only one image is needed. However, multiple images are possible, particularly when simulating different patient positions (prone, lateral) or projection images with varying energy levels. In such cases, several CT images with different acquisition energies may be required. Alternatively, a single CT image could be used where each pixel represents a vector of values ​​acquired at different energy levels.

[0021] However, for a good understanding of the following steps, it is sufficient to imagine that a working image is provided which represents a human body in three dimensions.

[0022] In the next step, a number of simulation images are generated from the at least one working image. These simulation images are preferably projection images. Only a single simulation image can be generated, for example, when systems of a radiography system are to be tested, or several can be generated, particularly when systems of a CT or tomosynthesis system are to be tested. In particular, simulation images in the form of projection images can be generated, showing the subject of the number of working images from different perspectives and corresponding specifically to raw data from a CT or tomosynthesis system.

[0023] It is important that for each pixel of the simulation image, the image values ​​of the working image are summed along a line. This can be done very quickly, as summing along a straight line is very simple and time-efficient to calculate with a computer. It is preferred that each pixel of a simulation image has its own line that passes through the working image and a predefined detector plane. The detector plane can be straight or curved and should correspond to the detector plane of the imaging system whose systems are to be tested. It is assumed that the respective pixel lies exactly at the intersection of the line with the detector plane.

[0024] Even if an imaging system does not yet exist, its geometric structure is known. Specifically, the location of the X-ray source, the position of the detector, the intended placement of the patient table, and any elements that could influence the X-ray beam are known. Before each run of the procedure, a bundle of lines can be defined that intersects the detector plane in a grid (corresponding to the pixels of the simulation image) and originates from a common point or at least a common surface. This bundle of lines is referred to as the "beam cone" because it simulates a cone-shaped X-ray beam. The beam cone should have the shape of an X-ray beam emitted from the X-ray source onto the detector, possibly influenced by other elements.Since the scanner geometry is known, the shape of a beam cone is always known, even with different beam variations. It should be noted that there are imaging systems that generate multiple cones of X-rays, for example, systems with two or more X-ray sources emitting simultaneously. In this case, the lines are simply arranged in the form of corresponding beam cones.

[0025] It should also be noted that scattering or cross-scattering can be taken into account. In this case, lines are added to simulate the X-ray radiation of these effects. These lines then add a value to the pixels of the simulated image, weighted according to the intensity of this scattered radiation. Generally, calculation methods for this are known.

[0026] The beam cone can certainly have a predefined intensity profile, for example, a profile where the rays in the center are brighter than those at the edges. This can be achieved with additional factors that are multiplied by the respective sum of the image values ​​of the working image. These factors can also be used to account for beam intensity and shadowing.

[0027] Similarly, consideration can be given to whether the detector allows for energy resolution and / or has photon-counting properties. However, an energy-resolved working image would be advantageous in this case.

[0028] The lines pass through the working image and intersect its pixels. The image value of the corresponding pixel in the simulation image can then be calculated from the sum of the image values ​​of the intersecting pixels. However, it should be noted that it is generally advantageous to normalize this pixel. For example, the path length of the line through each voxel could be taken into account, or the total length of the line through the working image could be considered, or the values ​​could be averaged over a specific number of voxels. In practice, weighting is preferably based on the path length of the ray or the line's trajectory within the image. The image values ​​can be, in particular, linear attenuation coefficients µ (usually µ). Wasser ) for each pixel of the simulation image. The total attenuation is then the sum of all µ * path length of the simulated ray (line) in the voxel.

[0029] To generate multiple simulation images in the form of CT raw data, the beam cone, along with the detector plane, can be rotated around the subject of the working image, and a simulation image can be created at each of the different rotation angles. This allows for the simulation of a patient table's movement. The working image is simply shifted relative to the beam cone and the detector plane.

[0030] It should also be noted that the working images can be manipulated to create different simulation images. For example, an element from other CT images can be added. A working image can be overlaid with a diseased area, or a healthy bone can be replaced with a fractured one. Generally, limited and individually documented disease patterns can be preferentially incorporated into the working images from other CT scans. Organs can also be swapped with organs from other 3D image volumes. It is advantageous to always use images or image sections of real-world subjects so that the simulation images are based on real images. However, it is also possible to use an image of a real-world subject and artificially alter parts of the image content.

[0031] After the simulation images are generated, they are output. They can be used directly for testing a module for processing, evaluating, or reconstructing images, or for training a machine learning network, or they can be saved. However, it is preferable to use them immediately, since the very aim of the invention is to minimize the amount of storage space occupied by test data (sets of simulation images) and instead generate them quickly as needed. If it is already clear from the outset that sets of simulation images will be used multiple times, then saving them could be advantageous.

[0032] A device according to the invention serves to test algorithms or systems for processing, evaluating, or reconstructing X-ray-based images. It comprises the following components: - a data interface designed to receive at least one working image in the form of a CT image, - a simulation unit designed to generate a number of simulation images from the at least one working image, wherein for each image point of the simulation images, image values ​​of the working image are summed along a line, - a data interface designed for outputting the simulation images, in particular for testing a module for processing, evaluating or reconstructing images or for training a machine learning network.

[0033] The function of the device's components has already been described. The device is preferably designed for carrying out a method according to the invention.

[0034] The invention enables the future use of a single input file (the number of working images) for a multitude of different (existing and non-existing) imaging systems, allowing for the quick and easy generation of test datasets tailored to specific situations. If a detector geometry is defined, this approach can generate simulation images for a desired imaging system without requiring actual hardware, preferably as simulated raw data. The input file consists solely of a number of essentially arbitrary 3D image volumes, with each working image containing the desired subject. These could be, for example, DICOM images. The working images are thus essentially used as a virtual patient.

[0035] The method can easily simulate different rotation speeds, integration times, X-ray voltages, beam currents, table speeds, gantry tilt angles, and other parameters, generating corresponding simulation images. Based on a known system architecture, new raw data can be created for previously non-existent imaging systems. Preliminary investigations can therefore be conducted without actual hardware. Using the same working images ensures that simulations of topographic scans, spiral scans, zigzag scans, sequence scans, and other scan types always start from the same initial data. This guarantees consistent training data. Instead of thousands of files, the method requires only a few working images, for example, four, because every typical patient position (face up and down, as well as left and right lateral decubitus positions) is represented.It is preferred that the working images contain stress information. This allows the stress effects in the raw data to be converted. If no images exist for a desired table position, the "air" method can generate raw data. This corresponds to reality, since if there is no patient table or patient in the acquisition area of ​​a CT scanner, "air" raw data is also generated there. Simulation images from aerial scans can be used for system tests, e.g., for calibration workflows.

[0036] AI algorithms can be validated more easily because the working images are defined and known. If working images of tuning phantoms exist, system tuning steps can also be successfully performed. The method according to the invention allows for the testing of very detailed and deeper system functions. This approach enables system experience at a real system level.

[0037] A control device according to the invention for an imaging X-ray system comprises a device according to the invention and / or is designed to carry out a method according to the invention.

[0038] An imaging X-ray system according to the invention comprises a control device according to the invention.

[0039] The invention can be implemented, in particular, in the form of a computer unit with suitable software. The computer unit can, for example, comprise one or more cooperating microprocessors or the like. In particular, it can be implemented in the form of suitable software program components within the computer unit. A largely software-based implementation has the advantage that even previously used computer units can be easily retrofitted by a software or firmware update to operate according to the invention. In this respect, the problem is also solved by a corresponding computer program product with a computer program that can be directly loaded into a memory device of a computer unit, containing program sections to execute all steps of the method according to the invention when the program is run in the computer unit.In addition to the computer program itself, such a computer program product may include additional components such as documentation and / or additional components, including hardware components such as hardware keys (dongles, etc.) for using the software.

[0040] For transport to the computer unit and / or for storage on or in the computer unit, a computer-readable medium, such as a memory stick, a hard drive or other portable or permanently installed data carrier, can be used, on which the program sections of the computer program that can be read and executed by a computer unit are stored.

[0041] Further, particularly advantageous embodiments and developments of the invention result from the dependent claims and the following description, wherein the claims of one claim category may also be further developed analogously to the claims and description parts of another claim category and, in particular, individual features of different embodiments or variants may be combined to form new embodiments or variants.

[0042] Preferably, a working image shows a patient in a supine, prone, right lateral, left lateral, or standing position. Ideally, several working images are available, showing the patient in different positions. Preferably, the patient is oriented with their head towards the gantry or with their feet facing it. Ideally, several working images of the same patient are available, taken in different positions. "Patient" here refers to people, animals, or objects that can be photographed in a prone, supine, or lateral position.

[0043] Preferably, several working images of the same subject are available, taken at different exposure energies. These are preferably dual- or multi-energy images.

[0044] According to a preferred embodiment of the method, a plurality of lines extending from the pixels of a simulation image to a predetermined position of an X-ray source are defined to generate the simulation image. These lines are then used to add the image values ​​of the working image for the respective pixel. As already explained above, it is preferred to define a beam cone (bundle of lines) for this purpose.

[0045] It is preferred that several simulation images are created for different positions of the X-ray source, lying on a circular or spiral path (with the working image unchanged or with a laterally shifted working image). For this purpose, the beam cone can preferably be rotated on a predefined path in space around the respective working image, and the pixels of the simulation images can be calculated for many positions of the beam cone.

[0046] According to a preferred embodiment of the method, the positions of the X-ray source are defined at (in particular constant) angular intervals, which have been determined according to a predetermined rotation time of the gantry and a predetermined integration time of the measurements. This simulates the acquisition of projection images at constant angular intervals.

[0047] According to a preferred embodiment of the method, a plurality of positions are defined based on a predetermined feed rate of a patient table, a predetermined rotation speed of an X-ray source, and predetermined acquisition times. In this way, for example, spiral or zigzag scans can be simulated. It is preferred that: - a spiral scan is simulated by selecting positions that correspond to a constant rotation around a moving patient bed, or - a topographic scan is simulated by selecting positions that correspond to a stationary X-ray source above (preferably at an angle of 0°) or next to (preferably at an angle of 90°) a moving patient table, or - a sequence scan is simulated by selecting positions on several circular paths, each spaced a predetermined detector width apart along the patient bed, or A zigzag scan is simulated by selecting positions that correspond to a patient table moving back and forth and an X-ray source rotating (especially at a constant speed). This allows, for example, a heart to be scanned multiple times to measure the temporal progression of a contrast agent or movement.

[0048] According to a preferred embodiment of the method, the lines are arranged to form a cone (said beam cone). The direction of its perpendicular center to the transverse plane of the working image is preferably inclined at a predetermined angle. It is particularly preferred that the inclination of the angle simulates the tilting of a gantry.

[0049] It is preferred that, in one embodiment of the method, the pixels of the simulation image lie on a flat or curved surface. The shape of the simulation image should correspond to the shape of the inner surface (facing the X-ray source) of the detector of the real or desired imaging system. It is particularly preferred that, if the simulation image (the detector plane) is curved, the radius of curvature corresponds to a defined distance from an X-ray source. For example, flat or curved detectors are frequently used in CT systems. Often, a curved detector with a constant distance of each detector point from the X-ray source is employed.

[0050] It is preferred to simulate the collimation of an X-ray beam by assigning a predetermined reduction in value to pixels at the edge of a projection image. This can be achieved, in particular, using the beam cone and factors as described above. Each line is assigned a factor by which the sum of the image values ​​is multiplied. Smaller factors correspond to lower beam intensity. If the beam is collimated in such a way that part of the "detector" would no longer see anything, the factors of the affected lines can even be zero.

[0051] It is preferred that, in one embodiment of the method, the definition of the lines is based on a predetermined beam direction of the X-ray source. It is further preferred that the weighting of the individual summed values ​​is defined in this respect. This simulates, for example, beams originating from an X-ray source with a spring focus that deflects a beam according to a predetermined beam direction. The intensity profile is preferably also simulated with the beam cone and the factors by representing the intensity profile through the factors.

[0052] According to a preferred embodiment of the method, before generating the number of simulation images, image values ​​of the working image are modified according to a function that simulates an acquisition at a different acquisition energy. It is preferred that the working image has been acquired at several acquisition energies and that the function depends on the difference between the image values ​​in a voxel of the working image and the difference between the respective image value at the different acquisition energies. If a distribution of base materials in the acquired image is known, then the absorption spectra of these base materials can also be used for the simulation. For the creation of a simulation image, an energy is specified, and during the summation, the image values ​​are weighted according to the absorption spectra of the base materials identified in the image pixels.

[0053] According to a preferred embodiment of the method, before generating the number of simulation images, image values ​​of the working image are modified according to a function that simulates an image taken at a predetermined radiation flux or collimator filter or a predetermined aperture. This function is known in advance or is defined by a user.

[0054] Additional image noise can be added to the simulation image if this is appropriate for the detector of the imaging system being simulated.

[0055] According to a preferred embodiment of the method, image information, preferably from other X-ray-based images, is added to the provided working images. This information modifies the depicted patient, in particular by adding a pathology or an implant and / or replacing an organ with a corresponding organ from another CT image. This allows the working images to be modified in a defined way to show predetermined features. This is particularly advantageous for generating training data. A defined modification can be added to a previously known working image, and the knowledge about this modification can be used as the basis for training. For example, a pathology can be added, and then a machine learning model can be specifically trained on this type of pathology.

[0056] Preferably, components of the invention are provided as a "cloud service." Such a cloud service serves to process data, particularly using artificial intelligence, but can also be a service based on conventional algorithms or a service where human evaluation takes place in the background. Generally, a cloud service (hereinafter also referred to simply as "cloud") is an IT infrastructure in which, for example, storage space or computing power and / or application software is provided via a network. Communication between the user and the cloud takes place via data interfaces and / or data transmission protocols. In the present case, it is particularly preferred that the cloud service provides both computing power and application software.

[0057] In a preferred method, data obtained within the scope of the invention is provided to the cloud service via the network. This cloud service comprises a computing system that typically does not include the user's local computer. The method can be implemented using a command structure within a network. The data processed in the cloud is subsequently sent back to the user's local computer via the network.

[0058] The invention is explained in more detail below with reference to the accompanying figures and exemplary embodiments. The same components are designated with identical reference numerals in the various figures. The figures are generally not to scale. They show: Fig. 1. A rough schematic representation of a CT system according to the state of the art, Fig. 2 a block diagram of an example of a method according to the invention, Fig. 3 ways to change parameters of a recording, Fig. 4. The generation of simulation images.

[0059] Fig. Figure 1 shows a computed tomography (CT) system 1 with a radiation detector 4 and an X-ray source 5. The X-ray source 5 is configured to expose the radiation detector 4 with X-rays. The CT system 1 shown comprises a gantry 2 with a rotor R. The rotor R includes the X-ray source 5 and the radiation detector 4.

[0060] The rotor R is rotatable about the axis of rotation 8. The patient P is positioned on the patient table L and can be moved along the axis of rotation 8 through the gantry 2. The processing unit 9 is provided for controlling the CT system 1 and / or for generating an image data set based on signals detected by the radiation detector 4.

[0061] Typically, a (raw) X-ray image dataset of the patient P is acquired from a variety of angular directions using the radiation detector 4. Subsequently, a (final) image dataset can be reconstructed from the (raw) X-ray image dataset using a mathematical procedure, for example, including a filtered backprojection or an iterative reconstruction method.

[0062] The processing unit 9 serves here as a control unit 9 for controlling the CT system 1. An input device 10 and an output device 11 are connected to this processing unit 9. The input device 10 and the output device 11 can, for example, enable interaction by a user or the display of a generated image data set B.

[0063] The computing unit 9 comprises a device 12 for testing algorithms or systems for processing, evaluating, or reconstructing X-ray-based images. The device 12 includes a data interface 13 and a simulation unit 14.

[0064] The data interface 13 is used to receive at least one working image A in the form of a CT image and to output simulation images S, in particular for testing a module for processing, evaluating or reconstructing images or for training a machine learning network.

[0065] Simulation unit 14 is used to generate a number of simulation images S from the at least one working image A, whereby for each pixel of the simulation images S, image values ​​of the working image A are summed along a line X. The generation is performed in Fig. 4. A closer look.

[0066] Fig. Figure 2 shows a method for testing algorithms or systems for processing, evaluating or reconstructing x-ray-based images.

[0067] In step I, a working image A is provided in the form of a CT image. It is indicated here that the working image A is a three-dimensional image in the form of a stack of tomographic images. It represents the body of a patient P and was generated, for example, by the CT system according to... Fig. 1 recorded.

[0068] In step II, a number of simulation images S are generated in the form of projection images from the working image A, whereby for each image point of the simulation images S, image values ​​of the working image A are summed along a line X (see Fig. 4).

[0069] In step III, the simulation images S are output, in particular for testing a module for processing, evaluating or reconstructing images or for training a machine learning network.

[0070] Fig. Figure 3 shows options for adjustable parameters of an image. On the left is gantry 2 of the CT system. Fig. Figure 1 shows a device that can be tilted according to the curved arrow up to the dashed positions. On the right is a patient stretcher that can be moved back and forth along the straight arrow at different speeds. Further parameters relate to the beam, the rotation of the rotator R, and the position of the patient P.

[0071] Fig.Figure 4 shows the generation of simulation images S. On the left, it is indicated that the patient P is irradiated by a beam cone (dashed lines) emanating from the X-ray source 5, and that the beam is captured by the radiation detector and its intensity is measured. If the patient P is the image information of the working image A and the beam cone is a bundle of lines X, the left image would also show the prerequisite for calculating the image values ​​of a simulation image B.

[0072] Suppose we want to calculate the value of the pixel V in the simulation image through which the vertical line X passes at the radiation detector 4. Then, as shown in the left image, the image values ​​of the pixels V of the working image A along the path of line X would be summed and preferably also normalized. As shown in the left image, the sum of all voxels V on the line equals one voxel V of the simulation image (below).

[0073] Finally, it should be noted once again that the invention described in detail above merely represents exemplary embodiments, which can be modified in various ways by a person skilled in the art without departing from the scope of the invention. Furthermore, the use of the indefinite articles "a" or "an" does not preclude the possibility that the features in question may be present multiple times. Likewise, terms such as "unit" do not preclude the possibility that the components in question consist of several interacting sub-components, which may also be spatially distributed. The term "a number" should be read as "at least one." Regardless of the grammatical gender of a particular term, persons of male, female, or other gender identities are included.

Images