Device for determining scan direction and stitching sequence of multiple X-ray images

Abstract

The present invention relates to an apparatus 10 for scan direction detection and stitching sequence determination of multiple X-ray images, the apparatus comprising an input unit 20, a processing unit 30, and an output unit 40. The input unit is configured to provide the processing unit with a first X-ray image acquired by an X-ray image acquisition system, the first image comprising patient image data. The input unit is configured to provide the processing unit with a second X-ray image acquired by the X-ray image acquisition system after the X-ray image acquisition system has been moved relative to the patient, the second image comprising patient image data. The processing unit is configured to determine an upper similarity value comprising a comparison of at least one region of the patient image data at an upper boundary of the first image and / or adjacent to the first image with at least one similarly sized region of the patient image data at a lower boundary of the second image and / or adjacent to the second image. The processing unit is configured to determine a right similarity value comprising a comparison of at least one region of the patient image data at a right boundary of the first image and / or adjacent to the first image with at least one similarly sized region of the patient image data at a left boundary of the second image and / or adjacent to the second image. The processing unit is configured to determine a bottom similarity value comprising a comparison of at least one region of the patient image data at a bottom boundary of the first image and / or adjacent to the first image with at least one similarly sized region of the patient image data at an top boundary of the second image and / or adjacent to the first image. The processing unit is configured to determine a left similarity value comprising a comparison of at least one region of the patient image data at a left boundary of the first image and / or adjacent to the first image with at least one similarly sized region of the patient image data at a right boundary of the second image and / or adjacent to the first image. The processing unit is configured to determine a scan direction and translation distance of the X-ray image acquisition system associated with a movement of the X-ray acquisition system comprising use of a maximum value of the top, right, bottom, or left similarity value, and / or determine a combined image formed from the first image and the second image comprising use of a maximum value of the top, right, bottom, or left similarity value.The output unit is configured to output the scan direction and translation distance and / or the combined image.

Description

[Technical Field]

[0001] The present invention relates to an apparatus for detecting the scan direction and determining the stitching sequence of multiple X-ray images, a system for detecting the scan direction and determining the stitching sequence of multiple X-ray images, a method for detecting the scan direction and determining the stitching sequence of multiple X-ray images, as well as a computer program element and a computer readable medium. [Background technology]

[0002] During an X-ray examination of a body part, the body part to be examined may be beyond the range of the X-ray detector, and several separate X-ray images may be acquired (for example, by moving the C-arm of a mobile C-arm X-ray system). The images are then combined; this has been called the image stitching process, and the sequence of acquired images can be called a stitched sequence of X-ray images. Summary of the Invention [Problem to be solved by the invention]

[0003] When acquiring such a stitched sequence of X-ray images, the device is typically moved linearly along the axis of one X-ray detector from one acquisition position to the next. Prior to acquisition, the user selects this scan direction according to clinical need. If the movement is not motorized but simply manual, there is no information about the detector position corresponding to the acquired subimage. Therefore, there is no information about which of the four subimage boundaries of subsequent subimages in the stitched sequence corresponds to the subimage's predecessor.

[0004] Also, for such manual movements, the movements may not be exactly linear, and after a movement from one subsequent image, the next movement may deviate from the previous direction, further complicating how the images are determined to be stitched together.

[0005] These issues need to be addressed. [Means for solving the problem]

[0006] It would be advantageous to have an improved means for acquiring a sequence of stitched images, determining the scan direction of an X-ray system, and determining how these images are stitched together. The object of the present invention is solved by the subject matter of the independent claims, and further embodiments are incorporated in the dependent claims. It is noted that the following described aspects and examples of the present invention also apply to an apparatus for scan direction detection and stitch sequence determination of multiple X-ray images, a system for scan direction detection and stitch sequence determination of multiple X-ray images, and a method for scan direction detection and stitch sequence determination of multiple X-ray images, as well as a computer program element and a computer-readable medium.

[0007] In a first aspect, there is provided an apparatus for scan direction detection and stitching sequence determination of a plurality of X-ray images, comprising: an input unit; a processing unit; - Output unit and An apparatus is provided comprising:

[0008] The input unit is configured to provide a first X-ray image acquired by the X-ray image acquisition system to the processing unit, the first image having patient image data. The input unit is configured to provide a second X-ray image acquired by the X-ray image acquisition system after the X-ray image acquisition system has been moved relative to the patient to the processing unit, the second image having patient image data. The processing unit is configured to determine an upper similarity value comprising a comparison of at least one region of the patient image data at and / or adjacent to an upper boundary of the first image with at least one equally sized region of the patient image data at and / or adjacent to a lower boundary of the second image. The processing unit is configured to determine a right similarity value comprising a comparison of at least one region of the patient image data at and / or adjacent to a right boundary of the first image with at least one equally sized region of the patient image data at and / or adjacent to a left boundary of the second image. The processing unit is configured to determine a lower similarity value comprising a comparison of at least one region of the patient image data at and / or adjacent to a lower boundary of the first image with at least one equally sized region of the patient image data at and / or adjacent to an upper boundary of the second image. The processing unit is configured to determine a left similarity value comprising a comparison of at least one region of the patient image data at and / or adjacent to a left boundary of the first image with at least one equally sized region of the patient image data at and / or adjacent to a right boundary of the second image. The processing unit is configured to determine a scan direction and translation distance of the X-ray image acquisition system associated with a movement of the X-ray acquisition system comprising use of a maximum value of the upper, right, lower, or left similarity value, and / or determine a combined image formed from the first image and the second image comprising use of a maximum value of the upper, right, lower, or left similarity value. The output unit is configured to output the scan direction and translation distance and / or the combined image.

[0009] In other words, the X-ray acquisition system acquires a first image, then moves and acquires a second image. Still, it is unclear in which direction the X-ray acquisition system was moved and how far it actually moved. Therefore, to fit or match the two images together, the second image must be positioned above the first image, to the right of the first image, below the first image, or to the left of the first image. Furthermore, it is unclear whether the second image butts the first image—in other words, whether the image acquisition system moved exactly the viewing distance—and how much the two images overlap if the image acquisition system moved less than the viewing distance. The new device addresses this situation by actually overlaying the second image on the first image at the top border and determining the similarity. This is then done using the second image to the right of the first image, then the second image below the first image, and then the second image to the left of the first image. The maximum similarity values ​​of these different positions then provide the correct positioning of the second image relative to the first image, providing the required degree of overlay, while this information also provides a determination of the direction in which the image acquisition system was moved and how much it moved between the acquisition of the two images.

[0010] Note that the "top" similarity value refers to the similarity value calculated when the second image is positioned on top of the first image, and the "bottom" value refers to the similarity value calculated when the second image is positioned on the bottom of the first image.

[0011] In examples, determining the top similarity value includes determining a plurality of similarity values ​​comprising comparing associated regions of different sizes of the patient image data in the first image and the second image and selecting a maximum similarity value from the plurality of similarity values ​​as the top similarity value; determining the right similarity value includes determining a plurality of similarity values ​​comprising comparing associated regions of different sizes of the patient image data in the first image and the second image and selecting a maximum similarity value from the plurality of similarity values ​​as the right similarity value; determining the bottom similarity value includes determining a plurality of similarity values ​​comprising comparing associated regions of different sizes of the patient image data in the first image and the second image and selecting a maximum similarity value from the plurality of similarity values ​​as the bottom similarity value; and determining the left similarity value includes determining a plurality of similarity values ​​comprising comparing associated regions of different sizes of the patient image data in the first image and the second image and selecting a maximum similarity value from the plurality of similarity values ​​as the left similarity value.

[0012] Thus, for example, with respect to positioning a second image on top of a first image, a series of numerous different similarity values ​​are determined for different degrees of overlap of the second image relative to the first image. This can involve the second image nearly butting the first image with a slight overlap, and similarity values ​​are calculated for the second image that actually butts the first image but is slightly shifted horizontally; in other words, the edges of the two images are thus minimally aligned. Here, butting means that there is at least some vertical overlap, but only a minimal amount. Similarity values ​​can also be calculated when the second image actually overlaps an area of ​​the first image; in this case, a translation of the overlap in the translation direction can also be used. In each of these situations, a similarity value is calculated. The maximum value then provides the best overlap between the second image and the first image when the second image is on top of the first image. This process is then repeated for the second image to the right, below, and left of the first image. The maximum similarity value among each of the four maximum similarity values ​​for the above, right, below, and left situations then provides the correct positioning of the second image relative to the first image, for example, the second image should be below the first image with a certain degree of overlap.

[0013] In an example, the comparison of each region of patient image data at and / or adjacent to the upper boundary of the first image with a similarly sized region of patient image data at and / or adjacent to the lower boundary of the second image comprises the use of an upper normalized cross-correlation algorithm.

[0014] In examples, comparing each region of patient image data at and / or adjacent the right border of a first image with a comparable sized region of patient image data at and / or adjacent the left border of a second image comprises utilizing a right-normalized cross-correlation algorithm, comparing each region of patient image data at and / or adjacent the bottom border of a first image with a comparable sized region of patient image data at and / or adjacent the top border of a second image comprises utilizing an top-normalized cross-correlation algorithm or a bottom-normalized cross-correlation algorithm, and comparing each region of patient image data at and / or adjacent the left border of a first image with a comparable sized region of patient image data at and / or adjacent the right border of a second image comprises utilizing a right-normalized cross-correlation algorithm or a left-normalized cross-correlation algorithm.

[0015] It should be noted that the "top normalized cross-correlation algorithm" refers to the normalized cross-correlation algorithm utilized when the second image is positioned on top of the first image, and the "bottom normalized cross-correlation algorithm" refers to the normalized cross-correlation algorithm utilized when the second image is positioned below the first image.

[0016] In examples, comparing each region of patient image data at and / or adjacent the right border of a first image with a comparable sized region of patient image data at and / or adjacent the left border of a second image comprises utilizing an upper normalized cross-correlation algorithm, comparing each region of patient image data at and / or adjacent the bottom border of a first image with a comparable sized region of patient image data at and / or adjacent the top border of a second image comprises utilizing an upper normalized cross-correlation algorithm, and comparing each region of patient image data at and / or adjacent the left border of a first image with a comparable sized region of patient image data at and / or adjacent the right border of a second image comprises utilizing an upper normalized cross-correlation algorithm.

[0017] In examples, comparing each region of patient image data at and / or adjacent to the right border of a first image with a comparable sized region of patient image data at and / or adjacent to the left border of a second image comprises rotating the first image 90 degrees counterclockwise and rotating the second image 90 degrees counterclockwise before using the top-normalized cross-correlation algorithm. Comparing each region of patient image data at and / or adjacent to the bottom border of a first image with a comparable sized region of patient image data at and / or adjacent to the top border of a second image comprises rotating the first image 180 degrees and rotating the second image 180 degrees before using the top-normalized cross-correlation algorithm. Comparing each region of patient image data at and / or adjacent to the left border of a first image with a comparable sized region of patient image data at and / or adjacent to the right border of a second image comprises rotating the first image 90 degrees clockwise and rotating the second image 90 degrees clockwise before using the top-normalized cross-correlation algorithm.

[0018] In other words, with the free effect of overlaying a second image on a first image with varying degrees of overlap, different algorithms can be utilized for top, right, bottom, and left situations when comparing the second image with the first image, so that, in effect, for the top situation, the second image can be moved down and slightly to the right and left, along with a downward movement if necessary. Then, for example, for the left situation, the second image can be moved right and slightly up and down, along with a right movement if necessary. Nevertheless, it has been found to be computationally efficient to rotate both images so that the same algorithm, e.g., the top algorithm, can be utilized in all situations.

[0019] In a second aspect, there is provided an apparatus for scan direction detection and stitching sequence determination of a plurality of X-ray images, comprising: an input unit; a processing unit; - Output unit and An apparatus is provided comprising:

[0020] The input unit is configured to provide a plurality of "N" X-ray images acquired by the X-ray image acquisition system to the processing unit. After each of the first N-1 images, the X-ray image acquisition is moved relative to the patient, and each of the N X-ray images has image data of the patient. One X-ray image of the N X-ray images is selected. This can be done by the processing unit or manually. The processing unit is configured to perform determining an upper similarity value, which includes determining a plurality of similarity values, which includes comparing at least one region of the patient image data at and / or adjacent to an upper boundary of the selected image with at least one equally sized region of the patient image data at and / or adjacent to a lower boundary of each of the other N-1 images, and the processing unit is configured to select a maximum similarity value from the plurality of similarity values ​​as the upper similarity value. The processing unit is configured to perform determining a right similarity value comprising determining a plurality of similarity values ​​comprising comparing at least one region of the patient image data at and / or adjacent a right boundary of the selected image with at least one similarly sized region of the patient image data at and / or adjacent a left boundary of each of the other N-1 images, the processing unit being configured to select a maximum similarity value from the plurality of similarity values ​​as the right similarity value. The processing unit is configured to perform determining a bottom similarity value comprising determining a plurality of similarity values ​​comprising comparing at least one region of the patient image data at and / or adjacent a bottom boundary of the selected image with at least one similarly sized region of the patient image data at and / or adjacent a top boundary of each of the other N-1 images, the processing unit being configured to select a maximum similarity value from the plurality of similarity values ​​as the bottom similarity value.The processing unit is configured to determine a left similarity value, the determination comprising determining a plurality of similarity values ​​comprising comparing at least one region of the patient image data at a left boundary of the selected image and / or adjacent to the selected image with at least one similarly sized region of the patient image data at a right boundary of each of the other N-1 images and / or adjacent to the selected image, and the processing unit is configured to select a maximum similarity value among the plurality of similarity values ​​as the left similarity value. The processing unit is configured to determine a scan direction and translation distance of the X-ray image acquisition system associated with a movement of the X-ray acquisition system using a maximum value of the top, right, bottom, or left similarity value, and / or determine a combined image formed from the selected image and the second image using a maximum value of the top, right, bottom, or left similarity value. The output unit is configured to output the scan direction and translation distance and / or the combined image.

[0021] In other words, the X-ray acquisition system has acquired several images between each image acquisition in which the X-ray acquisition system moved. Yet, it is unknown in what order the images were acquired and in which direction or directions the X-ray acquisition system actually moved. The new device addresses this situation. An arbitrary one of the images is first selected, which can be done by a processing unit or a human, and can be completely random. Then, all other images are individually matched to the selected image, from top to right or bottom to left, to determine their best positioning relative to the first image. Then, among all the different image situations that matched the first image, the highest similarity value for a particular situation not only provides the correct second image to match the first image, but also how this second image matches the first image, and this information also provides the distance and direction the X-ray imaging system moved. The process can then be performed again for the first image; in this case, for example, the first image may not have been acquired, and there may be an image that should have been combined on the other side. Alternatively, the process can be repeated for the image that was simply combined with the first image to determine the next image to be combined with this further image, as well as the positioning and movement of the X-ray image acquisition system, which does not have to be in the same direction, for example, as dogleg movements of the image acquisition system may occur. Nevertheless, the new device can stitch all of the images together regardless of how the X-ray image acquisition system moved, as long as the movement is not greater than the field of view of the image acquisition system and the movement is not reversed.

[0022] Note that time information is not required to stitch images together as described above. Still, the scan direction can be determined in the manner described above, but it is not possible to determine whether this was, for example, from top to bottom or from bottom to top. Thus, here, "scan direction" means, for example, from top to bottom or from bottom to top. Still, using the image acquisition times, the images can be stitched together in the manner described above, and the absolute scan direction can also be determined, and thus it can be determined that the scan direction was from bottom to top.

[0023] In an example, determining the top similarity value for each pair of the selected image and one of the other N-1 images includes determining a plurality of similarity values ​​comprising comparing associated regions of different sizes of the patient image data in the selected image and one of the other N-1 images, and selecting a maximum similarity value from the plurality of similarity values ​​as the top similarity value. Determining the right similarity value for each pair of the selected image and one of the other N-1 images includes determining a plurality of similarity values ​​comprising comparing associated regions of different sizes of the patient image data in the selected image and one of the other N-1 images, and selecting a maximum similarity value from the plurality of similarity values ​​as the right similarity value. Determining the bottom similarity value for each pair of the selected image and one of the other N-1 images includes determining a plurality of similarity values ​​with a comparison of associated regions of different sizes of the patient image data in the selected image and one of the other N-1 images, and selecting a maximum similarity value from the plurality of similarity values ​​as the bottom similarity value. Determining the left similarity value for each pair of the selected image and one of the other N-1 images includes determining a plurality of similarity values ​​with a comparison of associated regions of different sizes of the patient image data in the selected image and one of the other N-1 images, and selecting a maximum similarity value from the plurality of similarity values ​​as the left similarity value.

[0024] In an example, the comparison of each region of patient image data at and / or adjacent to the upper boundary of the selected image with an equivalent sized region of patient image data at and / or adjacent to the lower boundary of each of the other N-1 images comprises the use of an upper normalized cross-correlation algorithm.

[0025] In examples, comparing each region of patient image data at and / or adjacent the right border of the selected image with a similarly sized region of patient image data at and / or adjacent the left border of each of the other N-1 images comprises utilizing a right-normalized cross-correlation algorithm, comparing each region of patient image data at and / or adjacent the bottom border of the selected image with a similarly sized region of patient image data at and / or adjacent the top border of each of the other N-1 images comprises utilizing a bottom-normalized cross-correlation algorithm, and comparing each region of patient image data at and / or adjacent the left border of the selected image with a similarly sized region of patient image data at and / or adjacent the right border of each of the other N-1 images comprises utilizing a left-normalized cross-correlation algorithm.

[0026] In examples, comparing each region of patient image data at and / or adjacent the right border of the selected image with a similarly sized region of patient image data at and / or adjacent the left border of each of the other N-1 images comprises utilizing an upper-normalized cross-correlation algorithm. Comparing each region of patient image data at and / or adjacent the bottom border of the selected image with a similarly sized region of patient image data at and / or adjacent the top border of each of the other N-1 images comprises utilizing an upper-normalized cross-correlation algorithm. Comparing each region of patient image data at and / or adjacent the left border of the selected image with a similarly sized region of patient image data at and / or adjacent the right border of each of the other N-1 images comprises utilizing an upper-normalized cross-correlation algorithm.

[0027] In an example, comparing each region of patient image data at the right boundary of the selected image and / or adjacent to a region of comparable size at the left boundary of each of the other N-1 images and / or adjacent to the patient image data comprises a 90-degree counterclockwise rotation of the selected image and a 90-degree counterclockwise rotation of each of the other N-1 images before using the top-normalized cross-correlation algorithm. Comparing each region of patient image data at the bottom boundary of the selected image and / or adjacent to a region of comparable size at the top boundary of each of the other N-1 images and / or adjacent to the patient image data comprises a 180-degree rotation of the selected image and a 180-degree rotation of each of the other N-1 images before using the top-normalized cross-correlation algorithm. Comparing each region of patient image data at the left boundary of the selected image and / or adjacent to a region of comparable size at the right boundary of each of the other N-1 images and / or adjacent to the patient image data comprises a 90-degree clockwise rotation of the selected image and a 90-degree clockwise rotation of each of the other N-1 images before using the top-normalized cross-correlation algorithm.

[0028] In a third aspect, there is provided a system for scan direction detection and stitching sequence determination for multiple X-ray images, comprising: - an image acquisition system, - a device according to the first aspect, and / or - a device according to the second aspect A system is provided, comprising:

[0029] In a fourth aspect, there is provided a method for scan direction detection and stitching sequence determination of multiple X-ray images, comprising: a) providing a first X-ray image acquired by an X-ray image acquisition system to a processing unit, the first image comprising image data of a patient; b) providing a second X-ray image acquired by the X-ray image acquisition system to the processing unit after the X-ray image acquisition system has been moved relative to the patient, the second image comprising image data of the patient; c) determining an upper similarity value by the processing unit, comprising comparing at least one region of the patient image data at and / or adjacent to the upper boundary of the first image with at least one similarly sized region of the patient image data at and / or adjacent to the lower boundary of the second image; d) determining by the processing unit a right similarity value comprising comparing at least one region of the patient image data at and / or adjacent to the right border of the first image with at least one similarly sized region of the patient image data at and / or adjacent to the left border of the second image; e) determining by the processing unit a lower similarity value comprising comparing at least one region of the patient image data at and / or adjacent to the lower boundary of the first image with at least one similarly sized region of the patient image data at and / or adjacent to the upper boundary of the second image; f) determining by the processing unit a left similarity value comprising comparing at least one region of the patient image data at and / or adjacent to the left border of the first image with at least one similarly sized region of the patient image data at and / or adjacent to the right border of the second image; g) determining by a processing unit a scan direction and a translation distance of the X-ray image acquisition system associated with a movement of the X-ray acquisition system, comprising utilizing a maximum value of the top, right, bottom, or left similarity value, and / or determining a combined image formed from the first image and the second image, comprising utilizing a maximum value of the top, right, bottom, or left similarity value; h) outputting the scanning direction and translation distance and / or the combined image by an output unit; A method is provided, comprising:

[0030] In a fifth aspect, there is provided a method for scan direction detection and stitching sequence determination of multiple X-ray images, comprising: a1) providing a plurality of "N" X-ray images acquired by an X-ray image acquisition system to a processing unit, where after each of the first N-1 images, the X-ray image acquisition is moved relative to the patient, and each of the N X-ray images has image data of the patient; b1) selecting one X-ray image from the N X-ray images; c1) determining an upper similarity value by a processing unit, the upper similarity value comprising comparing at least one region of the patient image data at and / or adjacent to an upper boundary of the selected image with at least one similarly sized region of the patient image data at and / or adjacent to a lower boundary of each of the other N-1 images, and selecting by the processing unit a maximum similarity value of the plurality of similarity values ​​as the upper similarity value; d1) determining a plurality of similarity values ​​by a processing unit, the similarity values ​​comprising comparing at least one region of the patient image data at and / or adjacent to a right boundary of the selected image with at least one similarly sized region of the patient image data at and / or adjacent to a left boundary of each of the other N-1 images, and selecting by the processing unit a maximum similarity value from the plurality of similarity values ​​as the right similarity value; e1) determining a plurality of similarity values ​​by a processing unit, the similarity values ​​comprising comparing at least one region of the patient image data at and / or adjacent to a lower boundary of the selected image with at least one similarly sized region of the patient image data at and / or adjacent to an upper boundary of each of the other N-1 images, and selecting by the processing unit a maximum similarity value from the plurality of similarity values ​​as the lower similarity value; f1) determining a plurality of similarity values ​​by a processing unit, the plurality of similarity values ​​comprising comparing at least one region of the patient image data at and / or adjacent to the left boundary of the selected image with at least one similarly sized region of the patient image data at and / or adjacent to the right boundary of each of the other N-1 images, and selecting by the processing unit the largest similarity value of the plurality of similarity values ​​as the left similarity value; g1) determining by a processing unit a scan direction and a translation distance of the X-ray image acquisition system associated with a movement of the X-ray acquisition system, comprising using a maximum value of the top, right, bottom, or left similarity value, and / or determining a combined image formed from the selected image and the second image, comprising using a maximum value of the top, right, bottom, or left similarity value; h1) outputting the scanning direction and translation distance and / or the combined image by the output unit; A method is provided, comprising:

[0031] According to another aspect, there is provided a computer program element for controlling one or more of the apparatuses or systems as described above, adapted to perform one or more of the methods as described above when the computer program element is executed by a processing unit.

[0032] According to another aspect, there is provided a computer readable medium having stored thereon a computer element as described above.

[0033] The computer program element may for example be a software program, but may also be an FPGA, a PLD, or any other suitable digital means.

[0034] Advantageously, benefits provided by any of the above aspects apply equally to all of the other aspects, and vice versa.

[0035] The above aspects and examples will be apparent from and elucidated with reference to the embodiments described hereinafter.

[0036] Exemplary embodiments are described below with reference to the following drawings: [Brief explanation of the drawings]

[0037] [Figure 1] FIG. 1 is a schematic set-up diagram of an example of an apparatus for scan direction detection and stitching sequence determination of two X-ray images. [Figure 2] FIG. 1 is a schematic set-up diagram of an example of an apparatus for scan direction detection and stitching sequence determination of multiple X-ray images. [Figure 3] FIG. 1 is a schematic set-up diagram of an example system for scan direction detection and stitching sequence determination of multiple X-ray images. [Figure 4] Diagram of the method for scan direction detection and stitching sequence determination of two X-ray images. [Figure 5] Diagram of the method for scan direction detection and stitching sequence determination for multiple X-ray images. [Figure 6] Figure 1 shows an example of hip stitching, showing five partial images acquired with a left-to-right scanning direction and the composite image following the right partial border stitching. [Figure 7] FIG. 10 is an example of spine stitching showing seven partial images acquired in a bottom-to-top scan direction and a composite image followed by top boundary image stitching. [Figure 8] FIG. 10 is a diagram of how scan direction and image position and overlay are automatically determined, showing the first image positioned at the center and the second image positioned above, right, below, and left, and calculating the similarity value features of these positions for the second image. [Figure 9] Figure 8 shows the conclusion of the process shown in Figure 8, where the second image is combined with the first image in the upper boundary stitching process. [Figure 10]FIG. 10 is a diagram of a four step procedure for determining scan direction and image position; in this case, rather than overlaying images at different positions around the previous image, both images are rotated through different 90° steps to determine scan direction and image stitching requirements; then a bottom stitching or overlay process is applied to each image rotation, allowing the same algorithm to be utilized. [Figure 11] This is a sequence diagram of the process of scan direction determination and image stitching sequence determination. DETAILED DESCRIPTION OF THE INVENTION

[0038] FIG. 1 shows a schematic example of an apparatus 10 for scan direction detection and stitching sequence determination of two X-ray images. The apparatus includes an input unit 20, a processing unit 30, and an output unit 40. The input unit is configured to provide the processing unit with a first X-ray image acquired by an X-ray image acquisition system, the first image comprising patient image data. The input unit is configured to provide the processing unit with a second X-ray image acquired by the X-ray image acquisition system after the X-ray image acquisition system has been moved relative to the patient, the second image comprising patient image data. The processing unit is configured to determine an upper similarity value comprising a comparison of at least one region of the patient image data at and / or adjacent to an upper boundary of the first image with at least one similarly sized region of the patient image data at and / or adjacent to a lower boundary of the second image. The processing unit is configured to determine a right similarity value comprising a comparison of at least one region of the patient image data at a right border of and / or adjacent the first image with at least one similarly sized region of the patient image data at a left border of and / or adjacent the second image. The processing unit is configured to determine a bottom similarity value comprising a comparison of at least one region of the patient image data at a bottom border of the first image with at least one similarly sized region of the patient image data at an upper border of and / or adjacent the second image. The processing unit is configured to determine a left similarity value comprising a comparison of at least one region of the patient image data at a left border of and / or adjacent the first image with at least one similarly sized region of the patient image data at a right border of and / or adjacent the second image. The processing unit is configured to determine a scan direction and translation distance of the X-ray image acquisition system associated with a movement of the X-ray acquisition system, with utilization of a maximum value of the top, right, bottom, or left similarity values, and / or to determine a combined image formed from the first image and the second image, with utilization of a maximum value of the top, right, bottom, or left similarity values.The output unit is configured to output the scan direction and translation distance and / or the combined image.

[0039] According to an example, determining the top similarity value comprises determining a plurality of similarity values ​​comprising comparing associated regions of different sizes of the patient image data in the first image and the second image and selecting a maximum similarity value from the plurality of similarity values ​​as the top similarity value. Determining the right similarity value comprises determining a plurality of similarity values ​​comprising comparing associated regions of different sizes of the patient image data in the first image and the second image and selecting a maximum similarity value from the plurality of similarity values ​​as the right similarity value. Determining the bottom similarity value comprises determining a plurality of similarity values ​​comprising comparing associated regions of different sizes of the patient image data in the first image and the second image and selecting a maximum similarity value from the plurality of similarity values ​​as the bottom similarity value. Determining the left similarity value comprises determining a plurality of similarity values ​​comprising comparing associated regions of different sizes of the patient image data in the first image and the second image and selecting a maximum similarity value from the plurality of similarity values ​​as the left similarity value.

[0040] According to an example, the comparison of each region of patient image data at and / or adjacent to the upper boundary of the first image with a similarly sized region of patient image data at and / or adjacent to the lower boundary of the second image comprises the use of an upper normalized cross-correlation algorithm.

[0041] According to an example, comparing each region of patient image data at and / or adjacent the right border of a first image with a comparable sized region of patient image data at and / or adjacent the left border of a second image comprises utilizing a right-normalized cross-correlation algorithm; comparing each region of patient image data at and / or adjacent the bottom border of a first image with a comparable sized region of patient image data at and / or adjacent the top border of a second image comprises utilizing an top-normalized cross-correlation algorithm or a bottom-normalized cross-correlation algorithm; and comparing each region of patient image data at and / or adjacent the left border of a first image with a comparable sized region of patient image data at and / or adjacent the right border of a second image comprises utilizing a right-normalized cross-correlation algorithm or a left-normalized cross-correlation algorithm.

[0042] According to an example, comparing each region of patient image data at and / or adjacent the right border of a first image with a comparable sized region of patient image data at and / or adjacent the left border of a second image comprises utilizing an upper normalized cross-correlation algorithm. Comparing each region of patient image data at and / or adjacent the bottom border of a first image with a comparable sized region of patient image data at and / or adjacent the top border of a second image comprises utilizing an upper normalized cross-correlation algorithm. Comparing each region of patient image data at and / or adjacent the left border of a first image with a comparable sized region of patient image data at and / or adjacent the right border of a second image comprises utilizing an upper normalized cross-correlation algorithm.

[0043] According to an example, comparing each region of patient image data at the right border and / or adjacent of a first image with a comparable sized region of patient image data at the left border and / or adjacent of a second image comprises rotating the first image 90 degrees counterclockwise and rotating the second image 90 degrees counterclockwise before using the top-normalized cross-correlation algorithm. Comparing each region of patient image data at the bottom border and / or adjacent of a first image with a comparable sized region of patient image data at the top border and / or adjacent of a second image comprises rotating the first image 180 degrees and rotating the second image 180 degrees before using the top-normalized cross-correlation algorithm. Comparing each region of patient image data at the left border and / or adjacent of a first image with a comparable sized region of patient image data at the right border and / or adjacent of a second image comprises rotating the first image 90 degrees clockwise and rotating the second image 90 degrees clockwise before using the top-normalized cross-correlation algorithm.

[0044] FIG. 2 shows a schematic example of an apparatus 100 for scan direction detection and stitch sequence determination of multiple X-ray images. The apparatus includes an input unit 110, a processing unit 120, and an output unit 130. The input unit is configured to provide a plurality of “N” X-ray images acquired by an X-ray image acquisition system to the processing unit, where after each of the first N−1 images, the X-ray image acquisition is moved relative to the patient, and each of the N X-ray images includes patient image data. One of the N X-ray images is selected. The processing unit is configured to perform a determination of an upper similarity value, which includes a comparison of at least one region of the patient image data at an upper boundary of the selected image and / or adjacent to the selected image with at least one similarly sized region of the patient image data at a lower boundary of each of the other N−1 images and / or adjacent to the selected image, and the processing unit is configured to select a maximum similarity value from the plurality of similarity values ​​as the upper similarity value. The processing unit is configured to perform determining a right similarity value comprising determining a plurality of similarity values ​​comprising comparing at least one region of the patient image data at a right boundary of the selected image and / or adjacent with at least one similarly sized region of the patient image data at a left boundary of each of the other N-1 images, the processing unit being configured to select a maximum similarity value from the plurality of similarity values ​​as the right similarity value. The processing unit is configured to perform determining a bottom similarity value comprising determining a plurality of similarity values ​​comprising comparing at least one region of the patient image data at a bottom boundary of the selected image and / or adjacent with at least one similarly sized region of the patient image data at an upper boundary of each of the other N-1 images, the processing unit being configured to select a maximum similarity value from the plurality of similarity values ​​as the bottom similarity value.The processing unit is configured to determine a left similarity value, the determination comprising determining a plurality of similarity values ​​comprising comparing at least one region of the patient image data at a left boundary of the selected image and / or adjacent to the selected image with at least one similarly sized region of the patient image data at a right boundary of each of the other N-1 images and / or adjacent to the selected image, and the processing unit is configured to select a maximum similarity value among the plurality of similarity values ​​as the left similarity value. The processing unit is configured to determine a scan direction and translation distance of the X-ray image acquisition system associated with a movement of the X-ray acquisition system using a maximum value of the top, right, bottom, or left similarity value, and / or determine a combined image formed from the selected image and the second image using a maximum value of the top, right, bottom, or left similarity value. The output unit is configured to output the scan direction and translation distance and / or the combined image.

[0045] According to an example, determining the top similarity value for each pair of the selected image and one of the other N-1 images comprises determining a plurality of similarity values ​​comprising comparing associated regions of different sizes of the patient image data in the selected image and one of the other N-1 images, and selecting the largest similarity value from the plurality of similarity values ​​as the top similarity value. Determining the right similarity value for each pair of the selected image and one of the other N-1 images comprises determining a plurality of similarity values ​​comprising comparing associated regions of different sizes of the patient image data in the selected image and one of the other N-1 images, and selecting the largest similarity value from the plurality of similarity values ​​as the top similarity value. Determining the bottom similarity value for each pair of the selected image and one of the other N-1 images includes determining a plurality of similarity values ​​with a comparison of associated regions of different sizes of the patient image data in the selected image and one of the other N-1 images, and selecting a maximum similarity value from the plurality of similarity values ​​as the bottom similarity value. Determining the left similarity value for each pair of the selected image and one of the other N-1 images includes determining a plurality of similarity values ​​with a comparison of associated regions of different sizes of the patient image data in the selected image and one of the other N-1 images, and selecting a maximum similarity value from the plurality of similarity values ​​as the left similarity value.

[0046] By way of example, the comparison of each region of patient image data at and / or adjacent to the upper boundary of the selected image with an equivalent sized region of patient image data at and / or adjacent to the lower boundary of each of the other N-1 images comprises the use of an upper normalized cross-correlation algorithm.

[0047] By way of example, comparing each region of patient image data at and / or adjacent the right border of the selected image with a similarly sized region of patient image data at and / or adjacent the left border of each of the other N-1 images comprises utilizing a right-normalized cross-correlation algorithm, comparing each region of patient image data at and / or adjacent the bottom border of the selected image with a similarly sized region of patient image data at and / or adjacent the top border of each of the other N-1 images comprises utilizing a bottom-normalized cross-correlation algorithm, and comparing each region of patient image data at and / or adjacent the left border of the selected image with a similarly sized region of patient image data at and / or adjacent the right border of each of the other N-1 images comprises utilizing a left-normalized cross-correlation algorithm.

[0048] By way of example, comparing each region of patient image data at and / or adjacent the right border of the selected image with a similarly sized region of patient image data at and / or adjacent the left border of each of the other N-1 images comprises utilizing an upper-normalized cross-correlation algorithm. Comparing each region of patient image data at and / or adjacent the bottom border of the selected image with a similarly sized region of patient image data at and / or adjacent the top border of each of the other N-1 images comprises utilizing an upper-normalized cross-correlation algorithm. Comparing each region of patient image data at and / or adjacent the left border of the selected image with a similarly sized region of patient image data at and / or adjacent the right border of each of the other N-1 images comprises utilizing an upper-normalized cross-correlation algorithm.

[0049] By way of example, comparing each region of patient image data at the right boundary of the selected image and / or adjacent to a region of comparable size at the left boundary of each of the other N-1 images and / or adjacent to the patient image data comprises a 90-degree counterclockwise rotation of the selected image and a 90-degree counterclockwise rotation of each of the other N-1 images before using the top-normalized cross-correlation algorithm. Comparing each region of patient image data at the bottom boundary of the selected image and / or adjacent to a region of comparable size at the top boundary of each of the other N-1 images and / or adjacent to the patient image data comprises a 180-degree rotation of the selected image and a 180-degree rotation of each of the other N-1 images before using the top-normalized cross-correlation algorithm. Comparing each region of patient image data at the left boundary of the selected image and / or adjacent to a region of comparable size at the right boundary of each of the other N-1 images and / or adjacent to the patient image data comprises a 90-degree clockwise rotation of the selected image and a 90-degree clockwise rotation of each of the other N-1 images before using the top-normalized cross-correlation algorithm.

[0050] 3 shows a schematic example of a system 200 for scan direction detection and stitching sequence determination of multiple X-ray images, comprising an image acquisition system 150, an apparatus 10 as described with respect to FIG. 1, and / or an apparatus 100 as described with respect to FIG. 2.

[0051] FIG. 4 shows a method 200 for scan direction detection and stitching sequence determination of two X-ray images. In a providing step 210, also referred to as step a), providing a first X-ray image acquired by an X-ray image acquisition system to a processing unit, the first image comprising image data of a patient; In a providing step 220, also referred to as step b), providing a second X-ray image acquired by the X-ray image acquisition system to the processing unit after the X-ray image acquisition system has been moved relative to the patient, the second image comprising image data of the patient; determining, in a determining step 230, also referred to as step c), by the processing unit, an upper similarity value, which comprises comparing at least one region of the patient image data at and / or adjacent to the upper boundary of the first image with at least one similarly sized region of the patient image data at and / or adjacent to the lower boundary of the second image; determining by the processing unit in a determining step 240, also referred to as step d), a right similarity value comprising comparing at least one region of the patient image data at and / or adjacent to the right border of the first image with at least one similarly sized region of the patient image data at and / or adjacent to the left border of the second image; determining by the processing unit a lower similarity value in a determining step 250, also referred to as step e), which comprises comparing at least one region of the patient image data at the lower boundary of the first image and / or adjacent thereto with at least one similarly sized region of the patient image data at the upper boundary of the second image and / or adjacent thereto; determining, in a determining step 260, also referred to as step f), a left similarity value by the processing unit, comprising comparing at least one region of the patient image data at and / or adjacent to the left border of the first image with at least one similarly sized region of the patient image data at and / or adjacent to the right border of the second image; determining, in a determining step 270, also referred to as step g), by the processing unit, a scan direction and a translation distance of the X-ray image acquisition system associated with the movement of the X-ray acquisition system, comprising utilizing a maximum value of the top, right, bottom, or left similarity value, and / or determining a combined image formed from the first image and the second image, comprising utilizing a maximum value of the top, right, bottom, or left similarity value; In a decision step 280, also referred to as step h), outputting the scan direction and translation distance and / or the combined image by an output unit. It has.

[0052] In an example, step c) comprises determining a plurality of similarity values ​​comprising comparing associated regions of different sizes of the patient's image data in the first image and the second image, and selecting a maximum similarity value from the plurality of similarity values ​​as the upper similarity value.

[0053] In an example, step d) includes determining a plurality of similarity values, which includes comparing associated regions of different sizes of the patient's image data in the first image and the second image, and selecting the largest similarity value among the plurality of similarity values ​​as the right similarity value.

[0054] In an example, step e) includes determining a plurality of similarity values, which includes comparing associated regions of different sizes of the patient's image data in the first image and the second image, and selecting a maximum similarity value from the plurality of similarity values ​​as the lower similarity value.

[0055] In an example, step f) includes determining a plurality of similarity values, which includes comparing associated regions of different sizes of the patient's image data in the first image and the second image, and selecting a maximum similarity value from the plurality of similarity values ​​as a left similarity value.

[0056] In an example, step c) comprises utilizing an upper normalized cross-correlation algorithm.

[0057] In an example, step d) comprises utilizing a right-normalized cross-correlation algorithm.

[0058] In an example, step e) comprises utilizing a lower normalized cross-correlation algorithm.

[0059] In an example, step f) comprises using a left-normalized cross-correlation algorithm.

[0060] In an example, step d) comprises utilizing an upper normalized cross-correlation algorithm.

[0061] In an example, step e) comprises utilizing an upper normalized cross-correlation algorithm.

[0062] In an example, step f) comprises utilizing an upper normalized cross-correlation algorithm.

[0063] In an example, step d) comprises rotating the first image 90 degrees counterclockwise and rotating the second image 90 degrees counterclockwise before utilizing the upper normalized cross-correlation algorithm.

[0064] In an example, step e) comprises rotating the first image by 180 degrees and rotating the second image by 180 degrees before applying the upper normalized cross-correlation algorithm.

[0065] In an example, step f) comprises rotating the first image 90 degrees clockwise and rotating the second image 90 degrees clockwise before utilizing the upper normalized cross-correlation algorithm.

[0066] FIG. 5 illustrates a method 300 for scan direction detection and stitching sequence determination for multiple X-ray images. providing, in a providing step 310, also referred to as step a1), a plurality of "N" X-ray images acquired by an X-ray image acquisition system to a processing unit, where after each of the first N-1 images, the X-ray image acquisition is moved relative to the patient, and each of the N X-ray images has image data of the patient; In a selection step 320, also referred to as step b1), selecting one X-ray image out of the N X-ray images; determining, in a determining step 330, also referred to as step c1), a plurality of similarity values ​​comprising comparing at least one region of the patient image data at the upper boundary of the selected image and / or adjacent to the selected image with at least one similarly sized region of the patient image data at the lower boundary of each of the other N-1 images and selecting, by the processing unit, the largest similarity value of the plurality of similarity values ​​as the upper similarity value; determining a plurality of similarity values ​​by the processing unit in a determining step 340, also referred to as step d1), which comprises comparing at least one region of the patient image data at and / or adjacent to the right border of the selected image with at least one similarly sized region of the patient image data at and / or adjacent to the left border of each of the other N-1 images; and selecting by the processing unit the largest similarity value of the plurality of similarity values ​​as the right similarity value; determining a plurality of similarity values ​​by a processing unit in a determining step 350, also referred to as step e1), comprising comparing at least one region of the patient image data at the lower boundary of the selected image and / or adjacent to the selected image with at least one similarly sized region of the patient image data at the upper boundary of each of the other N-1 images and selecting by the processing unit the largest similarity value of the plurality of similarity values ​​as the lower similarity value; determining a plurality of similarity values ​​by the processing unit in a determining step 360, also referred to as step f1), comprising comparing at least one region of the patient image data at and / or adjacent to the left border of the selected image with at least one similarly sized region of the patient image data at and / or adjacent to the right border of each of the other N-1 images; and selecting by the processing unit the largest similarity value of the plurality of similarity values ​​as the left similarity value; determining, in a determining step 370, also referred to as step g1), by the processing unit, a scan direction and a translation distance of the X-ray image acquisition system associated with the movement of the X-ray acquisition system, comprising using a maximum value of the top, right, bottom, or left similarity value, and / or determining a combined image formed from the selected image and the second image, comprising using a maximum value of the top, right, bottom, or left similarity value; In an output step 380, also referred to as step h1), outputting the scan direction and translation distance and / or the combined image by an output unit. It has.

[0067] In an example, in step c1), determining an upper similarity value for each pair of the selected image and one of the other N-1 images includes determining a plurality of similarity values ​​by comparing associated regions of different sizes of the patient's image data in the selected image and one of the other N-1 images, and selecting the largest similarity value from the plurality of similarity values ​​as the upper similarity value.

[0068] In an example, in step d1), determining a right similarity value for each pair of the selected image and one of the other N-1 images includes determining a plurality of similarity values ​​by comparing associated regions of different sizes of the patient's image data in the selected image and one of the other N-1 images, and selecting the largest similarity value from the plurality of similarity values ​​as the right similarity value.

[0069] In an example, in step e1), determining a lower similarity value for each pair of the selected image and one of the other N-1 images comprises determining a plurality of similarity values ​​by comparing associated regions of different sizes of the patient's image data in the selected image and one of the other N-1 images, and selecting the largest similarity value from the plurality of similarity values ​​as the lower similarity value.

[0070] In an example, in step f1), determining a left similarity value for each pair of the selected image and one of the other N-1 images includes determining a plurality of similarity values ​​by comparing associated regions of different sizes of the patient's image data in the selected image and one of the other N-1 images, and selecting the largest similarity value from the plurality of similarity values ​​as the left similarity value.

[0071] In an example, step c1) comprises using an upper normalized cross-correlation algorithm.

[0072] In the example, step d1) comprises applying a right-normalized cross-correlation algorithm.

[0073] In the example, step e1) comprises applying a lower normalized cross-correlation algorithm.

[0074] In the example, step f1) comprises applying a left-normalized cross-correlation algorithm.

[0075] In an example, step d1) comprises using an upper normalized cross-correlation algorithm.

[0076] In the example, step e1) comprises using an upper normalized cross-correlation algorithm.

[0077] In the example, step f1) comprises using an upper normalized cross-correlation algorithm.

[0078] In the example, step d1) comprises rotating the selected image 90 degrees counterclockwise and rotating each of the other N-1 images 90 degrees counterclockwise prior to application of the upper normalized cross-correlation algorithm.

[0079] In the example, step e1) comprises rotating the selected image by 180 degrees and rotating each of the other N-1 images by 180 degrees before applying the upper normalized cross-correlation algorithm.

[0080] In the example, step f1) comprises rotating the selected image 90 degrees clockwise and rotating each of the other N-1 images 90 degrees clockwise prior to application of the upper normalized cross-correlation algorithm.

[0081] Therefore, a new technique is provided for automatic detection of one of the four sub-image boundaries where sub-images are stitched together, which is equivalent to detecting one of the following four scan directions:

number

number

number

number

[0082] The apparatus, system, and method for scan direction detection and stitching sequence determination of multiple X-ray images will now be described in further specific detail, and reference is made to FIGS. 6-11.

[0083] As mentioned above, for some mobile C-arm X-ray systems, the C-arm is moved manually and there is no information about the detector position corresponding to the acquired sub-images. As a result, the system has no information about which of the four image boundaries the sub-images should be stitched together, and the user must specify the scan direction. This means that the user must either have knowledge of the system geometry and acquisition context, or the user must visually detect the scan direction from the image content of the sub-images, which is inconvenient and error-prone.

[0084] Nevertheless, the devices, systems, and methods described herein address this. Figures 6 and 7 show the situation for two different scan directions and anatomical structures, where image sequences are stitched together. This is achieved by automatically detecting image boundaries where partial images are stitched together, without any external input provided to the stitching algorithm. This is therefore equivalent to automatic detection of scan direction.

[0085] To automatically select the correct stitching boundary, the following steps are performed: Without loss of generality, it is assumed that two temporally later images outside the stitching sequence are chosen; otherwise, the resulting maximum similarity will be smaller than the similarity of the two later images considered here.

[0086] We match the subimage to all four boundaries of the original image and measure the similarity as the normalized cross-correlation (NCC, see below).

[0087] b 1,2 Let E{left, right, top, bottom} be the boundaries corresponding to the first and second highest similarities, respectively, where b 1 is the detected image boundary for stitching, and the importance of this image boundary is S=(NCC b1 -NCC b2 ) / NCC b1The importance S is a number between 0 (no importance) and 1 (complete importance).

[0088] Implement stitching for the selected boundaries with corresponding translation.

[0089] Image matching is defined so that determining the displacement of a subimage relative to the original image leads to the maximum similarity between these subimages. To obtain maximum similarity, the images can be displaced in two orthogonal axes, with a large displacement along one axis and a small displacement along the orthogonal axis. This generally takes into account the movement of the X-ray system along its axis, but in this case, the system was also moved slightly sideways. Thus, the combined image then shifts the edge boundaries slightly. Even so, the movement is often directly along an axis, but we do not know which axis, positive or negative x, positive or negative y, or how much movement there was. The procedure is illustrated in Figures 8 and 9. Figure 8 shows the matching of a subimage to four boundaries of the original image. The original image is the center image, and subsequent subimages are displayed at the top, bottom, left, and right, respectively. Matching to the top boundary has the highest NCC value, with significance S = (0.33 - 0.11) / 0.33 = 0.67. Figure 9 shows stitching the top boundary of a subimage (top) to the original image (bottom) using matching translation. The result is the composite image on the right.

[0090] The similarity referred to above is measured as the normalized cross-correlation (NCC) between image I1 and translation image I2.

number

[0091] The following definitions are used herein: Pixel Average:

number

[0092] The NCC is a number between -1 and 1, with the following interpretation: NCC=1: fully correlated, e.g., I2=I1 NCC=0: No correlation NCC=-1: perfectly anti-correlated, e.g., I2=-I1

[0093] Rather than determining stitching in all four directions, it is possible to rotate both images and perform, for example, a bottom boundary stitching similarity determination, where bottom is just an example and could be top, right, or left, but what is important is that the same bottom boundary stitching similarity determination is performed on the rotated images.

[0094] It was found that scan direction detection is equivalent to sub-image rotation detection due to the following one-to-one relationship between sub-image rotation and scan direction:

number

number

number

number

[0095] Thus, if a stitching algorithm only allows for bottom boundary stitching, for example, the three-step procedure described above can be mapped to the following equivalent four-step procedure:

[0096] The partial image and the original image of the partial image are rotated discretely in 90° steps, and bottom boundary image matching is performed.

[0097] r 1,2 Let E{0°, 90°, 180°, 270°} be the rotations corresponding to the first and second highest similarities, respectively. Then, r 1 is the detected image rotation for stitching, and the importance of this rotation is S = (NCC r1 -NCC r2 ) / NCC r1 It is defined as follows.

[0098] A bottom border stitch is performed on the subimage rotated with the selected rotation.

[0099] The composite image is rotated inversely with the inverse rotation of the selected subimage rotation.

[0100] This procedure is illustrated in Figure 10, meaning that only one stitching algorithm needs to be implemented. Figure 10 shows a four-step procedure: partial image rotation and bottom boundary matching, rotation selection, bottom boundary stitching, and finally composite image derotation. This procedure is equivalent to the three-step procedure described with reference to Figures 8 and 9. The first four image columns show bottom edge image matching for all possible 90° rotations of the partial images. The third column, with a 180° rotation, yields the highest NCC = 0.33 and significance S = (0.33 - 0.11) / 0.33 = 0.67. Therefore, this rotation is selected for image merging. The resulting composite image in column 3 is then derotated -180°, resulting in the composite image in column 5. This image is therefore equivalent to the top edge stitching result for a bottom-to-top scan direction.

[0101] Here, assuming that the temporal order of the subimages is known and the scanning direction does not change, for a stitched sequence consisting of multiple N subimages, the success rate of automatic scanning direction detection can be improved by performing majority voting on all subimages in the stitched sequence in the following manner.

[0102] Perform image matching at all image boundaries of all sub-images out of the stitching sequence.

[0103] Stacking NCC values ​​for all subimage matchings of the sequence for all four image boundaries.

[0104] b 1,2 Let E{left, right, top, bottom} be the boundaries corresponding to the first and second maximum values ​​of the four stacked NCC values, respectively. Then, b 1 is the detected image boundary for stitching, and the importance of this image boundary is S=(NCC b1 -NCC b2 ) / NCC b1 It is defined as follows.

[0105] Image synthesis and scan direction determination can also be determined simultaneously. Because it is not known in advance when X-ray image acquisition will stop, the synthetic image should be created after adding each newly acquired partial X-ray image. Therefore, each partial image N is added to the synthetic image according to the scan direction detected on partial images 1,...,N. As a result, if the detected scan direction of images 1,...,N differs from the scan direction of images 1,...,N-1, images 1,...,N-1 must be re-stitched according to the newly detected scan direction. To optimize performance, as soon as the significance exceeds a predetermined threshold, the corresponding scan direction of each next acquired image will be used, and scan direction detection will be omitted thereafter (assuming the actual scan direction does not change during sequence acquisition).

[0106] Figure 11 shows a sequence diagram for detecting partial image rotation, which is equivalent to detecting scan direction as described above. In Figure 11, the numbers refer to the following logical software units and functions: Stitch Applications: 1 RotationDetector: 2 StitchingPipeline: 3 AlignmentAnalyzer: 4 Init(): 5 ClearSimilarities(): 6 Full image loop: 7 Addimage: 8 Total rotation loops: 9 Update(previousImage, rotate, isFirst): 10 Rotate image: 11 Process(previous image, isfirst): 12 Update(Image, Rotate): 13 Rotate image: 14 Process(Image): 15 Move, similarity=getAlignmentResults():pairs:16 Move, similarity=getAlignmentResults():pairs:17 Rotationsimilarities=setSimilarity(Similarity): 18 Similarities=addRotationSimilarities(Rotationsimilarities): 19 Previousimage=coptToPreviousimage(image): 20 rotation, importance = getRotation(): pairs: 21 Total rotation loops: 22 Full image loop: 23 cumSimilarities[rotation]=cumulate(similarity): 24 Rotation, significance = significanceAnalysis(cumSimilarities) 25

[0107] Continuing with Figure 11, for each additional image outside the stitching sequence, the stitching pipeline is first initialized with the previous image as the first image (update(...,isFirst)). In a second call, the current image is aligned with the previous image, and the alignment results (translation and similarity) are returned. The variable "rotationSimilarities" is a four-dimensional vector that stores the similarities for the four rotation indices. The variable "similarities" is a matrix, where the rows and columns are indexed by the image number and rotation indices, respectively. The function "cumulate" stacks the "similarities" for all image numbers. Thus, the variable "cumSimilarities" is a four-dimensional vector that stores the stacked similarities for the four rotation indices. The function "significanceAnalysis" determines the first and second maximum similarity values ​​NCCr1, NCCr2, and the corresponding rotation. The significance S of the detected rotation r1 with the maximum similarity is therefore S = (NCCr1 - NCCr2) / NCCr1.

[0108] Therefore, as shown in Figure 11, scan direction detection can be realized by executing the following logic software units:

[0109] Stitch Application This represents a SW application that allows the user to perform image stitching. The stitching application has access to the StitchingPipeline and RotationDetector.

[0110] Stitching Pipeline It encapsulates incremental stitching functionality, orchestrating data transfers or function calls into dedicated stitching classes, such as AlignmentAnalyzer or ImageComposer.

[0111] RotationDetector This encapsulates the detection of the rotation of sub-images within a stitched sequence. The detection of sub-image rotation is equivalent to the detection of the scan direction, which means the direction of detector movement during stitched acquisition.

[0112] AlignmentAnalyzer It is responsible for aligning or matching subsequent sub-images. Image matching is defined as determining the displacement between subsequent sub-images. The displacement found is the maximum similarity in the overlapping regions of the subsequent sub-images. The similarity is quantified by the normalized cross-correlation (NCC).

[0113] ImageComposer (not shown as such) It is responsible for combining the sub-images into a composite image. The processing method receives the displacement vectors between the subsequent sub-images. The displacement vectors can come from the AlignmentAnalyzer (automatic stitching) or from the user (manual stitching).

[0114] [Table 1]

[0115] In another exemplary embodiment, there is provided a computer program or a computer program element characterized in that it is configured to perform, on a suitable device or system, the method steps of a method according to one of the previous embodiments.

[0116] The computer program element is therefore stored on a computer unit, which is also part of an embodiment. This computing unit may be configured to perform or direct the execution of the steps of the above-described method. Furthermore, the computing unit may be configured to operate the components of the above-described device and / or system. The computing unit can be configured to operate automatically and / or to execute user instructions. The computer program is loaded into the working memory of a data processor. The data processor may therefore be equipped to perform the method according to one of the previous embodiments.

[0117] This exemplary embodiment of the present invention covers both computer programs that use the present invention from the beginning, and computer programs that are updated to turn existing programs into programs that use the present invention.

[0118] Furthermore, the computer program element is capable of providing all the steps necessary to implement the procedures of the exemplary embodiments of the method as described above.

[0119] According to a further exemplary embodiment of the present invention, a computer readable medium such as a CD-ROM, USB stick, or the like is presented, the computer readable medium having computer program elements stored thereon, the computer program elements being as described in the previous section.

[0120] The computer program may be stored and / or distributed on any suitable medium, such as an optical storage medium or a solid-state medium, supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems.

[0121] Nevertheless, the computer program may also be presented via a network such as the World Wide Web and downloaded from such a network into the working memory of a data processor. According to a further exemplary embodiment of the present invention, a medium for making a computer program element available for downloading is provided, the computer program element being arranged to perform a method according to one of the aforementioned embodiments of the present invention.

[0122] It should be noted that embodiments of the present invention are described with reference to different subject matters. In particular, some embodiments are described with reference to method-type claims, while other embodiments are described with reference to device-type claims. Nevertheless, those skilled in the art will infer from the above and following description that, unless otherwise indicated, any combination of features belonging to one type of subject matter, as well as any combination between features relating to different subject matters, is considered to be disclosed in this application. Nevertheless, all features can be combined, providing synergistic effects that are greater than the simple sum of the features.

[0123] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description is to be considered illustrative or exemplary and not restrictive. The invention is not limited to the disclosed embodiments. Other variations of the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the dependent claims.

[0124] In the claims, the word "comprises" does not exclude other elements or steps, and the words "a" or "an" in the singular do not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope.

Claims

1. 1. An apparatus for scan direction detection and stitching sequence determination for a plurality of X-ray images, the apparatus comprising: An input unit; a processing unit; Output unit and Equipped with the input unit provides the processing unit with a first X-ray image acquired by an X-ray image acquisition system, the first image having image data of a patient; the input unit provides the processing unit with a second X-ray image acquired by the X-ray image acquisition system after the X-ray image acquisition system has been moved relative to a patient, the second image having image data of the patient; the processing unit determines an upper similarity value comprising a comparison of at least one region of the patient image data at and / or adjacent to an upper boundary of the first image with at least one similarly sized region of the patient image data at and / or adjacent to a lower boundary of the second image; the processing unit determines a right similarity value comprising a comparison of at least one region of the patient image data at and / or adjacent to a right border of the first image with at least one similarly sized region of the patient image data at and / or adjacent to a left border of the second image; the processing unit determines a lower similarity value comprising a comparison of at least one region of the patient image data at and / or adjacent to a lower boundary of the first image with at least one similarly sized region of the patient image data at and / or adjacent to an upper boundary of the second image; the processing unit determines a left similarity value comprising a comparison of at least one region of the patient image data at and / or adjacent to a left border of the first image with at least one similarly sized region of the patient image data at and / or adjacent to a right border of the second image; the processing unit determining a scan direction and a translation distance of the X-ray image acquisition system associated with the movement of the X-ray image acquisition system, comprising utilizing a maximum value of the top, right, bottom, or left similarity value; the output unit outputs the scanning direction and the translation distance. Device.

2. 2. The apparatus of claim 1, wherein the processing unit further determines a combined image formed from the first image and the second image using the maximum of the top, right, bottom, or left similarity values, and the output unit outputs the combined image.

3. determining the upper similarity value comprises determining a plurality of similarity values ​​comprising comparing associated regions of different sizes of the patient's image data in the first image and the second image, and selecting a maximum similarity value from the plurality of similarity values ​​as the upper similarity value; determining the right similarity value comprises determining a plurality of similarity values ​​comprising comparing associated regions of different sizes of the patient's image data in the first image and the second image, and selecting a maximum similarity value from the plurality of similarity values ​​as the right similarity value; 3. The apparatus of claim 1, wherein determining the left similarity value comprises determining a plurality of similarity values ​​comprising a comparison of associated regions of different sizes of the patient's image data in the first image and the second image, and selecting a maximum similarity value from the plurality of similarity values ​​as the left similarity value, and wherein determining the left similarity value comprises a comparison of associated regions of different sizes of the patient's image data in the first image and the second image, and selecting a maximum similarity value from the plurality of similarity values ​​as the left similarity value.

4. 3. The apparatus of claim 1, wherein the comparison of each region of the patient's image data at and / or adjacent to the upper boundary of the first image with the equivalent sized region of the patient's image data at and / or adjacent to the lower boundary of the second image comprises utilizing an upper normalized cross-correlation algorithm.

5. 5. The apparatus of claim 4, wherein the comparison of each region of the patient image data at and / or adjacent to the right border of the first image with the equivalent sized region of the patient image data at and / or adjacent to the left border of the second image comprises utilizing a right-normalized cross-correlation algorithm; the comparison of each region of the patient image data at and / or adjacent to the bottom border of the first image with the equivalent sized region of the patient image data at and / or adjacent to the top border of the second image comprises utilizing the top-normalized cross-correlation algorithm or the bottom-normalized cross-correlation algorithm; and the comparison of each region of the patient image data at and / or adjacent to the left border of the first image with the equivalent sized region of the patient image data at and / or adjacent to the right border of the second image comprises utilizing the right-normalized cross-correlation algorithm or the left-normalized cross-correlation algorithm.

6. 5. The apparatus of claim 4, wherein the comparison of each region of the patient image data at and / or adjacent to the right border of the first image with the equivalent sized region of the patient image data at and / or adjacent to the left border of the second image comprises utilizing the upper normalized cross-correlation algorithm; the comparison of each region of the patient image data at and / or adjacent to the lower border of the first image with the equivalent sized region of the patient image data at and / or adjacent to the upper border of the second image comprises utilizing the upper normalized cross-correlation algorithm; and the comparison of each region of the patient image data at and / or adjacent to the left border of the first image with the equivalent sized region of the patient image data at and / or adjacent to the right border of the second image comprises utilizing the upper normalized cross-correlation algorithm.

7. The comparison of each region of the patient image data at and / or adjacent to the right boundary of the first image with the equivalent sized region of the patient image data at and / or adjacent to the left boundary of the second image includes a 90 degree counterclockwise rotation of the first image and a 90 degree counterclockwise rotation of the second image before using the upper normalized cross-correlation algorithm, and the comparison of each region of the patient image data at and / or adjacent to the lower boundary of the first image with the equivalent sized region of the patient image data at and / or adjacent to the upper boundary of the second image includes a 90 degree counterclockwise rotation of the first image and a 90 degree counterclockwise rotation of the second image before using the upper normalized cross-correlation algorithm.

6. The apparatus of claim 5, wherein the comparison of each region of the patient image data at and / or adjacent the left boundary of the first image with the equivalent sized region of the patient image data at and / or adjacent the right boundary of the second image comprises a 90 degree clockwise rotation of the first image and a 90 degree clockwise rotation of the second image before the use of the upper normalized cross-correlation algorithm.

8. 1. An apparatus for scan direction detection and stitching sequence determination for a plurality of X-ray images, said apparatus comprising: An input unit; a processing unit; Output unit and Equipped with the input unit provides to the processing unit a plurality of "N" x-ray images acquired by an x-ray image acquisition system, wherein after each of the first N-1 images, the x-ray image acquisition system is moved relative to a patient, each of the N x-ray images having image data of the patient; One X-ray image is selected from the N X-ray images; determining an upper similarity value, the determining a plurality of similarity values ​​comprising a comparison of at least one region of the patient image data at and / or adjacent to an upper boundary of the selected image with at least one similarly sized region of the patient image data at and / or adjacent to a lower boundary of each of the other N-1 images; and selecting a maximum similarity value from the plurality of similarity values ​​as the upper similarity value. determining a right similarity value, the processing unit performing a determination of a plurality of similarity values ​​comprising a comparison of at least one region of the patient image data at and / or adjacent a right boundary of the selected image with at least one similarly sized region of the patient image data at and / or adjacent a left boundary of each of the other N-1 images; and selecting a maximum similarity value from the plurality of similarity values ​​as the right similarity value. determining a lower similarity value, the determining a plurality of similarity values ​​comprising a comparison of at least one region of the patient image data at and / or adjacent a lower boundary of the selected image with at least one similarly sized region of the patient image data at and / or adjacent an upper boundary of each of the other N-1 images; and the processing unit selecting a maximum similarity value from the plurality of similarity values ​​as the lower similarity value. determining a left similarity value, the processing unit performing a determination of a plurality of similarity values ​​comprising a comparison of at least one region of the patient image data at and / or adjacent a left boundary of the selected image with at least one similarly sized region of the patient image data at and / or adjacent a right boundary of each of the other N-1 images; and selecting a maximum similarity value from the plurality of similarity values ​​as the left similarity value. the processing unit determining a scan direction and a translation distance of the X-ray image acquisition system associated with the movement of the X-ray image acquisition system, the scan direction and translation distance having a maximum value of the top, right, bottom, or left similarity value; the output unit outputs the scanning direction and the translation distance. Device.

9. 9. The apparatus of claim 8, wherein the processing unit further determines a combined image formed from the selected image and the other N-1 images using the maximum of the top, right, bottom, or left similarity values, and the output unit outputs the combined image.

10. determining the upper similarity value for each pair of the selected image and one of the other N-1 images comprises determining a plurality of similarity values ​​comprising comparing associated regions of different sizes of the patient's image data in the selected image and the one of the other N-1 images, and selecting a maximum similarity value from the plurality of similarity values ​​as the upper similarity value; determining the right similarity value for each pair of the selected image and one of the other N-1 images comprises determining a plurality of similarity values ​​comprising comparing associated regions of different sizes of the patient's image data in the selected image and the one of the other N-1 images, and selecting a maximum similarity value from the plurality of similarity values ​​as the right similarity value; 10. The apparatus of claim 8 or 9, wherein for each pair of an image and one of the other N-1 images, determining the bottom similarity value comprises determining a plurality of similarity values ​​comprising comparing a plurality of differently sized associated regions of the patient's image data in the selected image and the one of the other N-1 images, and selecting a maximum similarity value from the plurality of similarity values ​​as the bottom similarity value; and wherein for each pair of an image of the selected image and one of the other N-1 images, determining a plurality of similarity values ​​comprises comparing a plurality of differently sized associated regions of the patient's image data in the selected image and the one of the other N-1 images, and selecting a maximum similarity value from the plurality of similarity values ​​as the left similarity value.

11. 11. The apparatus of claim 8, wherein the comparison of each region of the patient's image data at and / or adjacent to the upper boundary of the selected image with the equivalent sized region of the patient's image data at and / or adjacent to the lower boundary of each of the other N-1 images comprises utilizing an upper normalized cross-correlation algorithm.

12. 10. The apparatus of claim 9, wherein the comparison of each region of the patient image data at and / or adjacent to the right boundary of the selected image with the equivalent sized region of the patient image data at and / or adjacent to the left boundary of each of the other N-1 images comprises utilizing a right-normalized cross-correlation algorithm; the comparison of each region of the patient image data at and / or adjacent to the lower boundary of the selected image with the equivalent sized region of the patient image data at and / or adjacent to the upper boundary of each of the other N-1 images comprises utilizing a bottom-normalized cross-correlation algorithm; and the comparison of each region of the patient image data at and / or adjacent to the left boundary of the selected image with the equivalent sized region of the patient image data at and / or adjacent to the right boundary of each of the other N-1 images comprises utilizing a left-normalized cross-correlation algorithm.

13. 12. The apparatus of claim 11, wherein the comparison of each region of the patient image data at and / or adjacent to the right boundary of the selected image with the equivalent sized region of the patient image data at and / or adjacent to the left boundary of each of the other N-1 images comprises utilizing the upper normalized cross-correlation algorithm, the comparison of each region of the patient image data at and / or adjacent to the lower boundary of the selected image with the equivalent sized region of the patient image data at and / or adjacent to the upper boundary of each of the other N-1 images comprises utilizing the upper normalized cross-correlation algorithm, and the comparison of each region of the patient image data at and / or adjacent to the left boundary of the selected image with the equivalent sized region of the patient image data at and / or adjacent to the right boundary of each of the other N-1 images comprises utilizing the upper normalized cross-correlation algorithm.

14. the comparison of each region of the patient image data at and / or adjacent to the right boundary of the selected image with the equivalent sized region of the patient image data at and / or adjacent to the left boundary of each of the other N-1 images comprises a 90 degree counterclockwise rotation of the selected image and a 90 degree counterclockwise rotation of each of the other N-1 images prior to the use of the upper normalized cross-correlation algorithm; 14. The apparatus of claim 13, wherein the comparison of each region of the patient image data at and / or adjacent to the left boundary of the selected image with the equivalent sized region of the patient image data at and / or adjacent to the right boundary of each of the other N-1 images comprises a 90 degree clockwise rotation of the selected image and a 90 degree clockwise rotation of each of the other N-1 images before the use of the upper normalized cross-correlation algorithm.

15. 1. A system for scan direction detection and stitching sequence determination for multiple X-ray images, the system comprising: Image acquisition system, A device according to any one of claims 1 to 7, and / or Apparatus according to any one of claims 8 to 14. A system comprising:

16. 1. A method for scan direction detection and stitching sequence determination for multiple X-ray images, the method comprising: a) providing a first X-ray image acquired by an X-ray image acquisition system to a processing unit, the first image comprising image data of a patient; b) providing a second X-ray image acquired by the X-ray image acquisition system to the processing unit after the X-ray image acquisition system has been moved relative to the patient, the second image comprising image data of the patient; c) determining by the processing unit an upper similarity value, comprising comparing at least one region of the patient image data at and / or adjacent to an upper boundary of the first image with at least one equally sized region of the patient image data at and / or adjacent to a lower boundary of the second image; d) determining by the processing unit a right similarity value comprising comparing at least one region of the patient image data at and / or adjacent to a right border of the first image with at least one similarly sized region of the patient image data at and / or adjacent to a left border of the second image; e) determining by the processing unit a lower similarity value, comprising comparing at least one region of the patient image data at and / or adjacent to a lower boundary of the first image with at least one similarly sized region of the patient image data at and / or adjacent to an upper boundary of the second image; f) determining by the processing unit a left similarity value comprising comparing at least one region of the patient image data at and / or adjacent to a left border of the first image with at least one equally sized region of the patient image data at and / or adjacent to a right border of the second image; g) determining by the processing unit a scan direction and a translation distance of the X-ray image acquisition system associated with the movement of the X-ray image acquisition system, the scan direction and translation distance comprising using the maximum value of the top, right, bottom, or left similarity values, and / or determining a combined image formed from the first image and the second image, the combined image comprising using the maximum value of the top, right, bottom, or left similarity values; h) outputting the scanning direction and translation distance and / or the combined image by an output unit; A method comprising:

17. 17. The method of claim 16, further comprising determining a combined image formed from the first image and the second image, comprising utilizing the maximum value of the top, right, bottom, or left similarity values, and outputting the combined image by an output unit.

18. 1. A method for scan direction detection and stitching sequence determination for multiple X-ray images, the method comprising: a1) providing a plurality of "N" x-ray images acquired by an x-ray image acquisition system to a processing unit, wherein after each of the first N-1 images, the x-ray image acquisition system is moved relative to a patient, and each of the N x-ray images comprises image data of the patient; b1) selecting one X-ray image from the N X-ray images; c1) determining an upper similarity value by the processing unit, the upper similarity value comprising comparing at least one region of the patient image data at and / or adjacent to an upper boundary of the selected image with at least one similarly sized region of the patient image data at and / or adjacent to a lower boundary of each of the other N-1 images; and selecting by the processing unit a maximum similarity value from the plurality of similarity values ​​as the upper similarity value. d1) determining a plurality of similarity values ​​by the processing unit, the similarity values ​​comprising comparing at least one region of the patient's image data at and / or adjacent to a right boundary of the selected image with at least one similarly sized region of the patient's image data at and / or adjacent to a left boundary of each of the other N-1 images; and selecting by the processing unit the largest similarity value of the plurality of similarity values ​​as the right similarity value. e1) determining a plurality of similarity values ​​by the processing unit, the similarity values ​​comprising comparing at least one region of the patient's image data at and / or adjacent to a lower boundary of the selected image with at least one similarly sized region of the patient's image data at and / or adjacent to an upper boundary of each of the other N-1 images; and selecting by the processing unit a maximum similarity value from the plurality of similarity values ​​as the lower similarity value. f1) determining a plurality of similarity values ​​by the processing unit, the plurality of similarity values ​​comprising comparing at least one region of the patient's image data at and / or adjacent to a left boundary of the selected image with at least one similarly sized region of the patient's image data at and / or adjacent to a right boundary of each of the other N-1 images; and selecting by the processing unit a maximum similarity value from the plurality of similarity values ​​as the left similarity value. g1) determining by the processing unit a scan direction and a translation distance of the X-ray image acquisition system associated with the movement of the X-ray image acquisition system, the scan direction and translation distance comprising using the maximum value of the top, right, bottom, or left similarity value; h1) outputting the scanning direction and translation distance by an output unit; A method comprising:

19. 20. The method of claim 18, further comprising determining a combined image formed from the selected image and the other N-1 images, comprising utilizing the maximum value of the top, right, bottom, or left similarity values, and outputting the combined image by an output unit.

Images