Training method and device for image stitching and image representation system

Abstract

The invention relates to an image processing system. The image processing system is configured for performing a computer-implemented method for generating a stitched image using a convolutional neural network (208), a computer-implemented method for generating a stitched image using a convolutional neural network, and a method of training a convolutional neural network for determining an image representation for image stitching. Training of the convolutional neural network comprises: receiving image data, where the image data comprises at least two images (101, 102) overlapping each other in an overlap region (103, 203), where the images overlap each other in a target anatomical structure (104), and where the overlap region comprises an optimal displacement between the two images, where the optimal displacement is greater than the optimal displacement between the two images; the two images can be correctly combined for image stitching; determining a cost function of the displacement of the two images in the overlapping region (105, 205); determining a deviation between the cost function and a reference cost function; and optimizing the CNN based on the deviation.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to the field of image stitching. More specifically, the present invention relates to the field of image processing systems configured to perform a computer-implemented method for generating an image representation based on image stitching using a neural network, a computer-implemented method for generating an image representation based on image stitching using a neural network, and a method for training a neural network for an image representation of an image stitching. Background Art

[0002] Image representation based on image stitching is a technique for creating images of anatomical structures that are larger than the detector size, for example in X-ray imaging. Typically, overlapping X-ray images of anatomical structures are required, in particular overlapping X-ray images from corresponding target anatomical structures that should be analyzed based on the stitched images. Images that can be described as being in a neighboring state are aligned and blended by image processing algorithms to obtain one image of the entire target anatomical structure.

[0003] Classical image processing algorithms usually estimate the expected displacement between two adjacent images by means of a similarity measure. The similarity measure is calculated on the overlapping area present in two adjacent images, wherein the similarity measure is directly calculated based on, for example, overlapping X-ray images. For each adjacent image pair combined during stitching, the overlapping area is exposed twice; this results in a higher dose for the patient. Large overlapping areas enable a more reliable stitching process, while they also increase the patient dose. In contrast, small overlapping areas reduce the patient dose, but make the stitching process less reliable. These measures, similarity measures, tend to local maxima or minima, which may be caused by noise or parallax errors. These local extremes of the similarity measure may lead to inaccurate stitching results, or even to failure of the entire stitching process.

[0004] WO2022 / 182293A1 describes an image stitching method, comprising generating a first stitched image based on multiple input images and based on an image domain stitching process.

[0005] Chukuri et al.: “I,r-Stitch Unit: Encoder-Decoder-CNN Based Image Mosaicing Mechanism for Stitching Non-Homogeneous Image Sequences” (IEEE Access, IEEE, USA, Vol. 9, January 18, 2021, pp. 16761-16782) describes a robust and reliable image stitching method based on a novel convolutional encoder-decoder deep neural network.

[0006] EP3726457 A1 describes a system and method for stitching images using non-linear optimization and multi-constrained cost function minimization.

[0007] EP2555156 A1 describes a method for image stitching implemented by an electronic device.

[0008] Levin et al.: “Seamless Image Stitching in the Gradient Domain” (16 April 2004, Computer Vision—ECCV 2004; [Lecture Notes in Computer Science; LNCS], Springer-Verlag, Berlin / Heidelberg, pp. 377-389) describes image stitching, which is used to combine several individual images with some overlap into a composite image using a cost function. Summary of the invention

[0009] Therefore, there is a need to optimize the quality of the image stitching process, and in particular, to optimize the quality of the image representation based on the image stitching. In particular, there is a need to improve the image stitching process and avoid inaccurate stitching results or failures in the entire stitching process.

[0010] The object of the invention is to provide a method for training, a method for generating image representations based on the method for training and a system for image representations, wherein malfunctions of the similarity measure can be reduced and the stitching result can be improved and less prone to malfunctions due to the similarity measure.

[0011] The objects of the invention are solved by the subject-matter of the independent claims, wherein further embodiments are incorporated in the dependent claims.

[0012] According to a first aspect of the invention, a method of training a convolutional neural network (CNN) for determining an image representation for image stitching is described. The method comprises the following steps: receiving image data, wherein the image data comprises at least two images overlapping each other in an overlapping region, wherein the images overlap each other in a target anatomical structure, wherein the overlapping region comprises an optimal displacement between the two images so that the two images can be correctly combined for image stitching. The method may also comprise the following steps: determining a cost function for the displacement of the two images in the overlapping region; determining a deviation of the cost function from a reference cost function; and optimizing the CNN based on the deviation.

[0013] In the context of the present invention, the term "image representation" should be understood as describing the display, illustration of at least one image or more than one image. In particular, the term "image representation should be understood as describing the result of applying a non-linear transformation to a pair of X-ray images. The non-linear transformation can be learned from the data by training a CNN. In other words, these images can be generated using image stitching methods, in particular image stitching methods using a training method for a CNN as described herein. The image representation can be, for example, an X-ray image representation of an image generated from a patient, in particular from a specific target anatomical structure that should be analyzed. The images generated from the represented patient can be pre-processed according to any method for training and / or according to a computer-implemented method for generating an image representation in an image representation system as described herein.

[0014] In the context of the present invention, the term "image stitching" or "stitching" should be understood as describing a method or process of combining at least two or more images to generate a new image, wherein the term image stitching can be used for a standard stitching process or for an image stitching process using methods, systems and components according to embodiments, as described herein. In particular, for the stitching of images, at least two images, wherein the images overlap in a specific area, the area of ​​the images is common. Image stitching is used to provide a so-called seamless result of combining images.

[0015] In the context of the present invention, the term "overlapping area" should be understood as describing an area that is included in the images to be stitched (e.g. in at least two images). In other words, at least two images that should undergo a stitching process are generated from, for example, the legs of a patient. The first image can be taken from the knee to the middle of the calf, and the second image can be taken from the foot to the middle of the calf. Both images include the same image information in the area in the middle of the calf and different image information corresponding to the knee (first image) and the foot (second image). The overlapping area is an area in which the image information and / or two displayed images include the same and / or similar image information. The size of the overlapping area depends in particular on the size of the corresponding single images themselves, the movement of the image processing system and / or the movement of the image device used to generate the images.

[0016] In the context of the present invention, the term "target anatomical structure" should be understood to describe, for example, a part of the body that should be examined by medical personnel for medical treatment, etc. The target anatomical structure may be any body part of a patient, such as a leg, foot, arm, chest, head, etc. In order to study the target anatomical structure, images may be generated, and at least two images including the target anatomical structure may be generated.

[0017] In the context of the present invention, the term "optimal displacement" should be understood to describe, for example, an arrangement of at least two images such that the combined image does not include any stitching failures and / or overlap / superposition failures. Therefore, a seamless result of the combined image should be achieved, which requires an almost exact overlap between the images and almost identical exposure to produce the optimal displacement. In particular, the CNN is trained with, for example, overlapping image pairs, wherein the corresponding displacement between them is the optimal correct displacement. The displacement is the real shift between the two images so that they can be correctly stitched. Therefore, the CNN is trained to predict the optimal displacement for unknown images (even for images with small overlapping areas). The CNN may be able to transform the image into a new image representation. The optimal displacement can be determined based on the new representation using a similarity measure, as will be described with respect to other embodiments of the present invention.

[0018] In other words, using the described method, a neural network, in particular a convolutional neural network, can be trained with corresponding optimal input parameters, so that the trained CNN is able to determine, predict optimal displacements during real-time applications. In particular, in order to learn image representations that are more suitable for image stitching. Therefore, a data-driven method is described, in which a CNN is trained to learn a representation of an image that represents the entire anatomical structure visible in the input image. The result of the method can be a smooth cost function with a significant minimum to determine the optimal displacement and / or shift for image stitching.

[0019] For example, when applying a trained neural network in a computer-implemented method for image representation based on image stitching and in a system using the method, the training method allows reducing the overlapping area and thus reducing the patient dose. In particular, the overlapping area can be reduced and can be as small as possible, because the proposed method (for training the CNN) is able to determine the optimal displacement even from a small overlapping area, and the stitching process using the trained CNN has no ambiguity in the cost function. In addition, when using the trained method in an imaging system, as will be described in other embodiments herein, the improved image representation method allows for an easy image stitching process, for example, a so-called stitching ruler (which is a ruler made of X-ray absorbing material) can be omitted, so that the workflow is enhanced.

[0020] The training of the neural network allows avoiding false global minima in the cost function, which results in a reduced stitching failure rate. The input images of the CNN are, for example, two complete images, not just the overlapping regions. Therefore, the CNN may be able to take into account all anatomical features from the entire input image, which in turn allows reducing the overlapping regions and therefore reducing the X-ray exposure to the patient. Furthermore, the CNN may be able to reduce the failure rate, in particular for long and thin anatomical structures such as the leg, which may result in incorrect displacements due to ambiguity in the cost function if classical techniques are used.

[0021] The described training methods and their application in computer implemented methods and image representation systems may be used in the fields of X-ray, radiography, fluoroscopy, orthopedics, bone radiography, and are not limited to any specific medical imaging.

[0022] According to an exemplary embodiment of the present invention, the step of optimizing the CNN may include optimizing at least one parameter of the CNN, for example, the filter kernel may be optimized. Alternatively or additionally, optimizing the CNN may include optimizing more than one parameter or all parameters, for example, optimizing all filter kernels. Optimizing the CNN may include determining the optimal weights of the CNN.

[0023] According to an exemplary embodiment of the present invention, the method may further include a step of penalizing deviations from a reference cost function. This step may be used to train and optimize CNN, in particular parameters of CNN, such as a filter kernel and / or multiple filter kernels.

[0024] In other words, a penalty for deviation can be described, and the CNN can reward those cost functions that satisfy the reference cost function. Therefore, the CNN can learn while training to obtain the image representation of the reference cost function. To reward the corresponding cost function, the deviation of the displacement from the reference cost function can be determined, where the reward can be greater when the deviation satisfies the reference cost function.

[0025] According to an exemplary embodiment of the present invention, the steps of determining the cost function, determining the deviation of the cost function from the reference cost function, and optimizing the CNN may be repeated until the cost function satisfies the reference cost function. Thus, several iterative steps may be required before the cost function satisfies the reference function.

[0026] According to an exemplary embodiment of the present invention, image data can be obtained from clinical mosaic sequences with annotated correct displacements and / or from artificially constructed image data with known displacements. A CNN can be trained on clinical image data to have an optimal cost function with a global extremum at the location of the optimal displacement between, for example, two X-ray images. Since the CNN can take into account anatomical features of the entire image from the input, it can also allow for reducing overlapping areas and thus reducing X-ray exposure to the patient.

[0027] Image data comprising annotated correct displacements may be understood as image data, wherein the correct displacement is known. For example, clinical staff and / or medical staff may have annotated pictures for determining the correct displacement. Thus, the annotations may be manual annotations, which may imply a higher cost due to the required personnel. On the other hand, this may have the advantage that real noise and parallax errors are present in the images, and this is included in the training method, such that the CNN is trained using images from a real environment.

[0028] Other image data based on artificially constructed / generated image data with known displacements can be understood to describe the use of enhanced clinical images. Enhancement can describe the modification of images using, for example, artificial noise, perspective distortion or contrast stretching and / or compression, wherein these examples are not limiting. Thus, an image date can include two images with the same content but different appearances. Two overlapping regions of the same size can be cropped from the two images. In fact, it is known how the overlapping regions overlap and the true displacement is known. The method can be used to artificially generate image pairs with known displacements. The method can be cheaper, can be automated, and can avoid the cost of manual image annotation. The two methods for determining the image data used for training can be combined during training.

[0029] According to an exemplary embodiment of the present invention, the step of determining a cost function for the displacement of the two images may comprise applying a similarity measure to a plurality of displacements between the two images.

[0030] The similarity measure can be applied to more than one displacement of two input images. The only restriction on the definition of such a similarity measure may be that the similarity measure is differentiable, since it is part of the training process, where all operations can be, should be, or specifically need to be differentiable. The output of the similarity measure applied to multiple displacements may be a cost function. Minimization, therefore the determination of the minimum cost value results in a displacement that maximizes the similarity in the overlapping region. In an ideal image, the overlapping images would be identical in the overlapping region. In order to determine the optimal displacement, it may only be necessary to minimize the cost function. However, in practice, parallax errors and noise interfere with the minimization process, and this may result in incorrect displacements. The similarity measure is applied to multiple displacements to find the optimal displacement. Since the similarity measure may be differentiable, the gradient of the loss can be back-propagated through the similarity measure and the parameters of the CNN can be optimized. Therefore, the trained CNN may be able to predict new images to which the similarity measure is applied to determine the optimal displacement.

[0031] According to an exemplary embodiment of the present invention, the step of applying a similarity measure may include determining the similarity of overlapping areas of possible displacements. Specifically, the similarity measure determines and calculates the similarity of each possible displacement. In the context of the present invention, the term "possible displacement" should be understood to describe displacements that can be found in images, where not the entire image overlaps each other, but only the relevant overlapping areas. For example, the overlap may depend on the imaging system settings for collimation and movement. Another example of a possible displacement may be the number of displacements between zero displacement and the maximum image width. For example, if the image has a size of 2400mm (length) x1800mm (width), the possible displacement may be in the range of 0 to 1800mm. Another example, for example, if the image has a size of 400mm x120mm, the possible displacement may be in the range of 0 to 120mm. It should be noted that if the displacement is zero, the images may be identical. In addition, the displacement may also be affected by the geometry of the imaging system and / or the movement of the imaging system. Therefore, if the movement of the imaging system is known with a certain uncertainty, the range of possible displacements can be defined based on the uncertainty. For example, if the movement of the system is to be known to be within the range of 60 mm to 100 mm, this range also defines the range of possible displacements.

[0032] According to an exemplary embodiment of the present invention, the step of applying the similarity measure may include generating a cost value for each of the plurality of displacements, wherein the cost function is generated based on the cost values ​​of the plurality of displacements. Thus, the distribution, process or trend of the cost values ​​at different displacements generates the cost function. For each of the plurality of displacements, a cost value may be generated, determined. Figure 2 A possible illustration of the process of describing the cost function is shown in .

[0033] According to an exemplary embodiment of the present invention, the reference cost function may be a convex cost function. An ideal cost function may be a parabola, which has its minimum at the true displacement and is therefore a convex cost function. Therefore, the reference cost function is a convex cost function, wherein deviations from the convex cost function may be determined. In view of the step of penalizing deviations from the reference cost function, deviations that do not match and satisfy the convex cost function may be penalized.

[0034] According to an exemplary embodiment of the present invention, the minimum value of the cost function may correspond to the optimal displacement of the two images. Therefore, when the cost function is determined and in particular when the cost function satisfies a convex cost function, the training method may be able to determine the optimal displacement because the convex cost function only includes one minimum value. Therefore, the failure rate can be minimized by the new image representation of the CNN, which can avoid local extreme values ​​in the cost function.

[0035] According to an exemplary embodiment of the present invention, the similarity metric may include at least one of the following: normalized cross correlation, zero mean normalized cross function, sum of absolute differences, or sum of squared differences. The similarity metric may not be limited to these explicitly mentioned similarity metrics, and other similarity metrics known to the technician may also be applicable. These above-mentioned similarity metrics are applied, and the corresponding output may be a cost value, wherein a cost function may be generated based on multiple cost values ​​corresponding to the corresponding displacements.

[0036] According to a second aspect of the invention, a computer-implemented method for generating a stitched image using a convolutional neural network (CNN) is described. The method comprises the following steps: receiving an input data set comprising at least two images, wherein the at least two images comprise an overlapping region, wherein the images overlap each other in a target anatomical structure; processing the input data set using a CNN trained using a method according to any one of the embodiments described herein, generating an output data set comprising at least two output images from the CNN, applying a stitching algorithm to the at least two output images of the CNN, determining an optimal displacement using the stitching algorithm, and generating a composite image, i.e., a stitched image, based on the determined optimal displacement and the input images.

[0037] Using the described computer-implemented method, image representations can be calculated / computed based on neural networks that are more suitable for image stitching. In other words, a pair of images (e.g., X-ray images) are processed by a CNN architecture, which outputs a pair of images. Therefore, the output data set can include at least two images, which are transformed into a new image representation that includes at least all image information from the input image. The output image can have the same size as the input image. If less accuracy is required and the calculation time can be limited, a lower resolution can be used to reduce the calculation time and therefore reduce the size of the output image. Since the stitching algorithm uses the output data set of the CNN, it is easy to determine the optimal displacement because the output data set of the CNN includes image representations, which results in a convex cost function. Therefore, the computer-implemented method is able to generate a synthetic image with an optimal displacement, i.e., a stitched image.

[0038] According to an exemplary embodiment of the present invention, the step of processing the input data set may include the step of applying at least one trained filter kernel. The CNN includes a filter kernel that can be optimized during the training method as described in any of the above embodiments.

[0039] According to a third aspect of the invention, an image processing system is described, which is configured to perform a computer-implemented method for generating an image representation, as described herein. The system includes at least one processor, and the at least one processor is configured to receive an input data set including at least two images, wherein the at least two images include an overlapping area, wherein the images overlap each other in the target anatomical structure. The processor can also be configured to process the input data set using a CNN, generate an output data set including at least two output images from the CNN, apply a stitching algorithm to the at least two output images of the CNN, determine an optimal displacement using the stitching algorithm, and generate a composite image based on the determined optimal displacement and the input image, i.e., a stitched image.

[0040] The image processing system may further comprise a user interface configured to receive the output data set from the processing system and configured to display the processing results, such as the combined image, to a user.

[0041] According to various embodiments of the present disclosure, the methods described herein can be implemented using a hardware computer system that executes a software program. In addition, in an exemplary non-limiting embodiment, implementations can include distributed processing, component / object distributed processing, and parallel processing. Virtual computer system processing can implement one or more methods or functions as described herein, and the processors described herein can be used to support a virtual processing environment.

[0042] According to a third aspect of the invention, a computer program comprising instructions is described, which, when executed by a computer, causes the computer to perform a method as described in any of the embodiments above. In detail, the program may cause the computer to perform a method for training a CNN as described in any of the embodiments herein. In addition, the program may additionally and / or separately cause the computer to perform a computer-implemented method for generating an image representation, as described in any of the embodiments herein.

[0043] A computer program element may be part of a computer program, but it may also be the entire program itself. For example, a computer program element may be used to update an already existing computer program to obtain the present invention.

[0044] The program element may be stored on a computer readable medium. A computer readable medium may be regarded as a storage medium, such as a USB stick, a CD, a DVD, a data storage device, a hard disk or any other medium on which a program element as described above may be stored.

[0045] According to a fourth aspect of the present invention, a computer-readable medium comprising instructions, which, when executed by a computer, causes the computer to perform a method as described in any embodiment herein. In detail, the computer-readable medium can cause the computer to perform a method for training a CNN as described in any embodiment herein. In addition, the computer-readable medium can additionally and / or separately cause the computer to perform a computer-implemented method for generating an image representation as described in any embodiment.

[0046] It should be noted that embodiments of the present invention have been described with reference to different subject matters. In particular, some embodiments have been described with reference to system, computer program type claims, while other embodiments have been described with reference to method type claims. However, a person skilled in the art will gather from the above and following description that, unless otherwise notified, in addition to any combination of features belonging to one type of subject matter, any combination between features relating to different subject matters, in particular any combination between features of system type claims and features of method type claims is considered to be disclosed in the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The aspects defined above and further aspects of the invention are apparent from the examples of embodiment to be described hereinafter and are explained with reference to the examples of embodiment.The invention will be described in more detail hereinafter with reference to examples of embodiment but to which the invention is not limited.

[0048] FIG. 1 illustrates an image stitching process according to the prior art.

[0049] Figure 2 The image stitching process according to an embodiment of the present invention is illustrated.

[0050] Figure 3 A flow chart illustrating method steps according to an embodiment of the invention is illustrated.

[0051] List of reference numerals:

[0052] 100 Image representation

[0053] 101 First Image

[0054] 102 Second Image

[0055] 103, 203 Displacement

[0056] 104 Target Anatomy

[0057] 105, 205 Cost Function

[0058] 106, 206 Minimum

[0059] 107 Error minimum

[0060] 208 CNN

[0061] S1-S7 Method Steps DETAILED DESCRIPTION

[0062] The illustrations in the drawings are schematically. Note that in different drawings, similar or identical elements are provided with the same reference signs.

[0063] FIG. 1 illustrates an image stitching process according to the prior art. FIG. 1 shows an overlap 103 of two images (a first image 101 and a second image 102). An image may be generated from a target anatomical structure 104, which in FIG. 1 is a portion of a leg. The stitching algorithm uses a similarity metric to estimate the expected displacement and outputs a cost function 105. As can be seen in the diagram of FIG. 1 , the process (x-axis) of the cost function is displaced on the displacement (y-axis), and the cost function is non-convex. The process of the cost function 105 as shown in FIG. 1 includes two minima 106 and 107. One minimum 106 is the correct minimum and corresponds to the optimal displacement. The other minimum 107 is a false minimum, which may occur due to a fault during the stitching process, such as parallax errors, noise, etc.

[0064] Figure 2 FIG. 1 is a diagram showing an image stitching process according to an embodiment of the present invention. Figure 2 A small overlapping area 203 of the first image 101 and the second image 102 is shown. A CNN 208 is used to process the first image 101 and the second image 102 and to determine a cost function 205 in the image representation. In addition, Figure 2 The convex cost function 205 is illustrated, which should be reached for determining the optimal displacement at the minimum 206 of the cost function 205. Specifically, Figure 2 A schematic diagram of a computer-implemented method for generating / computing an image representation based on image stitching using a convolutional neural network (CNN) is illustrated, as described in any of the embodiments herein. An input data set may include a first image 101 and a second image 102. The input data sets 101, 102 may be processed using a CNN 208 using a method according to any of the embodiments, as described herein. The CNN 208 generates an output data set including at least two images. Thereafter, a stitching algorithm is applied to the output image of the CNN, and an optimal displacement is determined. The optimal displacement at the minimum 206 of the cost function 205 for the two input images 101 and 102 is used to generate a composite image, i.e., a stitched image.

[0065] Figure 3 A flow chart illustrating method steps according to an embodiment of the present invention is illustrated. Figure 3 The steps of the training method are illustrated, wherein the training method can be performed according to Figure 3 On the other hand, Figure 3 The order of the steps in may not be limiting, so the training method may be performed in a different order. In addition, the order of the steps may be changed or some steps may be replaced or some steps from the training method may be eliminated. In step S1, image data may be received, wherein the image data includes at least two images overlapping each other in an overlapping region, wherein the images overlap each other in a target anatomical structure, and wherein the overlapping region includes an optimal displacement between the two images so that the two images can be correctly combined for image stitching. Step S1 may also be divided into three steps, for example: S1a receives input data, S1b processes the image data through a CNN, and S1c processes the image data. In step S2, the training method may determine a cost function for the displacement of the two images in the overlapping region. Then, in step S3, a deviation of the cost function from a reference cost function may be determined. In step S5, the CNN is optimized based on the deviation. The method may also include a step S4 of penalizing a deviation from a reference cost function to determine a cost function that satisfies the reference cost function. Step S7 comprises generating an output, thus processed input image data from the CNN, which is used to optimize the CNN, on the other hand, step S7 can be used as an output of the CNN for the stitching algorithm in the computer-implemented method. In the training method of step S6, the other steps starting from step S2 to step S5 can be repeated until the optimal displacement is determined, thus until the cost function satisfies the reference cost function. For the described method steps, further method steps can be added, for example, step S2 in which the cost function of the displacement of the two images is determined comprises: applying a similarity measure to multiple displacements between the two images. These steps can be further divided into sub-steps, for example, the step of applying the similarity measure may include: determining the similarity of the overlapping areas of the possible displacements, and / or the step of applying the similarity measure may include: generating a cost value for each of the multiple displacements, wherein the cost function can be generated based on the cost values ​​of the multiple displacements.

[0066] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments.

[0067] By studying the drawings, the disclosure and the claims, those skilled in the art can understand and implement other variations to the disclosed embodiments when practicing the claimed invention. In the claims, the word "comprising" does not exclude other elements or steps, and the words "one" or "an" do not exclude multiple. A single element or other unit can perform the functions of several items recorded in the claims. Although specific measures are recorded in different dependent claims, this does not indicate that a combination of these measures cannot be used to advantage. Any figure marks in the claims should not be interpreted as limiting the scope.

Claims

1. A method for training a convolutional neural network (CNN) (208) for determining an image representation (100) for image stitching, wherein: The method comprises the following steps: Receive image data, wherein the image data includes: at least two images (101, 102) overlapping each other in an overlapping region (103, 203), wherein the images overlap each other in a target anatomical structure (104), and wherein the overlap region includes an optimal displacement between the two images so that the two images can be correctly combined for image stitching, determining a cost function (105, 205) for the displacement of the two images in the overlapping region, determining a deviation of the cost function from a reference cost function, The CNN is optimized based on the bias.

2. The method according to claim 1, Also included is the step of penalizing deviations from the reference cost function.

3. The method according to claim 1 or 2, in, Repeat the following steps until the cost function satisfies the reference cost function: determining the cost function, determining the deviation of the cost function from the reference cost function, and optimizing the CNN.

4. A training method according to any one of the preceding claims, in, The image data are obtained from clinically stitched sequences with annotated correct displacements and / or from manually constructed image data with known displacements.

5. The method according to any one of the preceding claims, in, The step of determining the cost function for the displacement of the two images comprises applying a similarity measure to a plurality of displacements between the two images.

6. The method according to claim 5, in, The step of applying the similarity measure comprises determining the similarity of the overlapping regions for possible displacements.

7. The method according to claim 5 or 6, in, The step of applying the similarity measure comprises generating a cost value for each of the plurality of displacements, The cost function is generated according to the cost values ​​of the multiple displacements.

8. The method according to any one of the preceding claims, in, The reference cost function is a convex cost function.

9. The method according to any one of the preceding claims, in, The minimum value of the cost function corresponds to the optimal displacement of the two images.

10. The method according to any one of the preceding claims, in, The similarity measure includes at least one of: a normalized cross correlation, a zero-mean normalized cross function, a sum of absolute differences, or a sum of squared differences.

11. A computer-implemented method for generating a stitched image using a convolutional neural network (CNN (208)), comprising the steps of: Receiving an input data set comprising at least two images (101, 102), wherein the at least two images comprise an overlapping region (103, 203) in which the images overlap each other in a target anatomical structure (104); processing the input data set using the CNN trained according to the method of any one of claims 1 to 10, generating an output dataset from the CNN, the output dataset comprising at least two output images, applying a stitching algorithm to the at least two output images of the CNN, Use the stitching algorithm to determine the optimal displacement, A composite image, ie, the stitched image, is generated based on the determined optimal displacement and the input image.

12. The method according to claim 11, in, The step of processing said input data set comprises the step of applying at least one trained filter kernel.

13. An image processing system configured to perform the method according to any one of claims 11 or 12, the system comprising: At least one processor configured to: receiving an input data set comprising at least two images, wherein the at least two images include an overlap region in which the images overlap each other in a target anatomical structure; Using the CNN to process the input data set, generating an output dataset from the CNN, the output dataset comprising at least two output images, applying a stitching algorithm to the at least two output images of the CNN, Use the stitching algorithm to determine the optimal displacement, A composite image, ie a stitched image, is generated based on the determined optimal displacement and the input image.

14. A computer program comprising instructions which, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 12.

15. A computer-readable medium comprising instructions which, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 12.

Images