Training method and device for image stitching and image representation system

Abstract

The invention relates to an image processing system configured for performing a computer-implemented method for generating a stitched image with a convolutional neural network (208), a computer-implemented method for generating a stitched image with a convolutional neural network, and a method of training a convolutional neural network for determining an image representation for image stitching. The training of the convolutional neural network comprises receiving image data, wherein the image data comprises at least two images (101, 102) overlapping each other in an overlap region (103, 203), wherein the images overlap each other in a target anatomical structure (104), and wherein the overlap region comprises an optimal displacement between the two images such that the two images can be correctly combined for image stitching; determining a cost function (105, 205) of the displacement of the two images in the overlap region; determining a deviation of the cost function from a reference cost function; and optimizing the CNN based on the deviation.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more particularly to the field of image stitching. More specifically, this invention relates to the field of image processing systems configured to perform computer-implemented methods for generating image representations based on image stitching using neural networks, computer-implemented methods for generating image representations based on image stitching using neural networks, and methods for training neural networks for image representations of image stitched images. Background Technology

[0002] Image stitching-based image representation is a technique for creating images of anatomical structures larger than the detector size, such as in X-ray imaging. Typically, overlapping X-ray images of anatomical structures are required, particularly overlapping X-ray images of the corresponding target anatomical structure that should be analyzed based on the stitched image. Image processing algorithms are used to align and blend images that can be described as being in an adjacent state to obtain a single image of the entire target anatomical structure.

[0003] Classical image processing algorithms typically estimate the expected displacement between two adjacent images using similarity metrics. The similarity metric is calculated over the overlapping regions present in the two adjacent images, sometimes directly based on, for example, overlapping X-ray images. For each pair of adjacent images combined during stitching, the overlapping region is exposed twice; this results in a higher dose to the patient. Larger overlapping regions enable a more reliable stitching process, but they also increase the patient dose. Conversely, smaller overlapping regions reduce the patient dose but make the stitching process less reliable. These metrics, similarity measures, tend to have local maxima or minima, which can be caused by noise or parallax errors. These local extrema of the similarity measure can lead to inaccurate stitching results or even cause the entire stitching process to fail.

[0004] WO2022 / 182293A1 describes an image stitching method that includes generating a first stitched image based on multiple input images and an image domain stitching process.

[0005] Chukuri et al., “I,r-Stitch Unit: Encoder-Decoder-CNN Based Image Mosaicing Mechanism for Stitching Non-Homogeneous Image Sequences” (IEEE Access, IEEE, USA, Vol. 9, January 18, 2021, pp. 16761-16782), describe a robust and reliable image stitching method based on a novel convolutional encoder-decoder deep neural network.

[0006] EP3726457 A1 describes a system and method for stitching images using nonlinear optimization and minimization of multi-constraint cost functions.

[0007] EP2555156 A1 describes a method for image stitching implemented by an electronic device.

[0008] Levin et al., “Seamless Image Stitching in the Gradient Domain” (16 April 2004, Computer Vision-ECCV 2004; [Lecture Notes in Computer Science; LNCS], Springer-Verlag, Berlin / Heidelberg, pp. 377-389), described image stitching, which is used to combine several individual images with some overlap into a synthetic image by using a cost function. Summary of the Invention

[0009] Therefore, it is necessary to optimize the quality of the image stitching process, especially to optimize the quality of image representation based on image stitching. In particular, it is necessary to improve the image stitching process and avoid inaccurate stitching results or malfunctions throughout the entire stitching workflow.

[0010] The purpose of this invention is to provide a method for training, a method for generating image representations based on the method for training, and a system for image representation, wherein similarity measurement faults can be reduced, the stitching results can be improved, and the system is less prone to faults due to the similarity measurement.

[0011] The object of the invention is achieved by the subject matter of the independent claims, wherein further embodiments are incorporated in the dependent claims.

[0012] According to a first aspect of the invention, a method for training a convolutional neural network (CNN) for determining an image representation for image stitching is described. The method includes the steps of: receiving image data, wherein the image data includes at least two images overlapping each other in an overlapping region, wherein the images overlap each other in a target anatomical structure, wherein the overlapping region includes an optimal displacement between the two images such that the two images can be correctly combined for image stitching. The method may further include the steps of: determining a cost function for the displacement of the two images in the overlapping region; determining a deviation between the cost function and a reference cost function; and optimizing the CNN based on the deviation.

[0013] In the context of this invention, the term "image representation" should be understood as describing the display or illustration of at least one or more images. Specifically, the term "image representation" should be understood as describing the result of applying a nonlinear transformation to a pair of X-ray images. The nonlinear transformation can be learned from the data by training a CNN. In other words, these images can be generated using image stitching methods, particularly image stitching methods using CNN training methods as described herein. The image representation can be, for example, an X-ray image representation of an image generated from a patient (particularly from a specific target anatomical structure to be analyzed). The images generated from the patient to be represented can be preprocessed according to any method used for training and / or according to a computer-implemented method for generating the image representation in the image representation system as described herein.

[0014] In the context of this invention, the term "image stitching" or "stitching" should be understood to describe a method or process of combining at least two or more images to generate a new image. The term "image stitching" can be used in standard stitching procedures or in image stitching procedures using methods, systems, and elements according to embodiments, as described herein. Specifically, for image stitching, at least two images are used, wherein the images overlap in a specific region that is common to all images. Image stitching is used to provide a so-called seamless result of combining images.

[0015] In the context of this invention, the term "overlapping region" should be understood to describe a region included in the images to be stitched together (e.g., in at least two images). In other words, at least two images to be stitched together are generated from, for example, the patient's leg. The first image can be taken from the knee to the middle of the calf, and the second image can be taken from the foot to the middle of the calf. Both images include the same image information in the region in the middle of the calf and different image information corresponding to the knee (first image) and the foot (second image). The overlapping region is the area where the image information and / or the two displayed images include the same and / or similar image information. The size of the overlapping region depends in particular on the size of the respective single image itself, the movement of the image processing system, and / or the movement of the image device used to generate the images.

[0016] In the context of this invention, the term "target anatomical structure" should be understood as describing, for example, a part of the body that should be examined by a medical professional for medical treatment. The target anatomical structure can be any body part of the patient, such as the legs, feet, arms, chest, head, etc. To study the target anatomical structure, images can be generated; at least two images including the target anatomical structure can be generated.

[0017] In the context of this invention, the term "optimal displacement" should be understood to describe, for example, the arrangement of at least two images such that the combined image does not contain any stitching errors and / or overlap / overlay errors. Therefore, a seamless result for the combined image should be achieved, which requires near-precise overlap and near-identical exposure between the images to produce the optimal displacement. Specifically, a CNN is trained, for example, with pairs of overlapping images, where the corresponding displacements between them are the optimal correct displacements. The displacement is the true shift between two images so that they can be correctly stitched together. Therefore, the CNN is trained to predict the optimal displacement for unknown images (even for images with small overlapping regions). The CNN may be able to transform the image into a new image representation. The optimal displacement can be determined based on this new representation using a similarity metric, as described in other embodiments of the invention.

[0018] In other words, using the described method, neural networks, particularly convolutional neural networks (CNNs), can be trained with appropriate optimal input parameters, enabling the trained CNN to determine and predict the optimal displacement during real-time applications. Specifically, this is for learning an image representation better suited for image stitching. Therefore, a data-driven approach is described, in which a CNN is trained to learn a representation of an image representing the entire anatomical structure visible in the input image. The result of this method can be a smoothed cost function with a significant minimum to determine the optimal displacement and / or shift for image stitching.

[0019] For example, when a trained neural network is applied in a computer-implemented method for image representation based on image stitching and in a system using that method, the training method allows for a reduction in overlapping regions and thus a reduction in patient dose. In particular, the overlapping region can be reduced and can be as small as possible because the proposed method (for training the CNN) is able to determine the optimal displacement even with small overlapping regions, and the stitching process using the trained CNN is free from ambiguity in the cost function. Furthermore, when the trained method is used in an imaging system, as will be described in other embodiments herein, the improved image representation method allows for a smoother image stitching process; for example, the so-called stitching scale (which is a scale made of X-ray absorbing material) can be omitted, thus enhancing the workflow.

[0020] Training a neural network allows for the avoidance of spurious global minima in the cost function, leading to a reduced stitching failure rate. The input image to a CNN is, for example, two complete images, not just the overlapping region. Therefore, a CNN may be able to consider all anatomical features from the entire input image, which in turn allows for a reduction in overlapping areas and thus a reduction in X-ray exposure to the patient. Furthermore, CNNs may be able to reduce the failure rate, particularly for slender anatomical structures such as the legs, where incorrect displacements could occur due to blurring in the cost function using classical techniques.

[0021] The described training method and its application in computer-implemented methods and image representation systems can be used in the fields of X-ray, radiography, fluoroscopy, orthopedics, and bone radiography, and are not limited to any specific medical imaging.

[0022] According to an exemplary embodiment of the present invention, optimizing a CNN may include optimizing at least one parameter of the CNN, for example, optimizing the filter kernel. Alternatively or additionally, optimizing a CNN may include optimizing more than one parameter or all parameters, for example, optimizing all filter kernels. Optimizing a CNN may include determining the optimal weights of the CNN.

[0023] According to an exemplary embodiment of the present invention, the method may further include a step of penalizing the deviation from a reference cost function. This step can be used to train and optimize a CNN, particularly the parameters of the CNN, such as filter kernels and / or multiple filter kernels.

[0024] In other words, a penalty for deviation can be described, and a CNN can reward cost functions that satisfy a reference cost function. Therefore, a CNN can learn while training to obtain an image representation of the reference cost function. To reward the corresponding cost function, the deviation of the displacement from the reference cost function can be determined, where the reward can be larger when the deviation satisfies the reference cost function.

[0025] According to an exemplary embodiment of the present invention, the steps of determining the cost function, determining the deviation between the cost function and the reference cost function, and optimizing the CNN can be repeated until the cost function satisfies the reference cost function. Therefore, several iterative steps may be required before the cost function satisfies the reference function.

[0026] According to exemplary embodiments of the invention, image data can be obtained from clinically assembled sequences with annotated correct displacements and / or from image data of artificially constructed images with known displacements. A CNN can be trained on the clinical image data to have an optimal cost function with a global extremum at, for example, the location of the optimal displacement between two X-ray images. Since the CNN can take into account anatomical features from the entire input image, it also allows for reduction of overlapping areas and thus reduces X-ray exposure to the patient.

[0027] Image data including annotated correct displacements can be understood as image data where the correct displacements are known. For example, clinicians and / or medical personnel may have annotated images to determine the correct displacements. Therefore, the annotations can be manual, which may refer to the higher cost due to the required personnel. On the other hand, this can have the advantage that real noise and parallax errors exist in the images, and this is included in the training method, allowing the CNN to be trained using images from the real environment.

[0028] Other image data based on artificially constructed / generated image data with known displacements can be understood as describing the use of enhanced clinical images. Enhancement can describe modifying an image using, for example, artificial noise, perspective distortion, or contrast stretching and / or compression, where these examples are not limiting. Therefore, an image pair can include two images with the same content but different appearances. Two overlapping regions of the same size can be cropped from the two images. In fact, how the overlapping regions overlap is known, and the true displacement is known. This method can be used to artificially generate image pairs with known displacements. This method can be cheaper, can be automated, and can avoid the cost of manual image annotation. Two methods for determining the image data used for training can be combined during training.

[0029] According to an exemplary embodiment of the present invention, the step of determining the cost function of the displacement of two images may include applying a similarity metric to multiple displacements between the two images.

[0030] Similarity measures can be applied to more than one displacement between two input images. The only restriction to the definition of such a similarity measure is that it must be differentiable, as it is part of the training process, where all operations can, should, or specifically need to be differentiable. The output of a similarity measure applied to multiple displacements can be a cost function. Minimizing this cost function, and thus determining the minimum cost value, results in a displacement that maximizes the similarity in the overlapping regions. In ideal images, overlapping images would be identical in the overlapping regions. To determine the optimal displacement, it might only be necessary to minimize the cost function. However, in practice, disparity errors and noise interfere with this minimization process, and this can lead to incorrect displacements. Similarity measures are applied to multiple displacements to find the optimal displacement. Since similarity measures can be differentiable, the gradient of the loss can be backpropagated through the similarity measure, and the parameters of the CNN can be optimized. Therefore, a trained CNN may be able to predict new images by applying similarity measures to determine the optimal displacement.

[0031] According to an exemplary embodiment of the present invention, the step of applying a similarity measure may include determining the similarity of overlapping regions of possible displacements. Specifically, the similarity measure determines and calculates the similarity of each possible displacement. In the context of the present invention, the term "possible displacement" should be understood as describing a displacement that can be found in an image, where not the entire image overlaps with each other, but only the relevant overlapping regions. For example, the overlap may depend on the imaging system settings used for collimation and movement. Another example of a possible displacement may be the number of displacements between zero displacement and the maximum image width. For example, if the image has dimensions of 2400 mm (length) x 1800 mm (width), the possible displacements may be in the range of 0 to 1800 mm. Another example, for instance, if the image has dimensions of 400 mm x 120 mm, the possible displacements may be in the range of 0 to 120 mm. It should be noted that if the displacement is zero, the images may be identical. Furthermore, the displacement may also be affected by the geometry of the imaging system and / or the movement of the imaging system. Therefore, if the movement of the imaging system is known with a certain uncertainty, the range of possible displacements can be defined according to the uncertainty. For example, if the movement of the system is known to be within the range of 60mm to 100mm, then that range also defines the range of possible displacements.

[0032] According to an exemplary embodiment of the present invention, the step of applying a similarity measure may include generating a cost value for each of a plurality of displacements, wherein the cost function is generated based on the cost values ​​of the plurality of displacements. Therefore, the distribution, process, or trend of cost values ​​at different displacements generates the cost function. For each of the plurality of displacements, a cost value can be generated and determined. Figure 2 The diagram illustrates a possible process for the cost function.

[0033] According to an exemplary embodiment of the present invention, the reference cost function may be a convex cost function. An ideal cost function may be a parabola, which has its minimum value at the actual displacement, and is therefore a convex cost function. Thus, the reference cost function is a convex cost function, wherein deviations from this convex cost function can be determined. Given the step of penalizing deviations from the reference cost function, deviations that do not match and satisfy the convex cost function can be penalized.

[0034] According to an exemplary embodiment of the present invention, the minimum value of the cost function can correspond to the optimal displacement between two images. Therefore, when the cost function is determined, and particularly when the cost function satisfies a convex cost function, the training method may be able to determine the optimal displacement, since a convex cost function includes only one minimum value. Thus, the failure rate can be minimized by the new image representation of the CNN, which avoids local extrema in the cost function.

[0035] According to an exemplary embodiment of the present invention, a similarity measure may include at least one of the following: normalized cross-correlation, zero-mean normalized cross function, sum of absolute differences, or sum of squared differences. The similarity measure may not be limited to these explicitly mentioned similarity measures, and other similarity measures known to those skilled in the art may also be applicable. Applying these aforementioned similarity measures, and with the corresponding output being a cost value, a cost function can be generated based on multiple cost values ​​corresponding to the respective displacements.

[0036] According to a second aspect of the invention, a computer-implemented method for generating stitched images using a convolutional neural network (CNN) is described. The method includes the steps of: receiving an input dataset comprising at least two images, wherein the at least two images include overlapping regions, wherein the images overlap each other in a target anatomical structure; processing the input dataset using a CNN trained according to the method of any embodiment described herein; generating an output dataset from the CNN comprising at least two output images; applying a stitching algorithm to the at least two output images of the CNN; determining an optimal displacement using the stitching algorithm; and generating a synthetic image, i.e., a stitched image, based on the determined optimal displacement and the input images.

[0037] Using the described computer-implemented method, image representations can be computed / calculated based on neural networks, which are better suited for image stitching. In other words, a pair of images (e.g., X-ray images) are processed by a CNN architecture, which outputs a pair of images. Therefore, the output dataset can include at least two images transformed into a new image representation that includes at least all image information from the input images. The output images can have the same size as the input images. If less accuracy is required and computation time can be limited, a lower resolution can be used to reduce computation time and thus the size of the output images. Since the stitching algorithm uses the output dataset of the CNN, the optimal displacement is easily determined because the CNN output dataset includes the image representation, which yields a convex cost function. Therefore, the computer-implemented method is capable of generating a synthetic image with the optimal displacement, i.e., the stitched image.

[0038] According to an exemplary embodiment of the present invention, the step of processing the input dataset may include applying at least one trained filter kernel. The CNN includes filter kernels that can be optimized during the training method as described in any of the preceding embodiments.

[0039] According to a third aspect of the invention, an image processing system is described, configured to perform a computer-implemented method for generating an image representation, as described herein. The system includes at least one processor configured to receive an input dataset comprising at least two images, wherein the at least two images include overlapping regions, wherein the images overlap each other in a target anatomical structure. The processor may also be configured to process the input dataset using a CNN, generate an output dataset from the CNN comprising at least two output images, apply a stitching algorithm to the at least two output images of the CNN, determine an optimal displacement using the stitching algorithm, and generate a synthetic image, i.e., a stitched image, based on the determined optimal displacement and the input images.

[0040] The image processing system may also include a user interface configured to receive an output dataset from the processing system and to display the processing results, such as combined images, to a user.

[0041] According to various embodiments of this disclosure, the methods described herein can be implemented using a hardware computer system that executes software programs. Furthermore, in exemplary non-limiting embodiments, implementations may include distributed processing, component / object distributed processing, and parallel processing. Virtual computer system processing can implement one or more methods or functions as described herein, and the processors described herein can be used to support virtual processing environments.

[0042] According to a third aspect of the invention, a computer program comprising instructions, when executed by a computer, causes the computer to perform the methods of any of the embodiments described above. Specifically, the program can cause the computer to perform methods for training a CNN as described in any embodiment herein. Furthermore, the program can additionally and / or separately cause the computer to perform computer-implemented methods for generating image representations, as described in any embodiment herein.

[0043] A computer program unit can be a part of a computer program, but it can also be the entire program itself. For example, a computer program unit can be used to update an existing computer program to obtain the present invention.

[0044] The program unit may be stored on a computer-readable medium. A computer-readable medium may be considered a storage medium, such as a USB stick, CD, DVD, data storage device, hard disk, or any other medium on which the program unit described above may be stored.

[0045] According to a fourth aspect of the invention, a computer-readable medium includes instructions that, when executed by a computer, cause the computer to perform a method as described herein in any of the embodiments. Specifically, the computer-readable medium can cause a computer to perform a method for training a CNN as described herein in any of the embodiments. Furthermore, the computer-readable medium can additionally and / or separately cause a computer to perform a computer-implemented method for generating image representations as described in any of the embodiments.

[0046] It should be noted that embodiments of the invention have been described with reference to various subjects. In particular, some embodiments have been described with reference to system and computer program type claims, while others have been described with reference to method type claims. However, those skilled in the art will, from the above and below description, consider, in addition to any combination of features belonging to one type of subject matter, any combination of features relating to different subjects, particularly any combination of features between system type claims and method type claims, as disclosed in this application. Attached Figure Description

[0047] The aspects defined above and other aspects of the invention will become apparent from the examples of embodiments described below, and will be explained with reference to these examples. The invention will be described in more detail below with reference to examples of embodiments, but the invention is not limited thereto.

[0048] Figure 1 illustrates the image stitching process according to existing technology.

[0049] Figure 2 The illustration shows an image stitching process according to an embodiment of the present invention.

[0050] Figure 3 The illustration shows a flowchart of method steps according to an embodiment of the present invention.

[0051] List of reference numerals in the attached diagram:

[0052] 100 Image Representation

[0053] 101 First Image

[0054] 102 Second Image

[0055] Displacement of 103 and 203

[0056] 104 Target Anatomical Structure

[0057] Cost functions 105 and 205

[0058] Minimum values ​​of 106 and 206

[0059] 107 Minimum Error Value

[0060] 208 CNN

[0061] S1-S7 Method Steps Detailed Implementation

[0062] The illustrations in the accompanying drawings are schematic. Note that similar or identical elements are provided with the same reference numerals in different drawings.

[0063] Figure 1 illustrates an image stitching process according to the prior art. Figure 1 shows the overlap 103 of two images (first image 101 and second image 102). The images can be generated from a target anatomical structure 104, which is a leg portion in Figure 1. The stitching algorithm uses a similarity metric to estimate the expected displacement and outputs a cost function 105. As can be seen in the figure of Figure 1, the cost function process (x-axis) is a displacement along the displacement (y-axis), and the cost function is non-convex. The cost function 105 process shown in Figure 1 includes two minima 106 and 107. One minima 106 is the correct minima and corresponds to the optimal displacement. The other minima 107 is a false minima, which may occur due to faults during the stitching process, such as parallax errors, noise, etc.

[0064] Figure 2 Partially illustrated is the image stitching process according to an embodiment of the present invention. Compared to Figure 1, Figure 2 A small overlapping region 203 of the first image 101 and the second image 102 is shown. A CNN 208 is used to process the first image 101 and the second image 102, and to determine the cost function 205 in this image representation. Furthermore, Figure 2 The diagram illustrates a convex cost function 205, which should be achieved at the optimal displacement 206 used to determine the minimum value of cost function 205. Specifically, Figure 2 The illustration shows a schematic diagram of a computer-implemented method for generating / computing image representations based on image stitching using a convolutional neural network (CNN), as described in any embodiment herein. The input dataset may include a first image 101 and a second image 102. The input datasets 101 and 102 can be processed using a CNN 208 according to any embodiment, as described herein. The CNN 208 produces an output dataset comprising at least two images. A stitching algorithm is then applied to the output image of the CNN, and an optimal displacement is determined. The optimal displacement at the minimum value 206 of the cost function 205 for the two input images 101 and 102 is used to generate the synthesized image, i.e., the stitched image.

[0065] Figure 3 The illustration shows a flowchart illustrating method steps according to an embodiment of the present invention. About Figure 3 The steps are illustrated in the diagram, showing the training method steps, which can be followed as follows: Figure 3 The sequential execution steps described in [the document]. On the other hand, Figure 3 The order of steps in the training method may not be restrictive, and the training method can be performed in different orders. Furthermore, the order of steps can be changed, some steps can be replaced, or some steps from the training method can be removed. In step S1, image data can be received, wherein the image data includes at least two images overlapping each other in an overlapping region, wherein the images overlap each other in the target anatomical structure, and wherein the overlapping region includes the optimal displacement between the two images such that the two images can be correctly combined for image stitching. Step S1 can also be divided into, for example, three steps: S1a receiving input data, S1b processing the image data through a CNN, and S1c processing the image data. In step S2, the training method can determine a cost function for the displacement of the two images in the overlapping region. Then, in step S3, the deviation of the cost function from a reference cost function can be determined. In step S5, the CNN is optimized based on this deviation. The method may also include step S4, penalizing the deviation from the reference cost function to determine a cost function that satisfies the reference cost function. Step S7 includes generating an output, thus the processed input image data from the CNN, which is used to optimize the CNN. Alternatively, step S7 can be used as the output of the CNN for a stitching algorithm in a computer-implemented method. In the training method of step S6, the other steps from step S2 to step S5 can be repeated until the optimal displacement is determined, and thus until the cost function satisfies the reference cost function. Additional method steps can be added to the described method steps, for example, where step S2, which determines the cost function of the displacements of two images, includes applying a similarity metric to multiple displacements between the two images. These steps can be further divided into sub-steps; for example, the step of applying the similarity metric may include determining the similarity of overlapping regions of possible displacements, and / or the step of applying the similarity metric may include generating a cost value for each of the multiple displacements, wherein the cost function can be generated based on the cost values ​​of the multiple displacements.

[0066] Although the invention has been illustrated and described in detail in the accompanying drawings and the foregoing description, such illustrations and descriptions should be regarded as illustrative or exemplary rather than restrictive; the invention is not limited to the disclosed embodiments.

[0067] By studying the accompanying drawings, the disclosure, and the claims, those skilled in the art can understand and implement other variations of the disclosed embodiments in practicing the claimed invention. In the claims, the word "comprising" does not exclude other elements or steps, and the words "a" or "an" do not exclude a plurality. A single element or other unit may perform the function of several items recited in the claims. Although specific measures are recited in dissimilar dependent claims, this does not indicate that combinations of these measures cannot be advantageously used. Any reference numerals in the claims should not be construed as limiting the scope.

Claims

1. A method for training a convolutional neural network CNN (208) to determine an image representation (100) for image stitching, wherein, The method includes the following steps: Receive image data, wherein the image data includes: At least two images (101, 102) overlap each other in overlapping regions (103, 203), wherein the at least two images overlap each other in the target anatomical structure (104) within the overlapping regions, and The overlapping region includes the optimal displacement between the at least two images, such that the at least two images can be correctly combined for image stitching. Determine the cost function (105, 205) of the displacement of the at least two images in the overlapping region. Determine the deviation between the cost function and the reference cost function, wherein the reference cost function is a convex cost function. The CNN is optimized based on the aforementioned deviation.

2. The method according to claim 1, It also includes a step of penalizing deviations from the reference cost function.

3. The method according to claim 1 or 2, in, Repeat the following steps until the cost function satisfies the reference cost function: determine the cost function, determine the deviation between the cost function and the reference cost function, and optimize the CNN.

4. The training method according to any one of the preceding claims, in, The image data is obtained from clinically assembled sequences with annotated correct displacements and / or from image data of artificially constructed sequences with known displacements.

5. The method according to any one of the preceding claims, in, The step of determining the cost function of the displacements of the at least two images includes applying a similarity metric to multiple displacements between the at least two images.

6. The method according to claim 5, in, The step of applying the similarity metric includes: determining the similarity of the overlapping regions for possible displacements.

7. The method according to claim 5 or 6, in, The step of applying the similarity metric includes: generating a cost value for each of the plurality of displacements. The cost function is generated based on the cost value of the plurality of displacements.

8. The method according to any one of the preceding claims, in, The minimum value of the cost function corresponds to the optimal displacement of the at least two images.

9. The method according to any one of the preceding claims, in, Similarity measures include at least one of the following: normalized cross-correlation, zero-mean normalized cross function, sum of absolute differences, or sum of squared differences.

10. A computer-implemented method for generating stitched images using a convolutional neural network (CNN(208)), comprising the following steps: Receive an input dataset comprising at least two images (101, 102), wherein the at least two images include overlapping regions (103, 203), wherein the at least two images overlap each other in a target anatomical structure (104) within the overlapping regions; The input dataset is processed using the CNN trained using the method according to any one of claims 1 to 9. The CNN generates an output dataset, which includes at least two output images. The stitching algorithm is applied to the at least two output images of the CNN. The stitching algorithm described above is used to determine the optimal displacement. A composite image, i.e., the stitched image, is generated based on the determined optimal displacement and the input image.

11. The method according to claim 10, in, The steps for processing the input dataset include applying at least one trained filter kernel.

12. An image processing system configured to perform the method according to claim 10 or 11, the system comprising: At least one processor is configured as follows: Receive an input dataset comprising at least two images, wherein the at least two images include an overlapping region in which the at least two images overlap each other within a target anatomical structure; The input dataset is processed using the CNN described above. The CNN generates an output dataset, which includes at least two output images. The stitching algorithm is applied to the at least two output images of the CNN. The stitching algorithm described above is used to determine the optimal displacement. A composite image, or stitched image, is generated based on the determined optimal displacement and the input image.

13. A computer program comprising instructions that, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 11.

14. A computer-readable medium comprising instructions that, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 11.

Images