Deep learning robustness against differences in the field of view

Abstract

To provide systems and methods that facilitate deep learning robustness against display field of view (DFOV) variations.SOLUTION: A system accesses a deep learning neural network and a medical image. If first spatial resolution, on which the deep learning neural network is trained, fails to match second spatial resolution, exhibited by the medical image, then the system executes the deep learning neural network on an image obtained by resampling the medical image, where the image obtained by resampling the medical image is generated by up-sampling or down-sampling the medical image until the medical image matches the first spatial resolution.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present disclosure generally relates to deep learning, and more specifically, to the robustness of deep learning with respect to differences in display fields of view.

Background Art

[0002] Deep learning neural networks can be trained to perform inference tasks on medical images. How accurately a deep learning neural network performs an inference task depends on the display field of view of the medical image.

[0003] Therefore, a system or technique that can address one or more of these technical problems is desired.

Summary of the Invention

[0004] The following presents an overview for providing a basic understanding of one or more embodiments of the present invention. This overview is not intended to identify key or critical elements or to delineate the scope of particular embodiments or the scope of the claims. Its sole purpose is to present concepts in a simplified form as a prelude to the more detailed description that is presented later. In one or more embodiments described herein, a device, system, method implemented by a computer, apparatus, or computer program product that assists in the robustness of deep learning with respect to differences in display fields of view is described.

[0005] According to one or more embodiments, a system is provided. The system can include a non-transitory computer-readable memory that can store computer-executable components. The system can further include a processor that can be operably coupled to the non-transitory computer-readable memory and can execute the computer-executable components stored in the non-transitory computer-readable memory. In various embodiments, the computer-executable components can include a deep learning neural network and an access component that can access medical images. In various aspects, the first spatial resolution trained by the deep learning neural network may not match the second spatial resolution shown by the medical image. In various embodiments, the computer-executable components can further include an execution component that can execute the deep learning neural network on an image obtained by resampling the medical image. The image obtained by resampling the medical image can exhibit the first spatial resolution trained by the deep learning neural network.

[0006] According to one or more embodiments, the above system can be implemented as a method implemented by a computer or a computer program product.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Best Mode for Carrying Out the Invention

[0008] The following embodiments for carrying out the invention are merely illustrative and are not intended to limit the embodiments or the use / usage of the embodiments. Further, it is not intended to be bound by the explicit or implicit information described in the above "Background Art" or "Summary of the Invention" column or "Embodiments for Carrying Out the Invention" column.

[0009] Next, one or more embodiments will be described with reference to the drawings, and like reference numerals are used throughout to refer to like elements. In the following description, for the purpose of better understanding one or more embodiments, a number of specific details are set forth for illustrative purposes. However, in various cases, it is clear that one or more embodiments can be practiced without these specific details.

[0010] A deep learning neural network can be trained (e.g., by supervised training, unsupervised training, reinforcement learning) so that inference tasks (e.g., image quality improvement, image noise removal, image kernel conversion) of medical images (e.g., scanned images / reconstructed images generated by a computed tomography (CT) scanner, scanned images / reconstructed images generated by a magnetic resonance imaging (MRI) scanner, scanned images / reconstructed images generated by a positron emission tomography (PET) scanner, scanned images / reconstructed images generated by an X-ray scanner, scanned images / reconstructed images generated by an ultrasonic scanner) are executed.

[0011] In various aspects, the display field of view (DFOV) is a characteristic, feature, or attribute of a medical image. More specifically, in various embodiments, the DFOV can be considered as a controllable parameter / setting of a medical imaging device (e.g., of a CT scanner, an MRI scanner, a PET scanner, an X-ray scanner, an ultrasound scanner), and such a controllable parameter / setting can affect how much of the imaging field of view of the medical imaging device is reconstructed into the medical image acquired / generated by the medical imaging device. Since the medical imaging device can be configured to acquire / generate a plurality of medical images, each having a predetermined number of pixels or voxels / a predetermined pixel array or voxel array, the selectable settable value for the DFOV parameter / setting of the medical imaging device can affect the spatial resolution (sometimes also referred to as the grid size) of the medical image. That is, the selectable settable value for the DFOV parameter / setting of the medical imaging device can affect how the physical feature portions are represented by each pixel / voxel of the medical image. In particular, when the DFOV of a medical image is given, the spatial resolution of a given feature portion of the medical image can be calculated by dividing the given DFOV by the number of pixels / voxels in which the given feature portion extends in the medical image (e.g., a DFOV with good granularity corresponds to a spatial resolution with good granularity, and a DFOV with poor granularity corresponds to a spatial resolution with poor granularity). In other words, the spatial resolution of a medical image can be considered as conveying how physically large (e.g., with poor granularity) or how physically small (e.g., with good granularity) the pixels / voxels of the medical image are, and the spatial resolution can be directly related to the DFOV. Thus, in various examples, the DFOV and the spatial resolution can be considered to be correlated with each other, or can be considered as related / interchangeable metrics with each other.

[0012] As a non-limiting example, assume that a medical imaging device is configured to acquire / generate medical images, and each medical image is a pixel array having a length of x pixels and a width of y pixels (x and y are any suitable positive integers). Further, assume that the DFOV parameter / setting of the medical imaging device is set to a relatively large value (such as 40 centimeters (cm)). In such a case, the medical image acquired / generated by the medical imaging device can be considered to represent an area of 40×40 cm 2 and each pixel of the medical image can be considered to represent an area of (40 / x)×(40 / y) cm 2 . This can be considered a relatively coarse (e.g., poor granularity) spatial resolution. In contrast, assume that the DFOV parameter / setting of the medical imaging device is set to a relatively small value (such as 10 cm). In such a case, the medical image acquired / generated by the medical imaging device can be considered to represent an area of 10×10 cm 2 and each pixel of the medical image can be considered to represent an area of (10 / x)×(10 / y) cm 2 . This can be considered a relatively fine (e.g., good granularity) spatial resolution. In other words, the pixels of a medical image with a DFOV of 40 cm can be considered larger, coarser, or have poorer granularity than the pixels of a medical image with a DFOV of 10 cm. In still other words, as the DFOV of the medical image decreases, the pixels / voxels of the medical image become smaller, and thus, can be considered to have good granularity (e.g., as the granularity of the DFOV improves, the spatial resolution can improve the granularity).

[0013] It is considered impossible to generate a training dataset from multiple DFOVs (from multiple spatial resolutions). Certainly, a model can be trained by combining images from different finite numbers of DFOV settings, but the performance of such a model will be averaged rather than optimized for that finite number of DFOV settings. Furthermore, since the DFOV (spatial resolution) can be considered to vary over a continuous range of possible values, a finite set of DFOVs cannot cover the entire such continuous range.

[0014] For at least these reasons, deep learning neural networks can be trained with medical images showing a single or uniform DFOV (a single or uniform spatial resolution). That is, all medical images for training a deep learning neural network can show the same DFOV (the same spatial resolution) as each other. Thus, a deep learning neural network can be considered to be trained for a particular DFOV, for the sake of a particular DFOV, or with respect to a particular DFOV (with respect to a particular spatial resolution).

[0015] Unfortunately, when a deep learning neural network is executed on a medical image having a different DFOV (spatial resolution) from the DFOV (spatial resolution) trained by the deep learning neural network, the deep learning neural network may exhibit a decrease in inference accuracy. For example, assume that the deep learning neural network is configured to improve the image quality of a medical image. In such a case, the deep learning neural network may not be able to improve the accuracy of the image quality of a medical image having a DFOV (spatial resolution) that does not match the DFOV (spatial resolution) trained by the deep learning neural network. As another example, assume that the deep learning neural network is configured to perform image noise removal on a medical image. In that case, the deep learning neural network may not be able to accurately remove the noise from a medical image having a DFOV (spatial resolution) that does not match the DFOV (spatial resolution) trained by the deep learning neural network. As yet another example, assume that the deep learning neural network is configured to perform image kernel conversion on a medical image. In such a case, the deep learning neural network may not be able to accurately apply the image kernel conversion to a medical image having a DFOV (spatial resolution) that does not match the DFOV (spatial resolution) trained by the deep learning neural network.

[0016] Therefore, a system or technique capable of addressing one or more of these technical problems is desired.

[0017] The various embodiments described herein can address one or more of these technical problems. One or more of the embodiments described herein can include a system, a method implemented by a computer, an apparatus, or a computer program product that can assist the robustness of deep learning with respect to differences in display fields of view (with respect to differences in spatial resolution). In other words, the inventors of the various embodiments described herein have thought of various techniques for accurately executing a deep learning neural network on medical images, although the DFOV (spatial resolution) of medical images may be different from the DFOV trained by the deep learning neural network. In particular, such various techniques can include the following. Resampling (e.g., upsampling) a medical image to match the DFOV (spatial resolution) of the image obtained by resampling the medical image with the DFOV trained by a deep learning neural network, Executing a deep learning neural network on the image obtained by resampling the medical image, thereby obtaining an output image whose DFOV (its spatial resolution) matches the DFOV trained by the deep learning neural network, Resampling (e.g., downsampling) the output image to match the DFOV of the medical image with the DFOV of the image obtained by resampling the output image.

[0018] More specifically, the various embodiments described herein can be considered as computer tools (e.g., any suitable combination of computer-executable hardware or computer-executable software) that can assist the robustness of deep learning with respect to differences in DFOV. In various aspects, such computer tools can comprise an access component, a pre-execution resample component, an execution component, a post-execution resample component, or a result component.

[0019] In various embodiments, medical images are used. In various aspects, a medical image can represent one or more anatomical structures (e.g., tissue, organ, body part, or a portion thereof) of a patient (e.g., human, animal, or otherwise). In various embodiments, a medical image can indicate any suitable format or dimension. For example, in some cases, a medical image can be a two-dimensional array of pixels. In other cases, a medical image can be a three-dimensional array of voxels. In various aspects, a medical image can be acquired or generated by any suitable medical imaging device (e.g., CT scanner, MRI scanner, PET scanner, X-ray scanner, ultrasound scanner) or by any suitable image reconstruction technique. In various aspects, a medical image can be acquired / generated according to any suitable DFOV (according to any suitable spatial resolution).

[0020] In various embodiments, a deep learning neural network is used. In various aspects, the deep learning neural network represents any suitable deep learning architecture. For example, the deep learning neural network can include any suitable number of any suitable type of layer (e.g., an input layer, one or more hidden layers, an output layer, and any of these layers can be a convolutional layer or a non-linear layer), can include any suitable number of neurons in the various layers (e.g., different layers can have the same number of neurons or different numbers of neurons from each other), can include any suitable activation function (e.g., softmax, sigmoid, hyperbolic tangent, rectified linear unit) in the various neurons (e.g., different neurons can have the same activation function or different activation functions from each other), or can include any suitable intervening neuron connections (e.g., forward connections, skip connections, recurrent connections). In some embodiments, the deep learning neural network can be made to not include dense layers (e.g., can be made to not include fully connected layers). In such cases, the deep learning neural network can be made executable for inputs of varying size (e.g., not fixed) (e.g., convolutional layers and non-linear layers are applicable to inputs of any size / inputs of varying size, while dense / fully connected layers are only applicable to inputs of fixed size).

[0021] In various aspects, a deep learning neural network can be configured to perform any suitable inference task on an input medical image. As a non-limiting example, the inference task can be a task of improving image quality (e.g., improving the visual quality of the input medical image). As another non-limiting example, the inference task can be image kernel conversion (e.g., rendering an input medical image according to different image kernels such as a bone kernel or a soft tissue kernel). As yet another non-limiting example, the inference task can be image noise removal (e.g., reducing the amount of visual noise present in the input medical image). As yet another non-limiting example, the inference task can be image segmentation (e.g., determining to which class each pixel / voxel of the input medical image belongs). In any case, a deep learning neural network can be configured to receive a medical image as input and generate an output corresponding to the inference task. For example, when the inference task is image quality improvement, the deep learning neural network can be configured to generate an image with improved image quality of the input medical image. As another example, when the inference task is image kernel conversion, the deep learning neural network can be configured to generate an image with the converted kernel of the input medical image. As yet another example, when the inference task is image noise removal, the deep learning neural network can be configured to generate an image with the noise removed from the input medical image. As yet another example, when the inference task is image segmentation, the deep learning neural network can be configured to generate a segmentation mask of the input medical image.

[0022] In various aspects, a deep learning neural network can be, or can have been, trained with an appropriate type of training or an appropriate paradigm of training. For example, a deep learning neural network can be trained with supervision based on an annotated training dataset. In such a case, the internal parameters (e.g., convolutional kernels) of the deep learning neural network can be randomly initialized. In various aspects, appropriate medical images for training and appropriate annotations corresponding to the medical images can be selected from the annotated training dataset.

[0023] In various aspects, the medical images selected for training are supplied as input to the deep learning neural network, whereby the deep learning neural network can generate some output. More specifically, in various aspects, the input layer of the deep learning neural network can receive the medical images selected for training, and the medical images selected for training cause a forward pass through one or more hidden layers of the deep learning neural network, and the output layer of the deep learning neural network can calculate an output based on the activation of one or more hidden layers of the deep learning neural network.

[0024] In various embodiments, the output is a prediction / inference (e.g., It can be considered as a predicted / inferred image with improved quality, a kernel-converted predicted / inferred image, a noise-removed predicted / inferred image, a predicted / inferred segmentation mask). In contrast, the selected annotation can be considered as known ground truth data or ground truth data that matches the selected training medical image (e.g., a ground truth image with good image quality, a kernel-converted ground truth image, a noise-removed ground truth image, a ground truth segmentation mask). Note that if the deep learning neural network has not been trained at all or has been trained very little so far, the output may be very inaccurate (e.g., the output may be very different from the selected annotation).

[0025] In any case, an error or loss (e.g., mean absolute error (MAE), mean squared error (MSE), cross entropy) can be calculated between the output and the selected annotation, and the internal parameters of the deep learning neural network can be updated by performing backpropagation (e.g., stochastic gradient descent) that operates based on the calculated error / loss.

[0026] In various embodiments, such a training procedure can be repeated for each training medical image in the annotated training data set. As a result, the internal parameters of the deep learning neural network (e.g., convolutional kernels) can be iteratively optimized to accurately generate predictions / inferences based on the input medical images. In various cases, any appropriate training batch size, any appropriate training termination criterion, or any appropriate error / loss function can be implemented during such training.

[0027] While the above embodiments focus on supervised training, this is merely a non-limiting example for ease of explanation. In various aspects, instead, the deep learning neural network may undergo or may already have undergone unsupervised training based on an unannotated training dataset, or may undergo or may already have undergone reinforcement training based on iterative rewards / penalties.

[0028] In some cases, the computer tools described herein can assist or perform such training on the deep learning neural network.

[0029] Regardless of the training method the deep learning neural network has undergone, the training medical images used by the deep learning neural network for training, in various examples, all have a specific DFOV (specific spatial resolution) for practical reasons related to whether it is easy to obtain / curate the training dataset. Therefore, the deep learning neural network can be considered to be trained or have been trained at the specific DFOV (the specific spatial resolution).

[0030] In various aspects, it is desirable to perform an inference task on a medical image. However, in various aspects, the DFOV (spatial resolution) of a medical image may not match a specific DFOV (specific spatial resolution) trained by a deep learning neural network. More specifically, in various cases, the DFOV (spatial resolution) of a medical image may be less granular than a specific DFOV (specific spatial resolution) trained by a deep learning neural network. That is, the pixels / voxels of the medical image may be larger than the pixels / voxels of the training medical image, and thus may be represented with poor granularity of physical area / volume. However, in other cases, the DFOV (spatial resolution) of a medical image may be represented with better granularity than the DFOV (spatial resolution) trained by a deep learning neural network. That is, the pixels / voxels of the medical image may be smaller than the pixels / voxels of the training medical image, and thus may have good granularity of physical area / volume. In any case, when a deep learning neural network is directly executed on a medical image, it is expected that the inference output will be inaccurate or the accuracy of the inference output will decrease due to such a DFOV mismatch (due to a spatial resolution mismatch such as lp). In various cases, the computer tool described herein can easily perform an inference task on a medical image without loss of accuracy / precision despite the DFOV mismatch (despite the spatial resolution mismatch).

[0031] In various embodiments, an access component of a computer tool can electronically receive or access a deep learning neural network or a medical image. In some aspects, the access component can electronically retrieve a deep learning neural network or a medical image from an appropriate centralized data structure or distributed data structure (e.g., a graph data structure, a relational data structure, a hybrid data structure), regardless of whether the centralized data structure or distributed data structure is remote or local to the access component. In any case, the access component can electronically obtain or access a deep learning neural network or a medical image and enable other components of the computer tool to electronically interact with (e.g., read, write, edit, copy, operate, control, activate) the deep learning neural network or the medical image.

[0032] In various embodiments, a pre-execution resample component of a computer tool can electronically generate an image obtained by resampling a medical image, and the image obtained by resampling the medical image can indicate a specific DFOV (specific spatial resolution) trained by a deep learning neural network.

[0033] As a non-limiting example, assume that the DFOV (spatial resolution) of a medical image has worse granularity than the DFOV (spatial resolution) trained by a deep learning neural network. In such a case, the pre-execution resample component can apply any suitable upsampling technique (e.g., nearest neighbor interpolation, bilinear interpolation, cubic interpolation, or bicubic interpolation) to the medical image in various manners. In various embodiments, such an upsampling technique can be considered to increase the number of pixels / voxels of the medical image. In other words, such an upsampling technique can be considered to represent the medical image using more pixels / voxels. Thus, in such a case, each pixel / voxel of the image obtained by resampling the medical image is smaller than each pixel / voxel of the medical image (e.g., of the original medical image / of the medical image that has not been changed at all), and thus, it can be considered that the granularity of the physical area / volume is well represented. In this way, it can be considered that upsampling the medical image improves the granularity of the DFOV (spatial resolution) of the medical image. In various aspects, since the DFOV (spatial resolution) of the medical image and the specific DFOV (specific spatial resolution) trained by the deep learning neural network are known, and furthermore, the DFOV (spatial resolution) of the medical image may have worse granularity than the specific DFOV (specific spatial resolution) trained by the deep learning neural network, the pre-execution resample component can iteratively or stepwise upsample the medical image until the DFOV (spatial resolution) of the image obtained by resampling the medical image matches the specific DFOV (specific spatial resolution) trained by the deep learning neural network (e.g., within an appropriate threshold margin of the DFOV (spatial resolution)).

[0034] As another non-limiting example, assume that the DFOV (spatial resolution) of a medical image has better granularity than the DFOV (spatial resolution) trained by a deep learning neural network. In such a case, the pre-execution resample component can apply any suitable downsampling technique (e.g., box sampling, mipmapping) to the medical image in various manners. In various manners, such downsampling techniques can be considered to reduce the number of pixels / voxels of the medical image. In other words, such downsampling techniques can be considered to represent the medical image using a smaller number of pixels / voxels. Therefore, in such a case, each pixel / voxel of the image obtained by resampling the medical image can be considered to represent a larger, and thus less granular, physical area / volume than each pixel / voxel of the medical image (e.g., of the original medical image / of the medical image without any changes). Thus, it can be considered that downsampling the medical image worsens the granularity of the DFOV (spatial resolution) of the medical image. In various manners, since the DFOV (spatial resolution) of the medical image and the specific DFOV (specific spatial resolution) trained by the deep learning neural network are known, and furthermore, the DFOV (spatial resolution) of the medical image may be less granular than the specific DFOV (specific spatial resolution) trained by the deep learning neural network, the pre-execution resample component can iteratively or stepwise downsample the medical image until the DFOV (spatial resolution) of the image obtained by resampling the medical image matches the specific DFOV (specific spatial resolution) trained by the deep learning neural network (e.g., within an appropriate threshold margin of the DFOV (spatial resolution)).

[0035] In various embodiments, the execution component of the computer tool can electronically execute a deep learning neural network on an image obtained by resampling a medical image, rather than on the medical image itself. In various aspects, the execution can cause the deep learning neural network to generate an output image. More specifically, in various embodiments, the execution component supplies an image obtained by resampling a medical image to the input layer of the deep learning neural network, and the upsampled medical image is processed through a forward pass through one or more hidden layers of the deep learning neural network, and the output layer of the deep learning neural network can calculate an output image based on the activations provided by the one or more hidden layers.

[0036] Note that the deep learning neural network can be made to not include dense layers (e.g., can be made to not include fully connected layers), so that the deep learning neural network can be executed on inputs of any size (e.g., dense layers can be configured to operate on inputs of a fixed size, whereas convolutional layers and non-linear layers can be applied to inputs regardless of size). Thus, the deep learning neural network can operate on an image obtained by resampling a medical image, regardless of changes in the input size caused by the pre-execution resampling component.

[0037] In various aspects, the output image can correspond to an inference task configured to be performed by a deep learning neural network. For example, when the inference task is image quality improvement, the output image can be considered to have improved image quality of an image obtained by resampling a medical image. As another example, when the inference task is image noise removal, the output image can be considered to be an image with noise removed from an image obtained by resampling a medical image. As yet another example, when the inference task is image kernel conversion, the output image can be considered to be an image obtained by resampling a medical image that has been kernel-converted. As yet another example, when the inference task is image segmentation, the output image can be considered to be a segmentation mask of an image obtained by resampling a medical image.

[0038] In any case, since the image obtained by resampling a medical image can indicate a specific DFOV (specific spatial resolution) trained by a deep learning neural network, the output image can similarly indicate a specific DFOV (specific spatial resolution) trained by a deep learning neural network. In other words, the pixels / voxels of the output image can represent a physical region / volume of the same size as the physical region / volume represented by the pixels / voxels of the image obtained by resampling the medical image.

[0039] In various embodiments, the post-execution resampling component of the computer tool can electronically generate an image obtained by resampling the output image, and the image obtained by resampling the output image can indicate the DFOV (spatial resolution) of the medical image (e.g., of the original medical image / the medical image with nothing changed). More specifically, the post-execution resampling component can be any suitable upsampling technique or downsampling technique, and apply to the output image a technique that is the reverse of the upsampling technique or downsampling technique applied by the pre-execution resampling component. For example, if the pre-execution resampling component applies an upsampling technique to the medical image, the post-execution resampling component can apply a downsampling technique to the output image. In such a case, the post-execution resampling component can be considered to perform (e.g., reverse) the reverse operation of the upsampling performed by the pre-execution resampling component (e.g., the pre-execution resampling component can increase the number of pixels / voxels of the medical image by upsampling, while the post-execution resampling component can decrease the number of pixels / voxels of the output image by downsampling). As another example, if the pre-execution resampling component applies a downsampling technique to the medical image, the post-execution resampling component can apply an upsampling technique to the output image. In this case, the post-execution resampling component can be considered to perform (e.g., reverse) the reverse operation of the downsampling performed by the pre-execution resampling component (e.g., the pre-execution resampling component can decrease the number of pixels / voxels of the medical image by downsampling, while the post-execution resampling component can increase the number of pixels / voxels of the output image by upsampling).In any case, the post-execution resampling component can resample the output image so that the image obtained by resampling the output image represents the DFOV (spatial resolution) of the medical image, rather than a specific DFOV (specific spatial resolution) trained by a deep learning neural network.

[0040] In various aspects, the DFOV (spatial resolution) of the output image can match (be within an appropriate threshold margin of) the DFOV (spatial resolution) of the medical image, so the image obtained by resampling the output image can be considered the result obtained when the inference task is applied to the medical image. However, since the image obtained by resampling the output image can be obtained without directly running the deep learning neural network on the medical image, the image obtained by resampling the output image can be made free from the inaccuracy / deterioration of accuracy that would otherwise occur because the DFOV does not match (the spatial resolution does not match) between the deep learning neural network and the medical image. Therefore, the computing tool described herein can be considered to make the deep learning neural network robust or agnostic to differences in DFOV (differences in spatial resolution).

[0041] In various embodiments, a result component of a computer tool can electronically initiate or assist any suitable electronic operation based on an image obtained by resampling an output image. For example, in some cases, the result component can electronically transmit an image obtained by resampling the output image to any suitable computing device so as to notify a technician of the image obtained by resampling the output image. As another example, in some cases, the result component can electronically display an image obtained by resampling the output image on any suitable computing display, screen, or monitor so that a technician can visually inspect the image obtained by resampling the output image.

[0042] Accordingly, the various embodiments described herein can be considered as computer tools that can assist the robustness of deep learning against differences in DFOV (against differences in spatial resolution).

[0043] Using the various embodiments described herein, (e.g., to assist the robustness of deep learning against differences in DFOV / spatial resolution), which are essentially highly technical problems that are not abstract and cannot be performed as a series of mental activities by humans, hardware or software can be used to solve the problems. Further, some of the processes executed can be performed by a dedicated computer (e.g., a deep learning neural network having internal parameters such as convolutional kernels) for performing defined tasks related to the robustness of deep learning against differences in DFOV. For example, such defined tasks may include accessing a deep learning neural network and a medical image by a device operably coupled to a processor, where the first display field of view (first spatial resolution) trained by the deep learning neural network does not match the second display field of view (second spatial resolution) shown by the medical image, accessing the deep learning neural network and the medical image, and executing the deep learning neural network on an image obtained by resampling the medical image by the device, where the image obtained by resampling the medical image shows the first display field of view (first spatial resolution) trained by the deep learning neural network. In various aspects, such defined tasks may include upsampling the medical image by the device, thereby generating an image obtained by resampling the medical image, and executing the deep learning neural network on the image obtained by resampling the medical image, whereby the deep learning neural network generates a first output image, and the first output image shows the first display field of view (first spatial resolution), upsampling the medical image, and further may include downsampling the first output image by the device, thereby obtaining a second output image showing the second display field of view (second spatial resolution), downsampling the first output image.

[0044] Such defined tasks are not manually executed by humans. In fact, even if a human thinks or holds a pen and paper, electronically accessing a trained deep learning neural network and medical images (2D pixel arrays, 3D voxel arrays), electronically resampling the medical images, and matching the DFOV (spatial resolution) of the resampled medical images to the DFOV (spatial resolution) trained by the deep learning neural network, electronically executing the deep learning neural network on the image obtained by resampling the medical images to generate an output image, and electronically resampling the output image to match the DFOV (spatial resolution) of the output image obtained by resampling to the DFOV (spatial resolution) of the medical image cannot be done. Instead, various embodiments described herein are essentially and closely tied to computer technology and cannot be implemented outside of a computer environment. In fact, a deep learning neural network is essentially a computerized configuration and cannot be implemented at all by a human thinking in their head without a computer. Furthermore, resampling (e.g., upsampling or downsampling) a pixel / voxel array is also an essentially computerized operation and cannot be implemented by a human thinking in their head without a computer. Therefore, a computer tool that resamples a medical image based on the scale of DFOV (spatial resolution), executes a deep learning neural network on the image obtained by resampling the medical image, and resamples the output of the deep learning neural network based on the scale of DFOV (spatial resolution) is also essentially computerized and cannot be implemented in a practical, useful, or reasonable way without a computer.

[0045] Furthermore, the various embodiments described herein can integrate various teachings related to the robustness of deep learning with respect to differences in DFOV into practical applications. As described above, when a medical image exhibits a DFOV (spatial resolution) different from a specific DFOV (specific spatial resolution) trained by a deep learning neural network, the deep learning neural network can be expected to analyze the medical image inaccurately / with degraded accuracy. The various embodiments described herein can address this technical problem. Specifically, the various embodiments described herein can resample (upsample or downsample) a medical image to match the DFOV (spatial resolution of the medical image) of the medical image obtained by resampling to a specific DFOV (specific spatial resolution) trained by a deep learning neural network; performing a deep learning neural network on the image obtained by resampling the medical image, thereby obtaining an output image having a DFOV (spatial resolution) that matches a specific DFOV (specific spatial resolution) trained by the deep learning neural network; and resampling the output image to match the DFOV (spatial resolution of the output image) of the output image obtained by resampling to the DFOV (spatial resolution) of the original medical image / non-resampled medical image. Since the image obtained by resampling the medical image can exhibit a DFOV (spatial resolution) that matches the DFOV (spatial resolution) trained by the deep learning neural network, the deep learning neural network can perform accurately or precisely on the image obtained by resampling the medical image. Furthermore, since the image obtained by resampling the output image can exhibit a DFOV (spatial resolution) that matches the DFOV (spatial resolution) of the medical image, the image obtained by resampling the output image can be considered the result that would have been obtained if the deep learning neural network had been performed accurately / precisely on the medical image.In this way, the inference tasks performed by the deep learning neural network can be accurately / precisely performed on medical images, even though the DFOV does not match (even though the spatial resolutions do not match) between the medical images and the deep learning neural network. That is, the various embodiments described herein can be considered to make the deep learning neural network robust or agnostic to differences in the DFOV (differences in spatial resolution) of the input medical images. Accordingly, the various embodiments described herein achieve specific and certain technical improvements in the field of deep learning. Accordingly, the various embodiments described herein are clearly suitable as useful and practical applications of computers.

[0046] Furthermore, the various embodiments described herein can control real-world tangible devices based on the disclosed teachings. For example, the various embodiments described herein can electronically resample real-world medical images generated by real-world medical imaging devices (e.g., CT scanners, MRI scanners, X-ray scanners, PET scanners, ultrasound scanners), electronically execute a real-world deep learning neural network on the images obtained by resampling such medical images, electronically resample the results output by such a real-world deep learning neural network, and electronically display the results of such resampling on a real-world computer screen.

[0047] It should be understood that the drawings and descriptions in this specification provide non-limiting examples of various embodiments and are not necessarily illustrated enlarged or reduced on the same scale.

[0048] FIG. 1 shows a block diagram of an exemplary and non - limiting system 100 that can assist in the robustness of deep learning against differences in DFOV according to one or more embodiments described herein. As shown, in various embodiments, a display field - of - view robustness system 102 (hereinafter, "DFOV robustness system 102") is electronically coupled to a deep learning neural network 104 or a medical image 108 through any suitable wired or wireless electronic connection.

[0049] In various embodiments, the medical image 108 can be suitable image data representing any suitable anatomical structure of any suitable patient. As some non - limiting examples, the anatomical structure can be any suitable tissue of the patient (e.g., bone tissue, lung tissue, muscle tissue), any suitable organ of the patient (e.g., heart, liver, lung, brain), any suitable body fluid of the patient (e.g., blood, amniotic fluid), any other suitable body part of the patient, or any suitable portion thereof.

[0050] In various aspects, the medical image 108 can have any suitable format or dimension. As non - limiting examples, the medical image 108 can be an array of pixels of Hounsfield unit values of x×y (x and y are any suitable positive integers). As another non - limiting example, the medical image 108 can be an array of voxels of Hounsfield unit values of x×y×z (x, y, and z are any suitable positive integers).

[0051] In various examples, the medical image 108 can be obtained by or generated in other ways by any suitable medical imaging device (not shown). As non-limiting examples, the medical image 108 can be obtained by or generated in other ways by a CT scanner, in which case the medical image 108 can be considered as an image obtained by a CT scan. As another non-limiting example, the medical image 108 can be obtained by or generated in other ways by an MRI scanner, in which case the medical image 108 can be considered as an image obtained by an MRI scan. As yet another non-limiting example, the medical image 108 can be obtained by or generated in other ways by a PET scanner, in which case the medical image 108 can be considered as an image obtained by a PET scan. As yet another non-limiting example, the medical image 108 can be obtained by or generated in other ways by an X-ray scanner, in which case the medical image 108 can be considered as an image obtained by an X-ray scan. As yet another non-limiting example, the medical image 108 can be obtained by or generated in other ways by an ultrasound scanner, in which case the medical image 108 can be considered as an image obtained by an ultrasound scan. As another non-limiting example, the medical image 108 can be obtained by or generated in other ways by a visible spectrum camera, in which case the medical image 108 can be regarded as an image taken in the visible spectrum. In various aspects, any suitable image reconstruction technique may be applied to the medical image 108 or an image reconstruction technique may already be applied.

[0052] In any case, the medical image 108 can indicate a display field of view 110 (hereinafter, "DFOV 110") (for example, the medical image 108 may have been acquired / generated according to the DFOV 110). In various aspects, the DFOV 110 can have any suitable value. In various examples, the DFOV 110 can result in or correspond to a spatial resolution 128. In various aspects, the DFOV 110 and the spatial resolution 128 can be considered as interchangeable characteristics, features, or attributes of the medical image 108 that directly or indirectly indicate how much physical area / volume is represented by each pixel / voxel of the medical image 108. That is, both the DFOV 110 and the spatial resolution 128 can be considered as measures of the size of the physical pixel / voxel.

[0053] In various embodiments, the deep learning neural network 104 can represent any suitable deep learning architecture. Thus, the deep learning neural network 104 can have any suitable number of layers of any suitable type. As some non-limiting examples, the deep learning neural network 104 can include any suitable convolutional layer (e.g., whose internal parameters are convolutional kernels) or any suitable non-linearity layer (e.g., using any suitable non-linear activation function such as sigmoid, softmax, hyperbolic tangent, or rectified linear unit) that can be arranged in any suitable manner or order. Regardless of the number or type of layers of the deep learning neural network 104, the deep learning neural network 104 can be considered to include an input layer, one or more hidden layers, and an output layer. Further, the deep learning neural network 104 can have any suitable number of neurons in various layers. For example, different layers of the deep learning neural network 104 can have the same number of neurons or different numbers of neurons from each other. Further, the deep learning neural network 104 can have any suitable activation function in various neurons. That is, in various cases, different neurons of the deep learning neural network 104 can have the same activation function or different activation functions from each other. Further, the deep learning neural network 104 can have any suitable neuron connection or neuron connection pattern. As some non-limiting examples, the deep learning neural network 104 can have any suitable forward connection, any suitable recurrent connection, or any suitable skip connection, and those connections can be provided in any suitable manner or order. In various examples, the deep learning neural network 104 can exclude or omit a dense layer (e.g., a fully connected layer). In such cases, the deep learning neural network 104 can be considered executable for various sizes of inputs (e.g., convolutional kernels and non-linear activation functions can be applied regardless of the input size, while dense layers can only be applied to fixed-size inputs).

[0054] In various aspects, the deep learning neural network 104 can be configured to perform any suitable inference task on the input medical image. As some non-limiting examples, the inference tasks can include improving image quality, removing image noise, converting the image kernel, or performing image segmentation. In any case, the deep learning neural network 104 can be configured to receive a medical image as input and generate an output corresponding to the inference task. As a non-limiting example, when the inference task is image quality improvement, the output generated by the deep learning neural network 104 can be considered as an image with improved quality of the input medical image (e.g., an image representing the same anatomical structure of the same patient as the input medical image, but with improved visual quality of the anatomical structure). As another non-limiting example, when the inference task is image noise removal, the output generated by the deep learning neural network 104 can be considered as an image with removed noise from the input medical image (e.g., an image representing the same anatomical structure of the same patient as the input medical image, but with reduced visual noise / blurring of the anatomical structure). As yet another non-limiting example, when the inference task is image kernel conversion, the output generated by the deep learning neural network 104 can be considered as an image with converted kernel of the input medical image (e.g., an image representing the same anatomical structure of the same patient as the input medical image, but representing the anatomical structure according to a different image kernel). As yet another non-limiting example, when the inference task is image segmentation, the output generated by the deep learning neural network 104 can be considered as a segmentation mask of the input medical image (e.g., a pixel-by-pixel or voxel-by-voxel mask indicating to which of a plurality of classes each pixel or voxel of the input medical image belongs).

[0055] In various aspects, to facilitate performing an inference task on an input medical image, the deep learning neural network 104 can be trained according to any suitable type of training technique. As a non-limiting example, the deep learning neural network 104 can receive supervised training based on an annotated training dataset, which can include a set of training medical images and corresponding annotations for each image. In that case, the internal parameters of the deep learning neural network 104 (e.g., elements of the convolutional kernels) can be initialized by any suitable method (e.g., random initialization). In various aspects, any suitable training medical image can be selected from the annotated training dataset. Further, any suitable annotation corresponding to the selected training medical image can be selected from the annotated training dataset. In various embodiments, the deep learning neural network 104 can be executed on the selected training medical image. That is, the selected training medical image can pass through the layers constituting the deep learning neural network 104 in the forward direction. In any case, such an execution can cause the deep learning neural network 104 to generate some output corresponding to the inference task (e.g., generate an image inferred to improve the quality of the selected training medical image, generate an image inferred to remove noise from the selected training medical image, generate an image inferred to transform the kernel of the selected training medical image, generate an inferred segmentation mask of the selected training medical image). In various examples, an error / loss between the generated output and the selected annotation can be calculated, and the internal parameters of the deep learning neural network 104 can be updated by backpropagation. Backpropagation can be performed by the calculated error / loss.In various aspects, such training procedures can be repeated for each training medical image in the annotated training dataset, whereby the internal parameters of the deep learning neural network 104 are iteratively optimized to perform an inference task on the input medical image. In various cases, any suitable training batch size, any suitable training termination criterion, or any suitable error / loss function can be implemented during training.

[0056] In some other non-limiting examples, the deep learning neural network 104 can undergo unsupervised training or reinforcement learning.

[0057] In any case, the deep learning neural network 104 can be trained to perform an inference task on the input medical image. In some cases, the DFOV robustness system 102 can perform such training on the deep learning neural network 104. In other cases, the deep learning neural network 104 can be trained by any other suitable computing device (not shown).

[0058] Regardless of the type of training that the deep learning neural network 104 undergoes, such training can include running the deep learning neural network 104 against a set of training medical images (not shown). Due to practical problems in acquiring / organizing data, all of such a set of training medical images may show the same DFOV with each other, and thus the same spatial resolution with each other. That is, the DFOV and the spatial resolution may be uniform across the set of training medical images. In other words, the physical area / volume of each pixel / voxel of each training medical image may represent the same size with each other. In various cases, such a DFOV can be referred to as the display field of view 106 (hereinafter, "DFOV 106"), and such a spatial resolution can be referred to as the spatial resolution 126. Therefore, it can be considered that the deep learning neural network 104 is being trained or the training is being continued at the DFOV 106 or the spatial resolution 126.

[0059] Similar to the above, in various aspects, the DFOV 106 can have any suitable value, and the spatial resolution 126 can be given by the DFOV 106, or otherwise the DFOV 106 can correspond to the spatial resolution 126. In various aspects, the DFOV 106 and the spatial resolution 126 can be regarded as interchangeable characteristics, features, or attributes of the medical images trained by the deep learning neural network 104, and these characteristics, features, or attributes directly or indirectly indicate how much physical area / volume is represented by each pixel / voxel of those training medical images. That is, both the DFOV 106 and the spatial resolution 126 can be considered as measures of the size of the physical pixel / voxel.

[0060] In various aspects, DFOV110 (spatial resolution 128) is different from DFOV106 (spatial resolution 126). That is, the pixels / voxels of the medical image 108 can represent a physical area / volume of a different size than the pixels / voxels of the medical image that the deep learning neural network 104 was trained on. In such a case, when the deep learning neural network 104 is directly executed on the medical image 108, the deep learning neural network 104 does not generate an output that is sufficiently accurate. That is, due to the mismatch between DFOV106 and DFOV110 (due to the mismatch between spatial resolution 126 and spatial resolution 128), the deep learning neural network 104 cannot accurately perform an inference task on the medical image 108 (e.g., the deep learning neural network 104 can be executed on the medical image 108, but the results generated by that execution may be inaccurate).

[0061] In various aspects, the DFOV robustness system 102 can address this technical problem, as described herein.

[0062] In various embodiments, the DFOV robustness system 102 can include a processor 112 (e.g., a computer processing unit, a microprocessor) and a non-transitory computer-readable memory 114 that is operably, usable, or communicably connected / coupled to the processor 112. The non-transitory computer-readable memory 114 can store computer-executable instructions. When executed by the processor 112, these computer-executable instructions can cause one or more operations to be performed on the processor 112 or other components of the DFOV robustness system 102 (e.g., access component 116, pre-execution resample component 118, execution component 120, post-execution resample component 122, result component 124). In various embodiments, the non-transitory computer-readable memory 114 can store computer-executable components (e.g., access component 116, pre-execution resample component 118, execution component 120, post-execution resample component 122, result component 124), and the processor 112 can execute the computer-executable components.

[0063] In various embodiments, the DFOV robustness system 102 can include an access component 116. In various aspects, the access component 116 can electronically receive or electronically access the deep learning neural network 104 or the medical image. In various embodiments, the access component 116 can electronically retrieve the deep learning neural network 104 or the medical image 108 from any suitable centralized or distributed data structure (not shown) or from any suitable centralized or distributed computing device (not shown). In any case, the access component 116 can electronically obtain or electronically access the deep learning neural network 104 or the medical image 108 so that other components of the DFOV robustness system 102 can electronically communicate with the deep learning neural network 104 or the medical image 108.

[0064] In various embodiments, the DFOV robustness system 102 can further include a pre-execution resample component 118. In various aspects, as described herein, the pre-execution resample component 118 can electronically generate an image obtained by resampling the medical image 108, and the image obtained by resampling can represent DFOV 106 (spatial resolution 126) instead of DFOV 110 (spatial resolution 128).

[0065] In various embodiments, the DFOV robustness system 102 can further include an execution component 120. In various embodiments, as described herein, the execution component 120 can electronically execute the deep learning neural network 104 on an image obtained by resampling the medical image 108 rather than on the medical image 108 itself. By this execution, the deep learning neural network 104 can generate an output image, which can indicate the DFOV 106 (spatial resolution 126) rather than the DFOV 110 (spatial resolution 128).

[0066] In various embodiments, the DFOV robustness system 102 can further include a post-execution resample component 122. In various cases, as described herein, the post-execution resample component 122 can electronically generate an image obtained by resampling the output image, and the image obtained by this resampling can indicate the DFOV 110 (spatial resolution 128) rather than the DFOV 106 (spatial resolution 126).

[0067] In various embodiments, the DFOV robustness system 102 can further include a result component 124. In various aspects, as described herein, the result component 124 can electronically transmit an image obtained by resampling the output image to any suitable computing device, or can electronically display an image obtained by resampling the output image on any suitable computer display.

[0068] FIG. 2 shows a block diagram of an exemplary non-limiting system 200 that includes a medical image obtained by resampling and can assist in the robustness of deep learning against differences in DFOV, according to one or more embodiments described herein. As shown, the system 200 can optionally include the same components as the system 100 and can further include a medical image 202 obtained by resampling.

[0069] In various embodiments, the pre-execution resample component 118 can electronically generate a medical image 202 obtained by resampling based on the medical image 108. Further, the medical image 202 obtained by resampling can indicate a DFOV 106 (spatial resolution 126) instead of a DFOV 110 (spatial resolution 128). This is further explained in FIG. 3.

[0070] FIG. 3 shows an exemplary and non-limiting block diagram 300 illustrating how a medical image 202 obtained by resampling can be generated in accordance with one or more embodiments described herein.

[0071] In various embodiments, as illustrated, the pre-execution resample component 118 can electronically resample the medical image 108, thereby generating a medical image 202 obtained by resampling.

[0072] In various aspects, assume that the DFOV110 (spatial resolution 128) has worse granularity than the DFOV106 (spatial resolution 126). In such a case, the pre-execution resample component 118 can electronically apply any suitable upsampling technique to the medical image 108. For example, the pre-execution resample component 118 can apply nearest neighbor interpolation to the medical image 108. As another example, the pre-execution resample component 118 can apply bilinear interpolation to the medical image 108. As yet another example, the pre-execution resample component 118 can apply cubic interpolation or bicubic interpolation to the medical image 108. In any case, by applying an upsampling technique to the medical image 108, the pre-execution resample component 118 can be considered to increase the number of pixels / voxels that make up the medical image 108. That is, the medical image 202 obtained by resampling can be considered to represent the same anatomical structure of the same patient as the medical image 108, but the medical image 202 obtained by resampling can be considered to represent this anatomical structure using a larger number of pixels / voxels than the medical image 108. Therefore, such upsampling can prevent the granularity of the DFOV (spatial resolution) of the medical image 108 from deteriorating or becoming coarser. That is, the DFOV (spatial resolution) of the medical image 202 obtained by resampling can be made higher resolution than the DFOV (spatial resolution) of the medical image 108.

[0073] As a non-limiting example, consider that the medical image 108 is an x×y pixel array (x and y are any suitable positive integers). In some embodiments, the pre-execution resample component 118 can upsample the medical image 108 by 50% to make the medical image 202 obtained by resampling an array of 1.5(x)×1.5(y) pixels (both 1.5(x) and 1.5(y) are positive integers). In such a case, each pixel of the medical image 202 obtained by resampling can be considered to represent a physical area having two-thirds the height and two-thirds the width of each pixel of the medical image 108. That is, the medical image 202 obtained by resampling can be considered to have two-thirds the DFOV of the medical image 108 (e.g., as having 50% more pixel / voxel spatial resolution than the medical image 108). In other embodiments, the pre-execution resample component 118 can upsample the medical image 108 by 100% to make the medical image 202 obtained by resampling an array of 2(x)×2(y) pixels. In such a case, each pixel of the medical image 202 obtained by resampling can be considered to represent a physical area having half the height and half the width of each pixel of the medical image 108. That is, the medical image 202 obtained by resampling can be considered to have one-half the DFOV of the medical image 108 (e.g., as having 100% more pixel / voxel spatial resolution than the medical image 108). In still other embodiments, the pre-execution resample component 118 can upsample the medical image 108 by 150% to make the medical image 202 obtained by resampling a 2.5(x)×2.5(y) pixel array (e.g., both 2.5(x) and 2.5(y) are positive integers). In such a case, each pixel of the medical image 202 obtained by resampling can be considered to represent a physical area having two-fifths the height and two-fifths the width of each pixel of the medical image 108. That is, the medical image 202 obtained by resampling can be considered to have two-fifths the DFOV of the medical image 108 (e.g., as having 150% more pixel / voxel spatial resolution than the medical image 108).In this way, it can be considered that the medical image 202 obtained by resampling has a better granularity DFOV (better spatial resolution in terms of granularity) than the medical image 108. It should be understood that the specific numbers provided in this embodiment (or other embodiments described herein) (for example, 50%, 100%, 150%) are non-limiting.

[0074] In various other aspects, it is assumed that the DFOV 110 (spatial resolution 128) has better granularity than the DFOV 106 (spatial resolution 126). In such a case, the pre-execution resample component 118 can electronically apply any suitable downsampling technique to the medical image 108. For example, the pre-execution resample component 118 can apply box sampling to the medical image 108. As another example, the pre-execution resample component 118 can apply the mipmap technique to the medical image 108. In any case, by applying the downsampling technique to the medical image 108, it can be considered that the pre-execution resample component 118 reduces the number of pixels / voxels that make up the medical image 108. That is, it can be considered that the medical image 202 obtained by resampling represents the same anatomical structure of the same patient as the medical image 108, but the medical image 202 obtained by resampling represents this anatomical structure using a smaller number of pixels / voxels than the medical image 108. Therefore, such downsampling causes the granularity of the DFOV (spatial resolution) of the medical image 108 to deteriorate or become coarser. That is, the DFOV (spatial resolution) of the medical image 202 obtained by resampling is coarser than the DFOV (spatial resolution) of the medical image 108.

[0075] As a non-limiting example, consider that the medical image 108 is an x×y pixel array (x and y are any suitable positive integers). In some embodiments, the pre-execution resample component 118 can downsample the medical image 108 by 25% to make the medical image 202 obtained by resampling an array of 0.75(x)×0.75(y) pixels (both 0.75(x) and 0.75(y) are positive integers). In such a case, each pixel of the medical image 202 obtained by resampling can be considered to represent a physical area having 3 / 4 the height and 3 / 4 the width of each pixel of the medical image 108. That is, the medical image 202 obtained by resampling can be considered to have 3 / 4 the DFOV of the medical image 108 (e.g., as having 25% lower pixel / voxel spatial resolution than the medical image 108). In other embodiments, the pre-execution resample component 118 can downsample the medical image 108 by 50% to make the medical image 202 obtained by resampling an array of 0.5(x)×0.5(y) pixels (both 0.5(x) and 0.5(y) are positive integers). In such a case, each pixel of the medical image 202 obtained by resampling can be considered to represent a physical area having twice the height and twice the width of each pixel of the medical image 108. That is, the medical image 202 obtained by resampling can be considered to have twice the DFOV of the medical image 108 (as having 50% lower pixel / voxel spatial resolution than the medical image 108). In this way, the medical image 202 obtained by resampling can be considered to have a DFOV (spatial resolution with worse granularity) with worse granularity than the medical image 108. It should be understood that the specific numbers provided in this example (or other examples described herein), such as 25%, 50%, are non-limiting.

[0076] In any case, through resampling, the medical image 202 obtained by resampling can have a different DFOV (different spatial resolution) from the medical image 108. Therefore, since DFOV 110 (spatial resolution 128) and DFOV 106 (spatial resolution 126) are known, the pre-execution resample component 118 can mathematically determine the resampling size (e.g., the size of upsampling or downsampling) sufficient for the medical image 202 obtained by resampling to exhibit DFOV 106 (spatial resolution 126) instead of DFOV 110 (spatial resolution 128), and the pre-execution resample component 118 can apply such a resampling size to the medical image 108, thereby generating the medical image 202 obtained by resampling.

[0077] FIG. 4 shows a block diagram of an exemplary non-limiting system 400 that includes an output image and can assist in the robustness of deep learning against differences in the display field of view, according to one or more embodiments described herein. As shown, system 400 can optionally include the same components as system 200 and can further include an output image 402.

[0078] In various embodiments, the execution component 120 can electronically generate the output image 402 based on the medical image 202 obtained by resampling. Further, the output image 402 can exhibit DFOV 106 (spatial resolution 126) instead of DFOV 110 (spatial resolution 128). This is further illustrated in FIG. 5.

[0079] FIG. 5 is an exemplary non-limiting block diagram 500 showing how the output image 402 is generated by one or more embodiments described herein.

[0080] In various embodiments, as illustrated, execution component 120 can electronically execute deep learning neural network 104 on medical image 202 obtained by resampling. More specifically, in various aspects, execution component 120 can supply medical image 202 obtained by resampling to the input layer of deep learning neural network 104. In various embodiments, medical image 202 obtained by resampling can pass through one or more hidden layers of deep learning neural network 104 in the forward direction, thereby generating various activation maps. In various cases, the output layer of deep learning neural network 104 can calculate output image 402 based on the activation maps generated by one or more hidden layers.

[0081] Note that deep learning neural network 104 can include convolutional layers or non-linear layers, but not fully connected layers, so that deep learning neural network 104 can be made not to be restricted by the input size. After all, fully connected layers can only be applied to inputs with a fixed size, while convolutional layers and non-linear layers can be applied to inputs regardless of size. Therefore, since deep learning neural network 104 can include convolutional layers or non-linear layers and not include fully connected layers, deep learning neural network 104 can be executed on medical image 202 obtained by resampling regardless of the size of medical image 202 obtained by resampling (e.g., regardless of the number of pixels / voxels of medical image 202 obtained by resampling).

[0082] In various aspects, the output image 402 can correspond to an inference task configured to be executed by the deep learning neural network 104. More specifically, the output image 402 can be considered as the result obtained when an inference task is executed on the medical image 202 obtained by resampling. As a non-limiting example, when the inference task is image quality improvement, the output image 402 can be considered as the one in which the image quality of the medical image 202 obtained by resampling is inferred and improved. As another non-limiting example, when the inference task is image noise removal, the output image 402 can be considered as the image in which the noise removal of the medical image 202 obtained by resampling is inferred. As yet another non-limiting example, when the inference task is image kernel conversion, the output image 402 can be considered as the image in which the kernel conversion of the medical image 202 obtained by resampling is inferred. As yet another non-limiting example, when the inference task is image segmentation, the output image 402 can be considered as the inferred segmentation mask of the medical image 202 obtained by resampling.

[0083] In various aspects, the output image 402 can indicate the same dimensions (e.g., the same number or the same array of pixels / voxels) as the medical image 202 obtained by resampling. Thus, the output image 402 can also indicate the same DFOV (the same spatial resolution) as the medical image 202 obtained by resampling. In other words, since the medical image 202 obtained by resampling can indicate the DFOV 106 (spatial resolution 126), the output image 402 can also indicate the DFOV 106 (spatial resolution 126). That is, the physical area / volume represented by each pixel / voxel of the output image 402 can be made equal to the area / volume represented by each pixel / voxel of the medical image 202 obtained by resampling.

[0084] FIG. 6 shows a block diagram of an exemplary non-limiting system 600 that can assist in the robustness of deep learning against differences in display fields of view, including an output image obtained by resampling, in accordance with one or more embodiments described herein. As shown, system 600 can optionally include the same components as system 400 and can further include an output image 602 obtained by resampling.

[0085] In various embodiments, the post-execution resample component 122 can electronically generate an output image 602 obtained by resampling based on the output image 402. Further, the output image 602 obtained by resampling can show a DFOV 110 (spatial resolution 128) instead of a DFOV 106 (spatial resolution 126). This is further explained in FIG. 7.

[0086] FIG. 7 shows an exemplary non-limiting block diagram showing how an output image 602 obtained by resampling can be generated in accordance with one or more embodiments described herein.

[0087] In various embodiments, as illustrated, the post-execution resample component 122 can electronically resample the output image 402, thereby generating a medical image 202 obtained by resampling. In particular, the post-execution resample component 122 can electronically apply any suitable resampling technique to the output image 402, and such a resampling technique can be considered as a technique opposite to the one applied by the pre-execution resample component 118. As a non-limiting example, when the pre-execution resample component 118 applies upsampling (e.g., nearest neighbor interpolation, bilinear interpolation, cubic interpolation, or bicubic interpolation) to the medical image 108, the post-execution resample component 122 can apply downsampling (e.g., box sampling, mipmapping) to the output image 402. In this case, the pre-execution resample component 118 can be considered to increase the number of pixels / voxels in the medical image 108, while the post-execution resample component 122 can be considered to decrease the number of pixels / voxels in the output image 402. As another non-limiting example, when the pre-execution resample component 118 applies downsampling (e.g., box sampling, mipmapping) to the medical image 108, the post-execution resample component 122 can apply upsampling (e.g., nearest neighbor interpolation, bilinear interpolation, cubic interpolation, or bicubic interpolation) to the output image 402. In this case, the pre-execution resample component 118 can be considered to decrease the number of pixels / voxels in the medical image 108, while the post-execution resample component 122 can be considered to increase the number of pixels / voxels in the output image 402. In this way, the post-execution resample component 122 can resample the output image 402 such that the output image 602 obtained by resampling shows DFOV 110 (spatial resolution 128) instead of DFOV 106 (spatial resolution 126).That is, since DFOV110 (spatial resolution 128) and DFOV106 (spatial resolution 126) are known, the post-execution resample component 122 can mathematically determine the resampling size (e.g., the size of upsampling or downsampling) sufficient for the output image 602 obtained by resampling to show DFOV110 (spatial resolution 128) instead of DFOV106 (spatial resolution 126). The post-execution resample component 122 can apply such a resampling size to the output image 402, thereby generating the output image 602 obtained by resampling.

[0088] In various aspects, the output image 602 obtained by resampling can be considered as the result obtained when the inference task is accurately executed on the medical image 108. As a non-limiting example, when the inference task is image quality improvement, the output image 602 obtained by resampling can be considered as an image in which the image quality of the medical image 108 is inferred to be improved. As another non-limiting example, when the inference task is image noise removal, the output image 602 obtained by resampling can be considered as an image in which the noise removal of the medical image 202 is inferred. As yet another non-limiting example, when the inference task is image kernel conversion, the output image 602 obtained by resampling can be considered as an image in which the kernel conversion of the medical image 108 is inferred. As yet another non-limiting example, when the inference task is image segmentation, the output image 602 obtained by resampling can be considered as the inferred segmentation mask of the medical image 108.

[0089] In any case, the output image 602 obtained by resampling can reduce inaccuracies / artifacts related to the DFOV mismatch (related to the spatial resolution mismatch). In fact, as described above, when the deep learning neural network 104 is directly executed on the medical image 108, it is considered that an accurate inference result cannot be obtained due to the mismatch between the DFOV 110 (spatial resolution 128) and the DFOV 106 (spatial resolution 126). However, as described in this specification, this inaccuracy can be avoided / improved by resampling the medical image 108 (e.g., by component 118) to match the DFOV 106 (spatial resolution 126), executing the deep learning neural network 104 (e.g., by component 120) on the image obtained by resampling the medical image 108, and resampling the result generated by the deep learning neural network 104 (e.g., by component 122) back to the DFOV 110 (spatial resolution 128). In this way, the result of the inference task related to the medical image 108 (e.g., the output image 602) can be obtained, and such a result can be prevented from being deteriorated / not deteriorated by the inaccuracies / artifacts considered to be caused by the mismatch between the DFOV 110 and the DFOV 106 (by the mismatch between the spatial resolution 128 and the spatial resolution 126).

[0090] FIG. 8 shows an exemplary non - limiting block diagram 800. This block diagram 800 shows how a medical image showing a first display field of view (and thus a first spatial resolution) can be accurately analyzed by a deep learning neural network trained with a different display field of view (and thus a different spatial resolution) according to one or more embodiments described in this specification. In other words, FIG. 8 serves to clarify the various teachings of this specification.

[0091] In various embodiments, as illustrated, the medical image 108 can indicate or have a DFOV 110 (spatial resolution 128). As described above, the DFOV 110 (spatial resolution 128) is different from the DFOV 106 (spatial resolution 126) trained by the deep learning neural network 104 (e.g., less grainy or not coarser than the DFOV 106 (spatial resolution 126)).

[0092] In various aspects, as illustrated, the medical image 108 can be converted from the DFOV 110 (spatial resolution 128) to the DFOV 106 (spatial resolution 126) by resampling. Such resampling can generate a medical image 202 obtained by resampling. In some cases, it can be considered that such resampling imports the medical image 108 into the DFOV / resolution space of the deep learning neural network 104.

[0093] In various embodiments, as further illustrated, the deep learning neural network 104 can be executed on the medical image 202 obtained by resampling, thereby obtaining an output image 402. Since the medical image 202 obtained by resampling can have / show a DFOV 106 (spatial resolution 126), the output image 402 can also have / show a DFOV 106 (spatial resolution 126).

[0094] In various cases, as also illustrated, the output image 402 can be converted back from the DFOV 106 (spatial resolution 126) to the DFOV 110 (spatial resolution 128) by resampling. Such resampling can generate an output image 602 obtained by resampling. In some cases, it can be considered that such resampling imports the output image 402 into the DFOV / resolution space of the medical image 108 (e.g., such resampling can be considered the reverse of the resampling performed on the medical image 108).

[0095] As described above, the output image 402 can be considered as the result of applying an inference task (e.g., image quality improvement, image noise removal, image kernel conversion, image segmentation) to the medical image 202 obtained by resampling. In contrast, the output image 602 obtained by resampling can be considered as the result of applying the inference task to the medical image 108. The medical image 202 obtained by resampling can have / show a DFOV 106 (spatial resolution 126), and since the deep learning neural network 104 was trained with the DFOV 106 (spatial resolution 126), it should be noted that the output image 402 is not affected or degraded by inaccuracies or image artifacts related to the DFOV (spatial resolution) mismatch. Ultimately, it is possible to ensure that there is no DFOV mismatch (spatial resolution mismatch) between the medical image 202 obtained by resampling and the deep learning neural network 104. Therefore, since the output image 402 is not deteriorated / degraded by inaccuracies / artifacts related to the DFOV (spatial resolution) mismatch, the output image 602 obtained by resampling can also be made such that there are no inaccuracies / artifacts related to the DFOV (spatial resolution) mismatch, even though there is a DFOV (spatial resolution) mismatch between the medical image 108 and the deep learning neural network 104. In other words, by resampling the medical image 108 to match the DFOV 106 (spatial resolution 126) before running the deep learning neural network 104, the DFOV (spatial resolution) mismatch between the medical image 108 and the deep learning neural network 104 can be overcome (e.g., inaccuracies / artifacts caused by the DFOV / spatial resolution mismatch can be avoided or reduced).

[0096] In any case, the output image 602 obtained by resampling can be considered as the result obtained by applying the inference task to the medical image 108. In various embodiments, the result component 124 can perform or initiate any suitable electronic operation based on the output image 602 obtained by resampling. As a non-limiting example, the result component 124 can electronically transmit the output image 602 obtained by resampling (or a suitable portion of the output image 602 obtained by resampling) to a suitable computing device (not shown). As another non-limiting example, the result component 124 can electronically display the output image 602 obtained by resampling (or a suitable portion of the output image 602 obtained by resampling) on a suitable computing display (not shown).

[0097] FIG. 9 shows a flowchart of an exemplary and non-limiting method 900 implemented by a computer that can assist in the robustness of deep learning against differences in display fields of view according to one or more embodiments described herein. In various cases, the DFOV robustness system 102 can assist in the method 900 implemented by a computer.

[0098] In various embodiments, operation 902 includes accessing, by a device operably coupled to a processor (e.g., by access component 116), a deep learning neural network (e.g., deep learning neural network 104) trained at a first DFOV or spatial resolution (e.g., display FOV 106 or spatial resolution 126) and a medical image (e.g., medical image 108) indicative of a second DFOV or spatial resolution (e.g., display FOV 110 or spatial resolution 128).

[0099] In various aspects, operation 904 includes resampling a medical image by the device (e.g., by the pre-execution resample component 118), where the image obtained by resampling the medical image (e.g., 202) exhibits a first DFOV or spatial resolution (e.g., display FOV 106 or spatial resolution 126) and does not exhibit a second DFOV or spatial resolution (e.g., display FOV 110 or spatial resolution 128).

[0100] In various embodiments, operation 906 includes executing a deep learning neural network on an image obtained by resampling a medical image by the device (e.g., by the execution component 120). Thereby, an output image (e.g., output image 402) can be generated. The output image exhibits a first DFOV or spatial resolution (e.g., display FOV 106 or spatial resolution 126) and does not exhibit a second DFOV or spatial resolution (e.g., display FOV 110 or spatial resolution 128).

[0101] In various aspects, operation 908 includes resampling the output image by the device (e.g., by the post-execution resample component 122), where the image obtained by resampling the output image (e.g., output image 602 obtained by resampling) exhibits a second DFOV or spatial resolution (e.g., display FOV 110 or spatial resolution 128) and does not exhibit a first DFOV or spatial resolution (e.g., display FOV 106 or spatial resolution 126).

[0102] In various embodiments, operation 910 includes displaying, by the device (e.g., by the result component 124), an image obtained by resampling the output image on a computing display, or transmitting, by the device (e.g., by the result component 124), an image obtained by resampling the output image to a computing device.

[0103] Previously, embodiments have been described in which the medical image 108 is resampled so that it matches the DFOV (spatial resolution) trained by the deep learning neural network 104. However, as the DFOV 110 (spatial resolution 128) moves away from the DFOV 106 (spatial resolution 126), resampling the medical image 108, running the deep learning neural network 104 on the medical image 202 obtained by resampling, and resampling the output image 402 are accompanied by an increase in computational complexity (e.g., the number of pixels / voxels increases exponentially). It should be noted that in various embodiments, the increase in computational complexity can be improved by having a plurality of trained deep learning neural networks from which a trained deep learning neural network can be selected, and by selecting which trained deep learning neural network has a DFOV with good granularity and is closest to the DFOV of the medical image 108. Various embodiments in this regard are described in FIGS. 10 - 12.

[0104] FIG. 10 shows a block diagram of an exemplary non - limiting system 1000 that can include a selection component to assist in the robustness of deep learning to differences in the display field of view, according to one or more embodiments described herein. As shown, system 1000 can optionally include the same components as system 600 and can further include a selection component 1002.

[0105] In various embodiments, the selection component 1002 can electronically store, electronically maintain, or otherwise electronically access a deep learning neural network vault. In various aspects, the deep learning neural network vault can be considered as a collection of any suitable number of deep learning neural networks, each of which can be (trained to be) trained with a different DFOV (with a different spatial resolution). In various embodiments, before the pre-execution resample component 118 resamples the medical image 108, the selection component 1002 can electronically select the deep learning neural network 104 from the deep learning neural network vault based on the DFOV 110 (spatial resolution 128). This is further illustrated in FIG. 11.

[0106] FIG. 11 shows an exemplary non-limiting block diagram 1100 of a deep learning neural network vault 1102 according to one or more embodiments described herein.

[0107] In various embodiments, as shown, the deep learning neural network bolt 1102 can include a set 1104 of deep learning neural networks and a set 1106 of DFOV or spatial resolutions. In various aspects, the set 1104 of deep learning neural networks can include n networks (n being any suitable positive integer), i.e., deep learning neural network 1 to deep learning neural network n. In various embodiments, different deep learning neural networks of the set 1104 of deep learning neural networks can have the same or different architectures from each other. In any case, each deep learning neural network of the set 1104 of deep learning neural networks can be set / trained to perform the same inference task as the deep learning neural network 104. In fact, in various aspects, the deep learning neural network 104 is one of the set 1104 of deep learning neural networks.

[0108] In various aspects, as shown, the set 1106 of DFOV or spatial resolutions respectively corresponds to the set 1104 of deep learning neural networks (e.g., one-to-one). Thus, since the set 1104 of deep learning neural networks can have n networks, the set 1106 of DFOV or spatial resolutions can similarly have n DFOV or spatial resolutions, i.e., display field of view 1 (hereinafter, "DFOV1") and spatial resolution 1 to display field of view n (hereinafter, "DFOVn") and spatial resolution n. In various embodiments, different DFOV or spatial resolutions of the set 1106 of DFOV or spatial resolutions are different from each other. That is, each DFOV or spatial resolution of the set 1106 of DFOV or spatial resolutions can represent a unique pixel / voxel granularity.

[0109] In various aspects, each deep learning neural network of the set 1104 of deep learning neural networks can be (trained) trained at each DFOV or spatial resolution of the set 1106 of DFOVs or spatial resolutions. As a non-limiting example, deep learning neural network 1 can correspond to DFOV1 (spatial resolution 1), which means that deep learning neural network 1 was trained at DFOV1 (spatial resolution 1). As another non-limiting example, deep learning neural network n can correspond to DFOVn (spatial resolution n), which means that deep learning neural network n was trained at DFOVn (spatial resolution n). In various embodiments, as described above, deep learning neural network 104 can be within the set 1104 of deep learning neural networks. Thus, DFOV106 (spatial resolution 126) can be the one corresponding to deep learning neural network 104 among the set 1106 of DFOVs or spatial resolutions.

[0110] In various aspects, access component 116 can electronically receive, retrieve, or access medical image 108 indicating DFOV110 (spatial resolution 128). In various aspects, selection component 1002 can search for DFOV110 (spatial resolution 128) among the set 1106 of DFOVs or spatial resolutions. If DFOV110 (spatial resolution 128) is in the set 1106 of DFOVs or spatial resolutions (e.g., is an element of set 1106), selection component 1002 can select the one corresponding to DFOV110 (corresponding to spatial resolution 128) among the set 1104 of deep learning neural networks, and the selected deep learning neural network can be regarded as deep learning neural network 104. In such a case, DFOV110 (spatial resolution 128) can be regarded as equal to DFOV106 (spatial resolution 126), meaning that pre-execution resampling and post-execution resampling can be omitted.

[0111] In other cases, if the DFOV110 (spatial resolution 128) is not in the set 1106 of DFOVs or spatial resolutions (e.g., not an element of the set 1106), the selection component 1002 can select a DFOV or spatial resolution from the set 1106 of DFOVs or spatial resolutions that has better granularity than the DFOV110 (spatial resolution 128) and is closest in size to the DFOV110 (spatial resolution 128). In various cases, the selected DFOV (spatial resolution) can be regarded as the DFOV106 (spatial resolution 126), and the selection component 1002 can select, as the deep learning neural network 104, the deep learning neural network corresponding to the DFOV106 (spatial resolution 126) from the set of deep learning neural networks 1104. In this way, the selection component 1002 can identify, as the deep learning neural network 104, the deep learning neural network trained with a DFOV (spatial resolution) that has better granularity than the DFOV110 (spatial resolution 128) and is closest to the DFOV110 (spatial resolution 128) from the set of deep learning neural networks 1104. This helps reduce the computational complexity involved when resampling the medical image 108, when executing the deep learning neural network 104, or when resampling the output image 402.

[0112] FIG. 12 shows a flowchart of an exemplary and non - limiting computer - implemented method 1200 that can assist in the robustness of deep learning with respect to differences in display fields of view, according to one or more embodiments described herein. In various cases, the DFOV robustness system 102 can assist in the computer - implemented method 1200.

[0113] In various embodiments, operation 1202 includes accessing, by a device (e.g., the access component 116) operably coupled to a processor, a medical image (e.g., the medical image 108) indicating a given DFOV or a given spatial resolution (e.g., the display FOV110 or the spatial resolution 128).

[0114] In various aspects, operation 1204 includes accessing, by the device (e.g., by selection component 1002), a set of deep learning neural networks (e.g., 1104) each trained with a set of DFOV or spatial resolutions (e.g., 1106).

[0115] In various embodiments, operation 1206 includes determining, by the device (e.g., by selection component 1002), whether a given DFOV or a given spatial resolution exists in the set of DFOV or spatial resolutions (e.g., whether it is an explicitly specified element in the set). If it exists, the method 1200 implemented by the computer proceeds to operation 1208. If it does not exist, the method 1200 implemented by the computer can proceed to operation 1212.

[0116] In various cases, operation 1208 includes selecting, from the set of deep learning neural networks, a deep learning neural network trained with a given DFOV or a given spatial resolution (e.g., deep learning neural network 104) (e.g., in this case, DFOV 106 (spatial resolution 126) is equal to DFOV 110 (spatial resolution 128)).

[0117] In various aspects, operation 1210 includes executing, by the device (e.g., by execution component 120), the selected deep learning neural network (the deep learning neural network selected in operation 1208) on the medical image (e.g., if DFOV 110 (spatial resolution 128) is equal to DFOV 106 (spatial resolution 126), resampling of the medical image 108 before execution of the deep learning neural network 104 can be omitted).

[0118] In various embodiments, operation 1212 includes identifying, by the device (e.g., by selection component 1002), from a set of DFOVs or spatial resolutions, a DFOV or spatial resolution that has better granularity than a given DFOV or given spatial resolution and is closest to the given DFOV or given spatial resolution (e.g., DFOV 106 or spatial resolution 126) (e.g., in this case, DFOV 106 (spatial resolution 126) is not equal to DFOV 110 (spatial resolution 128)).

[0119] In various cases, operation 1214 includes selecting, by the device (e.g., by selection component 1002), from a set of deep learning neural networks, a deep learning neural network (e.g., deep learning neural network 104) trained with the identified DFOV (e.g., DFOV 106) or the identified spatial resolution (e.g., spatial resolution 126).

[0120] In various aspects, operation 1216 includes resampling, by the device (e.g., by pre-execution resample component 118), a medical image. The image obtained by resampling the medical image (e.g., medical image 202) exhibits the identified DFOV (e.g., DFOV 106) or the identified spatial resolution (e.g., spatial resolution 126), rather than a given DFOV (e.g., DFOV 110) or a given spatial resolution (e.g., spatial resolution 128).

[0121] In various embodiments, operation 1218 includes executing, by the device (e.g., by execution component 120), the selected deep learning neural network (the deep learning neural network selected in 1214) on the image obtained by resampling the medical image.

[0122] In various embodiments, downsampling the medical image 108 can be considered a lossy operation (e.g., information is lost because the number of pixels / voxels is reduced). Thus, in some aspects, it is desirable for the pre-execution resample component 118 to avoid downsampling the medical image 108. As described above, avoidance of downsampling can be achieved when the DFOV 106 (spatial resolution 126) has better granularity than the DFOV 110 (spatial resolution 128). Ultimately, in this case, the pre-execution resample component 118 can apply an upsampling technique (opposite to the downsampling technique) to the medical image 108 such that the medical image 202 obtained by resampling exhibits the DFOV 106 (spatial resolution 126). In situations where the DFOV 106 (spatial resolution 126) has worse granularity than the DFOV 110 (spatial resolution 128), avoidance of downsampling can be achieved by the selection component 1002 selecting a deep learning neural network trained at a DFOV with good granularity (good granularity spatial resolution) from the deep learning neural network bolt 1102.

[0123] However, in some cases, the DFOV 110 (spatial resolution 128) may have better granularity than all of the DFOVs or spatial resolutions of the set 1106 of DFOVs or spatial resolutions. In such cases, there may be a risk that the pre-execution resample component 118 cannot avoid applying a downsampling technique to the medical image 108. However, in various aspects as described with respect to FIGS. 13-15, when the DFOV or spatial resolution with the best granularity among the set 1106 of DFOVs or spatial resolutions is obtained from the maximum cutoff frequency of the modulation transfer function of the medical imaging device that acquired / generated the medical image 108, such information loss associated with downsampling can be reduced or eliminated.

[0124] Figures 13-14 show exemplary and non-limiting graphs regarding the maximum cut-off frequency of the modulation transfer function according to one or more embodiments described herein.

[0125] Consider Figure 13. As shown, Figure 13 shows graph 1302. In various aspects, graph 1302 can be considered to represent a non-limiting and exemplary modulation transfer function (MTF) of a medical imaging device that acquired / generated medical image 108. In various aspects, the horizontal axis of graph 1302 represents line pairs per millimeter (LPMM). LPMM can be regarded as the spatial scanning frequency of the medical imaging device. In various aspects, the vertical axis of graph 1302 represents the signal amplitude acquired / generated by the medical imaging device. In various aspects, a curve 1304 can be plotted on graph 1302, and such a curve can be considered to show the modulation transfer function of the medical imaging device (e.g., showing how the signal amplitude of the medical imaging device changes based on LPMM). As shown in the non-limiting example of Figure 13, curve 1304 can be considered to have a maximum cut-off frequency when LPMM is 2 (e.g., the signal amplitude is zero or near zero when LPMM exceeds 2).

[0126] Also as shown, FIG. 13 shows a graph 1306. In various aspects, graph 1306 can be considered to represent a non-limiting and exemplary relationship between DFOV (spatial resolution) and LPMM. In various aspects, the horizontal axis of graph 1306 represents LPMM, and the vertical axis of graph 1306 represents DFOV (spatial resolution). In particular, a curve 1308 can be plotted on graph 1306, and such a curve can be considered to indicate the DFOV (spatial resolution with the best granularity) that can achieve the best granularity at a given LPMM. Equivalently, curve 1308 can be considered to indicate the maximum allowable LPMM (also called the Nyquist frequency) for a given DFOV (a given spatial resolution). In various aspects, curve 1308 can be obtained mathematically. More specifically, the spatial resolution of a given dimension in a given DFOV can be obtained by dividing the given DFOV by the total number of pixels / voxels of the given dimension, the sampling frequency in a given DFOV can be obtained as the reciprocal of the spatial resolution in the given DFOV, and the Nyquist frequency (maximum LPMM) in a given DFOV is equal to half of the sampling frequency.

[0127] Next, consider FIG. 14. As shown, FIG. 14 shows a graph 1400 in which curve 1304 and curve 1308 are superimposed. As described above, curve 1304 can be considered to show that the MTF of the medical imaging device has a maximum cut-off frequency of 2 LPMM (in this non-limiting example). As indicated by reference numerals 1402 and 1404, in curve 1308, a DFOV of approximately 13 cm can be obtained for this maximum cut-off frequency (for example, when the LPMM is 2). In other words, a DFOV of 13 cm (spatial resolution) can be considered the DFOV (spatial resolution with the best granularity) supported by the medical imaging device when the MTF of the medical imaging device has a maximum cut-off frequency of 2. Stated more specifically, since 2 LPMM can be considered the maximum cut-off frequency of the MTF of the medical imaging device, the medical imaging device cannot reliably obtain information on a DFOV (spatial resolution) with better granularity than 13 cm. In other words, even if the medical imaging device attempts to obtain information on a DFOV (spatial resolution) with better granularity than 13 cm, it can be considered that no more useful information can be obtained than the information obtained at a DFOV of 13 cm. In such a non-limiting example, if DFOV 110 (spatial resolution 128) has better granularity than 13 cm, medical image 108 can be downsampled to a DFOV of 13 cm (spatial resolution) without loss of information. That is, downsampling in such a case can be considered an operation that does not cause loss. Therefore, in such a non-limiting example, when deep learning neural network bolt 1102 includes a deep learning neural network trained at a DFOV of 13 cm (spatial resolution), losses associated with downsampling can be avoided.

[0128] It should be understood that the specific numerical values presented above (or elsewhere in this specification), such as 13 cm and 2 of LPMM, are non-limiting numerical values.

[0129] More generally, in various aspects, the deep learning neural network bolt 1102 can include a deep learning neural network trained at a DFOV (spatial resolution) corresponding to the maximum cut-off frequency of the modulation transfer function of a medical imaging device that generates / acquires the medical image 108. As described above, this DFOV (spatial resolution) can be determined by empirically obtaining the modulation transfer function (MTF) of the medical imaging device that acquires / generates the medical image 108, identifying the maximum cut-off frequency of the MTF (e.g., identifying the LPMM value at which the MTF is band-limited), and calculating the DFOV (spatial resolution with the best granularity) that can support the best granularity at the maximum cut-off frequency based on the above-described Nyquist frequency calculation method. When the deep learning neural network bolt 1102 includes a deep learning neural network trained at a DFOV corresponding to this maximum cut-off frequency, information loss associated with downsampling can be avoided.

[0130] FIG. 15 shows a flowchart of an exemplary and non-limiting method 1500 implemented by a computer that can assist in the robustness of deep learning with respect to differences in the display field of view based on the maximum cut-off frequency of the modulation transfer function, according to one or more embodiments described herein. In various cases, the DFOV robustness system 102 can assist in the method 1500 implemented by a computer.

[0131] In various aspects, operation 1502 can include accessing, by a device operably coupled to a processor (e.g., by access component 116), a deep learning neural network (e.g., deep learning neural network 104) trained at a first DFOV or a first spatial resolution (e.g., DFOV 106 or spatial resolution 126). In various cases, the first DFOV or the first spatial resolution corresponds to the maximum cutoff frequency of the modulation transfer function of the medical imaging device (e.g., as shown in FIGS. 13 - 14). Since the first DFOV or the first spatial resolution corresponds to the maximum cutoff frequency of the modulation transfer function of the medical imaging device, the first DFOV or the first spatial resolution can be considered to be the DFOV with the best granularity or the spatial resolution with the best granularity that can be supported by the medical imaging device.

[0132] In various embodiments, operation 1504 can include accessing, by a device (e.g., by access component 116), a medical image (e.g., medical image 108) generated by a medical imaging device. In various embodiments, the medical image can indicate a second DFOV or a second spatial resolution.

[0133] In various aspects, operation 1506 can include determining, by a device (e.g., by pre - execution resample component 118), whether the second DFOV or the second spatial resolution has better granularity than the first DFOV or the first spatial resolution. If the granularity is good, the method 1500 implemented by the computer can proceed to operation 1508. If the granularity is not good, the method 1500 implemented by the computer can proceed to operation 1512.

[0134] In various aspects, operation 1508 can include downsampling a medical image by the device (e.g., by the pre-execution resample component 118), and the image obtained by downsampling the medical image (e.g., medical image 202) exhibits a first DFOV or a first spatial resolution. It should be noted that since the first DFOV or the first spatial resolution can correspond to the maximum cutoff frequency of the modulation transfer function of the medical imaging device, this downsampling is considered an operation without loss. In other words, although the second DFOV or the second spatial resolution is nominally grainier than the first DFOV or the first spatial resolution, the medical image does not actually contain information that is grainier than the information that would be obtained at the first DFOV or the first spatial resolution, and it can be considered that no information is lost when downsampling to the first DFOV or the first spatial resolution.

[0135] In various embodiments, operation 1510 can include executing a deep learning neural network on an image obtained by downsampling a medical image by the device (e.g., by the execution component 120).

[0136] In various aspects, operation 1512 includes upsampling a medical image by the device (e.g., by the pre-execution resample component 118), and the image obtained by upsampling the medical image (e.g., medical image 202) exhibits a first DFOV or a first spatial resolution.

[0137] In various embodiments, operation 1514 can include executing a deep learning neural network on an image obtained by upsampling a medical image by the device (e.g., by the execution component 120).

[0138] Figures 16 - 18 show exemplary and non-limiting experimental results demonstrating various advantages of one or more embodiments described herein.

[0139] Consider FIG. 16. FIG. 16 shows various CT images 1600 that are useful for showing the advantages of the various embodiments described herein. In particular, FIG. 16 shows a CT scan image 1602 of a patient's anatomical structure. The CT scan image 1602 is an image acquired / generated according to a soft tissue kernel. Thus, it may be desirable to perform an image kernel conversion on the CT scan image 1602 to obtain an image corresponding to the bone kernel of the CT scan image 1602. Although not explicitly shown in FIG. 16, the CT scan image 1602 is an image acquired / generated according to a 15 cm DFOV. It should be noted that the spatial resolution of the CT scan image 1602 in any given dimension is obtained by dividing 15 cm by the number of pixels arranged for the size of the given dimension.

[0140] In various aspects, FIG. 16 also shows a CT image 1604. In various embodiments, the CT image 1604 can be considered as an image obtained by performing a grand-to-truth kernel conversion on the CT scan image 1602. In various embodiments, the CT image 1604 is generated using an analytical bone kernel conversion technique.

[0141] In various aspects, FIG. 16 further shows CT image 1606 and CT image 1608. In various embodiments, a deep learning neural network has been trained to perform a bone kernel conversion, and based on the CT scan image 1602, both the CT image 1606 and the CT image 1608 have been generated by the deep learning neural network. However, the deep learning neural network was trained with a 10 cm DFOV that does not match the 15 cm DFOV shown by the CT scan image 1602. In various aspects, the deep learning neural network is executed directly on the CT scan image 1602, whereby the CT image 1606 is obtained. In various other aspects, the CT scan image 1602 is upsampled to match a 10 cm DFOV, the deep learning neural network is executed on the image obtained by upsampling the CT scan image, and the result output by the deep learning neural network is downsampled and returned to a 15 cm DFOV, whereby the CT image 1608 is obtained. As can be understood by looking at the figure, it can be considered that the CT image 1606 is over-processed. That is, it can be considered that the CT image 1606 is filled with many image artifacts caused by the mismatch between the DFOV of the CT scan image 1602 and the DFOV of the deep learning neural network. In stark contrast to this, it can be considered that the CT image 1608 is not over-processed. In fact, it is considered that the CT image 1608 quite well matches the CT image 1604 (for example, it can be considered that it quite well matches the ground truth). Therefore, even though the deep learning neural network was trained with a 10 cm DFOV and the CT scan image 1602 was acquired / generated according to a 15 cm DFOV, the deep learning neural network was able to execute accurately by the pre-execution and post-execution resampling operations based on the DFOV (based on the spatial resolution) described herein. Such experimental results are beneficial for demonstrating the technical advantages of the various embodiments described herein.

[0142] Here, consider FIG. 17. FIG. 17 shows various CT images 1700 that are also useful for demonstrating the advantages of the various embodiments described herein. Specifically, FIG. 17 shows a CT scan image 1702 of a patient's anatomical structure. Here too, the CT scan image 1702 was acquired according to a soft tissue kernel, and thus, bone kernel conversion is effective. Similar to FIG. 16, the CT scan image 1702 was acquired / generated according to a 15 cm DFOV.

[0143] In various aspects, FIG. 17 shows a CT image 1704 that can be considered a grand-to-truths kernel-converted image with respect to the CT scan image 1702. Here too, the CT image 1704 was generated using an analytical bone kernel conversion technique.

[0144] In various aspects, FIG. 17 further shows CT image 1706 and CT image 1708. As described above, the deep learning neural network was trained to perform bone kernel conversion, and based on the CT scan image 1702, both the CT image 1706 and the CT image 1708 were generated by the deep learning neural network. However, the deep learning neural network was trained with a 10 cm DFOV instead of a 15 cm DFOV. In various aspects, the deep learning neural network was executed directly on the CT scan image 1702, thereby obtaining the CT image 1706. In various other aspects, the CT scan image 1702 was upsampled to match a 10 cm DFOV, the deep learning neural network was executed on the image obtained by upsampling the CT scan image, and the result output by the deep learning neural network was downsampled and returned to a 15 cm DFOV, thereby obtaining the CT image 1708. As can be understood by looking at the figure, it can be considered that the CT image 1706 is overprocessed (e.g., filled with many image artifacts caused by the mismatch between the DFOV of the CT scan image 1702 and the DFOV of the deep learning neural network). In stark contrast to this, it can be considered that the CT image 1708 is not overprocessed. In fact, the CT image 1708 is considered to closely match the CT image 1704 (e.g., appears to closely match the ground truth). Therefore, even though the deep learning neural network was trained with a 10 cm DFOV and the CT scan image 1702 was acquired / generated according to a 15 cm DFOV, the deep learning neural network was able to execute accurately through the pre-execution and post-execution resampling operations based on the DFOV described herein (based on the spatial resolution). Again, such experimental results are beneficial for demonstrating the technical advantages of the various embodiments described herein.

[0145] Here, consider FIG. 18. FIG. 18 shows various CT images 1800 that are further useful for demonstrating the advantages of the various embodiments described herein. In particular, FIG. 18 shows a CT scan image 1802 of a patient's anatomical structure acquired according to a 10 cm DFOV. Also, FIG. 18 shows a CT scan image 1806 of the same anatomical structure of the same patient acquired according to a 15 cm DFOV.

[0146] The deep learning neural network was trained to perform image quality improvement using a 10 cm DFOV. In various aspects, the deep learning neural network was performed directly on the CT scan image 1802, thereby obtaining the CT image 1804. As can be understood from the figure, since the DFOV of the CT scan image 1802 matched the DFOV of the deep learning neural network (e.g., both had a 10 cm DFOV), the CT image 1804 could significantly remove image artifacts.

[0147] In various embodiments, the deep learning neural network was performed directly on the CT scan image 1806, thereby obtaining the CT image 1808. As can be understood from the figure, the CT image 1808 contains significant image artifacts. This is because the DFOV does not match between the CT scan image 1806 (e.g., 15 cm) and the deep learning neural network (e.g., 10 cm).

[0148] In various other embodiments, the CT scan image 1806 is upsampled to match a 10 cm DFOV, the deep learning neural network is run on the image obtained by upsampling the CT scan image, and the result generated by the deep learning neural network is downsampled and returned to a 15 cm DFOV, thereby obtaining the CT image 1810. As can be understood by looking at the figure, the CT image 1810 has significantly fewer image artifacts even though the DFOV does not match between the CT scan image 1806 and the deep learning neural network. In fact, as can be further understood by looking at the figure, the visual quality of the CT image 1810 closely resembles the visual quality of the CT image 1804 generated when there is no DFOV mismatch. Again, these results are useful for demonstrating the technical advantages of the various embodiments described herein (e.g., the deep learning neural network can be accurately run despite a DFOV (spatial resolution) mismatch).

[0149] Now, consider FIG. 19. FIG. 19 shows various CT images 1900 that are further useful for demonstrating the advantages of the various embodiments described herein. Specifically, FIG. 19 shows a CT scan image 1902 of a patient's anatomical structure. The CT scan image 1902 was acquired / generated according to a 25 cm DFOV.

[0150] In various aspects, FIG. 19 further shows CT image 1904 and CT image 1906. As described above, the deep learning neural network is trained to perform image quality improvement, and based on the CT scan image 1902, both the CT image 1904 and the CT image 1906 are generated by the deep learning neural network. However, the deep learning neural network was trained with a 10 cm DFOV instead of a 25 cm DFOV. In various aspects, the deep learning neural network is executed directly on the CT scan image 1902, thereby obtaining the CT image 1904. In various other aspects, the CT scan image 1902 is upsampled to match a 10 cm DFOV, the deep learning neural network is executed on the image obtained by upsampling the CT scan image, and the result output by the deep learning neural network is downsampled and returned to a 25 cm DFOV, thereby obtaining the CT image 1906. As can be understood by looking at the figure, it can be considered that the CT image 1904 is over-processed (e.g., filled with many image artifacts caused by the mismatch between the DFOV of the CT scan image 1902 and the DFOV of the deep learning neural network). In stark contrast to this, it can be considered that the CT image 1906 is not over-processed even though the DFOV does not match between the DFOV of the CT scan image 1902 and the DFOV of the deep learning neural network. Therefore, even though the deep learning neural network was trained with a 10 cm DFOV and the CT scan image 1902 was acquired / generated according to a 25 cm DFOV, the deep learning neural network was able to execute accurately through the pre-execution and post-execution resampling operations using the DFOV described herein (using the spatial resolution). Again, such experimental results are beneficial for demonstrating the technical advantages of the various embodiments described herein.

[0151] FIG. 20 shows a flowchart of a method 2000 implemented by an exemplary and non-limiting computer that can assist in the robustness of deep learning against differences in display fields of view, according to one or more embodiments described herein. In various cases, the DFOV robustness system 102 can assist in the method 2000 implemented by a computer.

[0152] In various embodiments, operation 2002 can include accessing a deep learning neural network (e.g., deep learning neural network 104) and a medical image (e.g., medical image 108) by a device operably coupled to a processor (e.g., by access component 116). In various cases, a first spatial resolution (e.g., spatial resolution 126) trained by the deep learning neural network may not match a second spatial resolution (e.g., spatial resolution 128) shown by the medical image.

[0153] In various aspects, operation 2004 can include executing a deep learning neural network on an image (e.g., medical image 202) obtained by resampling a medical image by a device (e.g., by execution component 120). In various aspects, the image obtained by resampling the medical image can exhibit a first spatial resolution (e.g., spatial resolution 126) trained by the deep learning neural network.

[0154] Although not explicitly shown in FIG. 20, the first spatial resolution (e.g., spatial resolution 126) may have better granularity than the second spatial resolution (e.g., spatial resolution 128), and the method 2000 implemented by a computer upsamples a medical image by an apparatus (e.g., by the pre-execution resampling component 118), thereby generating an image obtained by resampling the medical image, and executes a deep learning neural network on the image obtained by resampling the medical image, whereby the deep learning neural network can generate a first output image (e.g., output image 402), and the first output image can indicate the first spatial resolution (e.g., spatial resolution 126). In some cases, the method 2000 implemented by a computer may further include downsampling the first output image by an apparatus (e.g., by the post-execution resampling component 122), thereby generating a second output image (e.g., output image 602) indicating the second spatial resolution (e.g., spatial resolution 128).

[0155] Although not explicitly shown in FIG. 20, the first spatial resolution (e.g., spatial resolution 126) may have worse granularity than the second spatial resolution (e.g., spatial resolution 128), and the method 2000 implemented by a computer re-samples a medical image by an apparatus (e.g., by the pre-execution re-sample component 118), thereby generating an image obtained by re-sampling the medical image, and by executing a deep learning neural network on the image obtained by re-sampling the medical image, the deep learning neural network can generate a first output image (e.g., output image 402), and the first output image can indicate the first spatial resolution (e.g., 126). In various embodiments, the method 2000 implemented by a computer can further include up-sampling the first output image by an apparatus (e.g., by the post-execution re-sample component 122), thereby generating a second output image (e.g., 602) indicating the second spatial resolution (e.g., spatial resolution 128). In various cases, the first spatial resolution (e.g., spatial resolution 126) can correspond to the maximum cut-off frequency of the modulation transfer function of the medical imaging apparatus that generated the medical image.

[0156] Although not explicitly shown in FIG. 20, a deep learning neural network can belong to a set of deep learning neural networks (e.g., set 1104), and the set of deep learning neural networks can be trained respectively at a set of different spatial resolutions (e.g., set 1106), and the first spatial resolution (e.g., spatial resolution 126) can exist within the set of different spatial resolutions. In various aspects, the computer-implemented method 2000, by an apparatus (e.g., by selection component 1002), determines that any spatial resolution of the set of different spatial resolutions is a spatial resolution with less granularity than a second spatial resolution (e.g., spatial resolution 128), and that any spatial resolution is not a spatial resolution closer to the second spatial resolution (e.g., spatial resolution 128) than the first spatial resolution (e.g., spatial resolution 126), and when this determination is made, by the apparatus (e.g., by selection component 1002), further selects a deep learning neural network (e.g., deep learning neural network 104) for analyzing a medical image from the set of deep learning neural networks.

[0157] The various embodiments described herein can be considered as computer tools for assisting the robustness of deep learning against differences in DFOV. As described herein, this computer tool can improve / cope with the problem of DFOV mismatch (spatial resolution mismatch) between a deep learning neural network and a medical image that is desired to be executed by the deep learning neural network. As described herein, this computer tool can resample a medical image to match the DFOV (spatial resolution) of the medical image obtained by resampling with the DFOV (spatial resolution) on which the deep learning neural network was trained. This computer tool can execute a deep learning neural network on an image obtained by upsampling a medical image. This computer tool can resample the result generated by the deep learning neural network and match the DFOV (spatial resolution) of the image obtained by resampling with the DFOV of the original medical image or the non-resampled medical image. Thus, despite the DFOV not matching (spatial resolution not matching) between the medical image and the deep learning neural network, the inference task performed by the deep learning neural network can be accurately applied to the medical image. Therefore, this computer tool realizes a specific and practical technical improvement in the field of deep learning.

[0158] The disclosure herein mainly describes various embodiments as being applicable to deep learning neural networks, which is merely a non-limiting example. In various aspects, the teachings described herein can be applied to any suitable machine learning model regardless of architecture (e.g., to neural networks, support vector machines, naive bayes models, decision trees, linear regression models, or logistic regression models).

[0159] In the disclosure of this specification, mainly, various embodiments are described as being applied to medical images, but this is merely a non-limiting example. In various aspects, the teachings described herein can be applied to appropriate types of image data (e.g., not limited to only image data in a medical / clinical context).

[0160] In various embodiments, a machine learning algorithm or model can be implemented in any suitable way for performing any suitable aspect described herein. For performing some aspects of the machine learning in the various embodiments described above, the following artificial intelligence (AI) is considered. The various embodiments described herein can employ artificial intelligence to easily automate one or more features or functions. Components can adopt various schemes using AI to execute the various embodiments / examples disclosed herein. To provide or assist with a number of determinations (e.g., determining, verifying, inferring, calculating, predicting, foreknowing, estimating, deriving, anticipating, detecting, computer calculating) described herein, the components described herein can examine all or a subset of the data with access rights granted, and can infer or determine the state of a system or environment from a set of information obtained by events or data. The determination can be employed to identify a particular situation or action, or, for example, can generate a probability distribution of multiple states. The determination is probabilistic, that is, it is to calculate a probability distribution for a state of interest based on the consideration of data and events. The determination may also represent a technique employed to construct a higher-level event from a set of events or data.

[0161] Such a determination enables the construction of new events or actions from a set of observed events or stored event data, regardless of whether multiple events are correlated in close proximity, and regardless of whether the events and data originated from one or more event sources and data sources. The components disclosed herein can employ various classification (explicitly trained (e.g., by training data) and implicitly trained (e.g., by observing behavior, preferences, historical information, receiving external information, etc.)) schemes or systems (e.g., support vector machines, neural networks, expert systems, Bayesian belief networks, fuzzy logic, data fusion engines, etc.) for performing automatic actions or determined actions regarding the claimed subject matter. Thus, classification schemes or systems can be used to automatically learn and perform a number of functions, actions, or determinations.

[0162] A classifier can associate an input attribute vector z = (z1, z2, z3, z4, zn) with a confidence that the input belongs to a class, such as f(z) = confidence(class). Such classification can employ probabilistic or statistical analysis (e.g., considering analysis utility and analysis cost) to determine actions that are automatically performed. Support vector machines (SVMs) are an example of a classifier that can be used. SVMs operate by finding a hypersurface in the space of possible inputs, where the hypersurface is a surface that attempts to separate trigger criteria from non-trigger events. Intuitively, the hypersurface correctly classifies test data that is close to but not identical to the training data. Other directed and undirected model classification techniques include, for example, naive Bayes, Bayesian networks, decision trees, neural networks, fuzzy logic models, or probabilistic classification models that provide different patterns of independence, and any of these can be used. Classification as used herein also includes statistical regression utilized to develop preference models.

[0163] The disclosure of this specification describes non-limiting examples. For ease of description or explanation, in various parts of the disclosure of this specification, when describing various embodiments, the terms "each", "any", or "all" are used. The use of such terms as "each", "any", or "all" is non-limiting. In other words, when the disclosure of this specification provides an explanation applicable to "each", "any", or "all" of a particular object or component, this should be understood as a non-limiting example, and in various other examples, it should be further understood that the explanation may apply to a number less than "each", "any", or "all" of the particular object or component.

[0164] To provide additional explanation for the various embodiments described herein, FIGS. 21 and the following description are intended to provide a concise and general description of a suitable computing environment 2100 in which the various embodiments of the embodiments described herein are implemented. The embodiments have been described in the general context of computer-executable instructions executable on one or more computers, but those skilled in the art will recognize that the embodiments can also be implemented in combination with other program modules or as a combination of hardware and software.

[0165] In general, program modules include routines, programs, components, data structures, etc. that perform specific tasks or implement specific abstract data types. Further, those skilled in the art will understand that the methods of the present invention can be implemented in other computer system configurations (e.g., single-processor or multi-processor computer systems, minicomputers, mainframe computers, Internet of Things (IoT) devices, distributed computing systems, as well as personal computers, handheld computing devices, home appliances using microprocessors or programmable, each of which can be operably coupled to one or more associated devices).

[0166] The exemplary embodiments described herein can also be executed in a distributed computing environment where a particular task is executed by a remote processing device linked through a communication network. In a distributed computing environment, program modules can be located in both local and remote memory storage devices.

[0167] A computing device typically includes various media, which can include computer-readable storage media, machine-readable storage media, or communication media, and these two terms are used in different contexts from each other as follows in this specification. Computer-readable storage media or machine-readable storage media are available storage media that can be accessed by a computer and include both volatile and non-volatile media, and both removable and non-removable media. By way of example, computer-readable storage media or machine-readable storage media can be implemented in connection with any method or technology for storing information (such as computer-readable instructions or machine-readable instructions, program modules, structured data or unstructured data, etc.), but is not limited thereto.

[0168] Computer-readable storage media include, but are not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD), Blu-ray disc (BD) or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, solid state drive or other solid state storage devices, or other tangible or non-transitory media that can be used to store desired information. In this regard, as used herein, the terms "tangible" or "non-transitory" when applied to storage, memory, or computer-readable media are understood as modifiers that exclude only transient propagated signals themselves and do not waive rights to all standard storage, memory, or computer-readable media other than transient propagated signals themselves.

[0169] A computer-readable storage medium can be accessed, for example, by one or more local or remote computing devices via an access request, query, or other data retrieval protocol, enabling various operations with respect to the information stored on the medium.

[0170] A communication medium typically embodies computer-readable instructions, data structures, program modules, or other structured or unstructured data in a data signal, such as a modulated data signal (e.g., a carrier wave or other transport technology), and includes any information delivery medium or transmission medium. The term "modulated data signal" represents a signal that has been changed in such a way as to have one or more characteristics from among a set of signal characteristics or to encode information of one or more signals. By way of example, and not limitation, communication media include wired media (such as a wired network or direct wired connection) and wireless media (such as acoustic waves, RF, infrared, and other wireless media), but are not limited thereto.

[0171] Referring again to FIG. 21, an exemplary environment 2100 for implementing various embodiments of the aspects described herein includes a computer 2102, which includes a processing unit 2104, a system memory 2106, and a system bus 2108. The system bus 2108 couples system components (not limited to the system memory 2106) including the system memory 2106 to the processing unit 2104. The processing unit 2104 can be any of a variety of commercially available processors. Dual microprocessors and other multiprocessor architectures can also be used as the processing unit 2104.

[0172] The system bus 2108 can be any of several types of bus structures using any of a variety of commercially available bus architectures to further interconnect to a memory bus (with or without a memory controller), a peripheral bus, and a local bus. The system memory 2106 includes a ROM 2110 and a RAM 2112. The basic input / output system (BIOS) can be stored in non-volatile memory (such as ROM, erasable programmable read-only memory (EPROM), EEPROM, etc.), and this BIOS includes basic routines that help transfer information between components within the computer 2102, such as during startup. Also, the RAM 2112 can include high-speed RAM (such as static RAM for caching data).

[0173] The computer 2102 further includes an internal hard disk drive (HDD) 2114 (e.g., EIDE, SATA), one or more external storage devices 2116 (e.g., magnetic floppy disk drive (FDD) 2116, memory stick or flash drive reader, memory card reader, etc.), and a drive 2120 (e.g., solid state drive, optical disk drive, etc.) that can read from or write to a disk 2122 (CD-ROM disk, DVD, BD, etc.). Alternatively, if a solid state drive is involved, the disk 2122 is not included unless it is separate. The built-in HDD 2114 is shown as being disposed within the computer 2102, but the built-in HDD 2114 can also be configured for external use within a suitable enclosure (not shown). Further, although not shown in the environment 2100, a solid state drive (SSD) can be used in addition to or in place of the HDD 2114. The HDD 2114, the external storage device 2116, and the drive 2120 can each be connected to the system bus 2108 by an HDD interface 2124, an external storage device interface 2126, and a drive interface 2128, respectively. The interface 2124 for external drive implementation can include at least one or both of a Universal Serial Bus (USB) and Institute of Electrical and Electronics Engineers (IEEE) 1394 interface technologies. Other external drive connection technologies are within the scope contemplated by the embodiments described herein.

[0174] Drives and their associated computer-readable storage media can store data, data structures, computer-executable instructions, etc. non-volatilely. In computer 2102, the drives and storage media store any data in a suitable digital format. In the above description of computer-readable storage media, each type of storage device is mentioned. However, those skilled in the art should understand that other types of storage media readable by a computer, whether existing or to be developed in the future, can also be used in the illustrated operating environment, and further, such storage media can include computer-executable instructions for performing the methods described herein.

[0175] The drives and RAM 2112 can store a number of program modules (including operating system 2130, one or more application programs 2132, other program modules 2134, and program data 2136). It is also possible to cache all or part of the operating system, applications, modules, or data in RAM 2112. The systems and methods described herein can be implemented using various commercially available operating systems or combinations of operating systems.

[0176] Computer 2102 can optionally include emulation technology. For example, a hypervisor (not shown) or other intermediary can emulate the hardware environment for operating system 2130, and the emulated hardware can optionally be different from the hardware shown in FIG. 21. In such embodiments, operating system 2130 can include one virtual machine (VM) out of a plurality of virtual machines (VMs) hosted on computer 2102. Further, operating system 2130 can provide a runtime environment (such as a Java runtime environment or a.NET framework) to application 2132. The runtime environment is a consistent execution environment that enables application 2132 to execute on any operating system that includes the runtime environment. Similarly, operating system 2130 can support containers, application 2132 can be in the form of a container, and the container is a lightweight, stand-alone, executable software package that includes, for example, system libraries and system settings for the application, system tools, a runtime, and code.

[0177] Furthermore, computer 2102 can use a security module (such as a Trusted Processing Module (TPM)). For example, using a TPM, a boot component hashes the next boot component in time and waits for the result to match a security-protected value before loading the next boot component. This process can be executed at any layer within the code execution stack of computer 2102 (e.g., applied at the application execution level or the operating system (OS) kernel level), thereby enabling security to be realized at any level of code execution.

[0178] The user can input commands and information into the computer 2102 by one or more wired / wireless input devices (e.g., keyboard 2138, touch screen 2140, and pointing devices such as mouse 2142). Other input devices (not shown) can include a microphone, infrared (IR) remote control, radio frequency (RF) remote control, or other remote controls, joystick, virtual reality controller or virtual reality headset, game pad, stylus pen, image input device (e.g., camera), gesture sensor input device, visual motion sensor input device, emotion detection device or face detection device, biometric input device (e.g., fingerprint or iris scanner, etc.). These and other input devices are often connected to the processing unit 2104 through an input device interface 2144 that can be coupled to the system bus 2108, but can also be connected by other interfaces (parallel port, IEEE 1394 serial port, game port, USB port, IR interface, BLUETOOTH (registered trademark) interface, etc.).

[0179] Also, the monitor 2146 or other types of display devices can be connected to the system bus 2108 through an interface (such as video adapter 2148). In addition to the monitor 2146, the computer typically includes other peripheral output devices (not shown) such as speakers and printers.

[0180] Computer 2102 can operate in a networked environment using a logical connection to one or more remote computers (such as remote computer 2150) through wired or wireless communication. Remote computer 2150 can be a workstation, server computer, router, personal computer, portable computer, entertainment appliance using a microprocessor, peer device, or other common network node, and typically includes many or all of the elements described in relation to computer 2102. For the sake of brevity, only memory / storage device 2152 is illustrated. The illustrated logical connections include wired / wireless connections to a local area network (LAN) 2154 or a large-scale network (e.g., wide area network (WAN) 2156). Such LAN and WAN networking environments are common in offices and enterprises, support enterprise-wide computer networks (such as intranets), and can connect all of the LAN and WAN networking environments to a global communication network (e.g., the Internet).

[0181] When computer 2102 is used in a LAN network environment, it can be connected to local network 2154 through a wired or wireless communication network interface or network adapter 2158. Adapter 2158 can facilitate wired or wireless communication to LAN 2154, and LAN 2154 can also include a wireless access point (AP) located in the LAN to communicate with adapter 2158 in wireless mode.

[0182] When used in a WAN network environment, computer 2102 can include a modem 2160 or connect to a communication server on WAN 2156 through other means for establishing communication on the WAN (e.g., through the Internet). The modem 2160 can be an internal or external device, and a wired or wireless device, and can be connected to the system bus 2108 through the input device interface 2144. In a network environment, program modules illustrated in relation to computer 2102 or a part thereof can be stored in the remote memory / storage device 2152. The illustrated network connections are exemplary, and it is understood that other means for establishing communication links between computers can be used.

[0183] When used in either a LAN or WAN networking environment, in addition to or instead of the external storage device 2116 described above, computer 2102 can access a cloud storage system or other storage system using a network, where the storage system can be, but is not limited to, a network virtual machine that provides one or more aspects of information storage or processing. Generally, the connection between computer 2102 and the cloud storage system can be established on LAN 2154 or WAN 2156 by, for example, an adapter 2158 or a modem 2160, respectively. When computer 2102 is connected to an associated cloud storage system, the external storage interface 2126 can manage the storage provided by the cloud storage system, with the help of the adapter 2158 or the modem 2160, in the same way as other types of external storage. For example, the external storage interface 2126 can provide access to the cloud storage source as if the cloud storage source were physically connected to computer 2102.

[0184] Computer 2102 is operable to communicate with any wireless device or entity (e.g., printer, scanner, desktop or portable computer, portable data assistant, communication satellite, device or location associated with a wirelessly detectable tag (e.g., kiosk, newsstand, store shelf, etc.), and telephone) that is operably disposed for wireless communication. This includes wireless fidelity (Wi-Fi) and BLUETOOTH (registered trademark) wireless technologies. Thus, communication can be in a predefined structure like a conventional network or can be just an ad hoc communication between at least two devices.

[0185] FIG. 22 is a schematic block diagram of a sample computing environment 2200 in which the disclosed subject matter can exchange information. The sample computing environment 2200 includes one or more clients 2210. The clients 2210 can be hardware or software (e.g., threads, processes, computing devices). Also, the sample computing environment 2200 includes one or more servers 2230. The servers 2230 can also be hardware or software (e.g., threads, processes, computing devices). The servers 2230 can accommodate threads for performing conversions, for example, by adopting one or more of the embodiments described herein. One possible communication between the clients 2210 and the servers 2230 can be in the form of data packets adapted to be transmitted between two or more computer processes. The sample computing environment 2200 includes a communication framework 2250 that can be employed to facilitate communication between the clients 2210 and the servers 2230. The clients 2210 are operably connected to one or more client data stores 2220 that can be used to locally store information for the clients 2210. Similarly, the servers 2230 are operably connected to one or more server data stores 2240 that can be used to locally store information for the servers 2230.

[0186] The present invention is a system, method, apparatus, or computer program product that can be integrated at a technically detailed level. The computer program product can include a computer-readable storage medium having computer-readable program instructions for causing a processor to execute aspects of the present invention. The computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. The computer-readable storage medium can be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples of the computer-readable storage medium are listed below, although it is not possible to list all possible examples: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanically encoded devices (such as punch cards or raised structures with instructions recorded in grooves), and any suitable combination of the foregoing. The computer-readable storage medium used herein is not construed to be a transient signal itself, such as a radio wave or other electromagnetic wave propagating freely, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., an optical pulse propagating through an optical fiber cable), or an electrical signal transmitted through a wire.

[0187] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to respective computing devices / processing devices, or downloaded to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, or a wireless network). The network can include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers. The network adapter card or network interface of each computing device / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions, and the instructions are stored in a computer-readable storage medium within each computing device / processing device. The computer-readable program instructions for performing the operations of the present invention can be any combination of assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or source code or object code described in any combination of one or more programming languages (such as object-oriented programming languages like Smalltalk, C++, and procedural programming languages like the "C" programming language or similar programming languages). The computer-readable program instructions can be a stand-alone software package that executes entirely on the user's computer, executes partially on the user's computer, executes partially on the user's computer and partially on a remote computer, or executes entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any type of network (such as a local area network (LAN) or a wide area network (WAN)), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).In some embodiments, to implement aspects of the present invention, an electronic circuit (e.g., a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), etc.) can utilize the state information of computer-readable program instructions to execute the computer-readable program instructions and personalize the electronic circuit.

[0188] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions. These computer-readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the computer-readable storage medium containing instructions comprises an article of manufacture including instructions for implementing the aspects of the functions / acts specified in one or more blocks of the flowchart and / or block diagram. Also, the computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other devices to produce a computer-implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other devices implement the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0189] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram represents a module, a segment, or a portion of instructions, which can include one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions described in the blocks can be executed in an order different from that shown in the figures. For example, two blocks shown in succession can actually be executed substantially simultaneously, or the blocks can sometimes be executed in the reverse order depending on the functions involved. It should also be noted that each block in the block diagram or flowchart diagram, and combinations of blocks in the block diagram or flowchart diagram, can be implemented by a system using dedicated hardware to perform the specified function or operation or to perform a combination of dedicated hardware and computer instructions.

[0190] The subject matter has been described in the general context of computer-executable instructions of a computer program product that runs on one or more computers. However, those skilled in the art will recognize that the present disclosure can also be implemented in combination with other program modules. In general, program modules include routines, programs, components, data structures, etc. that perform specific tasks or implement specific abstract data types. Further, those skilled in the art will understand that the computer-implemented methods of the present invention can also be implemented in other computer system configurations (single-processor or multi-processor computer systems, minicomputer devices, mainframe computers, as well as computers, handheld computer devices (e.g., PDAs, telephones), consumer or industrial electronic devices using microprocessors or programmable). Also, the illustrated aspects can also be implemented in a distributed computing environment where tasks are performed by remote processing devices linked through a communication network. However, although not all aspects of the present disclosure, some aspects can be implemented on a stand-alone computer. In a distributed computing environment, program modules can be located in both local and remote memory storage devices.

[0191] In this application, terms such as "component", "system", "platform", "interface", etc. represent computer-related entities having one or more specific functions or entities related to an operating machine having one or more specific functions, or can include those entities. The entities disclosed herein can be either hardware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, or a computer. By way of illustration, both an application running on a server and the server can be components. One or more components can exist within an execution process or execution thread, a component can exist on one computer, or components can be distributed among two or more computers. In another example, each component can be executed from various computer-readable media storing various data structures. A component can communicate, for example, according to a signal having one or more data packets (e.g., data from a certain component that exchanges data with another component in a local system, in a distributed system, or in other multiple systems with signals interposed on a network such as the Internet) through a local process or a remote process. As another example, a component can be a device in which a specific function is provided by a mechanical component operated by an electrical circuit or an electronic circuit driven by software or a firmware application executed by a processor. In such a case, the processor can be inside or outside the device and can execute at least a part of the software or firmware application.As yet another example, the component can be a device that provides a particular function through an electronic component without mechanical parts, and the electronic component can include a processor or other means that executes software or firmware that imparts at least a portion of the function of the electronic component. In one aspect, the component can emulate the electronic component by, for example, a virtual machine within a cloud computing system.

[0192] Furthermore, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless otherwise specified or clear from the context, "X uses A or B" is intended to mean any inclusive and natural combination. That is, if X uses A, X uses B, or X uses both A and B, any of these examples will satisfy "X uses A or B". Further, in this specification, the term "and / or" is intended to include the same meaning as "or". Further, the articles "a" and "an" used in this specification and the accompanying drawings should generally be construed to mean "one or more" unless otherwise specified or it is clear from the context that the singular form is intended. In this specification, the words "example" and "exemplary" are used in the sense of serving as an example, an instance, or an illustration. To avoid misunderstanding, the subject matter disclosed in this specification is not limited to such examples. In addition, aspects or designs described as "example" and / or "exemplary" in this specification should not necessarily be construed as being more preferable or advantageous than other aspects or designs, nor are they intended to exclude exemplary equivalent structures and techniques known to those skilled in the art.

[0193] As used herein, the term "processor" can represent substantially any computing processing unit or device, which can include a single-core processor, a single processor with software having multi-threaded execution capabilities, a multi-core processor, a multi-core processor with software having multi-threaded execution capabilities, a multi-core processor using hardware multi-threading technology, a parallel platform, and a parallel platform with distributed shared memory, but is not limited thereto. Further, a processor can represent an integrated circuit, an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic controller (PLC), a complex programmable logic device (CPLD), discrete gates or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. Additionally, a processor can utilize nanoscale architectures such as transistors, switches, and gates using molecules and quantum dots for the purpose of optimizing space usage or improving the performance of user equipment, but is not limited thereto. A processor can also be implemented as a combination of multiple computing processing units. In the present disclosure, terms such as "store", "storage", "data store", "data storage", "database", and substantially any other information storage element related to the operation and function of components are used to represent an entity embodied in a "memory component", "memory", or an element including a memory. It should be understood that the memory or memory component described herein can be a volatile memory or a non-volatile memory, or can include both a volatile memory and a non-volatile memory.As an example, the non-volatile memory can be, but is not limited to, read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), flash memory, or non-volatile random access memory (RAM) (e.g., ferroelectric RAM (FeRAM)). As the volatile memory, for example, there is RAM, and the RAM can function as an external cache memory. As an example, the RAM can be utilized in many forms such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), direct rambus RAM (DRRAM), direct rambus dynamic RAM (DRDRAM), rambus dynamic RAM (RDRAM), etc., but is not limited thereto. Further, the memory components of the systems or methods implemented by the computers disclosed herein are intended to include these and any other suitable types of memory, but are not limited thereto.

[0194] What has been described above merely includes simple examples of the systems and methods implemented by the computers. Of course, for the purpose of explaining the present disclosure, it is impossible to describe all possible combinations of the components or the methods implemented by the computers, but more combinations and permutations are possible. Further, when terms such as "comprising", "having", "owning", etc. are used in the embodiments for carrying out the invention, the claims, the accompanying documents, and the drawings, such terms are intended to be inclusive as interpreted in the same way as the term "comprising" when "comprising" is used as a transitional term in the claims.

[0195] The descriptions of various embodiments are provided for illustrative purposes and are not intended to include all possible embodiments, nor are they intended to limit the disclosed embodiments. Many modifications and variations will be apparent without departing from the scope and spirit of the described embodiments. The terms used herein are selected to best describe the principles of the embodiments, practical applications that are superior to technologies found in the market, or technologies that are technically improved compared to technologies found in the market, or to enable those skilled in the art to understand the embodiments disclosed herein. [Embodiment 1] A system comprising a processor that executes computer-executable components stored in a non-transitory computer-readable memory, wherein the computer-executable components include an access component that accesses a deep learning neural network and a medical image, wherein a first spatial resolution trained by the deep learning neural network does not match a second spatial resolution indicated by the medical image, and an execution component that executes the deep learning neural network on an image obtained by resampling the medical image, wherein the image obtained by resampling the medical image exhibits the first spatial resolution trained by the deep learning neural network The system comprising. [Embodiment 2] The first spatial resolution has better granularity than the second spatial resolution, and the computer-executable components include A pre-execution resampling component that upsamples the medical image and thereby generates an image obtained by resampling the medical image, and executes the deep learning neural network on the image obtained by resampling the medical image, so that the deep learning neural network generates a first output image, and the first output image shows a first spatial resolution, the pre-execution resampling component The system according to Embodiment 1, including [Embodiment 3] The computer-executable component The system according to Embodiment 2, including a post-execution resampling component that downsamples the first output image and thereby generates a second output image showing the second spatial resolution. [Embodiment 4] The first spatial resolution is less granular than the second spatial resolution, and the computer-executable component A pre-execution resampling component that downsamples the medical image and thereby generates an image obtained by resampling the medical image, and executes the deep learning neural network on the image obtained by resampling the medical image, so that the deep learning neural network generates a first output image, and the first output image shows the first spatial resolution, the pre-execution resampling component The system according to Embodiment 1, including [Embodiment 5] The computer-executable component A post-execution resampling component that upsamples the first output image and thereby generates a second output image showing the second spatial resolution The system according to Embodiment 4, including [Embodiment 6] The system according to Embodiment 5, wherein the first spatial resolution corresponds to the maximum cut-off frequency in the modulation transfer function of the medical imaging device that generated the medical image. [Embodiment 7] The deep learning neural network belongs to a set of a plurality of deep learning neural networks, and the set of the plurality of deep learning neural networks is respectively trained by a set of different spatial resolutions. The first spatial resolution is included in the set of the different spatial resolutions, and the computer-executable component Based on the determination that any spatial resolution in the set of the different spatial resolutions has a worse granularity than the second spatial resolution and that any spatial resolution is not closer to the second spatial resolution than the first spatial resolution, a selection component that selects the deep learning neural network for analyzing the medical image from the set of the plurality of deep learning neural networks The system according to Embodiment 1, comprising [Embodiment 8] The system according to Embodiment 1, wherein the deep learning neural network is configured to perform image quality improvement, image noise removal, image kernel conversion, or image segmentation. [Embodiment 9] Accessing a deep learning neural network and a medical image by means of a device operably coupled to a processor, wherein a first spatial resolution trained by the deep learning neural network does not match a second spatial resolution indicated by the medical image, accessing the deep learning neural network and the medical image, and Executing the deep learning neural network on an image obtained by resampling the medical image by means of the device, wherein the image obtained by resampling the medical image indicates the first spatial resolution trained by the deep learning neural network, executing the deep learning neural network A method implemented by a computer, comprising [Embodiment 10] The first spatial resolution has a better granularity than the second spatial resolution, Upsampling the medical image by the device, thereby generating an image obtained by resampling the medical image, and executing the deep learning neural network on the image obtained by resampling the medical image, whereby the deep learning neural network generates a first output image, and the first output image indicates the first spatial resolution, and upsampling the medical image A method implemented by a computer according to Embodiment 9, including [Embodiment 11] Downsampling the first output image by the device, thereby generating a second output image indicating the second spatial resolution, and downsampling the first output image A method implemented by a computer according to Embodiment 10, including [Embodiment 12] The first spatial resolution has worse granularity than the second spatial resolution Downsampling the medical image by the device, thereby generating an image obtained by resampling the medical image, and executing the deep learning neural network on the image obtained by resampling the medical image, whereby the deep learning neural network generates a first output image, and the first output image indicates the first spatial resolution, and downsampling the medical image A method implemented by a computer according to Embodiment 9, including [Embodiment 13] Upsampling the first output image by the device, thereby generating a second output image indicating the second spatial resolution, and upsampling the first output image A method implemented by a computer according to Embodiment 12, including [Embodiment 14] The first spatial resolution corresponds to the maximum cut-off frequency in the modulation transfer function of the medical imaging device that generated the medical image, and is a method implemented by a computer according to Embodiment 13. [Embodiment 15] The deep learning neural network belongs to a set of multiple deep learning neural networks, and the set of multiple deep learning neural networks is trained by a set of different spatial resolutions respectively. The first spatial resolution is included in the set of different spatial resolutions. Determining by the device that any spatial resolution in the set of different spatial resolutions has worse granularity than the second spatial resolution, and that any spatial resolution is not closer to the second spatial resolution than the first spatial resolution, and In response to the determining, selecting, by the device, a deep learning neural network for analyzing the medical image from the set of multiple deep learning neural networks A method implemented by a computer according to Embodiment 9, including. [Embodiment 16] The deep learning neural network is configured to perform image quality improvement, image noise removal, image kernel conversion, or image segmentation, and is a method executed by a computer according to Embodiment 9. [Embodiment 17] A computer program product for assisting the robustness of deep learning against differences in display fields of view. The computer program product includes a computer-readable memory having program instructions. When the program instructions are executed by a processor, the processor is caused to Access a deep learning neural network and a medical image, where the first spatial resolution trained by the deep learning neural network does not match the second spatial resolution shown by the medical image, access the deep learning neural network and the medical image, and Executing the deep learning neural network on an image obtained by resampling the medical image, wherein the image obtained by resampling the medical image exhibits a first spatial resolution trained by the deep learning neural network, and executing the deep learning neural network A computer program product for causing execution. [Embodiment 18] The first spatial resolution has better granularity than the second spatial resolution, and the program instructions cause the processor to Upsampling the medical image, thereby generating an image obtained by resampling the medical image, and executing the deep learning neural network on the image obtained by resampling the medical image, whereby the deep learning neural network generates a first output image, and the first output image exhibits the first spatial resolution, and upsampling the medical image Downsampling the first output image, thereby generating a second output image exhibiting the second spatial resolution, and downsampling the first output image A computer program product executable to cause execution. [Embodiment 19] The first spatial resolution has worse granularity than the second spatial resolution, and the program instructions cause the processor to Downsampling the medical image, thereby generating an image obtained by resampling the medical image, and executing the deep learning neural network on the image obtained by resampling the medical image, whereby the deep learning neural network generates a first output image, and the first output image exhibits the first spatial resolution, and downsampling the medical image, and Upsampling the first output image to thereby generate a second output image indicative of the second spatial resolution, the act of upsampling the first output image A computer program product according to Embodiment 17, which is executable to cause the above to be executed. [Embodiment 20] The deep learning neural network belongs to a set of a plurality of deep learning neural networks, the set of the plurality of deep learning neural networks are each trained by a set of different spatial resolutions, the first spatial resolution is included within the set of different spatial resolutions, and the program instructions cause the processor to determine that any spatial resolution of the set of different spatial resolutions is a spatial resolution with worse granularity than the second spatial resolution and that any spatial resolution is not a spatial resolution closer to the second spatial resolution than the first spatial resolution, and in response to the determining, select the deep learning neural network for analyzing the medical image from the set of the plurality of deep learning neural networks A computer program product according to Embodiment 17, which is executable to cause the above to be executed.

Explanation of Signs

[0196] 104 Deep learning neural network 108 Medical image 110 Display field of view 112 Processor 114 Non-transitory computer-readable memory 116 Access component 118 Pre-execution resample component 120 Execution component 122 Post-execution resample component 126 Spatial resolution 128 Spatial resolution 202 Medical image 402 Output image 602 Output image 1002 Selection Component 1102 Deep Learning Neural Network Bolt 1104 Deep Learning Neural Network Set 1106 Spatial Resolution Set 1304 Curve

Claims

1. A system comprising: A processor that executes computer-executable components stored in a non-transitory computer-readable memory, wherein the computer-executable components include: An access component that accesses a deep learning neural network and a medical image, wherein a first spatial resolution trained by the deep learning neural network does not match a second spatial resolution shown by the medical image; A pre-execution resampling component that upsamples the medical image to generate an image obtained by resampling the medical image; An execution component that executes the deep learning neural network on the image obtained by resampling the medical image, wherein the image obtained by resampling the medical image shows the first spatial resolution trained by the deep learning neural network; and A post-execution resampling component that downsamples a first output image from the deep learning neural network to generate a second output image showing the second spatial resolution. The system.

2. The first spatial resolution has better granularity than the second spatial resolution. By executing the deep learning neural network on the image obtained by resampling the medical image, the deep learning neural network generates a first output image, and the first output image shows the first spatial resolution. The system according to claim 1.

3. The first spatial resolution has worse granularity than the second spatial resolution, and the computer-executable components include: A pre-execution resampling component that downsamples the medical image and thereby generates an image obtained by resampling the medical image, and by executing the deep learning neural network on the image obtained by resampling the medical image, the deep learning neural network generates a first output image, the first output image indicating the first spatial resolution, the system according to claim 1.

4. The computer-executable component A post-execution resampling component that upsamples the first output image and thereby generates a second output image indicating the second spatial resolution The system according to claim 3, comprising

5. The first spatial resolution corresponds to the maximum cut-off frequency in the modulation transfer function of the medical imaging device that generated the medical image, the system according to claim 4.

6. The deep learning neural network belongs to a set of multiple deep learning neural networks, the set of multiple deep learning neural networks is trained by a set of different spatial resolutions respectively, the first spatial resolution is included within the set of different spatial resolutions, and the computer-executable component Based on the determination that any spatial resolution in the set of different spatial resolutions is a spatial resolution with worse granularity than the second spatial resolution and none of the spatial resolutions is closer to the second spatial resolution than the first spatial resolution, a selection component that selects a deep learning neural network for analyzing the medical image from the set of multiple deep learning neural networks Including The selection component selects, from the set of different spatial resolutions, the spatial resolution with better granularity than the second spatial resolution and closest to the second spatial resolution, and as the deep learning neural network for analyzing the medical image, selects, from the set of deep learning neural networks, the deep learning neural network corresponding to the selected spatial resolution. The system according to claim 1. **Claim 7** The deep learning neural network is configured to perform image quality improvement, image noise removal, image kernel conversion, or image segmentation. The system according to claim 1. **Claim 8** Accessing a deep learning neural network and a medical image by a device operably coupled to a processor, wherein a first spatial resolution trained by the deep learning neural network does not match a second spatial resolution indicated by the medical image, accessing the deep learning neural network and the medical image, Upsampling the medical image by the device to thereby generate an image obtained by resampling the medical image, Executing the deep learning neural network on an image obtained by resampling the medical image by the device, wherein the image obtained by resampling the medical image indicates the first spatial resolution trained by the deep learning neural network, executing the deep learning neural network, and Downsampling a first output image from the deep learning neural network by the device to thereby generate a second output image indicating the second spatial resolution A method implemented by a computer, comprising: **Claim 9** The first spatial resolution has better granularity than the second spatial resolution, By executing the deep learning neural network on the image obtained by resampling the medical image, the deep learning neural network generates a first output image, and the first output image indicates the first spatial resolution. The method implemented by the computer according to claim 8.

10. The first spatial resolution has worse granularity than the second spatial resolution. The method includes Downsampling the medical image by the device, thereby generating an image obtained by resampling the medical image. By executing the deep learning neural network on the image obtained by resampling the medical image, the deep learning neural network generates a first output image, and the first output image indicates the first spatial resolution. The method implemented by the computer according to claim 8.

11. Upsampling the first output image by the device, thereby generating a second output image indicating the second spatial resolution by upsampling the first output image. The method implemented by the computer according to claim 10, including this.

12. The first spatial resolution corresponds to the maximum cut-off frequency in the modulation transfer function of the medical imaging device that generated the medical image. The method implemented by the computer according to claim 11.

13. The deep learning neural network belongs to a set of multiple deep learning neural networks. The set of multiple deep learning neural networks is each trained by a different set of spatial resolutions, and the first spatial resolution is included within the different set of spatial resolutions. Determining by the device that each spatial resolution of the set of different spatial resolutions has a granularity worse than the second spatial resolution and that each spatial resolution is not a spatial resolution closer to the second spatial resolution than the first spatial resolution, and In response to the determining, selecting, by the device, a deep learning neural network for analyzing the medical image from the set of deep learning neural networks comprising The selecting includes selecting, from the set of different spatial resolutions, a spatial resolution having a better granularity than the second spatial resolution and being closest to the second spatial resolution, and selecting, as the deep learning neural network for analyzing the medical image, the deep learning neural network corresponding to the selected spatial resolution from the set of deep learning neural networks. The method according to claim 8, which is implemented by a computer

14. The method according to claim 8, wherein the deep learning neural network is configured to perform image quality improvement, image noise removal, image kernel conversion, or image segmentation

15. A computer program for assisting the robustness of deep learning against differences in display fields of view, the program instructions being executable by a processor, and causing the processor to access a deep learning neural network and a medical image, wherein a first spatial resolution trained by the deep learning neural network does not match a second spatial resolution indicated by the medical image, access the deep learning neural network and the medical image upsample the medical image, thereby generating an image obtained by resampling the medical image Executing the deep learning neural network on the image obtained by resampling the medical image, wherein the image obtained by resampling the medical image shows a first spatial resolution trained by the deep learning neural network, and by executing the deep learning neural network on the image obtained by resampling the medical image, the deep learning neural network generates a first output image, executing the deep learning neural network, and Downsampling the first output image to thereby generate a second output image showing the second spatial resolution A computer program for causing execution.

16. The computer program according to claim 15, wherein the first spatial resolution has better granularity than the second spatial resolution.

17. The first spatial resolution has worse granularity than the second spatial resolution, and the program instructions cause the processor to Downsample the medical image to thereby generate an image obtained by resampling the medical image, and by executing the deep learning neural network on the image obtained by resampling the medical image, the deep learning neural network generates a first output image, and the first output image shows a first spatial resolution, The program instructions cause the processor to Upsample the first output image to thereby generate a second output image showing the second spatial resolution, which is executable, the computer program according to claim 15.

18. The deep learning neural network belongs to a set of multiple deep learning neural networks, and the set of the multiple deep learning neural networks is trained by a set of different spatial resolutions respectively. The first spatial resolution is included in the set of the different spatial resolutions. The program instructions cause the processor to determine that any spatial resolution in the set of the different spatial resolutions has worse granularity than the second spatial resolution and is not closer to the second spatial resolution than the first spatial resolution, and in response to the determining, it is executable to cause the processor to select the deep learning neural network for analyzing the medical image from the set of the multiple deep learning neural networks, wherein the selecting includes selecting, from the set of the different spatial resolutions, a spatial resolution that has better granularity than the second spatial resolution and is closest to the second spatial resolution, and selecting, as the deep learning neural network for analyzing the medical image, the deep learning neural network corresponding to the selected spatial resolution from the set of the deep learning neural networks. The computer program according to claim 15.

Images