Image processing systems and methods of using the same

Abstract

To visualize blood vessel and a target therapeutic part of a subject, for example, a tumor or lesion in the gastrointestinal tract of the subject.SOLUTION: The current invention relates to a method for processing an image. The method comprises: receiving an image (or an image frame); removing a plurality of first pixel values from the received image; determining a second plurality of second pixel values at a plurality of first locations and a plurality of third locations on the image; determining a third plurality of second predicted pixel values at the plurality of first locations and the plurality of second locations on the image; and generating a processed image on the basis of the plurality of second pixel values, the plurality of second predicted pixel values, a plurality of third pixel values, and a plurality of third predicted pixel values.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] Various aspects of the present disclosure generally relate to image processing systems, devices, and related methods. Examples of the present disclosure relate to systems, devices, and related methods for digital chromoendoscopy and the like. [Background technology]

[0002] Technological developments allow users of medical systems, devices, and methods to perform increasingly complex procedures on subjects. One challenge in the art of minimally invasive surgery involves visualizing a target treatment site within a subject's body, such as a tumor or lesion within the subject's gastrointestinal tract. Chromoendoscopy, which involves the injection of dyes, can facilitate the detection of changes in the mucosal surface within the luminal gastrointestinal tract. However, limited imaging methods and devices for visualizing blood vessels can prolong procedures, limit their effectiveness, and / or cause harm to the patient. Summary of the Invention

[0003] Aspects of the present disclosure relate to, among other things, systems, apparatus, and methods for providing an image processing system, and wavelength-shifting demosaicing logic, among other things. Each of the aspects disclosed herein may include one or more of the features described with respect to any of the other disclosed aspects.

[0004] According to one example, a medical device includes a shaft and a sensor coupled to a distal end of the shaft and including a filter array. The sensor is configured to capture an original image, and the filter array is configured to filter the original image to obtain a frame of original pixel values ​​including a plurality of first pixel values, a plurality of second pixel values, and a plurality of third pixel values. The medical device includes a processor and a non-transitory computer-readable medium storing demosaicing instructions that, when executed by the processor, cause the processor to exclude the plurality of first pixel values ​​from the frame of original pixel values. The processor generates a plurality of second predicted pixel values ​​at locations on the frame of the excluded first pixel values ​​and the plurality of third pixel values. The processor generates a plurality of third predicted pixel values ​​at locations on the frame of the excluded first pixel values ​​and the plurality of second pixel values. The processor generates a processed image having a partially resolved image frame from the plurality of second pixel values, the plurality of second predicted pixel values, the plurality of third pixel values, and the plurality of third predicted pixel values.

[0005] Any of the medical devices described herein may include any of the following features: the demosaicing instructions stored in a non-transitory computer-readable medium cause the processor to detect one or more edges in the original image and perform sharpened enhancement of the one or more edges to increase edge detail in the processed image; the demosaicing instructions stored in the non-transitory computer-readable medium cause the processor to output a sharpened enhanced image created from performing the sharpened enhancement step and fuse the sharpened enhanced image with the processed image; the demosaicing instructions stored in the non-transitory computer-readable medium cause the processor to set a luminance value for each of the plurality of second pixel values ​​and the plurality of third pixel values ​​and perform contrast enhancement of the plurality of second pixel values ​​and the plurality of third pixel values ​​by adjusting the luminance value to increase contrast in the processed image. The demosaicing instructions stored on the non-transitory computer-readable medium cause the processor to output a contrast-enhanced image created from performing the contrast enhancement step and blend the contrast-enhanced image with the processed image. The demosaicing instructions stored on the non-transitory computer-readable medium cause the processor to repeat all of the above steps up to a threshold. The demosaicing instructions stored on the non-transitory computer-readable medium also cause the processor to receive a wavelength shift input to identify color pixel values ​​for the plurality of first pixel values. The demosaicing instructions further include a user interface communicatively coupled to the processor and configured to transmit a signal indicative of the wavelength shift input to the processor. The sensor includes an RGB image sensor, and the filter array includes a red-green-blue Bayer color filter array. The plurality of first pixel values ​​include red pixel values, the plurality of second pixel values ​​include blue pixel values, and the plurality of third pixel values ​​include green pixel values. The sensor includes an RGB+Ir image sensor, and the filter array includes a red-green-blue-infrared Bayer color filter array. The plurality of first pixel values ​​includes blue pixel values, the plurality of second pixel values ​​includes red pixel values ​​and green pixel values, and the plurality of third pixel values ​​includes infrared pixel values.The sensor includes an RGB image sensor and a monochrome sensor. Each location in the partially resolved image frame includes one imaged color pixel value and one reconstructed color pixel value, thereby excluding at least one color pixel value from the frame of original pixel values. The system further includes a light source coupled to the distal end of the shaft, the light source being an optical fiber, an ultraviolet light, or a multicolor LED array. Demosaicing instructions stored on a non-transitory computer-readable medium cause the processor to output processed images of the partially resolved image frames to a display device.

[0006] According to another example, an image processing method includes capturing an original image and filtering the original image to obtain a frame of original pixel values ​​including a plurality of first pixel values, a plurality of second pixel values, and a plurality of third pixel values. The method includes excluding at least the plurality of first pixel values ​​and generating missing second pixel values ​​at pixel locations of the excluded first pixel values ​​and the plurality of third pixel values ​​along the frame. The method includes generating missing third pixel values ​​at pixel locations of the excluded first pixel values ​​and the plurality of second pixel values ​​along the frame, and constructing a partially sampled digital image from the plurality of second pixel values, the plurality of third pixel values, the generated second pixel values, and the generated third pixel values.

[0007] Any of the methods described herein may include any of the following steps: a method including detecting edges in a frame and enhancing edge sharpness to increase edge detail in the partially sampled digital image; a method including modifying luminance values ​​for each of a plurality of second pixel values ​​and a plurality of third pixel values ​​to increase contrast between the plurality of second pixel values ​​and a plurality of third pixel values ​​in the partially sampled digital image; a method including receiving a wavelength shift input to identify color pixel values ​​for the plurality of first pixel values; a frame of original pixel values ​​further including a plurality of fourth pixel values; or a method including excluding at least a plurality of fourth pixel values.

[0008] According to another example, there is provided a processor and a non-transitory computer-readable medium storing instructions that, when executed by the processor, cause the processor to transmit light having multiple wavelengths from an illumination source and remove at least a fraction of multiple pixel values ​​detected by a digital image sensor communicatively coupled to the processor, the digital image sensor including a filter array configured to filter the multiple wavelengths of light to obtain multiple pixel values, the processor generating multiple predicted pixel values, creating a processed image having the multiple pixel values ​​and a partially resolved image frame including the multiple predicted pixel values, and removing at least a fraction of the multiple pixel values ​​from the processed image.

[0009] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary aspects of the present disclosure and, together with the description, serve to explain the principles of the disclosure. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a schematic diagram of an exemplary medical system according to aspects of the present disclosure. [Figure 2A]2 is a partial perspective view of a medical device of the medical system of FIG. 1 including a sensor and a light source, according to an embodiment of the present disclosure. [Figure 2B] 10 is a partial perspective view of another medical device of the medical system of FIG. 1 including a sensor and multiple light sources, according to an embodiment of the present disclosure. [Figure 2C] 10 is a partial perspective view of another medical device of the medical system of FIG. 1 including a pair of sensors and a light source, according to an embodiment of the present disclosure. [Figure 3] 2B is a schematic diagram of an exemplary image sensor of the medical device of FIG. 2A, according to an embodiment of the present disclosure. [Figure 4] 4 is a schematic diagram of a frame of raw pixel data received from an image captured by the image sensor of FIG. 3, according to an embodiment of the present disclosure. [Figure 5] 2 is a block diagram of an exemplary method for imaging a target region using the medical system of FIG. 1 according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0011] Examples of the present disclosure include systems, devices, and methods for enhancing images of one or more target treatment sites within a subject's (e.g., patient's) body by highlighting one or more features of the target sites (e.g., blood vessels, vasculature, etc.) in the processed image. Aspects of the present disclosure are described in detail below, examples of which are illustrated in the accompanying drawings. Wherever possible, the same or similar reference numerals will be used throughout the drawings to refer to the same or similar parts. The term "distal" refers to the portion of the device that is furthest from the user when introduced into the patient's body. Conversely, the term "proximal" refers to the portion of the device that is closest to the user when placed within the subject's body. As used herein, the terms "comprises," "comprising," or any other variant thereof are intended to cover a non-exclusive inclusion, and a process, method, article, or device that includes listed elements does not necessarily include only those elements, but may include other elements not expressly listed or inherent in such process, method, article, or device. The term "exemplary" is used to mean "example" rather than "ideal." As used herein, the terms "about," "substantially," and "nearly" indicate a range of values ​​of + / - 10% of the stated value.

[0012] Examples of the present disclosure may be used to identify a target site within a subject's body by generating a processed image having a partial-resolution frame of pixel values ​​that visually enhances one or more features and / or characteristics of the subject's luminal gastrointestinal tract. Such features and / or characteristics may include, for example, tumors, lesions, blood vessels, changes in mucosal surfaces, etc. In some embodiments, the medical device may include an image processor, which includes a processor and memory that stores one or more algorithms for generating the partial-resolution image frames. In embodiments, the memory may include programmable instructions according to demosaicing logic, edge enhancement logic, and / or contrast enhancement logic. Additionally, the image processor may include a user interface operable to receive user input therefrom, such as a wavelength shift input for filtering out at least one color pixel value, followed by interpolation and enhancement of other color pixel values ​​captured by the image sensor. The processed image generated by the image processor of the medical device may include the partial-resolution frame of pixel values, which may be output to a display device.

[0013] Examples of the present disclosure may relate to devices and methods for performing various medical procedures and / or treating the large intestine (colon), small intestine, cecum, esophagus, and any other portion of the gastrointestinal tract and / or any other suitable portion of a patient's internal structure (collectively referred to herein as "target treatment sites"). The various examples described herein include single-use or disposable medical devices. The following provides a detailed description of the examples of the present disclosure described above and shown in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or similar parts.

[0014] 1 shows a schematic diagram of an exemplary medical system 100 according to an example of the present disclosure. The medical system 100 may include one or more light sources 130, an image processing device 101, a medical instrument 110, and a medical device 140. The image processing device 101 may be communicatively coupled to the medical instrument 110, for example, via a wired connection, a wireless connection, etc. In an example, the image processing device 101 is a computer system incorporating multiple hardware components that enable the image processing device 101 to receive data (e.g., image sensor data), process information (e.g., wavelength data), and / or generate processed images for output to a user of the medical system 100. Exemplary hardware components of the image processing device 101 may include at least one processor 102, at least one memory 103, at least one user interface 108, and at least one display 109.

[0015] The processor 102 of the image processing device 101 may include any computing device capable of executing machine-readable instructions, which may be stored on a non-transitory computer-readable medium, such as the memory 103 of the image processing device 101. For example, the processor 102 may include a controller, an integrated circuit, a microchip, a computer, and / or any other computer processing unit operable to perform the computational and logical operations necessary to execute a program. As described in more detail herein, the processor 102 is configured to perform one or more operations, such as imaging logic 104, demosaicing logic 105, edge enhancement logic 106, contrast enhancement logic 107, etc., in accordance with instructions stored on the memory 103.

[0016] 1, the memory 103 of the image processing device 101 may include a non-transitory computer-readable medium having stored thereon machine-readable instructions, such as imaging logic 104, demosaicing logic 105, edge enhancement logic 106, and contrast enhancement logic 107. The imaging logic 104 may include executable instructions that enable the medical system 100 to capture raw digital images by operating one or more components of the medical instrument 110, such as one or more image sensors 150, 150A, 150B (FIGS. 2A-2C).

[0017] Additionally, the demosaicing logic 105 may include executable instructions that enable the medical system 100 to process a digital image (e.g., a mosaic image) by demosaicing the image and reconstructing missing and / or unknown pixel values ​​in the mosaic image. It should be understood that a digital image captured by an image sensor using a color filter sensor array may provide an original image having various color pixel values ​​arranged in a mosaic pattern. Each pixel array in the pattern includes only one color pixel value, whereby one or more color pixel values ​​may be removed therefrom. As described in more detail herein, a digital image includes a two-dimensional array of pixel values, each corresponding to light intensity (e.g., color pixel value) in one of multiple spectral bands at a pixel location in the image.

[0018] 1 , the edge enhancement logic 106 may include executable instructions that enable the medical system 100 to process the mosaic image of the target area and increase the definition of one or more edges within the mosaic image. It should be understood that the demosaicing process may include inherent side effects, such as a reduction in the sharpness of one or more edges within the image. For example, the demosaicing process may attenuate high-frequency detail in the image and / or enhance low-frequency detail in the image.

[0019] In this example, color fringing may occur at the edges of sharp contrast boundaries in the image, where the edges of sharp contrast boundaries may contain fringe artifacts in the color pixel values ​​of the mosaic image. As described further below, the edge enhancement logic 106 may include executable instructions that enable the medical system 100 to process a digital image (e.g., a mosaic image) by detecting edges and increasing detail at the edges to provide sharper definition of the image in the color pixel values.

[0020] 1 , the contrast enhancement logic 107 may include executable instructions that enable the medical system 100 to process the mosaic image of the target area and enhance the contrast of one or more pixel values ​​within the mosaic image. It should be understood that the demosaicing process may include inherent side effects, such as a reduction in image contrast due to a reduction in color difference signals between pixel values ​​within the image.

[0021] In this example, a resolution frame of color pixel values ​​may lack sufficient brightness to differentiate one or more features of the image between various color pixel values. As described further below, the contrast enhancement logic 107 may include executable instructions that enable the medical system 100 to process a digital image (e.g., a mosaic image) by scaling the brightness of certain color pixel values ​​and increasing the brightness of the resolution frame to provide greater clarity of the image within the color pixel values.

[0022] In some embodiments, the imaging logic 104, the demosaicing logic 105, the edge enhancement logic 106, and / or the contrast enhancement logic 107 may include executable instructions that enable the medical system 100 to automatically perform periodic imaging of the target region without requiring user input. In other embodiments, the imaging device 101 may be configured to receive user input, such as from a user interface 108 of the imaging device 101, that initiates imaging of the target region. It should be recognized that in some embodiments, the user interface 108 may be a device integral with the imaging device 101, while in other embodiments, the user interface 108 may be a remote device that communicates (e.g., wirelessly, wired, etc.) with the imaging device 101.

[0023] It should be understood that various programming algorithms and data that support the operation of medical system 100 may reside in whole or in part in memory 103. Memory 103 may include any type of computer-readable medium suitable for storing data and algorithms, such as random access memory (RAM), read-only memory (ROM), flash memory, a hard drive, and / or any device capable of storing machine-readable instructions. Memory 103 may include one or more data sets, including, but not limited to, image data from one or more components of medical system 100 (e.g., medical instrument 110, medical device 140, etc.).

[0024] Continuing with reference to FIG. 1 , medical instrument 110 may be configured to facilitate positioning one or more components of medical system 100, such as medical device 140, relative to a subject (e.g., a patient). In embodiments, medical instrument 110 may be any type of endoscope, duodenoscope, gastroscope, colonoscope, ureteroscope, bronchoscope, catheter, or other delivery system, and may include a handle 112, a steering mechanism 114, at least one port 116, and a shaft 120. Handle 112 of medical instrument 110 may have one or more lumens (not shown) that communicate with lumens of one or more other components of medical system 100. Handle 112 further includes at least one port 116 that opens into the one or more lumens of handle 112. As described in more detail herein, at least one port 116 is sized and shaped to receive one or more instruments, such as medical device 140 of medical system 100, therein.

[0025] The shaft 120 of the medical instrument 110 may comprise a sufficiently flexible tube, such that the shaft 120 is configured to selectively bend, rotate, and / or twist during insertion into and / or through a subject's tortuous anatomy to a target treatment site. The shaft 120 may have one or more lumens (not shown) extending therethrough, including, for example, a working lumen for receiving an instrument (e.g., a medical device 140). In other examples, the shaft 120 may include additional lumens, such as a control wire lumen for receiving one or more control wires for operating one or more distal components / tools (e.g., an articulation joint, an elevator, etc.), a fluid lumen for delivering a fluid, an illumination lumen for receiving at least a portion of an illumination assembly (not shown), and / or an imaging lumen for receiving at least a portion of an imaging assembly (not shown).

[0026] Continuing to refer to FIG. 1 , the medical instrument 110 may further include a tip 122 at the distal end of the shaft 120. In some embodiments, the tip 122 may be attached to the distal end of the shaft 120, while in other embodiments, the tip 122 may be integral with the shaft 120. For example, the tip 122 may include a cap configured to receive the distal end of the shaft 120 therein. The tip 122 may include one or more openings that communicate with one or more lumens of the shaft 120. For example, the tip 122 may include a working port 123 through which the medical device 140 may exit the working lumen of the shaft 120. It should be appreciated that one or more other openings at the tip 122 of the shaft 120 are not shown. The operating mechanism 114 of the medical instrument 110 is located on the handle 112 and may include one or more knobs, buttons, levers, switches, and / or suitable actuators thereof. The operating mechanism 114 is configured to control at least the orientation of the shaft 120 (e.g., through manipulation of a control wire).

[0027] The medical device 140 of the medical system 100 may include a catheter having a longitudinal body 142 between a proximal end 141 of the medical device 140 and a distal end 144 of the medical device 140. The longitudinal body 142 of the medical device 140 may be flexible such that the medical device 140 is configured to bend, rotate, and / or twist when inserted into the working lumen of the medical instrument 110. The medical device 140 may include a handle at the proximal end 141 of the longitudinal body 142, which may be configured to move, rotate, and / or bend the longitudinal body 142. Additionally, the handle at the proximal end 141 of the medical device 140 may define one or more ports (not shown) sized to receive one or more tools through the longitudinal body 142 of the medical device 140.

[0028] 1 , medical instrument 110 may be configured to receive medical device 140 from at least one port 116, through the working lumen, through shaft 120, and into working port 123 of tip 122. In this example, medical device 140 may extend distally from working port 123 into the surrounding environment of tip 122, e.g., at a target treatment site in a subject, as described in more detail below. A distal end 144 of medical device 140 may extend distally from tip 122 in response to movement of longitudinal body 142 within the working lumen of shaft 120. Medical device 140 may include one or more end effectors (not shown) at the distal end 144 of longitudinal body 142 for performing one or more operations at the target treatment site.

[0029] The medical instrument 110 may further be configured to receive one or more light sources 130 through the shaft 120 via at least one of the lumens of the medical instrument 110 to be connected to the optical fiber 146. In this example, the one or more light sources 130 are shown as separate components from the imaging device 101, whereby the light sources 130 are coupled to the medical instrument 110 separately from the imaging device (e.g., via a cable). It should be recognized that in other embodiments, the one or more light sources 130 may be included in the imaging device 101, whereby the light sources 130 may be communicatively coupled to the medical instrument 110 along with the imaging device 101.

[0030] 2A-2C, a tip 122 of a medical device 110 is shown in accordance with one or more examples of the present disclosure. Referring first to FIG. 2A, in one embodiment, the tip 122 of the medical device 110 may include an optical fiber 146 and an image sensor 150 thereon. In this example, the optical fiber 146 may be coupled to one or more light sources 130 of the medical system 100, such that each of the one or more light sources 130 may transmit light through a single optical fiber 146. Although not shown, it should be appreciated that multiple light sources 130 may be coupled to the optical fiber 146 via a fiber splitter / combiner. The optical fiber 146 of the medical device 110 may be configured and operable to deliver light of various amplitudes from the one or more light sources 130 distally from the tip 122 of the shaft 120. In some embodiments, the optical fiber 146 may be configured to deliver white light, ultraviolet light, near-infrared (NIR) light, and / or various other wavelengths within or beyond the visible spectrum.

[0031] 2A , the image sensor 150 of the medical instrument 110 may be communicatively coupled to the image processing device 101 of the medical system 100, such as via a wired and / or wireless connection. The image sensor 150 of the medical instrument 110 may be configured and operable to capture an original image (e.g., a digital image) of the environment surrounding the tip 122 of the shaft 120. In some embodiments, the image sensor 150 may include an image sensor, such as an RGB (i.e., red-green-blue) digital sensor, an RGB-Ir (i.e., red-green-blue-infrared) digital sensor, and / or a monochrome sensor. As described in more detail herein, the image sensor 150 may include one or more components for filtering white light, ultraviolet light, near-infrared light, and / or colors from other wavelengths within or beyond the visible spectrum.

[0032] 2B, in another embodiment, the medical instrument 110 may include a multi-color LED assembly at the tip 122 of the shaft 120. In this example, the multi-color LED assembly may include one or more light emitting diodes (hereinafter, LEDs) 146A, 146B, 146C, and 146D arranged in an annular array around the image sensor 150. Each of the LEDs 146A, 146B, 146C, and 146D may be configured and operable to transmit a different wavelength and / or amplitude (e.g., color) of light relative to one another. It should be understood that different illumination sources may produce different spectra. It should be recognized that the LEDs 146A, 146B, 146C, and 146D of the medical instrument 110 may include more and / or fewer diodes at the tip 122 than shown and described herein without departing from the scope of the present disclosure.

[0033] In another embodiment, referring now to FIG. 2C , medical instrument 110 may include a multi-sensor assembly at tip 122 of shaft 120. In this example, multi-sensor assembly may include color image sensor 150A and monochrome image sensor 150B. As described in more detail herein, color image sensor 150A may be configured and operable to capture a portion of incident light at each individual pixel location of color image sensor 150A according to the color of the incident light. In some embodiments, color image sensor 150A may include, for example, an RGB (red-green-blue digital sensor), an RGB-Ir (red-green-blue-infrared) digital sensor, or the like. As described in more detail herein, monochrome image sensor 150B may be configured and operable to completely capture all of the incident light at each individual pixel location of monochrome image sensor 150B, regardless of the color of the incident light.

[0034] 3 , the image sensor 150 of the medical instrument 110 may include an exterior surface 152 having a plurality of microlenses 154 disposed thereon. In some examples, the exterior surface 152 and / or the plurality of microlenses 154 may be formed of glass, plastic, and / or other transparent materials. The image sensor 150 may be a color image sensor including a color filter array 156 disposed relatively below the exterior surface 152. The color filter array 156 may include a fiber optic device having a plurality of color pixel locations 158A, 158B, 158C arranged in a predetermined pattern. In this example, each of the plurality of microlenses 154 may be positioned in registration with at least one of the plurality of color pixel locations 158A, 158B, 158C of the color filter array 156 disposed below the exterior surface 152.

[0035] In some embodiments, color filter array 156 may include a plurality of first color pixel locations 158A, a plurality of second color pixel locations 158B, and / or a plurality of third color pixel locations 158C. The plurality of color pixel locations 158A, 158B, 158C may be arranged in a mosaic pattern, such as a Bayer pattern, along color filter array 156. In this example, the plurality of first color pixel locations 158A may include red filters, the plurality of second color pixel locations 158B may include green filters, and the plurality of third color pixel locations 158C may include blue filters. In other embodiments, the plurality of color pixel locations 158A, 158B, 158C may include various suitable color filters and / or patterns other than those shown and described herein. For example, in an embodiment in which image sensor 150 includes an RGB-IR sensor, it should be understood that color filter array 156 may additionally include a plurality of fourth color pixel locations corresponding to infrared color filters.

[0036] 3 , the color filter array 156 of the image sensor 150 may be configured and operable to selectively transmit one or more wavelengths 12, 14, 16 (e.g., light intensity, spectral band, color, etc.) of the light beam 10. For example, each of the color pixel locations 158A, 158B, 158C may be configured to transmit and / or transmit a portion of the light beam 10 (e.g., at least one wavelength 12, 14, 16) therethrough depending on the color filter of the color pixel location 158A, 158B, 158C. Thus, at each color pixel location 158A, 158B, 158C, only one color component (e.g., a color pixel value) may be measured by the image sensor 150. As described further herein, the color pixel value may include an amount of energy in a colored range of the spectrum (e.g., a red range, a green range, a blue range, etc.).

[0037] In this example, each of the plurality of color pixel locations 158A, 158B, 158C may allow only one wavelength 12, 14, 16 of the light beam 10 to pass through the color filter array 156. The image sensor 150 may further include a photosensor array 160 disposed relatively below the color filter array 156, such that the color filter array 156 of the image sensor 150 may be positioned between the exterior surface 152 and the photosensor array 160. The photosensor array 160 of the image sensor 150 may include a photodiode (e.g., a semiconductor device) having a plurality of photosites 162 and circuitry 164 communicatively coupled to the plurality of photosites 162.

[0038] 3, the plurality of photosites 162 are arranged in an array (e.g., a grid), with each of the plurality of photosites 162 positioned in alignment with at least one of a plurality of color pixel locations 158A, 158B, 158C of the color filter array 156 disposed above the photosensor array 160. The photosensor array 160 may be configured and operable to convert the light beam 10 received through the exterior surface 152 and the color filter array 156 into an electrical current. In this example, an electrical current may be generated by the photosensor array 160 when photons from the received light are absorbed by the plurality of photosites 162.

[0039] In this example, each of the plurality of photosites 162 may be configured to measure the quantity of only one color pixel value (e.g., red, green, or blue) in the incident light 10 at the photosite's 162 location along the surface of the photosensor array 160. Accordingly, the plurality of photosites 162 may image the incident light 10 and generate electrical signals that may be quantified and stored as numerical values ​​in the resulting processed image file. It should be recognized that the photosensor array 160 may include a variety of suitable shapes, sizes, and / or configurations other than those shown and described herein. In other embodiments, the image sensor 150 may be a monochrome sensor (e.g., monochrome sensor 150B), whereby the color filter array 156 shown and described above may be completely eliminated from between the exterior surface 152 and the photosensor array 160. In this example, each photosite 162 along the photosensor array 160 may be operable to receive, image, and absorb all three wavelengths 12, 14, and 16 of the light beam 10.

[0040] 3 and 4, in conjunction with the flow diagram of FIG. 5, there is shown generally an exemplary method 200 for generating processed images of a target region using medical system 100. The illustrations of FIGS. 3-5 and the following description thereof are not intended to limit the subject matter described herein to any particular method.

[0041] 1 , the medical instrument 110 of the medical system 100 may be inserted into a subject's body (not shown) and positioned such that the tip 122 is adjacent to a target site. For example, the shaft 120 may be navigated through the subject's (e.g., patient's) digestive tract by inserting the tip 122 into the subject's nose or mouth (or other suitable natural orifice in the body) and through the subject's gastrointestinal tract (e.g., esophagus, stomach, small intestine, etc.) to reach the target site. It should be appreciated that the length of the shaft 120 may be sufficient to enable the proximal end of the medical instrument 110 (including the handle 112) to be outside the subject's body while the tip 122 of the medical instrument 110 is within the subject's body. While the present disclosure relates to the use of the medical system 100 in the digestive tract of a subject, it should be understood that the features of the present disclosure may also be used in various other locations within the subject's body (e.g., other organs, tissues, etc.).

[0042] Additionally, once the medical instrument 110 is received within the subject's body and the tip 122 of the shaft 120 is positioned relatively close to the target site, a medical device 140 may be received within the medical instrument 110 through at least one port 116. In this example, the longitudinal body 142 of the medical device 140 is moved within the shaft 120 through at least one of the lumens (e.g., the working lumen) of the shaft 120. The distal end 144 of the longitudinal body 142 may extend distally from the tip 122 of the shaft 120 through a working port 123 that communicates with the working lumen of the shaft 120. It should be recognized that this step may be optional, and receiving the medical device 140 within the medical instrument 110 may occur at various other steps of the method 200 and / or may be omitted entirely. The tip 122 may be positioned adjacent to and facing the target treatment site.

[0043] 5, in step 202, in response to the processor 102 of the image processing device 101 executing the imaging logic 104 to activate one or more light sources 130, one or more target objects may be illuminated by the medical instrument 110. In the example of a medical instrument 110 including an optical fiber 146 (FIG. 2A) and / or multi-color LED assemblies 146A, 146B, 146C, 146D (FIG. 2B), light from one or more light sources 130 may be emitted from the chip 122 of the medical instrument 110 to illuminate the target objects.

[0044] In step 204, once the target object is illuminated with light from the medical instrument 110, the image sensor 150 may be activated by the processor 102 executing the imaging logic 104 to capture one or more original digital images of the target object. It should be understood that the processor 102 of the image processing device 101 may be communicatively coupled to the image sensor 150 of the medical instrument 110 via circuitry 164. For example, referring again to FIG. 3 , light 10 transmitted to the target object by the optical fiber 146 and / or multi-color LED assemblies 146A, 146B, 146C, 146D may reflect off the target object and be received by the image sensor 150. In this example, multiple wavelengths 12, 14, 16 of the light 10 may be received through one or more of multiple microlenses 154 on the exterior surface 152.

[0045] The plurality of wavelengths 12, 14, 16 may be received at one or more corresponding color pixel locations 158A, 158B, 158C of the color filter array 156, e.g., aligned with the microlens 154 through which the light beam 10 is received. At each of the plurality of color pixel locations 158A, 158B, 158C, one or more of the plurality of wavelengths 12, 14, 16 of the light 10 may be prevented from transmitting (e.g., filtered, filtered out, excluded, blocked) through the color filter array 156 depending on the color filter of the color pixel location 158A, 158B, 158C. Thus, the color filter configuration (e.g., red, green, blue, etc.) of the color pixel location 158A, 158B, 158C that receives the light 10 determines which of the wavelengths 12, 14, 16 (e.g., red, blue, green, etc.) can transmit through the color filter array 156 at that location.

[0046] Thus, it should be understood that each of color pixel locations 158A, 158B, 158C may allow only about one-third (e.g., 33%) of incident light 10 at that location to be transmitted through photosensor array 160. For example, at each of first color pixel locations 158A (e.g., red filter), only wavelength 12 (e.g., the red range of the light spectrum) of light 10 may be transmitted through color filter array 156, whereby wavelengths 14 and 16 (e.g., blue and green, respectively) may be filtered out by color filter array 156 at first color pixel location 158A. It should be appreciated that in this embodiment, second color pixel locations 158B (e.g., green filter) may be configured to transmit wavelength 14 (e.g., the green range of the light spectrum), and third color pixel locations 158C (e.g., blue filter) may be configured to transmit wavelength 16 (e.g., the blue range of the light spectrum), respectively.

[0047] 3, individual wavelengths 12, 14, 16 of light 10 transmitted through color filter array 156 may be detected along photosensor array 160 and absorbed by one or more of a plurality of photosites 162 (i.e., those aligned with color pixel locations 158A, 158B, 158C that receive light beam 10 therethrough). In this example, the portion of light 10 absorbed by each of the plurality of photosites 162 may be converted into electrical current. The raw digital image captured by image sensor 150 may include a quantitative record of the light energy measured at each grid location of photosites 162 along photosensor array 160, with each photosite 162 configured to identify a color pixel value for the wavelengths 12, 14, 16 received thereon.

[0048] In this example, processor 102 of image processing device 101, when executing imaging logic 104, may cause photosensor array 160 to transmit color pixel value electrical signals to image processing device 101, for example via circuitry 164. The color pixel value electrical signals may be stored in memory 103 of image processing device 101 and used by demosaicing logic 105, edge enhancement logic 106, and / or contrast enhancement logic 107 to generate a processed image.

[0049] 5 , in step 206, a wavelength shift input may be entered by a user of the medical system 100, for example, via the user interface 108 of the image processing device 101. The wavelength shift input may include the identity, amplitude, and / or color pixel values ​​of one or more light wavelengths for adjusting the demosaicing process of the raw digital image captured by the image sensor 150. It should be appreciated that in some embodiments, step 206 may occur before steps 202 and 204 and / or may be automatically pre-programmed into the memory 103 of the image processing device 101, such that the method 200 may proceed to step 208 with the wavelength shift input pre-specified.

[0050] 4, the raw digital image received by the image processing device 101 may include a frame 20 (e.g., a grid) of raw pixel values ​​formed of a plurality of color pixel values ​​22A, 22B, 22C measured by the image sensor 150. It should be understood that each of the grid locations along the frame 20 of raw pixel values ​​may correspond to the location of a photosite 162 on the photosensor array 160. Thus, each of the grid locations on the frame 20 of raw pixel values ​​includes at least one of a first color pixel value 22A, a second color pixel value 22B, or a third color pixel value 22C based on the relative position of the grid location with respect to the color pixel locations 158A, 158B, 158C of the color filter array 156 through which the photosite 162 receives the wavelengths 12, 14, 16.

[0051] In step 208, the processor 102 may separate the frame of original pixel values ​​20 by color pixel values ​​22A, 22B, 22C and filter (e.g., filter out) at least one of the first color pixel value 22A, the second color pixel value 22B, or the third color pixel value 22C based on the wavelength shift input received in step 206. In other words, the image processing device 101 is configured to perform a demosaicing process on the original digital image captured by the image sensor 150 to shift from the plurality of color pixel values ​​22A, 22B, 22C in the frame of original pixel values ​​20 to a fraction of the color pixel values ​​22A, 22B, 22C.

[0052] Thus, as will be described in more detail below, the demosaicing process (steps 210A, 210B), edge enhancement process (steps 212A, 212B), and / or contrast enhancement process (steps 214A, 214B) of the original digital image are performed on only a portion of the frame 20 of original pixel values ​​contained in the original digital image captured by the image sensor 150. For example, the processor 102 of the image processing device 101 removes a plurality of first color pixel values ​​22A according to the wavelength shift input, such that the resulting processed image produced by the image processing device 101 includes a partially resolved frame of pixels.

[0053] 5, in step 210A, processor 102 performs a demosaicing process on a plurality of second color pixel values ​​22B (when executing demosaicing logic 105) to calculate estimates of second color pixel values ​​22B (e.g., green) at grid locations along frame 20 of original pixel values ​​that did not receive wavelength 14 (e.g., green), such as grid locations that received wavelength 12 (e.g., red) and wavelength 16 (e.g., blue). Processor 102 may estimate the amount of unknown second color pixel values ​​22B at grid locations on frame 20 of original pixel values ​​that did not receive wavelength 14 by interpolation from measurements of known second color pixel values ​​22B.

[0054] In this example, processor 102 executing demosaicing logic 105 interpolates missing second color pixel value 22B from adjacent (e.g., neighboring) grid locations that contain measured second color pixel values ​​22B. Processor 102 determines the amount of unknown second color pixel value 22B from the neighboring grid locations that received wavelength 14. It should be understood that processor 102 may execute demosaicing logic 105 in step 210B to reconstruct unknown and / or missing third color pixel values ​​22C along frame 20 of original pixel values ​​in a manner substantially similar to that described above with respect to step 210A.

[0055] 5 , in step 212A, the processor 102 of the image processing device 101 may execute the edge enhancement logic 106 to further reconstruct the original digital image captured by the image sensor 150. In this example, the edge enhancement logic 106, when executed by the processor 102, may increase the detail and / or definition of one or more edges in the frame of original pixel values ​​20. For example, the processor 102 may detect the location of one or more edges in each grid location in the frame of original pixel values ​​20 and minimize the noise level around the edge by adjusting one or more of the plurality of second color pixel values ​​22B at that grid location. In some embodiments, the processor 102 may improve the sharpness of the one or more edges in the grid location by increasing the gradient (e.g., magnitude) of the plurality of second color pixel values ​​22B.

[0056] It should be understood that in step 212B, processor 102 may execute edge enhancement logic 106 in a manner substantially similar to that described above with respect to step 212A to reconstruct edges of the digital image by adjusting one or more of the plurality of third color pixel values ​​22C along the frame of original pixel values ​​20. It should be recognized that in other embodiments, a variety of other suitable edge enhancement processes may be included in edge enhancement logic 106 and executed by processor 102.

[0057] 5 , in step 214A, the processor 102 of the image processing device 101 may execute the contrast enhancement logic 107 to further reconstruct the original digital image captured by the image sensor 150. In this example, the contrast enhancement logic 107, when executed by the processor 102, may enhance the contrast of the representation of the frame 20 of original pixel values. For example, the processor 102 may increase one or more brightness components (e.g., luminance) of a plurality of second color pixel values ​​22B within each grid location in the frame 20 of original pixel values. In some embodiments, the processor 102 may adjust the brightness of one or more grid locations in the frame 20 of original pixel values ​​by downscaling the second color pixel values ​​22B therein to reduce the contrast contribution provided by the second color pixel values ​​22B.

[0058] It should be understood that in step 214B, processor 102 may execute contrast enhancement logic 107 in a manner substantially similar to that described above with respect to step 214A to enhance the local contrast of the digital image by adjusting one or more of the plurality of third color pixel values ​​22C along the frame of original pixel values ​​20. It should be recognized that in other embodiments, a variety of other suitable contrast enhancement processes may be included in contrast enhancement logic 107 and executed by processor 102.

[0059] 5, in step 216, the processor 102 of the image processing device 101 may determine whether the current iteration of the demosaicing process (steps 210A, 210B), the edge enhancement process (steps 212A, 212B), and the contrast enhancement process (steps 214A, 214B) is equal to or greater than a predetermined and / or dynamic iteration threshold. In some embodiments, the predetermined iteration threshold may be predetermined and stored in the memory 103 of the image processing device 101, or may be selectively input by a user of the medical system 100.

[0060] In another embodiment, the iteration threshold may be dynamically determined by processor 102 based on one or more factors, including, for example, peak definition, contrast, and visibility from an initial transformation of the digital image through the demosaicing, edge enhancement, and contrast enhancement processes. In this example, data indicative of the initial state of the raw pixel values ​​20 (e.g., a histogram of the frame) may be analyzed by processor 102 upon completion of the first iteration, and peak values ​​for the definition, contrast, and visibility characteristics may be identified. Thus, processor 102 may continually determine the current definition, contrast, and visibility of the digital image relative to the initial peak values ​​(i.e., the dynamic iteration threshold) upon completion of each iteration of the process.

[0061] In response to processor 102 determining at step 216 that the current iteration of method 200 is below the predetermined (or dynamic) threshold, image processing device 101 may be configured and operable to return to steps 210A, 210B and perform one or more of demosaicing, edge enhancement, and / or contrast enhancement processes. In response to processor 102 determining at step 216 that the current iteration of method 200 is at least equal to or greater than the predetermined (or dynamic) threshold, image processing device 101 may be configured and operable to generate an interpolated output image. It should be appreciated that in response to the iterative enhancement of frame 20 of raw pixel values ​​originally captured by image sensor 150, an image with improved definition, contrast, and / or visibility may be provided.

[0062] 5 , in step 218, the processor 102 of the image processing device 101 may generate a processed image resulting from the process of the method 200 shown and described herein. In this example, the output image may include a partial-resolution frame of color pixel values, such as a plurality of second color pixel values ​​22B and a plurality of third color pixel values ​​22C. Thus, the digital image generated by the processor 102 of the image processing device 101 may omit at least a plurality of first color pixel values ​​22A from one or more of the grid locations of the frame 20 of original pixel values ​​originally captured by the image sensor 150. In this example, the generated image excludes the first color pixel values ​​22A at each of the grid locations of the frame 20 of original pixel values ​​such that the image processing device 101 generates a partial-resolution image.

[0063] With the display 109 of the medical system 100 communicatively coupled to the processor 102 of the image processing device 101, the processor 102 may be operable to transmit the partially resolved images to the display 109 for viewing by a user of the medical system 100. In some examples, the medical system 100 may be configured and operable to continuously perform the method 200 shown and described herein, whereby the display 109 may output multiple partially resolved images to provide continuous (e.g., live, real-time, etc.) imaging of one or more target objects.

[0064] It should be appreciated that excluding at least one of the plurality of color pixel values ​​22A, 22B, 22C originally captured (by image sensor 150) in frame 20 of raw pixel values ​​from the processed image may facilitate differentiation of one or more features and / or structures (e.g., target objects) of the target treatment site. For example, in step 206, processor 102 may filter out at least one of color pixel values ​​22A, 22B, 22C according to the wavelength shift input, thereby better distinguishing blood vessels from surrounding tissue. In some examples, the primary colors of one or more target objects may be substantially similar to one another, e.g., tissue and blood vessels may appear generally red, thereby making the features less distinguishable in the resulting image. In this example, enhancing blue and / or green color pixel values ​​(e.g., second color pixel value 22B and third color pixel value 22C, respectively) may better distinguish the features.

[0065] Additionally, it should be understood that one or more color components (e.g., red) may have lower contrast compared to one or more other color components (e.g., green, blue) within the target treatment area. By excluding one or more of the color components, e.g., first color pixel value 22A (e.g., red), from the resulting digital image, the subject's vessels may be enhanced because only second color pixel value 22B (e.g., blue) and third color pixel value 22C (e.g., green) are retained from frame 20 of original pixel values.

[0066] In embodiments in which the optical fiber 146 of the medical device 140 is operable to generate ultraviolet light, the medical system 100 may be configured to generate partially resolved image frames that are capable of distinguishing desirable (e.g., healthy) tissue from undesirable (e.g., unhealthy) tissue because the tissues may fluoresce different colors under ultraviolet light. In this example, the wavelength shift input of step 206 may include the first color pixel value 22A (e.g., red) and the third color pixel value 22C (e.g., green), such that only the second color pixel value 22B (e.g., blue) may be included in the resulting processed image.

[0067] In an embodiment in which the image sensor 150 of the medical instrument 110 includes an RGB-IR sensor, the frame 20 of raw pixel values ​​detected and captured by the image sensor may include a plurality of first color pixel values ​​22A (e.g., red), a plurality of second color pixel values ​​22B (e.g., blue), a plurality of third color pixel values ​​22C (e.g., green), and a plurality of fourth color pixel values ​​(e.g., infrared) that are outside the visible spectrum and closer to the infrared spectrum. In this example, the medical system 100 may be configured to generate a partially resolved image frame that can make relatively dark areas within a target treatment site (e.g., a body lumen) more visible by enhancing the first color pixel values ​​22A, the third color pixel values ​​22C, and / or the fourth color pixel values. Thus, the wavelength shift input of step 206 may include the second color pixel value 22B (e.g., blue) for exclusion from the processed image.

[0068] In embodiments in which the tip 122 of the shaft 120 includes a color image sensor 150A (e.g., an RGB-IR sensor) and a monochrome image sensor 150B, the image processor 101 may be configured to generate a partially resolved image captured by the color image sensor 150A while further enhancing the contrast of relatively dark areas in the frame 20 of raw pixel values ​​with the monochrome image sensor 150B. In this example, the monochrome image sensor 150B may be sensitive to near-infrared wavelengths illuminated in the light 10, so that a fourth color pixel value (e.g., infrared) may be readily detected. It should be appreciated that visualization of one or more target objects by the monochrome image sensor 150B may be facilitated by providing one or more materials, such as fluorescent dyes, within the target treatment area.

[0069] (Addendum) As a preferred embodiment, the technical concept that can be grasped from the above embodiment will be described below. [Item 1] A medical device comprising: The medical device comprises a shaft; the medical device comprises a sensor coupled to a distal end of the shaft and including a filter array, the sensor configured to capture an original image, and the filter array configured to filter the original image into a frame of original pixel values, the frame of original pixel values ​​including a plurality of first pixel values, a plurality of second pixel values, and a plurality of third pixel values; The medical device includes a processor and a non-transitory computer-readable medium storing demosaicing instructions that, when executed by the processor, cause the processor to: excluding the plurality of first pixel values ​​from the frame of original pixel values; generating a plurality of second predicted pixel values ​​at locations of the plurality of excluded first pixel values ​​and the plurality of third pixel values ​​on the frame; generating a plurality of third predicted pixel values ​​at locations of the plurality of excluded first pixel values ​​and the plurality of second pixel values ​​on the frame; generating a processed image having a partially resolved image frame from the plurality of second pixel values, the plurality of second predicted pixel values, the plurality of third pixel values, and the plurality of third predicted pixel values; Medical devices. [Item 2] The demosaicing instructions stored on the non-transitory computer-readable medium cause the processor to: detecting one or more edges in the original image; Item 14. The medical device of item 1, wherein the one or more edge sharpening enhancements are performed to increase edge detail of the processed image. [Item 3] The demosaicing instructions stored on the non-transitory computer-readable medium cause the processor to: outputting the sharpening enhancement image produced from performing the sharpening enhancement step; 3. The medical device according to item 2, wherein the sharpened enhanced image is fused with the processed image. [Item 4] The demosaicing instructions stored on the non-transitory computer-readable medium cause the processor to: setting a luminance value for each of the plurality of second pixel values ​​and the plurality of third pixel values; 4. The medical device of item 3, wherein contrast enhancement of the plurality of second pixel values ​​and the plurality of third pixel values ​​is performed by adjusting the brightness values ​​to increase the contrast of the processed image. [Item 5] The demosaicing instructions stored on the non-transitory computer-readable medium cause the processor to: outputting a contrast-enhanced image produced from said performing contrast enhancement; 5. The medical device according to item 4, wherein the contrast-enhanced image is fused with the processed image. [Item 6] The demosaicing instructions stored on the non-transitory computer-readable medium cause the processor to: Item 6. The medical device according to item 5, wherein all of the above steps are repeated up to a threshold value. [Item 7] The demosaicing instructions stored on the non-transitory computer-readable medium cause the processor to: 7. The medical device of any one of items 1 to 6, wherein a wavelength shift input is received to identify color pixel values ​​of the plurality of first pixel values. [Item 8] Item 8. The medical device of item 7, further comprising a user interface communicatively coupled to the processor and configured to transmit a signal indicative of the wavelength shift input to the processor. [Item 9] the sensor comprises an RGB image sensor, and the filter array comprises a red-green-blue Bayer color filter array; A medical device described in any one of items 1 to 8, wherein the plurality of first pixel values ​​include red pixel values, the plurality of second pixel values ​​include blue pixel values, and the plurality of third pixel values ​​include green pixel values. [Item 10] 10. The medical device according to any one of items 1 to 9, wherein the sensor includes an RGB+Ir image sensor and the filter array includes a red-green-blue-infrared Bayer color filter array. [Item 11] the sensor includes an RGB+Ir image sensor, and the filter array includes a red-green-blue-infrared Bayer color filter array; The medical device of any one of items 1 to 8, wherein the plurality of first pixel values ​​include blue pixel values, the plurality of second pixel values ​​include red pixel values ​​and green pixel values, and the plurality of third pixel values ​​include infrared pixel values. [Item 12] 12. The medical device according to any one of items 1 to 11, wherein the sensor includes an RGB image sensor and a monochrome sensor. [Item 13] A medical device described in any one of items 1 to 12, wherein each location within the partially resolved image frame includes one captured color pixel value and one reconstructed color pixel value such that at least one color pixel value is excluded from the frame of original pixel values. [Item 14] 14. The medical device of any one of items 1 to 13, further comprising a light source coupled to the distal end of the shaft, the light source being an optical fiber or a multicolor LED array. [Item 15] The demosaicing instructions stored on the non-transitory computer-readable medium cause the processor to: 15. The medical device according to any one of items 1 to 14, wherein the processed image of the partially resolved image frame is output to a display device. Each of the above-described systems, devices, assemblies, and methods can be used to generate sub-resolution frames of pixel values ​​of a subject. By providing a medical device including an image processing system storing wavelength-shifting demosaicing logic, a user may better visualize one or more features and / or characteristics of a target site within a subject's body during treatment without manipulating a light source. The medical device may enable a user to accurately locate the target site, thereby reducing overall treatment time, improving treatment efficiency, and avoiding unnecessary harm to the subject's body that may result from inaccurate localization of a target object at a target treatment site.

[0070] It will be apparent to those skilled in the art that various modifications and variations can be made in the disclosed apparatus and methods without departing from the scope of the present disclosure. It should be recognized that the disclosed apparatus can include various suitable computer systems and / or computing units incorporating multiple hardware components, such as a processor and non-transitory computer-readable media that enable the apparatus to perform one or more operations during processing in accordance with those described herein. Other aspects of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features disclosed therein. The specification and examples should be considered exemplary only.

Claims

1. 1. A method for processing an image, comprising: The method comprises receiving an image frame, the image frame including a plurality of first pixel values ​​at a plurality of first locations on the image frame, a plurality of second pixel values ​​at a plurality of second locations on the image frame, and a plurality of third pixel values ​​at a plurality of third locations on the image frame; The method comprises removing the first plurality of pixel values ​​from the image frame; The method comprises determining a plurality of second predicted pixel values ​​at the plurality of first locations and the plurality of third locations on the image frame; The method comprises determining a third plurality of predicted pixel values ​​at the first plurality of locations and the second plurality of locations on the image frame; The method comprises generating a processed image frame based on the plurality of second pixel values, the plurality of second predicted pixel values, the plurality of third pixel values, and the plurality of third predicted pixel values.

2. receiving a wavelength-shifted input; 10. The method of claim 1, further comprising: excluding the plurality of first pixel values ​​from the image frame based on a received wavelength shift input and a first color associated with the plurality of first pixel values ​​that corresponds to the wavelength shift input.

3. 3. The method of claim 2, wherein the image frame is of a target region, the target region including at least a first feature and a second feature having a color substantially similar to the first color.

4. Determining the second predicted pixel values ​​at the first locations and the third locations on the image frame includes: interpolating from one or more of the second pixel values ​​at one or more of the second locations adjacent to each first location of the plurality of first locations to determine a corresponding second predicted pixel value at each first location of the plurality of first locations; interpolating from one or more of the second pixel values ​​at one or more of the second locations adjacent to each third location of the plurality of third locations to determine a corresponding predicted second pixel value at each third location of the plurality of third locations; The method of claim 1 , comprising:

5. Determining the third predicted pixel values ​​at the first locations and the second locations on the image frame includes: interpolating from one or more third pixel values ​​at one or more third locations adjacent each first location of the plurality of first locations to determine a corresponding third predicted pixel value at each first location of the plurality of first locations; interpolating from one or more third pixel values ​​at one or more third locations adjacent to each second location of the plurality of second locations to determine a corresponding predicted third pixel value at each second location of the plurality of second locations; The method of claim 1 , comprising:

6. enhancing sharpness of one or more edges detected at one or more of the plurality of second locations or the plurality of third locations to generate a sharpened image frame; fusing the sharpened image frame with the processed image frame. The method of claim 1 further comprising:

7. adjusting a luminance component of one or more of the second plurality of pixel values ​​or one or more of the third plurality of pixel values ​​to generate a contrast-enhanced image frame; fusing the contrast-enhanced image frame with the processed image frame; The method of claim 1 further comprising:

8. generating a sharpened image frame from the image frame; generating a contrast-enhanced image frame from the image frame; generating a partial resolution image frame based on the processed image frame, the sharpened image frame, and the contrast enhanced image frame; The method of claim 1 further comprising:

9. determining that a current iteration of image processing has reached a threshold based on one or more of the image frame, the processed image frame, the sharpened image frame, or the contrast enhanced image frame; generating the partially resolved image frame in response to the determination; and The method of claim 8 further comprising:

10. 2. The method of claim 1, wherein the plurality of first pixel values ​​corresponds to a first color, the plurality of second pixel values ​​corresponds to a second color different from the first color, and the plurality of third pixel values ​​corresponds to a third color different from the first color and the second color.

11. 11. The method of claim 10, wherein the image frames are captured by and received from a sensor including a Bayer color filter array, and the first color, the second color, and the third color correspond to either red, green, or blue.

12. 1. A method for processing an image, the method comprising: receiving an image including a plurality of pixels having a plurality of imaged pixel values ​​at a plurality of locations; removing from the image a first fraction of the plurality of imaged pixel values ​​for a first fraction of a plurality of pixels at a first fraction of the plurality of locations; estimating a plurality of unknown pixel values ​​for a remaining second fraction of the plurality of pixels in the first fraction of the plurality of locations; generating a processed image using a second subset of the captured pixel values ​​for the remaining second subset of the pixels at the first subset of the locations and the estimated unknown pixel values ​​for the remaining second subset of the pixels at the first subset of the locations; A method comprising:

13. 13. The method of claim 12, wherein the first subset of the plurality of pixels includes a first color pixel at the first subset of the plurality of locations in the image, and the second subset of the plurality of pixels includes a second color pixel at a second subset of the plurality of locations in the image and a third color pixel at a third subset of the plurality of locations in the image.

14. estimating a plurality of unknown pixel values ​​for the second color pixels in the third sub-portion of the plurality of locations within the image; estimating a plurality of unknown pixel values ​​for the third color pixels in the second subset of the plurality of locations within the image; further generating the processed image based on the plurality of unknown pixel values ​​estimated for the second color pixels in the third subset of the plurality of locations within the image and the plurality of unknown pixel values ​​estimated for the third color pixels in the second subset of the plurality of locations within the image; The method of claim 13 further comprising:

15. 14. The method of claim 13, wherein the image is captured by and received from a sensor including a Bayer color filter array, and the first color pixel, the second color pixel, and the third color pixel correspond to one of red, green, or blue.

16. generating one or more of a sharpened image or a contrast-enhanced image from said image; generating a partial resolution image based on the processed image and one or more of the sharpened image or the contrast enhanced image; The method of claim 12 further comprising:

17. generating the sharpened image from the image, 17. The method of claim 16, comprising increasing a sharpness of one or more edges detected at one or more of the plurality of locations associated with the remaining second fraction of the plurality of pixels to generate a sharpened image frame.

18. Generating the contrast-enhanced image from the image includes: adjusting a luminance component for one or more of the remaining second subset of the plurality of pixels to generate a contrast-enhanced image frame; The method of claim 16, comprising:

19. 1. A method for processing an image, the method comprising: receiving an image comprising a plurality of captured pixel values; filtering out a first fraction of the plurality of imaged pixel values ​​from the image; generating a plurality of predicted pixel values; generating a processed image using a remaining second fraction of the plurality of captured pixel values ​​and the plurality of predicted pixel values; removing the first fraction of the plurality of imaged pixel values ​​that are filtered out; and A method comprising:

20. generating a sharpened image from the image; generating a contrast-enhanced image from said image; generating a partial resolution image based on the processed image, the sharpened image, and the contrast-enhanced image; 20. The method of claim 19 further comprising:

Images