Image processing method, image processing device, image processing program, and endoscope system

Abstract

To provide an image processing technology that improves resolution or visibility of obtained images.SOLUTION: An image processing device (10) generates a super-resolution image by acquiring a plurality of object images obtained through imaging of one end of an image guide (13q) which includes a plurality of cores and in which the other end faces an object (O), and by obtaining a simple average or a weighted average of pixel values of pixels corresponding to the same point of the object (O) in the plurality of object images.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an image processing method, an image processing apparatus, and an image processing program for processing an image captured using an image guide. The present invention also relates to an endoscope system including such an image processing apparatus.

Background Art

[0002] An endoscope system that obtains an image including an object in the body as a subject by imaging the other end of an image guide having one end inserted into the body is widely used. For example, Patent Document 1 discloses an intravascular endoscope system in which the risk of damage to the inner wall of a small-diameter blood vessel is improved by reducing the diameter and increasing the flexibility of the image guide.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In an endoscope system using an image guide, the resolution is determined by the core density of the image guide. For example, a structure having a size of about 100 times the core pitch can be visually recognized from the image formed on the end face of the image guide, but a structure having a size of about 10 times the core pitch is difficult to visually recognize from the image formed on the end face of the image guide, and a structure having a size comparable to the core pitch cannot be visually recognized from the image formed on the end face of the image guide.

[0005] In the endovascular endoscope system described in Patent Document 1, the image quality of the obtained image is improved by correcting the differences in brightness and color tone for each pixel caused by the differences in the transmission characteristics of each core of the image guide. However, such correction cannot exceed the resolution determined by the core density of the image guide.

[0006] One aspect of the present invention has been made in view of the above problems, and an object thereof is to provide an image processing technique that improves the resolution or visibility of the obtained image.

Means for Solving the Problems

[0007] An image processing method according to one aspect of the present invention includes an acquisition process of acquiring a plurality of target images obtained by imaging the other end of an image guide having a plurality of cores and having one end facing an object, and a super-resolution process of generating a super-resolution image by simply averaging or weighted-averaging the pixel values of the pixels corresponding to the same point of the object in the plurality of target images.

[0008] An image processing apparatus according to one aspect of the present invention is an image processing apparatus including at least one processor, and the processor executes an acquisition process of acquiring a plurality of target images obtained by imaging the other end of an image guide having a plurality of cores and having one end facing an object, and a super-resolution process of generating a super-resolution image by averaging or weighted-averaging the pixel values of the pixels corresponding to the same point of the object in the plurality of target images.

[0009] An image processing program according to one aspect of the present invention is an image processing program for operating at least one processor, and causes the processor to execute an acquisition process of acquiring a plurality of target images obtained by imaging the other end of an image guide having a plurality of cores and having one end facing an object, and a super-resolution process of generating a super-resolution image by averaging or weighted-averaging the pixel values of the pixels corresponding to the same point of the object in the plurality of target images.

Effects of the Invention

[0010] According to one aspect of the present invention, the resolution or visibility of the obtained image can be improved.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Mode for Carrying Out the Invention

[0012] (Configuration of Endoscope System) An endoscope system 1 according to an embodiment of the present invention will be described with reference to FIG. 1. In FIG. 1, (a) is a schematic diagram showing the configuration of the endoscope system 1, (b) is a plan view and a cross-sectional view showing the structure of the light guide 13p in the light guide plug portion 13a, (c) is a plan view and a cross-sectional view showing the structure of the image guide 13q in the image guide plug portion 13c, and (d) is a plan view and a cross-sectional view showing the structures of the light guide 13p and the image guide 13q in the insertion portion 13g.

[0013] The endoscope system 1 is a system for displaying an image of an object O to be observed on a display D. As shown in FIG. 1(a), the endoscope system 1 includes an image processing apparatus 10, a light source apparatus 11, a camera apparatus 12, and an endoscope probe 13.

[0014] As shown in FIG. 1(a), the light source apparatus 11 includes a light source 11a and a condenser lens 11b. The light source 11a is disposed on the optical axis of the condenser lens 11b. As the light source 11a, for example, an LED (Light Emitting Diode), a xenon lamp, a halogen lamp, or the like is used.

[0015] As shown in FIG. 1(a), the camera apparatus 12 includes an image sensor 12a, an objective lens 12b, and a signal processing circuit 12c. The image sensor 12a is disposed on the optical axis of the objective lens 12b. The signal processing circuit 12c is electrically connected to the image sensor 12a. As the image sensor 12a, for example, a CMOS (Complementary Metal Oxide Semiconductor) sensor, a CCD (Charge Coupled Device) sensor, or the like is used.

[0016] As shown in Fig. 1(a), the endoscope probe 13 includes: (1) a light guide plug portion 13a mechanically connected to the light source device 11; (2) a light guide lead portion 13b having one end connected to the light guide plug portion 13a; (3) an image guide plug portion 13c mechanically connected to the camera device 12; (4) an image guide lead portion 13d having one end connected to the image guide plug portion 13c; (5) a branch portion 13e having one end connected to the other ends of the light guide lead portion 13b and the image guide lead portion 13d; (6) an intermediate portion 13f having one end connected to the other end of the branch portion 13e; and (7) an insertion portion 13g having one end connected to the other end of the intermediate portion 13f. The endoscope probe 13 may further include a water passage lumen, a working lumen, an insertion portion bending mechanism, and a bending operation portion (all not shown).

[0017] As shown in Figs. 1(b) and 1(d), one or more light guides 13p that function as waveguides for guiding the illumination light generated by the light source device 11 are provided inside the light guide plug portion 13a, the light guide lead portion 13b, the branch portion 13e, the intermediate portion 13f, and the insertion portion 13g. Each light guide 13p is embedded in the cladding. The light guides 13p included in the light guide plug portion 13a and the light guide lead portion 13b are bundled with an image guide 13q described later in the branch portion 13e, and are included in the intermediate portion 13f and the insertion portion 13g together with this image guide 13q.

[0018] Further, as shown in FIGS. 1(c) and 1(d), an image guide 13q, which is a waveguide for guiding scattered light generated by the object O, is provided inside the image guide plug portion 13c, the image guide lead portion 13d, the branching portion 13e, the intermediate portion 13f, and the insertion portion 13g. The image guide 13q is composed of a cladding and a plurality of cores (for example, several thousand to several tens of thousands of cores) regularly or randomly arranged inside the cladding. The image guide 13q included in the image guide plug portion 13c and the image guide lead portion 13d is bundled with the light guide 13p described above in the branching portion 13e, and is included in the intermediate portion 13f and the insertion portion 13g together with this light guide 13p.

[0019] As the image guide 13q, for example, an image guide fiber, an image fiber, a fiber bundle, a fiber optic can be used. The material of the image guide 13q may be any of a quartz-based, multi-component glass-based, or plastic-based. Also, the number of cores and the core arrangement of the image guide 13q are not particularly limited. As an example, an image guide fiber in which 3000 cores are arranged on the lattice points of a hexagonal close-packed lattice can be used as the image guide 13q. However, the resolution of the image guide 13q is lower than the resolution of the image sensor 12a, that is, the number of cores of the image guide 13q is less than the number of cells of the image sensor 12a.

[0020] Further, as shown in FIG. 1(d), an objective lens 13r is provided at the tip of the insertion portion 13g. One end of the objective lens 13r constitutes the end face of the insertion portion 13g, and the other end of the objective lens portion 13s faces one end of the image guide 13q. As the objective lens 13r, for example, a GRIN lens or a lens unit composed of a plurality of lenses can be used.

[0021] The endoscope system 1 operates as follows, for example. That is, illumination light is emitted from the light source 11a with the insertion portion 13g inserted into the patient's body. The illumination light emitted from the light source 11a is condensed by the condenser lens 11b onto the end face on the light guide plug portion 13a side of the light guide 13p. The illumination light incident on the core of the light guide 13p at the end face on the light guide plug portion 13a side of the light guide 13p is guided through the core of the light guide 13p and exits from the core of the light guide 13p at the end face on the insertion portion 13g side of the light guide 13p. The illumination light exiting from the core of the light guide 13p at the end face on the insertion portion 13g side of the light guide 13p irradiates the object O in the patient's body.

[0022] The scattered light generated at each point of the object O by scattering the illumination light in the patient's body is condensed by the objective lens 13r onto each point of the end face on the insertion portion 13g side of the image guide 13q. As a result, an image of the object O is formed on the end face on the insertion portion 13g side of the image guide 13q. The scattered light incident on the core of the image guide 13q at the end face on the insertion portion 13g side of the image guide 13q by the objective lens 13r is guided through the core of the image guide 13q and exits from the core of the image guide 13q at the end face on the image guide plug portion 13c side of the image guide 13q. As a result, an image of the object O is also formed on the end face on the image guide plug portion 13c side of the image guide 13q, similar to the end face on the insertion portion 13g side of the image guide 13q.

[0023] The light (light derived from the scattered light generated at each point of the object O) exiting from each core of the image guide 13q at the end face on the image guide plug portion 13c side of the image guide 13q is condensed by the objective lens 12b onto each point of the light receiving surface of the image sensor 12a. As a result, an image of the object O is formed on the light receiving surface of the image sensor 12a. The image sensor 12a generates an electrical signal representing the image of the object O and inputs the generated electrical signal to the signal processing circuit 12c. The signal processing circuit 12c generates a target image representing the image of the object O from the electrical signal input from the image sensor 12a.

[0024] The camera device 12 inputs the generated target image into the image processing device 10. The image processing device 10 generates an output image to be displayed on the display D from the target image input from the camera device 12 according to an image processing method S100 described later. Note that the functions performed by the image processing device 10 may be performed by the camera device 12. That is, instead of the configuration in which the image processing device 10 performs the image processing method S100 described later, a configuration in which the camera device 12 performs it may be adopted. In this case, the image processing device 10 can be omitted from the endoscope system 1.

[0025] In this specification, the "image" refers to a two-dimensional array of pixel values. The pixel value of a monochrome image consists of, for example, a single numerical value corresponding to luminance. In a D-bit monochrome image, the luminance takes an integer value of 0 or more and D - 1 or less. The pixel value in a color image consists of, for example, three numerical values corresponding to RGB. Here, R represents the red component, G represents the green component, and B represents the blue component. In a D-bit color image, the red component, the green component, and the blue component each take an integer value of 0 or more and D - 1 or less. The pixel value in a transmissive color image consists of, for example, four numerical values corresponding to RGBA. Here, A represents the opacity. In the case of a D-bit transmissive color image, the red component, the green component, the blue component, and the opacity each take an integer value of 0 or more and D - 1 or less.

[0026] Pixels in an image are identified by coordinates (x, y). When the size of the image is W pixels wide × H pixels high, x is an integer from 0 to W - 1, and y is an integer from 0 to H - 1. A pixel with coordinates (x, y) will hereinafter also be referred to as pixel (x, y). For example, pixel (0, 0) is the pixel in the upper left corner of the image, pixel (1, 0) is the pixel immediately to the right of pixel (0, 0), and pixel (0, 1) is the pixel immediately below pixel (0, 0). The pixel value (luminance value) of pixel (x, y) in a monochrome image is denoted as A(x, y). Also, for the pixel value of pixel (x, y) in a color image, its red component is denoted as R(x, y), its green component as G(x, y), and its blue component as B(x, y). Similarly, for the pixel value of pixel (x, y) in a color image with opacity, its red component is denoted as R(x, y), its green component as G(x, y), its blue component as B(x, y), and its opacity as A(x, y).

[0027] The file format for saving an image is not particularly limited. For example, using file formats such as PNG, JPEG, or GIF, it is possible to save a monochrome image or a color image. Also, using a file format such as a transparent PNG, it is possible to save a transparent color image.

[0028] (Configuration of the Image Processing Apparatus) The configuration of the image processing apparatus 10 will be described with reference to FIG. 2. FIG. 2 is a block diagram showing the configuration of the image processing apparatus 10.

[0029] As shown in FIG. 2, the image processing apparatus 10 includes a processor 101, a memory 102, a storage 103, an input / output interface 104, and a bus 105. The processor 101, the memory 102, the storage 103, and the input / output interface 104 are connected to each other via the bus 105.

[0030] The memory 102 is configured to expand and store an image processing program P100 for implementing an image processing method S100, which will be described later, in a state where it can be referred to by the processor 101. The memory 102 is also used to store various images acquired from the camera device 12. Note that, for example, a semiconductor RAM (Random Access Memory) or the like can be used as the memory 102.

[0031] The processor 101 is configured to implement the image processing method S100, which will be described later, according to the image processing program P100 expanded in the memory 102. As the processor 101, for example, a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), or a combination thereof can be used.

[0032] The storage 103 is configured to store (non-volatile storage) the image processing program P100. When implementing the image processing method S100, which will be described later, the processor 101 expands and refers to the image processing program P100 stored in the storage 103 on the memory 102. Note that, for example, a flash memory, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a combination thereof can be used as the storage 103.

[0033] The input / output interface 104 is configured to connect the camera device 12 and the display D to the image processing apparatus 10, respectively. Note that, for example, a USB (Universal Serial Bus) interface or the like can be used as the input / output interface 104.

[0034] Here, although the image processing method S100 described below is configured to be executed by a single processor 101 provided in a single computer (a computer functioning as the image processing apparatus 10), the present invention is not limited thereto. That is, it is also possible to adopt a configuration in which a plurality of processors provided concentratedly in a single computer or distributedly in a plurality of computers cooperate to execute the image processing method S100 described below.

[0035] Note that the image processing program P100 can be recorded on a non-transitory tangible computer-readable recording medium. This recording medium may be the memory 102, the storage 103, or other recording media. For example, a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, etc. can be used as other recording media.

[0036] (Flow of the image processing method) The flow of the image processing method S100 according to an embodiment of the present invention will be described with reference to FIG. 3. FIG. 3 is a flowchart showing the flow of the image processing method S100.

[0037] As shown in FIG. 3, the image processing method S100 includes a white reference plate imaging process S101, a white balance adjustment process S102, an object imaging process S103, a target image acquisition process S104, a super-resolution process S105, a mask process S106, and an image display process S107. Here, the white reference plate imaging process S101, the white balance adjustment process S102, and the object imaging process S103 are processes mainly implemented by the camera device 12. On the other hand, the target image acquisition process S104, the super-resolution process S105, the mask process S106, and the image display process S107 are processes mainly implemented by the image processing apparatus 10.

[0038] The white standard plate imaging process S101 is a process in which the user makes the objective lens 13r of the endoscope probe 13 face the white standard plate, and the camera device 12 images the end face on the image guide plug portion 13c side of the image guide 13q. The white balance adjustment process S102 is a process in which the camera device 12 adjusts the white balance with reference to the image captured in the white standard plate imaging process S101. The image captured in the white standard plate imaging process S101 and referred to in the white balance adjustment process S102 is a color image in which the area corresponding to the core of the image guide 13q is bright and the other areas are dark. This image will be hereinafter referred to as the mesh image I0. Regarding a specific example of the mesh image I0 obtained in the white standard plate imaging process S101, it will be described later with reference to different drawings.

[0039] The object imaging process S103 is a process in which the user makes the objective lens 13r of the endoscope probe 13 face the object O, and the camera device 12 images the end face on the image guide plug portion 13c side of the image guide 13q. The target image acquisition process S104 is a process in which the image processing device 10 acquires the image captured in the object imaging process S103 from the camera device 12 and stores it in the memory 102. The object imaging process S103 and the target image acquisition process S104 are repeated n times (n is a predetermined natural number). The image captured in the object imaging process S103 and acquired in the target image acquisition process S104 is a color image representing the image of the object O formed on the end face on the image guide plug portion 13c side of the image guide 13q. These images will be hereinafter referred to as the target images I1, I2,..., In. Regarding specific examples of the target images I1, I2,..., In obtained in the object imaging process S103, they will be described later with reference to different drawings.

[0040] The super-resolution process S105 is a process of generating a super-resolution image I by averaging the pixel values of the pixels corresponding to the same point of the object O in the target images I1, I2,..., In. In the present embodiment, the super-resolution process S105 is composed of a registration process S105a and an averaging process S105b.

[0041] The alignment process S105a is a process in which the image processing apparatus 10 calculates, for each target image Ii (where i is each natural number from 1 to n−1) other than the target image In that was last acquired, (1) the displacement vector vi of the target image Ii with respect to the target image In, and (2) shifts the pixel values of each pixel of the target image Ii by the displacement vector vi. As a method for calculating the displacement vector vi, for example, a known method that uses the feature amounts of the target images Ii and In can be used. The image obtained by the alignment process S105a for the target image Ii is a color image representing an image of the object O formed on the end face on the image guide plug portion 13c side of the image guide 13q. This image is hereinafter referred to as the target image Ii'.

[0042] When the pixel (xi, yi) of the target image Ii and the pixel (xn, yn) of the target image In correspond to the same point of the subject, the displacement vector vi = (vix, viy) is given, for example, by the following formula.

[0043] vix = xi − xn, viy = yi − yn.

[0044] Also, when the pixel value at the pixel (x, y) of the target image Ii is given by {Ri(x, y), Gi(x, y), Bi(x, y)}, the pixel value {Ri'(x, y), Gi'(x, y), Bi'(x, y)} at the pixel (x, y) of the target image Ii' is given, for example, by the following formula.

[0045] When 0 ≤ x + vix ≤ W and 0 ≤ y + viy ≤ H: Ri'(x, y) = Ri(x + vix, y + viy), Gi'(x, y) = Gi(x + vix, y + viy), Bi'(x, y) = Bi(x + vix, y + viy), When x + vix < 0, W < x + vix, y + viy < 0, or H < y + viy: Ri'(x, y) = 0, Gi'(x, y) = 0, Bi'(x, y) = 0.

[0046] Note that for the last acquired target image In, the alignment process S105a is not performed. However, hereinafter, for convenience of explanation, the target image In is also referred to as the target image In'. That is, the pixel values {Rn'(x, y), Gn'(x, y), Bn'(x, y)} at the pixel (x, y) of the target image In' are the same as the pixel values {Rn(x, y), Gn(x, y), Bn(x, y)} at the pixel (x, y) of the target image In. Specific examples of the target images I1', I2',..., In' obtained by the alignment process S105a will be described later with reference to different drawings.

[0047] Here, a configuration has been described in which the pixel values of each target image Ii other than the reference image are shifted with the last acquired target image In as the reference image. However, the present invention is not limited to this. That is, a configuration may be adopted in which a target image Ip (p is a predetermined natural number from 1 to n - 1) other than the last acquired target image In is used as the reference image, and the pixel values of each target image Iq (q is each natural number from 1 to n different from p) other than the reference image are shifted.

[0048] The averaging process S105b is a process in which the image processing apparatus 10 simply averages the pixel values {Rj'(x, y), Gj'(x, y), Bj'(x, y)} of each target image Ij' (j is each natural number from 1 to n) for each pixel (x, y). The image obtained by the averaging process S105b is a color image representing an image of the object O formed on the end face on the image guide plug portion 13c side of the image guide 13q. Hereinafter, this image is referred to as the super-resolution image I. The pixel values {R(x, y), G(x, y), B(x, y)} at the pixel (x, y) of the super-resolution image I are given by, for example, the following equations. Specific examples of the super-resolution image I generated by the super-resolution process S105 will be described later with reference to different drawings.

[0049] R(x, y) = {R1'(x, y) + R2'(x, y) +... + Rn'(x, y)} / n, G(x, y) = {G1'(x, y) + G2'(x, y) +... + Gn'(x, y)} / n, B(x,y) = {B1’(x,y) + B2’(x,y) + … + Bn’(x,y)} / n。

[0050] The masking process S106 is a process in which the image processing apparatus 10 generates an output image I’ by masking an area outside the field of view circle in the super-resolution image I. Here, masking the area outside the field of view circle means, for example, setting the pixel values {R(x,y), G(x,y), B(x,y)} of the pixels (x,y) located outside the field of view circle in the super-resolution image I to R(x,y) = 0, G(x,y) = 0, and B(x,y) = 0. A specific example of the output image I’ generated in the masking process S106 will be described later with reference to different drawings.

[0051] The image display process S107 is a process in which the image processing apparatus 10 displays the output image I’ on the display D. Thereby, a series of processes of the image processing method S100 is completed.

[0052] Here, the image processing method S100 for displaying a still image of the object O on the display D has been described. However, this method may be used to display a moving image of the object O on the display D. In this case, the camera device 12 repeatedly executes the object imaging process S103 at a period corresponding to the frame rate, and the image processing apparatus 10 executes the super-resolution process S105, the masking process S106, and the image display process S107 on the latest n object images among the object images captured by the camera device 12 so far. Thereby, a moving image of the object O can be displayed on the display D.

[0053] Note that, as a pre - processing of the super - resolution process S105, deformation correction such as affine transformation may be performed on each of the target images I1, I2, …, In. When there are differences in the position and shape of the core images included in the target images I1, I2, …, In due to differences in the skew angles during imaging, by eliminating these differences in advance through deformation correction, a super - resolution image I with higher visibility can be obtained. Also, as a pre - processing of the super - resolution process S105, color unevenness correction and luminance unevenness correction described in Japanese Patent Application Laid - Open No. 2021 - 177604 (Patent No. 7189173) may be performed on each of the target images I1, I2, …, In. When there are color unevenness and luminance unevenness for each core image included in the target images I1, I2, …, In, by eliminating these in advance through color unevenness correction and luminance unevenness correction, a super - resolution image I with higher visibility can be obtained.

[0054] (Execution example of the image processing method) An execution example of the image processing method S100 will be described with reference to FIGS. 4 to 11. In this specific example, for simplicity, the number n of the target images I1, I2, …, In is set to 4.

[0055] FIG. 4 is a plan view of the object O used as a sample in this execution example. The object O used as a sample in this execution example is a white standard plate on which alphabets from A to Z and numbers from 0 to 9 are printed.

[0056] FIG. 5 is a diagram illustrating the mesh image I0 obtained in the white standard plate imaging process S101. As described above, the mesh image I0 is an image in which the area corresponding to the core of the image guide 13q is bright and the other areas are dark.

[0057] FIG. 6 is a diagram illustrating a target image I1 obtained in the target object imaging process S103 and a target image I1' obtained in the alignment process S105a. The positions of the characters and alphabets included as subjects in the target image I1 are closer to the lower right than the positions of the characters and alphabets included as subjects in the target image I4 used as the reference image. Therefore, as shown by the white arrow in FIG. 6, the misregistration vector v1 points to the lower right. The positions of the characters and alphabets included as subjects in the target image I1' obtained by shifting the pixel values of each pixel of the target image I1 by the misregistration vector v1 coincide or substantially coincide with the positions of the characters and alphabets included as subjects in the target image I4 used as the reference image.

[0058] FIG. 7 is a diagram illustrating a target image I2 obtained in the target object imaging process S103 and a target image I2' obtained in the alignment process S105a. The positions of the characters and alphabets included as subjects in the target image I2 are closer to the left than the positions of the characters and alphabets included as subjects in the target image I4 used as the reference image. Therefore, as shown by the white arrow in FIG. 7, the misregistration vector v2 points to the left. The positions of the characters and alphabets included as subjects in the target image I2' obtained by shifting the pixel values of each pixel of the target image I2 by the misregistration vector v2 coincide or substantially coincide with the positions of the characters and alphabets included as subjects in the target image I4 used as the reference image.

[0059] FIG. 8 is a diagram illustrating a target image I3 obtained in the target object imaging process S103 and a target image I3' obtained in the alignment process S105a. The positions of the characters and alphabets included as subjects in the target image I3 are closer to the upper right than the positions of the characters and alphabets included as subjects in the target image I4 used as the reference image. Therefore, as shown by the white arrow in FIG. 8, the displacement vector v3 points to the upper right. By shifting the pixel values of each pixel of the target image I3 by the displacement vector v3, the positions of the characters and alphabets included as subjects in the target image I3' obtained are the same as or substantially the same as the positions of the characters and alphabets included as subjects in the target image I4 used as the reference image.

[0060] FIG. 9 is a diagram illustrating a target image I4 obtained in the target object imaging process S103. Since the target image I4 is used as the reference image in the alignment process S105a, the alignment process S105a for the target image I4 is not executed.

[0061] FIG. 10 is a diagram illustrating a super-resolution image I obtained in the super-resolution process S105. In the target images I1 to I4, it is difficult to read the alphabets and characters included as subjects. On the other hand, in the super-resolution image I, it is easy to read the alphabets and characters included as subjects. That is, it was confirmed that by executing the super-resolution process S105, a super-resolution image I with higher visibility than the target images I1 to I4 can be obtained.

[0062] FIG. 11 is a diagram illustrating an output image I' obtained in the masking process S106. In the super-resolution image I, it is possible to visually recognize jaggies (the jaggedness caused by the image guide 13q being composed of a plurality of cores) at the boundary of the end face of the image guide 13q included as a subject. On the other hand, in the output image I', it is not possible to visually recognize jaggies at the boundary of the end face of the image guide 13q included as a subject. That is, it was confirmed that by executing the masking process S106, an output image I' with fewer jaggies than the super-resolution image I can be obtained.

[0063] (Modification example of the image processing method) A modification example of the image processing method S100 will be described with reference to FIG. 12. FIG. 12 is a flowchart showing the flow of the image processing method S100' according to this modification example.

[0064] The image processing method S100' according to this modification example is obtained by (1) adding a white standard plate imaging process S110, a mesh image acquisition process S111, and a transparency process S112 to the above-described image processing method S100, and (2) replacing the average process S105b that takes a simple average included in the above-described super-resolution process S105 with an average process S105c that takes a weighted average.

[0065] The white standard plate imaging process S110 and the mesh image acquisition process S111 are processes executed between the white balance adjustment process S102 and the object imaging process S103 described above.

[0066] The white standard plate imaging process S110 is a process in which the user makes the objective lens 13r of the endoscope probe 13 face the white standard plate, and the camera device 12 images the end face on the image guide plug portion 13c side of the image guide 13q. The mesh image acquisition process S111 is a process in which the image processing device 10 acquires the image captured in the white standard plate imaging process S110 from the camera device 12 and stores it in the memory 102. The image captured in the white standard plate imaging process S101 and acquired in the mesh image acquisition process S111 is a color image in which the region corresponding to the core of the image guide 13q is bright and the other regions are dark. The image processing device 10 uses this color image after converting it to grayscale. The monochrome image obtained by converting this color image to grayscale is hereinafter referred to as the mesh image I0'. The difference from the above-described mesh image I0 is that the mesh image I0 is a color image captured before white balance adjustment, while the mesh image I0' is a monochrome image obtained by converting a color image captured after white balance adjustment to grayscale.

[0067] The transparency process S112 and the core drawing process S113 are pre - processes of the super - resolution process S105, which are executed between the above - mentioned target image acquisition process S104 and the super - resolution process S105.

[0068] In the transparency process S112, for each target image Ij (where j is each natural number from 1 to n), the image processing apparatus 10 refers to the mesh image I0' captured in the white standard plate imaging process S110, and performs a process of making transparent or semi - transparent the areas other than the area corresponding to the core of the image guide 13q in the target image Ij (that is, the areas corresponding to the mesh and the background in the mesh image I0'). The image generated by the transparency process S112 is a color image with opacity, where the opacity is high in the area corresponding to the core of the image guide 13q and low in other areas. This image is hereinafter referred to as the "transparent target image Ij". The pixel value at the pixel (x, y) of the transparent target image Ij is, for example, the pixel value {Rj(x, y), Gj(x, y), Bj(x, y)} at the pixel (x, y) of the target image Ij, with the pixel value A(x, y) at the pixel (x, y) of the mesh image I0' added as the opaque component. That is, the pixel value at the pixel (x, y) of the transparent target image Ij is {Rj(x, y), Gj(x, y), Bj(x, y), A(x, y)}.

[0069] In the alignment process S105a of the super-resolution process S105, for each transparent target image Ii (where i is each natural number from 1 to n - 1) other than the transparent target image In, the image processing apparatus 10 calculates (1) the displacement vector vi of the transparent target image Ii with respect to the transparent target image In, and (2) shifts the pixel values of each pixel of the transparent target image Ii by the displacement vector vi. The image obtained by the alignment process S105a for the transparent target image Ii is an opaque color image representing the image of the object O formed on the end face on the image guide plug portion 13c side of the image guide 13q. This image is hereinafter referred to as the transparent target image Ii'. Note that for the transparent target image In, the alignment process S105a is not performed, but hereinafter, for convenience of explanation, the transparent target image In is also referred to as the transparent target image In'. Specific examples of the transparent target images I1', I2',..., In' obtained by the alignment process S105a will be described later with reference to different drawings.

[0070] In the averaging process S105c of the super-resolution process S105, for each pixel (x, y), the image processing apparatus 10 generates a super-resolution image I by weighted-averaging the pixel values {Rj'(x, y), Gj'(x, y), Bj'(x, y), Aj'(x, y)} of each transparent target image Ij' for each component. The opacity Aj'(x, y) is used as the weight in the weighted-averaging. The image obtained by the averaging process S105c is an opaque color image representing the image of the object O formed on the end face on the image guide plug portion 13c side of the image guide 13q. This image is hereinafter referred to as the super-resolution image I. The pixel value {R(x, y), G(x, y), B(x, y), A(x, y)} at the pixel (x, y) of the super-resolution image I is given, for example, by the following formula. Specific examples of the super-resolution image I generated by the averaging process S105c will be described later with reference to different drawings.

[0071] D(x, y) = A1'(x, y) + A2'(x, y) +... + An'(x, y), When D(x, y) ≠ 0: R(x, y) = {R1'(x, y)A1'(x, y) + R2'(x, y)A2'(x, y) + … + Rn'(x, y)An'(x, y)} / D(x, y), G(x, y) = {G1'(x, y)A1'(x, y) + G2'(x, y)A2'(x, y) + … + Gn'(x, y)An'(x, y)} / D(x, y), B(x, y) = {B1'(x, y)A1'(x, y) + B2'(x, y)A2'(x, y) + … + Bn'(x, y)An'(x, y)} / D(x, y), A(x, y) = [{A1'(x, y)} 2 + {A2'(x, y)} 2 + … + {An'(x, y)} 2 / D(x, y), When D(x, y) = 0: R(x, y) = 0, G(x, y) = 0, B(x, y) = 0, A(x, y) = 0.

[0072] (Example of the execution of the image processing method according to the modified example) An example of the execution of the image processing method S100 according to the above-described modified example will be described with reference to FIGS. 13 to 18. In this specific example, for simplicity, the number n of target images I1, I2, …, In is set to 4.

[0073] FIG. 13 is a diagram illustrating the transparent target image I1 obtained by the transparency processing S112. FIG. 14 is a diagram illustrating the transparent target image I2 obtained by the transparency processing S112. FIG. 15 is a diagram illustrating the transparent target image I3 obtained by the transparency processing S112. FIG. 16 is a diagram illustrating the transparent target image I4 obtained by the transparency processing S112. It can be seen that in any of the transparent target images Ii, regions other than the region corresponding to the core of the image guide 13q are transparent.

[0074] FIG. 17 is a diagram illustrating a super-resolution image I obtained by the super-resolution process S105. In the target images I1 to I4, it is difficult to read the alphabets and characters included as subjects. On the other hand, in the super-resolution image I, it is easy to read the alphabets and characters included as subjects. That is, it was confirmed that by executing the super-resolution process S105, a super-resolution image I with higher visibility than the target images I1 to I4 can be obtained.

[0075] Further, when comparing the super-resolution image I (without transparency processing) shown in FIG. 10 and the super-resolution image I (with transparency processing) shown in FIG. 17, the contrast between the alphabets, characters, and their background is higher in the latter. As a result, it can be seen that the latter has higher visibility of the alphabets and characters. This is because in each target image Ii, the luminance of the region other than the region corresponding to the core of the image guide 13q is low. Therefore, in the super-resolution image I shown in FIG. 10, a decrease in luminance and image noise are likely to occur within the viewing circle. However, in each transparent target image Ii, since the region other than the region corresponding to the core of the image guide 13q is transparent, a decrease in luminance and image noise are less likely to occur within the viewing circle.

[0076] Note that the decrease in luminance and image noise occur, for example, when the pixel value of a pixel belonging to the low-luminance region of the target image Ij is averaged with the pixel value of a pixel belonging to the high-luminance region of another target image Ij', or when the pixel value of a pixel belonging to the low-luminance region of the target image Ij is averaged with the pixel value of a pixel belonging to the low-luminance region of another target image I'.

[0077] FIG. 18 is a diagram illustrating the output image I′ obtained in the mask process S106. In the super-resolution image I, jaggies (the jaggedness caused by the image guide 13q being composed of a plurality of cores) can be visually recognized at the boundary of the end face of the image guide 13q included as a subject. On the other hand, in the output image I′, jaggies cannot be visually recognized at the boundary of the end face of the image guide 13q included as a subject. That is, it was confirmed that by executing the mask process S106, an output image I′ with less jaggies than the super-resolution image I can be obtained.

[0078] (Further Modification Example 1 of the Image Processing Method) As preprocessing for the super-resolution process S105 described above, the image processing apparatus 10 may perform core drawing processing on each transparent target image Ij.

[0079] In the core drawing process, the image processing apparatus 10 first identifies a region corresponding to the core of the image guide 13q in the transparent target image Ij (hereinafter also referred to as the “core region”). For example, a set of pixels whose pixel values of pixels having the same coordinates in the mesh image I0′ are equal to or greater than a predetermined threshold value is identified as the core region.

[0080] Next, for each of the red component, green component, and blue component, the image processing apparatus 10 resets the pixel value of each pixel included in the core region in the transparent target image Ij to be the same as (a1) the pixel value of the pixel located at the center of the core region (hereinafter also referred to as the "center pixel value"), or (a2) the average value of the pixel values of each pixel included in the core region (hereinafter also referred to as the "average pixel value"). Further, the image processing apparatus 10 sets the opacity of the pixel located at the center of the core region in the transparent target image Ij to 100%. Then, the image processing apparatus 10 resets the opacity of each pixel in the transparent target image Ij where the distance r from the center of the core region is equal to or less than a predetermined threshold value R so that (b1) it decreases linearly as the distance r increases, or (b2) it decreases according to a Gaussian distribution as the distance r increases. When the opacity is not reset, the opacity of each pixel included in the core region coincides with the pixel value of the pixel having the same coordinates as that pixel in the mesh image I0' as described above.

[0081] FIG. 19(a) is an enlarged view of the transparent target image Ij and the super-resolution image I obtained when, for the red component, green component, and blue component, the pixel value of each pixel included in the core region is reset to the average pixel value and the opacity is not reset.

[0082] FIG. 19(b) is an enlarged view of the transparent target image Ij and the super-resolution image I obtained when, for the red component, green component, and blue component, the pixel value of each pixel included in the core region is reset to the average pixel value and, for the opacity, the opacity of each pixel where the distance r from the center of the core region is equal to or less than a predetermined threshold value R is reset so that it decreases linearly as the distance r increases. Comparing FIG. 19(a) and (b), it can be seen that by resetting the opacity, the decrease in luminance and the image noise in the super-resolution image I can be further suppressed. Note that the same effect can be obtained even when the pixel value of each pixel included in the core region is set to the center pixel value for each of the red component, green component, and blue component.

[0083] FIG. 19(c) is an enlarged view of the transparency target image Ij and the super-resolution image I obtained when, for the red, green, and blue components, the pixel value of each pixel included in the core region is reset to the average pixel value, and for the opacity, the opacity of each pixel whose distance r from the center of the core region is equal to or less than a predetermined threshold value R is reset so as to decrease according to a Gaussian distribution as the distance r increases. Comparing FIG. 19(a) and FIG. 19(c), it can be seen that by resetting the opacity, it is possible to further suppress a decrease in luminance and image roughness in the super-resolution image I. Further, comparing FIG. 19(b) and FIG. 19(c), it can be seen that resetting the opacity according to a Gaussian distribution as the distance r increases is more advantageous for further suppressing a decrease in luminance and image roughness in the super-resolution image I than resetting the opacity to decrease linearly as the distance r increases. Note that the same effect can be obtained even when the pixel value of each pixel included in the core region is set to the center pixel value for each of the red, green, and blue components.

[0084] (Further Modification Example 2 of Image Processing Method) In the above-described super-resolution process S105, the image processing apparatus 10 may perform the averaging process S105c on all of the transparency target images I1, I2,..., In, or may perform the averaging process S105c on a part of the transparency target images I1, I2,..., In.

[0085] As an example of the latter, in the transparency target images I1, I2,..., In, a configuration can be considered in which the transparency target images other than the transparency target image with a large amount of misregistration are the targets of the averaging process S105c. Since the transparency target image with a large amount of misregistration has little overlap with the transparency target image In used as the reference image in the alignment process S105a, its contribution to improving the visibility of the super-resolution image I is limited, and conversely, it may also cause noise in the super-resolution image I. Therefore, by excluding the transparency target image with a large amount of misregistration from the targets of the averaging process S105c, it is possible to suppress the generation of noise in the super-resolution image I without inhibiting the effect of improving the visibility of the super-resolution image I.

[0086] In the transparent target images I1, I2, …, In, as a method for selecting a transparent target image with a large amount of misregistration, for example, a method of selecting a transparent target image Ii in which the misregistration vector vi exceeds a predetermined threshold value in the transparent target images I1, I2, …, In can be mentioned.

[0087] Alternatively, in the transparent target images I1, I2, …, In, a configuration can be considered in which the transparent target images other than the transparent target images in which blurring or blurring has occurred are targeted for the averaging process S105c. A transparent target image with a large amount of blurring or blurring is difficult to perform the alignment process S105a with high accuracy, and the outline of the object O included as the subject is unclear. Therefore, the contribution to improving the visibility of the super-resolution image I is limited. On the contrary, it can also cause noise in the super-resolution image I. For this reason, by excluding the transparent target image in which blurring or blurring has occurred from the target of the averaging process S105c, it is possible to suppress the generation of noise in the super-resolution image I without inhibiting the effect of improving the visibility of the super-resolution image I.

[0088] In the transparent target images I1, I2, …, In, as a method for selecting a transparent target image Ii in which blurring or blurring has occurred, for example, a method of selecting a transparent target image Ii in which the difference |(vi-1)-(vi)|, |(vi+1)-(vi)| of the misregistration vectors from the adjacent transparent target images Ii-1, Ii+1 exceeds a predetermined threshold value can be mentioned. However, the method for selecting the transparent target image Ii in which blurring or blurring has occurred is not limited to this, and a known method can be arbitrarily adopted.

[0089] (Further modification example 3 of the image processing method) As post-processing of the above-described super-resolution process S105, the image processing apparatus 10 may perform a base image generation process of generating a base image by smoothing the finally acquired target image In, and an output image generation process of generating an output image by superimposing the super-resolution image I on the generated base image. Here, the method of smoothing is not particularly limited. The base image may be generated by smoothing using an averaging filter, or the base image may be generated by smoothing using a Gaussian filter.

[0090] If the opacity of a certain pixel is 0 in all the transparent target images I1’, I2’, …, In’, the opacity of that image will also be 0 in the super-resolution image I. That is, the pixel becomes transparent in the super-resolution image I. When such a super-resolution image I is displayed, the background that can be seen through at the transparent pixels may be visually recognized, giving an unnatural impression. For example, when using a completely black background, when such a super-resolution image I is displayed, the background that can be seen through at the transparent pixels is visually recognized like scattered black dots, giving an unnatural impression. By superimposing the super-resolution image I on the base image generated by smoothing the target image In, the concern that the background that can be seen through at the transparent pixels is visually recognized can be eliminated, and the super-resolution image I can be displayed more naturally.

[0091] Note that in this modification example, a configuration of generating a base image by smoothing the finally acquired target image In has been described, but it is not limited to this. That is, among the target images I1, I2, …, In, a configuration of generating a base image by smoothing a target image Ii other than the finally acquired target image In may be adopted.

[0092] (Summary) This embodiment includes the following aspects.

[0093] 〔Aspect 1〕 An acquisition process of acquiring a plurality of target images obtained by imaging the other end of an image guide having a plurality of cores and having one end facing an object, A super-resolution process of generating a super-resolution image by simply averaging or weighted-averaging the pixel values of pixels corresponding to the same point of the object in the plurality of target images, and the like. An image processing method.

[0094] According to this aspect, a super-resolution image with higher resolution or visibility than the target image can be obtained.

[0095] [Aspect 2] As a preprocessing of the super-resolution process, it further includes a transparency process of making the areas other than the areas corresponding to the core of the image guide transparent or semi-transparent in each of the plurality of target images. The super-resolution process is a process of generating the super-resolution image by weighted-averaging the pixel values of pixels corresponding to the same point of the object in the plurality of target images after the preprocessing, with the opacity as the weight. The image processing method according to Aspect 1.

[0096] According to this aspect, by suppressing the brightness reduction and image unevenness that may occur in the super-resolution image, a super-resolution image with higher visibility can be obtained.

[0097] [Aspect 3] As the preprocessing, it further includes a core drawing process of setting the pixel value of each pixel included in the area corresponding to the core of the image guide in each of the plurality of target images to the pixel value of the pixel located at the center of the area. The image processing method according to Aspect 2.

[0098] According to this aspect, by further suppressing the brightness reduction and image unevenness that may occur in the super-resolution image, a super-resolution image with higher visibility can be obtained.

[0099] [Aspect 4] As the preprocessing, it further includes a core drawing process of setting the pixel value of each pixel included in the area corresponding to the core of the image guide in each of the plurality of target images to the average value of the pixel values of each pixel included in the area. The image processing method according to Aspect 2.

[0100] According to this aspect, by further suppressing the luminance decrease and image unevenness that may occur in the super-resolution image, a super-resolution image with higher visibility can be obtained.

[0101] 〔Aspect 5〕 In the core drawing process, further, the opacity of each pixel whose distance from the center of the region in each of the plurality of target images is equal to or less than a threshold value is set to decrease according to a Gaussian distribution as the distance increases. The image processing method according to Aspect 3 or 4.

[0102] According to this aspect, by further suppressing the luminance decrease and image unevenness that may occur in the super-resolution image, a super-resolution image with higher visibility can be obtained.

[0103] 〔Aspect 6〕 As post-processing of the super-resolution process, a base image generation process for generating a base image by smoothing any one of the plurality of target images, and a superimposing process for generating an output image by superimposing the super-resolution image on the base image are further included. The image processing method according to any one of Aspects 2 to 5.

[0104] According to this aspect, by eliminating the possibility that the background is seen through in the super-resolution image, a more natural super-resolution image can be obtained.

[0105] 〔Aspect 7〕 The super-resolution process is a process of generating the super-resolution image by weighted-averaging the pixel values of the pixels corresponding to the same point of the object in the target image determined to have a small misregistration among the plurality of target images. The image processing method according to any one of Aspects 2 to 6.

[0106] According to this aspect, by reducing noise that may occur in a super-resolution image, a super-resolution image with higher visibility can be obtained.

[0107] Aspect 8 The super-resolution processing is a process of generating the super-resolution image by taking a weighted average of pixel values of pixels corresponding to the same point of the object in a target image determined to be free of blur or shaking among the plurality of target images. 7. The image processing method according to any one of aspects 2 to 6.

[0108] According to this aspect, by reducing noise that may occur in a super-resolution image, a super-resolution image with higher visibility can be obtained.

[0109] Aspect 9 An image processing device comprising at least one processor, The processor: an acquisition process for acquiring a plurality of target images obtained by imaging one end of an image guide having a plurality of cores and facing the target; and performing a super-resolution process to generate a super-resolution image by averaging or weighted averaging pixel values of pixels corresponding to the same point of the object in the plurality of object images. Image processing device.

[0110] According to this aspect, it is possible to obtain a super-resolution image having higher resolution or visibility than the target image.

[0111] Aspect 10 The processor further performs a transparency process as a preprocessing of the super-resolution process, which makes transparent or semi-transparent an area other than an area corresponding to a core of the image guide in each of the plurality of target images; The super-resolution processing is a process of generating the super-resolution image by weighting pixel values of pixels corresponding to the same point of the object in the plurality of target images after the preprocessing, using opacity as a weight. An image processing device according to embodiment 9.

[0112] According to this aspect, it is possible to obtain a super-resolution image with higher visibility by suppressing the reduction in brightness and image roughness that can occur in the super-resolution image.

[0113] Aspect 11 The processor further performs, as the pre-processing, a core drawing process of setting a pixel value of each pixel included in an area corresponding to a core of the image guide in each of the plurality of target images to a pixel value of a pixel located at the center of the area. An image processing device according to embodiment 10.

[0114] According to this aspect, it is possible to obtain a super-resolution image with higher visibility by further suppressing the reduction in brightness and image roughness that may occur in the super-resolution image.

[0115] Aspect 12 An image processing program for operating at least one processor, the program comprising: an acquisition process for acquiring a plurality of target images obtained by imaging one end of an image guide having a plurality of cores and facing the target; a super-resolution process for generating a super-resolution image by averaging or weighted averaging pixel values of pixels corresponding to the same point of the object in the plurality of object images; Image processing program.

[0116] According to this aspect, it is possible to obtain a super-resolution image having higher resolution or visibility than the target image.

[0117] Aspect 13 The processor further executes a transparency process as a preprocessing of the super-resolution process, which makes transparent or semi-transparent an area other than an area corresponding to a core of the image guide in each of the plurality of target images; The super-resolution processing is a process of generating the super-resolution image by weighting pixel values of pixels corresponding to the same point of the object in the plurality of target images after the preprocessing, using opacity as a weight. An image processing program according to embodiment 12.

[0118] According to this aspect, it is possible to obtain a super-resolution image with higher visibility by further suppressing the reduction in brightness and image roughness that may occur in the super-resolution image.

[0119] Aspect 14 and causing the processor to further execute, as the pre-processing, a core drawing process of setting a pixel value of each pixel included in an area corresponding to a core of the image guide in each of the plurality of target images to a pixel value of a pixel located at the center of the area. An image processing program according to embodiment 13.

[0120] According to this aspect, it is possible to obtain a super-resolution image with higher visibility by further suppressing the reduction in brightness and image roughness that may occur in the super-resolution image.

[0121] Aspect 15 The image processing device according to any one of aspects 9 to 11; an endoscopic probe including the image guide, Endoscopy system.

[0122] According to this aspect, it is possible to realize an endoscope system that can obtain a super-resolution image with higher resolution or visibility than the target image.

[0123] (Additional notes) The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0124] 1 Endoscopy System 10 Image processing device 101 processors 102 memory 103 Storage 104 Input / Output Interface 105 Bus 11 Light source device 12 Camera equipment 13 Endoscopic probe 13p Light Guide 13q Image Guide

Claims

1. An acquisition process for acquiring a plurality of target images obtained by imaging the other end of an image guide having a plurality of cores and having one end facing an object; A super-resolution process for generating a super-resolution image by simply averaging or weighted-averaging the pixel values of pixels corresponding to the same point of the object in the plurality of target images, and An image processing method.

2. As a pre-process of the super-resolution process, further includes a transparency process of making a region other than the region corresponding to the core of the image guide transparent or semi-transparent in each of the plurality of target images, The super-resolution process is a process of generating the super-resolution image by weighted-averaging the pixel values of pixels corresponding to the same point of the object in the plurality of target images after the pre-process, with the opacity as the weight. The image processing method according to claim 1.

3. As the pre-process, further includes a core drawing process of setting the pixel value of each pixel included in the region corresponding to the core of the image guide in each of the plurality of target images to the pixel value of the pixel located at the center of the region, The image processing method according to claim 2. 【Claim ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ The super-resolution process is a process of generating the super-resolution image by weighted-averaging pixel values of pixels corresponding to the same point of the object in the target image determined not to have blur or shake among the plurality of target images. The image processing method according to any one of claims 2 to 4.

9. An image processing apparatus including at least one processor, wherein the processor has a plurality of cores, and executes an acquisition process of acquiring a plurality of target images obtained by imaging the other end of an image guide having one end facing an object, and executes a super-resolution process of generating a super-resolution image by averaging or weighted-averaging pixel values of pixels corresponding to the same point of the object in the plurality of target images. Image processing apparatus.

10. As preprocessing of the super-resolution process, the processor further executes a transparency process of making a region other than a region corresponding to the core of the image guide transparent or semi-transparent in each of the plurality of target images, and the super-resolution process is a process of generating the super-resolution image by weighted-averaging pixel values of pixels corresponding to the same point of the object in the plurality of target images after the preprocessing, with the opacity as a weight. The image processing apparatus according to claim 9.

11. As the preprocessing, the processor further executes a core drawing process of setting the pixel value of each pixel included in the region corresponding to the core of the image guide to the pixel value of the pixel located at the center of the region in each of the plurality of target images. The image processing apparatus according to claim 10.

12. An image processing program for operating at least one processor, the program causing the processor to have a plurality of cores and execute an acquisition process of acquiring a plurality of target images obtained by imaging the other end of an image guide having one end facing an object, and execute a super-resolution process of generating a super-resolution image by averaging or weighted-averaging pixel values of pixels corresponding to the same point of the object in the plurality of target images. Image processing program.

13. As preprocessing of the super-resolution process, the program further causes the processor to execute a transparency process of making a region other than a region corresponding to the core of the image guide transparent or semi-transparent in each of the plurality of target images. The super-resolution processing is a process of generating the super-resolution image by weighted-averaging pixel values of pixels corresponding to the same point of the object in the plurality of target images after the pre-processing, with opacity as a weight. The image processing program according to claim 12.

14. The processor is further caused to execute a core drawing process of setting, as the pre-processing, pixel values of each pixel included in a region corresponding to the core of the image guide in each of the plurality of target images to pixel values of a pixel located at the center of the region. The image processing program according to claim 13.

15. The image processing apparatus according to any one of claims 9 to 11, and an endoscope probe including the image guide. An endoscope system.

Images