Image processing method, image processing device, image processing program, and endoscope system

Abstract

To provide an image processing technology that improves resolution or visibility of obtained images.SOLUTION: An image processing device (10) generates a super-resolution image by acquiring a plurality of object images obtained through imaging of one end of an image guide (13q) which includes a plurality of cores and in which the other end faces an object (O), and by obtaining a simple average or a weighted average of pixel values of pixels corresponding to the same point of the object (O) in the plurality of object images.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image processing method, an image processing device, and an image processing program for processing images captured using an image guide, and also to an endoscope system equipped with such an image processing device. [Background technology]

[0002] Endoscope systems are widely used that obtain images including internal objects as subjects by imaging one end of an image guide inserted into the body. For example, Patent Document 1 discloses a vascular endoscope system that reduces the risk of damage to the intima of small blood vessels by making the image guide thinner and more flexible. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 8-191439 Summary of the Invention [Problem to be solved by the invention]

[0004] In an endoscope system using an image guide, the resolution is determined by the core density of the image guide. For example, a structure about 100 times the inter-core distance can be seen from the image formed on the end face of the image guide, but a structure about 10 times the inter-core distance is difficult to see from the image formed on the end face of the image guide, and a structure about the same size as the inter-core distance cannot be seen from the image formed on the end face of the image guide.

[0005] In the angioscope system described in Patent Document 1, the image quality of the obtained image is improved by correcting the differences in brightness and color tone for each pixel caused by the differences in transmission characteristics of each core of the image guide. However, such correction cannot exceed the resolution determined by the core density of the image guide.

[0006] One aspect of the present invention has been made in view of the above-mentioned problems, and aims to provide an image processing technique that improves the resolution or visibility of the resulting image. [Means for solving the problem]

[0007] An image processing method according to one aspect of the present invention includes an acquisition process for acquiring a plurality of target images obtained by imaging one end of an image guide having a plurality of cores, the other end of which faces an object, and a super-resolution process for generating a super-resolution image by simply averaging or weighted averaging pixel values ​​of pixels corresponding to the same point on the object in the plurality of target images.

[0008] An image processing device according to one embodiment of the present invention is an image processing device having at least one processor, the processor having multiple cores, and performing an acquisition process to acquire multiple target images obtained by imaging one end of an image guide whose other end faces an target, and a super-resolution process to generate a super-resolution image by averaging or weighted averaging pixel values ​​of pixels corresponding to the same point on the target in the multiple target images.

[0009] An image processing program according to one embodiment of the present invention is an image processing program for operating at least one processor, which causes the processor to perform an acquisition process for acquiring multiple target images obtained by imaging one end of an image guide having multiple cores and one end facing an object, and a super-resolution process for generating a super-resolution image by averaging or weighted averaging pixel values ​​of pixels corresponding to the same point on the object in the multiple target images. [Effects of the Invention]

[0010] According to one aspect of the present invention, the resolution or visibility of the obtained image can be improved. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a schematic diagram showing the configuration of an endoscope system according to an embodiment of the present invention. [Figure 2] 2 is a block diagram showing the configuration of an image processing device included in the endoscope system shown in FIG. 1. FIG. [Figure 3] FIG. 1 is a flowchart showing the flow of an image processing method according to an embodiment of the present invention. [Figure 4] 4 is a diagram showing an example of an execution of the image processing method shown in Fig. 3. Specifically, it is a diagram showing an object to be observed in the execution example. [Figure 5] 4 is a diagram showing an example of an execution of the image processing method shown in Fig. 3. Specifically, it is a diagram showing a mesh image obtained by processing an image of a white standard board in this execution example. [Figure 6] 4 is a diagram showing an example of execution of the image processing method shown in Fig. 3. Specifically, it is a diagram showing a first object image obtained by object imaging processing in this example of execution. [Figure 7] 4 is a diagram showing an example of execution of the image processing method shown in Fig. 3. Specifically, it is a diagram showing a second object image obtained by object imaging processing in this example of execution. [Figure 8] 4 is a diagram showing an example of execution of the image processing method shown in Fig. 3. Specifically, it is a diagram showing a third object image obtained by object imaging processing in this example of execution. [Figure 9] 4 is a diagram showing an example of execution of the image processing method shown in Fig. 3. Specifically, it is a diagram showing a fourth object image obtained by object imaging processing in this example of execution. [Figure 10] 4 is a diagram showing an example of an execution of the image processing method shown in Fig. 3. Specifically, it is a diagram showing a super-resolution image obtained by super-resolution processing (simple averaging) in this execution example. [Figure 11]4 is a diagram showing an example of execution of the image processing method shown in Fig. 3. Specifically, it is a diagram showing an output image obtained by mask processing in this example of execution. [Figure 12] 4 is a flowchart showing the flow of an image processing method according to a modified example of the image processing method shown in FIG. [Figure 13] 13 is a diagram showing an example of execution of the image processing method shown in Fig. 12. Specifically, it is a diagram showing a first target image obtained by transparency processing in this example of execution. [Figure 14] 13 is a diagram showing an example of execution of the image processing method shown in Fig. 12. Specifically, it is a diagram showing a second target image obtained by transparency processing in this example of execution. [Figure 15] 13 is a diagram showing an example of execution of the image processing method shown in Fig. 12. Specifically, it is a diagram showing a third target image obtained by transparency processing in this example of execution. [Figure 16] 13 is a diagram showing an example of execution of the image processing method shown in Fig. 12. Specifically, it is a diagram showing a fourth target image obtained by transparency processing in this example of execution. [Figure 17] 13 is a diagram showing an example of an execution of the image processing method shown in Fig. 12. Specifically, it is a diagram showing a super-resolution image obtained by super-resolution processing (weighted average) in this execution example. [Figure 18] 13 is a diagram showing an example of execution of the image processing method shown in Fig. 12. Specifically, it is a diagram showing an output image obtained by mask processing in this example of execution. [Figure 19] 13A and 13B are diagrams illustrating an example of execution of the image processing method shown in Fig. 12. Specifically, they are enlarged views showing a portion of a super-resolution image obtained by core drawing processing in this example. (a) corresponds to a case where opacity resetting is not performed, (b) corresponds to a case where opacity resetting is performed so that the opacity of each pixel whose distance from the center of the core region is equal to or less than a predetermined threshold decreases linearly as the distance increases, and (c) corresponds to a case where the opacity of each pixel whose distance from the center of the core region is equal to or less than a predetermined threshold decreases according to a Gaussian distribution as the distance increases. DETAILED DESCRIPTION OF THE INVENTION

[0012] (Configuration of endoscope system) An endoscope system 1 according to one embodiment of the present invention will be described with reference to Fig. 1. In Fig. 1, (a) is a schematic diagram showing the configuration of the endoscope system 1, (b) is a plan view and a cross-sectional view showing the structure of a light guide 13p in a light guide plug portion 13a, (c) is a plan view and a cross-sectional view showing the structure of an image guide 13q in an image guide plug portion 13c, and (d) is a plan view and a cross-sectional view showing the structures of the light guide 13p and image guide 13q in an insertion portion 13g.

[0013] The endoscope system 1 is a system for displaying an image of an object O to be observed on a display D. As shown in FIG. 1(a), the endoscope system 1 includes an image processing device 10, a light source device 11, a camera device 12, and an endoscope probe 13.

[0014] 1(a), the light source device 11 includes a light source 11a and a condenser lens 11b. The light source 11a is disposed on the optical axis of the condenser lens 11b. The light source 11a may be, for example, an LED (Light Emitting Diode), a xenon lamp, or a halogen lamp.

[0015] As shown in Fig. 1(a), the camera device 12 includes an image sensor 12a, an objective lens 12b, and a signal processing circuit 12c. The image sensor 12a is disposed on the optical axis of the objective lens 12b. The signal processing circuit 12c is electrically connected to the image sensor 12a. The image sensor 12a may be, for example, a CMOS (Complementary Metal Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) sensor.

[0016] 1(a), the endoscopic probe 13 includes: (1) a light guide plug portion 13a mechanically connected to the light source device 11; (2) a light guide lead portion 13b having one end connected to the light guide plug portion 13a; (3) an image guide plug portion 13c mechanically connected to the camera device 12; (4) an image guide lead portion 13d having one end connected to the image guide plug portion 13c; (5) a branch portion 13e having one end connected to the other ends of the light guide lead portion 13b and the image guide lead portion 13d; (6) an intermediate portion 13f having one end connected to the other end of the branch portion 13e; and (7) an insertion portion 13g having one end connected to the other end of the intermediate portion 13f. The endoscopic probe 13 may further include a water passage lumen, a working lumen, an insertion portion bending mechanism, and a bending operation portion (none of which are shown).

[0017] 1(b) and 1(d), one or more light guides 13p that function as waveguides for guiding illumination light generated by the light source device 11 are provided inside the light guide plug portion 13a, the light guide lead portion 13b, the branch portion 13e, the intermediate portion 13f, and the insertion portion 13g. Each light guide 13p is embedded in the cladding. The light guides 13p contained in the light guide plug portion 13a and the light guide lead portion 13b are bundled with an image guide 13q (described later) at the branch portion 13e, and are contained together with this image guide 13q in the intermediate portion 13f and the insertion portion 13g.

[0018] 1(c) and 1(d), image guide 13q, which is a waveguide that guides scattered light generated by object O, is provided inside image guide plug portion 13c, image guide lead portion 13d, branch portion 13e, intermediate portion 13f, and insertion portion 13g. Image guide 13q is composed of a cladding and a plurality of cores (e.g., several thousand to several tens of thousands of cores) arranged regularly or randomly inside the cladding. Image guide 13q contained in image guide plug portion 13c and image guide lead portion 13d is bundled with light guide 13p described above at branch portion 13e, and is contained together with light guide 13p in intermediate portion 13f and insertion portion 13g.

[0019] The image guide 13q may be, for example, an image guide fiber, an image fiber, a fiber bundle, or a fiber conduit. The material of the image guide 13q may be any of quartz, multi-component glass, and plastic. The number and arrangement of cores in the image guide 13q are not particularly limited. As an example, an image guide fiber in which 3,000 cores are arranged on the lattice points of a hexagonal close-packed lattice may be used as the image guide 13q. However, the resolution of the image guide 13q is lower than the resolution of the image sensor 12a, i.e., the number of cores in the image guide 13q is smaller than the number of cells in the image sensor 12a.

[0020] 1(d), an objective lens 13r is provided at the tip of the insertion section 13g. One end of the objective lens 13r forms the end face of the insertion section 13g, and the other end of the objective lens section 13s faces one end of the image guide 13q. For example, a GRIN lens or a lens unit consisting of multiple lenses can be used as the objective lens 13r.

[0021] The endoscope system 1 operates, for example, as follows. That is, with the insertion section 13g inserted into the patient's body, illumination light is emitted from the light source 11a. The illumination light emitted from the light source 11a is condensed by the condenser lens 11b onto the end face of the light guide 13p on the light guide plug section 13a side. The illumination light that enters the core of the light guide 13p at the end face of the light guide 13p on the light guide plug section 13a side is guided through the core of the light guide 13p and emitted from the core of the light guide 13p at the end face of the light guide 13p on the insertion section 13g side. The illumination light that is emitted from the core of the light guide 13p at the end face of the light guide 13p on the insertion section 13g side is irradiated onto an object O inside the patient's body.

[0022] The scattered light generated at each point on the object O by scattering the illumination light inside the patient's body is focused by the objective lens 13r at each point on the end face of the image guide 13q on the insertion section 13g side. As a result, an image of the object O is formed on the end face of the image guide 13q on the insertion section 13g side. The scattered light incident on the core of the image guide 13q at the end face of the image guide 13q on the insertion section 13g side by the objective lens 13r is guided through the core of the image guide 13q and exits from the core of the image guide 13q at the end face of the image guide 13q on the image guide plug section 13c side. As a result, an image of the object O is formed on the end face of the image guide 13q on the image guide plug section 13c side, just like on the end face of the image guide 13q on the insertion section 13g side.

[0023] Light emitted from each core of image guide 13q at the end face of image guide 13q on the image guide plug portion 13c side (light originating from scattered light generated at each point on object O) is focused by objective lens 12b at each point on the light receiving surface of image sensor 12a. As a result, an image of object O is formed on the light receiving surface of image sensor 12a. Image sensor 12a generates an electrical signal representing the image of object O and inputs the generated electrical signal to signal processing circuit 12c. Signal processing circuit 12c generates an object image representing the image of object O from the electrical signal input from image sensor 12a.

[0024] The camera device 12 inputs the generated target image to the image processing device 10. The image processing device 10 generates an output image to be displayed on the display D from the target image input from the camera device 12 according to an image processing method S100 described later. Note that the functions performed by the image processing device 10 may be performed by the camera device 12. That is, a configuration in which the image processing method S100 described later is performed by the camera device 12 may be adopted instead of a configuration in which the image processing device 10 performs the image processing method S100 described later. In this case, the image processing device 10 can be omitted from the endoscope system 1.

[0025] In this specification, the term "image" refers to a two-dimensional array of pixel values. A pixel value in a monochrome image consists of, for example, a single numerical value corresponding to brightness. In a D-level monochrome image, brightness takes an integer value between 0 and D-1 inclusive. A pixel value in a color image consists of, for example, three numerical values ​​corresponding to RGB. Here, R represents the red component, G represents the green component, and B represents the blue component. In a D-level color image, the red component, green component, and blue component each take an integer value between 0 and D-1 inclusive. A pixel value in a transparent color image consists of, for example, four numerical values ​​corresponding to RGBA. Here, A represents opacity. In the case of a D-level transparent color image, the red component, green component, blue component, and opacity each take an integer value between 0 and D-1 inclusive.

[0026] Pixels in an image are identified by their coordinates (x, y). If the image size is W pixels wide by H pixels high, then x is an integer between 0 and W-1, inclusive, and y is an integer between 0 and H-1, inclusive. A pixel with coordinates (x, y) will hereinafter be referred to as pixel (x, y). For example, pixel (0, 0) is the pixel in the upper left corner of the image, pixel (1, 0) is the pixel immediately to the right of pixel (0, 0), and pixel (0, 1) is the pixel immediately below pixel (0, 0). The pixel value (brightness value) of pixel (x, y) in a monochrome image is referred to as A(x, y). Furthermore, for the pixel value of pixel (x, y) in a color image, its red component is referred to as R(x, y), its green component is referred to as G(x, y), and its blue component is referred to as B(x, y). Similarly, for the pixel value of pixel (x, y) in an opacity-added color image, its red component is written as R(x, y), its green component as G(x, y), its blue component as B(x, y), and its opacity as A(x, y).

[0027] The file format for saving images is not particularly limited. For example, file formats such as PNG, JPEG, and GIF can be used to save monochrome or color images. Furthermore, file formats such as transparent PNG can be used to save transparent color images.

[0028] (Configuration of image processing device) The configuration of the image processing device 10 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the configuration of the image processing device 10.

[0029] 2, the image processing device 10 includes a processor 101, a memory 102, a storage 103, an input / output interface 104, and a bus 105. The processor 101, the memory 102, the storage 103, and the input / output interface 104 are connected to one another via the bus 105.

[0030] The memory 102 is configured to expand and store an image processing program P100 for implementing an image processing method S100 (described later) in a state that can be referenced by the processor 101. The memory 102 is also used to store various images acquired from the camera device 12. Note that the memory 102 may be, for example, a semiconductor RAM (Random Access Memory).

[0031] The processor 101 is configured to execute an image processing method S100 (described later) in accordance with an image processing program P100 loaded in the memory 102. The processor 101 may be, for example, a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), or a combination thereof.

[0032] The storage 103 is configured to store (non-volatilely store) the image processing program P100. When the processor 101 performs an image processing method S100 (described later), the processor 101 loads the image processing program P100 stored in the storage 103 onto the memory 102 and references it. Note that the storage 103 may be, for example, a flash memory, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a combination of these.

[0033] The input / output interface 104 is configured to connect the camera device 12 and the display D to the image processing device 10. As the input / output interface 104, for example, a USB (Universal Serial Bus) interface or the like can be used.

[0034] Although the configuration in which the image processing method S100 described below is executed by a single processor 101 provided in a single computer (a computer that functions as the image processing device 10) has been described above, the present invention is not limited to this. That is, it is also possible to adopt a configuration in which the image processing method S100 described below is executed jointly by multiple processors provided centrally in a single computer or distributed across multiple computers.

[0035] The image processing program P100 may be recorded on a computer-readable, non-transitory, tangible recording medium. This recording medium may be the memory 102, the storage 103, or another recording medium. For example, a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like may be used as the other recording medium.

[0036] (Flow of Image Processing Method) The flow of the image processing method S100 according to one embodiment of the present invention will be described with reference to Fig. 3. Fig. 3 is a flow chart showing the flow of the image processing method S100.

[0037] 3, the image processing method S100 includes a white standard plate imaging process S101, a white balance adjustment process S102, an object imaging process S103, an object image acquisition process S104, a super-resolution process S105, a masking process S106, and an image display process S107. Here, the white standard plate imaging process S101, the white balance adjustment process S102, and the object imaging process S103 are processes that are mainly performed by the camera device 12. On the other hand, the object image acquisition process S104, the super-resolution process S105, the masking process S106, and the image display process S107 are processes that are mainly performed by the image processing device 10.

[0038] In the white standard plate imaging process S101, the user positions the objective lens 13r of the endoscopic probe 13 toward the white standard plate, and the camera device 12 images the end face of the image guide 13q on the image guide plug portion 13c side. In the white balance adjustment process S102, the camera device 12 adjusts the white balance by referring to the image captured in the white standard plate imaging process S101. The image captured in the white standard plate imaging process S101 and referenced in the white balance adjustment process S102 is a color image in which areas corresponding to the cores of the image guide 13q are bright and other areas are dark. This image will be referred to as a mesh image I0 below. Specific examples of the mesh image I0 obtained in the white standard plate imaging process S101 will be described later with reference to different drawings.

[0039] In the object imaging process S103, the user positions the objective lens 13r of the endoscopic probe 13 toward the object O, and the camera device 12 images the end face of the image guide 13q on the image guide plug portion 13c side. In the object image acquisition process S104, the image processing device 10 acquires the image captured in the object imaging process S103 from the camera device 12 and stores it in the memory 102. The object imaging process S103 and the object image acquisition process S104 are repeated n times (n is a predetermined natural number). The image captured in the object imaging process S103 and acquired in the object image acquisition process S104 is a color image representing the image of the object O formed on the end face of the image guide 13q on the image guide plug portion 13c side. These images will be referred to as object images I1, I2, ..., In hereinafter. Specific examples of the target images I1, I2, . . . , In obtained in the target image capturing process S103 will be described later with reference to different drawings.

[0040] The super-resolution process S105 is a process for generating a super-resolution image I by averaging pixel values ​​of pixels corresponding to the same point of an object O in target images I1, I2, ..., In. In this embodiment, the super-resolution process S105 is composed of a positioning process S105a and an averaging process S105b.

[0041] The alignment process S105a is a process in which the image processing apparatus 10 calculates, for each target image Ii (where i is each natural number from 1 to n - 1) other than the target image In that was last acquired, (1) the displacement vector vi of the target image Ii with respect to the target image In, and (2) shifts the pixel values of each pixel of the target image Ii by the displacement vector vi. As a method for calculating the displacement vector vi, for example, a known method that utilizes the feature amounts of the target images Ii and In can be used. The image obtained by the alignment process S105a for the target image Ii is a color image representing an image of the object O formed on the end face on the image guide plug portion 13c side of the image guide 13q. This image will be hereinafter referred to as the target image Ii'.

[0042] When the pixel (xi, yi) of the target image Ii and the pixel (xn, yn) of the target image In correspond to the same point of the subject, the displacement vector vi = (vix, viy) is given, for example, by the following formula.

[0043] vix = xi - xn, viy = yi - yn.

[0044] Also, when the pixel value at the pixel (x, y) of the target image Ii is given by {Ri(x, y), Gi(x, y), Bi(x, y)}, the pixel value {Ri'(x, y), Gi'(x, y), Bi'(x, y)} at the pixel (x, y) of the target image Ii' is given, for example, by the following formula.

[0045] When 0 ≤ x + vix ≤ W and 0 ≤ y + viy ≤ H: Ri'(x, y) = Ri(x + vix, y + viy), Gi'(x, y) = Gi(x + vix, y + viy), Bi'(x, y) = Bi(x + vix, y + viy), When x + vix < 0, W < x + vix, y + viy < 0, or H < y + viy: Ri'(x, y) = 0, Gi'(x, y) = 0, Bi'(x, y) = 0.

[0046] Note that the last acquired target image In is not subjected to the registration process S105a, but for convenience of explanation, the target image In will also be referred to as target image In' below. In other words, the pixel values ​​{Rn'(x,y), Gn'(x,y), Bn'(x,y)} at pixel (x,y) of the target image In' are the same as the pixel values ​​{Rn(x,y), Gn(x,y), Bn(x,y)} at pixel (x,y) of the target image In. Specific examples of the target images I1', I2', ..., In' obtained by the registration process S105a will be described later with reference to different drawings.

[0047] Although the configuration has been described above in which the last acquired target image In is used as a reference image and pixel values ​​of each target image Ii other than the reference image are shifted, the present invention is not limited to this. That is, a configuration may be adopted in which a target image Ip (p is a predetermined natural number between 1 and n-1) other than the last acquired target image In is used as a reference image and pixel values ​​of each target image Iq (q is a natural number between 1 and n, different from p) other than the reference image are shifted.

[0048] The averaging process S105b is a process in which the image processing device 10 simply averages the pixel values ​​{Rj'(x,y), Gj'(x,y), Bj'(x,y)} of each target image Ij' (j is a natural number between 1 and n) for each pixel (x,y) for each component. The image obtained by the averaging process S105b is a color image representing an image of the target O formed on the end face of the image guide 13q on the image guide plug portion 13c side. This image will be referred to as a super-resolution image I hereinafter. The pixel values ​​{R(x,y), G(x,y), B(x,y)} of the pixel (x,y) of the super-resolution image I are given by, for example, the following equations: A specific example of the super-resolution image I generated by the super-resolution process S105 will be described later with reference to different drawings.

[0049] R(x,y)={R1'(x,y)+R2'(x,y)+…+Rn'(x,y)} / n, G(x,y)={G1'(x,y)+G2'(x,y)+…+Gn'(x,y)} / n, B(x,y)={B1'(x,y)+B2'(x,y)+…+Bn'(x,y)} / n.

[0050] The masking process S106 is a process in which the image processing device 10 generates an output image I' by masking an area outside the field of view circle in the super-resolution image I. Here, masking an area outside the field of view circle means, for example, setting the pixel values ​​{R(x,y), G(x,y), B(x,y)} of pixels (x,y) located outside the field of view circle in the super-resolution image I to R(x,y)=0, G(x,y)=0, B(x,y)=0. A specific example of the output image I' generated in the masking process S106 will be described later with reference to different drawings.

[0051] The image display process S107 is a process in which the image processing device 10 displays the output image I' on the display D. This completes the series of processes of the image processing method S100.

[0052] While the image processing method S100 for displaying a still image of the object O on the display D has been described above, this method may also be used to display a moving image of the object O on the display D. In this case, the camera device 12 repeatedly executes the object imaging process S103 at a cycle corresponding to the frame rate, and the image processing device 10 executes the super-resolution process S105, the mask process S106, and the image display process S107 on the latest n target images among the target images captured by the camera device 12 up to that point. This allows a moving image of the object O to be displayed on the display D.

[0053] As a pre-processing step for the super-resolution processing S105, deformation correction such as affine transformation may be performed on each of the target images I1, I2, ..., In. If differences in the position and shape of the core images contained in the target images I1, I2, ..., In occur due to differences in the tilt angle during image capture, these differences can be eliminated in advance by deformation correction, thereby obtaining a super-resolution image I with higher visibility. Furthermore, as a pre-processing step for the super-resolution processing S105, color unevenness correction and brightness unevenness correction described in JP 2021-177604 A (Patent No. 7189173) may be performed on each of the target images I1, I2, ..., In. If color unevenness and brightness unevenness occur among the core images contained in the target images I1, I2, ..., In, these unevenness can be eliminated in advance by color unevenness correction and brightness unevenness correction, thereby obtaining a super-resolution image I with higher visibility.

[0054] (Example of image processing method) An example of execution of the image processing method S100 will be described with reference to Figures 4 to 11. In this example, for simplicity, the number n of target images I1, I2, . . . , In is set to 4.

[0055] 4 is a plan view of the object O used as a sample in this example. The object O used as a sample in this example is a white standard plate on which the letters A to Z and the numbers 0 to 9 are printed.

[0056] 5 is a diagram illustrating a mesh image I0 obtained in the white standard plate imaging process S101. As described above, the mesh image I0 is an image in which the areas corresponding to the cores of the image guide 13q are bright and the other areas are dark.

[0057] 6 is a diagram illustrating a target image I1 obtained in the target image capturing process S103 and a target image I1' obtained in the registration process S105a. The positions of the letters and alphabets included as subjects in the target image I1 are shifted to the lower right relative to the positions of the letters and alphabets included as subjects in the target image I4 used as the reference image. Therefore, the positional deviation vector v1 points to the lower right, as indicated by the white arrow in FIG. 6. The positions of the letters and alphabets included as subjects in the target image I1' obtained by shifting the pixel values ​​of each pixel of the target image I1 by the positional deviation vector v1 match or approximately match the positions of the letters and alphabets included as subjects in the target image I4 used as the reference image.

[0058] 7 is a diagram illustrating a target image I2 obtained in the target image capturing process S103 and a target image I2' obtained in the registration process S105a. The positions of the letters and alphabets included as subjects in the target image I2 are shifted to the left of the positions of the letters and alphabets included as subjects in the target image I4 used as the reference image. Therefore, the displacement vector v2 points to the left, as indicated by the white arrow in FIG. 7. The positions of the letters and alphabets included as subjects in the target image I2' obtained by shifting the pixel values ​​of each pixel of the target image I2 by the displacement vector v2 match or approximately match the positions of the letters and alphabets included as subjects in the target image I4 used as the reference image.

[0059] 8 is a diagram illustrating a target image I3 obtained in the target image capturing process S103 and a target image I3' obtained in the registration process S105a. The positions of the letters and alphabets included as subjects in the target image I3 are shifted to the upper right and to the right of the positions of the letters and alphabets included as subjects in the target image I4 used as the reference image. Therefore, the displacement vector v3 points to the upper right, as indicated by the white arrow in FIG. 8. The positions of the letters and alphabets included as subjects in the target image I3' obtained by shifting the pixel values ​​of each pixel of the target image I3 by the displacement vector v3 match or approximately match the positions of the letters and alphabets included as subjects in the target image I4 used as the reference image.

[0060] 9 is a diagram illustrating an example of a target image I4 obtained in the target image capturing process S103. Since the target image I4 is used as a reference image in the registration process S105a, the registration process S105a is not executed on the target image I4.

[0061] 10 is a diagram illustrating a super-resolution image I obtained by the super-resolution process S105. In the target images I1 to I4, it is difficult to read the alphabet and letters contained as subjects. In contrast, in the super-resolution image I, it is easy to read the alphabet and letters contained as subjects. In other words, it was confirmed that by performing the super-resolution process S105, a super-resolution image I with higher visibility than the target images I1 to I4 can be obtained.

[0062] FIG. 11 is a diagram illustrating an output image I' obtained by the masking process S106. In the super-resolution image I, jaggies (jagged edges resulting from the image guide 13q being composed of multiple cores) can be seen at the boundaries of the end faces of the image guide 13q included as the subject. On the other hand, in the output image I', jaggies cannot be seen at the boundaries of the end faces of the image guide 13q included as the subject. In other words, it was confirmed that by performing the masking process S106, an output image I' with fewer jaggies than the super-resolution image I can be obtained.

[0063] (Modification of image processing method) A modified example of the image processing method S100 will be described with reference to Fig. 12. Fig. 12 is a flow chart showing the flow of the image processing method S100' according to this modified example.

[0064] The image processing method S100′ according to this modified example (1) adds a white standard plate imaging process S110, a mesh image acquisition process S111, and a transparency process S112 to the image processing method S100 described above, and (2) replaces the averaging process S105b, which takes a simple average, included in the super-resolution process S105 described above with an averaging process S105c, which takes a weighted average.

[0065] The white standard plate imaging process S110 and the mesh image acquisition process S111 are processes executed between the white balance adjustment process S102 and the object imaging process S103 described above.

[0066] In the white standard plate imaging process S110, the user positions the objective lens 13r of the endoscopic probe 13 toward the white standard plate, and the camera device 12 images the end face of the image guide 13q on the image guide plug portion 13c side. In the mesh image acquisition process S111, the image processing device 10 acquires the image captured in the white standard plate imaging process S110 from the camera device 12 and stores it in the memory 102. The image captured in the white standard plate imaging process S101 and acquired in the mesh image acquisition process S111 is a color image in which areas corresponding to the cores of the image guide 13q are bright and other areas are dark. The image processing device 10 converts this color image into a grayscale image for use. The monochrome image obtained by converting this color image into a grayscale image will be referred to as a mesh image I0' below. The difference between the mesh image I0 and the mesh image I0 described above is that the mesh image I0 is a color image captured before white balance adjustment, while the mesh image I0' is a monochrome image obtained by grayscaling a color image captured after white balance adjustment.

[0067] The transparency processing S112 and the core drawing processing S113 are pre-processing for the super-resolution processing S105, which are executed between the target image acquisition processing S104 and the super-resolution processing S105 described above.

[0068] The transparency process S112 is a process in which the image processing device 10, for each target image Ij (j is a natural number between 1 and n), refers to the mesh image I0′ captured in the white standard plate capturing process S110 and makes transparent or semi-transparent areas of the target image Ij other than the area corresponding to the core of the image guide 13q (i.e., the areas corresponding to the mesh and background in the mesh image I0′). The image generated by the transparency process S112 is an opaque color image in which the area corresponding to the core of the image guide 13q is highly opaque and the other areas are less opaque. This image will be referred to as the “transparent target image Ij” hereinafter. The pixel value of pixel (x, y) of the transparent target image Ij is, for example, the pixel value {Rj(x, y), Gj(x, y), Bj(x, y)} of pixel (x, y) of the target image Ij plus the pixel value A(x, y) of pixel (x, y) of the mesh image I0′ as an opaque component. That is, the pixel value at pixel (x, y) of the transparent target image Ij is {Rj(x, y), Gj(x, y), Bj(x, y), A(x, y)}.

[0069] In the alignment process S105a of the super-resolution process S105, for each transparent target image Ii (i is a natural number between 1 and n-1) other than the transparent target image In, the image processing device 10 (1) calculates a positional deviation vector vi of the transparent target image Ii relative to the transparent target image In, and (2) shifts the pixel value of each pixel of the transparent target image Ii by the positional deviation vector vi. The image obtained in the alignment process S105a for the transparent target image Ii is a color image with opacity representing an image of the object O formed on the end face of the image guide 13q on the image guide plug portion 13c side. This image will be referred to as the transparent target image Ii' hereinafter. Note that the alignment process S105a is not performed on the transparent target image In, but for convenience of explanation, the transparent target image In will also be referred to as the transparent target image In' hereinafter. Specific examples of the transparent target images I1', I2', ..., In' obtained in the alignment process S105a will be described later with reference to different drawings.

[0070] In the averaging process S105c of the super-resolution process S105, the image processing device 10 generates a super-resolution image I by performing a weighted average of the pixel values ​​{Rj'(x,y), Gj'(x,y), Bj'(x,y), Aj'(x,y)} of each transparent object image Ij' for each pixel (x,y). The opacity Aj'(x,y) is used as the weight in the weighted average. The image obtained in the averaging process S105c is a color image with opacity representing the image of the object O formed on the end face of the image guide plug portion 13c side of the image guide 13q. This image will be referred to as the super-resolution image I hereinafter. The pixel values ​​{R(x,y), G(x,y), B(x,y), A(x,y)} of the pixel (x,y) of the super-resolution image I are given, for example, by the following equation: A specific example of the super-resolution image I generated in the averaging process S105c will be described later with reference to a different drawing.

[0071] D(x,y)=A1'(x,y)+A2'(x,y)+...+An'(x,y), If D(x,y)≠0: R(x,y)={R1'(x,y)A1'(x,y)+R2'(x,y)A2'(x,y)+...+Rn'(x,y)An'(x,y)} / D(x,y), G(x,y)={G1'(x,y)A1'(x,y)+G2'(x,y)A2'(x,y)+...+Gn'(x,y)An'(x,y)} / D(x,y), B(x,y)={B1'(x,y)A1'(x,y)+B2'(x,y)A2'(x,y)+…+Bn'(x,y)An'(x,y)} / D(x,y), A(x,y)=[{A1'(x,y)} 2 +{A2'(x,y)} 2 +…+{An'(x,y)} 2 ] / D(x,y), If D(x,y)=0: R(x,y)=0, G(x,y)=0, B(x,y)=0, A(x,y)=0.

[0072] (Example of execution of image processing method according to modified example) An example of execution of the image processing method S100 according to the above-mentioned modified example will be described with reference to Figures 13 to 18. In this example, for simplicity, the number n of target images I1, I2, . . . , In is set to 4.

[0073] Fig. 13 is a diagram illustrating a transparent target image I1 obtained by the transparency processing S112. Fig. 14 is a diagram illustrating a transparent target image I2 obtained by the transparency processing S112. Fig. 15 is a diagram illustrating a transparent target image I3 obtained by the transparency processing S112. Fig. 16 is a diagram illustrating a transparent target image I4 obtained by the transparency processing S112. It can be seen that in each transparent target image Ii, the area other than the area corresponding to the core of the image guide 13q is made transparent.

[0074] 17 is a diagram illustrating a super-resolution image I obtained by the super-resolution process S105. In the target images I1 to I4, it is difficult to read the alphabet and letters contained as subjects. In contrast, in the super-resolution image I, it is easy to read the alphabet and letters contained as subjects. In other words, it was confirmed that by performing the super-resolution process S105, a super-resolution image I with higher visibility than the target images I1 to I4 can be obtained.

[0075] 17, the contrast between the alphabet and letters and their background is higher in the latter, and as a result, the visibility of the alphabet and letters is higher in the latter. This is because, in each target image Ii, the brightness of areas other than the area corresponding to the core of image guide 13q is low, and therefore, in the super-resolution image I shown in FIG. 10, a decrease in brightness within the field of view and image graininess are likely to occur. However, in each transparent target image Ii, the area other than the area corresponding to the core of image guide 13q is made transparent, and therefore, a decrease in brightness within the field of view and image graininess are less likely to occur.

[0076] Note that a decrease in brightness and image graininess occur, for example, when the pixel values ​​of pixels belonging to a low-brightness region of the target image Ij are averaged with the pixel values ​​of pixels belonging to a high-brightness region of another target image Ij', or when the pixel values ​​of pixels belonging to a low-brightness region of the target image Ij are averaged with the pixel values ​​of pixels belonging to a low-brightness region of another target image I'.

[0077] 18 is a diagram illustrating an output image I' obtained by the masking process S106. In the super-resolution image I, jaggies (jagged edges resulting from the image guide 13q being composed of multiple cores) can be seen at the boundaries of the end faces of the image guide 13q included as the subject. On the other hand, in the output image I', jaggies cannot be seen at the boundaries of the end faces of the image guide 13q included as the subject. In other words, it was confirmed that by performing the masking process S106, an output image I' with fewer jaggies than the super-resolution image I can be obtained.

[0078] (Further modification 1 of the image processing method) As a pre-processing for the super-resolution processing S105 described above, the image processing device 10 may perform core drawing processing on each transparent target image Ij.

[0079] In the core drawing process, the image processing device 10 first identifies an area (hereinafter also referred to as a "core area") in the transparent target image Ij that corresponds to the core of the image guide 13q. For example, a set of pixels having the same coordinates in the mesh image I0', whose pixel values ​​are equal to or greater than a predetermined threshold, is identified as the core area.

[0080] Next, the image processing device 10 resets the pixel values ​​of each pixel included in the core region of the transparent target image Ij for each of the red, green, and blue components so that they match either (a1) the pixel value of the pixel located at the center of the core region (hereinafter also referred to as the "center pixel value") or (a2) the average value of the pixel values ​​of each pixel included in the core region (hereinafter also referred to as the "average pixel value"). The image processing device 10 also sets the opacity of the pixel located at the center of the core region of the transparent target image Ij to 100%. The image processing device 10 then resets the opacity of each pixel in the transparent target image Ij whose distance r from the center of the core region is equal to or less than a predetermined threshold R so that (b1) the opacity decreases linearly as the distance r increases, or (b2) the opacity decreases according to a Gaussian distribution as the distance r increases. If the opacity resetting is not performed, the opacity of each pixel included in the core region will match the pixel value of the pixel having the same coordinates as the pixel in the mesh image I0′, as described above.

[0081] (a) of Figure 19 is an enlarged view of the transparent object image Ij and super-resolution image I obtained when the pixel values ​​of each pixel included in the core region are reset to the average pixel value for the red, green, and blue components, but the opacity is not reset.

[0082] FIG. 19(b) is an enlarged view of a transparent target image Ij and a super-resolution image I obtained when the pixel values ​​of each pixel included in the core region for the red, green, and blue components are reset to the average pixel value, and the opacity of each pixel whose distance r from the center of the core region is equal to or less than a predetermined threshold R is reset to linearly decrease as the distance r increases. Comparing FIG. 19(a) and FIG. 19(b) shows that resetting the opacity further reduces the decrease in brightness and graininess in the super-resolution image I. Note that a similar effect can be obtained when the pixel values ​​of each pixel included in the core region for each of the red, green, and blue components are set to the center pixel value.

[0083] FIG. 19(c) is an enlarged view of a transparent target image Ij and a super-resolution image I obtained when the pixel values ​​of each pixel included in the core region for the red, green, and blue components are reset to the average pixel value, and the opacity of each pixel whose distance r from the center of the core region is equal to or less than a predetermined threshold R is reset to decrease according to a Gaussian distribution as the distance r increases. Comparing FIG. 19(a) and FIG. 19(c) reveals that resetting the opacity can further reduce brightness reduction and image graininess in the super-resolution image I. Furthermore, comparing FIG. 19(b) and FIG. 19(c) reveals that resetting the opacity so that it decreases according to a Gaussian distribution as the distance r increases is more advantageous in further reducing brightness reduction and image graininess in the super-resolution image I than resetting the opacity so that it decreases linearly as the distance r increases. Note that the same effect can be obtained even if the pixel value of each pixel included in the core region is set as the central pixel value for each of the red, green, and blue components.

[0084] (Further modification 2 of the image processing method) In the above-mentioned super-resolution processing S105, the image processing device 10 may perform averaging processing S105c on all of the transparent target images I1, I2, ..., In, or may perform averaging processing S105c on only a portion of the transparent target images I1, I2, ..., In.

[0085] An example of the latter is a configuration in which transparent object images I1, I2, ..., In other than those with large amounts of positional deviation are subjected to the averaging process S105c. Transparent object images with large amounts of positional deviation have little overlap with the transparent object image In used as the reference image in the registration process S105a, and therefore have limited contribution to improving the visibility of the super-resolution image I. Conversely, they may cause noise to occur in the super-resolution image I. Therefore, by excluding transparent object images with large amounts of positional deviation from the averaging process S105c, it is possible to suppress the occurrence of noise in the super-resolution image I without impeding the effect of improving the visibility of the super-resolution image I.

[0086] In addition, a method for selecting a transparent target image with a large amount of positional deviation from the transparent target images I1, I2, ..., In includes, for example, a method for selecting a transparent target image Ii from the transparent target images I1, I2, ..., In whose positional deviation vector vi exceeds a predetermined threshold.

[0087] Alternatively, a configuration is conceivable in which transparent object images I1, I2, ..., In other than the transparent object images that have blurred or blurred, are subjected to the averaging process S105c. Transparent object images that are significantly blurred or blurred are difficult to accurately perform the alignment process S105a on, and the contours of the object O included as a subject are unclear, so that their contribution to improving the visibility of the super-resolution image I is limited. Conversely, they may even cause noise to occur in the super-resolution image I. For this reason, by excluding the transparent object images that have blurred or blurred from the averaging process S105c, it is possible to suppress the occurrence of noise in the super-resolution image I without impeding the effect of improving the visibility of the super-resolution image I.

[0088] Note that, among the transparent target images I1, I2, ..., In, a method for selecting a blurred or shaky transparent target image Ii may include, for example, a method for selecting a transparent target image Ii whose positional deviation vector differences |(vi-1)-(vi)|, |(vi+1)-(vi)| with the adjacent transparent target images Ii-1, Ii+1 exceed a predetermined threshold. However, the method for selecting a blurred or shaky transparent target image Ii is not limited to this, and any known method may be adopted.

[0089] (Further modification example 3 of the image processing method) As post-processing of the super-resolution processing S105 described above, the image processing device 10 may perform a base image generation process in which a base image is generated by smoothing the last acquired target image In, and an output image generation process in which an output image is generated by superimposing the super-resolution image I on the generated base image. Here, the smoothing method is not particularly limited. The base image may be generated by smoothing using an averaging filter, or may be generated by smoothing using a Gaussian filter.

[0090] If the opacity of a pixel is 0 in all transparent target images I1', I2', ..., In', the opacity of that image will also be 0 in super-resolution image I. In other words, that pixel will be transparent in super-resolution image I. When such a super-resolution image I is displayed, the background visible through the transparent pixels may be perceived as being transparent, giving an unnatural impression. For example, when an all-black background is used, when such a super-resolution image I is displayed, the background visible through the transparent pixels may be perceived as scattered black dots, giving an unnatural impression. By superimposing super-resolution image I on a base image generated by smoothing target image In, concerns about the background visible through the transparent pixels can be eliminated, making it possible to display super-resolution image I more naturally.

[0091] In this modified example, a configuration has been described in which a base image is generated by smoothing the last acquired target image In, but the present invention is not limited to this. That is, a configuration may be adopted in which a base image is generated by smoothing a target image Ii other than the last acquired target image In among the target images I1, I2, ..., In.

[0092] (summary) This embodiment includes the following aspects.

[0093] [Aspect 1] an acquisition process for acquiring a plurality of target images obtained by imaging one end of an image guide having a plurality of cores and facing the target; a super-resolution process for generating a super-resolution image by simply averaging or weighted averaging pixel values ​​of pixels corresponding to the same point of the object in the plurality of object images; Image processing methods.

[0094] According to this aspect, it is possible to obtain a super-resolution image having higher resolution or visibility than the target image.

[0095] [Aspect 2] As a pre-processing of the super-resolution processing, a transparency processing is further included in which an area other than an area corresponding to a core of the image guide in each of the plurality of target images is made transparent or semi-transparent, The super-resolution processing is a process of generating the super-resolution image by weighting pixel values ​​of pixels corresponding to the same point of the object in the plurality of target images after the pre-processing, using opacity as a weight. 2. The image processing method according to embodiment 1.

[0096] According to this aspect, it is possible to obtain a super-resolution image with higher visibility by suppressing the reduction in brightness and image roughness that can occur in the super-resolution image.

[0097] Aspect 3 The pre-processing further includes a core drawing process for setting a pixel value of each pixel included in an area corresponding to the core of the image guide in each of the plurality of target images to a pixel value of a pixel located at the center of the area. 3. An image processing method according to aspect 2.

[0098] According to this aspect, it is possible to obtain a super-resolution image with higher visibility by further suppressing the reduction in brightness and image roughness that may occur in the super-resolution image.

[0099] Aspect 4 The pre-processing further includes a core drawing process for setting the pixel value of each pixel included in a region corresponding to the core of the image guide in each of the plurality of target images to an average value of each pixel included in the region. 3. An image processing method according to aspect 2.

[0100] According to this aspect, it is possible to obtain a super-resolution image with higher visibility by further suppressing the reduction in brightness and image roughness that may occur in the super-resolution image.

[0101] Aspect 5 In the core drawing process, the opacity of each pixel in each of the plurality of target images whose distance from the center of the region is equal to or less than a threshold is set so as to decrease according to a Gaussian distribution as the distance increases. 5. The image processing method according to aspect 3 or 4.

[0102] According to this aspect, it is possible to obtain a super-resolution image with higher visibility by further suppressing the reduction in brightness and image roughness that may occur in the super-resolution image.

[0103] Aspect 6 The method further includes, as post-processing of the super-resolution processing, a base image generation process of generating a base image by smoothing any of the plurality of target images, and a superimposition process of generating an output image by superimposing the super-resolution image on the base image. 6. The image processing method according to any one of aspects 2 to 5.

[0104] According to this aspect, by eliminating the possibility that the background will be visible in the super-resolution image, a more natural super-resolution image can be obtained.

[0105] Aspect 7 The super-resolution processing is a process of generating the super-resolution image by taking a weighted average of pixel values ​​of pixels corresponding to the same point of the object in a target image determined to have a small positional deviation among the plurality of target images. 7. The image processing method according to any one of aspects 2 to 6.

[0106] According to this aspect, by reducing noise that may occur in a super-resolution image, a super-resolution image with higher visibility can be obtained.

[0107] Aspect 8 The super-resolution processing is a process of generating the super-resolution image by taking a weighted average of pixel values ​​of pixels corresponding to the same point of the object in a target image determined to be free of blur or shake among the plurality of target images. 7. The image processing method according to any one of aspects 2 to 6.

[0108] According to this aspect, by reducing noise that may occur in a super-resolution image, a super-resolution image with higher visibility can be obtained.

[0109] Aspect 9 An image processing device comprising at least one processor, The processor: an acquisition process for acquiring a plurality of target images obtained by imaging one end of an image guide having a plurality of cores and facing the target; and performing a super-resolution process to generate a super-resolution image by averaging or weighted averaging pixel values ​​of pixels corresponding to the same point of the object in the plurality of object images. Image processing device.

[0110] According to this aspect, it is possible to obtain a super-resolution image having higher resolution or visibility than the target image.

[0111] Aspect 10 The processor further performs a transparency process as a preprocessing of the super-resolution process, which makes transparent or semi-transparent an area other than an area corresponding to a core of the image guide in each of the plurality of target images; The super-resolution processing is a process of generating the super-resolution image by weighting pixel values ​​of pixels corresponding to the same point of the object in the plurality of target images after the pre-processing, using opacity as a weight. An image processing device according to embodiment 9.

[0112] According to this aspect, it is possible to obtain a super-resolution image with higher visibility by suppressing the reduction in brightness and image roughness that can occur in the super-resolution image.

[0113] Aspect 11 As the pre-processing, the processor further executes a core drawing process of setting a pixel value of each pixel included in an area corresponding to a core of the image guide in each of the plurality of target images to a pixel value of a pixel located at the center of the area. An image processing device according to embodiment 10.

[0114] According to this aspect, it is possible to obtain a super-resolution image with higher visibility by further suppressing the reduction in brightness and image roughness that may occur in the super-resolution image.

[0115] Aspect 12 An image processing program for operating at least one processor, the program comprising: an acquisition process for acquiring a plurality of target images obtained by imaging one end of an image guide having a plurality of cores and facing the target; a super-resolution process for generating a super-resolution image by averaging or weighted averaging pixel values ​​of pixels corresponding to the same point of the object in the plurality of object images; Image processing program.

[0116] According to this aspect, it is possible to obtain a super-resolution image having higher resolution or visibility than the target image.

[0117] Aspect 13 The processor further executes a transparency process as a preprocessing of the super-resolution process, which makes transparent or semi-transparent an area other than an area corresponding to a core of the image guide in each of the plurality of target images; The super-resolution processing is a process of generating the super-resolution image by weighting pixel values ​​of pixels corresponding to the same point of the object in the plurality of target images after the pre-processing, using opacity as a weight. An image processing program according to embodiment 12.

[0118] According to this aspect, it is possible to obtain a super-resolution image with higher visibility by further suppressing the reduction in brightness and image roughness that may occur in the super-resolution image.

[0119] Aspect 14 and causing the processor to further execute, as the pre-processing, a core drawing process of setting a pixel value of each pixel included in an area corresponding to a core of the image guide in each of the plurality of target images to a pixel value of a pixel located at the center of the area. An image processing program according to embodiment 13.

[0120] According to this aspect, it is possible to obtain a super-resolution image with higher visibility by further suppressing the reduction in brightness and image roughness that may occur in the super-resolution image.

[0121] Aspect 15 The image processing device according to any one of aspects 9 to 11; an endoscopic probe including the image guide, Endoscopy system.

[0122] According to this aspect, it is possible to realize an endoscope system that can obtain a super-resolution image with higher resolution or visibility than the target image.

[0123] (Additional notes) The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0124] 1. Endoscopy system 10 Image processing device 101 processors 102 memory 103 Storage 104 Input / Output Interface 105 Bus 11 Light source device 12 Camera equipment 13 Endoscopic probe 13p Light Guide 13q Image Guide

Claims

1. an acquisition process for acquiring a plurality of target images obtained by imaging one end of an image guide having a plurality of cores and facing the target; a super-resolution process for generating a super-resolution image by simply averaging or weighted averaging pixel values ​​of pixels corresponding to the same point of the object in the plurality of object images; Image processing methods.

2. As a pre-processing of the super-resolution processing, a transparency processing is further included in which an area other than an area corresponding to a core of the image guide in each of the plurality of target images is made transparent or semi-transparent, The super-resolution processing is a process of generating the super-resolution image by weighting pixel values ​​of pixels corresponding to the same point of the object in the plurality of target images after the pre-processing, using opacity as a weight. The image processing method according to claim 1 .

3. The pre-processing further includes a core drawing process for setting a pixel value of each pixel included in an area corresponding to the core of the image guide in each of the plurality of target images to a pixel value of a pixel located at the center of the area. The image processing method according to claim 2 .

4. The pre-processing further includes a core drawing process for setting the pixel value of each pixel included in a region corresponding to the core of the image guide in each of the plurality of target images to an average value of each pixel included in the region. The image processing method according to claim 2 .

5. In the core drawing process, the opacity of each pixel in each of the plurality of target images whose distance from the center of the region is equal to or less than a threshold is set so as to decrease according to a Gaussian distribution as the distance increases.

5. The image processing method according to claim 3 or 4.

6. The method further includes, as post-processing of the super-resolution processing, a base image generation process of generating a base image by smoothing any of the plurality of target images, and a superimposition process of generating an output image by superimposing the super-resolution image on the base image. The image processing method according to any one of claims 2 to 4.

7. The super-resolution processing is a process of generating the super-resolution image by taking a weighted average of pixel values ​​of pixels corresponding to the same point of the object in a target image determined to have a small positional deviation among the plurality of target images. The image processing method according to any one of claims 2 to 4.

8. The super-resolution processing is a process of generating the super-resolution image by taking a weighted average of pixel values ​​of pixels corresponding to the same point of the object in a target image determined to be free of blur or shake among the plurality of target images. The image processing method according to any one of claims 2 to 4.

9. An image processing device comprising at least one processor, The processor: an acquisition process for acquiring a plurality of target images obtained by imaging one end of an image guide having a plurality of cores and facing the target; and performing a super-resolution process to generate a super-resolution image by averaging or weighted averaging pixel values ​​of pixels corresponding to the same point of the object in the plurality of object images. Image processing device.

10. The processor further performs a transparency process as a preprocessing of the super-resolution process, which makes transparent or semi-transparent an area other than an area corresponding to a core of the image guide in each of the plurality of target images; The super-resolution processing is a process of generating the super-resolution image by weighting pixel values ​​of pixels corresponding to the same point of the object in the plurality of target images after the pre-processing, using opacity as a weight. The image processing device according to claim 9 .

11. As the pre-processing, the processor further executes a core drawing process of setting a pixel value of each pixel included in an area corresponding to a core of the image guide in each of the plurality of target images to a pixel value of a pixel located at the center of the area. The image processing device according to claim 10.

12. An image processing program for operating at least one processor, the program comprising: an acquisition process for acquiring a plurality of target images obtained by imaging one end of an image guide having a plurality of cores and facing the target; a super-resolution process for generating a super-resolution image by averaging or weighted averaging pixel values ​​of pixels corresponding to the same point of the object in the plurality of object images; Image processing program.

13. The processor further executes a transparency process as a preprocessing of the super-resolution process, which makes transparent or semi-transparent an area other than an area corresponding to a core of the image guide in each of the plurality of target images; The super-resolution processing is a process of generating the super-resolution image by weighting pixel values ​​of pixels corresponding to the same point of the object in the plurality of target images after the pre-processing, using opacity as a weight. The image processing program according to claim 12.

14. and causing the processor to further execute, as the pre-processing, a core drawing process of setting a pixel value of each pixel included in an area corresponding to a core of the image guide in each of the plurality of target images to a pixel value of a pixel located at the center of the area. The image processing program according to claim 13.

15. An image processing device according to any one of claims 9 to 11; an endoscopic probe including the image guide, Endoscopy system.

Images