Endoscope system, endoscope, and distance calculation method

Abstract

An endoscope system (10) includes a pattern light projecting section (250), an imaging section (270), and a processing section (110). The pattern light projecting section projects pattern light (PT1 to PT3) having a stripe-like or lattice-like pattern and the phase of the pattern and the wavelength of the light being different from each other to an object (5). The imaging section photographs an image of the object to which the pattern light is projected as one frame. The processing section calculates the distance to the object or the shape of the object from the image of one frame.

Description

TECHNICAL FIELD

[0001] The present application relates to an endoscope system, an endoscope, and a distance calculation method. BACKGROUND

[0002] Sometimes, it is desired to measure the size of a lesion or the like in an endoscope, but in order to accurately measure the size, it is necessary to measure the distance from the scope to the lesion. In the past, ranging has been performed in three-dimensional measurement of an object or the like, and as this ranging method, a parallax method, a TOF (Time Of Flight) method, and a structured light method are known. As a viewpoint for evaluating whether these ranging methods are suitable for an endoscope, real-time measurement, real-time processing, and slimming are considered. Real-time measurement is a viewpoint indicating whether measurement is completed in a short time for high-precision ranging of a subject having motion such as a living body. Real-time processing is a viewpoint indicating whether ranging operation can be processed in a short time for real-time presentation of information when a subject is observed. Slimming is a viewpoint indicating whether the diameter of the tip of the scope is too thick when a ranging mechanism is mounted on the tip of the scope.

[0003] The parallax method is also called stereo vision, and a parallax image is obtained by two imaging systems. The parallax method can obtain a parallax image with one frame, and thus can perform real-time measurement. However, the operation load of parallax is large, real-time processing is difficult, and two imaging systems are required, and thus slimming is difficult.

[0004] The TOF method is a method of measuring the time until a reflected wave of light reaches an image sensor. The TOF method can perform ranging with one frame, and thus can perform real-time measurement, and the processing load of converting time to distance is small, and thus real-time processing can be performed. However, since a TOF-dedicated image sensor is provided in addition to an image sensor for capturing an observation image, slimming is difficult.

[0005] In the structured light method, the processing load of converting the imaging method of each pattern light to distance is small, and thus real-time processing can be performed, and since the projection mechanism of the pattern light is smaller than the image sensor, slimming can be achieved compared to other ranging methods. However, the existing structured light method performs imaging of one frame for one pattern projection, and thus in order to capture all pattern projections, imaging of multiple frames is required. For example, a ranging method is disclosed in Patent Literature 1 in which three light sources and a grating are included, the light sources are sequentially turned on one by one, and thus three pattern lights having mutually different phases are sequentially projected, three images are obtained by capturing a subject on which each pattern light is projected, and distance is calculated from the three images.

[0006] Prior Art Documents

[0007] Patent Documents

[0008] Patent Document 1: U.S. Patent Application Publication No. 2009 / 0225321 Specification SUMMARY

[0009] Problems to be Solved by the Invention

[0010] As described above, in an endoscope, from the viewpoint of important miniaturization, it is preferable to adopt a structured light method, but the existing structured light method requires a plurality of frames to be captured, and thus it is difficult to perform real-time measurement, and there is a problem that it is not suitable for distance measurement of a living body or the like having motion with high accuracy.

[0011] Means for Solving the Problems

[0012] One embodiment of the present application relates to an endoscope system including: a pattern light projecting section that projects first to nth pattern lights (n is an integer of 2 or more) having a stripe pattern or a lattice pattern and phases of the patterns different from each other to an object; an imaging section that captures an image of the object to which the first to nth pattern lights are projected as one frame; and a processing section that calculates a distance to the object or a shape of the object from the one frame of image.

[0013] Another embodiment of the present application relates to an endoscope system including: a pattern light projecting section that projects first to nth pattern lights (n is an integer of 2 or more) having a stripe pattern or a lattice pattern and phases of the patterns different from each other to an object; an imaging section that captures an image of the object to which the first to nth pattern lights are projected; and a processing section that calculates a distance to the object or a shape of the object from the image captured by the imaging section, the pattern light projecting section including: a DOE (Diffractive Optical Element); an entrance section that causes parallel light including components of first to nth wavelengths different from each other to be incident to the DOE; and a slit section, the exit light of the DOE being incident to the slit section, the first to nth pattern lights of the first to nth wavelengths being projected to the object. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 is a configuration example of an endoscope system.

[0015] Figure 2 is a diagram for explaining a first action example of an endoscope system.

[0016] Figure 3is a view for explaining a second example of operation of the endoscope system.

[0017] Figure 4 is a view for explaining a wavelength of the pattern light.

[0018] Figure 5 is an example of spectral characteristics of an image sensor possessed by the imaging section.

[0019] Figure 6 is a first detailed configuration example of the endoscope system.

[0020] Figure 7 is a second detailed configuration example of the endoscope system.

[0021] Figure 8 is a first detailed configuration example of the pattern light projection section.

[0022] Figure 9 is a second detailed configuration example of the pattern light projection section. DETAILED DESCRIPTION

[0023] Hereinafter, the present embodiment will be described. Note that the present embodiment described below does not unduly limit the content recited in the claims. Furthermore, all the structures described in the present embodiment are not necessarily essential structural elements of the present application.

[0024] 1. Configuration Example

[0025] Figure 1 is a configuration example of the endoscope system 10. The endoscope system 10 includes a pattern light projection section 250, an imaging section 270, a processing section 110, and an observation-use illumination light exit section 260. In the endoscope system 10, the pattern light projection section 250 projects the first to nth pattern lights to the subject 5, the imaging section 270 images the image of the subject 5 on which the pattern lights are projected, and the processing section 110 calculates the distance to the subject 5 or the shape of the subject 5 on the basis of the image. Figure 1 The endoscope system 10 includes a control device 100, and the processing section 110 that performs the distance measurement processing is included in the control device 100, but is not limited thereto, and can be configured to be provided in an information processing device or the like provided outside the control device 100. The endoscope system 10 is, for example, a medical endoscope system, and can be a video endoscope used in the upper digestive tract or the lower digestive tract, or a rigid scope used in a surgical operation, or the like.

[0026] The pattern light projection section 250 projects the first to nth pattern lights to the subject 5. n is an integer of two or more, and is set to n = 3 here. The pattern lights PT1 to PT3 are the first to third pattern lights. The pattern lights PT1 to PT3 have a stripe pattern or a lattice pattern, and the phases and the wavelengths of the lights are different from each other. The imaging section 270 images the image of the subject 5 on which the pattern lights PT1 to PT3 are projected as one frame. The processing section 110 calculates the distance to the subject 5 or the shape of the subject 5 on the basis of the one frame.

[0027] Here, a frame is the exposure period used to capture one image. For example, when capturing moving images, frames repeat periodically, but the image of that one frame is captured within one of those frames. For example, as in... Figure 2 , Figure 3 As described later, between frames in which the observed images are captured, the image of the subject 5, on which patterned lights PT1 to PT3 are projected, is captured.

[0028] According to this embodiment, an image of the subject 5 projected with patterned lights PT1 to PT3 is captured in a single frame, thus enabling the capture of images required for distance measurement using structured light in a short time. Therefore, real-time measurement is possible in structured light mode, allowing for high-precision distance measurement of moving objects such as living organisms. Since the patterned lights PT1 to PT3 have different wavelengths, the differences in these wavelengths can be used to separate the subject image projected with each patterned light PT1 to PT3 from the single-frame image, and the distance can be calculated based on this information.

[0029] Furthermore, the endoscope system 10 can also provide AI-based diagnostic assistance. In this case, information about the distance or shape of the subject 5 is input into the AI ​​along with the observed image, thereby improving the accuracy of the diagnostic assistance.

[0030] Furthermore, the shape obtained through measurement is important as evidence when diagnosing whether a site of interest is a lesion. For example, in the case of a polyp, the size determination of the polyp provides important evidence in diagnosing whether it is cancerous.

[0031] Below, on Figure 1 The detailed structure of the example will be described below. The patterned light projection unit 250 includes first to third light sources S1 to S3 that emit light of first to third wavelengths λ1 to λ3, and a slit unit 252 provided with multiple slits. The patterned light projection unit 250 is also referred to as a patterned light projection device. As described above, the patterned lights PT1 to PT3 are striped or grid-like.

[0032] The striped pattern is a pattern in which parallel lines repeat periodically or approximately periodically. When the pattern light PT1 to PT3 are striped, a plurality of linear slits are provided in the slit portion 252. These linear slits are parallel to each other and arranged in a direction orthogonal to the linear slits.

[0033] The lattice pattern is a pattern in which lines of the first line group and the second line group are orthogonal and in which the lines in each line group are periodically or substantially periodically repeated. In the case where the pattern light PT1 to PT3 is in a lattice pattern, the slit portion 252 is provided with slits in a lattice pattern. That is, the slit portion 252 is provided with a first plurality of linear slits and a second plurality of linear slits orthogonal to the first plurality of linear slits. The slit portion 252 is also referred to as a grating. Further, the slit portion 252 is a structure in which a slit is provided on a plate-shaped member, and the slit portion 252 is also referred to as a slit plate.

[0034] The light sources S1 to S3 emit light whose peak wavelength is in the spectrum of wavelengths λ1 to λ3. The light sources S1 to S3 emit light whose spectrum is sufficiently separated from each other in line width, for example, light having a line width of several nm to several tens of nm. As described later in the second embodiment, the light sources S1 to S3 are virtual light sources generated using a laser diode and a DOE (Diffractive Optical Element), or the like. Alternatively, the light sources S1 to S3 can be constituted by a light emitting element such as a light emitting diode and a band pass filter. In the case where the pattern light PT1 to PT3 is in a stripe pattern, the light sources S1 to S3 are linear light sources parallel to the linear slits, respectively. The light sources S1 to S3 are arranged in a plane parallel to the plane of the slit portion 252, and are arranged in the same direction as the direction in which the linear slits are arranged. In the case where the pattern light PT1 to PT3 is in a lattice pattern, the light sources S1 to S3 are point light sources arranged at different positions in a plane parallel to the plane of the slit portion 252.

[0035] Light from the light sources S1 to S3 passes through the slits of the slit portion 252, and thereby the pattern light PT1 to PT3 is generated. When the pattern light PT1 to PT3 is projected onto a flat object parallel to the plane of the slit portion 252, the pattern light PT1 to PT3 is in a stripe pattern or a lattice pattern, and the phases of the stripes or lattices are different from each other. For example, in the case of a stripe pattern, one period of the stripe is set to 360 degrees of phase, and with respect to a certain position of 0 degrees as a reference, the stripes of the pattern light PT1 to PT3 are at ph1 degrees to ph3 degrees, and the ph1 degrees to ph3 degrees are different values from each other. The positions of the light sources S1 to S3 are different, and thus the phase relationship of the pattern light PT1 to PT3 varies depending on the distance to the object. However, in the case where the light sources S1 to S3 are arranged at positions very close to each other, the phase relationship of the pattern light PT1 to PT3 can be regarded as constant regardless of the distance to the object.

[0036] The observation illuminating light emitting section 260 emits observation illuminating light for capturing an observation image toward the object 5. The observation image is an image for the user to observe the object 5. The observation illuminating light is also referred to as general light, and the observation image is also referred to as general image, in the sense of being contrasted with the ranging light and the image. The observation illuminating light is light of a light distribution characteristic corresponding to the observation purpose, and is, for example, white light or special light. An example of the special light is illuminating light for NBI composed of narrow-band light of green and narrow-band light of blue. The observation illuminating light emitting section 260 is also referred to as an observation illuminating light emitting device.

[0037] The imaging section 270 includes an objective lens that images the object 5 and an image sensor that captures the object 5 imaged by the objective lens. Either one of the pattern light projecting section 250 and the observation illuminating light emitting section 260 emits light. When the pattern light projecting section 250 emits light, the imaging section 270 captures an image of the object 5 on which the pattern light PT1 to PT3 is projected, and when the observation illuminating light emitting section 260 emits light, the imaging section 270 captures an observation image. The imaging section 270 includes one image sensor, and this common image sensor captures the observation illuminating light and the pattern light.

[0038] The control device 100 is a device that performs control of the endoscope system 10 and image processing and the like. The control device 100 is connected to a scope on which the pattern light projecting section 250, the observation illuminating light emitting section 260, and the imaging section 270 are provided. The control device 100 includes a processing section 110.

[0039] The processing section 110 is realized by a circuit device in which a plurality of circuit components are mounted on a substrate. Alternatively, the processing section 110 can be an integrated circuit device such as a processor, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array). The processor is a CPU, a microcomputer, a DSP, or the like. In the case where the processing section 110 is a processor, the processor executes a program that describes the operation of the processing section 110, thereby realizing the operation of the processing section 110. The program is stored in a memory, not shown, for example. The processing section 110 is also referred to as a processing circuit or a processing device.

[0040] The processing section 110 calculates the phase at each position of the image from the image captured by the imaging section 270 when the pattern light PT1 to PT3 is projected, and calculates the distance to the subject 5 up to each position of the image from the phase. The information of the distance is information such as a Z map in which the distance is calculated for each pixel, and indicates the three-dimensional shape of the subject 5. The processing section 110 calculates the shape of the subject 5 from the calculated distance. As the information of the calculated shape, various information such as the length, width, major diameter, minor diameter, height, or depth of the site of interest, or the outline of the site of interest, or information obtained by combining some of them can be envisaged.

[0041] The method of calculating the length in the actual space from the length on the image will be described taking the length of the site of interest as an example. The processing section 110 calculates the length of the site of interest in the actual space from the length of the site of interest on the image and the distance of the site of interest. That is, from the field angle of view of the imaging section 270 and the length of the site of interest on the image, the angle of view of the site of interest as viewed from the imaging section 270 is known. The processing section 110 calculates the length of the site of interest in the actual space from the angle of view and the distance of the site of interest. Approximately, the value obtained by multiplying the angle of view by the distance of the site of interest becomes the length of the site of interest in the actual space.

[0042] The processing section 110 can also perform inclination correction when calculating the shape of the site of interest. That is, the processing section 110 calculates the inclination of the site of interest from the distances of the periphery of the site of interest, and performs inclination correction on the length or the like calculated on the image, thereby converting it to the length or the like when the site of interest is directly facing the imaging section 270, and outputs the length or the like information after the correction as the shape information of the subject 5.

[0043] The processing section 110 can also calculate not only the shape of the site of interest but also the distance between two sites. For example, it is envisaged that the distance between a polyp and the anus is measured in a colonoscope. In a case where the two sites are separated and not captured in one image, the distance is measured by dividing the path. That is, the processing section 110 acquires a plurality of pattern images in the path between the two sites, connects the distances calculated from the respective pattern images, and thereby calculates the distance between the two sites. The distance between the lesion and the anus becomes a material for determining whether or not it is a suitable subject for a function-preserving surgery.

[0044] In the present embodiment, the endoscope system 10 switches the pattern light PT1 to PT3 and the observation illumination light to irradiate the subject 5, and acquires the image based on the pattern light PT1 to PT3 and the observation image in units of one frame. As one example of the operation, two examples will be shown below. Hereinafter, the image captured when the pattern light PT1 to PT3 is projected will be referred to as a pattern image.

[0045] Figure 2is a diagram illustrating a first example of operation of the endoscope system 10. The observation illumination light emitting section 260 emits observation illumination light in F1, F3, F5, and F7 among the consecutive frames F1 to F7. In Figure 2 , the high level of the waveform indicates lighting, and the low level indicates extinguishing. The imaging section 270 performs imaging in the frames F1, F3, F5, and F7 in which the observation illumination light is emitted. This image becomes an observation image.

[0046] The pattern light projecting section 250 projects the pattern light PT1 to PT3 in the frames in which the observation illumination light is not emitted. In Figure 2 , for example, in the frame F4 after the input of the trigger signal, the pattern light projecting section 250 projects the pattern light PT1 to PT3, and the imaging section 270 captures a pattern image. In Figure 2 , the high level of the waveform indicates projection of the pattern light PT1 to PT3, and the low level indicates extinguishing of the pattern light PT1 to PT3. The time for which the pattern light PT1 to PT3 is projected is arbitrary, but from the viewpoint of the measurement accuracy, it is preferable to be short. For example, this time can be set in accordance with the brightness of the pattern light PT1 to PT3 and the necessary measurement accuracy, and the like. The processing section 110 performs a measurement processing based on the pattern image captured in the frame F4. In Figure 2 , the high level of the waveform indicates execution of the measurement processing. In addition, in the endoscope, due to the problem of the light quantity of the observation light source, it is sometimes desirable to extend the accumulation time for obtaining the observation image as much as possible. In this case, it is considered to obtain the observation image by regarding 2 frames of Figure 2 as 1 frame, to suspend the obtaining of the observation image only at the time of input of the trigger signal, and to irradiate the pattern light to obtain the pattern image. In this case, when the observation image is displayed, it is necessary to somehow display the observation image of the previous frame, and the like.

[0047] The trigger signal is input to the processing section 110, for example, by user operation. For example, a button for measurement instruction is provided in the scope operation section, and when this button is pressed, the trigger signal is input from the scope operation section to the processing section 110. Alternatively, the trigger signal can be generated inside the processing section 110. For example, the processing section 110 determines whether or not a site of interest exists in the observation image, and when the site of interest is detected in the observation image, the trigger signal is generated. The site of interest is, for example, a lesion such as cancer or a polyp. The processing section 110 detects the site of interest by AI processing or the like, and generates the trigger signal. Furthermore, the processing section 110 can further perform AI processing using the measurement result of the site of interest detected by the AI processing, thereby improving the determination accuracy of the site of interest. For example, the AI processing uses the size or shape of the site of interest obtained by the measurement, and the like.

[0048] Figure 3 is a diagram illustrating a second example of operation of the endoscope system 10. As with Figure 2Likewise, the observation image is captured in frames F1, F3, F5, F7. In Figure 3 In the present embodiment, the pattern light PT1 to PT3 is projected by the pattern light projection section 250 to the object 5 at the same time regardless of the trigger signal. The "at the same time" means that there is at least a time point at which the pattern light PT1 to PT3 is projected. The projection period of the pattern light PT1 to PT3 can not be consistent, but it is more preferable that the projection period is consistent.

[0049] As described above, in the present embodiment, the pattern light PT1 to PT3 is projected by the pattern light projection section 250 to the object 5 at the same time. The "at the same time" means that there is at least a time point at which the pattern light PT1 to PT3 is projected. The projection period of the pattern light PT1 to PT3 can not be consistent, but it is more preferable that the projection period is consistent. Figure 2 Figure 3 In the present embodiment, the pattern light PT1 to PT3 is projected by the pattern light projection section 250 to the object 5 at the same time. The "at the same time" means that there is at least a time point at which the pattern light PT1 to PT3 is projected. The projection period of the pattern light PT1 to PT3 can not be consistent, but it is more preferable that the projection period is consistent.

[0050] According to the present embodiment, the pattern light PT1 to PT3 is projected at the same time, and thus the pattern light PT1 to PT3 is projected without time difference compared to the method in which the observation image is captured one frame by one frame. Thereby, the pattern images based on the three pattern lights are obtained at the same time for the object having movement such as a living body, and high-precision distance measurement is possible.

[0051] Further, in the present embodiment, in the first frame, the observation illumination light is emitted by the observation illumination light emission section 260 to the object 5, and the observation image is captured by the imaging section 270. In the second frame different from the first frame, the pattern light PT1 to PT3 is projected by the pattern light projection section 250 to the object 5, and the pattern image is captured by the imaging section 270. The first frame corresponds to any one of F1, F3, F5, F7 of Figure 2 and Figure 3 . The second frame corresponds to F4 of Figure 2 , or any one of F2, F4, F6 of Figure 3 .

[0052] According to the present embodiment, the observation image can be captured and prompted to the user, and the distance measurement is performed in the background thereof, and the information of the distance or the shape obtained by the distance measurement can be prompted to the user together with the observation image.

[0053] 2. Wavelength of light source and distance measurement processing

[0054] Figure 4 is a diagram illustrating the wavelengths λ1 to λ3 of the pattern light PT1 to PT3. In Figure 4 ​The spectroscopic properties of hemoglobin (Hb) and oxidized hemoglobin (HbO2) are shown in the figure. Furthermore, hemoglobin (Hb) and oxidized hemoglobin (HbO2) will be referred to together as hemoglobin below.

[0055] In medical endoscopes, the object of observation is the interior of a living organism, but the spectroscopic properties of a living organism are mainly determined by the spectroscopic properties of hemoglobin. Therefore, in this embodiment, the wavelengths λ1 to λ3 of the patterned light PT1 to PT3 are set according to the spectroscopic properties of hemoglobin.

[0056] Monochromatic light is used in existing structured light methods to ensure equal reflectance of each pattern, unaffected by the spectral characteristics of the subject. When using different wavelengths λ1 to λ3 as in this embodiment, it is preferable that the reflectance of the subject is the same at each wavelength. Therefore, wavelengths with the flattest absorption coefficient in the spectral characteristics of hemoglobin are used. Specifically, wavelengths λ1 to λ3 are set to avoid large absorption peaks below 450 nm and surrounding regions with large variations in absorption coefficient.

[0057] Specifically, the wavelengths λ1 to λ3 of the patterned light PT1 to PT3 fall within the range of 460 nm to 700 nm. Within this range, the spectroscopic properties of hemoglobin exhibit minimal variation, resulting in approximately the same reflectivity for each pattern. Furthermore, it is preferable that the wavelengths λ1 to λ3 of the patterned light PT1 to PT3 fall within the range of 460 nm to 520 nm. Within this range, the spectroscopic properties of hemoglobin show even less variation compared to the range of 460 nm to 700 nm.

[0058] Mucosa used as the target of medical endoscopes has a large number of capillaries near its surface. For example... Figure 4 As shown, hemoglobin absorbs light strongly in wavelengths below 460 nm, resulting in very weak reflected light at capillary locations. In structured light methods, distance is determined by the ratio of light intensity at each point of the patterned lights PT1-PT3. However, when factors other than the intensity of the patterned light exist, such as differences in reflectivity due to capillaries, resulting in varying reflected light intensity, accurate distance measurement is impossible. For example, if the wavelength of one of the patterned lights PT1-PT3 is below 460 nm, the reflected light from the capillaries is very weak compared to the other patterned lights, leading to an inaccurate light intensity ratio and making it impossible to accurately measure the distance to the capillaries. In this embodiment, the wavelengths of the patterned lights PT1-PT3 are set to a range of 460 nm to 700 nm or 460 nm to 520 nm, thus ensuring an accurate reflected light intensity ratio and enabling accurate distance measurement.

[0059] Next, the distance measurement process for determining the distance from images captured by simultaneously illuminating patterned lights PT1 to PT3 will be explained. The following explanation will use the cases of λ1 = 520 nm, λ2 = 500 nm, and λ3 = 480 nm as examples.

[0060] Figure 5 This is an example of the light-splitting characteristics of the image sensor in the camera unit 270. The image sensor has pixels of colors 1 to n that receive light of colors 1 to n. Here, let n = 3, the image sensor be an RGB primary color Bayer type, where R is color 1, G is color 2, and B is color 3. Figure 5 In this context, KR represents the relative sensitivity of the R pixel, KG represents the relative sensitivity of the G pixel, and KB represents the relative sensitivity of the B pixel.

[0061] like Figure 5 As shown, the photosensitivity of the i-th color pixel at the j-th wavelength λj is set to a. ij i and j are integers greater than 1 and less than n. The processing unit 110 determines the sensitivity a based on the photosensitivity. ij The image of the subject 5 when each pattern light is projected is extracted using the intensity values ​​p1, p2, and p3 of R, G, and B in the pattern image. Then, the processing unit 110 calculates the distance to the subject 5 or the shape of the subject 5 based on the phase of the image of the subject 5 when each pattern light is projected. The details of this processing will be explained below.

[0062] First, the intensity values ​​p1, p2, p3 of R, G, and B that can be obtained from the pattern image and the image of the subject 5, which is the object to be obtained, when each pattern light is projected, are related by the following equation (1). q1 is the intensity value in the image of the subject 5 when pattern light PT1 with wavelength λ1 is projected. Similarly, q2 and q3 are the intensity values ​​in the image of the subject 5 when pattern light PT2 and PT3 with wavelengths λ2 and λ3 are projected. Let the position in the pattern image be (x, y). The position (x, y) is, for example, pixel coordinates. In the following equation (1), q1, q2, q3 on the left and p1, p2, p3 on the right are intensity values ​​related to the same position (x, y).

[0063]

Mathematical Formula 1

[0064]

[0065] will be a ij Let the matrix with elements be A. Pre-select wavelengths λ1 to λ3 such that the row vectors of matrix A are linearly independent, i.e., (a...). 11 ,a 12 ,a 13 ), (a 21 ,a 22 ,a 23) and (a 31 32 33 ) become linearly independent. In this way, the matrix A has an inverse matrix, and thus, the above equation (1) can be transformed into the following equation (2). The processing section 110 calculates the intensity values q1, q2, q3 at each (x, y) using the following equation (2).

[0066] [Math. 2]

[0067]

[0068] In addition, the following equations (3), (4) are obtained by rewriting the recorded forms of the above equations (1), (2) into other forms, and have the same meanings. ij represents the ij component of the matrix A.

[0069] [Math. 3]

[0070]

[0071] [Math. 4]

[0072] q j = (A -1 ) ji p i ···(4)

[0073] As shown in the following equation (5), the processing section 110 converts the intensity values q1, q2, q3 into a phase WPh using a LUT (Look Up Table). The LUT is a table in which combinations of the intensity values q1, q2, q3 are associated with the phase WPh, and is stored in advance in a memory or the like in the control device 100. The phase WPh is a phase in which wrapping has occurred, and the processing section 110 performs unwrapping processing on the phase WPh, and calculates a distance based on the phase after the unwrapping processing. The unwrapping processing is processing in which phases that become discontinuous at the boundary of the period of the fringe are connected to become continuous phases. That is, in a phase in which wrapping has occurred, the phase of one fringe is 0 to 360 degrees, and the adjacent fringe is again 0 to 360 degrees, and the unwrapping processing is processing in which these phases are connected to become 0 to 720 degrees.

[0074] [Math. 5]

[0075]

[0076] If a reference surface is set for a certain reference distance, the phase of the pattern light in the reference surface is determined. The difference between the phase that becomes the reference and the phase calculated by the above processing represents the relative distance between the reference surface and the subject 5. That is, the processing section 110 calculates the distance to the subject 5 based on the difference between the phase that becomes the prescribed reference and the phase calculated by the above processing.​​

[0077] In Figure 1 which the positions of the light sources S1-S3 are very close to each other, the phase WPh can also be calculated by a function operation as shown in the following equation (6). arctan2 is a function that calculates the angle of a point (u, v) in uv rectangular coordinates when the arguments are set to v / u. The argument u of arctan2 can also be negative, and the range is -π to +π.

[0078] [Equation 6]

[0079]

[0080] According to the present embodiment, the phase can be determined from the image of 1 frame in which the pattern light PT1-PT3 is projected at the same time, and the distance to the subject 5 can be calculated using the phase. Further, if the full space tabular method is used, the distance can be measured without converting the ratio of the pattern light PT1-PT3 of each point into the phase.

[0081] 3. Detailed Configuration Example

[0082] Figure 6 is a first detailed configuration example of the endoscope system 10. The endoscope system 10 includes a scope 200, a control device 100, and a display section 300. In addition, the same reference numerals are assigned to the structural elements already described in the above-described endoscope system 10, and the description of the structural elements is appropriately omitted. Figure 1

[0083] The scope 200 includes a flexible section 210 to be inserted into a living body, an operation section 220, a connector 240 to connect the scope 200 and the control device 100, the operation section 220, and a general-purpose cord 230 to connect the connector 240.

[0084] A pattern light projecting section 250, an observation illuminating light emitting section 260, and an imaging section 270 are provided at the distal end of the flexible section 210. The end portion of the flexible section 210 on the side opposite to the distal end is connected to the operation section 220. The operation section 220 is a device for performing angle operation of the flexible section 210, operation of a treatment instrument, operation of air and water feeding, and the like. An optical fiber 251, a light guide 261, and a signal line 271 are provided inside the flexible section 210, the operation section 220, and the general-purpose cord 230. The optical fiber 251 connects the pattern light projecting section 250 and the connector 240. The light guide 261 connects the observation illuminating light emitting section 260 and the connector 240. The signal line 271 connects the imaging section 270 and the connector 240. The optical fiber 251, the light guide 261, and the signal line 271 are connected to the optical fiber, the light guide, and the signal line inside the control device 100 through the connector 240.

[0085] ​The control device 100 includes a processing section 110, a storage section 120, a pattern light source 150, and an observation light source 160.

[0086] The observation light source 160 is a light source that generates observation illumination light. The observation light source 160 includes a white light source and an optical system that causes light emitted by the white light source to be incident on a light guide. The white light source is, for example, a xenon lamp or a white LED.

[0087] The pattern light source 150 is a light source that emits laser light of wavelengths λ1 to λ3. The pattern light source 150 includes first to third laser diodes that generate laser light of wavelengths λ1 to λ3, and an optical system that causes laser light emitted by the first to third laser diodes to be incident on an optical fiber.

[0088] The storage section 120 is a memory or a storage device such as a hard disk drive. The memory is a semiconductor memory, and is a volatile memory such as a RAM or a nonvolatile memory such as an EEPROM. The storage section 120 stores programs and data and the like that are necessary for the operation of the processing section 110. In addition, the storage section 120 stores the LUT described in the above expression (5) as a table 121. In the case where the phase is calculated by a function operation as in the above expression (6), the table 121 can also be omitted.

[0089] The processing section 110 includes a light source controller 111, an image processing section 112, a distance measuring processing section 113, and an image output section 114. These sections can be realized by separate hardware circuits. Alternatively, the functions of the sections can be realized by a processor executing a program that describes the operation of each section.

[0090] The light source controller 111 controls the pattern light source 150 and the observation light source 160. That is, the light source controller 111 controls the emission timing, the emission period, and the light quantity of the pattern light source 150 and the observation light source 160.

[0091] The image processing section 112 performs image processing on an image signal input from the imaging section 270 via a signal line 271. The image processing section 112 performs processing for generating an RGB color image from a RAW image. In addition, the image processing section 112 can also perform, for example, white balance processing, gradation processing, or emphasis processing. The image output by the image processing section 112 in a frame in which observation illumination light is emitted is an observation image, and the image output by the image processing section 112 in a frame in which a pattern is projected is a pattern image.

[0092] The distance measurement processing section 113 performs the distance measurement processing explained in the above equations (1) to (6), thereby calculating the distance to each position of the subject from the pattern image. Further, the distance measurement processing section 113 calculates the shape from the distance to each position of the subject. As explained above, the shape is the short diameter, long diameter, width, length, height, or depth, or the like, of the region of interest. Hereinafter, the information of the distance or shape will be collectively referred to as distance measurement information. In addition, the distance measurement processing section 113 can calculate the distance of each position in the entire region of the pattern image, or can calculate the distance of each position in only a part of the region such as the region of interest. Further, the distance measurement processing section 113 can calculate the length or height, or the like, between points specified by the user as the shape information.

[0093] The distance measurement processing section 113 can also calculate the inclination of the region of interest from the distance to the periphery of the region of interest which is the calculation target of the shape, and perform inclination correction of the shape of the region of interest from the inclination. For example, the processing section 110 detects the region of interest by AI processing or the like described later. The distance measurement processing section 113 calculates the distance of three or more points of the periphery of the region of interest, and calculates the inclination of the subject surface of the periphery of the region of interest from the distance. The inclination is the angle formed by the camera line of sight and the surface of the imaging section 270. The distance measurement processing section 113 performs projection transformation so that the subject surface faces the imaging section 270, thereby calculating the shape of the region of interest in the subject facing the imaging section 270. The shape here is so-called dimensions such as length, width, long diameter, or short diameter, or the like.

[0094] The image output section 114 outputs a display image to the display section 300 from the observation image and the distance measurement information. The display section 300 is a display such as a liquid crystal display device or an EL display device. The image output section 114 displays the observation image 301 and the distance measurement information 302, for example, in parallel in the display region of the display section 300. Alternatively, the image output section 114 can display the distance measurement information superimposed on the observation image. For example, information indicating the shape of the region of interest can be superimposed in the region of interest.

[0095] Specifically, the image processing section 112 generates an observation image from the image captured by the imaging section 270 in the first frame. The image output section 114 displays the observation image 301 on the display section 300. The first frame corresponds to any of F1, F3, F5, and F7 of Figure 2 or Figure 3 . As a background process of the display of the observation image 301, the distance measurement processing section 113 calculates the distance to the subject or the shape of the subject from the image captured by the imaging section 270 in the first frame. The second frame corresponds to F4 of Figure 2 or Figure 3Any frame among F2, F4, and F6. The image output unit 114 appends ranging information 302 based on the distance to the subject or the shape of the subject to the observed image 301 and displays it on the display unit 300.

[0096] In addition, Figure 6 In this configuration, the pattern light source 150 is located in the control device 100, but the pattern light source 150 can also be located in the operation section 220 of the mirror body 200. Furthermore, in Figure 6 In this system, a pattern light source 150 and an observation light source 160 are separately provided, but they can also be combined into one observation light source 160. In this case, the optical fiber connecting the observation light source 160 and the observation illumination light emitting unit 260 is branched within the mirror body 200, and the branched optical fiber is connected to the pattern light projection unit 250. This eliminates the need to connect the pattern light projection unit 250 to the control device 100.

[0097] Figure 7 This is the second detailed structural example of the endoscope system 10. Figure 7 In the middle, the processing unit 110 also includes an AI processing unit 115. Additionally, for... Figure 1 and Figure 6 Structural elements already described in the previous section are labeled with the same number, and the description of the structural element is omitted appropriately.

[0098] The image processing unit 112 generates an observation image based on the image captured by the imaging unit 270 in the first frame. The first frame corresponds to... Figure 2 or Figure 3 Any frame from F1, F3, F5, and F7. The ranging processing unit 113 calculates the distance to the subject or the shape of the subject based on the image captured by the camera unit in the second frame. The second frame corresponds to Figure 2 F4 or Figure 3 Any frame from F2, F4, and F6. The AI ​​processing unit 115 performs AI processing based on the observed image and the distance to the subject or the shape of the subject to detect the presence of a region of interest or to determine its state. Detecting the presence of a region of interest means detecting whether the region of interest exists within the image. Determining the state of a region of interest means determining the classification category representing the state of the region of interest. Classification categories may include, for example, indicators representing the type of lesion such as cancer or polyps, or the stage of cancer progression.

[0099] by Figure 2Further detailed description will be given by way of example. The observation image taken in frame Fl is input to the AI processing section 115, and the AI processing section 115 detects the attention site by AI processing based on the observation image. The AI processing section 115 outputs a trigger signal to the distance measurement processing section 113 in a case where the attention site is detected. The distance measurement processing section 113 calculates the distance to the subject or the shape of the subject based on the image taken in frame F4 after the trigger signal is input, and inputs the distance to the subject or the shape of the subject to the AI processing section 115. Further, the observation image taken in frame F3 or F5 or the like is input to the AI processing section 115. The AI processing section 115 performs detection of the presence of the attention site or determination related to the state based on the observation image and the distance to the subject or the shape of the subject. The result of the second determination can also be used to generate a trigger signal again. Alternatively, the result of the second determination can be output to the image output section 130, and the image output section 130 can cause the display section 300 to display the observation image with the result of the determination superimposed thereon.

[0100] Figure 8 is a first detailed configuration example of the pattern light projecting section 250. The pattern light projecting section 250 includes an incidence section 256, a DOE 253, and a slit section 252. The incidence section 256 causes parallel light including components of wavelengths λ1 to λ3 to be incident on the DOE 253. The emergent light of the DOE 253 is incident on the slit section 252, and the slit section 252 projects pattern light PT1 to PT3 of wavelengths λ1 to λ3 to the subject 5.

[0101] According to the present embodiment, the pattern light source 150 as the laser light source is provided to the control device 100, and only a simple optical system composed of the DOE 253 or the like is provided to the front end of the scope 200. Thereby, the scope 200 can be made thin in diameter, and high-brightness pattern light PT1 to PT3 can be projected by the laser light source. In order to measure the distance to a subject such as a living body with motion with high accuracy, it is necessary to make the emission time of the pattern light PT1 to PT3 as short as possible, but by using the laser light source, the emission time can be shortened. The element of the laser light source is larger than that of a light-emitting diode or the like, but by being configured to use the DOE 253 or the like, the laser light source can be provided to the control device 100, and the scope 200 can be made thin in diameter. Further, it is not necessary to provide a heat source such as a light-emitting diode to the front end of the scope 200, and thus, unnecessary heat generation at the front end of the scope 200 can be prevented.

[0102] Next, detailed description will be given of the configuration of the pattern light projecting section 250. Figure 8 The incidence section 256 includes an optical fiber 251 that guides laser light and a collimator lens 254 that makes the emergent light of the optical fiber 251 into parallel light. Laser light of wavelengths λ1 to λ3 is guided by the optical fiber 251, and diffused from the emission end of the optical fiber 251. The collimator lens 254 makes the diffused laser light into parallel light.

[0103] The DOE 253 causes the components of wavelengths λ1 to λ3 included in the parallel light to converge into the first to third linear lights LL1 to LL3 that are different in position from each other. The slit section 252 has a plurality of slits that are parallel to each other. Further, the linear lights LL1 to LL3 pass through the plurality of slits, and thereby the pattern lights PT1 to PT3 are projected to the subject.

[0104] The linear lights LL1 to LL3 function as virtual light sources that emit light to the slit section 252, and correspond to Figure 1 the light sources S1 to S3 of formula (6). Each linear light is parallel to the slits of the slit section 252. The linear lights LL1 to LL3 are arranged at different positions in a direction that is parallel to the plane of the slit section 252 and orthogonal to the slits.

[0105] The DOE 253 is an optical element that controls the outgoing light into a specific shape using the diffraction phenomenon. The specific shape is determined by the fine structure of the DOE 253, and by designing the fine structure, light of a desired shape can be obtained. In the present embodiment, the DOE 253 causes the m-order diffracted light of the incident parallel light to converge into a linear light at a prescribed focal distance. The convergence position of the m-order diffracted light differs depending on the wavelength. The incident light has components of wavelengths λ1 to λ3, and therefore the m-order diffracted light of each wavelength converges into the linear lights LL1 to LL3 that are different in position from each other. In addition, m is an integer of 1 or more. Here, it is simply described as m-order, but the m-order can be either one of +m-order and -m-order.

[0106] The DOE 253 selectively causes the m-order diffracted light among the 0-order, 1-order, 2-order,... diffracted light to converge. That is, the intensity of the m-order diffracted light emitted by the DOE 253 is higher than the intensity of the diffracted light other than the m-order. More specifically, the DOE 253 emits only the approximately m-order diffracted light among the 0-order, 1-order, 2-order,... diffracted light.

[0107] The wavelengths λ1 to λ3 of the laser light are, for example, equidistant. In this case, the DOE 253 causes the linear lights LL1 to LL3 to converge at equidistance. The "distance" here is the distance in a direction that is parallel to the plane of the slit section 252 and orthogonal to the slits. The linear lights LL1 to LL3 are at equidistance, and thereby the phases of the pattern lights PT1 to PT3 become equidistant. Thereby, it is possible to generate a pattern light suitable for the structured light method. Further, in the case of using the function operation explained in the above formula (6), it is necessary for the phases of the pattern lights PT1 to PT3 to become equidistant. In addition, the wavelengths λ1 to λ3 of the laser light can also be non-equidistant, and the linear lights LL1 to LL3 that the DOE 253 converges can also be non-equidistant. In this case, as explained in the above formula (5), by using the LUT, it is possible to convert the pattern image into the distance.

[0108] Figure 9is a second detailed configuration example of the pattern light projecting section 250. In Figure 9 The pattern light projecting section 250 further includes a mask section 255 in Figure 8 The same reference numerals are assigned to the structural elements explained in

[0109] The mask section 255 is provided between the DOE 253 and the slit section 252. The mask section 255 passes the linear lights LL1 to LL3 based on the m-order diffracted light and blocks the diffracted light other than the m-order. In Figure 9 In the example shown in FIG. 25, the mask section 255 passes the linear lights LL1 to LL3 based on the 1-order diffracted light and blocks the diffracted light other than the 1-order such as the 0-order and the 2-order. The mask section 255 is a plate-like member parallel to the slit section 252, and an opening is provided in the plate-like member. The mask section 255 is provided so that the linear lights LL1 to LL3 pass through the opening.

[0110] The DOE 253 selectively converges the m-order diffracted light, but the diffracted light other than the m-order is also included in the emergent light of the DOE 253. When the diffracted light other than the m-order passes through the slit section 252, unwanted pattern light other than the original pattern lights PT1 to PT3 can be mixed, and the distance measuring accuracy can be reduced. According to the present embodiment, by providing the mask section 255, the diffracted light other than the m-order is blocked, and thus only the original pattern lights PT1 to PT3 are projected, and high-accuracy distance measurement can be performed.

[0111] In Figure 6 The processing section 110 can also generate the diagnosis assistance information by AI processing. In this case, the storage section 120 stores a learned model that has been learned for generating the diagnosis assistance information, and the processing section 110 generates the diagnosis assistance information by performing AI processing using the learned model. The content of the AI processing will be described below.

[0112] When a focus site is specified in the observation image, the processing section 110 acquires distance information of the subject, and determines the length or height of the focus site or the like based on the distance information. The focus site is specified, for example, by user operation. Alternatively, the processing section 110 can also detect the focus site by performing AI image recognition on the observation image, thereby specifying the focus site. The processing section 110 generates diagnosis assistance information by inputting the acquired length or height of the focus site or the like and the observation image to AI processing. The diagnosis assistance information is, for example, estimation information such as whether or not it is a lesion, the kind of the lesion, the malignancy degree of the lesion, or the shape of the lesion. The image output section 114 of the processing section 110 displays the diagnosis assistance information together with the observation image on the display section 300.

[0113] The processing section 110 can also generate the above-described diagnosis assistance information through post-processing. That is, the storage section 120 can also record the observation image and the pattern image in advance, and the processing section 110 performs the above-described AI processing using the observation image and the pattern image recorded in the storage section 120.

[0114] The above describes the present embodiment and its modification, but the present application is not limited to the present embodiment and its modification, and the structural elements can be modified and embodied in the implementation stage within the scope of the gist. Further, the plurality of structural elements disclosed in the above-described present embodiment and its modification can be appropriately combined. For example, several structural elements can be deleted from all the structural elements described in the present embodiment and its modification. Further, the structural elements described in different present embodiments and its modification can be appropriately combined. In this way, various modifications and applications can be made within the scope of the gist of the present application. Further, in the specification or the drawings, a term recorded at least once together with a different term that is more general or synonymous can be replaced with the different term at any place in the specification or the drawings.

[0115] Label Explanation

[0116] 5: object; 10: endoscope system; 100: control device; 110: processing section; 111: light source controller; 112: image processing section; 113: distance measuring processing section; 114: image output section; 115: AI processing section; 120: storage section; 121: table; 150: pattern light source; 151: optical fiber; 160: observation light source; 200: scope; 210: flexible section; 220: operation section; 230: general-purpose flexible cord; 240: connector; 250: pattern light projection section; 251: optical fiber; 252: slit section; 253: DOE; 254: collimator lens; 255: mask section; 256: entrance section; 260: observation illumination light exit section; 261: light guide; 270: imaging section; 271: signal line; 300: display section; 301: observation image; 302: distance measuring information; LL1 to LL3: linear light; PT1 to PT3: pattern light; λ1 to λ3: wavelength.

Claims

1. An endoscope system, characterized in that, The endoscope system includes: A light source that emits light of wavelengths from the 1st to the nth. A lens that makes the light of wavelengths 1 to n parallel; A diffractive optical element that converges the components of light of wavelengths 1 to n contained in the parallel light into linear light of wavelengths 1 to n with different positions. A slit that projects patterned light from the first to the nth linear light onto the subject; An imager that captures an image of the subject onto which the first to nth patterned lights are projected, as an image of one frame; as well as A processor configured to calculate the distance to the subject or the shape of the subject based on the image captured by the imager; The endoscope system includes an observation light source that emits illumination light for observation. In the first frame, the observation light source emits the observation illumination light, and the imager captures an image of the subject illuminated by the observation illumination light. In a second frame, which is different from the first frame, the light source, the lens, the diffractive optical element, and the slit project the first to nth patterned light onto the subject, and the imager captures an image of the subject on which the first to nth patterned light has been projected.

2. The endoscope system according to claim 1, characterized in that, In the frame of capturing the image of the first frame, the light source, the lens, the diffractive optical element, and the slit simultaneously project the first to nth patterned lights onto the subject.

3. The endoscope system according to claim 1, characterized in that, The processor generates the observation image based on the image captured by the imager in the first frame. The processor calculates the distance or the shape based on the image captured by the imager in the second frame. The processor performs AI processing based on the observed image and the distance or shape to detect the presence of the area of ​​interest or to make a determination related to the state.

4. The endoscope system according to claim 1, characterized in that, The processor generates an observation image based on the image captured by the imager in the first frame, and displays the observation image on the display. The processor calculates the distance or shape based on the image captured by the imager in the second frame as background processing for the display of the observed image. The processor appends information based on the distance or shape to the observed image and displays it on the display.

5. The endoscope system according to claim 1, characterized in that, The wavelengths of the first to nth patterned lights are in the range of 460nm to 700nm.

6. The endoscope system according to claim 5, characterized in that, The wavelengths of the first to nth patterned lights are in the range of 460nm to 520nm.

7. The endoscope system according to claim 1, characterized in that, The imager includes an image sensor having pixels of colors 1 to n that receive light of colors 1 to n. The wavelengths of the first to nth pattern lights are set to the first to nth wavelengths, and the photosensitivity of the i-th color pixel in the j-th wavelength is set to a. ij When, i is an integer greater than or equal to n, and j is an integer greater than or equal to n. The processor determines the photosensitivity a. ij The image of the subject when it is projected with each pattern light of the first to nth pattern light is extracted using the intensity values ​​of the first to nth colors in the image of the 1st frame, and the distance to the subject or the shape of the subject is calculated based on the phase of the image of the subject when it is projected with each pattern light.

8. The endoscope system according to claim 7, characterized in that, n＝3， The first to nth colors are R, G, and B. With the a mentioned ij The row vectors of a matrix A with elements are linearly independent. The processor assigns the intensity value p of the i-th color at each position in the image of the first frame. i Perform q j =(A -1 ) ji p i The calculation determines the intensity value q at each position of the image of the subject projected with the j-th pattern light. j And based on the intensity value q j The phase is used to calculate the distance to the subject or the shape of the subject.

9. The endoscope system according to claim 1, characterized in that, The processor calculates the tilt of the region of interest based on the distance to the periphery of the region of interest, and performs tilt correction on the shape of the region of interest based on the tilt, wherein the region of interest is the object of the shape calculation.

10. The endoscope system according to claim 1, characterized in that, The diffractive optical element emits m-th order diffracted light containing the first to nth wavelength components of the parallel light, where m is an integer greater than or equal to 1. The intensity of the m-order diffracted light is higher than that of the diffracted light beyond the m-order.

11. The endoscope system according to claim 1, characterized in that, The endoscope system includes a shield disposed between the diffractive optical element and the slit. The diffractive optical element emits m-order diffracted light containing the first to nth wavelength components of the parallel light. The intensity of the m-order diffracted light is higher than that of the diffracted light beyond the m-order. The shielding portion allows the first to nth linear light based on the m-th order diffraction light to pass through, while blocking diffraction light other than the m-th order.

12. The endoscope system according to claim 1, characterized in that, The first to nth wavelengths are equally spaced.

13. The endoscope system according to claim 1, characterized in that, The endoscope system includes optical fibers. The light source emits laser light of wavelengths 1 to n. The optical fiber guides the laser. The lens is a collimating lens that makes the outgoing light from the optical fiber become parallel light.

14. An endoscope, characterized in that, The endoscope comprises: A lens that makes light of wavelengths 1 to n parallel; A diffractive optical element that converges the components of light of wavelengths 1 to n contained in the parallel light into linear light of wavelengths 1 to n with different positions. A slit that projects patterned light from the first to the nth linear light onto the subject; as well as An imager that captures an image of the subject onto which the first to nth patterned lights are projected, as an image of one frame; In the first frame, the imager captures an image of the subject illuminated by observation lighting. In a second frame, which is different from the first frame, the lens, the diffractive optical element, and the slit project the first to nth patterned light onto the subject, and the imager captures an image of the subject on which the first to nth patterned light has been projected.

15. A distance calculation method, characterized in that, The distance calculation method includes the following steps: The light source emits light of wavelengths from the 1st to the nth. The lens makes the light of wavelengths 1 to n parallel; The diffractive optical element converges the components of the parallel light of wavelengths 1 to n into linear light of wavelengths 1 to n with different positions. The slit projects the patterned light from the first to the nth linear light onto the subject; The imager captures an image of the subject projected with patterned light from the first to the nth time as one frame; and The processor calculates the distance to the subject or the shape of the subject based on the image captured by the imager; The light source emits illumination for observation. In the first frame, the observation light source emits the observation illumination light, and the imager captures an image of the subject illuminated by the observation illumination light. In a second frame, which is different from the first frame, the light source, the lens, the diffractive optical element, and the slit project the first to nth patterned light onto the subject, and the imager captures an image of the subject on which the first to nth patterned light has been projected.

Images