Image processing device, image processing method, and image processing program

Abstract

To provide an image processing device which can prevent a false detection of an unobserved region.SOLUTION: An image processing device is applied in an endoscope system, and processes an image of a subject imaged by the endoscope device. A processor of the image processing device creates S2 a three-dimensional model of the subject from an image group imaged by the endoscope device; detects S3 an unobserved region which has not been imaged by the endoscope device based on the three-dimensional model; obtains an operation signal of the endoscope system; and temporarily stops a detection of the unobserved region in a case where the operation signal matches a predefined condition. The predefined condition includes an operation of the endoscope system that causes a lack of a part of the image group.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, an image processing method, and an image processing program.

Background Art

[0002] Conventionally, a technique for creating a three-dimensional model of a subject from a group of images acquired by an endoscope has been known (see, for example, Patent Document 1). According to Patent Document 1, a blank area where a three-dimensional model is not created is determined as an unobserved area that has not been observed by the endoscope, and the unobserved area is displayed so as to be visible.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Due to the operation of the endoscope system, a part of the group of images may be missing. A three-dimensional model created from a group of images with a missing part lacks accuracy, and as a result, problems may occur in the determination of the unobserved area.

Means for Solving the Problems

[0005] One aspect of the present invention is an image processing apparatus applied to an endoscope system for processing an image of a subject photographed by an endoscope, including a processor that creates a three-dimensional model of the subject from a group of images photographed by the endoscope, detects an unobserved area that has not been photographed by the endoscope based on the three-dimensional model, acquires an operation signal of the endoscope system, and temporarily stops the detection of the unobserved area when the operation signal matches a predetermined condition, where the predetermined condition includes an operation of the endoscope system that causes a part of the group of images to be missing.

[0006] One aspect of the present invention is an image processing method for processing an image of a subject photographed by an endoscope, including creating a three-dimensional model of the subject from a group of images photographed by the endoscope, detecting an unobserved region that has not been photographed by the endoscope based on the three-dimensional model, and obtaining an operation signal of an endoscope system including the endoscope. When the operation signal matches a predetermined condition, temporarily stopping the detection of the unobserved region, and it is an image processing method.

[0007] One aspect of the present invention is an image processing program for causing a computer to execute an image of a subject photographed by an endoscope, including creating a three-dimensional model of the subject from a group of images photographed by the endoscope, detecting an unobserved region that has not been photographed by the endoscope based on the three-dimensional model, and obtaining an operation signal of an endoscope system including the endoscope. When the operation signal matches a predetermined condition, causing the computer to temporarily stop the detection of the unobserved region, and the predetermined condition includes an operation of the endoscope system that causes a missing part of the group of images, and it is an image processing program.

Brief Description of the Drawings

[0008]

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8A

Fig. 8B

Fig. 9

Fig. 10

Fig. 11

Mode for Carrying Out the Invention

[0009] (First Embodiment) An image processing apparatus, an image processing method, an image processing program, and a storage medium according to a first embodiment of the present invention will be described with reference to the drawings. As shown in FIG. 1, an image processing apparatus 10 according to the present embodiment is applied to an endoscope system 100. The endoscope system 100 includes an image processing apparatus 10, an endoscope 20, a light source device 30, a control device 40, and a display device 50. Further, the endoscope system 100 includes peripheral devices 60 for an endoscope 20 used for endoscopy. The peripheral devices 60 include, for example, an air supply pump 61, an air intake pump 62, a water supply pump 63, a water suction pump 64, a high-frequency treatment tool 65, and a UPD (endoscope insertion shape observation device) 66.

[0010] The endoscope 20 is, for example, a flexible endoscope for a digestive organ such as the large intestine. The endoscope 20 has a long flexible insertion portion 20a, a bending portion 20b provided at the distal end of the insertion portion 20a, and an operation portion (not shown) connected to the proximal end of the insertion portion 20a (see FIG. 2).

[0011] In addition, the endoscope 20 includes an imaging optical system 21 for imaging a subject, a zoom mechanism 22 for magnifying and reducing the subject in the image, and an angle sensor 23 for detecting the bending angle of the bending portion 20b. The imaging optical system 21 includes an objective lens and an imaging element such as a CMOS image sensor. The zoom mechanism 22 changes the magnification of the subject in the image by optical zoom or digital zoom. For example, when a zoom switch provided on the operation unit is turned on by the user, the zoom mechanism 22 switches from the normal magnification to the high magnification.

[0012] The light source device 30 is connected to the endoscope 20 and supplies illumination light to the endoscope 20. The light source device 30 has a plurality of color LEDs and can output a plurality of types of illumination light by turning on and off each LED. For example, the light source device 30 has five LEDs of purple, blue, green, amber, and red. The plurality of types of illumination light include white light for normal observation and special light for special light observation. The white light is composed of five colors of light. The special light is composed of, for example, blue and green light for NBI (narrow band imaging).

[0013] The control device 40 includes a processor 41, an input / output unit 42, and a user interface 43. The input / output unit 42 has a known input / output interface, and the control device 40 is connected to the image processing device 10, the endoscope 20, the light source device 30, and the peripheral device 60 through the input / output unit 42. The image taken by the endoscope 20 is input to the display device 50 via the control device 40 and the image processing device 10 and is displayed on the display device 50. The display device 50 is an arbitrary type of display such as a liquid crystal display.

[0014] The control device 40 controls the operations of the light source device 30 and the peripheral device 60. For example, the processor 41 generates a control signal for controlling the light source device 30 and the peripheral device 60 based on the operation of the user interface 43 by the user, and transmits the control signal to the light source device 30 and the peripheral device 60.

[0015] The image processing apparatus 10 includes a processor 1 such as a central processing unit, a storage unit 2, a memory 3, and a user interface 4. For example, the image processing apparatus 10 is composed of any computer such as a personal computer. The storage unit 2 is a computer-readable non-transitory storage medium, such as a known magnetic disk, optical disk, or flash memory. The storage unit 2 stores an image processing program 2a that causes the processor 1 to execute an image processing method described later.

[0016] The memory 3 is composed of a volatile storage device such as a RAM (random access memory) and is used as a working area for the processor 1. The user interface 4 has input devices such as a mouse, a keyboard, and a touch panel, and receives operations of the input devices by the user.

[0017] The processor 1 includes, as functional units, a 3D (three-dimensional) model creation unit 11, a determination unit 12, a data removal unit 13, an unobserved area detection unit 14, and a display control unit 15. Further, the image processing apparatus 10 includes an image storage unit 5 that stores an image group used for creating a 3D model, and a condition storage unit 6 that stores predetermined conditions for temporarily stopping the detection of an unobserved area. The storage units 5 and 6 are composed of, for example, the storage unit 2, the memory 3, or other storage devices.

[0018] The processor 1 stores an image input to the image processing apparatus 10 from the endoscope 20 via the control device 40 in the image storage unit 5, and generates an image group composed of a plurality of images for creating a 3D model. The processor 1 may store all the images input to the image processing apparatus 10, or may select and store images suitable for creating a 3D model. The 3D model creation unit 11 acquires an image group from the image storage unit 5, and creates a 3D model representing the 3D shape of a subject in the image group. For creating the 3D model, a known 3D reconstruction technique such as Visual SLAM (Simultaneous Localization and Mapping) is used.

[0019] Figures 2 and 3 respectively illustrate a colonoscopy and the creation of a 3D model of the colon. In a colonoscopy, after inserting the endoscope 20 from the anus to the cecum, while removing the endoscope 20 towards the anus, the cecum, ascending colon, transverse colon, descending colon, sigmoid colon, and rectum are observed in order. If necessary, for observing the mucosa of the colon, washing of the mucosa by water injection and air supply and aspiration are performed. When a lesion is found, magnified observation or special light observation of the lesion is performed, and treatment using the high-frequency treatment device 65 is performed if necessary.

[0020] As shown in Figure 3, as the endoscope 20 moves, the 3D model C is created in order from the cecum. The 3D model C may include a missing part D where the model of the subject is not created. The missing part D corresponds to the unobserved region B of the subject that was not observed by the endoscope 20. For example, in a colonoscopy, the region on the back side of the fold A that is likely to be a blind spot can be the unobserved region B. In Figures 2 and 3, at time t2, an unobserved region B occurs on the back side of the fold A, and a missing part D corresponding to the unobserved region B is formed in the 3D model C.

[0021] In order to create a continuous 3D model C over the moving range of the visual field of the endoscope 20, it is necessary for the subject to be continuous in a plurality of images used for creating the 3D model C. However, due to the operation of the endoscope system 100, an image without information on the form of the subject may be input to the image processing device 10, or the image may not be input to the image processing device 10 temporarily. In such a case, a part of the image group is missing. For example, some of the images in the image group do not have information on the form of the subject, or the image group does not include images of a partial region of the subject. Therefore, the image group with a missing part includes a discontinuous part of the subject without information on the form of the subject.

[0022] For example, when switching from white light to special light for NBI, the intensity of the illumination light temporarily decreases, and a dark image without information on the form of the subject is temporarily acquired by the endoscope 20. In the image group including the dark image, a part of the subject corresponding to the dark image becomes a discontinuous part. The 3D model creation unit 11 continues to create the 3D model C regardless of the presence or absence of a missing part in a part of the image group. The 3D model C created from the image group with a missing part includes a missing part corresponding to the missing part of the image group.

[0023] The determination unit 12 acquires the operation signal of the endoscope system 100 and compares the operation signal with a predetermined condition stored in the condition storage unit 6. The determination unit 12 determines whether the operation signal matches the predetermined condition. The operation signal is a signal indicating the operations of the devices 20, 30, 61 to 66 constituting the endoscope system 100, and particularly includes a signal indicating a predetermined operation of the devices 20, 30, 61 to 66 that causes a missing part in a part of the image group. The determination unit 12 acquires the operation signal from the control device 40 or from the devices 20, 30, 61 to 66.

[0024] An example of the operation signal is a signal indicating the switching of the type of illumination light supplied from the light source device 30 to the endoscope 20, for example, a control signal output by the control device 40 to the light source device 30 to execute the switching of the type of illumination light. The determination unit 12 acquires the control signal from the control device 40. The predetermined condition is a predetermined operation of the endoscope system 100 that causes a missing part in a part of the image group. For example, the predetermined condition includes the switching of the type of illumination light supplied from the light source device 30 to the endoscope 20.

[0025] When the operation signal matches the predetermined condition, the data removal unit 13 detects a missing part in the 3D model C, which is a part corresponding to the predetermined condition (that is, a part corresponding to the missing part of the image group), and removes the data of the missing part from the 3D model C. Thereby, a 3D model C that does not include a missing part caused by the operation of the endoscope system 100 is created.

[0026] The unobserved area detection unit 14 detects an unobserved area in the 3D model C based on the determination result by the determination unit 12. Specifically, when it is determined that the operation signal does not match the predetermined condition, the unobserved area detection unit 14 detects the missing part D in the 3D model C created by the 3D model creation unit 11 as the unobserved area.

[0027] On the other hand, when it is determined that the operation signal matches the predetermined condition, the unobserved area detection unit 14 temporarily stops detecting the unobserved area. After the missing part caused by the operation of the endoscope system 100 is removed by the data removal unit 13, the unobserved area detection unit 14 resumes detecting the unobserved area and detects the missing part D in the 3D model C from which the missing part caused by the operation of the endoscope system 100 has been removed as the unobserved area.

[0028] As shown in FIG. 4, the display control unit 15 generates displays E1, E2, E3 indicating that an unobserved area has been detected, and outputs the displays E1, E2, E3 to the display device 50 together with the current image F for display on the display device 50. The display control unit 15 may output the displays E1, E2, E3 together with the image F at a position a certain distance away from the unobserved area D, or the image F after a certain time from the detection of the unobserved area D.

[0029] In the example of FIG. 4, the unobserved area D is detected at time t2, and then the displays E1, E2, E3 are displayed on the display device 50 at time t3. An example of the display is the image E1 when the unobserved area D is detected, and a marker may be attached to the unobserved area B in the image E1. Another example of the display is the arrow E2 indicating the position of the unobserved area B in the current image F. Another example of the display is the frame E3 of a predetermined color attached to the image F. By displaying at least one of the displays E1, E2, E3 together with the image F, the user is notified of the existence of the unobserved area B.

[0030] Next, an image processing method executed by the image processing apparatus 10 will be described. As shown in FIG. 5, the image processing method according to this embodiment includes a step S1 of acquiring an image captured by the endoscope 20, a step S2 of creating a 3D model C, a step S3 of detecting an unobserved region D in the 3D model C, and a step S4 of generating and outputting a display indicating the unobserved region D. Further, the image processing method includes a step S5 of acquiring an operation signal of the endoscope system 100, a step S6 of determining whether the operation signal matches a predetermined condition, a step S7 of temporarily stopping the detection of the unobserved region, and a step S8 of removing data of a missing part from the 3D model C.

[0031] The processor 1 starts acquiring an image input from the endoscope 20 to the image processing apparatus 10 and sequentially stores the image in the image storage unit 5 (step S1). Subsequently, the 3D model creation unit 11 starts creating a 3D model C of the subject from the image group (step S2). The unobserved region detection unit 14 executes detection of an unobserved region in the created 3D model C (step S3). When the unobserved region D is detected, the display control unit 15 generates displays E1, E2, E3 indicating the detected unobserved region D and outputs the displays E1, E2, E3 to the display device 50 together with the image F (step S4).

[0032] Here, after starting the creation of the 3D model C, the determination unit 12 starts acquiring an operation signal of the endoscope system 100 (step S5). When an operation signal is acquired along with the operation of the endoscope system 100 (YES in step S5), the determination unit 12 subsequently determines whether the operation signal matches a predetermined condition (step S6).

[0033] When the operation signal does not match the predetermined condition (NO in step S6), step S3 is then executed. On the other hand, when the operation signal matches the predetermined condition (YES in step S6), the unobserved region detection unit 14 temporarily stops the detection of the unobserved region (step S7), and the data removal unit 13 removes data of a missing part corresponding to the predetermined condition from the 3D model C (step S8). After removing the data of the missing part, the unobserved region detection unit 14 resumes the detection of the unobserved region (step S3).

[0034] As described above, according to the image processing apparatus 10 according to the present embodiment, by determining whether or not the operation signal of the endoscope system 100 matches a predetermined condition, a predetermined operation of the endoscope system 100 in which a part of the image group is missing is detected. After the detection of the predetermined operation, the detection of the unobserved region is temporarily stopped, the data of the missing portion caused by the predetermined operation is removed from the 3D model, and then the detection of the unobserved region is resumed. Thereby, even when a part of the image group is missing, it is possible to prevent the missing portion due to the missing of the image group from being erroneously detected as an unobserved region, and thereby prevent a problem from occurring in the detection of the unobserved region.

[0035] In the present embodiment, the 3D model creation unit 11 continues to create the 3D model regardless of the operation of the endoscope system 100. Instead, the creation of the 3D model may be temporarily stopped according to the operation of the endoscope system 100. Specifically, as shown in FIG. 6, when the operation signal matches a predetermined condition (YES in step S6), the 3D model creation unit 11 temporarily stops creating the 3D model (step S9), and the unobserved region detection unit 14 temporarily stops detecting the unobserved region (step S7). The data removal unit 13 removes an image corresponding to the predetermined condition (for example, a dark image due to switching of illumination light) from the image group (step S10), and then the 3D model creation unit 11 resumes creating the 3D model (step S11). Therefore, the unobserved region detection unit 14 detects the missing portion D in the 3D model C that does not include the missing portion caused by the operation of the endoscope system 100 as the unobserved region.

[0036] When continuously creating the 3D model, the data amount of the 3D model becomes enormous, and it is necessary to store a large amount of data. Also, it takes time for data processing for creating the 3D model. By temporarily stopping the creation of the 3D model when a predetermined operation of the endoscope system 100 is detected, these inconveniences can be eliminated and the processing speed of the processor 1 can be improved.

[0037] In the present embodiment, in addition to when a part of the image group is missing, the processor 1 may temporarily stop detecting the unobserved area when it is not necessary to detect the unobserved area. In this case, the operation signal acquired by the determination unit 12 in step S5 includes a signal indicating a predetermined operation of the endoscope system 100 in which detection of the unobserved area is unnecessary. Further, the predetermined condition that the determination unit 12 compares with the operation signal in step S6 includes a predetermined operation of the endoscope system 100 in which detection of the unobserved area is unnecessary.

[0038] The operation signal indicating the operation of the endoscope system 100 in which detection of the unobserved area is unnecessary includes at least one of the following signals. The first example of the operation signal is a signal indicating a change in the magnification of the zoom mechanism 22, for example, an on signal of the zoom switch. In this case, the predetermined condition includes a change in the zoom magnification of the endoscope 20, for example, a switch from the normal magnification to the high magnification. The second example of the operation signal is a signal indicating the output of special light from the light source device 30, for example, a control signal that the control device 40 outputs to the light source device 30 to switch from white light to special light. In this case, the predetermined condition includes a switch of the illumination light output from the light source device 30 to special light.

[0039] The third example of the operation signal is a signal indicating the operation of each of the air supply pump 61 and the air intake pump 62, for example, a control signal that the control device 40 outputs to the pumps 61, 62 to operate the pumps 61, 62. In this case, the predetermined condition includes the operation of the pumps 61, 62. The fourth example of the operation signal is a signal indicating the operation of each of the water supply pump 63 and the water intake pump 64, for example, a control signal that the control device 40 outputs to the pumps 63, 64 to operate the pumps 63, 64. In this case, the predetermined condition includes the operation of the pumps 63, 64.

[0040] The fifth example of the operation signal is a signal indicating the operation of the high-frequency treatment tool 65, for example, an on signal of the switch of the high-frequency treatment tool 65. In this case, the predetermined condition includes the operation of the high-frequency treatment tool 65. The sixth example of the operation signal is a signal output by the angle sensor 23 or UPD66 that detects the bending angle of the bending portion 20b. In this case, the predetermined condition includes that the bending portion 20b bends at a bending angle equal to or greater than a predetermined value.

[0041] When differentiating or treating a lesion, the user can perform magnified observation of the mucosa at a high magnification, perform special light observation of the lesion, treat the lesion using the high-frequency treatment instrument 65, or perform a reverse view for observing the rear by bending the bending portion 20b at a large angle. During differentiation or treatment, detection of unobserved areas is unnecessary for the user. During air supply / aspiration and water supply / aspiration, detection of unobserved areas is also unnecessary for the user.

[0042] The unobserved area detection unit 14 may resume detection of the unobserved area after a certain period of time has elapsed after temporarily stopping the detection of the unobserved area. For example, the unobserved area detection unit 14 may resume detection of the unobserved area after a certain period of time has elapsed after air supply or water supply is started by the operation of the pumps 61 and 63.

[0043] In the present embodiment, the processor 1 may set a predetermined condition for temporarily stopping the detection of the unobserved area based on the operation of the user interface 4 by the user. According to this configuration, the user can set the condition for temporarily stopping the detection of the undetected area.

[0044] For example, as shown in FIG. 7, the processor 1 causes the display device 50 to display a selection screen including a plurality of conditions. A user such as a doctor operates the user interface 4 to select a desired condition from among the plurality of conditions. In FIG. 7, a condition is selected when a checkbox is checked. The processor 1 sets the selected condition as a predetermined condition, and the determination unit 12 determines whether the operation signal matches the selected condition.

[0045] In this embodiment, the display control unit 15 may create a report regarding the stop of detection of the unobserved area after the end of the endoscopic examination. FIGS. 8A and 8B show examples of the report. The report in FIG. 8A includes a schematic diagram of the large intestine, and a marker is attached to the position where the detection of the unobserved area has been stopped. The report in FIG. 8B is a table showing the number of times the detection of the unobserved area has been stopped at each part of the large intestine.

[0046] (Second Embodiment) Next, an image processing apparatus, an image processing method, an image processing program, and a storage medium according to a second embodiment of the present invention will be described. This embodiment is different from the first embodiment in that predetermined conditions are set for each combination of devices constituting the endoscopic system. In this embodiment, the configuration different from that of the first embodiment will be described, and the same reference numerals will be given to the configurations common to the first embodiment, and the description thereof will be omitted.

[0047] As shown in FIG. 9, the image processing apparatus 101 according to this embodiment is applied to an endoscopic system 200 including an endoscope 20, a light source device 30, a control device 40, a display device 50, and peripheral devices 60. The image processing apparatus 101 includes a processor 102, a storage unit 2, a memory 3, a user interface 4, an image storage unit 5, and a condition storage unit 6.

[0048] FIG. 10 shows combinations of models of devices constituting the endoscopic system 200 and predetermined conditions corresponding to each combination. The condition storage unit 6 stores the correspondence between the combination of models and the predetermined conditions.

[0049] As shown in FIG. 10, endoscopes 20 of a plurality of models "Y1", "Y2", "Y3", and "Y4" can be used in the endoscopic system 200. The functions of the endoscope 20 differ depending on the model. Similarly, light source devices 30 of a plurality of models and control devices 40 of a plurality of models can be used in the endoscopic system 200.

[0050] In the example of FIG. 10, the control device 40 of the model "X1" supports the functions of NBI, RDI (Red Dichromatic Imaging), and EDOF (Extended Depth of Field). RDI is an observation method in which amber, green, and red lights are irradiated onto a subject as illumination lights in order to make it easier to see deep blood vessels and bleeding. EDOF is a technique for generating an image with a wide range of focus from two images focused on the near point and the far point, respectively.

[0051] The endoscopes 20 of the models "Y1" and "Y2" have the functions of NBI, RDI, and EDOF. The endoscopes 20 of the models "Y3" and "Y4" do not have the functions of NBI and RDI, and only normal observation using white light is possible. The light source device 30 of the model "Z1" has five-color LEDs of purple, blue, green, amber, and red.

[0052] For example, the predetermined condition corresponding to the combination of "X1", "Y1", and "Z1" is the on-operation of the switch for executing the functions of NBI, RDI, or EDOF. There is no predetermined condition corresponding to the combination of "X1", "Y3", and "Z1".

[0053] The processor 102 includes, as functional units, a 3D model creation unit 11, a determination unit 12, an unobserved region detection unit 14, and a display control unit 15. The determination unit 12 acquires system information from, for example, the control device 40. The system information is information on the models of the devices 20, 30, and 40 that constitute the endoscope system 200 and are directly or indirectly connected to the image processing device 101. The determination unit 12 acquires a predetermined condition corresponding to the combination of the models of the devices 20, 30, and 40 from the condition storage unit 6, compares the operation signal with the predetermined condition, and determines whether the operation signal matches the predetermined condition.

[0054] When it is determined that the operation signal does not match the predetermined conditions, the unobserved area detection unit 14 detects the missing part D in the 3D model created by the 3D model creation unit 11 as an unobserved area. When it is determined that the operation signal matches the predetermined conditions, the unobserved area detection unit 14 temporarily stops detecting the unobserved area.

[0055] Next, an image processing method executed by the image processing apparatus 101 will be described. As shown in FIG. 11, the image processing method according to the present embodiment includes a step S0 of acquiring system information and steps S1 to S7.

[0056] For example, when the endoscope 20 is connected to the control device 40, the control device 40 inputs the system information to the image processing apparatus 101. The processor 102 acquires the system information input to the image processing apparatus 101 and stores the system information in a storage device in the image processing apparatus 101, for example, the condition storage unit 6 (step S0). Next, similar to the first embodiment, steps S1 to S7 are executed.

[0057] After temporarily stopping the detection of the unobserved area in the 3D model (step S7), the unobserved area detection unit 14 may automatically resume the detection of the unobserved area. For example, the unobserved area detection unit 14 may resume the detection of the unobserved area based on a signal indicating the off operation of the NBI, RDI, or EDOF switch. After step S7, step S8 or steps S10, S11 described in the first embodiment may be executed.

[0058] As described above, according to the image processing apparatus 101 according to the present embodiment, by determining whether or not the operation signal of the endoscope system 200 matches a predetermined condition, a predetermined operation of the endoscope system 200 in which a part of the image group is missing is detected. After the detection of the predetermined operation, the detection of the unobserved region is temporarily stopped. Thereby, even when a part of the image group is missing, it is possible to prevent a missing portion due to the missing of the image group from being erroneously detected as an unobserved region, and thereby prevent a problem from occurring in the detection of the unobserved region.

[0059] In addition, since the functions of the respective devices 20, 30, and 40 differ depending on the model, the operations of the endoscope system 200 that cause a part of the image group to be missing are different for each combination of the devices 20, 30, and 40. According to the present embodiment, predetermined conditions suitable for the combination of the models of the devices 20, 30, and 40 can be automatically set.

[0060] In the present embodiment, the modification described in the first embodiment may be applied. That is, the predetermined condition may further include an operation of the endoscope system in which detection of a missing portion is unnecessary. Further, the predetermined condition may be set based on an operation of the user interface 4 by the user from among a plurality of conditions (see FIG. 7). The display control unit 15 may create a report after the end of the endoscope examination (see FIGS. 8A and 8B).

[0061] As described above, the embodiments and modifications of the present invention have been described in detail with reference to the drawings. However, the specific configuration of the present invention is not limited to the above embodiments and modifications, and various design changes are possible without departing from the gist of the present invention. Further, the components shown in the above embodiments and modifications can be combined as appropriate. For example, the subject observed by the endoscope may be an organ other than the large intestine.

Description of Reference Numerals

[0062] 1, 102 Processor 2 Memory unit (recording medium) 2a Image processing program 4 User interface 6 Condition storage unit (memory unit) 10, 101 Image processing apparatus 20 Endoscope 22 Zoom mechanism 23 Angle sensor (sensor) 30 Light source device 40 Control device 61 Air supply pump 62 Air intake pump 63 Water supply pump 64 Water suction pump 65 High-frequency treatment tool 66 UPD (sensor) C 3D model D Unobserved area

Claims

1. An image processing device that is applied to an endoscope system and processes an image of a subject captured by an endoscope, A processor is provided. The processor creating a three-dimensional model of the subject from a group of images captured by the endoscope; detecting an unobserved region not photographed by the endoscope based on the three-dimensional model; Acquiring an operational signal of the endoscope system; An image processing device that includes an operation of the endoscopic system that temporarily stops detection of the unobserved area when the operation signal matches a predetermined condition, the predetermined condition causing a portion of the image group to be missing.

2. the operation signal includes a signal indicating switching of a type of illumination light supplied from a light source device to the endoscope, The image processing device according to claim 1 , wherein the predetermined condition includes switching of the type of the illumination light.

3. The image processing device according to claim 1 , wherein the predetermined condition includes an operation of the endoscope system in which detection of the unobserved region is not required.

4. the operation signal includes a signal indicative of a change in magnification of a zoom mechanism of the endoscope; The image processing device according to claim 3 , wherein the predetermined condition includes a change in a zoom magnification of the endoscope.

5. the operation signal includes a signal indicating operation of an air supply pump or an air intake pump; The image processing apparatus according to claim 3 , wherein the predetermined condition includes an operation of the air supply pump or the air intake pump.

6. the operation signal includes a signal indicating operation of a water supply pump or a water intake pump, The image processing apparatus according to claim 3 , wherein the predetermined condition includes operation of the water supply pump or the water intake pump.

7. The operation signal includes a signal indicating the operation of a high-frequency treatment tool, The image processing device according to claim 3 , wherein the predetermined condition includes activation of the high-frequency treatment tool.

8. the operation signal includes a signal from a sensor that detects a bending angle of a bending portion of the endoscope, The image processing device according to claim 3 , wherein the predetermined condition includes bending the bending portion at a bending angle equal to or larger than a predetermined value.

9. A user interface for accepting an operation by a user is further provided, The processor, The image processing device according to claim 1 , further comprising: a step of selecting at least one of a plurality of conditions based on the operation received by the user interface, and setting the selected condition as the predetermined condition.

10. Further comprising a storage unit, the storage unit stores a correspondence relationship between a combination of models of a plurality of devices constituting the endoscope system and the predetermined condition, the plurality of devices including the endoscope and a light source device that supplies illumination light to the endoscope; The processor, Acquire information on the model of each of a plurality of devices to which the image processing device is connected; The image processing apparatus according to claim 1 , wherein the predetermined condition corresponding to the combination of models is acquired from the storage unit.

11. An image processing method for processing an image of a subject captured by an endoscope, comprising: creating a three-dimensional model of the subject from a group of images captured by the endoscope; detecting an unobserved region not photographed by the endoscope based on the three-dimensional model; acquiring an operational signal of an endoscope system including the endoscope; temporarily stopping detection of the unobserved area when the operation signal matches a predetermined condition.

12. An image processing program for causing a computer to execute an image of a subject captured by an endoscope, Creating a three-dimensional model of the subject from a group of images captured by an endoscope; detecting an unobserved region not photographed by the endoscope based on the three-dimensional model; acquiring an operational signal of an endoscope system including the endoscope; causing the computer to temporarily stop detecting the unobserved area when the operation signal matches a predetermined condition; An image processing program, wherein the predetermined condition includes an operation of the endoscope system that causes a portion of the image group to be missing.

Images