Three dimensional image display device, three dimensional image display method, and program

Abstract

To provide a three dimensional image display device, a three dimensional image display method, and a program capable of visualizing fine undulations of a subject.SOLUTION: The three dimensional image display device includes a control unit. The processing circuitry sets a reference plane based on one or more points included in three dimensional image data, and divides the three dimensional image data into a first region and a second region with the reference plane as a boundary. The processing circuitry superimposes, on a three dimensional shape of a subject, a graphic having a display state set in accordance with a distance between the reference plane and each of three or more points included in the three dimensional image data. The control unit sets a gradation of a display state of first graphics superimposed on the three dimensional shape corresponding to the first region. The control unit sets a gradation of a display state of second graphics superimposed on the three dimensional shape corresponding to the second region.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a three-dimensional image display device, a three-dimensional image display method, and a program. [Background technology]

[0002] Industrial endoscope devices are used for inspecting (endoscopic inspection) the internal structure of boilers, pipes, aircraft engines, heat exchangers, etc. for abnormalities and corrosion. The device disclosed in Patent Document 1 has a long, thin probe including an insertion tube that can be inserted into an object to be observed, and generates an image based on an optical image acquired via the probe. The device determines three-dimensional (3D) coordinates of points on the object to be observed by using the image of the object to be observed, and determines a reference plane by using the 3D coordinates of three or more points. The device calculates the distance between each point and the reference plane and displays a color map of each point, with each point colored according to the distance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6030837 Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional technology, the color of each point in a color map changes depending on the distance between the reference plane and each point. However, when the surface of an object to be observed has both large and small undulations, conventional technology can visualize the large undulations of the object, but it is difficult to visualize the small undulations of the object's surface. Therefore, when a user conducts observations focusing on small changes in shape, it is difficult to intuitively grasp such changes.

[0005] An object of the present invention is to provide a three-dimensional image display device, a three-dimensional image display method, and a program that can visualize fine undulations of a subject. [Means for solving the problem]

[0006] The present invention has a control unit, and the control unit acquires three-dimensional image data including three-dimensional coordinates of three or more points on a subject calculated based on an endoscopic image of the subject, displays a three-dimensional shape of the subject on a display based on the three-dimensional image data, sets a reference plane based on one or more points included in the three or more points, divides the three-dimensional image data into a first region and a second region using the reference plane as a boundary, superimposes a graphic having a display state set according to a distance between the reference plane and each of the three or more points on the three-dimensional shape, and displays the first graphic, which is the graphic superimposed on the three-dimensional shape corresponding to the first region. a 3D image display device that receives first information indicating a first range in which a gradation of a display state of a second graphic, the second graphic being the graphic superimposed on the three-dimensional shape corresponding to the second region, is set; sets the gradation of the display state of the first graphic in the first range indicated by the first information; and sets the gradation of the display state of the second graphic in the second range indicated by the second information, wherein setting of the gradation of the display state of the first graphic and setting of the gradation of the display state of the second graphic are performed independently of each other.

[0007] In the three-dimensional image display device of the present invention, the control unit receives the first information indicating the first range from the reference surface to a first position, the first position being a first distance away from the reference surface in the first region, and receives the second information indicating the second range from the reference surface to a second position, the second position being a second distance away from the reference surface in the second region.

[0008] In the three-dimensional image display device of the present invention, the first range includes a first partial range from the reference surface to a third position and a second partial range from the third position to the first position, and the third position is separated from the reference surface in the first region by a third distance that is smaller than the first distance, and the control unit receives partial range information, which is the first information indicating the first partial range and the second partial range, and sets the gradation of the display state of the first graphic in the first partial range indicated by the partial range information, and sets the gradation of the display state of the first graphic in the second partial range indicated by the partial range information.

[0009] In the three-dimensional image display device of the present invention, the second range includes a third partial range from the reference surface to a fourth position and a fourth partial range from the fourth position to the second position, and the fourth position is located in the second region at a fourth distance from the reference surface that is smaller than the second distance, and the control unit receives partial range information, which is the second information indicating the third partial range and the fourth partial range, and sets the gradation of the display state of the second graphic in the third partial range indicated by the partial range information, and sets the gradation of the display state of the second graphic in the fourth partial range indicated by the partial range information.

[0010] In the three-dimensional image display device of the present invention, the control unit superimposes the graphic having a color as the display state on the three-dimensional shape.

[0011] In the three-dimensional image display device of the present invention, the control unit superimposes a first color on the three-dimensional shape as the first graphic at the first position, and superimposes a second color different from the first color on the three-dimensional shape as the second graphic at the second position.

[0012] In the three-dimensional image display device of the present invention, the control unit superimposes a third color, which is different from the first color and the second color, on the three-dimensional shape as the graphic on the reference surface.

[0013] In the three-dimensional image display device of the present invention, the control unit superimposes the first color on the three-dimensional shape as the first graphic in an area in the first region that is a distance from the reference surface that is greater than the first distance, and superimposes the second color on the three-dimensional shape as the second graphic in an area in the second region that is a distance from the reference surface that is greater than the second distance.

[0014] In the three-dimensional image display device of the present invention, the control unit receives the first information including a numerical value indicating the first position, and receives the second information including a numerical value indicating the second position.

[0015] In the three-dimensional image display device of the present invention, the control unit sets a measurement point, and sets a position that is separated from the reference plane by a distance between the reference plane and the measurement point as the first position or the second position.

[0016] In the three-dimensional image display device of the present invention, the control unit enlarges an area of ​​the three-dimensional shape on which the first graphic is superimposed and an area of ​​the three-dimensional shape on which the second graphic is superimposed.

[0017] In the three-dimensional image display device of the present invention, the control unit sets one of a switchable first mode and a switchable second mode, and in the first mode, the setting of the gradation of the display state of the first graphic and the setting of the gradation of the display state of the second graphic are executed independently of each other, and in the second mode, the setting of the gradation of the display state of the first graphic and the setting of the gradation of the display state of the second graphic are executed in conjunction with each other.

[0018] In the three-dimensional image display device of the present invention, the control unit receives an instruction to designate a point on the subject, and sets the reference plane based on the point designated by the instruction.

[0019] In the three-dimensional image display device of the present invention, if the number of points included in the first region or the second region is less than a predetermined number, the control unit outputs information prompting the user to re-specify points on the subject.

[0020] In the three-dimensional image display device of the present invention, the control unit displays a graphical user interface including a movable first slider and a movable second slider on the display, accepts the first information in response to operation of the first slider, and accepts the second information in response to operation of the second slider.

[0021] In the three-dimensional image display device of the present invention, the control unit switches between a state in which the entire reference plane is displayed on the display and a state in which at least a part of the reference plane is not displayed.

[0022] The present invention relates to a method for displaying a three-dimensional shape of an object on a display, the method comprising: a control unit acquiring three-dimensional image data including three-dimensional coordinates of three or more points on the object calculated based on an endoscopic image of the object; displaying a three-dimensional shape of the object on a display based on the three-dimensional image data; setting a reference plane based on one or more points included in the three or more points; dividing the three-dimensional image data into a first region and a second region using the reference plane as a boundary; superimposing a graphic having a display state set according to a distance between the reference plane and each of the three or more points on the three-dimensional shape; and displaying a tier of display states of the first graphic, which is the graphic superimposed on the three-dimensional shape corresponding to the first region. a 3D image display method including: receiving first information indicating a first range in which a gradation of a display state of a second graphic, the second graphic being the graphic superimposed on the three-dimensional shape corresponding to the second region, is to be set; receiving second information indicating a second range in which a gradation of a display state of the first graphic in the first range indicated by the first information; setting the gradation of the display state of the second graphic in the second range indicated by the second information; and setting the gradation of the display state of the first graphic and the gradation of the display state of the second graphic independently of each other.

[0023] The present invention provides a method for displaying a three-dimensional shape of the subject on a display, the method comprising the steps of: acquiring three-dimensional image data including three-dimensional coordinates of three or more points on the subject calculated based on an endoscopic image of the subject; displaying a three-dimensional shape of the subject on a display based on the three-dimensional image data; setting a reference plane based on one or more points included in the three or more points; dividing the three-dimensional image data into a first region and a second region using the reference plane as a boundary; superimposing a graphic having a display state set according to the distance between the reference plane and each of the three or more points on the three-dimensional shape; and setting a first gradation of the display state of the first graphic, which is the graphic superimposed on the three-dimensional shape corresponding to the first region. a step of receiving first information indicating a range; a step of receiving second information indicating a second range in which the gradation of the display state of a second graphic, which is the graphic superimposed on the three-dimensional shape corresponding to the second region, is set; a step of setting the gradation of the display state of the first graphic in the first range indicated by the first information; and a step of setting the gradation of the display state of the second graphic in the second range indicated by the second information, wherein the setting of the gradation of the display state of the first graphic and the setting of the gradation of the display state of the second graphic are executed independently of each other. [Effects of the Invention]

[0024] According to the present invention, the three-dimensional image display device, the three-dimensional image display method, and the program are capable of visualizing fine undulations of an object. [Brief explanation of the drawings]

[0025] [Figure 1] 1 is a block diagram showing an example of the configuration of an endoscope system according to a first embodiment of the present invention. [Figure 2] 5 is a flowchart showing an example of a procedure for a three-dimensional (3D) shape display process according to the first embodiment of the present invention. [Figure 3]3A to 3C are diagrams showing examples of images displayed on a display included in the endoscope system according to the first embodiment of the present invention. [Figure 4] 3A to 3C are diagrams showing examples of images displayed on a display included in the endoscope system according to the first embodiment of the present invention. [Figure 5] 3A to 3C are diagrams showing examples of images displayed on a display included in the endoscope system according to the first embodiment of the present invention. [Figure 6] FIG. 1 is a diagram showing a 3D space defined in 3D image data in a first embodiment of the present invention. [Figure 7] 2 is a diagram showing an example of a graphical user interface (GUI) displayed on a display included in the endoscope system according to the first embodiment of the present invention. FIG. [Figure 8] 2A to 2C are diagrams showing examples of images of a 3D shape of a subject in the first embodiment of the present invention. [Figure 9] 3A to 3C are diagrams showing examples of images displayed on a display included in the endoscope system according to the first embodiment of the present invention. [Figure 10] FIG. 1 is a diagram showing an example of a GUI in the prior art to be compared with the first embodiment of the present invention. [Figure 11] 1A and 1B are diagrams showing examples of images in the prior art to be compared with the first embodiment of the present invention. [Figure 12] FIG. 2 is a diagram showing the positional relationship between a reference surface, a first region, and a second region in the first embodiment of the present invention. [Figure 13] FIG. 2 is a diagram showing the positional relationship between a reference surface, a first region, and a second region in the first embodiment of the present invention. [Figure 14] FIG. 2 is a diagram showing the positional relationship between a reference surface, a first region, and a second region in the first embodiment of the present invention. [Figure 15] FIG. 10 is a diagram showing an example of a GUI displayed on a display included in an endoscope system according to a first modified example of the first embodiment of the present invention. [Figure 16]FIG. 10 is a diagram showing an example of an image displayed on a display included in an endoscope system according to a second modified example of the first embodiment of the present invention. [Figure 17] FIG. 10 is a block diagram showing an example of the configuration of an endoscope system according to a third modified example of the first embodiment of the present invention. [Figure 18] FIG. 10 is a block diagram showing an example of the configuration of an endoscope system according to a fourth modified example of the first embodiment of the present invention. [Figure 19] 10 is a flowchart showing an example of a procedure for 3D shape display processing according to the second embodiment of the present invention. [Figure 20] 10A and 10B are diagrams showing examples of images displayed on a display included in an endoscope system according to a second embodiment of the present invention. [Figure 21] FIG. 11 is a diagram showing an example of a method for setting the color gradation of a graphic to be superimposed on a 3D shape of a subject in the third embodiment of the present invention. [Figure 22] FIG. 11 is a diagram showing an example of a method for setting the color gradation of a graphic to be superimposed on a 3D shape of a subject in the third embodiment of the present invention. [Figure 23] FIG. 11 is a diagram showing an example of a method for setting the color gradation of a graphic to be superimposed on a 3D shape of a subject in the third embodiment of the present invention. [Figure 24] FIG. 11 is a diagram showing an example of a method for setting the color gradation of a graphic to be superimposed on a 3D shape of a subject in the third embodiment of the present invention. [Figure 25] FIG. 10 is a diagram showing an example of a GUI displayed on a display included in an endoscope system according to a fourth embodiment of the present invention. [Figure 26] FIG. 10 is a diagram showing an example of a GUI displayed on a display included in an endoscope system according to a fourth embodiment of the present invention. [Figure 27] FIG. 13 is a diagram showing an example of an image of a 3D shape of a subject in the fifth embodiment of the present invention. [Figure 28] 13 is a flowchart showing an example of a processing procedure for specifying an area to be extracted from an image of a 3D shape of a subject in the fifth embodiment of the present invention. [Figure 29]FIG. 20 is a diagram showing an example of the positional relationship between the 3D shape of a subject and a reference plane in the sixth embodiment of the present invention. [Figure 30] FIG. 13 is a diagram showing an example of information displayed on a display included in an endoscope system according to a sixth embodiment of the present invention. [Figure 31] FIG. 13 is a diagram showing an example of an image displayed on a display of an endoscope system according to a seventh embodiment of the present invention. [Figure 32] FIG. 13 is a diagram showing an example of an image displayed on a display of an endoscope system according to a seventh embodiment of the present invention. [Figure 33] FIG. 13 is a diagram showing an example of an image displayed on a display of an endoscope system according to a seventh embodiment of the present invention. [Figure 34] FIG. 13 is a diagram showing an example of an image displayed on a display of an endoscope system according to a seventh embodiment of the present invention. [Figure 35] FIG. 13 is a diagram showing an example of an image displayed on a display of an endoscope system according to a seventh embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0026] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will now be described with reference to the accompanying drawings. An endoscope system will be described below as an example of a three-dimensional (3D) image display device.

[0027] (First embodiment) Fig. 1 shows an example of the configuration of an endoscope system 1 according to a first embodiment of the present invention. The endoscope system 1 shown in Fig. 1 includes an insertion section 2, a scope unit 3, a base unit 4, and a main unit 5. The insertion section 2, the scope unit 3, and the base unit 4 constitute an endoscope device 10. The main unit 5 is an operation device.

[0028] The insertion section 2 is inserted into the inside of the subject to be observed. The subject is an industrial product. The insertion section 2 is a long, thin tube that is bendable. The user performs an insertion operation to insert the insertion section 2 into the subject. An optical adapter is attached to the tip of the insertion section 2. The insertion section 2 acquires an optical image of the inside of the subject. The insertion section 2 has an imaging section 20, a bending section 21, and an illumination window 22.

[0029] The imaging unit 20 is disposed in the distal end portion 2a including the distal end of the insertion portion 2. The imaging unit 20 is an image sensor such as a CCD (Charge-Coupled Device) image sensor or a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor. The imaging unit 20 generates an image based on an optical image acquired by the insertion portion 2. The image generated by the imaging unit 20 is output to the scope unit 3.

[0030] The bending section 21 bends the insertion section 2 in the upward (U), downward (D), leftward (L), or rightward (R) direction. Alternatively, the bending section 21 bends the insertion section 2 in the upward-left (UL), upward-right (UR), downward-left (DL), or downward-right (DR) direction.

[0031] Illumination light is generated by a light source 35 included in the scope unit 3 and output to the distal end portion 2a through a light guide (not shown) arranged inside the insertion portion 2. The illumination light is irradiated from the illumination window 22 into the inside of the subject.

[0032] The scope unit 3 has an imaging drive circuit 30, an image processing unit 31, a UD drive unit 32, a RL drive unit 33, a bending control unit 34, a light source 35, and a light source control unit 36. The base unit 4 has a control unit 40, a communication unit 41, a volatile memory 42, and a non-volatile memory 43.

[0033] The imaging drive circuit 30 controls the imaging unit 20 and outputs the image output from the imaging unit 20 to the image processing unit 31. The image processing unit 31 performs image processing such as noise reduction on the image output from the imaging unit 20 and outputs the image to the control unit 40.

[0034] The UD driving unit 32 is connected to a UD bending wire for bending the bending portion 21 in the U direction or the D direction. The UD driving unit 32 has a motor, and bends the bending portion 21 in the U direction or the D direction by pulling the UD bending wire. The RL driving unit 33 is connected to an RL bending wire for bending the bending portion 21 in the R direction or the L direction. The RL driving unit 33 has a motor, and bends the bending portion 21 in the R direction or the L direction by pulling the RL bending wire. The bending control unit 34 controls the UD driving unit 32 and the RL driving unit 33.

[0035] The UD drive unit 32 and the RL drive unit 33 can operate simultaneously. For example, the UD drive unit 32 and the RL drive unit 33 can bend the bending section 21 in the UL direction.

[0036] The light source 35 is an LED (Light-Emitting Diode) or the like, and generates illumination light. The illumination light is output from the light source 35 to a light guide (not shown). The light source control unit 36 ​​controls the light source 35.

[0037] The control unit 40 controls each unit of the scope unit 3 and the base unit 4. At least one of the control unit 40, the image processing unit 31, the bending control unit 34, and the light source control unit 36 ​​may be configured with at least one of a processor and a logic circuit. For example, the processor is at least one of a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and a GPU (Graphics Processing Unit). For example, the logic circuit is at least one of an ASIC (Application Specific Integrated Circuit) and an FPGA (Field-Programmable Gate Array). At least one of the control unit 40, the image processing unit 31, the bending control unit 34, and the light source control unit 36 ​​may include one or more processors. At least one of the control unit 40, the image processing unit 31, the bending control unit 34, and the light source control unit 36 ​​may include one or more logic circuits.

[0038] The computer of the endoscope system 1 may load a program and execute the loaded program. The program includes instructions that define the operation of at least one of the control unit 40, the image processing unit 31, the bending control unit 34, and the light source control unit 36. In other words, the function of at least one of the control unit 40, the image processing unit 31, the bending control unit 34, and the light source control unit 36 ​​may be realized by software.

[0039] The above program may be provided by a "computer-readable recording medium" such as a flash memory. The program may be transmitted from a computer storing the program to the endoscope system 1 via a transmission medium or by transmission waves in the transmission medium. A "transmission medium" that transmits the program is a medium that has the function of transmitting information. Media that have the function of transmitting information include networks (communication networks) such as the Internet and communication lines (communication lines) such as telephone lines. The above program may realize some of the above functions. Furthermore, the above program may be a difference file (difference program). The above functions may be realized by combining a program already recorded on a computer with a difference program.

[0040] The communication unit 41 has a communication circuit and performs wired or wireless communication with the main unit 5 for curvature control and the like. The volatile memory 42 is RAM (Random Access Memory) or DRAM (Dynamic RAM), etc. The volatile memory 42 stores various information processed by the control unit 40. The non-volatile memory 43 is SRAM (Static RAM), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable ROM), or flash memory. The non-volatile memory 43 may be detachable from the base unit 4. The non-volatile memory 43 stores images generated by the imaging unit 20 and various information processed by the control unit 40.

[0041] The main unit 5 has a control unit 50, a display 51, a touch panel 52, operation buttons 53, a communication unit 54, a communication unit 55, a volatile memory 56, and a non-volatile memory 57. The main unit 5 may be an information terminal such as a smartphone or a tablet terminal.

[0042] The control unit 50 controls each part of the main unit 5. The control unit 50 may be composed of at least one of a processor and a logic circuit. The control unit 50 may include one or more processors. The control unit 50 may include one or more logic circuits. The computer of the endoscope system 1 may load a program and execute the loaded program. The program includes instructions that define the operation of the control unit 50. In other words, the functions of the control unit 50 may be realized by software. The program that realizes the functions of the control unit 50 may be realized in the same way as the program that realizes the functions of the control unit 40, etc.

[0043] The display 51 is a monitor such as an LCD (Liquid Crystal Display). The display 51 displays an image generated by the imaging unit 20. The touch panel 52 accepts operations for inputting information necessary for controlling the endoscope system 1. The touch panel 52 is arranged on the screen of the display 51. By operating the touch panel 52, the user can input instructions to change the settings of the endoscope system 1, instructions necessary for operating the endoscope system 1, and the like to the endoscope system 1.

[0044] The operation button 53 accepts various instructions from the user. By pressing the operation button 53, the user can input instructions regarding the power supply or lighting to the endoscope system 1. The communication unit 54 performs wired or wireless communication with the base unit 4 for bending control and the like. The communication unit 55 performs wired or wireless communication with the external device 11. The external device 11 is a remote control, a keyboard, a mouse, or the like.

[0045] The control unit 50 executes a 3D shape display process for displaying an image of the 3D shape of the subject on the display 51. An outline of the 3D shape display process will be described below.

[0046] The control unit 50 acquires 3D image data including 3D coordinates of three or more points on the subject. The 3D coordinates are defined in a 3D space corresponding to the real space. The scale of the 3D space may be the same as or different from the scale of the real space. The control unit 50 displays an image of the 3D shape of the subject on the display 51 based on the 3D image data. The control unit 50 accepts the designation of one or more reference points on the subject. In the first embodiment, the control unit 50 accepts the designation of two reference points.

[0047] The control unit 50 sets a reference plane (cutting reference) in the 3D space based on one or more reference points. Specifically, the control unit 50 sets a reference figure based on one or more reference points, and sets a reference plane based on the reference figure. The reference figure is a point, a line, a polygon, a plane, or a curved surface. In the first embodiment, an example will be described in which a boundary line is used as the reference figure.

[0048] The control unit 50 divides the entire area of ​​the subject into two areas by the reference plane. The cross section of the subject is formed by the intersection of the reference plane and the subject. The control unit 50 sets the display state of the graphic (computer graphic) according to the distance between the reference plane and each of three or more points included in the 3D image data. In the following example, the control unit 50 sets the color of the graphic. The control unit 50 superimposes a graphic with a color set according to the above distance on an image of the 3D shape of the subject. The control unit 50 sets the color gradation of the graphic in each of the two areas according to instructions from the user.

[0049] 2 shows an example of the procedure for the 3D shape display process. The operation of the endoscope system 1 will be described with reference to FIG.

[0050] The control unit 50 acquires 3D image data (step S100).

[0051] The control unit 50 executes the following process in step S100. First to third examples will be described below.

[0052] First, a first example will be described. The optical adapter in the first example is a stereo optical adapter with two fields of view. The optical adapter has a first optical system and a second optical system corresponding to the two fields of view. The first optical system and the second optical system form two optical images of the subject on the imaging unit 20. The imaging unit 20 generates a stereo image corresponding to the first optical image and the second optical image. The stereo image includes a pair of two images (a first image and a second image). In other words, the stereo image includes an image of the subject viewed from a first viewpoint and an image of the subject viewed from a second viewpoint. The control unit 50 calculates the 3D coordinates of three or more points on the subject by using one or more stereo images generated by the imaging unit 20, and generates 3D image data including the 3D coordinates.

[0053] Next, a second example will be described. The optical adapter in the second example is a monocular optical adapter with one field of view. While the optical adapter in the first example forms two optical images of the subject, the optical adapter in the second example forms one optical image of the subject. The imaging unit 20 generates an image corresponding to the optical image formed by the optical adapter. The imaging unit 20 performs imaging from two or more different viewpoints and generates two or more images. The control unit 50 calculates the 3D coordinates of three or more points on the subject by using the two or more images generated by the imaging unit 20, and generates 3D image data including the 3D coordinates.

[0054] Next, a third example will be described. The 3D image data generated in the first or second example is stored in advance in the nonvolatile memory 57. The control unit 50 acquires the 3D image data from the nonvolatile memory 57.

[0055] After step S100, the control unit 50 generates a 3D image based on the 3D image data and outputs the 3D image to the display 51. The display 51 displays the 3D image (step S101). The 3D image is an image of the 3D shape of the subject.

[0056] 3 shows an example of a 3D image displayed on the display 51 in step S101. The control unit 50 displays the 3D image IMG10 shown in FIG. 3 on the display 51. The 3D image IMG10 includes a 3D shape SH10 of the subject. The control unit 50 displays an icon IC10 on the 3D image IMG10. The icon IC10 includes an illustration of the subject and prompts the user to set a first reference point.

[0057] The control unit 50 displays a cursor CS10 on the 3D image IMG10. The user moves the cursor CS10 on the 3D image IMG10 by operating the touch panel 52. The user may also move the cursor CS10 on the 3D image IMG10 by operating the operation button 53, the external device 11, or the like.

[0058] The user operates the touch panel 52 or the like to input an instruction to set a reference point to the endoscope system 1. The control unit 50 receives the instruction and sets the reference point at the position of the cursor CS10 (step S102).

[0059] The following describes an example in which a boundary line serving as a reference figure is set using two reference points. When an instruction to set a first reference point is input, the control unit 50 accepts the instruction and sets the first reference point at the position of the cursor CS10. The first reference point is one of three or more points included in the 3D image data. Information about the first reference point is stored in the volatile memory 56.

[0060] 4 shows an example of a 3D image displayed on the display 51 when the first reference point is set. Portions that are the same as those shown in FIG. 3 will not be described.

[0061] The control unit 50 displays an icon IC11 on the 3D image IMG 10. The icon IC11 includes an illustration of the subject and prompts the user to set a second reference point.

[0062] The user operates the touch panel 52 or the like to move the cursor CS10 on the 3D image IMG10. The user operates the touch panel 52 or the like to input an instruction to the endoscope system 1 to set a second reference point.

[0063] 5 shows an example of a 3D image displayed on the display 51 after the first reference point has been set. The same parts as those shown in FIG. 4 will not be described.

[0064] The control unit 50 displays a line L10 on the 3D image IMG10. The line L10 is a straight line connecting the first reference point and the point indicated by the cursor CS10. The line L10 is displayed as a thick line. When the user moves the cursor CS10, the line L10 moves.

[0065] When an instruction to set a second reference point is input, the control unit 50 accepts the instruction and sets the second reference point at the position of the cursor CS10. The second reference point is one of three or more points included in the 3D image data and is different from the first reference point. Information about the second reference point is stored in the volatile memory 56.

[0066] After step S102, the control unit 50 sets a reference plane that passes through the first reference point and the second reference point (step S103).

[0067] Step S103 will now be described in detail. Fig. 6 shows the 3D space defined in the 3D image data. The 3D image data includes 3D coordinates of three or more points that form the 3D shape SH11 of the subject. The 3D coordinates of the three or more points are composed of coordinate values ​​on the mutually orthogonal X-axis, Y-axis, and Z-axis.

[0068] In step S102, the control unit 50 sets a first reference point RP10 and a second reference point RP11. In step S103, the control unit 50 sets a boundary line BL10 that passes through the first reference point RP10 and the second reference point RP11. In step S103, the control unit 50 also sets a plane PL10 that includes the boundary line BL10 and is parallel to the Z axis. The plane PL10 is a reference surface. In the above example, the reference surface is a flat surface, but the reference surface may also be a curved surface such as a cylindrical surface or a spherical surface.

[0069] After step S103, the control unit 50 divides the entire region of the 3D image data into a first region and a second region (step S104). Dividing the entire region into two regions means defining the two regions in the entire region. In other words, dividing the entire region into two regions means assigning each point included in the entire region to one of the two regions. The boundary between the first region and the second region is the reference plane set in step S103. Each of the three or more points included in the 3D image data is included in the first region or the second region.

[0070] After step S104, the control unit 50 calculates the 3D distance between the reference plane and each of the three or more points included in the 3D image data (step S105).

[0071] After step S105, the control unit 50 sets the color of the graphic at the reference position. The reference position indicates the position of a reference plane in 3D space. For example, the control unit 50 sets the color of the graphic at the reference position to green (step S106).

[0072] The control unit 50 may set the color of the graphic at the reference position based on information preset in the endoscope system 1. The control unit 50 may set the color of the graphic at the reference position in response to an instruction from the user.

[0073] As will be described later, the user can input information indicating a first position in the first area to the endoscope system 1 by operating the touch panel 52 or the like. After step S106, the control unit 50 sets the first position according to the information input by the user (step S107).

[0074] The first position indicates the boundary between the range in which the color gradation is set in the first region and the range in which the color does not change in the first region. The first position is separated from the reference plane in the first region by a first reference value. When the first position is set, a first range between the reference plane and the first position is set in the first region. If the user has not input information indicating the first position into the endoscope system 1, the control unit 50 does not execute step S107.

[0075] As will be described later, the user can input information indicating a second position in the second area to the endoscope system 1 by operating the touch panel 52 or the like. After step S107, the control unit 50 sets the second position according to the information input by the user (step S108).

[0076] The second position indicates the boundary between the range in which the color gradation is set in the second region and the range in which the color does not change in the second region. The second position is separated from the reference plane in the second region by a second reference value. When the second position is set, a second range between the reference plane and the second position is set in the second region. If the user has not input information indicating the second position into the endoscope system 1, the control unit 50 does not execute step S108.

[0077] The order in which steps S107 and S108 are performed is not limited to the order shown in Fig. 2. Step S107 may be performed after step S108 is performed.

[0078] After step S108, the control unit 50 sets the color of the graphic at the first position and the color of the graphic at the second position. For example, the control unit 50 sets the color of the graphic at the first position to red and the color of the graphic at the second position to blue (step S109).

[0079] The control unit 50 may set the color of the graphic at each of the first position and the second position based on information preset in the endoscope system 1. The control unit 50 may set the color of the graphic at each of the first position and the second position in response to an instruction from the user.

[0080] When the control unit 50 does not execute step S107, the control unit 50 does not set the color of the graphic at the first position in step S109. When the control unit 50 does not execute step S108, the control unit 50 does not set the color of the graphic at the second position in step S109.

[0081] After step S109, the control unit 50 sets the color of the graphic of each point according to the distance between the reference plane and each point included in the 3D image data (step S110).

[0082] As a point in the first region moves away from the reference plane toward the first position, the graphic color of the point gradually changes from green to red. If the position of the reference plane is defined as the center, the graphic color of points in the first region outside the first position is set to red.

[0083] As a point in the second region moves away from the reference plane toward the second position, the graphic color of the point gradually changes from green to blue. If the position of the reference plane is defined as the center, the graphic color of points in the second region outside the second position is set to blue.

[0084] After step S110, the control unit 50 superimposes colors on the surface of the 3D shape formed by three or more points included in the 3D image data. Specifically, the control unit 50 superimposes colors on the area between two adjacent points so that the area has gradations according to the two colors set for the two points (step S111).

[0085] After step S111, the control unit 50 displays the image of the 3D shape on which the color has been superimposed in step S111 on the display 51 (step S112).

[0086] After step S112, the control unit 50 determines whether to end the setting of the color gradation of the 3D shape of the object in response to an instruction from the user (step S113). For example, when the user has not input an instruction to end the setting of the color gradation, the control unit 50 determines not to end the setting of the color gradation. When the user has input an instruction to end the setting of the color gradation, the control unit 50 determines to end the setting of the color gradation.

[0087] When the control unit 50 determines in step S113 that the setting of the color gradation is not to be ended, step S107 is executed. When the control unit 50 determines in step S113 that the setting of the color gradation is to be ended, the 3D shape display process shown in FIG. 2 ends.

[0088] 7(a) and 7(b) show examples of the graphical user interface (GUI) displayed on the display 51 in steps S107 and S108.

[0089] As shown in FIG. 7(a), the control unit 50 displays an operation panel PN10, which is a GUI, on the display 51. The operation panel PN10 includes a slider bar SB10 and a slider bar SB11. The slider bar SB10 and the slider bar SB11 can be moved left or right on the operation panel PN10. The user moves the slider bar SB10 and the slider bar SB11 by operating the touch panel 52 or the like.

[0090] The control unit 50 sets a first inspection reference value and a second inspection reference value to be used in determining the inspection result. The first inspection reference value indicates a reference value for the height of a convex portion as a distance from a reference surface. The second inspection reference value indicates a reference value for the depth of a concave portion as a distance from the reference surface. For example, the first inspection reference value is a positive value, and the second inspection reference value is a negative value.

[0091] When the height (positive value) of a convex portion is equal to or less than the first inspection standard value, the inspection result of that convex portion is pass. When the height (positive value) of a convex portion is greater than the first inspection standard value, the inspection result of that convex portion is fail. When the depth (negative value) of a concave portion is equal to or greater than the second inspection standard value, the inspection result of that concave portion is pass. When the depth (negative value) of a concave portion is less than the second inspection standard value, the inspection result of that concave portion is fail.

[0092] In the example shown in Figure 7(a), the first inspection reference value is 8.46 mm, and the second inspection reference value is -8.46 mm. Reference position P10 shown in Figure 7(a) indicates the position (0.00 mm) of the reference plane. Each position in the distance from the position (0.00 mm) of the reference plane to the first inspection reference value (8.46 mm) corresponds to a value from 0% to 50%. Also, each position in the distance from the position (0.00 mm) of the reference plane to the second inspection reference value (-8.46 mm) corresponds to a value from 0% to 50%.

[0093] The user can move the slider bar SB10 between the reference position P10 and the position P11 of the first inspection reference value, and the user can move the slider bar SB12 between the reference position P10 and the position P12 of the second inspection reference value.

[0094] When the user moves slider bar SB10 or slider bar SB11, touch panel 52 or the like outputs information indicating the position of slider bar SB10 or slider bar SB11. Control unit 50 receives the information and sets a first position in the first area or a second position in the second area in accordance with the information. In this way, control unit 50 sets the range within which the color gradation of the graphic is set.

[0095] By operating the slider bar SB10, the user can input information indicating a first position (first reference value) to the endoscope system 1. The information indicating the position of the slider bar SB10 is equivalent to information indicating the range from the reference position to the first position. By operating the slider bar SB11, the user can input information indicating a second position (second reference value) to the endoscope system 1. The information indicating the position of the slider bar SB11 is equivalent to information indicating the range from the reference position to the second position.

[0096] In the example shown in Figure 7(a), the first position corresponding to the position of slider bar SB10 is 2.54 mm away from the reference plane, and the second position corresponding to the position of slider bar SB11 is 5.07 mm away from the reference plane. The distance between the reference plane and the first position (2.54 mm) corresponds to the first reference value. The distance between the reference plane and the second position (-5.07 mm) corresponds to the second reference value.

[0097] 7(a), the distance between the reference plane and each point in the first region is expressed as a positive value, and the distance between the reference plane and each point in the second region is expressed as a negative value. The distance between the reference plane and each point in the first region may be expressed as a negative value, and the distance between the reference plane and each point in the second region may be expressed as a positive value.

[0098] The control unit 50 sets the color of the graphic at the reference position P10 to green, the color of the graphic at the first position corresponding to the position of the slider bar SB10 to red, and the color of the graphic at the second position corresponding to the position of the slider bar SB11 to blue. The control unit 50 sets the color of the graphic at each position in the range R10 from the first position to the position P11 to red. The control unit 50 sets the color of the graphic at each position in the range R11 from the second position to the position P12 to blue.

[0099] The control unit 50 sets the color of the graphic at each position in a range R12 from the reference position P10 to the first position. That is, the control unit 50 sets the gradation of the color of the graphic in the range R12. The color of the graphic changes between green and red depending on the distance between the reference position P10 and each position. The control unit 50 sets the color of the graphic at each position in a range R13 from the reference position P10 to the second position. That is, the control unit 50 sets the gradation of the color of the graphic in the range R13. The color of the graphic changes between green and blue depending on the distance between the reference position P10 and each position.

[0100] The user can operate slider bar SB10 and slider bar SB11 separately, which allows control unit 50 to set the color of the graphic at each position in range R12 and the color of the graphic at each position in range R13 separately.

[0101] Fig. 7(b) shows another example of the operation panel PN10. The operation panel PN10 shown in Fig. 7(b) includes slider bars SB10 and SB11, similar to the operation panel PN10 shown in Fig. 7(a).

[0102] In the example shown in Fig. 7(a), the reference position P10 is the center between the position P11 of the first inspection reference value and the position P12 of the second inspection reference value. In the example shown in Fig. 7(b), the reference position P10 is not the center between the position P11 of the first inspection reference value and the position P12 of the second inspection reference value.

[0103] In the examples shown in FIGS. 7(a) and 7(b), the value indicating the position is a signed value, but it may be an unsigned value.

[0104] Figure 8 shows an example of a 3D image of a subject. The 3D image IMG11 shown in Figure 8 includes a 3D shape SH12 of the subject having a circular recess. A reference plane RS10 is set that includes a triangle as a reference figure with three points on the 3D shape SH12 as vertices. The reference plane RS10 is a plane.

[0105] 9 shows an example of a 3D image displayed on the display 51 in step S112. The control unit 50 displays the 3D image IMG13 on the display 51. The 3D image IMG13 includes a 3D shape SH13 to which a graphic is set. The graphic of each point in the 3D shape SH13 has the color set on the operation panel PN10 shown in FIG. 7(a) or 7(b).

[0106] Each point in the range R14 shown in Figure 9 is included in the recess. The 3D shape SH13 in the range R14 includes many small undulations, which are shown as changes in color. The user can intuitively grasp the undulations.

[0107] Fig. 10 shows an example of a GUI in the prior art. A reference position P100 is set on an operation panel PN100 shown in Fig. 10. Also, a position P101 corresponding to a first inspection reference value and a position P102 corresponding to a second inspection reference value are set.

[0108] The color of the graphic at reference position P100 is set to green, the color of the graphic at position P101 is set to red, and the color of the graphic at position P102 is set to blue. The color of the graphic at each position in range R100 from reference position P100 to position P101 changes between green and red depending on the distance between reference position P100 and each position. The color of the graphic at each position in range R101 from reference position P100 to position P102 changes between green and blue depending on the distance between reference position P100 and each position.

[0109] FIG. 11 shows an example of a 3D image of a subject in the prior art. The 3D image IMG100 shown in FIG. 11 is a 3D image of the same subject as the subject in the 3D image IMG11 shown in FIG. 8. The 3D image IMG100 includes a 3D shape SH100. The graphics of each point in the 3D shape SH100 have the color set on the operation panel PN100 shown in FIG. 10.

[0110] Each point in range R100 shown in Fig. 11 is included in a recess. Range R100 corresponds to range R14 shown in Fig. 9. Although the 3D shape SH100 in range R100 includes many small undulations, the 3D shape SH100 in range R100 is displayed in a uniform color. Therefore, the user cannot intuitively grasp the undulations.

[0111] Figures 12(a), 12(b), 13(a), 13(b), 14(a), and 14(b) show the positional relationship between the reference plane, the first region, and the second region. In Figures 12(a), 12(b), 13(a), and 13(b), the line of sight when viewing the subject from viewpoint VP10 corresponding to the position of tip portion 2a is parallel to the Z axis.

[0112] 12(a), a reference plane RS11 is parallel to the Y-axis. The 3D shape SH14 of the object is divided into a first region FR10 and a second region SR10. The first region FR10 extends from the reference plane RS11 in the negative direction of the Z-axis. The second region SR10 extends from the reference plane RS11 in the positive direction of the Z-axis.

[0113] 12(b), the reference plane RS12 is parallel to the Z axis. The 3D shape SH15 of the object is divided into a first region FR11 and a second region SR11. The first region FR11 extends from the reference plane RS12 in the positive direction of the Y axis. The second region SR11 extends from the reference plane RS12 in the negative direction of the Y axis.

[0114] 13(a), a reference plane RS13 is parallel to the Z axis. The 3D shape SH16 of the object is divided into a first region FR12 and a second region SR12. The first region FR12 extends from the reference plane RS13 in the negative direction of the Y axis. The second region SR12 extends from the reference plane RS13 in the positive direction of the Y axis.

[0115] 13(b), reference plane RS14 is not parallel to the Y axis and not parallel to the Z axis. The 3D shape SH17 of the object is divided into a first region FR13 and a second region SR13. The first region FR13 extends from reference plane RS14 in the positive direction of the Y axis and the negative direction of the Z axis. The second region SR13 extends from reference plane RS14 in the negative direction of the Y axis and the positive direction of the Z axis.

[0116] 14(a) and 14(b), the object is cylindrical. In FIG. 14(a), the viewpoint is inside the 3D shape SH18 of the object. The reference plane RS15 is a cylindrical surface. The 3D shape SH18 is divided into a first region outside the reference plane RS15 and a second region inside the reference plane RS15.

[0117] 14(b), the viewpoint VP11 is outside the 3D shape SH19 of the object. The reference surface RS16 is a cylindrical surface. The 3D shape SH19 is divided into a first region inside the reference surface RS16 and a second region outside the reference surface RS16.

[0118] In the first embodiment, the control unit 50 controls the color of a graphic as the display state of the graphic superimposed on the 3D image of the subject. The display state of the graphic is not limited to color, and may be expressed by light and shade or shading of a pattern, etc. For example, the control unit 50 may superimpose a graphic represented by multiple lines on the 3D image and control the density of the lines according to the distance between the reference plane and each point included in the 3D image data. The control unit 50 may superimpose a graphic represented by dots on the 3D image and control the density of the dots according to the distance between the reference plane and each point included in the 3D image data.

[0119] Each embodiment of the 3D image display device of the present invention includes a control unit 50. The control unit 50 acquires 3D image data including 3D coordinates of three or more points on a subject calculated based on an endoscopic image of the subject. The control unit 50 displays a 3D shape of the subject on a display 51 based on the 3D image data. The control unit 50 sets a reference plane based on one or more points included in the three or more points. The control unit 50 divides the 3D image data into a first region and a second region using the reference plane as a boundary. The control unit 50 superimposes a graphic, the display state of which is set according to the distance between the reference plane and each of the three or more points, on the 3D shape. The control unit 50 accepts first information indicating a first range in which the gradation of the display state of the first graphic, which is a graphic superimposed on the 3D shape corresponding to the first region, is set. For example, the first range corresponds to range R12 shown in FIG. 7. The control unit 50 accepts second information indicating a second range in which the gradation of the display state of the second graphic, which is a graphic superimposed on the 3D shape corresponding to the second region, is set. For example, the second range corresponds to range R13 shown in Fig. 7. The control unit 50 sets the gradation of the display state of the first graphic in the first range indicated by the first information. The control unit 50 sets the gradation of the display state of the second graphic in the second range indicated by the second information. The setting of the gradation of the display state of the first graphic and the setting of the gradation of the display state of the second graphic are performed independently of each other.

[0120] A 3D image display method according to each aspect of the present invention includes first to ninth steps. In a first step (step S100), the control unit 50 acquires 3D image data including 3D coordinates of three or more points on the subject calculated based on an endoscopic image of the subject. In a second step (step S101), the control unit 50 displays a 3D shape of the subject on the display 51 based on the 3D image data. In a third step (step S103), the control unit 50 sets a reference plane based on one or more points included in the three or more points. In a fourth step (step S104), the control unit 50 divides the 3D image data into a first region and a second region using the reference plane as a boundary. In a fifth step (step S111), the control unit 50 superimposes a graphic, the display state of which is set according to the distance between the reference plane and each of the three or more points, on the 3D shape. In a sixth step (step S107), the control unit 50 receives first information indicating a first range in which the gradation of the display state of a first graphic, which is a graphic superimposed on a 3D shape corresponding to the first region, is set. In a seventh step (step S108), the control unit 50 receives second information indicating a second range in which the gradation of the display state of a second graphic, which is a graphic superimposed on a 3D shape corresponding to the second region, is set. In an eighth step (step S110), the control unit 50 sets the gradation of the display state of the first graphic in the first range indicated by the first information. In a ninth step (step S110), the control unit 50 sets the gradation of the display state of the second graphic in the second range indicated by the second information. The setting of the gradation of the display state of the first graphic and the setting of the gradation of the display state of the second graphic are performed independently of each other.

[0121] The program according to each aspect of the present invention causes a computer to execute the first to ninth steps described above.

[0122] Each aspect of the present invention may include the following modifications. The control unit 50 receives first information indicating a first range from the reference surface to a first position. The first position is a first distance away from the reference surface in a first region. For example, the first position corresponds to the position of the slider bar SB10 shown in FIG. 7(a). The control unit 50 receives second information indicating a second range from the reference surface to a second position. The second position is a second distance away from the reference surface in a second region. For example, the second position corresponds to the position of the slider bar SB11 shown in FIG. 7(a). The second distance may be the same as the first distance or may be different from the first distance.

[0123] Each aspect of the present invention may include the following modifications: The control unit 50 superimposes a graphic having a color as a display state on the 3D shape of the subject.

[0124] Each aspect of the present invention may include the following modifications: The control unit 50 superimposes a first color on the 3D shape as a first graphic at a first position, and superimposes a second color different from the first color on the 3D shape as a second graphic at a second position. For example, the first color is red and the second color is blue.

[0125] Each aspect of the present invention may include the following modifications: The control unit 50 superimposes a third color different from the first and second colors onto the 3D shape as a graphic on the reference surface. For example, the third color is green.

[0126] Each aspect of the present invention may include the following modifications. The control unit 50 superimposes a first color onto the 3D shape as a first graphic in a range in a first region that is a distance greater than a first distance from the reference surface. For example, this range corresponds to the range R10 shown in FIG. 7(a). The control unit 50 superimposes a second color onto the 3D shape as a second graphic in a range in a second region that is a distance greater than a second distance from the reference surface. For example, this range corresponds to the range R11 shown in FIG. 7(a).

[0127] Each aspect of the present invention may include the following modifications: The control unit 50 receives an instruction to designate a point on the subject, and sets a reference plane based on the point designated by the instruction.

[0128] Each aspect of the present invention may include the following modifications. The control unit 50 displays a GUI including a movable first slider and a movable second slider on the display 51. For example, the first slider corresponds to the slider bar SB10 shown in FIG. 7(a), and the second slider corresponds to the slider bar SB11 shown in FIG. 7(a). The control unit 50 receives first information in response to the operation of the first slider, and receives second information in response to the operation of the second slider.

[0129] As described above, the control unit 50 receives information indicating the ranges for setting the gradation of the display state of the graphic superimposed on the 3D shapes corresponding to each of the first and second regions, and sets the gradation of the display state of the graphic within the ranges indicated by the information. Therefore, the endoscope system 1 can visualize fine undulations of the subject. Furthermore, for example, in an examination in which the presence of convex shapes is not a problem but concave shapes need to be detected with high sensitivity, the first and second ranges can be set independently to visualize the 3D shape in a way that accentuates fine concave shapes while not accentuating convex shapes. Therefore, the endoscope system 1 can improve the efficiency of the examination.

[0130] (First Modification of the First Embodiment) A first modified example of the first embodiment of the present invention will be described. In the above-described first embodiment, a user operates a slider bar on an operation panel to input information (first reference value or second reference value) indicating a first position in a first area or a second position in a second area to the endoscope system 1. On the other hand, in a first modified example of the first embodiment, a user inputs the first reference value or second reference value to the endoscope system 1 by directly inputting the first reference value or second reference value to the operation panel.

[0131] 15 shows an example of a GUI displayed on the display 51 in steps S107 and S108 shown in FIG. 2. The control unit 50 displays an operation panel PN11 on the display 51. The operation panel PN11 includes an input field IF10 and an input field IF11. The user can directly input a first reference value into the input field IF10 and can directly input a second reference value into the input field IF11.

[0132] The control unit 50 accepts a first reference value input into the input field IF10 and accepts a second reference value input into the input field IF11. The control unit 50 sets a first position indicated by the first reference value and sets a second position indicated by the second reference value.

[0133] The control unit 50 sets the color of the graphic at a reference position corresponding to the reference surface to green, the color of the graphic at the first position to red, and the color of the graphic at the second position to blue. The control unit 50 sets the color gradation of the graphic in the range from the reference position to the first position. The color of the graphic changes between green and red depending on the distance between the reference position and each position. The control unit 50 sets the color gradation of the graphic in the range from the reference position to the second position. The color of the graphic changes between green and blue depending on the distance between the reference position and each position.

[0134] The control unit 50 sets the color of the graphic at each position outside the first position to red. The distance between the reference plane and each position outside the first position is greater than a first reference value (positive value). The control unit 50 sets the color of the graphic at each position outside the second position to blue. The distance between the reference plane and each position outside the second position is less than a second reference value (negative value).

[0135] 15, the unit of the first reference value or the second reference value is mm. The first reference value or the second reference value may be expressed as a percentage (%) of a preset reference length.

[0136] Each aspect of the present invention may include the following modifications: The control unit 50 receives first information including a numerical value indicating a first position, and receives second information including a numerical value indicating a second position.

[0137] In a first modification of the first embodiment, the user can directly specify a first or second reference value that is known in advance. By directly specifying a threshold value for determining whether the test is pass or fail as the reference value, the area exceeding the threshold is displayed in the same color, which has the effect of making it easier for the user to visually identify the range of the area exceeding the threshold.

[0138] (Second Modification of the First Embodiment) A second modified example of the first embodiment of the present invention will be described. In the second modified example of the first embodiment, the user specifies a first position in a first region or a second position in a second region on a 3D image displayed on the display 51. In this way, the user inputs a first reference value or a second reference value into the endoscope system 1.

[0139] Fig. 16 shows an example of a 3D image displayed on the display 51 in step S112 shown in Fig. 2. The control unit 50 displays a 3D image IMG14 on the display 51. The 3D image IMG14 includes a 3D shape SH18 to which a graphic is set.

[0140] The control unit 50 displays cursors CS11 and CS12 on the 3D image IMG14. The user moves cursors CS11 and CS12 on the 3D image IMG14 by operating the touch panel 52 or the like. The control unit 50 sets the distance between the reference plane and the point at the position of cursor CS11 as a first reference value. The control unit 50 also sets the distance between the reference plane and the point at the position of cursor CS12 as a second reference value.

[0141] The control unit 50 sets the color of the graphic at each position in the range from the reference position corresponding to the reference plane to the first position indicated by the first reference value. The control unit 50 also sets the color of the graphic at each position in the range from the reference position to the second position indicated by the second reference value. The graphic at each point in the 3D shape SH18 has the color set as described above.

[0142] In the second modification of the first embodiment, the user can easily input the first reference value or the second reference value to the endoscope system 1 by specifying the first position or the second position on the 3D image displayed on the display 51. If a shape corresponding to the inspection threshold exists on the subject, the user can set the threshold more easily.

[0143] (Third Modification of the First Embodiment) A third modified example of the first embodiment of the present invention will be described. Fig. 17 shows an example of the configuration of an endoscope system 1a according to the third modified example of the first embodiment. The same parts as those shown in Fig. 1 will not be described.

[0144] 17 includes an insertion portion 2 and a main unit 6. The insertion portion 2 and the main unit 6 constitute an endoscope device 10a.

[0145] The insertion section 2 shown in Fig. 17 is the same as the insertion section 2 shown in Fig. 1. The main unit 6 has an imaging drive circuit 30, an image processing section 31, a UD drive section 32, a RL drive section 33, a bending control section 34, a light source 35, a light source control section 36, a display 51, a touch panel 52, operation buttons 53, a communication section 55, a volatile memory 56, a non-volatile memory 57, and a control section 60. The same reference symbols as those in Fig. 1 are assigned to the same blocks as those in Fig. 1.

[0146] The control unit 60 has both the functions of the control unit 40 shown in Fig. 1 and the functions of the control unit 50 shown in Fig. 1. The control unit 60 executes the processing shown in Fig. 2.

[0147] In the third modified example of the first embodiment, the endoscope system 1 can visualize fine undulations of the subject, similarly to the first embodiment.

[0148] (Fourth Modification of the First Embodiment) A fourth modified example of the first embodiment of the present invention will be described. Fig. 18 shows an example of the configuration of an endoscope system 1b according to the fourth modified example of the first embodiment. Portions that are the same as those shown in Fig. 1 will not be described.

[0149] 18 includes an insertion section 2, a scope unit 3b, and a base unit 7. The insertion section 2 and the scope unit 3b constitute an endoscope device 10b. The scope unit 3b and the base unit 7 are connected by a cable 8.

[0150] The insertion section 2 shown in Fig. 18 is the same as the insertion section 2 shown in Fig. 1. The scope unit 3b shown in Fig. 18 is the same as the scope unit 3 shown in Fig. 1 except that it does not have the image processing section 31. The base unit 7 has the image processing section 31, a display 51, a touch panel 52, operation buttons 53, a communication section 55, a volatile memory 56, a non-volatile memory 57, and a control section 70. The same reference symbols as those in Fig. 1 are assigned to blocks that are the same as those in Fig. 1.

[0151] The control unit 70 has both the functions of the control unit 40 shown in Fig. 1 and the functions of the control unit 50 shown in Fig. 1. The control unit 70 executes the processing shown in Fig. 2.

[0152] In the fourth modification of the first embodiment, the endoscope system 1 can visualize fine undulations of the subject, similarly to the first embodiment.

[0153] (Second embodiment) A second embodiment of the present invention will be described. In the second embodiment, an endoscope system 1 shown in Fig. 1 is used. An endoscope system 1a shown in Fig. 17 or an endoscope system 1b shown in Fig. 18 may also be used.

[0154] After the reference plane is set, the control unit 50 sets measurement points on the 3D image of the subject in accordance with instructions from the user. The control unit 50 calculates the distance between the reference plane and the measurement points. The control unit 50 uses the calculated distance as the first reference value or the second reference value.

[0155] The endoscope system 1 executes the 3D shape display process shown in Fig. 19. The operation of the endoscope system 1 will be described with reference to Fig. 19. Processes that are the same as those shown in Fig. 2 will not be described.

[0156] The user inputs an instruction to set a reference point into the endoscope system 1, and then operates the touch panel 52 or the like to input an instruction to set a measurement point into the endoscope system 1. After step S106, the control unit 50 accepts the instruction and sets the measurement point (step S120). The measurement point is one of three or more points included in the 3D image data. Information about the measurement point is stored in the volatile memory 56.

[0157] After step S120, the control unit 50 calculates the 3D distance between the reference plane set in step S103 and the measurement point set in step S120 (step S121). The 3D distance is used as a first reference value.

[0158] After step S121, the control unit 50 sets a position that is away from the reference plane by the 3D distance calculated in step S121 as the first position (step S122). After step S122, step S108 is executed.

[0159] The 3D distance calculated in step S121 may be used as the second reference value. The control unit 50 may set a position that is the 3D distance calculated in step S121 away from the reference plane as the second position.

[0160] When the control unit 50 determines in step S113 that the setting of the color gradation is not to be completed, step S120 is executed. If the position of the measurement point is different from the desired position, the user can set the measurement point again.

[0161] The user may input a first instruction to set a first measurement point and a second instruction to set a second measurement point to the endoscope system 1. The control unit 50 may accept the first instruction and the second instruction and may set the first measurement point and the second measurement point. The control unit 50 may calculate the 3D distance between a reference plane and the first measurement point, and may set a position that is that 3D distance away from the reference plane as the first position. The control unit 50 may calculate the 3D distance between the reference plane and the second measurement point, and may set a position that is that 3D distance away from the reference plane as the second position.

[0162] 20 shows an example of the 3D image displayed on the display 51 in step S112. The control unit 50 displays a 3D image IMG20 on the display 51. The 3D image IMG20 includes a 3D shape SH20 to which a graphic is set.

[0163] Before the graphic with set gradation is superimposed on the 3D shape SH20, the user operates the touch panel 52 or the like to input an instruction to set a measurement point to the endoscope system 1. The control unit 50 accepts the instruction and sets the measurement point MP20. The control unit 50 calculates the 3D distance between the reference plane and the measurement point MP20 and displays a measurement result MR20 indicating the 3D distance on the display 51. The control unit 50 sets a position that is the 3D distance away from the reference plane as a first position. Thereafter, the control unit 50 sets the color of the graphic to be superimposed on the 3D shape SH20 using a method similar to the method in the first embodiment.

[0164] Each aspect of the present invention may include the following modifications: The control unit 50 sets a measurement point, and sets a position that is separated from the reference surface by the distance between the reference surface and the measurement point as the first position or the second position.

[0165] In the second embodiment, the operation for setting the measurement point and the operation for setting the first position or the second position are common. This simplifies the operation performed by the user, reducing the burden on the user. Furthermore, the user can reset the measurement point by referring to the measurement results displayed on the display 51. For example, when it is necessary to set the measurement point at the deepest point of a recess or the highest point of a protrusion, the accuracy of the position of the measurement point is improved.

[0166] (Third embodiment) A third embodiment of the present invention will be described. In the third embodiment, the endoscope system 1 shown in Fig. 1 is used. An endoscope system 1a shown in Fig. 17 or an endoscope system 1b shown in Fig. 18 may also be used.

[0167] In the first embodiment described above, the control unit 50 sets a first position indicating the boundary between a range R12 in which color gradations are set in the first region and a range R10 in which color does not change in the first region. Also, in the first embodiment, the control unit 50 sets a second position indicating the boundary between a range R13 in which color gradations are set in the second region and a range R11 in which color does not change in the second region. In the third embodiment, the control unit 50 adds a boundary to the range in which color gradations are set in the first region or the second region, and sets an arbitrary color at the boundary.

[0168] FIG. 21 shows an example of a method for setting the color gradation of a graphic to be superimposed on the 3D shape of the subject in the third embodiment.

[0169] First, a first example will be described. The control unit 50 displays an operation panel PN30 shown in Fig. 21 on the display 51. The operation panel PN30 includes a slider bar SB30 and a slider bar SB31. The user moves the slider bar SB30 and the slider bar SB31 by operating the touch panel 52 or the like.

[0170] When the user moves slider bar SB30 or slider bar SB31, touch panel 52 or the like outputs information indicating the position of slider bar SB30 or slider bar SB31. Control unit 50 receives the information and sets a first position in the first area or a second position in the second area according to the information.

[0171] The control unit 50 sets the color of the graphic at the reference position P30 to green, sets the color of the graphic at a first position corresponding to the position of the slider bar SB30 to red, and sets the color of the graphic at a second position corresponding to the position of the slider bar SB31 to blue.

[0172] The control unit 50 sets the color gradation of the graphic in a range R30 from the reference position P30 to the first position. The color of the graphic changes between green and red depending on the distance between the reference position P30 and each position. The control unit 50 sets the color gradation of the graphic in a range R31 from the reference position P30 to the second position. The color of the graphic changes between green and blue depending on the distance between the reference position P30 and each position.

[0173] The control unit 50 superimposes a graphic having the color set as described above onto the 3D shape SH30 of the subject. The control unit 50 superimposes a green graphic on the point at reference position P31 of the 3D shape SH30. The control unit 50 superimposes a red graphic on the point at position P32 of the 3D shape SH30. The control unit 50 superimposes a blue graphic on the point at position P33 of the 3D shape SH30.

[0174] Next, a second example will be described. The control unit 50 displays an operation panel PN31 shown in Fig. 21 on the display 51. The operation panel PN31 includes slider bars SB30 to SB33. The user moves these slider bars by operating the touch panel 52 or the like.

[0175] As in the first example, control unit 50 sets a first position in the first area or a second position in the second area depending on the position of slider bar SB30 or slider bar SB31. When the user moves slider bar SB32 or slider bar SB33, touch panel 52 or the like outputs information indicating the position of slider bar SB32 or slider bar SB33.

[0176] The control unit 50 receives information indicating the position of the slider bar SB32 and sets a third position in the first region based on the information. The distance between the reference plane and the third position is smaller than the distance between the reference plane and the first position. The user can input information indicating the third position to the endoscope system 1 by operating the slider bar SB32. The information indicating the position of the slider bar SB32 is equivalent to information indicating the range from the reference position to the third position and the range from the third position to the first position.

[0177] The control unit 50 receives information indicating the position of the slider bar SB33 and sets a fourth position in the second region based on the information. The distance between the reference plane and the fourth position is smaller than the distance between the reference plane and the second position. The user can input information indicating the fourth position to the endoscope system 1 by operating the slider bar SB33. The information indicating the position of the slider bar SB33 is equivalent to information indicating the range from the reference position to the fourth position and the range from the fourth position to the second position.

[0178] The control unit 50 sets the color of the graphic at the reference position P34 to green, the color of the graphic at the first position corresponding to the position of the slider bar SB30 to red, and the color of the graphic at the second position corresponding to the position of the slider bar SB31 to blue. The control unit 50 sets the color of the graphic at the third position corresponding to the position of the slider bar SB32 to yellow. The control unit 50 sets the color of the graphic at the fourth position corresponding to the position of the slider bar SB33 to cyan.

[0179] The range from the reference position P34 to the first position includes a partial range PR30 from the reference position P34 to the third position and a partial range PR31 from the third position to the first position. The control unit 50 sets the color gradation of the graphic in the partial range PR30. The color of the graphic varies between green and yellow depending on the distance between the reference position P34 and each position. The control unit 50 sets the color gradation of the graphic in the partial range PR31. The color of the graphic varies between yellow and red depending on the distance between the reference position P34 and each position.

[0180] The range from the reference position P34 to the second position includes a partial range PR32 from the reference position P34 to the fourth position and a partial range PR33 from the fourth position to the second position. The control unit 50 sets the color gradation of the graphic in the partial range PR32. The color of the graphic varies between green and cyan depending on the distance between the reference position P34 and each position. The control unit 50 sets the color gradation of the graphic in the partial range PR33. The color of the graphic varies between cyan and blue depending on the distance between the reference position P34 and each position.

[0181] The control unit 50 superimposes a graphic having the color set as described above onto the 3D shape SH31 of the subject. The control unit 50 superimposes a green graphic on the point at reference position P31 of the 3D shape SH31. The control unit 50 superimposes a red graphic on the point at position P32 of the 3D shape SH31. The control unit 50 superimposes a blue graphic on the point at position P33 of the 3D shape SH31.

[0182] The control unit 50 superimposes a yellow graphic on the point at position P35 of the 3D shape SH31, and a cyan graphic on the point at position P36 of the 3D shape SH31.

[0183] The control unit 50 can set the color of the graphic at each of the first to fourth positions to any color. The control unit 50 may set the color of the graphic at two of the first to fourth positions to the same color. The control unit 50 may set only one of the second position and the third position. The control unit 50 may set one or more new positions in addition to the first to fourth positions.

[0184] The control unit 50 may set the color gradation of the graphic according to the method shown in Fig. 22. Fig. 22 shows an example of a method for setting the color gradation of the graphic to be superimposed on the 3D shape of the subject.

[0185] First, a first example will be described. Line L30 indicates the green component of the graphic at each position on the 3D shape SH32 of the subject. The green component decreases as each position approaches position P32 from reference position P31 on the 3D shape SH32 to position P32 on the 3D shape SH32. Furthermore, the green component decreases as each position approaches position P33 from reference position P31 to position P33 on the 3D shape SH32.

[0186] Line L31 indicates the red component of the graphic at each position of the 3D shape SH32. The red component increases as each position approaches position P32 from reference position P31 to position P32. The red component is 0 at each position from reference position P31 to position P33.

[0187] Line L32 indicates the blue component of the graphic at each position of the 3D shape SH32. The blue component increases from reference position P31 to position P33 as each position approaches position P33. The blue component is 0 at each position from reference position P31 to position P32.

[0188] The control unit 50 calculates the value C of each component of the color of the graphic at each position from the reference position P31 to the position P33 by using the following formula (1). Each component of the color of the graphic is red, green, or blue. C = (C1 × (1 - D)) + (C2 × D) (1)

[0189] The value C1 in equation (1) represents the value of each component of the color of the graphic at position P33. The value C2 in equation (1) represents the value of each component of the color of the graphic at reference position P31. The value D in equation (1) represents the ratio of the distance D1 between position P33 and each position to the distance D2 between reference position P31 and position P33. In other words, the value D is expressed by the following equation (2). D=D1 / D2 (2)

[0190] The control unit 50 calculates the value C of each component of the color of the graphic at each position from the reference position P31 to the position P32 by using the formula (1). The value C1 in the formula (1) indicates the value of each component of the color of the graphic at the position P32. The value C2 in the formula (1) indicates the value of each component of the color of the graphic at the reference position P31. The value D in the formula (1) indicates the ratio of the distance D1 between the position P32 and each position to the distance D2 between the reference position P31 and the position P32.

[0191] In the first example in FIG. 22, it is possible to set the color gradation of the graphic in the same way as in the first example in FIG.

[0192] Next, a second example will be described. Line L33 indicates the green component of the graphic at each position on the 3D shape SH33 of the subject. The green component is the same at each position between position P35 on the 3D shape SH33 and position P36 on the 3D shape SH33. The green component decreases as each position approaches position P32 from position P35 to position P32 on the 3D shape SH33. The green component decreases as each position approaches position P33 from position P36 to position P33 on the 3D shape SH33.

[0193] Line L34 indicates the red component of the graphic at each position of the 3D shape SH33 of the subject. The red component increases from reference position P31 to position P35 as each position approaches position P35. The red component is the same at each position between position P35 and position P32. The red component is 0 at each position from reference position P31 to position P33.

[0194] Line L35 indicates the blue component of the graphic at each position on the 3D shape SH33 of the subject. The blue component increases as each position approaches position P36 from reference position P31 to position P36 on the 3D shape SH33. The blue component is the same at each position between position P36 and position P33. The blue component is 0 at each position from reference position P31 to position P32.

[0195] In the second example in FIG. 22, it is possible to set the gradation of the color of the graphic in the same way as in the second example in FIG.

[0196] The color of the graphic at the third position corresponding to the position of slider bar SB32 may be the same as the color (green) of the graphic at the reference position. Similarly, the color of the graphic at the fourth position corresponding to the position of slider bar SB33 may be the same as the color (green) of the graphic at the reference position. Below, an example will be described in which the color of the graphic at each of the third and fourth positions is set to the same color as the color at the reference position.

[0197] The control unit 50 may set the color gradation of the graphic according to the method shown in Fig. 23. Fig. 23 shows an example of a method for setting the color gradation of the graphic to be superimposed on the 3D shape of the subject. The operation panel PN30 shown in Fig. 23 is the same as the operation panel PN30 shown in Fig. 21. A description of the method for setting the color gradation of the graphic according to the settings of the operation panel PN30 will be omitted.

[0198] The control unit 50 displays an operation panel PN32 shown in Fig. 23 on the display 51. The operation panel PN32 includes slider bars SB30 to SB33.

[0199] The control unit 50 sets the color of the graphic at the reference position P36 to green, the color of the graphic at the first position corresponding to the position of the slider bar SB30 to red, the color of the graphic at the second position corresponding to the position of the slider bar SB31 to blue, the color of the graphic at the third position corresponding to the position of the slider bar SB32 to green, and the color of the graphic at the fourth position corresponding to the position of the slider bar SB33 to green.

[0200] 21, the range from reference position P34 to the first position includes partial ranges PR30 and PR31. The control unit 50 sets the color of the graphic at each position in partial range PR30 to green. The control unit 50 sets the color gradation of the graphic in partial range PR31. The color of the graphic changes between green and red depending on the distance between reference position P34 and each position.

[0201] 21, the range from reference position P34 to the second position includes partial ranges PR32 and PR33. The control unit 50 sets the color of the graphic at each position in partial range PR32 to green. The control unit 50 sets the color gradation of the graphic in partial range PR33. The color of the graphic changes between green and blue depending on the distance between reference position P34 and each position.

[0202] The control unit 50 superimposes a graphic having the color set as described above onto the 3D shape SH34 of the subject. The control unit 50 superimposes a green graphic on the point at each of the reference position P31, position P35, and position P36 of the 3D shape SH34. The control unit 50 superimposes a red graphic on the point at position P32 of the 3D shape SH34. The control unit 50 superimposes a blue graphic on the point at position P33 of the 3D shape SH34.

[0203] 23, the partial ranges PR30 and PR34, which are set to green, are considered to be safe areas, and the ranges set to red or blue are considered to be outside the observation range. Meanwhile, the color of the graphics at each position in the partial ranges PR31 and PR33, which are far from the reference position P34, changes depending on the distance between the reference position P34 and each position. In this example, the gradation of the area that the user focuses on when observing the subject is emphasized.

[0204] The control unit 50 may set the color gradation of the graphic according to the method shown in Figures 24(a) and 24(b), which show an example of a method for setting the color gradation of the graphic to be superimposed on the 3D shape of the subject.

[0205] 24(a) will be described. The control unit 50 sets the color of the graphic at reference position P37 to green, the color of the graphic at position P38 to red, and the color of the graphic at position P39 to blue. The control unit 50 sets the color of the graphic at each position from reference position P37 to position P38 according to the distance between reference position P37 and each position.

[0206] The control unit 50 sets the color of the graphic at position P39 to green, and sets the color of the graphic at each position from reference position P37 to position P39 to green. The control unit 50 sets the color of the graphic at position P40 adjacent to position P39 to cyan, and sets the color of the graphic at each position from position P40 to position P41 to cyan. The control unit 50 sets the color of the graphic at position P42 adjacent to position P41 to blue, and sets the color of the graphic at each position from position P42 to position P39 to blue.

[0207] 24(a), the graphic color of the 3D shape SH35 changes smoothly from the reference position P37 to the position P38, while the graphic color of the 3D shape SH35 changes stepwise from the reference position P37 to the position P39.

[0208] The example shown in Figure 24(b) will be described. The same parts as those in the example shown in Figure 24(a) will not be described. The control unit 50 sets the color of the graphic between positions P39 and P40 to black. The control unit 50 also sets the color of the graphic between positions P41 and P42 to black.

[0209] In the example shown in Figure 24(b), the color of the graphic of the 3D shape SH36 changes stepwise from the reference position P37 to the position P39. Also, the boundary between green and cyan is displayed in black, and the boundary between cyan and blue is displayed in black. Therefore, these boundaries are emphasized.

[0210] Each aspect of the present invention may include the following modifications. A first range from the reference surface to the first position includes a first partial range from the reference surface to a third position and a second partial range from the third position to the first position. For example, the first partial range corresponds to partial range PR30 shown in FIG. 21, and the second partial range corresponds to partial range PR31 shown in FIG. 21. The first position is a first distance away from the reference surface in the first region. The third position is a third distance away from the reference surface in the first region, the third distance being smaller than the first distance. The control unit 50 receives partial range information, which is first information indicating the first partial range and the second partial range. The control unit 50 sets a gradation of the display state of the first graphic in the first partial range indicated by the partial range information, and sets a gradation of the display state of the first graphic in the second partial range indicated by the partial range information.

[0211] Each aspect of the present invention may include the following modifications. The second range from the reference surface to the second position includes a third partial range from the reference surface to a fourth position and a fourth partial range from the fourth position to the second position. For example, the third partial range corresponds to partial range PR32 shown in FIG. 21 , and the fourth partial range corresponds to partial range PR33 shown in FIG. 21 . The second position is a second distance away from the reference surface in the second region. The fourth position is a fourth distance away from the reference surface in the second region that is smaller than the second distance. The control unit 50 receives partial range information, which is second information indicating the third partial range and the fourth partial range. The control unit 50 sets the gradation of the display state of the second graphic in the third partial range indicated by the partial range information, and sets the gradation of the display state of the second graphic in the fourth partial range indicated by the partial range information.

[0212] In the third embodiment, the control unit 50 can set the gradation of the graphic display state in any partial range included in the first range or the second range. When a specific distance within the range included in the first range or the second range is important, the endoscope system 1 can visualize the 3D shape so that the location of that distance is clarified.

[0213] (Fourth embodiment) A fourth embodiment of the present invention will be described. In the fourth embodiment, an endoscope system 1 shown in Fig. 1 is used. An endoscope system 1a shown in Fig. 17 or an endoscope system 1b shown in Fig. 18 may also be used.

[0214] In the fourth embodiment, the endoscope system 1 operates in either a first mode or a second mode. In the first mode, the gradation setting of the display state of the first graphic and the gradation setting of the display state of the second graphic are performed independently of each other. The first graphic is superimposed on a 3D shape corresponding to the first region. The second graphic is superimposed on a 3D shape corresponding to the second region. In the second mode, the gradation setting of the display state of the first graphic and the gradation setting of the display state of the second graphic are performed in conjunction with each other. The control unit 50 can switch between the first mode and the second mode.

[0215] The control unit 50 sets one of the first mode and the second mode. The endoscope system 1 operates in the set first mode or second mode. The control unit 50 switches between the first mode and the second mode in response to an instruction from the user.

[0216] 25(a), 25(b), and 25(c) show a first example of a GUI displayed on the display 51. FIG.

[0217] As shown in Fig. 25(a), control unit 50 displays operation panel PN40 on display 51. Operation panel PN40 includes slider bar SB40 and slider bar SB41. Slider bar SB40 and slider bar SB41 are the same as slider bar SB10 and slider bar SB11, respectively, shown in Fig. 7(a) and other figures.

[0218] The control unit 50 sets the color of the graphic at the reference position P40 corresponding to the reference surface to green, sets the color of the graphic at the first position P41 corresponding to the position of the slider bar SB40 to red, and sets the color of the graphic at the second position P42 corresponding to the position of the slider bar SB11 to blue.

[0219] The control unit 50 sets the color gradation of the graphic in a range R40 from the reference position P40 to the first position P41. The color of the graphic changes between green and red depending on the distance between the reference position P40 and each position. The control unit 50 sets the color gradation of the graphic in a range R41 from the reference position P40 to the second position P42. The color of the graphic changes between green and blue depending on the distance between the reference position P40 and each position.

[0220] FIG. 25(b) shows a first example in which the movements of slider bar SB40 and slider bar SB41 are linked to each other. When the second mode is set and the user moves slider bar SB40, slider bar SB41 automatically moves in accordance with the movement of slider bar SB40. Alternatively, when the second mode is set and the user moves slider bar SB41, slider bar SB40 automatically moves in accordance with the movement of slider bar SB41. In the example shown in FIG. 25(b), control unit 50 sets the positions of slider bar SB40 and slider bar SB41 symmetrically with respect to reference position P40.

[0221] In the example shown in FIG. 25(b), when the first reference value (or second reference value) corresponding to slider bar SB40 (or slider bar SB41) decreases, the second reference value (or first reference value) corresponding to slider bar SB41 (or slider bar SB40) increases. Alternatively, when the first reference value (or second reference value) increases, the second reference value (or first reference value) decreases. In other words, the direction of change of the first reference value and the direction of change of the second reference value are different.

[0222] When the first reference value (or the second reference value) decreases, the second reference value (or the first reference value) may decrease. Alternatively, when the first reference value (or the second reference value) increases, the second reference value (or the first reference value) may increase. In other words, the direction of change of the first reference value and the direction of change of the second reference value may be the same.

[0223] FIG. 25(c) shows a second example in which the movements of slider bar SB40 and slider bar SB41 are linked to each other. The reference position P41 corresponding to the reference plane is not the center of the range from the first inspection reference value (2.50 mm) to the second inspection reference value (-7.50 mm). When the user moves slider bar SB41 to the right, the control unit 50 cannot move slider bar SB40 to the right by the same amount as the movement of slider bar SB41. Therefore, the control unit 50 moves slider bar SB40 to the position of the first inspection reference value.

[0224] The control unit 50 determines the amount of change in the distance corresponding to one slider bar according to the amount of change in the distance corresponding to the other slider bar, or determines the rate of change in the distance corresponding to one slider bar according to the rate of change in the distance corresponding to the other slider bar.

[0225] Examples of changes in the first and second reference values ​​will be described below. First, the first example will be described. In the first example, the direction of change in the first reference value and the direction of change in the second reference value are the same, and the control unit 50 determines the amount of change in the distance corresponding to one slider bar according to the amount of change in the distance corresponding to the other slider bar.

[0226] For example, the control unit 50 changes the first reference value from a value corresponding to 10% to a value corresponding to 7%. The control unit 50 also changes the second reference value from a value corresponding to 20% to a value corresponding to 17%. That is, the amount of change in each of the first and second reference values ​​is a value corresponding to 3%. The first and second reference values ​​also decrease.

[0227] Next, a second example will be described. In the second example, the direction of change of the first reference value is different from the direction of change of the second reference value, and the control unit 50 determines the amount of change in the distance corresponding to one slider bar according to the amount of change in the distance corresponding to the other slider bar.

[0228] For example, the control unit 50 changes the first reference value from a value corresponding to 10% to a value corresponding to 7%. The control unit 50 also changes the second reference value from a value corresponding to 20% to a value corresponding to 23%. That is, the amount of change in each of the first and second reference values ​​is a value corresponding to 3%. The first reference value decreases, and the second reference value increases.

[0229] Next, a third example will be described. In the third example, the direction of change of the first reference value and the direction of change of the second reference value are the same, and the control unit 50 determines the rate of change of the distance corresponding to one slider bar according to the rate of change of the distance corresponding to the other slider bar.

[0230] For example, the control unit 50 changes the first reference value from a value corresponding to 10% to a value corresponding to 7%. Also, the control unit 50 changes the second reference value from a value corresponding to 20% to a value corresponding to 14%. In other words, the rate of change of each of the first reference value and the second reference value is 30% of the original value (10% or 20%). Also, the first reference value and the second reference value decrease.

[0231] Next, a fourth example will be described. In the fourth example, the direction of change of the first reference value is different from the direction of change of the second reference value, and the control unit 50 determines the rate of change of the distance corresponding to one slider bar according to the rate of change of the distance corresponding to the other slider bar.

[0232] For example, the control unit 50 changes the first reference value from a value corresponding to 10% to a value corresponding to 7%. The control unit 50 also changes the second reference value from a value corresponding to 20% to a value corresponding to 26%. In other words, the rate of change of each of the first and second reference values ​​is 30% of the original value (10% or 20%). The first reference value also decreases, and the second reference value also increases.

[0233] 26(a) and 26(b) show a second example of the GUI displayed on the display 51. FIG.

[0234] As shown in FIG. 26(a), control unit 50 displays operation panel PN41 on display 51. Operation panel PN41 includes slider bars SB40 to SB43, buttons BT40 and BT41. Slider bars SB40 and SB41 are the same as slider bars SB40 and SB41 shown in FIGS. 26(a) and 26(b), respectively. Slider bars SB42 and SB43 are the same as slider bars SB32 and SB33 shown in FIG. 21, respectively.

[0235] The user moves slider bar SB40 and slider bar SB41. When the user presses button BT40, control unit 50 moves slider bar SB42 to the center between the reference position corresponding to the reference surface and the position of slider bar SB40, as shown in FIG. 26(b). When the user presses button BT41, control unit 50 moves slider bar SB43 to the center between the reference position corresponding to the reference surface and the position of slider bar SB41, as shown in FIG. 26(b).

[0236] Each aspect of the present invention may include the following modifications. The control unit 50 sets one of a first mode and a second mode that are switchable. In the first mode, the setting of the gradation of the display state of the first graphic and the setting of the gradation of the display state of the second graphic are performed independently of each other. In the second mode, the setting of the gradation of the display state of the first graphic and the setting of the gradation of the display state of the second graphic are performed in conjunction with each other.

[0237] In the fourth embodiment, the control unit 50 switches between a first mode and a second mode. In the second mode, the control unit 50 links the setting of the color gradation of the graphic in the range from the reference position to the first position with the setting of the color gradation of the graphic in the range from the reference position to the second position. The user can easily set the color gradation of the graphic by operating one of two slider bars.

[0238] (Fifth embodiment) A fifth embodiment of the present invention will be described. In the fifth embodiment, an endoscope system 1 shown in Fig. 1 is used. An endoscope system 1a shown in Fig. 17 or an endoscope system 1b shown in Fig. 18 may also be used.

[0239] In the fifth embodiment, the control unit 50 enlarges an image of a 3D shape that includes a region from the reference plane to a first position in the first region and a region from the reference plane to a second position in the second region.

[0240] 27(a) and 27(b) show examples of 3D images. 3D image IMG50 shown in FIG. 27(a) includes a 3D shape SH50 to which a graphic is set. A range R50 in the 3D shape SH50 includes an area to which the color gradation of the graphic is set. The control unit 50 generates 3D image IMG51 shown in FIG. 27(b) by removing from 3D image IMG50 a color that is the same as or similar to the color (red) of the graphic at a first position in a first area.

[0241] The control unit 50 identifies the coordinates of each of the left, right, top, and bottom edges of the 3D image IMG51, each of which has a color set. The coordinates of each point in the 3D image IMG51 include an X coordinate and a Y coordinate. The X axis defining the X coordinate points to the right as shown in FIG. 27(b). The Y axis defining the Y coordinate points to the downward as shown in FIG. 27(b). The control unit 50 extracts from the 3D image IMG51 an area including points having coordinates between the X coordinate of the left edge and the X coordinate of the right edge and between the Y coordinate of the top edge and the Y coordinate of the bottom edge. The control unit 50 generates a 3D image by enlarging the extracted area and displays the 3D image on the display 51.

[0242] 28 shows an example of the procedure for processing to identify the coordinates of the left edge. The operation of the endoscope system 1 will be described with reference to FIG.

[0243] The control unit 50 sets the X coordinate held in the volatile memory 56 to the maximum value (step S300). Hereinafter, this X coordinate will be referred to as the held X coordinate.

[0244] After step S300, the control unit 50 sets the Y coordinate to be searched to 0 (step S301). Hereinafter, this Y coordinate will be referred to as the search Y coordinate.

[0245] After step S301, the control unit 50 sets the X coordinate to be searched to 0 (step S302). Hereinafter, this X coordinate will be referred to as the search X coordinate.

[0246] After step S302, the control unit 50 acquires the color of the pixel specified by the search Y coordinate and the search X coordinate from the 3D image of the subject (step S303).

[0247] After step S302, the control unit 50 determines whether the color acquired in step S303 is a color other than the background color (step S304).

[0248] If the control unit 50 determines in step S304 that the color acquired in step S303 is the background color, the control unit 50 executes step S307. If the control unit 50 determines in step S304 that the color acquired in step S303 is a color other than the background color, the control unit 50 determines whether the search X coordinate is smaller than the stored X coordinate (step S305).

[0249] If control unit 50 determines in step S305 that the search X coordinate is equal to or greater than the retained X coordinate, step S307 is executed. If control unit 50 determines in step S305 that the search X coordinate is smaller than the retained X coordinate, control unit 50 sets the retained X coordinate to the search X coordinate. The retained X coordinate retained in volatile memory 56 is updated with the search X coordinate (step S306).

[0250] After step S306, the control unit 50 increments the search X coordinate, that is, increases the search X coordinate by 1 (step S307).

[0251] After step S307, the control unit 50 determines whether the search X coordinate is the same as the width of the 3D image in the horizontal direction. That is, the control unit 50 determines whether the search X coordinate is the same as the X coordinate of the right end of one row of the 3D image (step S308).

[0252] When the control unit 50 determines in step S308 that the search X coordinate is different from the width of the 3D image in the horizontal direction, step S303 is executed. When the control unit 50 determines in step S308 that the search X coordinate is the same as the width of the 3D image in the horizontal direction, the control unit 50 increments the search Y coordinate, that is, increases the search Y coordinate by 1 (step S309).

[0253] After step S309, the control unit 50 determines whether the search Y coordinate is the same as the height of the 3D image in the vertical direction (step S310).

[0254] When the control unit 50 determines in step S310 that the search Y coordinate is different from the height of the 3D image in the vertical direction, step S302 is executed. When the control unit 50 determines in step S310 that the search Y coordinate is the same as the height of the 3D image in the vertical direction, the process shown in FIG. 28 ends.

[0255] The control unit 50 identifies the coordinates of the right end, the top end, and the bottom end by performing the same process as the process shown in Fig. 28. The control unit 50 may reduce the area extracted from the 3D image in the above process.

[0256] Each aspect of the present invention may include the following modifications: The control unit 50 superimposes a first graphic on a 3D shape corresponding to a first region, and superimposes a second graphic on a 3D shape corresponding to a second region. The control unit 50 enlarges the region of the 3D shape on which the first graphic is superimposed and the region of the 3D shape on which the second graphic is superimposed.

[0257] In the fifth embodiment, the area where the graphic color gradation is set is enlarged and the image of the enlarged area is displayed, thereby improving the visibility of the area.

[0258] (Sixth embodiment) A sixth embodiment of the present invention will be described. In the sixth embodiment, the endoscope system 1 shown in Fig. 1 is used. An endoscope system 1a shown in Fig. 17 or an endoscope system 1b shown in Fig. 18 may also be used.

[0259] As described above, the control unit 50 divides the entire area of ​​the 3D image data into a first area and a second area. If the first area or the second area does not contain any points or the first area or the second area contains very few points, the reference plane needs to be set again.

[0260] 29 shows an example of the positional relationship between the 3D shape of the object and the reference plane. A reference plane RS60 is set on the 3D shape SH60 of the object. When a first region R60 and a second region R61 shown in FIG. 29 are set, the first region R60 includes all points of the 3D data, and the second region R61 does not include any points.

[0261] The control unit 50 checks the number of points included in the first area and the number of points included in the second area. If the number of points included in the first area or the second area is smaller than the number previously stored in the nonvolatile memory 57, the control unit 50 displays information on the display 51 prompting the user to re-designate the reference point (reference plane).

[0262] 30(a), 30(b), and 30(c) show examples of information displayed on the display 51. FIG.

[0263] 30(a), a slider bar SB60 for setting the first position in the first region is displayed, but a slider bar for setting the second position in the second region is not displayed. Also, region R62 for displaying the slider bar for setting the second position is displayed in a color (e.g., gray) that indicates that it is disabled.

[0264] In the example shown in Figure 30(b), operation panel PN60 includes input fields IF60 and IF61 that are similar to input fields IF10 and IF11 shown in Figure 15. No numerical values ​​are displayed in input field IF61 for setting the second position.

[0265] In the example shown in FIG. 30(c), the control unit 50 displays a message MS60 on the display 51, urging the user to reset the reference plane, since the first region does not include any points.

[0266] Each aspect of the present invention may include the following modifications: If the number of points included in the first region or the second region is less than a preset number, the control unit 50 outputs information prompting the user to re-designate the reference points on the subject.

[0267] In the sixth embodiment, the endoscope system 1 can avoid setting the reference plane at a position that is not appropriate for setting the first region and the second region.

[0268] (Seventh embodiment) A seventh embodiment of the present invention will be described. In the seventh embodiment, an endoscope system 1 shown in Fig. 1 is used. An endoscope system 1a shown in Fig. 17 or an endoscope system 1b shown in Fig. 18 may also be used.

[0269] In the seventh embodiment, the control unit 50 displays the reference surface and switches the display state of the reference surface.

[0270] First, a first example will be described. Figures 31, 32, and 33 show examples of 3D images displayed on the display 51. The control unit 50 displays a 3D image IMG70 shown in Figure 31 on the display 51. The 3D image IMG70 includes a reference plane RS70 and a 3D shape SH70 of the subject.

[0271] The reference surface RS70 is a cylindrical surface, and a portion of the reference surface RS70 is located in front of the 3D shape SH70. Because a portion of the reference surface RS70 overlaps with the 3D shape SH70, the visibility of the 3D shape SH70 is reduced. In response to an instruction from the user, the control unit 50 switches between a state in which the reference surface is displayed on the display 51 and a state in which the reference surface is not displayed.

[0272] When an instruction to hide the reference plane is input, the control unit 50 accepts the instruction and hides the reference plane in the 3D image. At this time, the control unit 50 displays a 3D image IMG71 shown in Fig. 32 on the display 51. The 3D image IMG71 includes a 3D shape SH70 of the subject. Because the reference plane RS70 shown in Fig. 31 is not displayed, the visibility of the 3D shape SH70 is improved.

[0273] The control unit 50 may hide a portion of the reference plane. For example, the control unit 50 displays a 3D image IMG72 shown in FIG. 33 on the display 51. The 3D image IMG72 includes a reference plane RS70 and a 3D shape SH70 of the subject. The control unit 50 hides a portion of the reference plane RS70 shown in FIG. 31. In other words, the control unit 50 hides the portion that overlaps with the 3D shape SH70. This improves the visibility of the 3D shape SH70.

[0274] The control unit 50 may switch between a 3D image IMG73 shown in Fig. 34 and a 3D image IMG74 shown in Fig. 35. The 3D image IMG73 and the 3D image IMG74 include a reference plane RS71 and a 3D shape SH71 of the subject.

[0275] In the 3D image IMG73 shown in Fig. 34, the reference plane RS71 is a part of the cylindrical surface. In the 3D image IMG74 shown in Fig. 35, the reference plane RS71 is the entire cylindrical surface. This makes it easier for the user to imagine the shape of the subject in areas where three-dimensional image data was not generated.

[0276] Each aspect of the present invention may include the following modifications: The control unit 50 switches between a state in which the entire reference surface is displayed on the display 51 and a state in which at least a part of the reference surface is not displayed.

[0277] In the seventh embodiment, the visibility of the 3D shape of the subject is improved.

[0278] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments and their modifications. Addition, omission, substitution, and other modifications of the configuration are possible within the scope of the spirit of the present invention. Furthermore, the present invention is not limited by the above description, but is limited only by the scope of the appended claims. [Explanation of symbols]

[0279] 1,1a,1b Endoscope System 2 Insertion section 2a Tip 3,3b Scope unit 4,7 Base Unit 5,6 Main unit 10, 10a, 10b Endoscopic device 11 External device 20 Imaging unit 21 Curved section 22 Lighting window 30 Imaging drive circuit 31 Image processing section 32 UD drive unit 33 RL drive unit 34 Bending control section 35 Light source 36 Light source control unit 40,50,60,70 Control section 41, 54, 55 Communications Department 42,56 Volatile memory 43,57 Non-volatile memory 51 Display 52 Touch Panel 53 Operation buttons

Claims

1. The control unit includes: acquiring three-dimensional image data including three-dimensional coordinates of three or more points on the subject calculated based on an endoscopic image of the subject; displaying a three-dimensional shape of the subject on a display based on the three-dimensional image data; setting a reference plane based on one or more points included in the three or more points; Dividing the three-dimensional image data into a first region and a second region using the reference plane as a boundary; superimposing a graphic having a display state set according to the distance between the reference plane and each of the three or more points on the three-dimensional shape; receiving first information indicating a first range in which a gradation of a display state of a first graphic, which is the graphic superimposed on the three-dimensional shape corresponding to the first region, is set; receiving second information indicating a second range in which a gradation of a display state of a second graphic, which is the graphic superimposed on the three-dimensional shape corresponding to the second region, is set; setting the gradation of the display state of the first graphic in the first range indicated by the first information; setting the gradation of the display state of the second graphic in the second range indicated by the second information; The setting of the gradation of the display state of the first graphic and the setting of the gradation of the display state of the second graphic are performed independently of each other. Three-dimensional image display device.

2. The control unit receiving the first information indicating the first range from the reference surface to a first position, the first position being a first distance away from the reference surface in the first region; The second information indicating the second range from the reference surface to a second position is received, and the second position is separated from the reference surface by a second distance in the second region. The three-dimensional image display device according to claim 1 .

3. the first range includes a first subrange from the reference surface to a third position and a second subrange from the third position to the first position, and the third position is separated from the reference surface in the first region by a third distance that is smaller than the first distance; The control unit receiving partial range information, which is the first information indicating the first partial range and the second partial range; setting the gradation of the display state of the first graphic in the first partial range indicated by the partial range information; Setting the gradation of the display state of the first graphic in the second partial range indicated by the partial range information The three-dimensional image display device according to claim 2 .

4. the second range includes a third subrange from the reference surface to a fourth position and a fourth subrange from the fourth position to the second position, and the fourth position is separated from the reference surface in the second region by a fourth distance that is smaller than the second distance; The control unit receiving partial range information, which is the second information indicating the third partial range and the fourth partial range; setting the gradation of the display state of the second graphic in the third partial range indicated by the partial range information; Setting the gradation of the display state of the second graphic in the fourth partial range indicated by the partial range information The three-dimensional image display device according to claim 2 .

5. The control unit superimposes the graphic having a color as the display state on the three-dimensional shape. The three-dimensional image display device according to claim 2 .

6. The control unit superimposing a first color onto the three-dimensional shape as the first graphic at the first location; a second color different from the first color is superimposed on the three-dimensional shape as the second graphic at the second position; The three-dimensional image display device according to claim 5 .

7. The control unit a third color different from the first color and the second color is superimposed on the three-dimensional shape as the graphic on the reference surface; 7. The three-dimensional image display device according to claim 6.

8. The control unit superimposing the first color on the three-dimensional shape as the first graphic in an area in the first region that is a distance greater than the first distance from the reference surface; The second color is superimposed on the three-dimensional shape as the second graphic in the second region in an area that is a distance greater than the second distance from the reference surface.

7. The three-dimensional image display device according to claim 6.

9. The control unit receiving the first information including a numerical value indicating the first position; receiving the second information including a numerical value indicating the second position; The three-dimensional image display device according to claim 2 .

10. The control unit Set the measurement points, A position that is separated from the reference plane by a distance between the reference plane and the measurement point is set as the first position or the second position. The three-dimensional image display device according to claim 2 .

11. The control unit enlarges an area of ​​the three-dimensional shape on which the first graphic is superimposed and an area of ​​the three-dimensional shape on which the second graphic is superimposed. The three-dimensional image display device according to claim 1 .

12. the control unit sets one of a first mode and a second mode that can be switched; in the first mode, the setting of the gradation of the display state of the first graphic and the setting of the gradation of the display state of the second graphic are performed independently of each other; In the second mode, the setting of the gradation of the display state of the first graphic and the setting of the gradation of the display state of the second graphic are executed in conjunction with each other. The three-dimensional image display device according to claim 1 .

13. The control unit Accepting an instruction to designate a point on the object; Set the reference plane based on the point specified by the instruction The three-dimensional image display device according to claim 1 .

14. If the number of points included in the first region or the second region is less than a preset number, the control unit outputs information prompting the user to re-designate points on the object. The three-dimensional image display device according to claim 13.

15. The control unit displaying a graphical user interface on the display, the graphical user interface including a movable first slider and a movable second slider; receiving the first information in response to an operation of the first slider; The second information is received in response to an operation of the second slider. The three-dimensional image display device according to claim 1 .

16. The control unit switches between a state in which the entire reference surface is displayed on the display and a state in which at least a part of the reference surface is not displayed. The three-dimensional image display device according to claim 1 .

17. The control unit acquiring three-dimensional image data including three-dimensional coordinates of three or more points on the subject calculated based on an endoscopic image of the subject; displaying a three-dimensional shape of the subject on a display based on the three-dimensional image data; setting a reference plane based on one or more points included in the three or more points; Dividing the three-dimensional image data into a first region and a second region using the reference plane as a boundary; superimposing a graphic having a display state set according to the distance between the reference plane and each of the three or more points on the three-dimensional shape; receiving first information indicating a first range in which a gradation of a display state of a first graphic, which is the graphic superimposed on the three-dimensional shape corresponding to the first region, is set; receiving second information indicating a second range in which a gradation of a display state of a second graphic, which is the graphic superimposed on the three-dimensional shape corresponding to the second region, is set; setting the gradation of the display state of the first graphic in the first range indicated by the first information; setting the gradation of the display state of the second graphic in the second range indicated by the second information; The setting of the gradation of the display state of the first graphic and the setting of the gradation of the display state of the second graphic are performed independently of each other. 3D image display method.

18. acquiring three-dimensional image data including three-dimensional coordinates of three or more points on the subject calculated based on an endoscopic image of the subject; displaying a three-dimensional shape of the subject on a display based on the three-dimensional image data; setting a reference plane based on one or more points included in the three or more points; Dividing the three-dimensional image data into a first region and a second region using the reference plane as a boundary; superimposing, on the three-dimensional shape, a graphic having a display state set according to the distance between the reference plane and each of the three or more points; receiving first information indicating a first range in which a gradation of a display state of a first graphic, which is the graphic superimposed on the three-dimensional shape corresponding to the first region, is set; receiving second information indicating a second range in which a gradation of a display state of a second graphic, which is the graphic superimposed on the three-dimensional shape corresponding to the second region, is set; setting the gradation of the display state of the first graphic in the first range indicated by the first information; setting the gradation of the display state of the second graphic in the second range indicated by the second information; A program for causing a computer to execute the above, The setting of the gradation of the display state of the first graphic and the setting of the gradation of the display state of the second graphic are performed independently of each other. program.

Images