Measurement device, measurement method, and program

Abstract

This measurement device has a control unit. The control unit estimates a curved surface of a specimen on the basis of three or more points on the specimen in three-dimensional data including the three-dimensional coordinates of the three or more points. The control unit acquires a first reference value indicating the absolute size of the curved surface, and calculates a second reference value indicating the relative size of the curved surface. The control unit measures the relative size of the specimen on the basis of one or more points in the three-dimensional data. The control unit converts the relative size of the specimen into an absolute size of the specimen on the basis of the first reference value and the second reference value.

Description

Measurement device, measurement method, and program

[0001] This application claims priority to Japanese Patent Application No. 2024-120925, filed on July 26, 2024, the contents of which are incorporated herein by reference.

[0002] Industrial endoscope devices are used to inspect the interior of boilers, pipes, aircraft engines, heat exchangers, etc. for abnormalities and corrosion. Applications used for inspection have also been developed. The applications quantify the severity of defects present inside the inspection object, determine whether the defect passes or fails, and generate an inspection report. The applications measure the size of the defects to quantify the severity of the defects.

[0003] The endoscope device disclosed in Patent Document 1 performs surface-based measurement, in which a reference plane is set based on three or more reference positions, and the distance between the reference plane and a measurement point is measured.

[0004] The measurement device disclosed in Patent Document 2 performs scaler measurement. In scaler measurement, two reference positions having known lengths are set, and the distance between the two reference positions is set as a reference value. Then, a measurement position is set, and the size of the object is measured based on the reference value.

[0005] The endoscopic device disclosed in Patent Document 3 superimposes a ruler (scale) for measuring the size of a lesion, etc., on an image of the lesion, etc. The endoscopic device calculates the position and angle of the tip of the endoscope and generates an image of the ruler with multiple concentric circles according to the position and angle.

[0006] Japanese Patent No. 5530225 Japanese Patent Application Laid-Open No. 2020-034442 International Publication No. 2016 / 039292

[0007] The endoscopic device disclosed in Patent Document 3 generates a ruler image according to the position and angle of the tip of the endoscope, but does not properly reflect the shape of the lesion, etc. in the ruler, which can result in a decrease in measurement accuracy.

[0008] An object of the present invention is to provide a measurement device, a measurement method, and a program that can improve measurement accuracy.

[0009] A measurement device according to an aspect of the present invention includes a control unit. The control unit acquires three-dimensional data generated based on two or more two-dimensional images of a subject generated by an endoscope device. The subject has a curved surface. The three-dimensional data includes virtual three-dimensional coordinates of three or more points on the subject. The control unit estimates the curved surface based on the three or more points in the three-dimensional data. The control unit acquires a first reference value indicating the absolute size of the curved surface and calculates a second reference value indicating the relative size of the curved surface. The control unit measures the relative size of the subject based on one or more points in the three-dimensional data. The control unit converts the relative size of the subject to the absolute size of the subject based on the first reference value and the second reference value.

[0010] In the measurement device according to this aspect of the present invention, the control unit may estimate the curved surface based on three or more points included in one or more regions in the three-dimensional data.

[0011] In a measurement device according to this aspect of the present invention, the control unit may estimate the curved surface based on three or more points in the three-dimensional data that correspond to three or more points contained in one or more areas in the two or more two-dimensional images.

[0012] In the measurement apparatus according to an aspect of the present invention, the control unit may estimate the curved surface based on four or more points included in two or more regions including the one or more regions, each of the two or more regions including two or more points.

[0013] In the measurement device according to an aspect of the present invention, the control unit may estimate the curved surface based on four or more points in the three-dimensional data corresponding to four or more points included in two or more regions including the one or more regions, each of the two or more regions including two or more points.

[0014] In the measurement device according to this aspect of the present invention, the curved surface may have all or part of a cylindrical surface, a spherical surface, a torus surface, an elliptical surface, or a paraboloid surface.

[0015] In a measurement device according to an aspect of the present invention, the control unit may estimate two or more curved surfaces including the curved surface, and may calculate the second reference value of one of the two or more curved surfaces.

[0016] In the measurement device according to this aspect of the present invention, the control unit may calculate the second reference value based on the size of an arc or chord of a figure formed by a curve on a cross section of the curved surface.

[0017] In the measurement device according to this aspect of the present invention, the control unit may acquire the first reference value input through an input device.

[0018] In the measurement device according to this aspect of the present invention, the control unit may acquire the first reference value from a storage medium.

[0019] In a measuring device according to this aspect of the present invention, the control unit may calculate a ratio between the first reference value and the second reference value, and may convert the relative size of the subject into an absolute size of the subject based on the ratio.

[0020] In the measurement apparatus according to the present invention, the object may have a cylindrical inner surface and an outer surface. The control unit may acquire a first reference distance or a second reference distance as the first reference value. The first reference distance indicates an absolute size of the inner surface, and the second reference distance indicates an absolute size of the outer surface.

[0021] In the measurement device according to an aspect of the present invention, the control unit may set the inner surface or the outer surface as a reference surface, and may calculate the distance between each of two or more points in the three-dimensional data and the reference surface. The control unit may display a distribution of the distances at the two or more points for which the distances have been calculated on a display.

[0022] In the measurement device according to this aspect of the present invention, the control unit may measure a distance between a point in the three-dimensional data and the curved surface.

[0023] In a measurement device according to an aspect of the present invention, the control unit may estimate a reference plane based on three or more points in the three-dimensional data, and may measure the distance between a point in the three-dimensional data and the reference plane.

[0024] In the measurement device according to this aspect of the present invention, the two or more images may be generated by the endoscope device from two or more viewpoints that are different from each other.

[0025] In the measurement device according to this aspect of the present invention, the endoscope device may have an insertion section with a monocular optical system attached to the tip.

[0026] In the measurement device according to this aspect of the present invention, the control unit may generate the three-dimensional data based on the two or more images.

[0027] In the measurement device according to this aspect of the present invention, the control unit may acquire the three-dimensional data from a storage medium.

[0028] A measurement device according to an aspect of the present invention includes a control unit. The control unit acquires first three-dimensional data generated based on two or more two-dimensional images of a subject generated by an endoscope device. The subject has a curved surface. The first three-dimensional data includes virtual three-dimensional coordinates of three or more points on the subject. The control unit estimates the curved surface based on the three or more points in the first three-dimensional data. The control unit acquires a first reference value indicating the absolute size of the curved surface and calculates a second reference value indicating the relative size of the curved surface. The control unit converts the first three-dimensional data into second three-dimensional data based on the first reference value and the second reference value. The second three-dimensional data includes absolute three-dimensional coordinates of three or more points on the subject. The control unit measures the size of the subject based on one or more points in the second three-dimensional data.

[0029] A measurement method according to an aspect of the present invention includes the following processes: a control unit acquires three-dimensional data generated based on two or more two-dimensional images of a subject generated by an endoscope device; the subject has a curved surface; the three-dimensional data includes virtual three-dimensional coordinates of three or more points on the subject; the control unit estimates the curved surface based on the three or more points in the three-dimensional data; the control unit acquires a first reference value indicating the absolute size of the curved surface, and calculates a second reference value indicating the relative size of the curved surface; the control unit measures the relative size of the subject based on one or more points in the three-dimensional data; and the control unit converts the relative size of the subject to the absolute size of the subject based on the first reference value and the second reference value.

[0030] A program according to an aspect of the present invention causes a computer to perform the following processes: The computer acquires three-dimensional data generated based on two or more two-dimensional images of a subject generated by an endoscope device. The subject has a curved surface. The three-dimensional data includes virtual three-dimensional coordinates of three or more points on the subject. The computer estimates the curved surface based on the three or more points in the three-dimensional data. The computer acquires a first reference value indicating the absolute size of the curved surface and calculates a second reference value indicating the relative size of the curved surface. The computer measures the relative size of the subject based on one or more points in the three-dimensional data. The computer converts the relative size of the subject to the absolute size of the subject based on the first reference value and the second reference value.

[0031] According to the above aspects, the measurement device, the measurement method, and the program can improve the measurement accuracy.

[0032] FIG. 1 is a block diagram showing an example of the configuration of an endoscopic system according to a first embodiment of the present invention. FIG. 2 is a flowchart showing an example of a procedure for measurement processing in the first embodiment of the present invention. FIG. 3 is a diagram showing an example of a screen of a display included in the endoscopic system according to the first embodiment of the present invention. FIG. 4 is a diagram showing an example of a screen of a display included in the endoscopic system according to the first embodiment of the present invention. FIG. 5 is a diagram showing an example of a screen of a display included in the endoscopic system according to the first embodiment of the present invention. FIG. 6 is a diagram showing an example of a screen of a display included in the endoscopic system according to the first embodiment of the present invention. FIG. 7 is a flowchart showing an example of a procedure for processing to set a reference plane in the first embodiment of the present invention. FIG. 8 is a diagram showing an example of a screen of a display included in the endoscopic system according to the first embodiment of the present invention. FIG. 9 is a diagram showing an example of a screen of a display included in the endoscopic system according to the first embodiment of the present invention. Fig. 10 is a block diagram showing an example of the configuration of an endoscope system according to a second modified example of the first embodiment of the present invention. Fig. 11 is a block diagram showing an example of the configuration of an endoscope system according to a third modified example of the first embodiment of the present invention. Fig. 12 is a flowchart showing an example of a processing procedure for setting a reference plane in the second embodiment of the present invention. Fig. 13 is a flowchart showing an example of a measurement processing procedure in the third embodiment of the present invention. Fig. 14 is a diagram showing positions on an object displayed as a thinning map in the third embodiment of the present invention. Fig. 15 is a diagram showing a first example of a thinning map in the third embodiment of the present invention. Fig. 16 is a diagram showing a second example of a thinning map in the third embodiment of the present invention.10 is a diagram showing a third example of a wall-thinning map in the third embodiment of the present invention. FIG. 11 is a flowchart showing an example of a measurement processing procedure in the fourth embodiment of the present invention. FIG. 12 is a flowchart showing an example of a processing procedure for converting the scale of three-dimensional data in the fourth embodiment of the present invention. FIG. 13 is a diagram showing an example of a screen of a display included in an endoscope system according to the fourth embodiment of the present invention. FIG. 14 is a diagram showing an example of a hierarchical structure of folders in the fourth embodiment of the present invention. FIG. 15 is a flowchart showing an example of a processing procedure for measuring the size of an object in the fifth embodiment of the present invention. FIG. 16 is a diagram showing an example of a screen of a display included in an endoscope system according to the fifth embodiment of the present invention. FIG. 17 is a diagram showing an example of a screen of a display included in an endoscope system according to the fifth embodiment of the present invention. FIG. 18 is a diagram showing an example of a screen of a display included in an endoscope system according to the fifth embodiment of the present invention. FIG. 19 is a diagram showing an example of a screen of a display included in an endoscope system according to the fifth embodiment of the present invention. FIG. 10 is a diagram showing an example of a three-dimensional (3D) shape of a blade in a sixth embodiment of the present invention. FIG. 11 is a diagram showing an example of a 3D shape of a blade in a sixth embodiment of the present invention. FIG. 12 is a diagram showing an example of a 3D shape of a blade in a sixth embodiment of the present invention. FIG. 13 is a diagram showing an example of a 3D shape of a blade in a sixth embodiment of the present invention. FIG. 14 is a diagram showing an example of a 3D shape of a blade in a sixth embodiment of the present invention. FIG. 15 is a diagram showing an example of an edge extracted from the 3D shape of a blade in a sixth embodiment of the present invention.

[0033] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the accompanying drawings. An endoscope system will be described below as an example of a measurement device.

[0034] First Embodiment Fig. 1 shows an example of the configuration of an endoscope system 1 according to a first embodiment of the present invention. The endoscope system 1 shown in Fig. 1 has an insertion section 2, a scope unit 3, a base unit 4, and a main unit 5. The insertion section 2, the scope unit 3, and the base unit 4 constitute an endoscope device 10. The main unit 5 is an operation device.

[0035] The insertion section 2 is inserted into the inside of the subject to be observed. The subject is an industrial product. The insertion section 2 is a long, thin tube that is bendable. A user performs an insertion operation to insert the insertion section 2 into the subject. An optical adapter is attached to the tip of the insertion section 2. The insertion section 2 acquires an optical image of the inside of the subject. The insertion section 2 has an imaging section 20, a bending section 21, and an illumination window 22.

[0036] The imaging unit 20 is disposed in the distal end portion 2a including the distal end of the insertion portion 2. The imaging unit 20 is an image sensor such as a CCD (Charge-Coupled Device) sensor or a CMOS (Complementary Metal-Oxide-Semiconductor) sensor. The imaging unit 20 generates an image based on an optical image acquired by the insertion portion 2. The image generated by the imaging unit 20 is output to the scope unit 3.

[0037] The bending section 21 bends the insertion section 2 in the upward (U), downward (D), leftward (L), or rightward (R) direction. Alternatively, the bending section 21 bends the insertion section 2 in the upward-left (UL), upward-right (UR), downward-left (DL), or downward-right (DR) direction.

[0038] Illumination light is generated by a light source 35 included in the scope unit 3 and output to the distal end portion 2a through a light guide disposed in the insertion portion 2. The illumination light is irradiated from the illumination window 22 into the inside of the subject.

[0039] The scope unit 3 has an imaging drive circuit 30, an image processing unit 31, a UD drive unit 32, an RL drive unit 33, a bending control unit 34, a light source 35, and a light source control unit 36. The base unit 4 has a control unit 40, a communication unit 41, a volatile memory 42, and a non-volatile memory 43.

[0040] The imaging drive circuit 30 controls the imaging unit 20 and outputs the image output from the imaging unit 20 to the image processing unit 31. The image processing unit 31 performs image processing such as noise reduction on the image output from the imaging unit 20 and outputs the image to the control unit 40.

[0041] The UD driving unit 32 is connected to a UD bending wire for bending the bending portion 21 in the U direction or the D direction. The UD driving unit 32 has a motor, and bends the bending portion 21 in the U direction or the D direction by pulling the UD bending wire. The RL driving unit 33 is connected to an RL bending wire for bending the bending portion 21 in the R direction or the L direction. The RL driving unit 33 has a motor, and bends the bending portion 21 in the R direction or the L direction by pulling the RL bending wire. The bending control unit 34 controls the UD driving unit 32 and the RL driving unit 33.

[0042] The UD drive unit 32 and the RL drive unit 33 can operate simultaneously. For example, the UD drive unit 32 and the RL drive unit 33 can bend the bending portion 21 in the UL direction.

[0043] The light source 35 is a light-emitting diode (LED) or the like, and generates illumination light. The illumination light is output from the light source 35 to a light guide. The light source control unit 36 ​​controls the light source 35.

[0044] The control unit 40 controls each unit of the scope unit 3 and the base unit 4. At least one of the control unit 40, the image processing unit 31, the bending control unit 34, and the light source control unit 36 ​​may be configured with at least one of a processor and a logic circuit. For example, the processor is at least one of a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and a GPU (Graphics Processing Unit). For example, the logic circuit is at least one of an ASIC (Application Specific Integrated Circuit) and an FPGA (Field-Programmable Gate Array). At least one of the control unit 40, the image processing unit 31, the bending control unit 34, and the light source control unit 36 ​​may include one or more processors. At least one of the control unit 40, the image processing unit 31, the bending control unit 34, and the light source control unit 36 ​​may include one or more logic circuits.

[0045] The computer of the endoscope system 1 may load a program and execute the loaded program. The program includes instructions that define the operation of at least one of the control unit 40, the image processing unit 31, the bending control unit 34, and the light source control unit 36. In other words, at least one function of the control unit 40, the image processing unit 31, the bending control unit 34, and the light source control unit 36 ​​may be realized by software.

[0046] The above program may be provided by a "computer-readable recording medium" such as a flash memory. The program may be transmitted from a computer storing the program to the endoscope system 1 via a transmission medium or by transmission waves in the transmission medium. The "transmission medium" that transmits the program is a medium that has the function of transmitting information. Media that have the function of transmitting information include networks (communication networks) such as the Internet and communication lines (communication lines) such as telephone lines. The above program may realize some of the functions described above. Furthermore, the above program may be a difference file (difference program). The functions described above may be realized by combining a program already recorded on the computer with the difference program.

[0047] The communication unit 41 has a communication circuit and performs wired or wireless communication with the main unit 5 for curvature control and the like. The communication unit 41 transmits images generated by the imaging unit 20 to the main unit 5. The volatile memory 42 is a RAM (Random Access Memory) or a DRAM (Dynamic RAM), for example. The volatile memory 42 stores various information processed by the control unit 40. The non-volatile memory 43 is a SRAM (Static RAM), an EPROM (Erasable Programmable Read-Only Memory), an EEPROM (Electrically Erasable Programmable ROM), or a flash memory. The non-volatile memory 43 may be detachable from the base unit 4. The non-volatile memory 43 stores images generated by the imaging unit 20 and various information processed by the control unit 40 .

[0048] The main unit 5 has a control unit 50, a display 51, a touch panel 52, operation buttons 53, a communication unit 54, a communication unit 55, a volatile memory 56, and a non-volatile memory 57. The main unit 5 may be an information terminal such as a smartphone or a tablet terminal.

[0049] The control unit 50 controls each part of the main unit 5. The control unit 50 may be composed of at least one of a processor and a logic circuit. The control unit 50 may include one or more processors. The control unit 50 may include one or more logic circuits. The computer of the endoscope system 1 may load a program and execute the loaded program. The program includes instructions that define the operation of the control unit 50. In other words, the functions of the control unit 50 may be realized by software. The program that realizes the functions of the control unit 50 may be realized in the same way as the program that realizes the functions of the control unit 40, etc.

[0050] The display 51 is a monitor such as an LCD (Liquid Crystal Display). The display 51 displays an image generated by the imaging unit 20. The touch panel 52 accepts operations for inputting information necessary for controlling the endoscope system 1. The touch panel 52 is arranged on the screen of the display 51. By operating the touch panel 52, the user can input to the endoscope system 1 instructions for changing the settings of the endoscope system 1, instructions necessary for operating the endoscope system 1, and the like.

[0051] The operation button 53 accepts various instructions from the user. By pressing the operation button 53, the user can input instructions regarding power or lighting to the endoscope system 1. The communication unit 54 performs wired or wireless communication with the base unit 4 for bending control and the like. The communication unit 54 receives images generated by the imaging unit 20 from the base unit 4. The communication unit 55 performs wired or wireless communication with the external device 11. The external device 11 is a remote control, a keyboard, a mouse, or the like.

[0052] A monocular optical adapter used for normal observation is attached to the tip of the insertion section 2. The imaging section 20 generates two or more two-dimensional (2D) images based on an optical image formed through the monocular optical adapter. The monocular optical adapter and the imaging section 20 form a monocular camera with one field of view.

[0053] The imaging unit 20 continuously generates images. That is, the imaging unit 20 generates images of each frame corresponding to a moving image. A moving image includes two or more frames. Each frame is made up of an image generated by the imaging unit 20. The two or more frames included in the moving image are associated with each other by a timestamp (time code). When the imaging unit 20 finishes capturing images, a moving image file including the moving image is recorded in the non-volatile memory 57.

[0054] At least a portion of the surface of the object is curved. For example, the object is a pipe having a cylindrical surface. The control unit 50 executes a measurement process to measure the size of the object. In the measurement process, the control unit 50 executes a 3D reconstruction process to generate 3D data representing the three-dimensional (3D) shape of the object, and measures the size of the object using the 3D data.

[0055] 2 shows an example of the procedure of the measurement process. The operation of the endoscope system 1 will be described with reference to FIG.

[0056] When the measurement process is started, the control unit 50 reads the video file from the nonvolatile memory 57 (step S100).

[0057] After step S100, the user plays back the video and determines two or more frames of interest that include the measurement target. The user operates the touch panel 52 to input information indicating the frames of interest to the endoscope system 1. The control unit 50 accepts the information input by the user (step S101).

[0058] 3 and 4 show examples of the screen of the display 51. The display 51 has a screen SC1. When the measurement process is started, the control unit 50 displays a frame FR1, buttons BT1 to BT5, and a seek bar SB1 shown in FIG. 3 on the screen SC1.

[0059] The user operates buttons BT1 to BT5 by touching the touch panel 52. The user operates button BT1 to load a video file containing the video to be played. The user operates button BT2 to play the video. After button BT2 is pressed, frame FR1 of the video is displayed. The user operates button BT3 to pause the video playback. The user operates button BT4 to fast-forward the video. The user operates button BT5 to rewind the video.

[0060] The user touches the touch panel 52 to specify a start frame at which the 3D reconstruction process will begin and an end frame at which the 3D reconstruction process will end. Two or more frames from the start frame to the end frame constitute frames of interest. The control unit 50 may automatically specify the start frame and the end frame. For example, the control unit 50 may detect a section of a video that shows an abnormality such as a scratch. The section includes two or more frames of the video. The control unit 50 may specify the first frame of the section as the start frame and the last frame of the section as the end frame.

[0061] Only one of the start frame and the end frame may be specified by the user. Alternatively, only one of the start frame and the end frame may be specified automatically. One of the two or more frames of interest may be specified by the user, and the control unit 50 may specify one or more frames of interest other than the frame of interest specified by the user. The method of setting the section including the frames used in the 3D reconstruction process is not limited to the above example.

[0062] When information indicating two or more frames of interest is input to the endoscope system 1, the control unit 50 displays the frame FR1, buttons BT1 to BT6, and seek bar SB1 shown in Fig. 4 on the screen SC1 of the display 51. The same parts as those shown in Fig. 3 will not be described.

[0063] The user presses button BT6 to start the 3D reconstruction process. When button BT6 is pressed, the control unit 50 executes the 3D reconstruction process using two or more frames of interest to generate 3D data. For example, the control unit 50 generates the 3D data by using the method disclosed in Japanese Patent Application Laid-Open No. 2020-12635 (step S102).

[0064] The 3D data includes 3D coordinates of two or more points (3D point cloud) of the subject, camera coordinates, and posture information. The 3D data may also include meshes, which are surfaces having the 3D point cloud as vertices, and mesh polygon data, which is a collection of texture information associated with the meshes.

[0065] The 3D coordinates are defined in a 3D space corresponding to the real space. The camera coordinates indicate the 3D coordinates of the camera that captured each of the two or more images, and are associated with each of the two or more images. The camera coordinates are the 3D coordinates of the viewpoint when each image was captured, and indicate the position of the camera. For example, the camera coordinates indicate the 3D coordinates of the observation optical system of the camera. The attitude information indicates the attitude of the camera that captured each of the two or more images, and is associated with each of the two or more images. For example, the attitude information indicates the attitude of the observation optical system of the camera.

[0066] 3D data indicates a relative shape without the dimension of length. Although the 3D data includes the 3D coordinates of each point, the relative distance calculated using the 3D coordinates is different from the absolute size of the object.

[0067] The 3D data generated in step S102 is stored in the non-volatile memory 57. The control unit 50 may obtain from the non-volatile memory 57 3D data generated in an examination that was previously performed.

[0068] After step S102, the control unit 50 displays the 3D shape of the subject on the display 51 based on the 3D data (step S103).

[0069] After the button BT6 shown in Fig. 4 is pressed, the control unit 50 displays the 3D shape SH1, the button BT1, the button BT6, and the button BT7 shown in Fig. 5 on the screen SC1 of the display 51. The same parts as those shown in Fig. 4 will not be described.

[0070] The 3D shape SH1 indicates the 3D shape of the subject reconstructed in the 3D reconstruction process. The user may input an instruction to rotate, enlarge, reduce, or translate the 3D shape SH1 by operating the touch panel 52. The control unit 50 may accept the instruction. When an instruction to rotate the 3D image is accepted, the control unit 50 may rotate the 3D shape SH1. When an instruction to enlarge or reduce the 3D image is accepted, the control unit 50 may enlarge or reduce the 3D shape SH1. When an instruction to translate the 3D image is accepted, the control unit 50 may move the 3D shape SH1 in any direction without changing the orientation of the 3D shape SH1.

[0071] The user presses button BT7 to set a reference plane on which a reference position (reference point) that defines a reference length in the 3D data is set. When button BT7 is pressed, the control unit 50 sets the reference plane in the 3D space based on three or more reference points (step S104).

[0072] 6 shows an example of the procedure of the process executed in step S104. The operation of the endoscope system 1 will be described with reference to FIG.

[0073] The control unit 50 sets a variable n for managing the number of reference points to 1 (step S200).

[0074] The user operates the touch panel 52 to set a reference point on the 3D shape displayed on the display 51, and inputs the reference point to the endoscope system 1. For example, the user touches the position of the reference point. The control unit 50 accepts the reference point input by the user and sets the reference point on the 3D shape (step S201). The reference point is included in three or more points in the 3D data. Information about the reference point is stored in the volatile memory 56. The reference point set in step S201 is treated as the nth reference point.

[0075] After step S201, the control unit 50 selects an area near the nth reference point in the 3D shape displayed on the display 51 (step S202). For example, the control unit 50 selects a spherical area centered on the reference point in step S202. After step S202, the control unit 50 displays the area selected in step S202 on the 3D shape (step S203).

[0076] After step S203, the control unit 50 estimates a curved surface that approximates the surface of the 3D shape of the subject by using the 3D coordinates of the points included in the region selected in step S202 (step S204), and displays the curved surface on the display 51 (step S205).

[0077] After step S205, the control unit 50 determines whether the curved surface has been accurately estimated (step S206).

[0078] Step S206 will now be described in detail. For example, the user checks the curved surface displayed on the display 51. The user determines the degree of coincidence between the surface of the subject and the curved surface, and inputs information indicating the determination result to the endoscope system 1. The control unit 50 determines whether the curved surface has been accurately estimated based on the information input by the user.

[0079] When the control unit 50 determines in step S206 that the curved surface has been accurately estimated, the control unit 50 sets the curved surface as a reference surface (step S207). Information about the reference surface is stored in the volatile memory 56. When step S207 is executed, the process shown in Fig. 6, i.e., step S104 shown in Fig. 2, ends.

[0080] If the control unit 50 determines in step S206 that the curved surface has not been accurately estimated, the control unit 50 increments the variable n by 1 (step S208). After step S208, step S201 is executed. After step S208, the control unit 50 may change the positions of the measurement points that have already been set.

[0081] When steps S201 to S204 are executed two or more times, the control unit 50 sets two or more reference points and selects two or more regions. The control unit 50 estimates a curved surface based on the 3D coordinates of three or more points, including one or more points included in each of the two or more regions. The control unit 50 may estimate a curved surface based on the 3D coordinates of four or more points, including two or more points included in each of the two or more regions.

[0082] When step S201 is executed three times and three reference points are set, the control unit 50 displays the 3D shape SH1, button BT1, button BT6, and button BT7 shown in Fig. 7 on the screen SC1 of the display 51. The same parts as those shown in Fig. 5 will not be described.

[0083] The control unit 50 displays regions R1 to R3 corresponding to three reference points on the 3D shape SH1. Region R1 includes a first reference point, region R2 includes a second reference point, and region R3 includes a third reference point. In the example shown in Fig. 7, the control unit 50 estimates a curved surface based on the 3D coordinates of six or more points, including two or more points in each region.

[0084] The control unit 50 may execute the following process. The control unit 50 displays at least one of the two or more images used to generate the 3D data on the display 51. The user inputs a reference point in the image displayed on the display 51 to the endoscope system 1. Positions on the image used to generate the 3D image data and points included in the 3D data are associated with each other. The control unit 50 identifies a point in the 3D data that corresponds to the reference point in the image displayed on the display 51. The control unit 50 uses the identified point as the reference point in step S201.

[0085] 2 again, the operation of the endoscope system 1 will be described. After step S104, the control unit 50 sets a known reference length of the 3D shape of the object in the 3D data (step S105). Information about the reference length is stored in the volatile memory 56. For example, if the object is a pipe, the reference length is the radius or diameter of the cylindrical surface that constitutes the pipe.

[0086] The control unit 50 displays the 3D shape SH1, button BT1, button BT6, button BT7, and window W1 shown in Fig. 8 on the screen SC1 of the display 51. Portions that are the same as those shown in Fig. 5 will not be described.

[0087] The user operates the touch panel 52 or the like to input the reference length to the endoscope system 1. The control unit 50 receives the reference length input by the user and sets the reference length.

[0088] After step S105, the control unit 50 displays the reference plane set in step S207 and the reference length set in step S105 on the display 51 (step S106).

[0089] After step S106, the control unit 50 measures the length on the 3D data corresponding to the reference length. For example, if the object is a pipe, the control unit 50 calculates the radius or diameter of the reference surface set in step S104 (step S107).

[0090] After step S107, the control unit 50 calculates the ratio between the reference length set in step S105 and the length calculated in step S107. The control unit 50 uses the ratio as a scale factor to convert the scale of the 3D data (step S108).

[0091] For example, if the reference length is X and the length in the 3D data corresponding to the reference length is Y, the scale factor is X / Y. In step S108, the control unit 50 converts the 3D data into 3D data having an absolute length dimension in a 3D space having three preset coordinate axes. This 3D space corresponds to real space. After step S108 is executed, the 3D coordinates in the 3D data are converted into absolute 3D coordinates. The 3D data with the converted scale is stored in the non-volatile memory 57.

[0092] After step S108, the control unit 50 measures the size of the subject based on one or more points in the 3D data (step S109). When step S109 is executed, the measurement process shown in FIG.

[0093] 9 shows an example of the procedure of the process executed in step S109. The operation of the endoscope system 1 will be described with reference to FIG.

[0094] The control unit 50 refers to the measurement mode stored in the volatile memory 56. The measurement mode indicates curved surface-based measurement, plane-based measurement, or two-point distance measurement. The curved surface-based measurement is a mode for measuring the 3D distance from a measurement point to a point where a straight line that includes the measurement point and is perpendicular to the reference curved surface intersects with the reference curved surface. The plane-based measurement is a mode for measuring the 3D distance from a measurement point to a point where a straight line that includes the measurement point and is perpendicular to the reference plane intersects with the reference plane. The two-point distance measurement is a mode for measuring the 3D distance between two measurement points. The control unit 50 determines whether the measurement mode is curved surface-based measurement (step S300).

[0095] When the control unit 50 determines in step S300 that the measurement mode is curved surface reference measurement, the control unit 50 sets a measurement point (step S301). The measurement point is included in three or more points in the 3D data. The measurement point may be a point not included in the 3D data. When the measurement point is not included in the 3D data, the control unit 50 calculates the 3D coordinates of the measurement point based on the 3D coordinates of points included in the 3D data. Information about the measurement point is stored in the volatile memory 56.

[0096] For example, the user operates the touch panel 52 to set a measurement point and inputs the measurement point to the endoscope system 1. For example, the user touches the position of the measurement point on the 3D shape displayed on the display 51. The control unit 50 accepts the measurement point input by the user and sets the measurement point on the 3D shape. The 3D coordinates of the measurement point may be the 3D coordinates of the point closest to the position touched by the user.

[0097] After step S301, the control unit 50 calculates the 3D distance from the reference curved surface to the measurement point (step S302). The reference curved surface is the reference surface set in step S207. Because the reference surface for converting the scale of the 3D data is used as the reference curved surface in the curved surface-reference measurement, the endoscope system 1 does not need to calculate a new reference curved surface.

[0098] After step S302, the control unit 50 displays the measurement results on the display 51 (step S303).

[0099] After step S303, the control unit 50 determines whether to continue the measurement (step S304). The control unit 50 may determine to continue the measurement until the user inputs information indicating that the measurement is to be ended into the endoscope system 1. When the user inputs information indicating that the measurement is to be ended into the endoscope system 1, the control unit 50 may determine not to continue the measurement.

[0100] When the control unit 50 determines in step S304 that the measurement is to be continued, step S300 is executed. When the control unit 50 determines in step S304 that the measurement is not to be continued, the process shown in FIG.

[0101] When the control unit 50 determines in step S300 that the measurement mode is not curved surface reference measurement, the control unit 50 determines whether the measurement mode is flat surface reference measurement (step S305).

[0102] When the control unit 50 determines in step S305 that the measurement mode is plane-reference measurement, the control unit 50 sets three reference points (step S306). The three reference points are included in three or more points in the 3D data. Information on the three reference points is stored in the volatile memory 56.

[0103] After step S306, the control unit 50 calculates a reference plane that passes through the three reference points set in step S306 (step S307), and then displays the reference plane on the display 51 (step S308).

[0104] After step S308, the control unit 50 sets a measurement point (step S309). The measurement point is included in three or more points in the 3D data. The measurement point may be a point that is not included in the 3D data. When the measurement point is not included in the 3D data, the control unit 50 calculates the 3D coordinates of the measurement point based on the 3D coordinates of points that are included in the 3D data. The information about the measurement point is stored in the volatile memory 56. The method for setting the measurement point in step S309 is the same as the method for setting the measurement point in step S301.

[0105] After step S309, the control unit 50 calculates the 3D distance from the reference plane to the measurement point (step S310). After step S310, step S303 is executed.

[0106] When the control unit 50 determines in step S305 that the measurement mode is not plane-reference measurement, the control unit 50 sets two measurement points to perform two-point distance measurement (steps S311 and S312). The two measurement points are included in three or more points in the 3D data. At least one of the two measurement points may be a point that is not included in the 3D data. Information on the two measurement points is stored in the volatile memory 56. The method for setting the measurement points in steps S311 and S312 is the same as the method for setting the measurement points in step S301.

[0107] After step S312, the control unit 50 calculates the 3D distance between the two measurement points (step S313). After step S313, step S303 is executed.

[0108] Figures 10 to 12 show examples of the screen of the display 51 in the process shown in Figure 9. The control unit 50 displays the 3D shape SH1, button BT1, and buttons BT6 to BT8 shown in Figures 10 to 12 on the screen SC1 of the display 51. The same parts as those shown in Figure 5 will not be described.

[0109] The user presses the button BT8 to start measurement. When the button BT8 is pressed, the control unit 50 executes the process shown in FIG.

[0110] When the curved surface reference measurement is performed, the control unit 50 sets the measurement point MP1 shown in Fig. 11 on the 3D shape SH1 and displays the measurement point MP1 on the 3D shape SH1. The control unit 50 calculates the 3D distance from the reference curved surface to the measurement point MP1 and displays the measurement result MR1 indicating the 3D distance on the display 51.

[0111] 12, after the curved surface-reference measurement is performed, the plane-reference measurement is performed. Therefore, the control unit 50 displays the measurement result MR1 of the curved surface-reference measurement on the display 51. In the plane-reference measurement, the control unit 50 sets reference points RP1 to RP3 on the 3D shape SH1 and displays the reference points RP1 to RP3 on the 3D shape SH1. In the plane-reference measurement, the control unit 50 sets measurement point MP2 on the 3D shape SH1 and displays measurement point MP2 on the 3D shape SH1. In the plane-reference measurement, the control unit 50 calculates the 3D distance from the reference plane to measurement point MP2 and displays the measurement result MR2 indicating the 3D distance on the display 51.

[0112] The measurement points in the curved surface reference measurement or the flat surface reference measurement are set at, for example, an abnormal part in the object to be measured. The abnormal part is a concave or convex part.

[0113] 13A and 13B show examples of an abnormality occurring in a test specimen. The test specimen SB1 shown in FIGS. 13A and 13B is a pipe. The test specimen SB1 has an abnormality AP1. The abnormality AP1 is a convex portion occurring on the inner surface of the test specimen SB1. The control unit 50 sets a reference plane that approximates the inner surface of the test specimen SB1, and also sets a measurement point MP3. For example, the measurement point MP3 is the highest point of the abnormality AP1. The control unit 50 calculates the distance D1 between the reference plane and the measurement point MP3.

[0114] As described above, the control unit 50 estimates the curved surface of the object and sets a reference length on that curved surface. The control unit 50 calculates a length on the 3D data that corresponds to the reference length, and converts the scale of the 3D data based on that length and the reference length. The control unit 50 performs measurement by using the 3D data with the converted scale. The length corresponding to the size of the curved surface in the 3D data (e.g., the radius or diameter of a cylindrical surface) is reflected in the scale of the 3D data. In other words, the scale of the 3D data is converted according to the shape of the object. This improves the accuracy of the scale conversion and the accuracy of the measurement results.

[0115] The curved surface of the object in the first embodiment may be a cylindrical surface, a spherical surface, a torus surface, an elliptical surface, a paraboloid surface, or a free-form surface. The object in the first embodiment does not need to have the entirety of such a curved surface. For example, the object in the first embodiment may have a part of a spherical surface.

[0116] 14A and 14B show examples of objects having curved surfaces other than cylindrical surfaces.

[0117] 14A shows an example of an object to be inspected that includes a part of a torus surface. The object to be inspected SB2 shown in FIG. 14A is a turbine blade. A root portion PT1 (Fillet Radius) of the object to be inspected SB2 can be approximated by a torus surface TS1.

[0118] Figure 14B shows an example of a test object that includes a portion of a spherical surface. There is industrial equipment that has a hemisphere, a sphere, or a spherical shell, and such curved surfaces can be approximated by a spherical surface. The test object SB3 shown in Figure 14B has a concave portion, which can be approximated by a spherical surface SS1 with a radius r.

[0119] The control unit 50 may select a cylindrical surface, a spherical surface, a torus surface, an elliptical surface, a paraboloid surface, or a free-form surface in accordance with an instruction from the user, and may estimate the selected surface.

[0120] The control unit 50 may estimate two or more curved surfaces from among a cylindrical surface, a spherical surface, a torus surface, an elliptical surface, a paraboloid surface, and a free-form surface, and may set one of the two or more curved surfaces as a reference surface. For example, the control unit 50 displays the two or more curved surfaces on the display 51. The user operates the touch panel 52 or the like to input an instruction to the endoscope system 1 to select one of the two or more curved surfaces. The control unit 50 accepts the instruction and sets the curved surface selected by the user as the reference surface. Alternatively, the control unit 50 sets the curved surface with the smallest error among the two or more curved surfaces as the reference surface.

[0121] By selecting an appropriate approximation model according to the shape of the curved surface of the subject, the endoscope system 1 can use a reference plane that is suitable for the actual shape of the subject, thereby reducing errors in the measurement results.

[0122] The control unit 50 may change the size of the region in step S202 in accordance with an instruction from the user. The control unit 50 may remove abnormal points included in the region in step S202.

[0123] The reference length may be set in advance before the test is performed. The reference length may be stored in advance in the nonvolatile memory 57. The control unit 50 may obtain the reference length from the nonvolatile memory 57 in step S105.

[0124] Two or more candidates for the reference length may be prepared in advance, and the control unit 50 may select one of the two or more candidates in step S105 in accordance with an instruction from the user.

[0125] When curved surface-based measurement and flat surface-based measurement are performed, two types of reference planes are set. The control unit 50 may calculate the distance between each reference plane and each point included in the 3D data. The control unit 50 may generate a color map by assigning a color to each point according to the distance, and may display the color map on the display 51. The control unit 50 may switch between the two types of color maps displayed on the display 51 in accordance with an instruction from the user.

[0126] Each aspect of the measurement device of the present invention includes a control unit 50. The control unit 50 acquires first 3D data generated based on two or more 2D images of the object generated by the endoscope device 10. The object has a curved surface (e.g., a cylindrical surface). The first 3D data includes virtual 3D coordinates of three or more points on the object. The control unit 50 estimates the curved surface based on the three or more points in the first 3D data. The control unit 50 acquires a first reference value indicating the absolute size of the curved surface and calculates a second reference value indicating the relative size of the curved surface. The control unit 50 converts the first 3D data into second 3D data based on the first and second reference values. The second 3D data includes absolute 3D coordinates of three or more points on the object. The control unit 50 measures the size of the object based on one or more points in the second 3D data.

[0127] The measurement method of each aspect of the present invention includes first to sixth steps. In a first step (step S102), the control unit 50 acquires first 3D data generated based on two or more 2D images of the object generated by the endoscope device 10. In a second step (step S104), the control unit 50 estimates a curved surface based on three or more points in the first 3D data. In a third step (step S105), the control unit 50 acquires a first reference value indicating the absolute size of the curved surface. In a fourth step (step S107), the control unit 50 calculates a second reference value indicating the relative size of the curved surface. In a fifth step (step S108), the control unit 50 converts the first 3D data into second 3D data based on the first and second reference values. In a sixth step (step S109), the control unit 50 measures the size of the object based on one or more points in the second 3D data.

[0128] The program according to each aspect of the present invention causes a computer to execute the first to sixth steps described above.

[0129] Each aspect of the present invention may include the following variations: The control unit 50 estimates a curved surface based on three or more points included in one or more regions (e.g., regions R1 to R3) in the 3D data, and the 3D coordinates of one or more points in each region are used to estimate the curved surface.

[0130] Each aspect of the present invention may include the following modifications: The control unit 50 estimates a curved surface based on four or more points included in two or more regions including the one or more regions described above, and each of the two or more regions includes two or more points.

[0131] Each aspect of the present invention may include the following modifications: The control unit 50 estimates a curved surface based on three or more points in the 3D data that correspond to three or more points included in one or more regions in two or more 2D images.

[0132] Each aspect of the present invention may include the following modifications: The control unit 50 estimates a curved surface based on four or more points in the 3D data corresponding to four or more points included in two or more regions including the one or more regions described above, each of the two or more regions including two or more points.

[0133] Each aspect of the present invention may include the following variations: The curved surface of the object has all or part of a cylindrical surface, a spherical surface, a torus surface, an elliptical surface, or a paraboloid surface.

[0134] Each aspect of the present invention may include the following modifications: The control unit 50 estimates two or more curved surfaces, and calculates a second reference value of one of the two or more curved surfaces.

[0135] Each aspect of the present invention may include the following modifications: The control unit 50 acquires a first reference value input through an input device, such as a touch panel 52.

[0136] Each aspect of the present invention may include the following modifications: The control unit 50 acquires the first reference value from the nonvolatile memory 57 (storage medium).

[0137] Each aspect of the present invention may include the following modifications: The control unit 50 measures the distance between a point in the 3D data and a curved surface, which corresponds to curved surface-based measurement.

[0138] Each aspect of the present invention may include the following modifications: The control unit 50 estimates a reference plane based on three or more points in the 3D data, and measures the distance between the points in the 3D data and the reference plane. This corresponds to curved surface-based measurement or planar surface-based measurement.

[0139] Each aspect of the present invention may include the following modifications: The two or more images used to generate the 3D data are generated by the endoscope device 10 from two or more viewpoints that are different from each other.

[0140] Each aspect of the present invention may include the following modifications: The endoscope device 10 has an insertion section 2 with a monocular optical adapter (monocular optical system) attached to the tip.

[0141] Each aspect of the present invention may include the following modifications: The control unit 50 generates 3D data based on two or more images generated by the endoscope device 10 .

[0142] Each aspect of the present invention may include the following modifications: The control unit 50 acquires 3D data from the non-volatile memory 57 (storage medium).

[0143] As described above, the control unit 50 estimates the curved surface of the object and obtains a reference value indicating the absolute size of the curved surface. The control unit 50 calculates the relative size of the curved surface and converts the scale of the 3D data based on the reference value and the relative size of the curved surface. Since the shape of the object is reflected in the scale of the 3D data, measurement accuracy is improved.

[0144] (First Modification of First Embodiment) A first modification of the first embodiment of the present invention will be described. In the above-described first embodiment, the control unit 50 converts the overall scale of the 3D data. On the other hand, in the first modification of the first embodiment, the control unit 50 measures the relative size of the subject by using the 3D data. Furthermore, the control unit 50 converts the relative size into absolute size. In the first modification of the first embodiment, it is not necessary to convert the overall scale of the 3D data.

[0145] Fig. 15 shows an example of the procedure of the measurement process. The operation of the endoscope system 1 will be described using Fig. 15. The same processes as those shown in Fig. 2 will not be described.

[0146] After step S107, step S109 is executed. Step S108 shown in Fig. 2 is not executed. In step S109, the control unit 50 measures the relative size of the subject based on one or more points in the 3D data.

[0147] After step S109, the control unit 50 calculates the ratio (scale factor) between the reference length set in step S105 and the length calculated in step S107. The control unit 50 multiplies the relative size measured in step S109 by the scale factor to convert the relative size to an absolute size (step S110). When step S110 is executed, the measurement process shown in FIG. 15 ends.

[0148] For example, if the reference length is X and the length on the 3D data corresponding to the reference length is Y, the scale factor is X / Y. For example, when the 3D distance Lt between two points is measured in step S109, the scale factor (X / Y) is multiplied by the 3D distance Lt to calculate the absolute distance (Lt*X / Y).

[0149] The measurement device according to each aspect of the present invention includes a control unit 50. The control unit 50 acquires 3D data generated based on two or more 2D images of the object generated by the endoscope device 10. The object has a curved surface. The 3D data includes virtual three-dimensional coordinates of three or more points on the object. The control unit 50 estimates the curved surface based on the three or more points in the 3D data. The control unit 50 acquires a first reference value indicating the absolute size of the curved surface and calculates a second reference value indicating the relative size of the curved surface. The control unit 50 measures the relative size of the object based on one or more points in the 3D data. The control unit 50 converts the relative size into an absolute size based on the first reference value and the second reference value.

[0150] The measurement method of each aspect of the present invention includes first to sixth steps. In a first step (step S102), the control unit 50 acquires 3D data generated based on two or more 2D images of the subject generated by the endoscope device 10. In a second step (step S104), the control unit 50 estimates a curved surface based on three or more points in the 3D data. In a third step (step S105), the control unit 50 acquires a first reference value indicating the absolute size of the curved surface. In a fourth step (step S107), the control unit 50 calculates a second reference value indicating the relative size of the curved surface. In a fifth step (step S109), the control unit 50 measures the relative size of the subject based on one or more points in the 3D data. In a sixth step (step S110), the control unit 50 converts the relative size into an absolute size based on the first and second reference values.

[0151] Each aspect of the present invention may include the following modifications: The control unit 50 calculates the ratio between the first reference value and the second reference value, and converts the relative size of the subject into the absolute size of the subject based on the ratio.

[0152] In the first embodiment, the control unit 50 converts the overall scale of the 3D data. On the other hand, in the first modification of the first embodiment, the control unit 50 converts the measured relative size of the object into absolute size, thereby reducing the processing load.

[0153] (Second Modification of First Embodiment) A second modification of the first embodiment of the present invention will be described. Fig. 16 shows an example of the configuration of an endoscope system 1a according to the second modification of the first embodiment. Portions that are the same as those shown in Fig. 1 will not be described.

[0154] 16 includes an insertion portion 2 and a main unit 6. The insertion portion 2 and the main unit 6 constitute an endoscope device 10a.

[0155] The insertion section 2 shown in Fig. 16 is the same as the insertion section 2 shown in Fig. 1. The main unit 6 has an imaging drive circuit 30, an image processing section 31, a UD drive section 32, an RL drive section 33, a bending control section 34, a light source 35, a light source control section 36, a display 51, a touch panel 52, operation buttons 53, a communication section 55, a volatile memory 56, a non-volatile memory 57, and a control section 60. The same reference symbols as those shown in Fig. 1 are assigned to blocks that are the same as those shown in Fig. 1.

[0156] The control unit 60 has both the functions of the control unit 40 shown in Fig. 1 and the functions of the control unit 50 shown in Fig. 1. The control unit 60 executes the process shown in Fig. 2.

[0157] In the second modification of the first embodiment, the measurement accuracy is improved in the same way as in the first embodiment.

[0158] (Third Modification of First Embodiment) A third modification of the first embodiment of the present invention will be described. Fig. 17 shows an example of the configuration of an endoscope system 1b according to the third modification of the first embodiment. Portions that are the same as those shown in Fig. 1 will not be described.

[0159] 17 includes an insertion portion 2, a scope unit 3b, and a base unit 7. The insertion portion 2 and the scope unit 3b constitute an endoscope device 10b. The scope unit 3b and the base unit 7 are connected by a cable 8.

[0160] The insertion section 2 shown in Fig. 17 is the same as the insertion section 2 shown in Fig. 1. The scope unit 3b shown in Fig. 17 is the same as the scope unit 3 shown in Fig. 1 except that it does not have the image processing section 31. The base unit 7 has the image processing section 31, a display 51, a touch panel 52, operation buttons 53, a communication section 55, a volatile memory 56, a non-volatile memory 57, and a control section 70. The same reference symbols as those in Fig. 1 are assigned to blocks that are the same as those in Fig. 1.

[0161] The control unit 70 has both the functions of the control unit 40 shown in Fig. 1 and the functions of the control unit 50 shown in Fig. 1. The control unit 70 executes the process shown in Fig. 2.

[0162] In the third modified example of the first embodiment, the measurement accuracy is improved in the same way as in the first embodiment.

[0163] Second Embodiment A second embodiment of the present invention will be described. In the second embodiment, the endoscope system 1 shown in Fig. 1 is used. An endoscope system 1a shown in Fig. 16 or an endoscope system 1b shown in Fig. 17 may also be used.

[0164] In the first embodiment, the user inputs an instruction to set a reference point to the endoscope system 1. In the first embodiment, the control unit 50 sets the reference point based on the instruction and estimates a curved surface based on points included in an area near the reference point. On the other hand, in the second embodiment, the control unit 50 estimates a curved surface based on 3D data without receiving a reference point from the user.

[0165] The endoscope system 1 executes the measurement process shown in Fig. 2. Fig. 18 shows an example of the procedure of the process executed in step S104 shown in Fig. 2. The operation of the endoscope system 1 will be described with reference to Fig. 18.

[0166] The control unit 50 analyzes the 3D data generated in step S102 and estimates a curved surface in the 3D shape of three or more points included in the 3D data (step S400).

[0167] Step S400 will be described in detail. For example, the control unit 50 extracts a characteristic pattern or shape (corners, edges, etc.) from the 3D data and estimates a curved surface from the extracted pattern or shape. Alternatively, the control unit 50 divides the 3D data into two or more small regions and sequentially estimates a cylindrical surface, a spherical surface, etc. in each small region. If the error in the estimated curved surface is large, the control unit 50 discards the estimated curved surface.

[0168] The control unit 50 gradually expands the small region and checks whether the estimated curved surface matches the expanded region. If the shape of the subject changes significantly within the expanded region, the estimation error of the curved surface will increase. Therefore, the control unit 50 stops expanding the region and sets the estimated curved surface as the reference surface.

[0169] After step S400, the control unit 50 displays the curved surface estimated in step S400 on the display 51 (step S401).

[0170] In step S401, the control unit 50 may perform semi-transparency processing on the curved surface and display the curved surface on the 3D shape of the subject. The control unit 50 may display the 3D shape or the curved surface on the display 51, or may switch between displaying the 3D shape and the curved surface in accordance with an instruction from the user. When two or more curved surfaces are estimated in step S400, the control unit 50 may display all of the two or more curved surfaces on the display 51, or may display some of the two or more curved surfaces on the display 51 in step S401.

[0171] For example, the control unit 50 may select one or more curved surfaces whose estimated curved surfaces occupy a large area of ​​the 3D data, and display the selected curved surfaces on the display 51. The control unit 50 may select one or more curved surfaces whose estimation error is small, and display the selected curved surfaces on the display 51. For example, the control unit 50 may calculate the estimation error (RMS error) by using a technique called ICP (Iterative Closest Point). The control unit 50 may select one or more curved surfaces depending on whether the estimated curved surface is located at the center or edge of the 3D data, and display the selected curved surfaces on the display 51.

[0172] After step S401, the control unit 50 determines whether the curved surface has been accurately estimated (step S402). Step S402 is the same as step S206 shown in FIG.

[0173] When the control unit 50 determines in step S402 that the curved surface has been accurately estimated, the control unit 50 sets the curved surface as a reference surface (step S403). Information about the reference surface is stored in the volatile memory 56. When step S403 is executed, the process shown in Fig. 18, i.e., step S104 shown in Fig. 2, ends.

[0174] If the control unit 50 determines in step S402 that the curved surface has not been accurately estimated, the control unit 50 changes the conditions for estimating the curved surface (step S404). After step S404, step S400 is executed.

[0175] Step S404 will be described in detail. First to fourth examples will be described below. In the first example, the control unit 50 limits the valid range of the 3D data for estimating the curved surface. If a valid range is set for the 3D data, the control unit 50 changes the valid range. In the second example, the control unit 50 increases the number of iterations when estimating the curved surface. Alternatively, the control unit 50 changes the RMS error threshold for determining that the curved surface has been estimated.

[0176] In a third example, the control unit 50 changes the algorithm for estimating the curved surface. For example, the control unit 50 changes the algorithm from ICP to another algorithm. In a fourth example, the control unit 50 changes the shape of the curved surface to be estimated to an approximation model that is more suitable for the curved surface to be estimated. For example, the control unit 50 changes the curved surface to be estimated from a cylindrical surface to a spherical surface. The control unit 50 may combine two or more of the first to fourth examples.

[0177] The control unit 50 may change the conditions for estimating a curved surface without receiving instructions from the user. The control unit 50 may display parameters related to the conditions for estimating a curved surface on the display 51, and may change the parameters in accordance with instructions from the user. The control unit 50 may automatically set some of the above conditions in accordance with instructions from the user, and then automatically set the remaining conditions.

[0178] Instead of executing step S404, the control unit 50 may display a message on the display 51 to prompt the user to re-acquire images for generating 3D data. Alternatively, the control unit 50 may display a message on the display 51 to prompt the user to use 3D data generated from a frame of interest different from the frame of interest accepted in step S101, among frames included in the video file loaded in step S100, in order to set the reference plane again. After the video file is generated again, the control unit 50 may use the video file to re-execute the measurement process shown in FIG. 2 .

[0179] In the second embodiment, the process for the user to set the reference point is omitted, which makes the user's work more efficient.

[0180] Third Embodiment A third embodiment of the present invention will be described. In the third embodiment, the endoscope system 1 shown in Fig. 1 is used. An endoscope system 1a shown in Fig. 16 or an endoscope system 1b shown in Fig. 17 may also be used.

[0181] For example, the test object in the third embodiment is a pipe. The pipe has a cylindrical inner surface and an outer surface. The inner surface and the outer surface share a common central axis. The user inputs the inner and outer diameters of the pipe's cylinder into the endoscope system 1. The inner or outer diameter is set as the reference length. The control unit 50 displays a wall-thinning map on the display 51, which indicates the distribution of recesses and protrusions on the inner surface of the cylinder.

[0182] Fig. 19 shows an example of the procedure of the measurement process. The operation of the endoscope system 1 will be described using Fig. 19. The same processes as those shown in Fig. 15 will not be described.

[0183] In step S104, the control unit 50 estimates the cylindrical surface and sets the inner or outer surface of the cylinder as a reference surface. The user operates the touch panel 52 or the like to input the inner and outer diameters of the pipe's cylinder to the endoscope system 1. The inner diameter of a cylinder is the radius or diameter of the inner surface of the cylinder. The outer diameter of a cylinder is the radius or diameter of the outer surface of the cylinder. The outer diameter of a cylinder is larger than the inner diameter of the cylinder. When the inner surface of the cylinder is observed from a viewpoint inside the pipe, the user inputs the inner diameter of the cylinder to the endoscope system 1. When the outer surface of the cylinder is observed from a viewpoint outside the pipe, the user inputs the outer diameter of the cylinder to the endoscope system 1.

[0184] In step S105, the control unit 50 receives the inner diameter and the outer diameter input by the user and sets one of the inner diameter and the outer diameter as a reference length. The other of the inner diameter and the outer diameter is not used as the reference length but is used to display a metal-loss map in step S121, which will be described later.

[0185] In step S107, the control unit 50 measures the length corresponding to the inner or outer diameter of the cylinder of the pipe. Specifically, the control unit 50 calculates the center axis of the cylinder and calculates the distance between the center axis and the inner surface of the cylinder. Alternatively, the control unit 50 calculates the distance between the center axis and the outer surface of the cylinder.

[0186] In step S110, the control unit 50 calculates the ratio (scale factor) between the reference length for the inner or outer diameter of the cylinder and the measured length for the inner or outer diameter of the cylinder. The control unit 50 multiplies the relative size measured in step S109 by the scale factor to convert the relative size into an absolute size.

[0187] After step S110, the control unit 50 analyzes the 3D shape of the object. Specifically, the control unit 50 calculates the 3D distance between each point in the 3D data and a reference plane (step S120). The reference plane is the inner or outer surface of a cylinder.

[0188] After step S120, the control unit 50 generates a wall-thinning map showing the distribution of the 3D distances calculated in step S120, and displays the wall-thinning map on the display 51 (step S121). When step S121 is executed, the measurement process shown in FIG. 19 ends.

[0189] Step S121 will be described in detail. The control unit 50 generates a metal-thinning map by arranging each point in the 3D data at a position corresponding to the 3D distance between that point and a reference surface (e.g., the inner surface of a cylinder). Each point in the 3D data is associated with an angle centered on the central axis of the reference surface. For example, the angle in the vertically upward direction is defined as 0 degrees. If a gravity sensor is disposed at the tip portion 2a, the angle in the vertically upward direction can be defined based on a signal output from the gravity sensor. Each point in the 3D data is arranged in the metal-thinning map at a position away from the central axis by the 3D distance between that point and the reference surface in the direction of the angle associated with that point.

[0190] FIG. 20 shows positions on the subject SB4 that are displayed as a wall-thinning map. The subject SB4 is a cylinder. The cylinder has a central axis CA1. The 3D data shows the 3D shape of the subject SB4 from position ST to position EN. For example, the control unit 50 generates a wall-thinning map of the subject SB4 in a cross section perpendicular to the central axis CA1 at position A1.

[0191] 21 shows a first example of a wall-thickness reduction map. The control unit 50 displays a wall-thickness reduction map MAP1 shown in FIG.

[0192] The metal-thinning map MAP1 shows the distribution of 3D distances in a cross section of the specimen SB4 perpendicular to the central axis CA1 shown in Fig. 20. The control unit 50 displays, on the display 51, the length RI indicating the inner diameter accepted in step S105 and the length RO indicating the outer diameter accepted in step S105. The control unit 50 also displays, on the display 51, an angle indicating the position in the cross section. For example, the angle is displayed in increments of 45 degrees starting from 0 degrees.

[0193] The specimen SB4 has abnormal parts AR1 to AR4. The abnormal parts AR1, AR2, and AR4 are concave parts on the inner surface of the specimen SB4. The abnormal part AR3 is a convex part on the inner surface of the specimen SB4.

[0194] 22 shows a second example of the wall-thickness reduction map. The control unit 50 displays the wall-thickness reduction map MAP2 shown in FIG.

[0195] The metal-thinning map MAP2 corresponds to a development of the metal-thinning map MAP1. The horizontal direction in Fig. 22 indicates the position in the cross section as an angle. The vertical direction in Fig. 22 indicates the position in the direction perpendicular to the central axis CA1 (radial direction).

[0196] The control unit 50 displays the position corresponding to the length RI and the position corresponding to the length RO on the display 51. The control unit 50 also displays on the display 51 a line LD1 indicating the position of the deepest abnormal portion AR1 among the abnormal portions AR1, AR2, and AR4, and a line LH1 indicating the position of the highest abnormal portion AR3.

[0197] The control unit 50 may display the metal-thickness reduction map MAP1 and the metal-thickness reduction map MAP2 on the display 51. The control unit 50 may display one of the metal-thickness reduction map MAP1 and the metal-thickness reduction map MAP2 on the display 51, or may switch between the display of the metal-thickness reduction map MAP1 and the display of the metal-thickness reduction map MAP2 in accordance with an instruction from the user.

[0198] 23 shows a third example of the wall-thickness reduction map. The control unit 50 displays the wall-thickness reduction map MAP3 shown in FIG.

[0199] The metal-thinning map MAP3 shows the distribution of 3D distances in the region of the specimen SB4 from position ST to position EN shown in Fig. 20. The horizontal direction shown in Fig. 23 indicates the position in the direction perpendicular to the central axis CA1 as an angle. The vertical direction shown in Fig. 23 indicates the position in the direction parallel to the central axis CA1.

[0200] The subject SB4 has abnormal areas AR1 to AR8. The control unit 50 calculates the depth or height of each abnormal area. The depth or height of each abnormal area is the maximum distance between the reference plane and a point included in each abnormal area. The control unit 50 displays each abnormal area in a color corresponding to the depth or height of the abnormal area. The control unit 50 may also display each position in a color corresponding to the depth or height of each position in each abnormal area.

[0201] The control unit 50 may display information indicating that each abnormal portion is a concave portion or a convex portion. For example, the control unit 50 may surround each abnormal portion with a type of line corresponding to whether it is a concave portion or a convex portion. The control unit 50 may display a combination of the metal-loss map MAP2 and the metal-loss map MAP3 on the display 51.

[0202] Each aspect of the present invention may include the following modifications: The subject has an inner and outer cylindrical surface. The control unit 50 acquires a first reference distance or a second reference distance as a first reference value. The first reference distance indicates the absolute size of the inner cylindrical surface. The second reference distance indicates the absolute size of the outer cylindrical surface.

[0203] Each aspect of the present invention may include the following modifications. The control unit 50 sets the inner or outer surface of the cylinder as a reference plane. The control unit 50 calculates the distance between each of two or more points in the 3D data and the reference plane. The control unit 50 displays on the display 51 the distribution of the distances between the two or more points for which the distances have been calculated.

[0204] In the third embodiment, the control unit 50 estimates the cylindrical surface of the object to be inspected and obtains a reference value indicating the absolute size of the cylindrical surface. When the object to be inspected is a pipe, measurement accuracy is improved.

[0205] Furthermore, the control unit 50 generates a wall-thinning map and displays the wall-thinning map on the display 51. The user can know the tendency of abnormal portions occurring according to the position or direction in the pipe.

[0206] (Fourth embodiment) A fourth embodiment of the present invention will be described. In the fourth embodiment, the endoscope system 1 shown in Fig. 1 is used. An endoscope system 1a shown in Fig. 16 or an endoscope system 1b shown in Fig. 17 may also be used.

[0207] In the fourth embodiment, the control unit 50 compares the 3D data with previously prepared reference data. The reference data is design data such as CAD (Computer Aided Design) data, or 3D data generated in a previous inspection. As will be described later, the reference data may be a 2D image of the object, i.e., 2D data. The reference data includes absolute 3D coordinates of three or more points on the object. The reference data is pre-stored in the non-volatile memory 57. The control unit 50 converts the scale of the 3D data so that the scale of the 3D data matches the scale of the reference data.

[0208] Fig. 24 shows an example of the procedure of the measurement process. The operation of the endoscope system 1 will be described using Fig. 24. The same processes as those shown in Fig. 2 will not be described.

[0209] After step S103, the control unit 50 calculates a scale factor and converts the scale of the 3D data based on the scale factor (step S130). After step S130, step S109 is executed.

[0210] Fig. 25 shows an example of the procedure of the process executed in step S130 shown in Fig. 24. The operation of the endoscope system 1 will be described with reference to Fig. 25.

[0211] The control unit 50 reads out the reference data from the non-volatile memory 57 (step S500). As described above, the reference data may be design data or 3D data generated in an inspection that was previously performed.

[0212] When an examination is performed with the monocular optical adapter attached to the tip of the insertion portion 2, the process shown in Fig. 2 is executed, and the scale of the 3D data is converted in step S108. The 3D data is used as reference data.

[0213] A stereo optical adapter having two fields of view may be attached to the tip of the insertion section 2. The stereo optical adapter has a first optical system and a second optical system corresponding to the two fields of view. The first optical system and the second optical system form two optical images of the subject on the imaging section 20. The imaging section 20 generates a stereo image corresponding to the first optical image and the second optical image. The stereo image includes a pair of two images (a first image and a second image). In other words, the stereo image includes an image of the subject seen from a first viewpoint and an image of the subject seen from a second viewpoint.

[0214] The control unit 50 can calculate the 3D coordinates of three or more points on the subject by using one or more stereo images generated by the imaging unit 20, and can generate 3D data including the 3D coordinates. The 3D data may be used as reference data. As long as the scale of the 3D data can be converted, 2D data may be used as reference data in addition to the 3D data. The method of acquiring the reference data is not limited to using a monocular optical adapter or a stereo optical adapter. As long as 3D data of the subject can be generated, any method may be used as a method of generating 3D data.

[0215] After step S500, the control unit 50 displays the first 3D shape indicated by the reference data and the second 3D shape indicated by the 3D data on the display 51. At this time, the two 3D shapes are aligned horizontally or vertically (step S501).

[0216] After the two 3D shapes are displayed, the user operates the touch panel 52 to input information indicating the same point on the first 3D shape and the second 3D shape to the endoscope system 1. The control unit 50 accepts the information input by the user (step S502). The input point information is stored in the volatile memory 56.

[0217] In step S501, the control unit 50 displays the first 3D shape SHR1, the second 3D shape SHO1, the button BT1, and the buttons BT6 to BT8 shown in Fig. 26 on the screen SC1 of the display 51. The same parts as those shown in Fig. 10 will not be described.

[0218] The control unit 50 displays a first 3D shape SHR1 based on the reference data and a second 3D shape SHO1 based on the 3D data. The user operates the touch panel 52 to set the same point on the first 3D shape SHR1 and the second 3D shape SHO1, and inputs the point to the endoscope system 1. For example, the user touches the position of the point on the first 3D shape SHR1 and the position of the point on the second 3D shape SHO1. The control unit 50 accepts the point input by the user in step S502 and sets the point on the first 3D shape SHR1 and the second 3D shape SHO1.

[0219] For example, a user inputs points PR1 through PR4 in a first 3D shape SHR1 and points PO1 through PO4 in a second 3D shape SHO1. Point PR1 is the same as point PO1, point PR2 is the same as point PO2, point PR3 is the same as point PO3, and point PR4 is the same as point PO4.

[0220] After step S502, the control unit 50 matches the positions and orientations of the reference data and the 3D data based on the points set in step S502 for the first 3D shape and the second 3D shape (step S503). Hereinafter, the process of matching the positions and orientations of the reference data and the 3D data will be referred to as "registration."

[0221] After step S503, the control unit 50 calculates a scale factor for matching the scale of the 3D data with the scale of the reference data (step S504).

[0222] The control unit 50 may simultaneously execute step S503 and step S504 as described below. Variables (Tx, Ty, Tz) related to the position of the 3D data in 3D space, variables (Rx, Ry, Rz) related to the orientation of the 3D data, and variables (Sx, Sy, Sz) related to the scale of the 3D data are defined. These nine variables need to be calculated. Because the scale-related variables Sx, Sy, and Sz are the same, seven variables actually need to be calculated.

[0223] Using one pair established in step S502, three equations can be introduced. The control unit 50 uses three or more pairs established in step S502 and calculates seven variables using a known solution method, such as an algorithm like the Levenberg-Marquardt method for solving nonlinear least-squares problems. This allows the control unit 50 to calculate the position, orientation, and scale factor of the 3D data that matches the reference data. Each of the three or more pairs includes a point in the reference data and a point in the 3D data.

[0224] After step S504, the control unit 50 converts the scale of the 3D data by multiplying the 3D coordinates of each point in the 3D data by the scale factor calculated in step S504. The control unit 50 displays a second 3D shape on the display 51 based on the 3D data with the converted scale (step S505).

[0225] In step S505, the control unit 50 displays the first 3D shape SHR1, the second 3D shape SHO1, the button BT1, and the buttons BT6 to BT8 shown in Fig. 27 on the screen SC1 of the display 51. The same parts as those shown in Fig. 10 will not be described.

[0226] The control unit 50 displays a first 3D shape SHR1 based on the reference data, and a second 3D shape SHO1 based on the 3D data. The control unit 50 displays the second 3D shape SHO1 on the first 3D shape SHR1. The first 3D shape SHR1 and the second 3D shape SHO1 overlap each other. The control unit 50 displays the scale factor SF1 calculated in step S504 on the display 51. The control unit 50 displays the registration error ER1 on the display 51.

[0227] After step S505, the control unit 50 determines whether the position of the reference data and the position of the 3D data match (step S506).

[0228] Step S506 will now be described in detail. For example, the user checks the first 3D shape and the second 3D shape displayed on the display 51. The user determines the degree of match between the first 3D shape and the second 3D shape and inputs information indicating the determination result to the endoscope system 1. Based on the information input by the user, the control unit 50 determines whether the position of the reference data and the position of the 3D data match. The control unit 50 may make the determination in step S506 based on a registration error or the like.

[0229] When the control unit 50 determines in step S506 that the position of the reference data and the position of the 3D data match, the control unit 50 confirms the alignment result. At this time, the scale factor calculated in step S504 is confirmed (step S507). The 3D data with the converted scale is stored in the non-volatile memory 57. When step S507 is executed, the process shown in FIG. 25, i.e., step S130 shown in FIG. 24, ends.

[0230] When the control unit 50 determines in step S506 that the position of the reference data and the position of the 3D data do not match, the control unit 50 changes the conditions for alignment (step S508). After step S508, step S502 is executed.

[0231] Details of step S508 will be described. First to fourth examples will be described below. In the first example, the control unit 50 performs rough alignment in accordance with instructions from the user before performing the alignment in step S503. The user operates the touch panel 52 to overlay the 3D data on the reference data. By performing the alignment in step S503 when the positions of the reference data and the 3D data match to a certain extent, the control unit 50 can perform the alignment with high accuracy.

[0232] In the second example, the control unit 50 changes the parameters related to the alignment. For example, when the control unit 50 makes the determination in step S506 without any instruction from the user, the control unit 50 relaxes the criteria related to the accuracy of the alignment. Alternatively, the control unit 50 increases the number of times the alignment is performed.

[0233] In the third example, the user operates the touch panel 52 to input information indicating the location of an abnormal point or abnormal area that is likely to cause a registration failure to the endoscope system 1. The control unit 50 accepts the information and removes the point or area indicated by the information from the 3D data.

[0234] In a fourth example, the control unit 50 increases the number of pairs of points in the first 3D shape represented by the reference data and points in the second 3D shape represented by the 3D data. After the number of pairs is increased, the control unit 50 performs the alignment again. The control unit 50 may combine two or more of the first to fourth examples.

[0235] The control unit 50 may change the alignment conditions without receiving instructions from the user. The control unit 50 may display parameters related to the alignment conditions on the display 51 and may change the parameters in accordance with instructions from the user. The control unit 50 may automatically set some of the above conditions in accordance with instructions from the user, and then automatically set the remaining conditions.

[0236] When an inspection is carried out, a folder for each type of inspection object and each inspection location is created in the non-volatile memory 57. The inspection results including 3D data are saved in the folder.

[0237] 28 shows an example of a hierarchical structure of folders in which inspection results are saved. For example, in an aircraft engine inspection, many folders are generated according to the inspection object, stage number, blade number, etc. The control unit 50 can acquire 3D data generated in previous inspections from a location identified by a folder name or folder structure corresponding to the inspection location in the currently being inspected inspection.

[0238] The 3D data is associated with the 2D image used to generate the 3D data. When the 3D data is associated with the 2D image, the 3D data and 2D image acquired in a previous examination can be used as reference data. The control unit 50 can perform registration according to the 3D shape characteristics of the subject as well as the texture and color of the subject in the 2D image.

[0239] The control unit 50 may select points that satisfy a predetermined condition from the points input by the user in step S502. For example, the control unit 50 may select points in a distinctive pattern.

[0240] The control unit 50 may extract the same points on the first 3D shape and the second 3D shape without user instructions. In this case, the control unit 50 can improve the accuracy of alignment by extracting points on a characteristic pattern. For example, the control unit 50 may extract points on a cooling hole or the like.

[0241] After step S109, step S110 shown in Fig. 15 may be executed. In this case, the control unit 50 does not need to convert the scale of the 3D data in step S130 (step S505).

[0242] The measurement device according to each aspect of the present invention includes a control unit 50. The control unit 50 acquires 3D data generated based on two or more 2D images of the object generated by the endoscope device 10. The object has a curved surface (e.g., a cylindrical surface). The 3D data includes virtual 3D coordinates of three or more points on the object. The control unit 50 acquires reference data including absolute 3D coordinates of three or more points on the object from a storage medium. The control unit 50 converts the scale of the 3D data so that the scale of the 3D data matches the scale of the reference data. The control unit 50 measures the size of the object based on one or more points in the 3D data having the converted scale.

[0243] The measurement method of each aspect of the present invention includes first to fourth steps. In a first step (step S102), the control unit 50 acquires 3D data generated based on two or more 2D images of the object generated by the endoscope device 10. In a second step (step S500), the control unit 50 acquires reference data including absolute 3D coordinates of three or more points on the object from a storage medium. In a third step (step S505), the control unit 50 converts the scale of the 3D data so that the scale of the 3D data matches the scale of the reference data. In a fourth step (step S109), the control unit 50 measures the size of the object based on one or more points in the 3D data having the converted scale.

[0244] In the measurement device according to each aspect of the present invention, the control unit 50 calculates a scale factor for matching the scale of the 3D data with the scale of the reference data. The control unit 50 measures the relative size of the object based on one or more points in the 3D data. The control unit 50 converts the relative size into absolute size based on the scale factor.

[0245] The measurement method of each aspect of the present invention includes fifth to seventh steps in addition to the first and second steps described above. In the fifth step (step S504), the control unit 50 calculates a scale factor for matching the scale of the 3D data with the scale of the reference data. In the sixth step (step S109), the control unit 50 measures the relative size of the subject based on one or more points in the 3D data. In the seventh step (step S110), the control unit 50 converts the relative size into absolute size based on the scale factor.

[0246] In the fourth embodiment, the control unit 50 converts the scale of the 3D data so that the scale of the 3D data matches the scale of the reference data. Even if the 3D shape of the subject is complex, the control unit 50 can accurately convert the scale of the 3D data, thereby improving measurement accuracy.

[0247] Fifth Embodiment A fifth embodiment of the present invention will be described. In the fifth embodiment, the endoscope system 1 shown in Fig. 1 is used. An endoscope system 1a shown in Fig. 16 or an endoscope system 1b shown in Fig. 17 may also be used.

[0248] As described above, the endoscopic device disclosed in Japanese Patent No. 5530225 performs surface-based measurement. The endoscopic device displays the reference surface and the measurement points. However, if the 3D shape of the object is complex, it is difficult for the user to grasp the exact positions of the measurement points.

[0249] The fifth embodiment aims to provide a function that can display information to assist in intuitively grasping the positions of measurement points.

[0250] The endoscope system 1 executes the measurement process shown in Fig. 2. Fig. 29 and Fig. 30 show an example of the procedure of the process executed in step S109 shown in Fig. 2. The operation of the endoscope system 1 will be described using Fig. 29 and Fig. 30. The measurement mode in the fifth embodiment indicates curved surface reference measurement or flat surface reference measurement.

[0251] The user operates the touch panel 52 to set measurement reference points on the 3D shape displayed on the display 51, and inputs the first and second measurement reference points to the endoscope system 1. For example, the user touches the positions of the first and second measurement reference points. The control unit 50 accepts the first and second measurement reference points input by the user and sets the first and second measurement reference points on the 3D shape (steps S600 and S601). The first and second measurement reference points are included in three or more points in the 3D data. Information on the first and second measurement reference points is stored in the volatile memory 56.

[0252] After step S601, the control unit 50 sets a variable n for managing the number of measurement reference points to 3 (step S602).

[0253] The user operates the touch panel 52 to set a measurement reference point on the 3D shape displayed on the display 51, and inputs the measurement reference point to the endoscope system 1. The control unit 50 accepts the measurement reference point input by the user and sets the measurement reference point on the 3D shape (step S603). The measurement reference point is included in three or more points in the 3D data. Information about the measurement reference point is stored in the volatile memory 56. The measurement reference point set in step S603 is treated as the nth measurement reference point.

[0254] After step S603, the control unit 50 calculates a measurement reference surface by using the 3D coordinates of the first to nth measurement reference points (step S604). When the measurement mode indicates curved surface-based measurement, the control unit 50 calculates a curved surface as the measurement reference surface. The type of curved surface may be any of a cylindrical surface, a spherical surface, a torus surface, an elliptical surface, a parabolic surface, and a free-form surface. When the measurement mode indicates plane-based measurement, the control unit 50 calculates a plane as the measurement reference surface.

[0255] After step S604, the control unit 50 displays the measurement reference plane calculated in step S604 on the display 51 (step S605).

[0256] The control unit 50 may set the measurement reference surface to a semi-transparent state, or may perform hidden surface removal on the measurement reference surface. The control unit 50 may also omit displaying the measurement reference surface.

[0257] The size and position of the measurement reference plane are arbitrary. The control unit 50 may align the center of the measurement reference plane with the center (circumcenter) of a circumscribing circle of a triangle having three measurement reference points as vertices. The control unit 50 may align the center of the measurement reference plane with the center of gravity of the three measurement reference points. The control unit 50 may align the center of the measurement reference plane with the center of a screen (3D view) of software that executes the measurement process.

[0258] The control unit 50 may set the size of the measurement reference plane to a size that includes the three measurement reference points. The control unit 50 may set the size of the measurement reference plane to a size that includes the entire 3D shape displayed based on the 3D data. The control unit 50 may set the size of the measurement reference plane to a size that does not include the three measurement reference points. For example, the control unit 50 may display, as the measurement reference plane, an inscribed circle or an inscribed rectangle of a triangle having the three measurement reference points as vertices.

[0259] After step S605, the control unit 50 determines whether or not to end the calculation of the measurement reference plane (step S606).

[0260] Details of step S606 will be described. For example, the number of required measurement reference points (first number) is set in advance. The control unit 50 determines whether the set number of measurement reference points (second number) has reached the first number. If the second number is the same as the first number, the control unit 50 determines to end calculation of the measurement reference plane. If the second number is smaller than the first number, the control unit 50 determines not to end calculation of the measurement reference plane.

[0261] If the control unit 50 determines in step S606 that the calculation of the measurement reference plane should not be completed, the control unit 50 increments the variable n by 1 (step S607). After step S607, step S603 is executed.

[0262] When the control unit 50 determines in step S606 that the calculation of the measurement reference plane is completed, the control unit 50 determines the measurement reference plane (step S608). Information about the measurement reference plane is stored in the volatile memory 56.

[0263] After step S608, the control unit 50 executes the same process as step S600, S601, or S603 to set a measurement point (step S609). The measurement point is included in three or more points in the 3D data. The measurement point may also be a point not included in the 3D data. When the measurement point is not included in the 3D data, the control unit 50 calculates the 3D coordinates of the measurement point based on the 3D coordinates of points included in the 3D data. Information about the measurement point is stored in the volatile memory 56.

[0264] After step S609, the control unit 50 calculates an auxiliary measurement plane that includes the measurement point set in step S609 and is parallel to the measurement reference plane (step S610). After step S610, the control unit 50 displays the auxiliary measurement plane calculated in step S610 on the display 51 (step S611).

[0265] The control unit 50 may set the auxiliary measurement surface to a semi-transparent state, or may perform hidden surface removal on the auxiliary measurement surface.

[0266] The size and position of the auxiliary measurement surface are arbitrary. The control unit 50 may align the center of the auxiliary measurement surface with the measurement point. The control unit 50 may align the center of the auxiliary measurement surface projected onto a plane including the three measurement reference points with the center (circumcenter) of the circumscribed circle of a triangle having the three measurement reference points as vertices. The control unit 50 may align the center of the auxiliary measurement surface projected onto the plane with the center of gravity of the three measurement reference points. The control unit 50 may align the center of the auxiliary measurement surface projected onto the screen (3D view) of software executing the measurement process with the center of the 3D view.

[0267] The control unit 50 may display, as the auxiliary measurement surface, a circle having a radius or diameter equal to the distance between the measurement point and the measurement reference surface. The control unit 50 may display, as the auxiliary measurement surface, a rectangle having sides with lengths equal to the distance between the measurement point and the measurement reference surface. The control unit 50 may match the radius or diameter of the circle or the length of the sides of the rectangle to a value obtained by multiplying the distance between the measurement point and the measurement reference surface by a predetermined coefficient. The control unit 50 may change the size of the auxiliary measurement surface depending on the position of the auxiliary measurement surface in the 3D view.

[0268] The measurement points set in step S609 are displayed on the 3D shape. The user observes the measurement points and determines whether they have been set at the correct positions. The control unit 50 determines whether to end the setting of the measurement points according to the operation performed by the user (step S612). The control unit 50 may determine not to end the setting of the measurement points until the user inputs information indicating that he or she wishes to end the measurement setting into the endoscope system 1. When the user inputs information indicating that he or she wishes to end the setting of the measurement points into the endoscope system 1, the control unit 50 may determine that he or she wishes to end the setting of the measurement points.

[0269] If the control unit 50 determines in step S612 that the setting of measurement points is not to be completed, step S609 is executed, and measurement points are set again. If the control unit 50 determines in step S612 that the setting of measurement points is to be completed, the control unit 50 determines the auxiliary measurement surface (step S613). Information on the auxiliary measurement surface is stored in the volatile memory 56.

[0270] After step S613, the control unit 50 calculates the 3D distance from the measurement reference plane to the measurement point (step S614). After step S614, the control unit 50 displays the measurement result on the display 51 (step S615). When step S615 is executed, the processing shown in Figures 29 and 30 ends. Steps S614 and S615 may be executed between steps S611 and S612.

[0271] Figures 31 to 33 show examples of the screen of the display 51 in the processing shown in Figures 29 and 30. Figures 31 to 33 show an example of plane-based measurement. The control unit 50 displays the 3D shape SH3 shown in Figure 31 or the 3D shape SH4 shown in Figures 32 and 33 on the screen SC1 of the display 51. The control unit 50 also displays button BT1 and buttons BT6 to BT8 on the screen SC1. Portions that are the same as those shown in Figure 10 will not be described.

[0272] As shown in Fig. 31, the control unit 50 displays the first measurement reference point MRP1, the second measurement reference point MRP2, and the third measurement reference point MRP3 on the 3D shape SH3, and displays the measurement reference plane MRS1 on the 3D shape SH3. When the measurement point MP4 is set as shown in Fig. 32, the control unit 50 displays the auxiliary measurement plane AMS1 on the 3D shape SH4. The control unit 50 also calculates the 3D distance from the measurement reference plane MRS1 to the measurement point MP4, and displays the measurement result MR3 indicating the 3D distance on the display 51.

[0273] For example, there may be a case where measurement of the depth or height of a depression or protrusion formed on the surface of a test object is required. In conventional technology, an auxiliary measurement surface is not displayed, making it difficult for the user to grasp the distance between the measurement point and the measurement reference surface. This makes it difficult for the user to set the measurement point at the deepest part of the depression or the highest part of the protrusion.

[0274] In the fifth embodiment, the user sets a measurement point with reference to the distance between the measurement reference surface and the auxiliary measurement surface. For example, when the user moves the measurement point, the distance between the measurement reference surface and the auxiliary measurement surface changes depending on the position of the measurement point. By setting the measurement point at the position where the distance is greatest, the user can easily set the measurement point at the deepest part of a recess or the highest part of a protrusion.

[0275] When the control unit 50 controls the size of the auxiliary measurement surface according to the distance between the measurement point and the measurement reference surface, the user can intuitively grasp the distance between the measurement reference surface and the auxiliary measurement surface from the size of the auxiliary measurement surface.

[0276] The control unit 50 may calculate a straight line that passes through the measurement point and is perpendicular to the measurement reference plane and the auxiliary measurement plane. The control unit 50 may calculate the point where the straight line intersects with the measurement reference plane, i.e., the foot of the perpendicular line. The control unit 50 may calculate an auxiliary measurement plane that includes the foot of the perpendicular line and is parallel to the measurement reference plane. The auxiliary measurement plane overlaps with the measurement reference plane. The user can easily grasp the distance between the two auxiliary measurement planes and can easily set the measurement point using that distance as a reference.

[0277] 33 shows an example in which two auxiliary measurement surfaces are displayed. The control unit 50 displays auxiliary measurement surface AMS1 and auxiliary measurement surface AMS2 on the 3D shape SH4. Auxiliary measurement surface AMS2 overlaps with measurement reference surface MRS1. Auxiliary measurement surface AMS1 and auxiliary measurement surface AMS2 are smaller than measurement reference surface MRS1.

[0278] Figures 34 to 36 show examples of the screen of the display 51 in the processing shown in Figures 29 and 30. Figures 34 to 36 show an example of curved surface reference measurement. The control unit 50 displays the 3D shape SH5, button BT1, and buttons BT6 to BT8 on the screen SC1. Portions that are the same as those shown in Figure 10 will not be described.

[0279] As shown in Fig. 34, the control unit 50 displays the first measurement reference point MRP4, the second measurement reference point MRP5, and the third measurement reference point MRP6 on the 3D shape SH5, and displays the measurement reference plane MRS2 on the 3D shape SH5. When the measurement point MP5 is set as shown in Fig. 35, the control unit 50 displays the auxiliary measurement plane AMS3 on the 3D shape SH5. The control unit 50 also calculates the 3D distance from the measurement reference plane MRS2 to the measurement point MP5, and displays a measurement result MR4 indicating the 3D distance on the display 51.

[0280] 36 shows an example in which two auxiliary measurement surfaces are displayed. The control unit 50 displays auxiliary measurement surface AMS3 and auxiliary measurement surface AMS4 on the 3D shape SH5. Auxiliary measurement surface AMS4 overlaps with measurement reference surface MRS2. Auxiliary measurement surface AMS3 and auxiliary measurement surface AMS4 are smaller than measurement reference surface MRS2.

[0281] In the above example, the auxiliary measurement surface is a flat surface. The auxiliary measurement surface may also be a curved surface.

[0282] 37A and 37B show an example in which the measurement reference surface and auxiliary measurement surface are curved surfaces. The control unit 50 calculates the measurement reference surface MRS3, which is a cylindrical surface. The control unit 50 sets a measurement point MP5 at the apex of a convex portion of the subject in accordance with instructions from the user. When the measurement point MP5 is set, the control unit 50 calculates an auxiliary measurement surface AMS5 that includes the measurement point and is parallel to the measurement reference surface. The auxiliary measurement surface AMS5 is a cylindrical surface, and the central axis of the auxiliary measurement surface AMS5 is the same as the central axis of the measurement reference surface MRS3. The diameter of the auxiliary measurement surface AMS5 is smaller than the diameter of the measurement reference surface MRS3.

[0283] For example, if the area near measurement point MP5 in the 3D shape of the subject protrudes from auxiliary measurement surface AMS5, the user can confirm that measurement point MP5 is not at the highest position of the convex portion. By setting measurement point MP5 so that the area near measurement point MP5 does not protrude from auxiliary measurement surface AMS5, the user can set measurement point MP5 to the highest position of the convex portion. If the measurement point is set in a concave portion of the subject, the user can set the measurement point to the deepest position of the concave portion by setting the measurement point so that the area near the measurement point does not protrude deep into the auxiliary measurement surface (curved surface).

[0284] The endoscope system 1 may execute the measurement process shown in FIG. 15, or may execute the processes shown in FIGS. 29 and 30 in step S109 shown in FIG.

[0285] Each aspect of the measurement device of the present invention includes a control unit 50. The control unit 50 acquires first 3D data generated based on two or more 2D images of the object generated by the endoscope device 10. The object has a curved surface (e.g., a cylindrical surface). The first 3D data includes virtual 3D coordinates of three or more points on the object. The control unit 50 estimates the curved surface based on the three or more points in the first 3D data. The control unit 50 acquires a first reference value indicating the absolute size of the curved surface and calculates a second reference value indicating the relative size of the curved surface. The control unit 50 converts the first 3D data into second 3D data based on the first and second reference values. The second 3D data includes absolute 3D coordinates of three or more points on the object. The control unit 50 sets a measurement reference plane based on the three or more points in the second 3D data. The control unit 50 sets an auxiliary measurement plane that includes measurement points included in the three or more points and is parallel to the measurement reference plane. The control unit 50 measures the size of the subject based on the measurement points.

[0286] The measurement method of each aspect of the present invention includes first to eighth steps. In a first step (step S102), the control unit 50 acquires first 3D data generated based on two or more 2D images of the subject generated by the endoscope device 10. In a second step (step S104), the control unit 50 estimates a curved surface based on three or more points in the first 3D data. In a third step (step S105), the control unit 50 acquires a first reference value indicating the absolute size of the curved surface. In a fourth step (step S107), the control unit 50 calculates a second reference value indicating the relative size of the curved surface. In a fifth step (step S108), the control unit 50 converts the first 3D data into second 3D data based on the first and second reference values. In a sixth step (step S604), the control unit 50 sets a measurement reference plane based on three or more points in the second 3D data. In a seventh step (step S610), the control unit 50 sets an auxiliary measurement plane that includes the measurement points included in the three or more points and is parallel to the measurement reference plane. In an eighth step (step S614), the control unit 50 measures the size of the object based on the measurement points.

[0287] In the measurement device according to each aspect of the present invention, the control unit 50 calculates a scale factor for matching the scale of the 3D data with the scale of the reference data. The control unit 50 measures the relative size of the object based on one or more points in the 3D data. The control unit 50 converts the relative size into absolute size based on the scale factor.

[0288] The measurement method of each aspect of the present invention includes the above-described first step, second step, and sixth to eighth steps, as well as ninth to eleventh steps. In the ninth step (step S110), the control unit 50 calculates a scale factor for matching the scale of the 3D data with the scale of the reference data. In the tenth step (step S614), the control unit 50 measures the relative size of the object based on one or more points in the 3D data. In the eleventh step (step S110), the control unit 50 converts the relative size into absolute size based on the scale factor.

[0289] In the fifth embodiment, the endoscope system 1 can display information for assisting the user in intuitively grasping the positions of the measurement points, thereby enabling the user to intuitively grasp the accurate positions of the measurement points.

[0290] Sixth Embodiment A sixth embodiment of the present invention will be described. In the sixth embodiment, the endoscope system 1 shown in Fig. 1 is used. An endoscope system 1a shown in Fig. 16 or an endoscope system 1b shown in Fig. 17 may also be used.

[0291] The test object in the sixth embodiment is a turbine blade. Hereinafter, the turbine blade will be referred to as a blade. The blade has a part of a cylindrical surface.

[0292] As described in the first embodiment, if the test object is a pipe, the reference length is the radius or diameter of the cylindrical surface that constitutes the pipe. On the other hand, the reference length in the sixth embodiment is the magnitude (length) of the arc or chord of a figure formed by a curve on the cross section of the cylindrical surface of the blade. The arc corresponds to at least a part of the curve. The chord is a line segment connecting two points on the curve.

[0293] For example, the curvature of the cylindrical surface of the blade may be small, and the radius or diameter of the cylindrical surface may be very large. In this case, the width or height of the blade may be much smaller than the radius or diameter of the cylindrical surface. Because the width or height is much smaller than the reference length, the width or height may not be measured accurately. In the sixth embodiment, an arc or chord length smaller than the radius or diameter of the cylindrical surface is used as the reference length, which increases the likelihood that the width or height of the blade will be measured accurately.

[0294] In an example in which the arc or chord length of the cross section of the cylindrical surface that constitutes the blade is used as the reference length, the following problem may occur. Two methods for calculating the arc or chord length will be described below. In the first method, the arc length is calculated. In the second method, the chord length is calculated. In the first and second methods, the control unit 50 calculates the arc or chord length without using the information about the curved surface estimated in step S104 shown in FIG. 2 or FIG. 15 .

[0295] A first example will be described. The control unit 50 calculates the length of the arc based on two or more points specified by the user on the 3D shape of the blade. In the first example, the user needs to specify a large number of points. Also, there is a possibility that the points on the arc may not be specified accurately.

[0296] A second example will be described. The control unit 50 calculates the length of the chord based on two points specified by the user on the 3D shape of the blade. Specifically, the control unit 50 calculates the distance between the two points. In the second example, there is a possibility that the two points corresponding to the chord are not specified accurately.

[0297] Fig. 38 shows an example of the 3D shape of a blade. In the example shown in Fig. 38 where the entire 3D shape SH6 is restored in the 3D reconstruction process, the user can easily specify two end points EP1 and EP2 corresponding to the width of the blade, and the control unit 50 can calculate the length of the chord CH1 connecting the end points EP1 and EP2.

[0298] On the other hand, in an example where only the range RA2 of the 3D shape SH6 is restored in the 3D reconstruction process and the ranges RA1 and RA3 of the 3D shape SH6 are not restored in the 3D reconstruction process, it is difficult for the user to specify two points in the range RA2 so that the line passing through those two points is parallel to the top end of the blade. Therefore, the control unit 50 may not be able to accurately calculate the length of the chord in the range RA1.

[0299] On the other hand, in the method of the sixth embodiment, the control unit 50 calculates the length of the arc or chord based on the curved surface estimated in step S104 shown in Fig. 2 or 15. This allows the control unit 50 to accurately calculate the length of the arc or chord.

[0300] The endoscope system 1 executes the measurement process shown in Fig. 2 or 15. The control unit 50 executes the following process in step S107. Three examples of the process executed by the control unit 50 in step S107 will be described below.

[0301] A first example will be described. Figures 39, 40, and 41 show examples of the 3D shape of the blade. The control unit 50 displays the 3D shape SH7 on the screen of the display 51.

[0302] The user operates the touch panel 52 or the like to specify a reference point RP4 on the 3D shape SH7, and inputs the reference point RP4 to the endoscope system 1. The reference point RP4 corresponds to one of the two end points at the upper end of the blade. The control unit 50 accepts the reference point RP4 input by the user and sets the reference point RP4 on the 3D shape SH7 ( FIG. 39 ). The reference point RP4 is included in three or more points in the 3D data.

[0303] The control unit 50 calculates a straight line SL1 that passes through the reference point RP4 and is perpendicular to the central axis CA2 of the cylindrical surface calculated in step S104. The control unit 50 calculates a point IT1 where the straight line SL1 intersects with the cylindrical surface, and calculates a cross section CS1 of the cylindrical surface ( FIG. 40 ). The cross section CS1 passes through the point IT1 and is perpendicular to the central axis CA2. The outline of the cross section CS1 is drawn in FIG. 40 . The cross section CS1 is circular.

[0304] The control unit 50 displays the cross section CS1 on the screen of the display 51. The user operates the touch panel 52 or the like to specify reference points RP5 and RP6 on the outer periphery of the cross section CS1, and inputs the reference points RP5 and RP6 to the endoscope system 1. The reference points RP5 and RP6 correspond to the two end points of the upper end of the blade. The control unit 50 accepts the reference points RP5 and RP6 input by the user and sets the reference points RP5 and RP6 on the 3D shape SH7 ( FIG. 41 ). The reference points RP5 and RP6 are on the outer periphery of the cross section CS1.

[0305] The control unit 50 calculates the length of the arc or chord of the cross section CS1 based on the reference points RP5 and RP6. The arc of the cross section CS1 is the portion between the reference points RP5 and RP6 on the outer periphery of the cross section CS1. The chord of the cross section CS1 is the line segment connecting the reference points RP5 and RP6.

[0306] A second example will be described. Figures 42 and 43 show examples of the 3D shape of a blade. The control unit 50 displays the 3D shape SH8 on the screen of the display 51.

[0307] The user operates the touch panel 52 or the like to specify a reference point RP7 on the 3D shape SH8, and inputs the reference point RP7 to the endoscope system 1. The reference point RP7 corresponds to one of the two end points at the upper end of the blade. The control unit 50 accepts the reference point RP7 input by the user and sets the reference point RP7 on the 3D shape SH8. The reference point RP7 is included in three or more points in the 3D data. The control unit 50 calculates a cross section CS2 perpendicular to the central axis CA3 of the cylindrical surface (FIG. 42) using the same method as the method for calculating the cross section CS1 shown in FIG. 40.

[0308] The user operates the touch panel 52 or the like to specify a reference point RP8 on the 3D shape SH8, and inputs the reference point RP8 to the endoscope system 1. The reference point RP8 is on an edge that includes the other of the two end points of the upper end of the blade. The control unit 50 accepts the reference point RP8 input by the user and sets the reference point RP8 on the 3D shape SH8. The reference point RP8 is included in three or more points in the 3D data. The control unit 50 calculates a cross section CS3 perpendicular to the central axis CA3 of the cylindrical surface (FIG. 42) using the same method as the method for calculating the cross section CS1 shown in FIG. 40.

[0309] The control unit 50 calculates a straight line SL2 that passes through the reference point RP8 and is parallel to the central axis CA3. The control unit 50 calculates a point IT2 where the straight line SL2 intersects with the cross section CS2 (FIG. 43).

[0310] The control unit 50 calculates the length of the arc or chord of the cross section CS2 based on the reference point RP7 and the point IT2. The arc of the cross section CS2 is the portion of the periphery of the cross section CS2 between the reference point RP7 and the point IT2. The chord of the cross section CS2 is the line segment connecting the reference point RP7 and the point IT2.

[0311] In the second example, even if the 3D shape around point IT2 is not restored in the 3D reconstruction process, the control unit 50 can set the reference point (point IT2) of the arc or chord of cross section CS2 based on reference point RP8.

[0312] A third example will be described. The control unit 50 extracts edges of the 3D shape based on the 3D data generated in step S102. Each point on each edge is associated with a point in the 3D data. Figure 44 shows an example of edges extracted from the 3D shape of a blade. The control unit 50 displays edge ED1 on the screen of the display 51.

[0313] The user operates the touch panel 52 or the like to specify a reference point RP9 on the edge ED1, and inputs the reference point RP9 to the endoscope system 1. The reference point RP9 is located on the upper edge of the blade. The control unit 50 accepts the reference point RP9 input by the user and sets the reference point RP9. The reference point RP9 is included in three or more points in the 3D data.

[0314] The control unit 50 identifies the end points EP3 and EP4 of the edge including the reference point RP9. The end points EP3 and EP4 are each a point where the edge including the reference point RP9 intersects with another edge.

[0315] The control unit 50 calculates the cross section of the cylindrical surface by using the same method as the method for calculating the cross section CS2 shown in Figure 43. The end point EP3 corresponds to the reference point RP7 shown in Figure 43. The control unit 50 calculates the length of the arc or chord of the cross section of the cylindrical surface based on the end points EP3 and EP4. The end point EP4 corresponds to point IT2 shown in Figure 43. The arc of the cross section is the portion between the end points EP3 and EP4 on the outer periphery of the cross section. The chord of the cross section is the line segment connecting the end points EP3 and EP4.

[0316] In the third method, compared to the first or second method, the number of operations required for the user to specify the reference point is reduced, and the variation in the reference length corresponding to the variation in the position of the reference point specified by the user is reduced.

[0317] The cross section of the curved surface of the object may be elliptical, and the control unit 50 may calculate the arc or chord of the ellipse.

[0318] Each aspect of the present invention may include the following modifications: The control unit 50 calculates the second reference value based on the size of an arc or a chord of a figure formed by a curve on a cross section of the curved surface of the object.

[0319] In the sixth embodiment, the endoscope system 1 estimates the cylindrical surface of the blade and calculates the length of the arc or chord of the cross section of the cylindrical surface, thereby enabling the endoscope system 1 to accurately calculate the length of the arc or chord.

[0320] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments and their modifications. Addition, omission, substitution, and other modifications of the configuration are possible within the scope of the spirit of the present invention. Furthermore, the present invention is not limited by the above description, but is limited only by the scope of the appended claims.

[0321] According to each embodiment of the present disclosure, the measurement device, the measurement method, and the program can improve measurement accuracy.

[0322] REFERENCE SIGNS LIST 1, 1a, 1b Endoscope system 2 Insertion section 2a Tip section 3, 3b Scope unit 4, 7 Base unit 5, 6 Main unit 10, 10a, 10b Endoscope device 11 External device 20 Imaging section 21 Bending section 22 Illumination window 30 Imaging drive circuit 31 Image processing section 32 UD drive section 33 RL drive section 34 Bending control section 35 Light source 36 Light source control section 40, 50, 60, 70 Control section 41, 54, 55 Communication section 42, 56 Volatile memory 43, 57 Non-volatile memory 51 Display 52 Touch panel 53 Operation button

Claims

1. A measuring device having a control unit that: acquires three-dimensional data generated based on two or more two-dimensional images of a subject generated by an endoscope device, the subject having a curved surface, the three-dimensional data including virtual three-dimensional coordinates of three or more points on the subject; estimates the curved surface based on the three or more points in the three-dimensional data; acquires a first reference value indicating the absolute size of the curved surface; calculates a second reference value indicating the relative size of the curved surface; measures the relative size of the subject based on one or more points in the three-dimensional data; and converts the relative size of the subject to the absolute size of the subject based on the first reference value and the second reference value.

2. The measurement device according to claim 1, wherein the control unit estimates the curved surface based on three or more points contained in one or more regions in the three-dimensional data.

3. The measurement device according to claim 1, wherein the control unit estimates the curved surface based on three or more points in the three-dimensional data that correspond to three or more points contained in one or more areas in the two or more two-dimensional images.

4. The measurement device of claim 2, wherein the control unit estimates the curved surface based on four or more points contained in two or more regions that include the one or more regions, and each of the two or more regions contains two or more points.

5. The measurement device of claim 3, wherein the control unit estimates the curved surface based on four or more points in the three-dimensional data corresponding to four or more points contained in two or more regions including the one or more regions, and each of the two or more regions contains two or more points.

6. The measurement device according to claim 1, wherein the curved surface comprises all or part of a cylindrical surface, a spherical surface, a torus surface, an elliptical surface, or a paraboloid surface.

7. The measurement device according to claim 6, wherein the control unit estimates two or more curved surfaces including the curved surface, and calculates the second reference value of one of the two or more curved surfaces.

8. The measuring device according to claim 6, wherein the control unit calculates the second reference value based on the size of an arc or chord of a figure formed by a curve on a cross section of the curved surface.

9. The measuring device according to claim 1, wherein the control unit acquires the first reference value input through an input device.

10. The measuring device according to claim 1, wherein the control unit acquires the first reference value from a storage medium.

11. The measurement device according to claim 1, wherein the control unit calculates a ratio between the first reference value and the second reference value, and converts the relative size of the subject into an absolute size of the subject based on the ratio.

12. The measuring device of claim 1, wherein the subject has a cylindrical inner surface and an outer surface, the control unit acquires a first reference distance or a second reference distance as the first reference value, the first reference distance indicating the absolute size of the inner surface, and the second reference distance indicating the absolute size of the outer surface.

13. The measurement device described in claim 12, wherein the control unit sets the inner surface or the outer surface as a reference surface, calculates the distance between each of two or more points in the three-dimensional data and the reference surface, and displays on a display the distribution of the distances at the two or more points for which the distances have been calculated.

14. The measurement device according to claim 1, wherein the control unit measures the distance between a point in the three-dimensional data and the curved surface.

15. The measurement device according to claim 1, wherein the control unit estimates a reference plane based on three or more points in the three-dimensional data, and measures the distance between a point in the three-dimensional data and the reference plane.

16. The measurement device according to claim 1, wherein the two or more images are generated by the endoscope device from two or more different viewpoints.

17. The measurement device according to claim 16, wherein the endoscope device has an insertion section with a monocular optical system attached to the tip.

18. The measuring device according to claim 1, wherein the control unit generates the three-dimensional data based on the two or more images.

19. The measuring device according to claim 1, wherein the control unit acquires the three-dimensional data from a storage medium.

20. A measurement device having a control unit, which: acquires first three-dimensional data generated based on two or more two-dimensional images of a subject generated by an endoscope device; the subject has a curved surface; the first three-dimensional data includes virtual three-dimensional coordinates of three or more points on the subject; estimates the curved surface based on the three or more points in the first three-dimensional data; acquires a first reference value indicating the absolute size of the curved surface; calculates a second reference value indicating the relative size of the curved surface; converts the first three-dimensional data into second three-dimensional data based on the first reference value and the second reference value; the second three-dimensional data includes absolute three-dimensional coordinates of three or more points on the subject; and measures the size of the subject based on one or more points in the second three-dimensional data.

21. A measurement method in which a control unit acquires three-dimensional data generated based on two or more two-dimensional images of a subject generated by an endoscopic device, the subject having a curved surface, the three-dimensional data including virtual three-dimensional coordinates of three or more points on the subject, estimates the curved surface based on the three or more points in the three-dimensional data, acquires a first reference value indicating the absolute size of the curved surface, calculates a second reference value indicating the relative size of the curved surface, measures the relative size of the subject based on one or more points in the three-dimensional data, and converts the relative size of the subject to the absolute size of the subject based on the first reference value and the second reference value.

22. A program for causing a computer to execute the steps of: acquiring three-dimensional data generated based on two or more two-dimensional images of a subject generated by an endoscopic device, the subject having a curved surface, the three-dimensional data including virtual three-dimensional coordinates of three or more points on the subject; estimating the curved surface based on three or more points in the three-dimensional data; acquiring a first reference value indicating the absolute size of the curved surface; calculating a second reference value indicating the relative size of the curved surface; measuring the relative size of the subject based on one or more points in the three-dimensional data; and converting the relative size of the subject to the absolute size of the subject based on the first reference value and the second reference value.

Images