Three-dimensional measuring device, three-dimensional measuring method, and three-dimensional measuring program

Abstract

To facilitate operations to scan an object to be measured using a three-dimensional scanner.SOLUTION: A three-dimensional measuring device comprises: a texture camera; three-dimensional data generation means that generates three-dimensional shape data of an object to be measured on the basis of an image including pattern light and the position and posture of a three-dimensional scanner; and a display data generation unit. The display data generation unit generates first display data that displays a texture image as a two-dimensional texture image seen from the position and posture of the texture camera when the texture image is acquired, and second display data that displays a three-dimensional texture image obtained by applying the texture of the object to be measured included in the texture image to the three-dimensional data on the basis of position and posture information corresponding to the texture image. The three-dimensional measuring device receives input of a measurement instruction for the three-dimensional texture image displayed on a display through the generation of the second display data by the display data generation unit, and executes processing of measuring the object to be measured on the basis of the measurement instruction and the three-dimensional shape data.SELECTED DRAWING: Figure 23

Description

Technical Field

[0001] The present disclosure relates to a three-dimensional measuring apparatus including a three-dimensional scanner, a three-dimensional measuring method using the three-dimensional scanner, and a three-dimensional measurement program.

Background Art

[0002] For example, Patent Document 1 discloses performing three-dimensional coordinate measurement of a measurement object using a contact probe having a contact portion that contacts a desired portion of the measurement object. In this Patent Document 1, a plurality of markers included in the contact probe are imaged by an imaging unit installed at a location away from the contact probe, and based on the marker image generated by the imaging unit, the three-dimensional coordinates of the contact position of the contact probe can be calculated.

[0003] The contact probe of Patent Document 1 is provided with a display unit that displays a setting screen including measurement items, and a measurement operator can perform a selection operation of setting items while viewing the setting screen displayed on the display unit.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] By the way, in the device of Patent Document 1 above, since it is a contact probe, coordinates can only be measured at the part contacted by the probe. Therefore, if a non-contact three-dimensional scanner is used, it becomes possible to measure a wider range of the measurement object, that is, to perform a wide-range scan. When a measurement operator scans a measurement object with a three-dimensional scanner, for example, in a state where only three-dimensional data is displayed on the display screen, since the data is monotonously displayed in a predetermined color, it may be difficult to identify where the part to be measured is on the display screen.

[0006] In this regard, although the contact probe of Patent Document 1 is provided with a display unit, since the contact probe measures by contacting the measurement object, there is no need to identify the part of the measurement object to be measured on the display screen, and also, since the measurement operator knows the contact part of the probe, there is no need to confirm it on the display screen. Therefore, in the case of the contact probe of Patent Document 1, problems such as those during the scan of the measurement object by the three-dimensional scanner described above could not occur.

[0007] The present disclosure is in view of such a point, and its object is to facilitate the scan operation of a measurement object by a three-dimensional scanner.

Means for Solving the Problem

[0008] To achieve the above object, one aspect of the present disclosure can be premised on a three-dimensional measuring device that measures the three-dimensional shape of an object to be measured. The three-dimensional measuring device includes a three-dimensional scanner having a scanner light source that irradiates pattern light, and a scanner imaging unit that captures the pattern light irradiated by the scanner light source and generates an image including the pattern light, a texture camera that captures the object to be measured and generates a texture image including the texture of the object to be measured, a position and orientation specifying unit that specifies the position and orientation of the three-dimensional scanner and the position and orientation of the texture camera, three-dimensional data generation means that generates three-dimensional shape data of the object to be measured based on the image including the pattern light generated by the scanner imaging unit and the position and orientation of the three-dimensional scanner specified by the position and orientation specifying unit, a texture image acquisition unit that receives an imaging instruction for the texture image and acquires, in association with each other, position and orientation information based on the position and orientation of the texture camera specified by the position and orientation specifying unit and the texture image generated by the texture camera, display data generation means that generates first display data for displaying the texture image acquired by the texture image acquisition unit as a two-dimensional texture image viewed from the position and orientation of the texture camera when the texture image was acquired, and second display data for displaying a three-dimensional texture image in which the texture of the object to be measured included in the texture image is applied to the three-dimensional shape data based on the position and orientation information corresponding to the texture image, a measurement processing unit that receives an input of a measurement instruction for the three-dimensional texture image displayed on the display unit when the second display data is generated by the display data generation means, and executes a measurement process of the object to be measured based on the measurement instruction and the three-dimensional shape data.

[0009] According to this configuration, since it is possible to display two-dimensional texture images and three-dimensional texture images on the display unit, by switching from a two-dimensional texture image to a three-dimensional texture image or from a three-dimensional texture image to a two-dimensional texture image as needed, it becomes possible to perform a scanning operation on the measurement object by the three-dimensional scanner while viewing the desired texture image. The switching of the texture image may be performed automatically or manually by the user. After the scanning operation of the measurement object, a measurement instruction may be input for the three-dimensional texture image, so that the measurement instruction can be easily input for a desired location on the scan data.

[0010] During the display of the first display data, the display data generation unit can switch the data to be displayed on the display unit to the second display data by receiving a drag input for changing the display position of the three-dimensional shape data. The display position of the three-dimensional shape data can be changed in the state where the data has been switched to the second display data.

[0011] During the display of the first display data, the display data generation unit can switch the data to be displayed on the display unit to the second display data by receiving an input for enlarging or reducing the display of the three-dimensional shape data. The display of the three-dimensional shape data can be enlarged or reduced in the state where the data has been switched to the second display data.

[0012] The position and orientation information based on the position and orientation of the texture camera specified by the position and orientation specifying unit can also be referred to as view information, for example.

[0013] In another aspect of the present disclosure, it is also possible to assume a three-dimensional measurement method for measuring the three-dimensional shape of an object to be measured. The three-dimensional measurement method includes a first step of specifying the position and orientation of a three-dimensional scanner having a scanner light source that irradiates pattern light and a scanner imaging unit that images the pattern light irradiated by the scanner light source to generate an image including the pattern light; a second step of specifying the position and orientation of a texture camera that images the object to be measured to generate a texture image including the texture of the object to be measured; a third step of generating three-dimensional shape data of the object to be measured based on the image including the pattern light generated by the scanner imaging unit and the position and orientation of the three-dimensional scanner specified by the position and orientation specifying unit; a fourth step of receiving an imaging instruction for the texture image, and associating and acquiring the position and orientation information based on the position and orientation of the texture camera specified in the second step and the texture image generated by the texture camera; a fifth step of generating first display data for displaying the texture image acquired in the fourth step as a two-dimensional texture image viewed from the position and orientation of the texture camera when the texture image was acquired, and second display data for displaying a three-dimensional texture image in which the texture of the object to be measured included in the texture image is applied to the three-dimensional shape data based on the position and orientation information corresponding to the texture image; and a sixth step of receiving an input of a measurement instruction for the three-dimensional texture image displayed on the display unit when the second display data is generated in the fifth step, and executing a measurement process of the object to be measured based on the measurement instruction and the three-dimensional shape data.

[0014] Furthermore, still another aspect of the present disclosure can be a three-dimensional measurement program for causing a computer to execute a three-dimensional measurement method for measuring the three-dimensional shape of an object to be measured.

Advantages of the Invention

[0015] As described above, since two-dimensional texture images and three-dimensional texture images can be displayed, the scanning operation of the object to be measured by the three-dimensional scanner can be facilitated.

Brief Description of Drawings

[0016]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Mode for Carrying Out the Invention

[0017] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. It should be noted that the following description of the preferred embodiments is merely illustrative in nature and is not intended to limit the present invention, its applications, or its uses.

[0018] FIG. 1 is a diagram showing the configuration of a three-dimensional measuring device 1 according to an embodiment of the present invention. The three-dimensional measuring device 1 is a shape measuring device that measures the three-dimensional shape and three-dimensional coordinates of a measurement object W without contacting the measurement object W, and includes a three-dimensional scanner 2 having a plurality of self-luminous markers, an imaging unit 3 that images the plurality of self-luminous markers included in the three-dimensional scanner 2, and a processing unit 4 that measures the three-dimensional shape and three-dimensional coordinates of the measurement object based on the marker image generated by the imaging unit 3 and the streak image generated by the three-dimensional scanner 2. The three-dimensional scanner 2 is separate from the imaging unit 3 and the processing unit 4, and the measurement operator can move the three-dimensional scanner 2 to the vicinity of the measurement object W located away from the imaging unit 3 and the processing unit 4 and cause the three-dimensional scanner 2 to generate a streak image.

[0019] The imaging unit 3 is an example of a position and orientation specifying unit that specifies the position and orientation of the three-dimensional scanner 2. For example, the imaging unit 3 is a unit that generates a marker image including the plurality of self-luminous markers by imaging the plurality of self-luminous markers (described later) provided on the three-dimensional scanner 2. Since a texture camera 66 (described later) is attached to the three-dimensional scanner 2, the position and orientation of the texture camera 66 can also be specified by specifying the position and orientation of the three-dimensional scanner 2. Therefore, the imaging unit 3 is also a member that specifies the position and orientation of the texture camera 66.

[0020] The marker image including the self-luminous markers of the three-dimensional scanner 2 captured by the imaging unit 3 can also be referred to as the second image. As shown in FIG. 2, the imaging unit 3 includes a base 30 and a movable imaging unit 3A that moves the field of view so that the three-dimensional scanner 2 is within the field of view and captures the self-luminous markers to measure the position and orientation of the three-dimensional scanner 2 and generates a marker image including the self-luminous markers. The movable imaging unit 3A includes a movable stage 31 supported by the base 30 and a scanner imaging camera 32 fixed to the upper part of the movable stage 31. The movable stage 31 includes a stage drive unit 31a. The stage drive unit 31a incorporates an actuator such as a motor and is configured to rotate the movable stage 31 around the vertical axis and also around the horizontal axis. By rotating the movable stage 31 around the vertical axis, the scanner imaging camera 32 rotates around the vertical axis, and by rotating the movable stage 31 around the horizontal axis, the scanner imaging camera 32 rotates around the horizontal axis. Thereby, the field of view of the scanner imaging camera 32 (schematically shown by the dashed line A in FIGS. 1 and 2) is moved to track the self-luminous markers so that the three-dimensional scanner 2, that is, a plurality of self-luminous markers included in the three-dimensional scanner 2 enter the field of view of the scanner imaging camera 32. The stage drive unit 31a is controlled by the main control unit 33 included in the imaging unit 3.

[0021] A plurality of light emitters 31b are provided at a predetermined interval on the lower part of the movable stage 31 in a two-dimensional plane, and the light emitters 31b can be switched between a lit state and an extinguished state by a lighting control unit 31c. The plurality of light emitters 31b move as the scanner imaging camera 32 and the movable stage 31 move. The lighting control unit 31c is controlled by the main body control unit 33. On the other hand, a reference camera 34 for imaging the movable imaging unit 3A is provided on the base 30. This reference camera 34 images the light emitter 31b that has been put in a lit state by the lighting control unit 31c. The reference camera 34 images the plurality of light emitters 31b provided in the movable imaging unit 3A and generates an image including the light emitter 31b. The reference camera 34 can also be referred to as a fixed imaging unit, and the image including the light emitter 31b can also be referred to as a third image. That is, a reference camera 34 for imaging the light emitter 31b that has been put in a lit state by the lighting control unit 31c is provided. Incidentally, the plurality of light emitters 31b can also be referred to as self-luminous markers provided in the movable imaging unit 3A. Markers provided in the movable imaging unit 3A may be constituted by members serving as marks other than the light emitter 31b.

[0022] An imaging unit 3 is provided with a camera image processing unit 35. The camera image processing unit 35 has an image processing circuit and controls the scanner imaging camera 32 to perform imaging at a predetermined timing. Examples of the image processing circuit include a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), and a DSP (Digital Signal Processor).

[0023] A marker image captured by the scanner imaging camera 32 and an image of the light emitter 31b captured by the reference camera 34 are input to the camera image processing unit 35.

[0024] The camera image processing unit 35 processes the marker image captured by the scanner imaging camera 32 to generate the center position information of the self-luminous marker (corresponding to the second measurement information of the present invention). Specifically, the camera image processing unit 35 performs a process of extracting the center of the self-luminous marker from the marker image. Then, based on the extracted result, the center position information of the self-luminous marker is generated. Further, as a result of the process of extracting the center of the self-luminous marker, the camera image processing unit 35 generates the position and orientation information of the self-luminous marker with respect to the movable imaging unit 3A based on the obtained center position information of the self-luminous marker.

[0025] The center position information of the self-luminous markers 71 to 77, 81 to 87, 91 to 97, and 101 to 107 is generated by the following method. First, the camera image processing unit 35 acquires the arrangement information of the respective self-luminous markers 71 to 77, 81 to 87, 91 to 97, and 101 to 107 stored in the three-dimensional scanner 2. Then, based on the arrangement information of the self-luminous markers 71 to 77, 81 to 87, 91 to 97, and 101 to 107 acquired from the three-dimensional scanner 2 and the relative three-dimensional position information between the markers included in the marker image generated by the camera image processing unit 35, when the relative position and orientation of the three-dimensional scanner 2 with respect to the imaging unit 3 are changed, it is calculated where each marker will be imaged by the imaging unit 3, and the calculated positions of each marker are matched with the marker positions in the image 102. Then, the relative position and orientation of the three-dimensional scanner 2 with respect to the imaging unit 3 where the error between the calculated positions of each marker and the marker positions in the image 102 is minimized are calculated and generated as the center position information of the self-luminous markers 71 to 77, 81 to 87, 91 to 97, and 101 to 107. That is, the camera image processing unit 35 virtually changes the position and orientation of the three-dimensional scanner 2, thereby virtually changing the arrangement information of the self-luminous markers 71 to 77, 81 to 87, 91 to 97, and 101 to 107 acquired from the three-dimensional scanner 2, calculates the position and orientation that matches the marker image generated by the camera image processing unit 35, and generates the center position information of the self-luminous markers 71 to 77, 81 to 87, 91 to 97, and 101 to 107. This calculation process of the position and orientation information may be referred to as bundle adjustment. Here, for the matching, a part of the self-luminous markers 71 to 77, 81 to 87, 91 to 97, and 101 to 107 included in the marker image may be selectively used as representative markers. The circular self-luminous markers 71 to 77, 81 to 87, 91 to 97, and 101 to 107 may become elliptical depending on the position and orientation of the three-dimensional scanner 2. Therefore, as an example, the flatness ratio, which is the ratio of the length of the long side to the length of the short side of the self-luminous markers 71 to 77, 81 to 87, 91 to 97, and 101 to 107 included in the marker image, is used. When the flatness ratio is below a predetermined value, it is not included in the calculation target, and the self-luminous markers 71 to 77, 81 to 87, 91 to 97, and 101 to 107 with a flatness ratio above the predetermined value may be used as representative markers.Also, among the marker blocks, those close to a perfect circle may be selected as representative markers. By limiting the self-luminous markers to be calculated to the representative markers in this way, the calculation speed can be improved and the decrease in measurement accuracy can be suppressed.

[0026] The center position information of the self-luminous markers 71 to 77, 81 to 87, 91 to 97, and 101 to 107 calculated here is based on the scanner imaging camera 32. Therefore, the camera image processing unit 35 calculates the position and orientation information of the scanner imaging camera 32 based on the reference camera 34, and the position and orientation information of the three-dimensional scanner 2 based on the scanner imaging camera 32, and calculates the position and orientation information of the three-dimensional scanner 2 based on the reference camera 34, thereby generating the center position information of the self-luminous markers based on the reference camera 34.

[0027] The imaging unit 3 includes a wireless communication unit 36 controlled by the main body control unit 33. The wireless communication unit 36 is a communication module or the like configured to be able to communicate with devices other than the imaging unit 3. In this example, the imaging unit 3 communicates with the three-dimensional scanner 2 via the wireless communication unit 36, and various data such as image data captured by the scanner imaging camera 32 and various signals can be transmitted and received. Further, the wireless communication unit 36 has an optical communication interface 36a and a radio wave communication interface 36b. The optical communication interface 36a is a part for performing optical communication using visible light or invisible light, and can be configured by, for example, an infrared communication interface or the like. The radio wave communication interface 36b may be a part for constructing a wireless LAN, for example, or a part capable of short-distance digital wireless communication using radio waves such as Bluetooth (registered trademark) communication. Optical communication has high directivity and accurate information transfer time as its characteristics.

[0028] The imaging unit 3 also includes a communication unit (corresponding to the third communication unit of the present invention) 37 that is controlled by the main body control unit 33. The communication unit 37 is a communication module or the like configured to be communicable with the processing unit 4. Through the communication unit 37, the imaging unit 3 can communicate with the processing unit 4, and for example, various data such as image data and various signals can be transmitted and received. The communication by the communication unit 37 may be wired communication or wireless communication. The communication unit 37 transmits the center position information of the self-luminous marker generated by the camera image processing unit 35.

[0029] The imaging unit 3 has a trigger generation unit 38 that generates identification information for identifying the timing of synchronous execution based on a measurement instruction. For example, when a measurement operator performs a predetermined measurement start operation, the main body control unit 33 of the imaging unit 3 receives the measurement start operation. When the main body control unit 33 receives the measurement start operation, it causes the trigger generation unit 38 to generate a trigger as the above identification information. The trigger is transmitted to the three-dimensional scanner 2 via, for example, the wireless communication unit 36. Note that the trigger generation unit 38 can also be referred to as a synchronization means.

[0030] In response to the generation of the trigger, the main body control unit 33 synchronizes and executes the emission of the self-luminous marker of the three-dimensional scanner 2, the imaging of the self-luminous marker of the three-dimensional scanner 2 by the movable imaging unit 3A, the lighting of the light emitter 31b of the movable stage 31, and the imaging of the light emitter 31b by the reference camera 34. Note that the light emitter 31b of the movable stage 31 may be constantly lit. Therefore, the main body control unit 33 synchronizes and executes at least the emission of the self-luminous marker of the three-dimensional scanner 2, the imaging by the movable imaging unit 3A, and the imaging by the reference camera 34. The timing of the emission of the self-luminous marker of the three-dimensional scanner 2 may be slightly earlier than the timing of the imaging by the movable imaging unit 3A. In this case as well, it is assumed that the emission of the self-luminous marker of the three-dimensional scanner 2 and the imaging by the movable imaging unit 3A are synchronized.

[0031] The communication unit 37 transmits by associating the center position information of the self-luminous marker generated by the camera image processing unit 35 with the identification information corresponding to the center position information of the self-luminous marker generated by the trigger generation unit 38. "Associating" means relating or correlating two or more pieces of information. In this case, the center position information of the self-luminous marker is associated with the identification information for distinguishing this center position information of the self-luminous marker from the center position information of other self-luminous markers. Therefore, based on the identification information, the center position information of a desired self-luminous marker can be specified. The communication unit 37 corresponds to the second transmission unit of the present invention. Incidentally, the center position information of the self-luminous marker and the identification information may be transmitted by wireless communication.

[0032] The processing unit 4 receives the position and orientation of a plurality of markers obtained by processing the marker image generated by the imaging unit 3 from the imaging unit 3, and receives the edge data of the streak image obtained by processing the streak image generated by the three-dimensional scanner 2. It is a part that measures the three-dimensional shape of the measurement object W based on the received position and orientation of the markers and the edge data.

[0033] As the method for measuring the three-dimensional shape, a conventionally well-known method can be used. Hereinafter, an example will be described. Since the plurality of light emitters 31b of the imaging unit 3 are provided on the movable stage 31 to which the scanner imaging camera 32 is fixed, the positional relationship between the plurality of light emitters 31b with respect to the scanner imaging camera 32 is known. When the scanner imaging camera 32 is moved by the stage drive unit 31a, the scanner imaging camera 32 moves within the range where those light emitters 31b can be imaged by the reference camera 34. The position and orientation of the three-dimensional scanner 2 with respect to the scanner imaging camera 32 are determined based on the marker image of the three-dimensional scanner 2 imaged by the scanner imaging camera 32.

[0034] Further, the reference camera 34 similarly determines the position and orientation of the scanner imaging camera 32 with respect to the reference camera 34 based on the images of the plurality of light emitters 31b captured. Specifically, the camera image processing unit 35 acquires the arrangement information of the light emitters 31b stored in the storage unit 39c of the imaging unit 3, processes the image of the light emitters 31b generated by the reference camera 34 based on the arrangement information of the light emitters 31b, and generates the position and orientation information of the scanner imaging camera 32 with respect to the reference camera 34. The position and orientation information of the scanner imaging camera 32 with respect to the reference camera 34 can be referred to as third measurement information.

[0035] By determining the position and orientation of the three-dimensional scanner 2 with respect to the reference camera 34 from the position and orientation of the three-dimensional scanner 2 with respect to the scanner imaging camera 32 and the position and orientation of the scanner imaging camera 32 with respect to the reference camera 34, and obtaining the coordinates of the measurement points, three-dimensional coordinate measurement, that is, measurement of the three-dimensional shape, becomes possible.

[0036] In FIG. 1, an example is shown in which the processing unit 4 is configured by a general-purpose notebook personal computer. However, it may be configured by a desktop personal computer or a controller (an example of a computer) dedicated to the three-dimensional measuring device 1, etc. In any case, by installing a program or application for realizing the functions of the three-dimensional measuring device 1, it can be used as the processing unit 4. The program installed in the processing unit 4 is a three-dimensional measurement program, and this three-dimensional measurement program can cause the computer to execute a three-dimensional measurement method. The place where the three-dimensional measurement program is installed is not limited to the processing unit 4, and it may be, for example, another storage device (not shown).

[0037] The processing unit 4 may be separate from the imaging unit 3 or may be integrated with the imaging unit 3. Also, a part of the processing unit 4 may be incorporated into the imaging unit 3, or a part of the imaging unit 3 may be incorporated into the processing unit 4.

[0038] As shown in FIG. 2, the processing unit 4 includes a control unit 40, a monitor (display unit) 41, and an operation input unit 42. The monitor 41 is composed of a liquid crystal display, an organic EL display, etc. configured to be able to display various images, user interfaces, etc.

[0039] The operation input unit 42 is a part where the user performs various input operations. The operation input unit 42 is composed of, for example, a keyboard, a mouse, etc.

[0040] The control unit 40 includes a control section 43, a display control section 44, a storage section 45, and a communication section 46. The display control section 44 is a part that controls the monitor 41 based on the signal output from the control section 43, and causes the monitor 41 to display various images, user interfaces, etc. Operations performed by the user on the user interface are acquired by the control section 43 based on the signal output from the operation input unit 42.

[0041] The storage section 45 may be a ROM, or may be a solid state drive, a hard disk drive, etc. The storage section 45 stores the arrangement information of each self-luminous marker in the marker block of the three-dimensional scanner 2. The marker block and the arrangement information of each self-luminous marker include the distance between the marker blocks, information indicating the relative positional relationship of the self-luminous markers provided on each marker block, etc.

[0042] Also, the communication section 46 of the processing unit 4 is controlled by the control section 43. The communication section 46 is a communication module, etc. configured to be able to communicate with the communication section 37 of the imaging unit 3.

[0043] The three-dimensional scanner 2 is configured such that a measurement operator can hold it with one or both hands and freely move it while measuring the shape of the measurement object W, and it is a handheld and portable scanner. The power supply may be supplied from the outside, or a battery may be built in and supplied from the battery. In the present embodiment, the front, rear, left, right, top, and bottom of the three-dimensional scanner 2 are defined as shown in FIGS. 3 to 7. That is, when the measurement operator holds the three-dimensional scanner 2 by hand, the side located on the right is called the right, and the side located on the left is called the left. The front of the three-dimensional scanner 2 is the side facing the measurement object W, and the rear side of the three-dimensional scanner 2 is the side opposite to the side facing the measurement object W. The top of the three-dimensional scanner 2 is the upper side in a state where the gripping portion 112 described later is gripped in a natural posture as determined, and the bottom of the three-dimensional scanner 2 is the lower side in a state where the gripping portion 112 is gripped in a natural posture as determined. However, as described above, since the three-dimensional shape of the measurement object W can be measured while holding the three-dimensional scanner 2 by hand and moving it, the orientation of the three-dimensional scanner 2 may be upside down, or the upper side may be located on the right or left, or the rear side may be located on the upper or lower side.

[0044] The three-dimensional scanner 2 includes a scanner main body 20, a first marker block 21, a second marker block 22, a third marker block 23, and a fourth marker block 24. Although details will be described later, the first to fourth marker blocks 21 to 24 each have self-luminous markers facing in a plurality of directions.

[0045] The scanner main body 20 includes a first arm portion 51 extending upward from the central portion, a second arm portion 52 extending downward from the central portion, a third arm portion 53 extending leftward from the central portion, and a fourth arm portion 54 extending rightward from the central portion.

[0046] The first marker block 21 is attached to the tip of the first arm portion 51, the second marker block 22 is attached to the tip of the second arm portion 52, the third marker block 23 is attached to the tip of the third arm portion 53, and the fourth marker block 24 is attached to the tip of the fourth arm portion 54.

[0047] As shown in FIG. 8, the scanner main body 20 has a scanner unit 60 and an optical base 61. As shown in FIG. 7, the scanner unit 60 has a first scanner light source 62, a second scanner light source 63, a first scanner imaging unit 64, a second scanner imaging unit 65, and a texture camera 66. As shown in FIG. 8, the upper portion of the optical base 61 above the central portion is a portion constituting the first arm portion 51 and serves as an upper support portion 61a that supports the first marker block 21. Therefore, the first marker block 21 is attached to the upper end portion of the upper support portion 61a. The lower portion of the optical base 61 below the central portion is a portion constituting the second arm portion 52 and serves as a lower support portion 61b that supports the second marker block 22. Therefore, the second marker block 22 is attached to the lower end portion of the lower support portion 61b.

[0048] At the vertical center of the optical base 61, that is, in the portion between the upper support portion 61a and the lower support portion 61b, two first scanner light sources 62 (shown in FIG. 7) are attached at intervals in the horizontal direction. The two first scanner light sources 62 are multi-line light sources that irradiate a plurality of linear lights in the measurement direction (forward), and the light emitting surfaces are arranged so as to face the measurement object W at the time of measurement. The light irradiated by the first scanner light source 62 can be called multi-line light, and the multi-line light is included in the pattern light.

[0049] Above the first scanner light source 62 at the vertical center of the optical base 61, a second scanner light source 63 is attached. The second scanner light source 63 is a single-line light source that irradiates a single linear light in the measurement direction (forward), and the light-emitting surface is arranged to face the object to be measured W during measurement. The light irradiated by the second scanner light source 63 can be called single-line light, and the single-line light is also included in the pattern light.

[0050] The first scanner light source 62 and the second scanner light source 63 have a laser light source that irradiates laser light, but the type of the light source is not particularly limited. Also, in this example, a total of three scanner light sources 62 and 63 are provided, but it is not limited to this, and one or more scanner light sources may be provided. Also, the type of the pattern light is not particularly limited, and the scanner light source may irradiate pattern light other than multi-line light and single-line light.

[0051] The first scanner imaging unit 64 and the second scanner imaging unit 65 have a light-receiving element such as a CMOS sensor, and an optical system for forming an image of the light incident from the outside on the light-receiving surface of the light-receiving element. The first scanner imaging unit 64 is attached to the upper part of the optical base 61, which is a part away from the scanner light sources 62 and 63 upward. The second scanner imaging unit 65 is attached to the lower part of the optical base 61, which is a part away from the scanner light sources 62 and 63 downward. The first scanner imaging unit 64 and the second scanner imaging unit 65 are arranged such that their respective optical axes face the irradiation direction of the pattern light by the scanner light sources 62 and 63. Thereby, it becomes possible to image the pattern light irradiated by the scanner light sources 62 and 63 in the measurement direction and generate a bright line image including the pattern light. The bright line image including the pattern light can also be called the first image.

[0052] Since the first scanner imaging unit 64 is attached to the upper part of the optical base 61 and the second scanner imaging unit 65 is attached to the lower part of the optical base 61, the distance between the first scanner imaging unit 64 and the second scanner imaging unit 65 can be ensured to be long, and the accuracy of the stereo measurement method can be improved. That is, the distance between the optical axes of the first scanner imaging unit 64 and the second scanner imaging unit 65 is known, and the corresponding points of the respective images generated by simultaneously imaging the pattern light irradiated from the first scanner light source 62 or the second scanner light source 63 with the first scanner imaging unit 64 and the second scanner imaging unit 65 are obtained, and by using the stereo measurement method, the three-dimensional coordinates of the corresponding points can be obtained. The stereo measurement method may be passive stereo using the first scanner imaging unit 64 and the second scanner imaging unit 65, or active stereo using one scanner imaging unit. In particular, when the measurement object W is specularly reflected or when measuring a deep hole, etc., there may be a case where one of the images generated by the first scanner imaging unit 64 and the second scanner imaging unit 65 does not contain pattern light. In such a case, the three-dimensional coordinates may be calculated by the active stereo method based on the positional relationship between the scanner imaging unit corresponding to the image in which the pattern light is photographed and the scanner light source.

[0053] The texture camera 66 has a light receiving element such as a CMOS sensor capable of acquiring a color image, for example, and an optical system for imaging the light incident from the outside on the light receiving surface of the light receiving element. The texture camera 66 is attached to the optical base 61 between the first scanner imaging unit 64 and the second scanner imaging unit 65. The texture camera 66 is arranged such that the optical axis faces the measurement object W at the time of measurement, and images the measurement object W to generate a texture image including the texture of the measurement object W. The texture image includes information such as the color and texture of the surface of the measurement object W.

[0054] The first to fourth marker blocks 21 to 24 have the same structure. The first marker block 21 has first to seventh self-luminous markers 71 to 77 facing in a plurality of directions.

[0055] The second to fourth marker blocks 22 to 24 are also configured in the same manner as the first marker block 21. That is, as shown in FIGS. 3 to 8, the second marker block 22 has the first to seventh self-luminous markers 81 to 87, the third marker block 23 has the first to seventh self-luminous markers 91 to 97, and the fourth marker block 24 has the first to seventh self-luminous markers 101 to 107.

[0056] As shown in FIG. 7, the first self-luminous marker 71 of the first marker block 21 and the first self-luminous marker 81 of the second marker block 22 are arranged so as to be displaced around the straight line B, and the optical axis of the first self-luminous marker 71 of the first marker block 21 and the optical axis of the first self-luminous marker 81 of the second marker block 22 are directed in different directions. This is because a plurality of side surfaces formed on the second marker block 22 are arranged so as to be displaced in position around an axis extending in the first direction (vertical direction) with respect to a plurality of side surfaces formed on the first marker block 21. Similarly, a plurality of side surfaces formed on the fourth marker block 24 are arranged so as to be displaced in position around an axis extending in the second direction (left-right direction) with respect to a plurality of side surfaces formed on the third marker block 23. This makes it difficult for multiple solutions to occur during the processing of the marker image described later.

[0057] The scanner main body 20 includes a resin exterior member 110 that covers the optical base 61. The front portion of the exterior member 110 has a scanner cover portion 111 that covers the first scanner light source 62, the second scanner light source 63, the first scanner imaging unit 64, and the second scanner imaging unit 65. The rear portion of the exterior member 110 has a grip portion 112 that is gripped by the measurement operator.

[0058] For example, as shown in FIG. 8, the grip portion 112 has a vertically long shape, the upper end portion thereof is integrated with the main body portion of the exterior member 110, and it is provided at a position away from the optical base 61 on the side opposite to the measurement direction (rear side).

[0059] As shown in FIGS. 5 and 8, at the upper end of the gripping portion 112, there are provided a scanner display portion 113 for displaying information regarding measurement results by the scanner portion 60, a setting screen, etc., and an operation portion 114 for operating the scanner portion 60. The scanner display portion 113 is composed of a liquid crystal display, an organic EL display, etc., and is arranged such that the display surface is inclined. Further, the display surface faces the measurement subject side, and the three-dimensional scanner 2 can be moved while viewing the display content of the scanner display portion 113. This scanner display portion 113 is a display portion incorporated in the three-dimensional scanner 2, a display portion provided integrally with the three-dimensional scanner 2, or a display portion attached inseparably from the main body portion of the three-dimensional scanner 2 during operation, that is, a so-called built-in type display portion. The scanner display portion 113 is above the gripping portion 112 and is provided in the vicinity of the portion where the gripping portion 112 and the scanner main body 20 are connected. Also, the display surface of the scanner display portion 113 is embedded above the gripping portion 112 so as to face the opposite direction to the direction in which light is irradiated from the scanner light sources 62, 63. Further, as shown in FIG. 8, the periphery of the scanner display portion 113 is covered with an exterior member 110, and is arranged continuously integrally with the exterior housing 110 that covers the gripping portion 112. The operation portion 114 is also an operation portion incorporated in the three-dimensional scanner 2 in the same manner as the scanner display portion 113, that is, a so-called built-in type operation portion. The operation portion 114 is arranged below the scanner display portion 113, and the periphery of the operation portion 114 is covered with the exterior housing 110. Here, the exterior housing 110 is provided with irregularities corresponding to the shape of the operation portion 114, and the operation portion 114 is also covered with the exterior housing 110.

[0060] On the display surface side of the scanner display portion 113, a touch panel 113a capable of touch operation is also provided. The operation portion 114 is composed of a plurality of operation buttons including, for example, a measurement start button, a measurement stop button, etc., and is arranged below the scanner display portion 113. The touch panel 113a can also be made a part of the operation portion.

[0061] Next, the circuit of the three-dimensional scanner 2 will be described with reference to FIG. 16. The three-dimensional scanner 2 includes a display control unit 140, a marker lighting control unit 141, a scanner control unit 142, and a storage unit 143. The display control unit 140 is a part that controls the scanner display unit 113 based on a signal output from the scanner control unit 142, and causes the scanner display unit 113 to display various images, user interfaces, and the like. Operations performed by the user on the scanner display unit 113 are acquired by the scanner control unit 142 based on signals output from the touch panel 113a.

[0062] The marker lighting control unit 141 is a part that controls the self-luminous markers 71 to 77, 81 to 87, 91 to 97, 101 to 107 (only 71 is shown in FIG. 16). The self-luminous markers 71 to 77, 81 to 87, 91 to 97, 101 to 107 can be switched between a lit state and an extinguished state by the marker lighting control unit 141. The marker lighting control unit 141 is controlled by the scanner control unit 142. The storage unit 143 can temporarily store programs, images captured by the scanner unit 60, and the like.

[0063] The three-dimensional scanner 2 includes a wireless communication unit (first communication unit) 144 controlled by the scanner control unit 142. The wireless communication unit 144 is a communication module or the like configured to be capable of communicating with devices other than the three-dimensional scanner 2. In this example, the three-dimensional scanner 2 communicates with the imaging unit 3 via the wireless communication unit 144, and transmits and receives various data such as image data captured by the scanner unit 60 and various signals, and receives display data and the like generated by the display data generation unit 410 described later.

[0064] The three-dimensional scanner 2 is provided with a motion sensor 145. The motion sensor 145 is composed of sensors that detect the acceleration and angular velocity of the three-dimensional scanner 2, and the detected values are output to the scanner control unit 142 and used for various arithmetic processes. For example, by using the values output from the motion sensor 145, the initial solution of the posture of the three-dimensional scanner 2, that is, the postures of the first to fourth marker blocks 21 to 24, can be obtained to improve the matching accuracy and the processing speed during posture calculation can be improved. The processing using the values output from the motion sensor 145 may be executed by the imaging unit 3 or the processing unit 4.

[0065] The three-dimensional scanner 2 includes a scanner light source control unit 146 and a scanner image processing unit 147. The scanner light source control unit 146 is a part that controls the first scanner light source 62 and the second scanner light source 63. The first scanner light source 62 and the second scanner light source 63 can be switched between a lit state and an extinguished state by the scanner light source control unit 146. The scanner light source control unit 146 is controlled by the scanner control unit 142. Further, the scanner image processing unit 147 controls the first scanner imaging unit 64, the second scanner imaging unit 65, and the texture camera 66 to execute imaging at a predetermined timing. Images captured by the first scanner imaging unit 64, the second scanner imaging unit 65, and the texture camera 66 are input to the scanner image processing unit 147. The scanner image processing unit 147 executes various image processes such as extraction of edge data on the input images.

[0066] That is, the scanner image processing unit 147 generates edge data (corresponding to the first measurement information of the present invention) by performing edge extraction processing on the streak image generated by the first scanner imaging unit 64 or the second scanner imaging unit 65. When the first scanner light source 62 irradiates multi-line light, the first scanner imaging unit 64 and the second scanner imaging unit 65 generate multi-line images. The scanner image processing unit 147 processes the multi-line images to generate edge data.

[0067] The wireless communication unit 144 transmits by associating the edge data generated by the scanner image processing unit 147 with the identification information corresponding to the edge data generated by the trigger generation unit 38. That is, the edge data is associated with the identification information for distinguishing this edge data from other edge data. Therefore, based on the identification information, the desired edge data can be specified. The wireless communication unit 144 corresponds to the first transmission unit of the present invention. Incidentally, the edge data and the identification information may be transmitted by wired communication.

[0068] Further, when the trigger generated by the trigger generation unit 38 of the imaging unit 3 is transmitted to the three-dimensional scanner 2, the scanner control unit 142 of the three-dimensional scanner 2 receives the trigger. When the scanner control unit 142 receives the trigger, the scanner light source control unit 146 executes irradiation of pattern light from the first scanner light source 62 or the second scanner light source 63, the scanner image processing unit 147 executes imaging by the first scanner imaging unit 64 and the second scanner imaging unit 65, and the marker lighting control unit 141 causes the self-luminous markers 71 to 77, 81 to 87, 91 to 97, and 101 to 107 to emit light. The irradiation of pattern light from the first scanner light source 62 or the second scanner light source 63, the imaging by the first scanner imaging unit 64 and the second scanner imaging unit 65, and the emission of the self-luminous markers 71 to 77, 81 to 87, 91 to 97, and 101 to 107 are synchronized.

[0069] In short, the main body control unit 33 of the imaging unit 3 and the scanner control unit 142 of the three-dimensional scanner 2 cooperate with each other, and in response to the trigger being generated by the trigger generation unit 38, the irradiation of pattern light from the scanner light sources 62 and 63, the imaging by the scanner imaging units 64 and 65, the emission of the self-luminous markers 71 to 77, 81 to 87, 91 to 97, and 101 to 107, and the imaging by the movable imaging unit 3A are synchronized.

[0070] The three-dimensional scanner 2 includes an indicator lamp 148 and a communication control unit 149. The indicator lamp 148 displays the operating state of the three-dimensional scanner 2 and is controlled by the scanner control unit 142. The communication control unit 149 is a part that performs execution processing of communication such as image data.

[0071] The processing unit 4 shown in FIG. 2 is a three-dimensional data generation means for generating three-dimensional shape data of the measurement object W based on an image including pattern light generated by the scanner imaging units 64 and 65 and the position and orientation of the three-dimensional scanner 2 specified by the imaging unit 3. For example, the processing unit 4 can generate a point cloud indicating the three-dimensional shape of the measurement object W based on the edge data generated by the scanner image processing unit 147, the center position information of the self-luminous marker generated by the camera image processing unit 35, and the position and orientation information of the scanner imaging camera 32. The point cloud data indicating the three-dimensional shape of the measurement object W is an example of display data indicating the three-dimensional shape of the measurement object W.

[0072] Specifically, the processing unit 4 includes a three-dimensional data generation unit 43a. When imaging is performed by the three-dimensional scanner 2 and the imaging unit 3, the processing unit 4 receives the edge data generated by the scanner image processing unit 147 of the three-dimensional scanner 2, the identification information corresponding to the edge data, the center position information of the self-luminous marker generated by the camera image processing unit 35 of the imaging unit 3, and the identification information corresponding to the center position information of the self-luminous marker. After receiving each data and information, the three-dimensional data generation unit 43a generates point cloud data indicating the three-dimensional shape of the measurement object W based on the received edge data, the identification information corresponding to the edge data, the center position information of the self-luminous marker, and the identification information corresponding to the center position information of the self-luminous marker.

[0073] In this example, as shown in FIG. 2, the imaging unit 3 is provided with a memory 39a that sequentially stores the edge data generated by the scanner image processing unit 147, and a correlation unit 39b that correlates the edge data with the center position information of the self-luminous marker based on the identification information. For example, when sequentially measuring a plurality of measurement objects W or when sequentially measuring different parts of the same measurement object W, the scanner image processing unit 147 generates a plurality of edge data. The plurality of generated edge data are transmitted from the wireless communication unit 144 of the three-dimensional scanner 2 to the imaging unit 3 in a state where different identification information is associated with each of them. The plurality of edge data transmitted from the wireless communication unit 144 of the three-dimensional scanner 2 are stored in the memory 39a of the imaging unit 3 in a state where the identification information is associated with them.

[0074] When causing the three-dimensional data generation unit 43a to generate a point group indicating a three-dimensional shape, the correlation unit 39b specifies the center position information of the self-luminous marker to be transmitted to the processing unit 4. The correlation unit 39b specifies, from among the plurality of edge data stored in the memory 39a, the edge data having the identification information associated with the specified center position information of the self-luminous marker. Thereafter, the correlation unit 39b correlates the specified edge data with the center position information of the self-luminous marker. The communication unit 37 of the imaging unit 3 transmits the edge data specified by the correlation unit 39b and the center position information of the self-luminous marker in a correlated state to the processing unit 4. That is, since the generation of the center position information of the self-luminous marker and the generation of the edge data have different processing contents, there may be a difference in the timing at which the processing ends between the two. However, by synchronizing with the trigger ID as in this example, regardless of the deviation in the timing at which the processing ends, a point group indicating a three-dimensional shape can be generated between corresponding ones.

[0075] As shown in FIG. 2, the processing unit 4 includes a measurement setting unit 48 that receives at least one of the types of pattern light irradiated by the scanner light sources 62 and 63 of the three-dimensional scanner 2 and the exposure time settings of the scanner imaging units 64 and 65. The types of pattern light include multi-line light and single-line light. The settings for the type of pattern light and the exposure time can be received via a setting screen described later, and this setting process will be described later.

[0076] The control unit 43 of the processing unit 4 is a measurement control unit that controls the scanner light sources 62 and 63 or the scanner imaging units 64 and 65 based on the settings received by the measurement setting unit 48. When the measurement setting unit 48 receives the setting of multi-line light, information (setting information) indicating that multi-line light is set is written to the measurement setting unit 48. When single-line light is set, information (setting information) indicating that single-line light is set is written to the measurement setting unit 48. Further, when the exposure time is set, the set exposure time (setting information) is written to the measurement setting unit 48.

[0077] The control unit 43 controls the scanner light sources 62 and 63 or the scanner imaging units 64 and 65 based on the setting information written to the measurement setting unit 48. For example, when multi-line light is set, when the control unit 43 reads from the measurement setting unit 48 information indicating that multi-line light is set, the read setting information is transmitted to the three-dimensional scanner 2 via the communication unit 46. The scanner light source control unit 146 of the three-dimensional scanner 2 controls the first scanner light source 62 so that multi-line light is irradiated. When single-line light is set, the scanner light source control unit 146 of the three-dimensional scanner 2 controls the second scanner light source 63 so that single-line light is irradiated.

[0078] Also, in the case of setting the exposure time, the control unit 43 reads the set exposure time from the measurement setting unit 48. The control unit 43 transmits the read exposure time to the three-dimensional scanner 2 via the communication unit 46. The scanner image processing unit 147 of the three-dimensional scanner 2 controls the scanner imaging units 64 and 65 so that the set exposure time is achieved.

[0079] Next, the display process of the scanner display unit 113 of the three-dimensional scanner 2 will be described. When the processing unit 4 generates display data indicating the three-dimensional shape of the measurement object W, the generated display data is transmitted by the communication unit (corresponding to the second communication unit of the present invention) 46. The wireless communication unit 144 of the three-dimensional scanner 2 receives the display data transmitted via the communication unit 46 of the processing unit 4. The scanner display unit 113 displays a display screen generated based on the display data received via the wireless communication unit 144. When the three-dimensional scanner 2 is connected to the imaging unit 3 or the processing unit 4 via a communication cable, the scanner display unit 113 may display a display screen generated based on the display data received via the communication control unit 149. Although the case where the three-dimensional scanner 2 wirelessly communicates with at least one of the imaging unit 3 and the processing unit 4 will be mainly described, wired communication via a communication cable may also be used.

[0080] In a more specific form, the processing unit 4 receives the edge data generated by the scanner image processing unit 147 of the three-dimensional scanner 2 and transmitted via the wireless communication unit 144, and the center position information of the self-luminous markers 71 to 77, 81 to 87, 91 to 97, 101 to 107 generated by the camera image processing unit 35 of the imaging unit 3 and transmitted via the wireless communication unit 36. Based on the received edge data and the center position information of the self-luminous markers 71 to 77, 81 to 87, 91 to 97, 101 to 107, the processing unit 4 generates display data indicating the three-dimensional shape of the measurement object W. The display data is generated every time imaging is completed, and the processing unit 4 transmits the generated display data to the three-dimensional scanner 2.

[0081] In addition, the scanner display unit 113 displays a setting screen 200 (Fig. 10) that accepts at least one of the types of pattern light irradiated by the above-described scanner light sources 62 and 63 and the exposure time settings of the scanner imaging units 64 and 65. The setting screen 200 is a so-called user interface screen, which is generated by the display control unit 140 and the scanner control unit 142 of the three-dimensional scanner 2 and displayed on the scanner display unit 113. The information required for generating the setting screen 200 may be transmitted from the processing unit 4 or may be generated by the imaging unit 3.

[0082] The setting screen 200 is provided with a pattern light setting area 201 for setting the type of pattern light, an exposure time setting area 202 for setting the exposure time, and a resolution setting area 203 for setting the resolution. The pattern light setting area 201 is provided with a first button 201a for setting multi-line light and a second button 201b for setting single-line light. When the measurement operator presses the first button 201a, the touch panel 113a detects that the first button 201a has been pressed, and the first button 201a is displayed in a form that allows the pressed state to be determined. At the same time, the scanner control unit 142 transmits the detected result to the processing unit 4. The control unit 43 of the processing unit 4 that has received the detected result writes information indicating that multi-line light has been set to the measurement setting unit 48. In addition, the scanner control unit 142 determines whether the current setting is multi-line light. If the current setting is not multi-line light, the scanner control unit 142 controls the first scanner light source unit 62 so that multi-line light is irradiated from the first scanner light source unit 62. Similarly, when the second button 201b is pressed, the second button 201b is displayed in a form that allows the pressed state to be determined, and the scanner control unit 142 transmits the detected result to the processing unit 4. The control unit 43 of the processing unit 4 that has received the detected result writes information indicating that single-line light has been set to the measurement setting unit 48. In addition, the scanner control unit 142 determines whether the current setting is single-line light. If the current setting is not single-line light, the scanner control unit 142 controls the second scanner light source unit 63 so that single-line light is irradiated from the second scanner light source unit 63.

[0083] In the exposure time setting area 202, there are provided a decrease button 202a to be operated when shortening the exposure time, an increase button 202b to be operated when lengthening the exposure time, and an exposure time display section 202c for displaying the set exposure time numerically. The measurement operator can easily set a desired exposure time by operating the increase button 202b or the decrease button 202a while looking at the exposure time displayed on the exposure time display section 202c. The scanner control section 142 transmits the set exposure time to the processing section 4. The control section 43 of the processing section 4 that has received the exposure time writes the exposure time into the measurement setting section 48. Further, the scanner control section 142 controls the scanner imaging sections 64 and 65 based on the set exposure time. Also, an automatic button 202d may be provided. When this automatic button 202d is operated, the three-dimensional measurement device 1 executes a process of automatically obtaining an optimal exposure time, and the obtained exposure time is automatically set.

[0084] In the resolution setting area 203, there are provided a low-resolution button 203a to be operated when increasing the decimation amount and decreasing the resolution when generating a point cloud, a high-resolution button 203b to be operated when decreasing the decimation amount and increasing the resolution when generating a point cloud, and a resolution display section 203c for displaying the set resolution. The measurement operator can set the desired resolution by operating the low-resolution button 203a or the high-resolution button 203b. The scanner light source control section 146 transmits the set resolution to the processing section 4. The control section 43 of the processing section 4 that has received the resolution writes the resolution into the measurement setting section 48. The three-dimensional data generation section 43a generates a point cloud so as to have the resolution written into the measurement setting section 48. The resolution can be set stepwise, for example, as "high resolution", "standard", "low resolution", etc.

[0085] As described above, in the setting screen 200 as shown in FIG. 10, it is possible to set both the settings for the three-dimensional scanner 2 and the settings for the processing unit 4. Here, when the settings for the three-dimensional scanner 2 are made, based on the setting information received on the setting screen 200, the scanner control unit 142 controls at least one of the scanner imaging units 64, 65 and the scanner light sources 62, 63. Also, the setting information of the three-dimensional scanner 2 received on the setting screen 200 is transmitted to the processing unit 4 and is used at the time of point cloud generation. Further, when the settings for the processing unit 4 are made on the setting screen 200, the scanner control unit 142 transmits the setting information received on the setting screen 200 to the processing unit 4. That is, the setting items set on the setting screen 200 may include setting items that control the operation of the three-dimensional scanner 2 and transfer the setting items to the processing unit, and setting items that transfer the setting items to the processing unit without controlling the operation of the three-dimensional scanner 2.

[0086] An example of the procedure for reflecting the above-described setting information will be described with reference to FIG. 11. In step S11, pressing (operation) of the setting button is detected. The setting button is the first button 201a, the second button 201b, the decrease button 202a, the increase button 202b, etc. shown in FIG. 10. In step S12, it is transmitted to the imaging unit 3 that the setting button has been pressed and the setting information has been received (including the setting information). In step S13, the imaging unit 3 receives that the setting information has been received. In step S14, the imaging unit 3 transmits to the processing unit 4 that the setting information has been received. In step S15, the processing unit 4 receives that the setting information has been received. In step S16, the setting information is reflected.

[0087] The three-dimensional data generation unit 43a of the processing unit 4 shown in FIG. 2 generates new display data indicating the three-dimensional shape of the measurement object W based on a new image including pattern light generated by the scanner imaging units 64 and 65 controlled based on the setting information written in the measurement setting unit 48, and the position and orientation of the three-dimensional scanner 2 specified by the imaging unit 3. That is, when the above-described setting operation is performed by the measurement operator and the exposure time is changed, the scanner imaging units 64 and 65 are controlled so that the exposure time after the change is obtained, and thus a new image different from the image generated before the change is generated. The display data generated based on this new image and the position and orientation of the three-dimensional scanner 2 is different from the display data before the change. Therefore, the processing unit 4 transmits the new display data after the exposure time change to the three-dimensional scanner 2. As a result, the scanner display unit 113 can display a display screen generated based on the new display data transmitted from the processing unit 4, so that the measurement operator can determine whether the exposure time after the change is appropriate just by looking at the scanner display unit 113 of the three-dimensional scanner 2. Similarly, in the case of the setting of the pattern light, it can be determined whether the pattern light after the change is appropriate.

[0088] FIG. 12 is a diagram showing a first example of a shape display screen 210 displaying a point cloud indicating the three-dimensional shape of the measurement object W. FIG. 12 shows an example in which a point cloud is shown as a measurement result of the measurement object W irradiated with a multi-line light source. The scanner display unit 113 can display a shape display screen 210 having a display area 211 for distance information representing the distance between the measurement object W and the three-dimensional scanner 2. The distance between the measurement object W and the three-dimensional scanner 2 can also be referred to as a working distance. Therefore, the display area 211 for distance information is also a working distance display area.

[0089] On the shape display screen 210, a viewpoint fixation button 500, a texture photographing button 501, a setting button 502, a scan stop button 503, and a scan start button 504 are provided. The viewpoint fixation button 500 is a button to be operated when fixing the viewpoint of the image displayed on the shape display screen 210. The texture photographing button 501 is a button to be operated when acquiring a texture image with the texture camera 66, and when the texture photographing button 501 is operated, a trigger signal for texture acquisition is generated. The setting button 502 is a button to be operated when performing various settings, and when the setting button 502 is operated, a setting screen (not shown) is displayed and various setting operations can be accepted. The scan stop button 503 is a button to be operated when stopping the scan by the three-dimensional scanner 2. The scan start button 504 is a button to be operated when starting the scan by the three-dimensional scanner 2.

[0090] In the display area 211 for distance information, it may display whether the distance between the measurement object W and the three-dimensional scanner 2 is relatively close or far, or may display the distance numerically. In this example, the distance between the measurement object W and the three-dimensional scanner 2 is displayed in a color bar format. Also, on the shape display screen 210, a scale change section 212 is provided. When the measurement operator operates the scale change section 212, the three-dimensional shape of the displayed measurement object W is enlarged or reduced.

[0091] FIG. 13 and FIG. 14 show a second example and a third example of a shape display screen 210 that displays a point cloud indicating the three-dimensional shape of the object W to be measured, and shows the same object W to be measured shown in FIG. 12 from another angle when measured multiple times. As described above, it can be seen that as the number of measurements of the same object W to be measured increases, the obtained point cloud increases, and it is also possible to grasp where the unmeasured portions (white portions in the object W to be measured) are. FIG. 15 shows a fourth example of the shape display screen 210 that displays a point cloud indicating the three-dimensional shape of the object W to be measured. By further increasing the number of measurements, it can be seen that the obtained point cloud becomes denser and the unmeasured portions decrease. Therefore, the measurement operator can easily confirm matters such as whether the working distance is appropriate, whether the pattern light can be irradiated on the portion to be measured in the object W to be measured, and the current scan completion range just by looking at the scanner display unit 113. The shape display screen 210 can also be referred to as an image indicating the measurement range by the three-dimensional scanner 2 or an image indicating the measurement completion area by the three-dimensional scanner 2.

[0092] The scanner display unit 113 can display a point cloud indicating the three-dimensional shape of the object W to be measured with the viewpoint fixed. The process when the viewpoint is fixed will be described with reference to FIG. 16. In step S21, it is detected that the viewpoint fixing button included in the operation unit 114 of the three-dimensional scanner 2 has been pressed. In step S22, the fact that the viewpoint fixing button has been pressed is transmitted to the imaging unit 3. In step S23, the imaging unit 3 receives the fact that the viewpoint fixing button has been pressed. In step S24, the imaging unit 3 transmits the fact that the viewpoint fixing button has been pressed to the processing unit 4. In step S25, the processing unit 4 receives the fact that the viewpoint fixing button has been pressed. In step S26, the setting for fixing the viewpoint is reflected.

[0093] In step S27, the viewpoint of the viewer is fixed. In step S28, display data is created with the viewpoint of the viewer fixed. In step S29, the display data created in step S28 is transmitted to the three-dimensional scanner 2 via the imaging unit 3. In step S30, the three-dimensional scanner 2 receives the display data. In step S31, the screen displayed on the scanner display unit 113 is updated based on the display data received in step S30.

[0094] FIG. 17 shows an example of a display screen 210 showing difference information representing the difference between the CAD data of the measurement object W and the three-dimensional shape of the measurement object W generated by the three-dimensional data generation unit 43a. The CAD data of the measurement object W can be input to the processing unit 4 from the outside. The control unit 43 calculates the difference between the CAD data and the three-dimensional shape data after aligning the origin on the CAD data with the origin of the three-dimensional shape data generated by the three-dimensional data generation unit 43a. On the display screen 210, the difference information between the CAD data and the three-dimensional shape data can be displayed in a heat map format. For example, in FIG. 17, it can be displayed so that the larger the difference, the brighter it becomes, or conversely, the larger the difference, the darker it becomes. That is, since the scanner display unit 113 can display the display screen 210 showing the difference information representing the difference between the CAD data and the three-dimensional shape data obtained by measurement, the measurement operator can check the difference information on the three-dimensional scanner 2.

[0095] FIG. 18 is a diagram showing an example of the display screen 210 when the texture of the object W to be measured is reflected. The texture of the object W to be measured can be acquired as a texture image (color image) by the texture camera 66 of the three-dimensional scanner 2. The texture image is transmitted to the processing unit 4. The control unit 43 of the processing unit 4 generates superimposed display data for superimposing the texture image on a point group showing the three-dimensional shape of the object W to be measured generated based on the display data. For example, the control unit 43 can generate the superimposed display data by matching the origin of the texture image with the origin of the three-dimensional shape data generated by the three-dimensional data generation unit 43a and superimposing the texture image on the three-dimensional shape data. The generated superimposed display data is transmitted to the three-dimensional scanner 2 and displayed on the scanner display unit 113. That is, the scanner display unit 113 displays the display screen 210 in which the color image of the object to be measured generated by the texture camera 66 is superimposed on a point group showing the three-dimensional shape of the object W to be measured generated based on the display data.

[0096] At this time, the position and orientation information of the texture camera 66 and the texture image can be acquired in association with each other. Specifically, as shown in FIG. 9, the three-dimensional scanner 2 includes a texture image acquisition unit 67. The texture image acquisition unit 67 receives a texture image generated by the texture camera 66 and output from the texture camera 66. The texture image acquisition unit 67 also receives position and orientation information based on the position and orientation of the texture camera 66 specified by the imaging unit 3. The texture image acquisition unit 67 acquires, in association with each other, the position and orientation information based on the position and orientation of the texture camera specified by the imaging unit 3 and the texture image generated by the texture camera 66, for example, when the texture shooting button 501 shown in FIG. 12 is operated to receive an imaging instruction for the texture image. The position and orientation information based on the position and orientation of the texture camera 66 can also be, for example, the view information or the viewpoint information of the texture camera 66. The texture image acquisition unit 67 can acquire not only a single texture image but also a plurality of texture images associated with the position and orientation information based on the position and orientation of the texture camera specified by the imaging unit 3.

[0097] The processing unit 4 includes a display data generation unit 410. The display data generation unit 410 is a part that enables the display of two-dimensional and three-dimensional texture images. Specifically, the display data generation unit 410 generates first display data for displaying the texture image acquired by the texture image acquisition unit 67 as a two-dimensional texture image viewed from the position and orientation of the texture camera 66 when the texture image was acquired, and second display data for displaying a three-dimensional texture image in which the texture of the measurement object W included in the texture image is applied to the three-dimensional shape data based on the position and orientation information corresponding to the texture image.

[0098] When a plurality of texture images are acquired by the texture image acquisition unit 67, the display data generation unit 410 can generate second display data for displaying a three-dimensional texture image in which the texture of the measurement object W included in the texture image is applied to the three-dimensional shape data based on the position and orientation information corresponding to each texture image.

[0099] The display data generation unit 410 acquires the exposure amount of the texture camera 66 and generates luminance information normalized by the acquired exposure amount. Based on the generated luminance information, the display data generation unit 410 corrects a plurality of texture images acquired by the texture image acquisition unit 67, and can generate second display data for displaying a three-dimensional texture image in which the texture of the measurement object W included in the texture image is applied to the three-dimensional shape data.

[0100] The processing unit 4 includes a measurement processing unit 411 that executes measurement processing of the measurement object W. Executable measurement processing includes, for example, geometric measurement, comparative measurement, cross-sectional measurement, and the like. Specifically, when executing the measurement processing, the measurement processing unit 411 receives an input of a measurement instruction by the user for the three-dimensional texture image displayed on the monitor 41 when the second display data is generated by the display data generation unit 410. The measurement processing unit 411 executes the measurement processing of the measurement object W based on the received measurement instruction and the three-dimensional shape data.

[0101] In this embodiment, it is possible to display first display data, i.e., a two-dimensional texture image, on the monitor 41, and to display second display data, i.e., a three-dimensional texture image, on the monitor 41. While the two-dimensional texture image is being displayed, the display data generation unit 410 accepts an operation input of a three-dimensional view for changing the display position of the three-dimensional shape data from the user, and switches the data to be displayed on the monitor 41 to a three-dimensional texture image. For example, when the user wants to change the display position of the three-dimensional shape data when trying to input a measurement instruction while a two-dimensional texture image is being displayed on the monitor 41, the user may perform an operation for changing the display position of the three-dimensional shape data using the operation input unit 42 or the like. Examples of the operation input of the three-dimensional view for changing the display position of the three-dimensional shape data include, for example, a drag input in which the three-dimensional shape data is dragged while being clicked with the mouse. When the user performs an operation input of the three-dimensional view, the three-dimensional texture image is automatically displayed on the monitor 41.

[0102] Also, an operation for rotating the display of the three-dimensional shape data is included in the drag input. For example, by accepting a drag input for rotating the display of the three-dimensional shape data from the user, the data to be displayed on the monitor 41 is switched to a three-dimensional texture image. After that, the display of the three-dimensional shape data is rotated.

[0103] The display data generation unit 410 can also receive an input for enlarging the display of the three-dimensional shape data while the two-dimensional texture image is being displayed on the monitor 41. By receiving an operation for enlarging the display of the three-dimensional shape data, the data to be displayed on the monitor 41 can be switched to a three-dimensional texture image. The display of the three-dimensional shape data can be enlarged in the state where the image has been switched to the three-dimensional texture image. Further, the display data generation unit 410 can also receive an input for reducing the display of the three-dimensional shape data while the two-dimensional texture image is being displayed on the monitor 41. By receiving an operation for reducing the display of the three-dimensional shape data, the data to be displayed on the monitor 41 can be switched to a three-dimensional texture image. The display of the three-dimensional shape data can be reduced in the state where the image has been switched to the three-dimensional texture image.

[0104] The processing unit 4 includes a region deletion unit 412 that deletes a part of the three-dimensional shape data. The region deletion unit 412 receives user input on the three-dimensional shape display data generated by the display data generation unit 410 and displayed on the monitor 41. For example, when the user wants to delete a part of the three-dimensional shape display data, the operation input unit 42 is operated to specify an unnecessary region, which is the region to be deleted, in an arbitrary size and arbitrary shape. This region may be specified as one or a plurality. The region deletion unit 412 executes a deletion process for deleting the three-dimensional shape data of the region indicated by the user input. Although not particularly limited, for example, the three-dimensional shape data of a region other than the measurement object W can be deleted by the region deletion unit 412.

[0105] When the region deletion unit 412 executes the deletion process, the display data generation unit 410 extracts a texture corresponding to the three-dimensional shape data from which the unnecessary region has been deleted by the region deletion unit 412. The display data generation unit 410 generates a three-dimensional texture image (second display data) by attaching the extracted texture to the three-dimensional shape data.

[0106] As shown in FIGS. 19 and 20, the three-dimensional measuring device 1 may include a contact-type probe 5 that indicates the position of a measurement point. The contact-type probe 5 is a hand-held and portable probe similar to the three-dimensional scanner 2. When using the probe 5, the trigger is transmitted to the probe 5 via the optical communication interface 36a. The imaging unit 3 can track a plurality of self-luminous probe markers included in the probe 5 and can also image the probe markers. The scanner imaging camera 32 can generate a probe marker image including the probe markers by imaging the probe markers of the probe 5. The camera image processing unit 35 processes the probe marker image captured by the scanner imaging camera 32 in the same manner as in the case of the scanner marker to generate the center position information of the probe markers. Based on the center position information of the scanner markers, the position and orientation information of the scanner markers with respect to the movable imaging unit 3A is generated.

[0107] The probe 5 is also separate from the imaging unit 3 and the processing unit 4. The measurement operator can take the probe 5 to the vicinity of the measurement object W located away from the imaging unit 3 and the processing unit 4 and can identify the measurement point by the probe 5.

[0108] The probe 5 includes a probe body 120 and a stylus 121 protruding from the probe body 120. A contact 121a for contacting the measurement object W is provided at the tip of the stylus 121. This contact 121a is, for example, spherical. The contact 121a is a part for indicating the position of the measurement point of the measurement object W. The probe body 120 has a grip portion 5A at an intermediate portion in its longitudinal direction, and the measurement operator can hold the grip portion 5A with one hand during measurement to move or change the orientation of the probe 5.

[0109] A plurality of probe markers 5B are provided on the probe body 120 at intervals from each other. For example, a plurality of probe markers 5B are provided at intervals from each other on one end side in the longitudinal direction of the probe body 120, and a plurality of probe markers 5B are also provided at intervals from each other on the other end side in the longitudinal direction of the probe body 120.

[0110] Fig. 20 shows the circuit configuration of the probe 5. In this Fig. 20, only one probe marker 5B is shown, but actually a plurality of probe markers 5B are provided. A probe camera 122 is provided in the vicinity of the stylus 121. The probe 5 includes a display unit 123a composed of a liquid crystal display, an organic EL display, etc., a touch panel 123b capable of touch operation, and a display control unit 123c. Also, an operation unit 124 having a plurality of buttons and the like is provided in the vicinity of the display unit 123a. The probe 5 further includes a probe control unit 125, a storage unit 126, a probe marker lighting control unit 127, a fourth wireless communication unit 128, a motion sensor 129, etc. The probe 5 also includes a battery 5C as a power source.

[0111] The display control unit 123c is a part that controls the display unit 123a based on the signal output from the probe control unit 125, and causes the display unit 123a to display various images, user interfaces, etc. Operations performed by the user on the display unit 123a are acquired by the probe control unit 125 based on the signal output from the touch panel 123b.

[0112] The probe marker lighting control unit 127 is a part that controls the probe marker 5B. The probe marker 5B can be switched between a lit state and an extinguished state by the probe marker lighting control unit 127. The probe marker lighting control unit 127 is controlled by the probe control unit 125. Programs and the like can be stored in the storage unit 126.

[0113] Similar to the first wireless communication unit 36 of the imaging unit 3, the fourth wireless communication unit 128 has an optical communication interface 128a and a radio communication interface 128b. The optical communication interface 128a is a part that receives a trigger transmitted via the optical communication interface 36a of the imaging unit 3. When receiving the trigger, the probe marker lighting control unit 127 lights the probe marker 5B. Thereby, imaging of the probe marker by the imaging unit 3 and lighting of the probe marker 5B can be synchronized. The radio communication interface 128b may have a different radio communication method from the radio communication interface 144b of the three-dimensional scanner 2. For example, when the radio communication interface 144b of the three-dimensional scanner 2 constructs a wireless LAN, the radio communication interface 128b of the fourth wireless communication unit 128 can be configured as a part capable of Bluetooth communication or the like with a communication speed slower than that of the wireless LAN. Note that, as described above, when the radio communication methods of the radio communication interface 144b of the three-dimensional scanner 2 and the radio communication interface 128b of the probe 5 are different, the radio communication interface 36b of the imaging unit 3 may correspond to both the radio communication interface 144b of the three-dimensional scanner 2 and the radio communication interface 128b of the probe 5. That is, when the radio communication interface 144b of the three-dimensional scanner 2 constructs a wireless LAN and the radio communication interface 128b of the probe 5 constructs Bluetooth communication, the radio communication interface 36b of the imaging unit 3 may correspond to both the wireless LAN and Bluetooth communication.

[0114] The data transmission and reception between the probe 5 and the imaging unit 3 has a smaller data capacity compared to the data transmission and reception between the three-dimensional scanner 2 that continuously transmits measurement data and the imaging unit 3. Therefore, the radio communication established between the probe 5 and the imaging unit 3 may use Bluetooth communication with low power consumption and expected long battery life.

[0115] The motion sensor 129 is composed of sensors that detect the acceleration and angular velocity of the probe 5, and the detected values are output to the probe control unit 125 and used for various arithmetic processes such as the attitude calculation of the probe 5, similar to the attitude calculation of the three-dimensional scanner 2.

[0116] The coordinate calculation unit 420 shown in FIG. 2 is a part that calculates the coordinates of a plurality of measurement points indicated by the probe 5 based on the position and attitude information of the probe 5 and the information output from the imaging unit 3 and the like. The coordinate system creation unit 421 is a part that creates a measurement coordinate system based on the coordinates of a plurality of measurement points calculated by the coordinate calculation unit 420.

[0117] Next, the three-dimensional shape measurement procedure of the measurement object W by the three-dimensional measurement device 1 configured as described above will be described based on the flowchart shown in FIG. 21. The measurement operator holds the gripping part 112 of the three-dimensional scanner 2 and turns the scanner part 60 toward the measurement object W, and then operates the measurement start button included in the operation part 114. Then, in step SA1, the imaging unit 3 issues a trigger. An ID is assigned to the trigger. The trigger issued by the imaging unit 3 is received by the wireless communication unit 144 of the three-dimensional scanner 2 via the wireless communication unit 36 of the imaging unit 3. Then, in step SA2, the scanner control unit 142 of the three-dimensional scanner 2 outputs a light emission instruction to the marker lighting control unit 141, and the marker lighting control unit 141 causes the self-luminous markers 71 to 77, 81 to 87, 91 to 97, and 101 to 107 to emit light. In step SA3, the scanner control unit 142 of the three-dimensional scanner 2 outputs a light emission instruction to the scanner light source control unit 146, and the scanner light source control unit 146 causes the first scanner light source 62 or the second scanner light source 63 to emit light. Which of the first scanner light source 62 and the second scanner light source 63 emits light is based on the setting information.

[0118] Also, in step SA4, simultaneously with step SA3, the scanner control unit 142 of the three-dimensional scanner 2 outputs an imaging instruction to the scanner image processing unit 147, and the scanner image processing unit 147 causes the first scanner imaging unit 64 and the second scanner imaging unit 65 to perform imaging. The exposure times of the first scanner imaging unit 64 and the second scanner imaging unit 65 are set based on the setting information. In step SA5, a streak image is acquired by imaging with the first scanner imaging unit 64 and the second scanner imaging unit 65. A trigger ID is assigned to the luminance image. In step SA6, the streak image is input to the scanner image processing unit 147, and the scanner image processing unit 147 extracts edge data from the streak image. The edge data is received by the wireless communication unit 36 of the imaging unit 3 via the wireless communication unit 144 of the three-dimensional scanner 2.

[0119] On the other hand, in the imaging unit 3, after issuing a trigger in step SA1, it proceeds to step SA7 where the main body control unit 33 outputs an imaging instruction to the camera image processing unit 35, and the camera image processing unit 35 causes the scanner imaging camera 32 to perform imaging. In step SA8, the scanner imaging camera 32 can acquire a marker image including a plurality of self-luminous markers. A trigger ID is assigned to the marker image.

[0120] In step SA9, the marker image is input to the camera image processing unit 35 of the imaging unit 3, and the camera image processing unit 35 extracts marker image coordinates. Also, in step SA10, an external marker parameter is calculated. The external marker parameter is a six-axis parameter. In step SA10, data matching between the edge data transmitted from the three-dimensional scanner 2 and the marker image coordinates is performed based on the trigger ID. Thereby, the first step of specifying the position and orientation of the three-dimensional scanner 2 is executed. Also, since the position and orientation of the texture camera 66 are also specified by specifying the position and orientation of the three-dimensional scanner 2, the second step is also executed. Details of the data matching will be described later.

[0121] In step SA12, the data obtained in step SA11 is transmitted to the communication unit 46 of the processing unit 4 via the communication unit 37. In step SA13, the control unit 43 of the processing unit 4 processes the data transmitted from the imaging unit 3. In step SA14, the control unit 43 executes three-dimensional point cloud generation. This is the third step of generating three-dimensional shape data of the measurement object W based on the image including the pattern light generated by the scanner imaging units 64 and 65 and the position and orientation of the three-dimensional scanner 2 specified in the first step. By executing the third step, the three-dimensional shape of the measurement object W is obtained.

[0122] At this time, the scanner display unit 113 receives, via the wireless communication unit 144 of the three-dimensional scanner 2, display data in which the three-dimensional shape based on the point cloud data of the measurement object W in the measurement coordinate system created by the coordinate system creation unit 421 is cumulatively displayed, and displays the received display data. That is, by moving the three-dimensional scanner 2 a plurality of times, point cloud data is acquired a plurality of times, and by generating data for cumulatively displaying the point cloud data acquired a plurality of times, display data in which the three-dimensional shape based on the point cloud data is cumulatively displayed can be acquired.

[0123] FIG. 22 is a flowchart showing an example of the processing procedure of data matching. In step SB1, the imaging unit 3 acquires the marker external parameter data calculated in step SA10 of the flowchart shown in FIG. 21. In step SB2, the three-dimensional scanner 2 acquires the edge data extracted in step SA6 of the flowchart shown in FIG. 21 and transmits it to the imaging unit 3. In step SB3, the imaging unit 3 temporarily stores the marker external parameter data acquired in step SB1 and the edge data acquired in step SB2.

[0124] In step SB4, ID matching between the marker external parameter data and the edge data is performed based on a pre-assigned trigger ID. In step SB5, it is determined whether the trigger IDs match. If the trigger IDs match, in step SB6, the marker external parameter data and the edge data are associated. If the trigger IDs do not match, in step SB7, the marker external parameter data and the edge data are discarded. After step SB6, in step SB8, data transmission processing to the processing unit 4 is executed. In step SB9, the processing unit 4 receives the data.

[0125] Figure 23 is a flowchart showing an example of a processing procedure for reflecting a texture in three-dimensional shape data. In step S41, the scanner control unit 142 detects that the texture camera button included in the operation unit 114 of the three-dimensional scanner 2 has been pressed. In step S42, the scanner control unit 142 activates the texture camera 66. In step S43, a texture camera preview, that is, an image captured by the texture camera 66, is displayed on the scanner display unit 113. Here, the scanner display unit 113 displays, as a live image, the images sequentially generated by the texture camera 66.

[0126] In step S44, the texture image acquisition unit 67 detects that the imaging button included in the operation unit 114 has been pressed. The pressing of the imaging button is an instruction to capture a texture image. In step S45, the image captured by the texture camera 66 when the imaging button is pressed is captured. In step S46, the texture image acquisition unit 67 detects that the confirmation button (included in the operation unit 114) for confirming the image captured by the texture camera 66 has been pressed. The texture image acquisition unit 67 executes a fourth step of acquiring, in association with each other, the position and orientation information based on the position and orientation of the texture camera 66 specified in the second step and the texture image generated by the texture camera 66 by receiving an instruction to capture a texture image.

[0127] Also, in step S47, the determined texture image is transmitted to the imaging unit 3. In step S48, the imaging unit 3 executes image bridge processing, which is the bridging of the image to the processing unit 4, and transmits it to the processing unit 4. In step S49, the processing unit 4 receives the texture image. In step S50, a process of superimposing and displaying the texture image received in step S49 on the point cloud indicating the three-dimensional shape of the measurement object W generated based on the display data, that is, texture processing, is executed.

[0128] In step S51, first display data for displaying the texture image obtained through step S46 as a two-dimensional texture image viewed from the position and orientation of the texture camera 66 when the texture image was obtained is generated. An example of the two-dimensional texture image is shown in FIG. 24.

[0129] Also, in step S52, second display data for displaying a three-dimensional texture image in which the texture of the measurement object W included in the texture image is applied to the three-dimensional shape data based on the position and orientation information corresponding to the texture image obtained through step S46 is generated. An example of the three-dimensional texture image is shown in FIG. 24. The three-dimensional texture image is obtained by projecting the texture image onto each point cloud data generated by the three-dimensional data generation unit 43a. The image shown at the lower side of FIG. 25 is an example of an image based on the point cloud data before the point cloud data of the unnecessary area is deleted by the area deletion unit 412. This image includes, in addition to the point cloud data of the measurement object W, the point cloud data (background point cloud data) around the measurement object W. Note that the lower side of FIG. 25 corresponds to the left side of FIG. 25 when FIG. 25 is viewed so that the arrow in the figure is horizontal. Also, the upper side of FIG. 25 corresponds to the right side of FIG. 25 when FIG. 25 is viewed so that the arrow in the figure is horizontal.

[0130] On the other hand, the image shown on the upper side of FIG. 25 is an example of an image when the point cloud data around the measurement object W is deleted by the region deletion unit 412 and the point cloud data of the measurement object W is retained. Since the point cloud data around the measurement object W is thus deleted, in the example shown in FIG. 24, the texture information around the measurement object W is deleted, and a three-dimensional texture image with texture information applied to the measurement object W where the point cloud data is retained is generated. Steps S51 and S52 are steps corresponding to the fifth step.

[0131] That is, the texture image acquired by the texture image acquisition unit 67 is transmitted from the wireless communication unit 144 of the three-dimensional scanner 2 to the communication unit 46 of the processing unit 4 and received by the communication unit 46. The display data generation unit 410 receives the texture image acquired by the texture image acquisition unit 67 via the wireless communication unit 144 of the three-dimensional scanner 2 and the communication unit 46 of the processing unit 4, and generates display data with the texture image applied to the three-dimensional shape data of the measurement object W generated by the processing unit 4. When applying the texture image to the three-dimensional shape data, for example, for the meshes in the shadow, the texture image may be projected onto the vertices of the mesh data without projecting the texture image, and the method is not particularly limited.

[0132] In addition, the display data generation unit 410 can generate the three-dimensional shape data of the measurement object W based on the edge data received via the wireless communication unit 144 of the three-dimensional scanner 2 and the communication unit 46 of the processing unit 4. In this case, the display data generation unit 410 generates display data with the texture image applied to the generated three-dimensional shape data.

[0133] In step S53, the two-dimensional texture image generated in step S51 is displayed on the monitor 41. When the user performs an operation such as a drag input using the operation input unit 42 while the two-dimensional texture image is being displayed on the monitor 41, the image displayed on the monitor 41 automatically switches from the two-dimensional texture image to the three-dimensional texture image. The image displayed on the monitor 41 can also be switched from the three-dimensional texture image to the two-dimensional texture image.

[0134] In step S54, the display data created in step S52 can also be transmitted to the three-dimensional scanner 2 via the imaging unit 3. In step S55, the three-dimensional scanner 2 receives the display data. In step S56, the screen displayed on the scanner display unit 113 is updated based on the display data received in step S55.

[0135] For example, the communication unit 46 of the processing unit 4 can receive a texture image from the wireless communication unit 144 of the three-dimensional scanner 2. Also, the communication unit 46 of the processing unit 4 can transmit the display data generated by the display data generation unit 410 to the wireless communication unit 144 of the three-dimensional scanner 2. When displaying the display data on the scanner display unit 113, the display data generated by the display data generation unit 410 and transmitted to the three-dimensional scanner 2 via the communication unit 46 of the processing unit 4 and the wireless communication unit 144 of the three-dimensional scanner 2 is displayed. Similarly, the texture image can be transmitted to the contact probe 5 and displayed on the display unit 123a of the probe 5. Thereby, the user can move the probe 5 to indicate a desired measurement point while looking at the texture image displayed on the display unit 123a.

[0136] In a state where the scanner display unit 113 is displaying, as live images, the images sequentially generated by the texture camera 66, the texture image acquisition unit 67 can receive an imaging instruction for the texture image. When the texture image acquisition unit 67 receives an imaging instruction for the texture image while the scanner display unit 113 is displaying the live images, the scanner display unit 113 switches the currently displayed live images to the display of the two-dimensional texture image or the three-dimensional texture image received via the wireless communication unit 144 of the three-dimensional scanner 2. That is, an update process for updating the previously generated texture image can be executed.

[0137] When second display data for displaying the three-dimensional texture image is generated in step S52 and the three-dimensional texture image is displayed on the monitor 41, when an input of a measurement instruction by the user is received for the three-dimensional texture image displayed on the monitor 41, the measurement processing unit 411 executes a measurement process for the measurement object W based on the measurement instruction and the three-dimensional shape data. This step is step 6.

[0138] At least one of the two-dimensional texture image and the three-dimensional texture image can be a constituent material of the measurement result report together with the measurement values acquired by the measurement processing unit 411. When creating a measurement result report for the measurement object W using the three-dimensional measurement device 1, the measurement values of each part of the measurement object W are incorporated into the measurement result report, and further at least one of the two-dimensional texture image and the three-dimensional texture image is incorporated into the measurement result report. Thereby, the creation efficiency of the measurement result report is improved.

[0139] (Operation and Effect of Embodiment) As described above, based on the images generated by the scanner imaging units 64 and 65 of the three-dimensional scanner 2 and the position and orientation of the three-dimensional scanner 2 identified by the imaging unit 3, the three-dimensional data generation unit 43a of the processing unit 4 can generate display data indicating the three-dimensional shape of the measurement object W. The display data generated by the three-dimensional data generation unit 43a is received by the three-dimensional scanner 2 and displayed as a display screen 210 on the scanner display unit 113 of the three-dimensional scanner 2.

[0140] Then, since it is possible to display the two-dimensional texture image and the three-dimensional texture image, by switching from the two-dimensional texture image to the three-dimensional texture image or from the three-dimensional texture image to the two-dimensional texture image as needed, it becomes possible to perform the scanning operation of the measurement object W by the three-dimensional scanner 2 while viewing the desired texture image. After the scanning operation of the measurement object W, a measurement instruction may be input to the three-dimensional texture image, so that the measurement instruction can be easily input to a desired location on the scan data.

[0141] The above-described embodiments are merely illustrative in every respect and should not be construed in a limiting sense. Furthermore, all modifications and changes belonging to the equivalent scope of the claims are within the scope of the present invention. In the above example, the case where the display screen 210 is a screen for displaying a point cloud indicating the three-dimensional shape of the measurement object W has been described, but the present invention is not limited thereto, and a screen for displaying mesh data indicating the three-dimensional shape of the measurement object W may also be used.

[0142] Also, as shown in FIG. 26, the present invention can also be applied when the three-dimensional scanner 2 is attached to and operated by the arm 600. The arm 600 is a multi-degree-of-freedom arm including a plurality of arm components 600a, 600b, 600c and rotating parts 600d, 600e, 600f that rotatably connect them. In this case, the position and orientation specifying unit 601 that specifies the position and orientation of the three-dimensional scanner 2 and the position and orientation of the texture camera 66 is composed of sensors that detect the rotation angles of the arm components 600a, 600b, 600c. That is, if the rotation angles of the respective arm components 600a, 600b, 600c are known, the position and orientation of the three-dimensional scanner 2 can be calculated based on a predetermined relational expression.

Industrial Applicability

[0143] As described above, the present invention can be used when measuring the three-dimensional shape of various measurement objects.

Explanation of Signs

[0144] 1 Three-dimensional measuring device 2 Three-dimensional scanner 3 Imaging unit (position and orientation specifying unit) 4 Processing unit (three-dimensional data generation means) 37 Communication unit (third communication unit) 43 Control unit (measurement control unit) 46 Communication unit (second communication unit) 48 Measurement setting unit 62, 63 Scanner light source 64, 65 Scanner imaging unit 66 Texture camera 67 Texture image acquisition unit 71 to 77 First to seventh scanner markers 113 Scanner display unit 144 Wireless communication unit (first communication unit) 410 Display data generation unit 411 Measurement processing unit 412 Region deletion unit 420 Coordinate calculation unit 421 Coordinate system creation unit 601 Position and Orientation Specifying Unit

Claims

1. A three-dimensional measuring device for measuring the three-dimensional shape of an object to be measured, a three-dimensional scanner having a scanner light source that irradiates pattern light and a scanner imaging unit that images the pattern light irradiated by the scanner light source to generate an image including the pattern light, a texture camera that images the object to be measured to generate a texture image including the texture of the object to be measured, a position and orientation specifying unit that specifies the position and orientation of the three-dimensional scanner and the position and orientation of the texture camera, three-dimensional data generation means for generating three-dimensional shape data of the object to be measured based on the image including the pattern light generated by the scanner imaging unit and the position and orientation of the three-dimensional scanner specified by the position and orientation specifying unit, a texture image acquisition unit that receives an imaging instruction for the texture image and acquires, in association with each other, position and orientation information based on the position and orientation of the texture camera specified by the position and orientation specifying unit and the texture image generated by the texture camera, display data generation means for generating first display data for displaying the texture image acquired by the texture image acquisition unit as a two-dimensional texture image viewed from the position and orientation of the texture camera when the texture image was acquired, and second display data for displaying a three-dimensional texture image in which the texture of the object to be measured included in the texture image is applied to the three-dimensional shape data based on the position and orientation information corresponding to the texture image; a three-dimensional measuring device comprising a measurement processing unit that receives an input of a measurement instruction for the three-dimensional texture image displayed on the display unit when the second display data is generated by the display data generation means and executes a measurement process of the object to be measured based on the measurement instruction and the three-dimensional shape data.

2. In the three-dimensional measuring device according to Claim 1, The three-dimensional measuring device, wherein the display data generation unit switches the data to be displayed on the display unit to the second display data by receiving an operation input of a three-dimensional view for changing the display position of the three-dimensional shape data during the display of the first display data.

3. In the three-dimensional measuring device according to claim 2, further comprising a region deletion unit that deletes the three-dimensional shape data of a region indicated by the user input by receiving a user input on the three-dimensional shape display data generated by the display data generation unit and displayed on the display unit, The three-dimensional measuring device, wherein the display data generation unit extracts a texture corresponding to the three-dimensional shape data from which a region has been deleted by the region deletion unit, and generates a three-dimensional texture image in which the texture is applied to the three-dimensional shape data.

4. In the three-dimensional measuring device according to claim 1, the texture image acquisition unit acquires a plurality of texture images associated with the position and orientation information based on the position and orientation of the texture camera specified by the position and orientation specifying unit, the three-dimensional measuring device, wherein the display data generation unit generates the second display data for displaying a three-dimensional texture image in which the texture of the measurement object included in the texture image is applied to the three-dimensional shape data based on the position and orientation information corresponding to each texture image.

5. In the three-dimensional measuring device according to claim 3, the three-dimensional measuring device, wherein the display data generation unit corrects a plurality of texture images acquired by the texture image acquisition unit based on the luminance information normalized by the exposure amount of the texture camera, and generates the second display data for displaying a three-dimensional texture image in which the texture of the measurement object included in the texture image is applied to the three-dimensional shape data.

6. In the three-dimensional measuring device according to claim 5, a contact probe indicating the position of the measurement point, A coordinate calculation unit that calculates the coordinates of a plurality of measurement points indicated by the contact probe, A three-dimensional measuring device comprising: a coordinate system creation unit that creates a measurement coordinate system based on the coordinates of a plurality of measurement points calculated by the coordinate calculation unit.

7. In the three-dimensional measuring device according to claim 6, The three-dimensional scanner further includes a scanner display unit and a first communication unit that transmits the texture image to the display data generation unit and receives the display data generated by the display data generation unit. The scanner display unit receives, via the first communication unit, display data in which a three-dimensional shape based on point cloud data of a measurement object in the measurement coordinate system created by the coordinate system creation unit is cumulatively displayed, and displays the received display data. A three-dimensional measuring device.

8. In the three-dimensional measuring device according to claim 7, The three-dimensional scanner further includes a second communication unit that receives the texture image from the first communication unit of the three-dimensional scanner and transmits display data to the first communication unit. The scanner display unit displays display data generated by the display data generation unit and transmitted to the three-dimensional scanner via the second communication unit and the first communication unit. A three-dimensional measuring device.

9. In the three-dimensional measuring device according to claim 8, The display data generation unit receives the texture image acquired by the texture image acquisition unit via the first communication unit and the second communication unit, and attaches the texture image to the three-dimensional shape data of the measurement object generated by the three-dimensional data generation means. Generate display data, The scanner display unit displays display data generated by the display data generation unit and transmitted to the three-dimensional scanner via the second communication unit and the first communication unit. A three-dimensional measuring device.

10. In the three-dimensional measuring device according to claim 9, The three-dimensional scanner further includes a scanner image processing unit that processes the image generated by the scanner imaging unit to generate first measurement information. The display data generation unit generates three-dimensional shape data of a measurement object based on the first measurement information received via the first communication unit and the second communication unit, and generates display data with the texture image applied thereon. A three-dimensional measuring device.

11. In the three-dimensional measuring device according to claim 6, The contact probe has a plurality of self-luminous markers. The position and orientation specifying unit moves the field of view so that the contact probe is within the field of view, processes an image including a plurality of self-luminous markers of the contact probe, and based on the positions and orientations of the plurality of self-luminous markers included in the image, specifies the position and orientation of the contact probe. The coordinate calculation unit calculates the coordinates of a plurality of measurement points indicated by the contact probe based on the position and orientation of the contact probe specified by the position and orientation specifying unit. A three-dimensional measuring device.

12. In the three-dimensional measuring device according to claim 11, The three-dimensional scanner has a plurality of self-luminous markers. The position and orientation specifying unit moves the field of view so that the three-dimensional scanner is within the field of view, processes an image including the self-luminous markers of the three-dimensional scanner, and based on the positions and orientations of the plurality of self-luminous markers included in the image, specifies the position and orientation of the three-dimensional scanner. A three-dimensional measuring device.

13. In the three-dimensional measuring device according to claim 7, In a state where the scanner display unit is displaying, as live images, the images sequentially generated by the texture camera, when the texture image acquisition unit receives an instruction to capture a texture image, the scanner display unit switches the live image being displayed to a display based on the first display data or the second display data received via the first communication unit. A three-dimensional measurement device.

14. A three-dimensional measurement method for measuring the three-dimensional shape of a measurement object, comprising: A first step of specifying the position and orientation of a three-dimensional scanner including a scanner light source that irradiates pattern light and a scanner imaging unit that captures the pattern light irradiated by the scanner light source and generates an image including the pattern light; A second step of specifying the position and orientation of a texture camera that captures a measurement object and generates a texture image including the texture of the measurement object; A third step of generating three-dimensional shape data of the measurement object based on the image including the pattern light generated by the scanner imaging unit and the position and orientation of the three-dimensional scanner specified in the first step; A fourth step of, by receiving an instruction to capture a texture image, associating and acquiring the position and orientation information based on the position and orientation of the texture camera specified in the second step and the texture image generated by the texture camera; A fifth step of generating first display data for displaying the texture image acquired in the fourth step as a two-dimensional texture image viewed from the position and orientation of the texture camera when the texture image was acquired, and second display data for displaying a three-dimensional texture image in which the texture of the measurement object included in the texture image is applied to the three-dimensional shape data based on the position and orientation information corresponding to the texture image; A sixth step of receiving an input of a measurement instruction for the three-dimensional texture image displayed on the display unit by generating the second display data in the fifth step, and executing a measurement process of a measurement object based on the measurement instruction and the three-dimensional shape data. A three-dimensional measurement method comprising:

15. A three-dimensional measurement program for causing a computer to execute a three-dimensional measurement method for measuring the three-dimensional shape of a measurement object, A first step of specifying the position and orientation of a three-dimensional scanner having a scanner light source that irradiates pattern light and a scanner imaging unit that captures the pattern light irradiated by the scanner light source and generates an image including the pattern light; A second step of specifying the position and orientation of a texture camera that captures a measurement object and generates a texture image including the texture of the measurement object; A third step of generating three-dimensional shape data of a measurement object based on the image including the pattern light generated by the scanner imaging unit and the position and orientation of the three-dimensional scanner specified in the first step; A fourth step of receiving an imaging instruction for a texture image, and associating and acquiring position and orientation information based on the position and orientation of the texture camera specified in the second step and the texture image generated by the texture camera; First display data for displaying the texture image acquired in the fourth step as a two-dimensional texture image viewed from the position and orientation of the texture camera when the texture image was acquired, and based on the position and orientation information corresponding to the texture image, A fifth step of generating second display data for displaying a three-dimensional texture image in which the texture of the measurement object included in the texture image is applied to the three-dimensional shape data; A three-dimensional measurement program that causes the computer to execute a sixth step of receiving an input of a measurement instruction for the three-dimensional texture image displayed on the display unit by generating the second display data in the fifth step, and executing a measurement process of a measurement object based on the measurement instruction and the three-dimensional shape data.

Images