Coordinate measuring apparatus, coordinate measuring method, and storage medium storing coordinate measuring program

Abstract

To allow a user to easily grasp whether or not necessary three-dimensional data are acquired when acquiring the three-dimensional data by a three-dimensional scanner.SOLUTION: The three dimensional coordinate generation unit 43a sequentially generates the three dimensional coordinates of the measurement workpiece based on the image including the pattern light generated by the scanner imaging unit. The display control unit 44 sequentially generates display data in which the three dimensional coordinates of the measurement work sequentially generated by the three dimensional coordinate generation unit 43a are superimposed on the three dimensional shape based on the reference three dimensional shape data registered by the registration unit 43g. The three dimensional data generation unit 43b generates three dimensional shape data of the measurement workpiece based on the three dimensional coordinates of the measurement workpiece. The measurement unit 43d performs measurement based on the reference measurement elements registered by the registration unit 43g and the three-dimensional shape data of the measurement workpiece.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a three-dimensional measuring device for measuring the three-dimensional shape of a workpiece, a three-dimensional measuring method, and a storage medium storing a three-dimensional measuring program. [Background technology]

[0002] For example, Patent Document 1 discloses a method for inspecting an object to be inspected, in which a three-dimensional scanner is used to obtain three-dimensional scan data showing the shape of the object to be inspected, the CAD data and the three-dimensional scan data are aligned to the same coordinates, and then the inspection elements of the CAD data and the inspection elements of the three-dimensional scan data are compared. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-194896 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in Patent Document 1, when three-dimensional data is acquired using a three-dimensional scanner, reference data such as CAD data is not provided, making it difficult for the user to determine whether the necessary three-dimensional data has been acquired.

[0005] The present disclosure has been made in consideration of such points, and its purpose is to make it easier for users to understand whether or not the necessary three-dimensional data has been acquired when acquiring three-dimensional data using a three-dimensional scanner. [Means for solving the problem]

[0006] To achieve the above object, one aspect of the present disclosure can be premised on a three-dimensional measuring device for measuring the three-dimensional shape of a workpiece. The three-dimensional measuring device includes a three-dimensional scanner having a scanner light source that irradiates pattern light and a scanner imaging unit that captures the pattern light irradiated by the scanner light source and generates an image including the pattern light, a three-dimensional coordinate generation unit that sequentially generates three-dimensional coordinates of a reference workpiece based on the image including the pattern light generated by the scanner imaging unit, a three-dimensional data generation unit that generates reference three-dimensional shape data of the reference workpiece based on the three-dimensional coordinates sequentially generated by the three-dimensional coordinate generation unit, a display control unit that generates display data for displaying the reference three-dimensional shape data generated by the three-dimensional data generation unit on a display unit, and a display control unit that controls the reference three-dimensional shape data generated by the three-dimensional data generation unit and displayed on the display unit. The system is equipped with a setting unit that sets measurement elements, a measurement unit that performs measurement based on the measurement elements set by the setting unit, a memory unit that stores a setting file that associates the reference three-dimensional shape data generated by the three-dimensional data generation unit with the measurement elements set by the setting unit as a measurement reproduction template, a reception unit that receives the selection of one template from the measurement reproduction templates stored in the memory unit, a reading unit that reads out the reference three-dimensional shape data and the measurement elements included in the one template received by the reception unit, and a registration unit that registers the reference three-dimensional shape data and the measurement elements read by the reading unit as reference three-dimensional shape data and reference measurement elements.

[0007] The three-dimensional coordinate generation unit sequentially generates three-dimensional coordinates of the measurement work based on an image including pattern light generated by the scanner imaging unit, the display control unit sequentially generates display data in which the three-dimensional coordinates of the measurement work sequentially generated by the three-dimensional coordinate generation unit are superimposed on a three-dimensional shape based on the reference three-dimensional shape data registered by the registration unit, the three-dimensional data generation unit generates three-dimensional shape data of the measurement work based on the three-dimensional coordinates of the measurement work, and the measurement unit can perform measurement based on the reference measurement element registered by the registration unit and the three-dimensional shape data of the measurement work.

[0008] According to this configuration, when a workpiece is imaged by the scanner imaging unit of the three-dimensional scanner, reference three-dimensional shape data of the reference workpiece is generated and displayed on the display unit. Measurement elements can be set for the reference three-dimensional shape data displayed on the display unit. The reference three-dimensional shape data and measurement elements are registered as reference three-dimensional shape data and reference measurement elements, respectively. When a measurement workpiece is imaged by the scanner imaging unit, display data is sequentially generated in which the sequentially generated three-dimensional coordinates of the measurement workpiece are superimposed on a three-dimensional shape based on the reference three-dimensional shape data. This makes it easier for the user to determine whether the required three-dimensional data has been acquired when acquiring three-dimensional data using the three-dimensional scanner.

[0009] Another aspect of the present disclosure can be based on a three-dimensional measurement method for measuring the three-dimensional shape of a workpiece. The three-dimensional measurement method includes capturing an image of pattern light irradiated by a scanner light source to generate an image including the pattern light, sequentially generating three-dimensional coordinates of a reference workpiece based on the generated image including the pattern light, generating reference three-dimensional shape data of the reference workpiece based on the sequentially generated three-dimensional coordinates, displaying the generated reference three-dimensional shape data on a display unit, setting measurement elements for the reference three-dimensional shape data displayed on the display unit, storing a setting file that associates the generated reference three-dimensional shape data with the set measurement elements as a measurement reproduction template, and selecting one of the stored measurement reproduction templates. The system accepts the selection of a template, reads out the reference three-dimensional shape data and the measurement elements contained in the accepted template, registers the read-out reference three-dimensional shape data and the measurement elements as reference three-dimensional shape data and reference measurement elements, sequentially generates display data in which the three-dimensional coordinates of the measurement workpiece that are sequentially generated are superimposed on a three-dimensional shape based on the registered reference three-dimensional shape data, generates three-dimensional shape data of the measurement workpiece based on the three-dimensional coordinates of the measurement workpiece, and performs the measurement based on the registered reference measurement elements and the three-dimensional shape data of the measurement workpiece.

[0010] Yet another aspect of the present disclosure may be a storage medium storing a three-dimensional measurement program that causes a computer to execute a three-dimensional measurement method for measuring a three-dimensional shape of a workpiece. [Effects of the Invention]

[0011] As described above, it becomes easier for the user to know whether or not the necessary three-dimensional data has been acquired. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram showing the configuration of a three-dimensional measuring device according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram of the imaging unit and the processing unit. [Figure 3] FIG. 3 is a perspective view of the three-dimensional scanner as seen from below. [Figure 4] FIG. 4 is a block diagram showing the circuit configuration of the three-dimensional scanner. [Figure 5] FIG. 5 is a block diagram showing the circuit configuration of the probe. [Figure 6] FIG. 6 is a diagram showing an example of a user interface screen. [Figure 7] FIG. 7 is a diagram showing an example of displaying mesh data. [Figure 8] FIG. 8 is a diagram showing an example in which a plane is set as a measurement element. [Figure 9] FIG. 9 is a flowchart showing an example of the flow of the measurement process. [Figure 10] FIG. 10 is a diagram showing an example of a measurement user interface screen showing how a measurement workpiece coordinate system has been generated. [Figure 11] FIG. 11 is a diagram showing an example of a measurement user interface screen showing how the reference workpiece and the measurement workpiece have been aligned. [Figure 12] FIG. 12 is a diagram showing an example of a measurement user interface screen during measurement by the three-dimensional scanner. [Figure 13] FIG. 13 is a diagram showing an example of the measurement user interface screen when measurement by the three-dimensional scanner is completed. [Figure 14] FIG. 14 is a diagram showing an example of the measurement user interface screen when mesh data of a measurement workpiece that is a certain distance away from the mesh data in the setting file is deleted. [Figure 15] FIG. 15 is a diagram showing an example of a measurement user interface screen when a measurement element of a workpiece to be measured is extracted. [Figure 16] FIG. 16 is a flowchart showing an example of processing when the display color of mesh data is changed according to settings. [Figure 17] FIG. 17 is a conceptual diagram showing how the display color of mesh data is changed according to settings. [Figure 18]FIG. 18 is a flowchart showing an example of processing according to the progress of the scan. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the following description of the preferred embodiments is merely exemplary in nature and is not intended to limit the present invention, its applications, or its uses.

[0014] FIG. 1 is a diagram showing the configuration of a three-dimensional measuring device 1 according to an embodiment of the present invention. The three-dimensional measuring device 1 is a measuring instrument for measuring the three-dimensional shape and three-dimensional coordinates of a workpiece W. The three-dimensional measuring device 1 includes a non-contact three-dimensional scanner 2 having multiple self-luminous markers (scanner markers), a contact probe 5 having multiple self-luminous markers (probe markers), an imaging unit 3 that captures the multiple scanner markers of the three-dimensional scanner 2 and the multiple probe markers of the probe 5, and a processing unit 4 that measures the three-dimensional shape and three-dimensional coordinates of the workpiece W. The markers do not have to be self-luminous markers. The three-dimensional scanner 2 is separate from the imaging unit 3 and processing unit 4, and a measurement operator can bring the three-dimensional scanner 2 to the vicinity of the workpiece W, which is located away from the imaging unit 3 and processing unit 4, and generate a bright line image using the three-dimensional scanner 2. The probe 5 is also separate from the imaging unit 3 and processing unit 4, and a measurement operator can bring the probe 5 to the vicinity of the workpiece W, which is located away from the imaging unit 3 and processing unit 4, and identify measurement points using the probe 5.

[0015] (Configuration of imaging unit 3) The imaging unit 3 is a unit that generates a scanner marker image including multiple scanner markers (described later) provided on the three-dimensional scanner 2 by capturing images of the multiple scanner markers, and also generates a probe marker image including multiple probe markers (described later) provided on the probe 5 by capturing images of the multiple probe markers. When measuring using the three-dimensional scanner 2, a scanner marker image including the scanner markers is generated, while when measuring using the probe 5, a probe marker image including the probe markers is generated.

[0016] As shown in FIG. 2 , the imaging unit 3 includes a base 30 and a movable imaging unit 3A that moves the field of view so that the three-dimensional scanner 2 is within the field of view and captures images of scanner markers to measure the position and orientation of the three-dimensional scanner 2 and generate a scanner marker image including the scanner markers. The movable imaging unit 3A includes a movable stage 31 supported by the base 30 and a scanner imaging camera 32 fixed to the upper part of the movable stage 31. The movable stage 31 includes a stage driver 31a. The stage driver 31a incorporates an actuator such as a motor and is configured to rotate the movable stage 31 about a vertical axis and also about a horizontal axis. Rotating the movable stage 31 about the vertical axis rotates the scanner imaging camera 32 about the vertical axis, and rotating the movable stage 31 about the horizontal axis rotates the scanner imaging camera 32 about the horizontal axis. This allows the field of view (schematically shown by dashed line A in FIGS. 1 and 2) of the scanner imaging camera 32 to be moved, thereby tracking the three-dimensional scanner 2, i.e., the multiple scanner markers possessed by the three-dimensional scanner 2, so that they come within the field of view of the scanner imaging camera 32. Similarly, the probe markers possessed by the probe 5 can also be tracked. The stage driving unit 31a is controlled by a main body control unit 33 possessed by the imaging unit 3.

[0017] A plurality of light emitters 31b are provided at predetermined intervals on a two-dimensional plane below the movable stage 31, and the light emitters 31b can be switched between a lit state and an extinguished state by a lighting control unit 31c. The arrangement information of the light emitters 31b is stored in advance in the imaging unit 3. Note that the light emitters may be landmarks other than light emitters. The lighting control unit 31c is controlled by the main body control unit 33. Meanwhile, a reference camera 34 that captures an image of the movable imaging unit 3A is provided on the base 30. The reference camera 34 captures the light emitters 31b that are turned on by the lighting control unit 31c. The reference camera 34 captures the light emitters 31b provided on the movable imaging unit 3A and generates an image including the light emitters 31b.

[0018] The imaging unit 3 is provided with a camera image processing unit 35. The camera image processing unit 35 has an image processing circuit and controls the scanner imaging camera 32 to perform imaging at a predetermined timing. The scanner imaging camera 32 captures the scanner markers of the three-dimensional scanner 2 to generate a scanner marker image including the scanner markers. The scanner imaging camera 32 also captures the probe markers of the probe 5 to generate a probe marker image including the probe markers.

[0019] Examples of the image processing circuit include a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), etc. The camera image processing unit 35 receives the scanner marker image or the probe marker image captured by the scanner imaging camera 32, as well as the image of the light-emitting body 31b captured by the reference camera 34.

[0020] The camera image processing unit 35 processes the scanner marker image captured by the scanner imaging camera 32 to generate center position information of the scanner marker. Specifically, the camera image processing unit 35 performs processing to extract the center of the scanner marker from the scanner marker image. Then, based on the extraction result, it generates center position information of the scanner marker. Furthermore, based on the center position information of the scanner marker obtained as a result of the processing to extract the center of the scanner marker, the camera image processing unit 35 generates position and orientation information of the scanner marker relative to the movable imaging unit 3A.

[0021] As in the case of the scanner marker, the camera image processing unit 35 processes the probe marker image captured by the scanner imaging camera 32 to generate center position information of the probe marker. Based on the center position information of the scanner marker, it generates position and orientation information of the scanner marker relative to the movable imaging unit 3A.

[0022] The imaging unit 3 includes a first wireless communication unit 36 ​​controlled by the main body control unit 33. The first wireless communication unit 36 ​​is a communication module or the like configured to be able to communicate with devices other than the imaging unit 3. In this example, the imaging unit 3 communicates with the three-dimensional scanner 2 and the probe 5 via the first wireless communication unit 36, and is able to send and receive various data, such as image data captured by the scanner imaging camera 32, and various signals, such as synchronization signals.

[0023] The first wireless communication unit 36 ​​transmits, for example, a synchronization signal to the three-dimensional scanner 2 and the probe 5, and receives measurement information (edge ​​data) generated by the scanner image processing unit 147 (described later). It also transmits, to the processing unit 4, center position information of the scanner marker, which is measurement information generated by the camera image processing unit 35, and center position information of the probe marker, which is measurement information generated by the camera image processing unit 35. The first wireless communication unit 36 ​​has an optical communication interface 36a and a radio wave communication interface 36b. The optical communication interface 36a is a unit for performing optical communication using visible or invisible light, and may be configured, for example, by an infrared communication interface. The radio wave communication interface 36b may be, for example, a unit for establishing a wireless LAN, or may be a unit capable of short-range digital wireless communication using radio waves, such as Bluetooth (registered trademark) communication.

[0024] The imaging unit 3 also includes a communication unit 37 controlled by the main body control unit 33. The communication unit 37 is a communication module or the like configured to be able to communicate with the processing unit 4. The imaging unit 3 communicates with the processing unit 4 via the communication unit 37, enabling transmission and reception of various data such as image data and various signals. The communication via the communication unit 37 may be wired communication or wireless communication.

[0025] The imaging unit 3 has a trigger generation unit 38 that generates identification information for identifying the timing of synchronization execution based on a measurement instruction. For example, when a measurement operator performs a predetermined measurement start operation, the main body control unit 33 of the imaging unit 3 accepts the measurement start operation. Upon accepting the measurement start operation, the main body control unit 33 causes the trigger generation unit 38 to generate a trigger as the above-mentioned identification information. The trigger is transmitted to the three-dimensional scanner 2, for example, via an optical communication interface 36a of the wireless communication unit 36. When a probe 5 is used, the trigger is transmitted to the probe 5 via the optical communication interface 36a.

[0026] In response to the generation of the trigger, the main body control unit 33 synchronizes and executes the emission of light by the scanner marker of the three-dimensional scanner 2, the imaging of the scanner marker of the three-dimensional scanner 2 by the movable imaging unit 3A, the lighting of the light emitter 31b of the movable stage 31, and the imaging of the light emitter 31b by the reference camera 34. The main body control unit 33 synchronizes and executes the emission of light by the scanner marker of the three-dimensional scanner 2, the imaging by the movable imaging unit 3A, and the imaging by the reference camera 34.

[0027] The radio wave communication interface 36b of the first wireless communication unit 36 ​​associates and transmits the scanner marker center position information generated by the camera image processing unit 35 with identification information corresponding to the scanner marker center position information generated by the trigger generation unit 38. For example, the scanner marker center position information is associated with identification information for distinguishing the scanner marker center position information from the center position information of other scanner markers. Therefore, the desired scanner marker center position information can be identified based on the identification information.

[0028] (Configuration of processing unit 4) The processing unit 4 is a three-dimensional data generation unit that receives the positions and orientations of multiple markers obtained by processing the scanner marker image generated by the imaging unit 3 from the imaging unit 3, and receives edge data of the bright line image obtained by processing the bright line image generated by the three-dimensional scanner 2, and measures the three-dimensional shape of the workpiece W based on the received positions and orientations of the scanner markers and edge data.

[0029] An example of a method for measuring a three-dimensional shape will now be described. The multiple light-emitting bodies 31b of the imaging unit 3 are mounted on a movable stage 31 to which the scanner imaging camera 32 is fixed, so the positional relationship of the multiple light-emitting bodies 31b with respect to the scanner imaging camera 32 is known. When the scanner imaging camera 32 is moved by the stage driving unit 31a, the scanner imaging camera 32 moves within a range in which the reference camera 34 can capture images of the light-emitting bodies 31b. The position and orientation of the three-dimensional scanner 2 with respect to the scanner imaging camera 32 is determined based on a scanner marker image of the three-dimensional scanner 2 captured by the scanner imaging camera 32.

[0030] Similarly, the reference camera 34 determines the position and orientation of the scanner imaging camera 32 relative to the reference camera 34 based on images of the light-emitting bodies 31b. Specifically, the camera image processing unit 35 processes the images of the light-emitting bodies 31b generated by the reference camera 34 to generate position and orientation information of the scanner imaging camera 32 relative to the reference camera 34.

[0031] The position and orientation of the three-dimensional scanner 2 relative to the scanner imaging camera 32 and the position and orientation of the scanner imaging camera 32 relative to the reference camera 34 are used to determine the position and orientation of the three-dimensional scanner 2 relative to the reference camera 34, and the coordinates of the measurement points are then obtained, making it possible to perform three-dimensional coordinate measurement, i.e., measurement of the three-dimensional shape.

[0032] By installing a three-dimensional measurement program or application for realizing the functions of the three-dimensional measuring device 1 in the processing unit 4, it can be used as the processing unit 4 and can also execute the three-dimensional measuring method according to the present invention. The three-dimensional measuring method is a method for measuring the three-dimensional shape of the workpiece W, and is executed by the processing unit 4, which is a computer. The three-dimensional measuring program that causes the computer to execute the three-dimensional measuring method can be recorded on a storage medium 1000. The storage medium 1000 may be, for example, an optical disk such as a CD-ROM or DVD-ROM, or a semiconductor memory such as a memory card.

[0033] The processing unit 4 may be separate from the imaging unit 3, or may be integrated with the imaging unit 3. Furthermore, a part of the processing unit 4 may be incorporated into the imaging unit 3, or a part of the imaging unit 3 may be incorporated into the processing unit 4.

[0034] 2, the processing unit 4 includes a control unit 40, a monitor 41, and an operation input unit 42. The monitor 41 is configured to be capable of displaying various images, user interfaces, etc., and is configured to be a liquid crystal display, an organic EL display, or the like.

[0035] The operation input unit 42 is a section where the user performs various input operations and is configured with, for example, a keyboard, a mouse, and the like.

[0036] The control unit 40 includes a control unit 43, a display control unit 44, a storage unit 45, and a second wireless communication unit 46. The display control unit 44 controls the monitor 41 based on signals output from the control unit 43, and causes the monitor 41 to display various images, a user interface, and the like. Operations performed by the user on the user interface are acquired by the control unit 43 based on signals output from the operation input unit 42.

[0037] The storage unit 45 may be a ROM, a solid state drive, a hard disk drive, etc. The storage unit 45 stores placement information of each scanner marker in the marker blocks of the three-dimensional scanner 2. The placement information of the marker blocks and each scanner marker includes information indicating the distance between the marker blocks, the relative positional relationship of the self-luminous markers provided in each marker block, etc.

[0038] The second wireless communication unit 46 of the processing unit 4 is controlled by the control unit 43. The second wireless communication unit 46 is a communication module or the like configured to be able to communicate with the first wireless communication unit 36 ​​of the imaging unit 3. The second wireless communication unit 46 has a radio wave communication interface 46a. The radio wave communication interface 46a of the second wireless communication unit 46 receives measurement information, such as edge data, scanner marker center position information, and probe marker center position information, transmitted via the radio wave communication interface 36b of the first wireless communication unit 36 ​​of the imaging unit 3.

[0039] (Configuration of 3D Scanner 2) The three-dimensional scanner 2 is a handheld, portable scanner configured so that a measurement operator can measure the shape of the workpiece W while holding it in one or both hands and moving it freely. In this embodiment, the front, back, left, right, and top and bottom of the three-dimensional scanner 2 are defined as shown in FIG. 3 . That is, when the measurement operator holds the three-dimensional scanner 2 in his / her hand, the side that is located to the right is referred to as the right, and the side that is located to the left is referred to as the left. The front of the three-dimensional scanner 2 is the side that faces the workpiece W, and the rear of the three-dimensional scanner 2 is the side opposite the side that faces the workpiece W. The top of the three-dimensional scanner 2 refers to the side that is the upper side when the gripper 112 (described below) is held in a predetermined, natural posture, and the bottom of the three-dimensional scanner 2 refers to the side that is the lower side when the gripper 112 is held in a predetermined, natural posture. However, since the three-dimensional shape of the workpiece W can be measured while the three-dimensional scanner 2 is held in his / her hand and moved as described above, the orientation of the three-dimensional scanner 2 may be upside down, or the top side may be positioned to the right or left, or the rear side may be positioned to the top or bottom.

[0040] The three-dimensional scanner 2 includes a scanner body 20, a first marker block 21, a second marker block 22, a third marker block 23, and a fourth marker block 24. The first to fourth marker blocks 21 to 24 each include self-luminous scanner markers 21a, 22a, 23a, and 24a that face in multiple directions. The scanner markers 21a, 22a, 23a, and 24a include, for example, light-emitting diodes (LEDs).

[0041] The scanner main body 20 has a scanner unit 60. The scanner unit 60 has two first scanner light sources 62, a second scanner light source 63, a first scanner imaging unit 64, a second scanner imaging unit 65, and a texture camera 66. The two first scanner light sources 62 are multi-line light sources that irradiate multiple linear light beams in the measurement direction (forward), and are arranged so that the light emission surface faces the workpiece W during measurement. The light irradiated by the first scanner light source 62 can be called multi-line light, and multi-line light is included in pattern light.

[0042] The second scanner light source 63 is a single-line light source that emits a single linear light in the measurement direction (forward), and is arranged so that the light emission surface faces the workpiece W during measurement. The light emitted by the second scanner light source 63 can be called single-line light, and single-line light is also included in pattern light.

[0043] The first scanner light source 62 and the second scanner light source 63 each have a laser light source that emits laser light, but the type of light source is not particularly limited. In this example, a total of three scanner light sources 62 and 63 are provided, but this is not a limitation, and it is sufficient that one or more scanner light sources are provided. In addition, the type of pattern light is not particularly limited, and the scanner light source may emit pattern light other than multi-line light or single-line light.

[0044] The first scanner imaging unit 64 and the second scanner imaging unit 65 each include a light-receiving element such as a CMOS sensor, an optical system for forming an image of externally incident light on the light-receiving surface of the light-receiving element, etc. The first scanner imaging unit 64 is attached at a location above and away from the scanner light sources 62 and 63. The second scanner imaging unit 65 is attached at a location below and away from the scanner light sources 62 and 63. The first scanner imaging unit 64 and the second scanner imaging unit 65 are arranged so that their optical axes are oriented in the direction of irradiation of the pattern light by the scanner light sources 62 and 63, making it possible to capture the pattern light irradiated by the scanner light sources 62 and 63 in the measurement direction and generate a bright line image including the pattern light.

[0045] The distance between the optical axes of the first scanner imaging unit 64 and the second scanner imaging unit 65 is known, and the pattern light irradiated from the first scanner light source 62 or the second scanner light source 63 is simultaneously captured by the first scanner imaging unit 64 and the second scanner imaging unit 65 to generate corresponding points in the respective images, and the three-dimensional coordinates of the corresponding points can be determined by using a stereo measurement method. The stereo measurement method may be passive stereo using the first scanner imaging unit 64 and the second scanner imaging unit 65, or active stereo using a single scanner imaging unit.

[0046] The texture camera 66 has a light receiving element such as a CMOS sensor capable of acquiring a color image, an optical system for forming an image of light incident from outside on the light receiving surface of the light receiving element, etc. The texture camera 66 captures an image of the workpiece W and generates a texture image.

[0047] As shown in Fig. 4, the scanner main body 20 is provided with a display unit (scanner display unit) 113 for displaying the measurement results obtained by the scanner unit 60, and an operation unit 114 for operating the scanner unit 60. The display unit 113 is configured with a liquid crystal display, an organic EL display, or the like. The display surface faces the person being measured, so that the three-dimensional scanner 2 can be operated while viewing the display content of the display unit 113.

[0048] A touch panel 113a that can be operated by touch is also provided on the display surface side of the display unit 113. The operation unit 114 is made up of a plurality of operation buttons, including, for example, a measurement start button, a measurement stop button, etc., and is arranged below the display unit 113. The touch panel 113a can also be part of the operation unit.

[0049] The three-dimensional scanner 2 includes a display control unit 140, a marker lighting control unit 141, a scanner control unit 142, and a storage unit 143. The display control unit 140 controls the display unit 113 based on signals output from the scanner control unit 142, and causes the display unit 113 to display various images, user interfaces, etc. Operations performed by the user on the display unit 113 are acquired by the scanner control unit 142 based on signals output from the touch panel 113a.

[0050] The marker lighting control unit 141 is a unit that controls the scanner markers. The scanner markers 21a, 22a, 23a, and 24a are configured to be switched between an on state and an off state by the marker lighting control unit 141. The marker lighting control unit 141 is controlled by the scanner control unit 142. The storage unit 143 is capable of temporarily storing a control program, an image captured by the scanner unit 60, and the like.

[0051] The three-dimensional scanner 2 includes a third wireless communication unit 144 controlled by the scanner control unit 142. The third wireless communication unit 144 is a communication module or the like configured to be able to communicate with devices other than the three-dimensional scanner 2. The third wireless communication unit 144 is a part that transmits edge data generated by the scanner image processing unit 147 to the first wireless communication unit 36 ​​of the imaging unit 3, and receives a synchronization signal transmitted by the first wireless communication unit 36. The third wireless communication unit 144 includes an optical communication interface 144a and a radio wave communication interface 144b.

[0052] The three-dimensional scanner 2 is equipped with a motion sensor 145. The motion sensor 145 is composed of sensors that detect the acceleration and angular velocity of the three-dimensional scanner 2, and the detected values ​​are output to the scanner control unit 142 and used in various calculation processes. For example, the value output from the motion sensor 145 can be used to obtain an initial solution for the attitude of the three-dimensional scanner 2, i.e., the attitudes of the first to fourth marker blocks 21 to 24, thereby improving matching accuracy and increasing the processing speed during attitude calculation. The processing using the value output from the motion sensor 145 may be performed by the imaging unit 3 or the processing unit 4.

[0053] The three-dimensional scanner 2 includes a scanner light source control unit 146 and a scanner image processing unit 147. The scanner light source control unit 146 controls the first scanner light source 62 and the second scanner light source 63. The first scanner light source 62 and the second scanner light source 63 are switched between an on state and an off state by the scanner light source control unit 146. The scanner light source control unit 146 is controlled by the scanner control unit 142. The scanner image processing unit 147 also controls the first scanner imaging unit 64, the second scanner imaging unit 65, and the texture camera 66 to perform imaging at predetermined timing. Images captured by the first scanner imaging unit 64, the second scanner imaging unit 65, and the texture camera 66 are input to the scanner image processing unit 147. The scanner image processing unit 147 performs various image processing on the input images, such as extracting edge data.

[0054] That is, the scanner image processing unit 147 generates edge data by performing edge extraction processing on the bright line image generated by the first scanner imaging unit 64 or the second scanner imaging unit 65. When the first scanner light source 62 emits multi-line light, the first scanner imaging unit 64 and the second scanner imaging unit 65 generate multi-line images. The scanner image processing unit 147 processes the multi-line images to generate edge data.

[0055] The third wireless communication unit 144 transmits the edge data generated by the scanner image processing unit 147 in association with the identification information corresponding to the edge data generated by the trigger generation unit 38. That is, the edge data is associated with the identification information for distinguishing the edge data from other edge data. Therefore, the desired edge data can be identified based on the identification information.

[0056] Furthermore, when a trigger generated by the trigger generation unit 38 of the imaging unit 3 is transmitted to the three-dimensional scanner 2, the three-dimensional scanner 2 receives the trigger as a synchronization signal via the optical communication interface 144a of the third wireless communication unit 144. Upon receiving the trigger, the scanner light source control unit 146 causes the first scanner light source 62 or the second scanner light source 63 to emit pattern light, the scanner image processing unit 147 causes the first scanner imaging unit 64 and the second scanner imaging unit 65 to capture images, and the marker illumination control unit 141 causes the scanner markers 21a, 22a, 23a, and 24a to emit light. The illumination of the pattern light from the first scanner light source 62 or the second scanner light source 63, the imaging by the first scanner imaging unit 64 and the second scanner imaging unit 65, and the illumination of the scanner markers 21a, 22a, 23a, and 24a are synchronized.

[0057] The three-dimensional scanner 2 transmits the edge data generated by the scanner image processing unit 147 to the imaging unit 3 via the radio wave communication interface 144b of the third wireless communication unit 144. The imaging unit 3 receives the edge data via the radio wave communication interface 36b of the first wireless communication unit 36, and transmits the received edge data and the center position information of the scanner marker generated by the camera image processing unit 35 to the second wireless communication unit 46 of the processing unit 4.

[0058] The three-dimensional scanner 2 is equipped with an indicator light 148 and a communication control unit 149. The indicator light 148 displays the operating status of the three-dimensional scanner 2, and is controlled by the scanner control unit 142. The communication control unit 149 is a part that performs processing to communicate, for example, image data and the like.

[0059] (contact probe) The contact-type probe 5 is a handheld, portable probe similar to the three-dimensional scanner 2. As shown in FIG. 1, the probe 5 comprises a probe body 120 and a stylus 121 protruding from the probe body 120. A contactor 121a for contacting the workpiece W is provided at the tip of the stylus 121. This contactor 121a is, for example, spherical. The contactor 121a is a part for specifying the position of a measurement point on the workpiece W and various other specified points. The probe body 120 also has a gripping portion 5A in the middle in the longitudinal direction, so that the measurement operator can hold the gripping portion 5A with one hand to move or change the orientation of the probe 5 during measurement.

[0060] A plurality of probe markers 5B are provided at intervals on the probe main body 120. For example, a plurality of probe markers 5B are provided at intervals on one longitudinal end side of the probe main body 120, and a plurality of probe markers 5B are also provided at intervals on the other longitudinal end side of the probe main body 120.

[0061] FIG. 5 shows the circuit configuration of the probe 5. Although FIG. 5 shows only one probe marker 5B, multiple probe markers 5B are actually provided. A probe camera 122 is provided near the stylus 121. The probe 5 includes a display unit 123a configured with a liquid crystal display, an organic EL display, or the like, a touch panel 123b that can be operated by touch, and a display control unit 123c. An operation unit 124 having multiple buttons and the like is also provided near the display unit 123a. The probe 5 also includes a probe control unit 125, a memory unit 126, a probe marker illumination control unit 127, a fourth wireless communication unit 128, a motion sensor 129, and the like. The probe 5 also includes a battery 5C that serves as a power source.

[0062] The display control unit 123c is a part that controls the display unit 123a based on signals output from the probe control unit 125, and causes the display unit 123a to display various images, user interfaces, etc. The probe control unit 125 acquires operations performed by the user on the display unit 123a based on signals output from the touch panel 123b.

[0063] The probe marker lighting control unit 127 is a unit that controls the probe marker 5B. The probe marker 5B can be switched between a lit state and an unlit state by the probe marker lighting control unit 127. The probe marker lighting control unit 127 is controlled by the probe control unit 125. The memory unit 126 is capable of storing control programs and the like.

[0064] The fourth wireless communication unit 128 has an optical communication interface 128a and a radio wave communication interface 128b, similar to the first wireless communication unit 36 ​​of the imaging unit 3. The optical communication interface 128a is a unit that receives a trigger transmitted via the optical communication interface 36a of the imaging unit 3. Upon receiving the trigger, the probe marker illumination control unit 127 illuminates the probe marker 5B. This allows the imaging of the probe marker by the imaging unit 3 to be synchronized with the illumination of the probe marker 5B. The radio wave communication interface 128b may use a different radio wave communication method from the radio wave communication interface 144b of the three-dimensional scanner 2. For example, if the radio wave communication interface 144b of the three-dimensional scanner 2 is configured to establish a wireless LAN, the radio wave communication interface 128b of the fourth wireless communication unit 128 may be configured as a unit capable of Bluetooth communication, which has a slower communication speed than a wireless LAN.

[0065] The motion sensor 129 is composed of a sensor that detects the acceleration and angular velocity of the probe 5, and the detected values ​​are output to the probe control unit 125 and used for various calculation processes, such as calculating the attitude of the probe 5, as well as calculating the attitude of the three-dimensional scanner 2.

[0066] (Processing content by processing unit) The processing unit 4 includes a three-dimensional coordinate generation unit 43a, a three-dimensional data generation unit 43b, a setting unit 43c, and a measurement unit 43d. The three-dimensional coordinate generation unit 43a sequentially generates three-dimensional coordinates of a reference workpiece and three-dimensional coordinates of a measurement workpiece based on images including pattern light generated by the first scanner imaging unit 64 and the second scanner imaging unit 65. The three-dimensional data generation unit 43b also generates three-dimensional shape data of the reference workpiece and three-dimensional shape data of the measurement workpiece based on the three-dimensional coordinates sequentially generated by the three-dimensional coordinate generation unit 43a. For example, if the three-dimensional coordinates generated by the three-dimensional coordinate generation unit 43a include incorrect points, the data editing unit 43h (described later) can remove the incorrect points, and the remaining final points can be converted into point cloud data or mesh data by the three-dimensional data generation unit 43b.

[0067] Specifically, when the first scanner imaging unit 64 and the second scanner imaging unit 65 of the three-dimensional scanner 2 capture images of a reference workpiece, the processing unit 4 receives the center position information of the scanner marker generated by the camera image processing unit 35 via the second wireless communication unit 46. The three-dimensional coordinate generation unit 43a sequentially generates the three-dimensional coordinates of the reference workpiece. The three-dimensional data generation unit 43b also acquires the edge data generated by the scanner image processing unit 147, the center position information of the scanner marker generated by the camera image processing unit 35, and the position and orientation information of the scanner imaging camera 32. The three-dimensional data generation unit 43b generates point cloud data and mesh data representing the three-dimensional shape of the reference workpiece based on the edge data, the center position information of the scanner marker, and the position and orientation information of the scanner imaging camera 32. Point cloud data and mesh data can also be generated for the measurement workpiece in a similar manner.

[0068] Edge data is calculated for each multi-line image generated by the first scanner imaging unit 64 and the second scanner imaging unit 65. The edge data is calculated by identifying changes in brightness value for each Y coordinate of the multi-line image and performing arithmetic processing such as differentiation on the changes in brightness value. In other words, the edge data is data that indicates the position (X coordinate) of the bright line at each Y coordinate.

[0069] Furthermore, the central position information of the self-luminous markers 21a, 22a, 23a, and 24a is generated in the following manner: First, the camera image processing unit 35 acquires arrangement information of each of the self-luminous markers 21a, 22a, 23a, and 24a from the storage unit 143 of the three-dimensional scanner 2. Then, based on the arrangement information of the self-luminous markers 21a, 22a, 23a, and 24a acquired from the storage unit 143 of the three-dimensional scanner 2 and the relative three-dimensional position information between the markers included in the marker image generated by the camera image processing unit 35, the camera image processing unit 3 calculates the positions at which each of the markers 21a, 22a, 23a, and 24a will be imaged by the imaging unit 3 when the relative position or attitude of the three-dimensional scanner 2 with respect to the imaging unit 3 is changed, and matches the calculated positions of each of the markers 21a, 22a, 23a, and 24a with the marker positions on the image. Then, the relative position and orientation of the three-dimensional scanner 2 with respect to the imaging unit 3 that minimizes the error between the calculated positions of each of the markers 21a, 22a, 23a, and 24a and the marker positions on the image is calculated, and generated as central position information of the self-luminous markers 21a, 22a, 23a, and 24a. That is, the camera image processing unit 35 virtually changes the position and orientation of the three-dimensional scanner 2, thereby virtually changing the arrangement information of the self-luminous markers 21a, 22a, 23a, and 24a acquired from the storage unit 143 of the three-dimensional scanner 2, and calculates the position and orientation that matches the marker image generated by the camera image processing unit 35, thereby generating central position information of the self-luminous markers 21a, 22a, 23a, and 24a. This position and orientation information calculation process may be referred to as bundle adjustment.

[0070] The central position information of the self-luminous markers 21a, 22a, 23a, and 24a calculated here is based on the scanner imaging camera 32. Therefore, the camera image processing unit 35 calculates position and orientation information of the three-dimensional scanner 2 based on the reference camera 34, based on position and orientation information of the scanner imaging camera 32 based on the reference camera 34, and position and orientation information of the three-dimensional scanner 2 based on the scanner imaging camera 32, thereby generating central position information of the self-luminous markers 21a, 22a, 23a, and 24a based on the reference camera 34.

[0071] When imaging is performed, the three-dimensional coordinate generation unit 43a receives the edge data generated by the scanner image processing unit 147, the identification information corresponding to the edge data, the center position information of the scanner marker generated by the camera image processing unit 35, and the identification information corresponding to the center position information of the scanner marker. The three-dimensional coordinate generation unit 43a can generate a point cloud indicating the three-dimensional shape of the workpiece W based on the edge data, the identification information corresponding to the edge data, the center position information of the scanner marker, the identification information corresponding to the center position information of the scanner marker, and the calibration data of the three-dimensional scanner 2 stored in the memory unit 45 of the processing unit 4.

[0072] As shown in Fig. 2, the imaging unit 3 is equipped with a memory 39a that sequentially stores edge data generated by the scanner image processing unit 147, and an association unit 39b that associates the edge data with center position information of the scanner marker based on identification information. For example, when sequentially measuring multiple workpieces W or sequentially measuring different parts of the same workpiece W, the scanner image processing unit 147 generates multiple pieces of edge data. Each of the generated multiple pieces of edge data is associated with different identification information and transmitted from the third wireless communication unit 144 of the three-dimensional scanner 2 to the imaging unit 3. The multiple pieces of edge data transmitted from the third wireless communication unit 144 of the three-dimensional scanner 2 are stored in the memory 39a of the imaging unit 3 with the identification information associated with them.

[0073] When the three-dimensional coordinate generation unit 43a generates a point cloud representing a three-dimensional shape, the association unit 39b identifies center position information of the scanner marker to be transmitted to the three-dimensional coordinate generation unit 43a. The association unit 39b identifies edge data having identification information linked to the identified center position information of the scanner marker from among the multiple edge data stored in the memory 39a. The association unit 39b then associates the identified edge data with the center position information of the scanner marker. The communication unit 37 of the imaging unit 3 transmits the edge data identified by the association unit 39b to the three-dimensional coordinate generation unit 43a in a state in which the edge data is associated with the center position information of the scanner marker.

[0074] When measuring with the probe 5, the measurement operator brings the contact 121a into contact with the point on the workpiece W to be measured (measurement point) and then operates the operation unit 124. The trigger generation unit 38 then generates a trigger, and the following processes are synchronously executed: emission of light from the probe marker 5B, imaging of the probe marker, emission of light from the light-emitting element 31b, and imaging of the light-emitting element 31b by the reference camera 34. The camera image processing unit 35 generates center position information of the probe marker. The processing unit 4 receives the center position information of the probe marker generated by the camera image processing unit 35 via the second wireless communication unit 46. The three-dimensional coordinate generation unit 43a generates three-dimensional coordinates of the measurement point indicated by the contact 121a based on the center position information of the probe marker generated by the camera image processing unit 35 and the position and orientation information of the scanner imaging camera 32.

[0075] (Creating a configuration file) When creating a setting file, first, a coordinate system is set using a contact-type probe 5. In this embodiment, for example, a reference coordinate system can be defined using any part of a reference workpiece as a reference. In this case, similar to the design drawing of the reference workpiece, a reference coordinate system is set using any part of the reference workpiece as a reference, and the coordinates of the measurement position are calculated according to that reference coordinate system. When setting the reference coordinate system, the reference coordinate system can be set using any part of the reference workpiece as a reference, regardless of the position and posture of the reference workpiece.

[0076] When the user performs a setting start operation, the display control unit 44 generates a user interface screen 100 shown in FIG. 6 and displays it on the monitor 41. The user interface screen 100 is provided with a coordinate system setting button 101, an image display area 102, a measurement item setting area 103, and a measurement element setting area 104. The image display area 102 displays the set reference coordinate system, various images, and the like. The measurement item setting area 103 includes measurement items such as "distance" for measuring the distance between two geometric elements and "angle" for measuring the angle between two geometric elements, which can be selected by the user. The measurement element setting area 104 includes measurement elements such as "plane," "straight line," "point," and "circle," which can be selected by the user.

[0077] The control unit 43 operates in a coordinate system setting mode when it detects that the coordinate system setting button 101 on the user interface screen 100 has been operated. In the coordinate system setting mode, the coordinate system generation unit 43i of the control unit 43 generates a reference workpiece coordinate system based on the positions of multiple measurement points specified on the reference workpiece by the contact-type probe 5.

[0078] In this embodiment, the coordinate system generation unit 43i sequentially sets one plane, one line, and one point to set the reference coordinate system. In the coordinate system setting mode, the user designates one surface (e.g., the top surface) of the reference workpiece as the "plane." When designating the "plane," the user sequentially designates four points spaced apart on the surface of the reference workpiece using the contact-type probe 5. Once the designation of three points is complete, the plane passing through the three points is set as the "plane," and the position of the plane is calculated. This plane is defined, for example, as the XY plane.

[0079] Furthermore, after specifying a plane, when specifying a "straight line", two points spaced apart from each other on one surface of the reference workpiece can be sequentially specified with the contact probe 5. When specifying a "point", points on one surface of the reference workpiece can be sequentially specified with the contact probe 5. The method for setting the reference coordinate system is not particularly limited, and for example, the reference coordinate system may be set by setting three planes including the point where the origin should be set as measurement planes.

[0080] Once the reference coordinate system has been set, the reference workpiece is scanned using the non-contact three-dimensional scanner 2. The three-dimensional coordinate generation unit 43a then sequentially generates three-dimensional coordinates of the reference workpiece based on images containing pattern light generated by the scanner imaging units 64 and 65. Once the three-dimensional coordinates of the reference workpiece have been generated, the three-dimensional data generation unit 43b generates reference three-dimensional shape data of the reference workpiece based on the three-dimensional coordinates sequentially generated by the three-dimensional coordinate generation unit 43a. Once the reference three-dimensional shape data has been generated by the three-dimensional data generation unit 43b, the display control unit 44 generates display data for displaying the reference three-dimensional shape data on the monitor (display unit) 41. For example, the monitor 41 displays the reference three-dimensional shape data in the image display area 102 of the user interface screen 100, as shown in FIG. 7 .

[0081] After acquiring mesh data representing the three-dimensional shape of the reference workpiece in this manner, the desired mesh data is obtained by partially deleting (removing) unnecessary data from the acquired mesh data. That is, for example, a user interface screen 100 shown in FIG. 7 has a data editing button 105. When the user operates the data editing button 105, data editing is enabled by the data editing unit 43h of the control unit 43. The data editing unit 43h may delete, for example, data of the background portion other than the reference workpiece or data of unnecessary portions of the reference workpiece. The data of the unnecessary portions may include invalid points. When deleting data, the user simply specifies the area to be deleted, and only the data of the area specified by the user is deleted, and the necessary mesh data is generated by the three-dimensional data generation unit 43b. After the three-dimensional data generation unit 43b generates the mesh data, unnecessary points may be deleted.

[0082] The setting unit 43c is a part that sets measurement elements for the reference three-dimensional shape data generated by the three-dimensional data generation unit 43b and displayed on the monitor 41. The measurement elements can be set using the measurement item setting area 103 and the measurement element setting area 104 on the user interface screen 100 shown in FIG. 7. For example, as shown in FIG. 8, if you want to set a "plane" as a measurement element for the reference three-dimensional shape data displayed on the monitor 41, you select "plane" in the measurement element setting area 104. Then, the setting unit 43c sets the "plane" as the measurement element.

[0083] The measurement unit 43d is a part that performs measurements based on the measurement elements set by the setting unit 43c. The measurement unit 43d measures the "distance" or "angle" selected in the measurement item setting area 103.

[0084] When the measurement elements are set by the setting unit 43c, a setting file is generated that associates the reference three-dimensional shape data generated by the three-dimensional data generation unit 43b with the measurement elements set by the setting unit 43c. This setting file is stored as a measurement reproduction template in the storage unit 45. Only one measurement reproduction template may be stored in the storage unit 45, or multiple measurement reproduction templates may be stored.

[0085] When setting measurement elements, the measurement elements can include not only the extraction of planes from the scan results, but also comparison with reference CAD data.

[0086] The processing unit 4 has a receiving unit 43e, a reading unit 43f, and a registration unit 43g. The receiving unit 43e is a unit that receives the selection of one measurement reproduction template from the measurement reproduction templates stored in the storage unit 45. For example, with a list of measurement reproduction templates stored in the storage unit 45 displayed on the monitor 41, the user can select a desired measurement reproduction template by operating the operation input unit 42. The receiving unit 43e receives this selection operation by the user, making it possible to identify the selected measurement reproduction template.

[0087] The reading unit 43f is a part that reads out the reference three-dimensional shape data and measurement elements included in one measurement / reproduction template accepted by the accepting unit 43e from the storage unit 45. The registration unit 43g is a part that registers the reference three-dimensional shape data and measurement elements read out by the reading unit 43f as reference three-dimensional shape data and reference measurement elements, respectively.

[0088] In order to reproduce the measurement of the measurement elements during the measurement process described below, the gaze direction, click position, operation history, etc. at the time of creating the setting file are saved in the setting file. For example, the reception unit 43e receives input of an extraction reference point when extracting a geometric element, which is a measurement element, on the monitor 41, and a selection of the type of geometric element to be extracted. The input of the extraction reference point and the selection of the type of geometric element are made by the user. The type of geometric element is, for example, a plane, a line, etc. The memory unit 45 stores the extraction reference point received by the reception unit 43e, the display attitude or gaze direction of the measurement workpiece on the monitor 41 when the extraction reference point was received, and the type of geometric element as measurement elements.

[0089] (Measurement processing) After storing the setting file as a measurement reproduction template in the memory unit 45, the user executes the measurement process. The measurement process can be performed according to the flowchart shown in FIG. 9. In step SA1 after starting, alignment with the measurement setting is performed using the contact probe 5. For example, as described above, one plane, one line, and one point are sequentially set using the contact probe 5. In this way, the coordinate system generation unit 43i can generate a measurement workpiece coordinate system based on multiple measurement points specified on the measurement workpiece W by the contact probe 5. FIG. 10 shows a measurement user interface screen 150 showing how the measurement workpiece coordinate system has been generated, and this measurement user interface screen 150 is displayed on the monitor 41.

[0090] 2, the control unit 43 has an alignment unit 43j. After the coordinate system generation unit 43i generates a measurement workpiece coordinate system, the alignment unit 43j of the control unit 43 aligns the reference workpiece with the measurement workpiece based on the reference workpiece coordinate system and measurement workpiece coordinate system generated by the coordinate system generation unit 43i. The alignment method using the contact-type probe 5 is not limited to the method of creating a coordinate system by creating and referencing elements such as planes and lines, and other methods may be used.

[0091] 11 shows the measurement user interface screen 150 that is displayed on the monitor 41 when the alignment of the reference workpiece and the measurement workpiece is complete. By performing alignment using the contact-type probe 5, the mesh data in the setting file and the position of the measurement workpiece match.

[0092] Once this alignment is performed, mesh data of the reference workpiece, which is the scan result of the setting file, i.e., reference three-dimensional shape data, is displayed on the measurement user interface screen 150 shown in Fig. 11. At this time, reference measurement elements may also be displayed on the measurement user interface screen 150. The mesh data of the reference workpiece serves as a guide when the user scans the measurement workpiece with the three-dimensional scanner 2.

[0093] In step SA1 shown in FIG. 9 , with the mesh data of the setting file serving as a guide displayed on the monitor 41, the process proceeds to step SA2, where the three-dimensional scanner 2 starts scanning the workpiece to be measured. While the reference CAD data can be displayed on the monitor 41 as a guide, there are cases where the CAD data alone cannot indicate which part of the workpiece to be measured should be scanned. In such cases, by displaying the mesh data of the setting file as a guide on the monitor 41 instead of the CAD data, as in this embodiment, the user can be presented with not only the shape of the workpiece but also the specific measurement points on the workpiece, allowing the user to more appropriately scan the measurement points.

[0094] 12 shows a measurement user interface screen 150 displayed on the monitor 41 when measurement is being performed with the three-dimensional scanner 2, with the mesh data in the setting file displayed as a guide on the monitor 41. The line indicated by the symbol L in FIG. 12 represents the multi-line light emitted from the first scanner light source 62 of the three-dimensional scanner 2.

[0095] As shown in FIG. 12, when a user scans a measurement workpiece using the three-dimensional scanner 2, the three-dimensional coordinate generation unit 43a sequentially generates three-dimensional coordinates of the measurement workpiece based on images containing pattern light generated by the scanner imaging units 64 and 65. The display control unit 44 then sequentially generates display data in which the three-dimensional coordinates of the measurement workpiece sequentially generated by the three-dimensional coordinate generation unit 43a are superimposed on a three-dimensional shape based on the reference three-dimensional shape data registered by the registration unit 43g. Images based on the generated display data are incorporated into the measurement user interface screen 150 and displayed on the monitor 41. By viewing the monitor 41, the user can perform efficient scanning while checking in real time whether the required range has been scanned. The user can also determine whether the measurement location has been scanned. If the measurement location has not been scanned, the user simply moves the three-dimensional scanner 2 so that the measurement location can be scanned.

[0096] The third wireless communication unit 144 of the three-dimensional scanner 2 sequentially receives display data in which the three-dimensional coordinates of the measurement workpiece are superimposed on a three-dimensional shape based on the reference three-dimensional shape data sequentially generated by the display control unit 44 of the processing unit 4. The display unit 113 of the three-dimensional scanner 2 displays the display data received by the third wireless communication unit 144. In this way, a measurement user interface screen 150 such as that shown in Fig. 12 is displayed on the display unit 113 of the three-dimensional scanner 2 held by the user, so that the user can see in real time whether the required range has been scanned simply by looking at their hand.

[0097] In step SA3, the three-dimensional data generation unit 43b generates three-dimensional shape data of the measurement workpiece based on the three-dimensional coordinates of the measurement workpiece acquired in step SA2. Figure 13 shows the state after scanning by the three-dimensional scanner 2 is completed, and the three-dimensional shape data of the measurement workpiece is displayed on the monitor 41.

[0098] In step SA4, mesh data of the measurement workpiece that is a certain distance away from the mesh data (reference three-dimensional shape data) in the setting file is deleted. This step SA4 is a step performed by the data editing unit 43h. With the reference three-dimensional shape data and the three-dimensional shape data of the measurement workpiece aligned by the alignment unit 43j, the data editing unit 43h partially removes three-dimensional shape data from the three-dimensional shape data of the measurement workpiece based on the distance between the reference three-dimensional shape data and the three-dimensional shape data of the measurement workpiece. For example, if the three-dimensional shape data of the measurement workpiece is away from the reference three-dimensional shape data by more than a certain distance, only the three-dimensional shape data of the measurement workpiece that is away from the reference three-dimensional shape data by more than the certain distance is removed. The "predetermined distance" may be set by the user.

[0099] In step SA5, the user determines whether the necessary areas of the measurement workpiece have been scanned by the three-dimensional scanner 2. If the necessary areas of the measurement workpiece have not been scanned, the process proceeds to step SA2, where the user performs an additional scan of the measurement workpiece using the three-dimensional scanner 2. If the necessary areas of the measurement workpiece have been scanned, the process proceeds to step SA5, where the measurement unit 43d performs measurements based on the reference measurement elements registered by the registration unit 43g and the three-dimensional shape data of the measurement workpiece. In other words, by obtaining mesh data that reproduces both the position and shape of the setting file, it is possible to measure the set measurement elements.

[0100] The measurement unit 32d acquires the display attitude or line of sight direction stored as a measurement element in the storage unit 45 from the setting file. Based on the acquired display attitude or line of sight direction, the measurement unit 32d updates the position and attitude of the three-dimensional data of the measurement workpiece, extracts geometric elements based on the extraction reference points and types of geometric elements included in the setting file in the updated position and attitude state, and performs various measurements based on the extracted geometric elements (see FIG. 15).

[0101] The control unit 43 includes a synthesis unit 43k. The synthesis unit 43k acquires three-dimensional coordinates using different parameters and synthesizes the acquired three-dimensional coordinates to generate one set of three-dimensional shape data. An example of a parameter is resolution, which is the density of measured points. When the synthesis unit 43k synthesizes three-dimensional coordinates acquired using multiple parameters to generate one set of three-dimensional shape data, the three-dimensional shape data is associated with point clouds acquired using each parameter and mesh data obtained by synthesizing the point clouds acquired using each parameter. When displaying the reference three-dimensional shape data on the monitor 41, the display control unit 44 projects the point clouds acquired using each parameter onto the mesh data, thereby displaying the data acquired using each parameter in a distinguishable manner. In addition to resolution, parameters such as exposure time and laser type may also be used, and multiple types of point clouds with different settings for these parameters may be stored in the memory unit 45. The laser type here refers to whether multi-line light or single-line light is used as the pattern light. Then, by projecting each point cloud acquired with different settings onto mesh data, it is possible to display the areas acquired with each setting on the mesh data in a distinguishable manner.

[0102] (Other embodiments) The above-described embodiments are merely examples in all respects and should not be construed as limiting. Furthermore, all modifications and variations within the scope of the claims are within the scope of the present invention.

[0103] For example, the display form of the three-dimensional shape of the reference workpiece may include, for example, opaque, semi-transparent, wireframe, etc. It may be possible to switch between opaque, semi-transparent, wireframe, etc. depending on the workpiece to be measured. In an opaque display form, the back of the workpiece cannot be seen, making it possible to distinguish between the front and back.

[0104] Along with displaying the three-dimensional shape of the reference workpiece, it is also possible to indicate to the user, for example, by color or transparency, whether a sufficient number of point clouds have been obtained by measurement using the three-dimensional scanner 2. For example, by making the color on the measurement user interface screen 150 darker or decreasing the transparency as the number of point clouds increases, the user can determine when to end the scan while looking at the color or transparency displayed on the monitor 41 or display unit 113.

[0105] Furthermore, the settings for scanning include, for example, resolution, exposure time, etc. A different three-dimensional guide may be displayed on the measurement user interface screen 150 for each resolution and exposure time.

[0106] In addition, the color and display state of the mesh data showing the three-dimensional shape of the reference workpiece can be changed depending on the scan settings, and the user can be informed of which settings to use for scanning. For example, as shown in the flowchart of FIG. 16 and the conceptual diagram of FIG. 17, the scan settings can be presented using not only mesh data information but also point cloud information at the time of scanning. That is, in step SB1, since the mesh data and point cloud are stored separately, the mesh data and point cloud are read separately and identical alignment is performed for each. In step SB2, the point cloud and mesh are associated. In this example, this is done by projecting the point cloud onto the mesh based on the position coordinates and normal information of the point cloud. In step SB3, the display color of the mesh data is changed according to the projected point cloud settings (Setting A and Setting B shown in FIG. 17).

[0107] The progress of the scan by the three-dimensional scanner 2 may be displayed to the user by comparing it with the mesh data in the setting file, or the scan may be performed automatically. In step SC1 of the flowchart shown in FIG. 18, the user scans the measurement workpiece using the three-dimensional scanner 2 to acquire a point cloud. In step SC2, the control unit 43 determines whether the distance between the acquired point cloud and the mesh data in the setting file is equal to or less than a certain value. If the distance between the acquired point cloud and the mesh data in the setting file is equal to or less than a certain value, the process proceeds to step SC3, where the point cloud is added as the point cloud measured in step SC1. If the distance between the acquired point cloud and the mesh data in the setting file is not equal to or less than a certain value, the process proceeds to step SC4, where the point cloud measured in step SC1 is deleted. At this time, the user is presented with a numerical value indicating the percentage of the mesh data in the setting file that has been scanned by the three-dimensional scanner 2. If the three-dimensional scanner 2 has scanned at least a certain percentage of the mesh data in the setting file, the control unit 43 may automatically stop the scan. When deleting the point cloud, rather than deleting the mesh data of the measurement workpiece that has moved away from the mesh data in the setting file after scanning, the point cloud may be deleted in real time during scanning by the three-dimensional scanner 2. [Industrial Applicability]

[0108] As described above, the present invention can be used to measure the three-dimensional shapes of various workpieces. [Explanation of symbols]

[0109] 1 Three-dimensional measuring device 2. 3D scanner 3 Imaging unit 43a 3D coordinate generator 43b 3D data generation unit 43c Setting section 43d Measurement section 43e Reception 43f Reading section 43g Registration Section 43h Data Editorial Department 43i Coordinate system generator 43j Alignment part 43k synthesis section 44 Display control unit

Claims

1. A three-dimensional measuring device for measuring a three-dimensional shape of a workpiece, a three-dimensional scanner having a scanner light source that irradiates pattern light and a scanner imaging unit that captures the pattern light irradiated by the scanner light source and generates an image including the pattern light; a three-dimensional coordinate generating unit that sequentially generates three-dimensional coordinates of a reference workpiece based on an image including pattern light generated by the scanner imaging unit; a three-dimensional data generation unit that generates reference three-dimensional shape data of the reference workpiece based on the three-dimensional coordinates sequentially generated by the three-dimensional coordinate generation unit; a display control unit that generates display data for displaying the reference three-dimensional shape data generated by the three-dimensional data generation unit on a display unit; a setting unit that sets measurement elements for the reference three-dimensional shape data generated by the three-dimensional data generation unit and displayed on the display unit; a measurement unit that performs measurement based on the measurement parameters set by the setting unit; a storage unit that stores, as a measurement reproduction template, a setting file that associates the reference three-dimensional shape data generated by the three-dimensional data generation unit with the measurement elements set by the setting unit; a reception unit that receives a selection of one template from the measurement reproduction templates stored in the storage unit; a reading unit that reads out the reference three-dimensional shape data and the measurement elements included in the one template accepted by the accepting unit; a registration unit that registers the reference three-dimensional shape data and the measurement element read by the reading unit as reference three-dimensional shape data and reference measurement element, the three-dimensional coordinate generation unit sequentially generates three-dimensional coordinates of the measurement workpiece based on an image including pattern light generated by the scanner imaging unit; the display control unit sequentially generates display data in which the three-dimensional coordinates of the measurement workpiece sequentially generated by the three-dimensional coordinate generation unit are superimposed on the three-dimensional shape based on the reference three-dimensional shape data registered by the registration unit, the three-dimensional data generation unit generates three-dimensional shape data of the measurement workpiece based on the three-dimensional coordinates of the measurement workpiece; The measurement unit performs measurement based on the reference measurement element registered by the registration unit and three-dimensional shape data of the measurement workpiece.

2. 2. The three-dimensional measuring apparatus according to claim 1, The three-dimensional measuring device further comprises a data editing unit that partially removes three-dimensional shape data from the reference three-dimensional shape data generated by the three-dimensional data generating unit.

3. 3. The three-dimensional measuring apparatus according to claim 2, A three-dimensional measuring device in which the data editing unit partially removes three-dimensional shape data from the three-dimensional shape data of the measurement work based on the distance between the reference three-dimensional shape data registered by the registration unit and the three-dimensional shape data of the measurement work.

4. 2. The three-dimensional measuring apparatus according to claim 1, a contact probe that indicates the position of the measurement point; a coordinate system generating unit that generates a coordinate system based on the positions of the plurality of measurement points indicated by the contact-type probe, The coordinate system generation unit generates a reference work coordinate system based on a plurality of measurement points indicated on the reference work by the contact probe, and generates a measurement work coordinate system based on a plurality of measurement points indicated on the measurement work by the contact probe, a three-dimensional measuring device.

5. 5. The three-dimensional measuring apparatus according to claim 4, an alignment unit that aligns the reference workpiece and the measurement workpiece based on the reference workpiece coordinate system and the measurement workpiece coordinate system generated by the coordinate system generation unit; a data editing unit that partially removes three-dimensional shape data from the three-dimensional shape data of the measurement workpiece based on the distance between the reference three-dimensional shape data registered by the registration unit and the three-dimensional shape data of the measurement workpiece, when the reference three-dimensional shape data and the three-dimensional shape data of the measurement workpiece are aligned by the alignment unit.

6. 2. The three-dimensional measuring apparatus according to claim 1, the receiving unit receives an input of an extraction reference point and a selection of a type of geometric element to be extracted; The memory unit stores the extracted reference point received by the receiving unit, the display posture of the measurement work on the display unit when the extracted reference point was received, and the type of the geometric element as the measurement elements.

7. 7. The three-dimensional measuring apparatus according to claim 6, The measurement unit updates the position and orientation of the three-dimensional data of the measurement workpiece based on the display orientation stored as the measurement element, extracts the geometric element based on the extraction reference point and the type of the geometric element in the updated position and orientation state, and performs measurement based on the extracted geometric element.

8. 2. The three-dimensional measuring apparatus according to claim 1, a synthesis unit that acquires three-dimensional coordinates using different parameters and synthesizes the acquired three-dimensional coordinates to generate one piece of three-dimensional shape data; A three-dimensional measuring device in which the three-dimensional shape data corresponds to a point cloud acquired using each parameter and mesh data obtained by combining the point clouds acquired using each parameter.

9. 9. The three-dimensional measuring apparatus according to claim 8, The display control unit, when displaying the reference three-dimensional shape data, projects the point cloud acquired with each parameter onto mesh data, thereby displaying the data acquired with each parameter in an identifiable manner.

10. 2. The three-dimensional measuring apparatus according to claim 1, the three-dimensional scanner has a communication unit and a scanner display unit; the communication unit sequentially receives display data in which the three-dimensional coordinates of the measurement workpiece are superimposed on the three-dimensional shape based on the reference three-dimensional shape data sequentially generated by the display control unit; The scanner display unit displays the display data received by the communication unit.

11. A three-dimensional measuring method for measuring a three-dimensional shape of a workpiece, comprising: capturing an image of pattern light irradiated by a scanner light source to generate an image including the pattern light; Based on the image including the generated pattern light, three-dimensional coordinates of the reference workpiece are sequentially generated; generating reference three-dimensional shape data of the reference workpiece based on the three-dimensional coordinates generated sequentially; displaying the generated reference three-dimensional shape data on a display unit, and setting measurement elements for the reference three-dimensional shape data displayed on the display unit; storing a setting file in which the generated reference three-dimensional shape data and the set measurement elements are associated with each other as a measurement reproduction template; Accepting a selection of one template from the stored measurement reproduction templates; reading out the reference three-dimensional shape data and the measurement elements included in the received template; The read reference three-dimensional shape data and the measurement element are registered as reference three-dimensional shape data and reference measurement element, Sequentially generating display data in which the three-dimensional coordinates of the sequentially generated measurement workpieces are superimposed on the three-dimensional shape based on the registered reference three-dimensional shape data; generating three-dimensional shape data of the measurement workpiece based on the three-dimensional coordinates of the measurement workpiece; A three-dimensional measuring method for measuring the workpiece to be measured based on the registered reference measurement element and three-dimensional shape data of the workpiece to be measured.

12. A storage medium storing a three-dimensional measurement program for causing a computer to execute a three-dimensional measurement method for measuring a three-dimensional shape of a workpiece, The three-dimensional measurement method includes: capturing an image of pattern light irradiated by a scanner light source to generate an image including the pattern light; Based on the image including the generated pattern light, three-dimensional coordinates of the reference workpiece are sequentially generated; generating reference three-dimensional shape data of the reference workpiece based on the three-dimensional coordinates generated sequentially; displaying the generated reference three-dimensional shape data on a display unit, and setting measurement elements for the reference three-dimensional shape data displayed on the display unit; storing a setting file in which the generated reference three-dimensional shape data and the set measurement elements are associated with each other as a measurement reproduction template; Accepting a selection of one template from the stored measurement reproduction templates; reading out the reference three-dimensional shape data and the measurement elements included in the received template; The read reference three-dimensional shape data and the measurement element are registered as reference three-dimensional shape data and reference measurement element, Sequentially generating display data in which the three-dimensional coordinates of the sequentially generated measurement workpieces are superimposed on the three-dimensional shape based on the registered reference three-dimensional shape data; generating three-dimensional shape data of the measurement workpiece based on the three-dimensional coordinates of the measurement workpiece; A method for measuring the measurement workpiece based on the registered reference measurement element and three-dimensional shape data of the measurement workpiece, A storage medium storing a three-dimensional measurement program that causes the computer to execute the three-dimensional measurement method.

Images