Three-dimensional measurement device, three-dimensional measurement method, and storage medium storing three-dimensional measurement program

Abstract

The invention provides a three-dimensional measurement apparatus, a three-dimensional measurement method, and a storage medium storing a three-dimensional measurement program. When acquiring three-dimensional data by a three-dimensional scanner, a user can easily know whether necessary three-dimensional data has been acquired. The three-dimensional coordinate generation unit sequentially generates three-dimensional coordinates of the measurement workpiece on the basis of the image including the pattern light generated by the scanner imaging section. The display control unit sequentially generates display data in which the three-dimensional coordinates of the measurement workpiece sequentially generated by the three-dimensional coordinate generation unit are superimposed on the three-dimensional shape based on the reference three-dimensional shape data registered by the registration unit. The three-dimensional data generation unit generates three-dimensional shape data of the measurement workpiece based on three-dimensional coordinates of the measurement workpiece. The measurement unit performs measurement based on the reference measurement element registered by the registration unit and the three-dimensional shape data of the measurement workpiece.

Description

Technical Field

[0001] This disclosure relates to a three-dimensional measuring apparatus, a three-dimensional measuring method, and a storage medium for storing three-dimensional measuring programs for measuring the three-dimensional shape of a workpiece. Background Technology

[0002] For example, JP 2016-194896 A discloses a method for inspecting an object, which uses a 3D scanner to acquire 3D scan data for indicating the shape of the object to be inspected, aligns CAD data and 3D scan data with the same coordinates, and then compares the inspection elements of the CAD data with the inspection elements of the 3D scan data.

[0003] However, in JP 2016-194896 A, when 3D data is acquired by a 3D scanner, data used as a reference, such as CAD data, is not indicated, making it difficult for users to determine whether the necessary 3D data has been acquired. Summary of the Invention

[0004] This disclosure is made in view of this, and the purpose of this disclosure is to make it easy for users to know whether the necessary three-dimensional data has been acquired when acquiring three-dimensional data by a 3D scanner.

[0005] To achieve the above objectives, according to one embodiment of the present disclosure, a three-dimensional measuring device for measuring the three-dimensional shape of a workpiece can be assumed.

[0006] A three-dimensional measuring device includes: a three-dimensional scanner, comprising a scanner light source for emitting patterned light and a scanner camera unit for capturing the patterned light emitted by the scanner light source and generating an image including the patterned light; a three-dimensional coordinate generation unit that sequentially generates three-dimensional coordinates of a reference workpiece based on the image including the patterned light generated by the scanner camera unit; a three-dimensional data generation unit that generates reference three-dimensional shape data of the reference workpiece based on the three-dimensional coordinates sequentially generated by the three-dimensional coordinate generation unit; a display control unit that generates display data for causing a display unit to display the reference three-dimensional shape data generated by the three-dimensional data generation unit; and a setting unit that sets the reference three-dimensional shape data generated by the three-dimensional data generation unit and displayed on the display unit. The system comprises: a 3D shape data setting unit for setting measurement elements; a measurement unit for performing measurements based on the measurement elements set by the setting unit; a storage unit for storing setting files as measurement reproduction templates, each setting file being obtained by associating the reference 3D shape data generated by the 3D data generation unit with the measurement elements set by the setting unit; a receiving unit for receiving the selection of one of the measurement reproduction templates stored in the storage unit; a reading unit for reading the reference 3D shape data and the measurement elements included in one of the templates received by the receiving unit; and a registration unit for registering the reference 3D shape data and the measurement elements read by the reading unit as reference 3D shape data and reference measurement elements.

[0007] The three-dimensional coordinate generation unit can sequentially generate the three-dimensional coordinates of the workpiece based on the image including the patterned light generated by the scanner camera unit. The display control unit can sequentially generate display data. In the display data, the three-dimensional coordinates of the workpiece generated sequentially by the three-dimensional coordinate generation unit are superimposed on a three-dimensional shape based on the reference three-dimensional shape data registered by the registration unit. The three-dimensional data generation unit can generate the three-dimensional shape data of the workpiece based on the three-dimensional coordinates of the workpiece. The measurement unit can perform measurement based on the reference measurement element registered by the registration unit and the three-dimensional shape data of the workpiece.

[0008] According to this configuration, when the scanner camera of a 3D scanner captures an image of a workpiece, reference 3D shape data of the reference workpiece is generated and displayed on the display unit. Measurement elements can be set for the reference 3D shape data displayed on the display unit. The reference 3D shape data and measurement elements are registered as reference 3D shape data and reference measurement elements, respectively. When the scanner camera captures an image of the measured workpiece, display data is generated sequentially, in which the sequentially generated 3D coordinates of the measured workpiece are superimposed on the 3D shape based on the reference 3D shape data. Therefore, when acquiring 3D data through a 3D scanner, the user can easily determine whether the necessary 3D data has been acquired.

[0009] According to another embodiment of this disclosure, a three-dimensional measurement method for measuring the three-dimensional shape of a workpiece can be assumed. In this three-dimensional measurement method, the following operations can be performed: capturing an image of patterned light emitted by a scanner light source to generate an image including the patterned light; sequentially generating three-dimensional coordinates of a reference workpiece based on the generated image including the patterned light; generating reference three-dimensional shape data of the reference workpiece based on the sequentially generated three-dimensional coordinates; displaying the generated reference three-dimensional shape data on a display unit, and setting measurement elements for the reference three-dimensional shape data displayed on the display unit; storing setting files as measurement reproduction templates, each setting file being obtained by associating the generated reference three-dimensional shape data with the set measurement elements; receiving data for… The process includes: selecting one template from the stored measurement reproduction templates; reading the reference 3D shape data and measurement elements included in the received template; registering the read reference 3D shape data and the read measurement elements as reference 3D shape data and reference measurement elements; sequentially generating display data, in which the sequentially generated 3D coordinates of the measurement workpiece are superimposed on the 3D shape based on the registered reference 3D shape data; generating the 3D shape data of the measurement workpiece based on the 3D coordinates of the measurement workpiece; and measuring the measurement workpiece based on the registered reference measurement elements and the 3D shape data of the measurement workpiece.

[0010] According to another embodiment of the present disclosure, a storage medium for storing a three-dimensional measurement program can be provided, the three-dimensional measurement program being used to enable a computer to execute a three-dimensional measurement method for measuring the three-dimensional shape of a workpiece.

[0011] As mentioned above, users can easily determine whether the necessary 3D data has been acquired. Attached Figure Description

[0012] Figure 1 This is a diagram illustrating the configuration of a three-dimensional measuring device according to an embodiment of the present invention.

[0013] Figure 2This is a block diagram of the camera unit and the processing unit.

[0014] Figure 3 This is a stereoscopic image of a 3D scanner viewed from below.

[0015] Figure 4 This is a block diagram showing the circuit configuration of a 3D scanner;

[0016] Figure 5 This is a block diagram showing the circuit configuration of the detector;

[0017] Figure 6 This is an example diagram showing a user interface screen;

[0018] Figure 7 This is a diagram showing an example of grid data;

[0019] Figure 8 This is a diagram illustrating an example where a plane is set as a measurement element;

[0020] Figure 9 This is a flowchart illustrating an example of the measurement processing flow;

[0021] Figure 10 This is a diagram showing an example of a user interface screen used for measurement, which illustrates the state of generating the coordinate system for the measurement workpiece;

[0022] Figure 11 This is an example diagram showing a user interface screen used for measurement, which illustrates the alignment of the reference workpiece and the measuring workpiece.

[0023] Figure 12 This is an example diagram showing a user interface screen used for measurement during the measurement process of a 3D scanner;

[0024] Figure 13 This is an example diagram showing the user interface screen used for measurement after the measurement by the 3D scanner has been completed;

[0025] Figure 14 This is a diagram illustrating an example of a user interface screen for measurement in the case where the grid data of the workpiece at a certain distance is deleted from the grid data in the settings file;

[0026] Figure 15 This is a diagram showing an example of a user interface screen for measurement when the measurement elements of the workpiece are extracted;

[0027] Figure 16 This is a flowchart illustrating an example of the process when the display color of the grid data is changed according to settings;

[0028] Figure 17This is a concept diagram showing the display color of the grid data being changed according to settings; and

[0029] Figure 18 This is a flowchart illustrating an example of processing based on the scanning progress. Detailed Implementation

[0030] In the following, embodiments of the invention will be described in detail with reference to the accompanying drawings. Note that the preferred embodiments described below are merely examples and are not intended to limit the invention, its application, or its use.

[0031] Figure 1 This diagram illustrates the configuration of a three-dimensional measuring device 1 according to an embodiment of the present invention. The three-dimensional measuring device 1 is a measuring instrument for measuring the three-dimensional shape and three-dimensional coordinates of a workpiece W, and includes: a non-contact three-dimensional scanner 2 comprising a plurality of self-illuminating marks (scanner marks); a contact detector 5 comprising a plurality of self-illuminating marks (detector marks); an imaging unit 3 for capturing images of the plurality of scanner marks included in the three-dimensional scanner 2 and images of the plurality of detector marks included in the detector 5; and a processing unit 4 for measuring the three-dimensional shape and three-dimensional coordinates of the workpiece W. The marks do not necessarily have to be self-illuminating marks. The three-dimensional scanner 2 is disposed separately from the imaging unit 3 and the processing unit 4, and the measuring operator can bring the three-dimensional scanner 2 to the vicinity of the workpiece W located away from the imaging unit 3 and the processing unit 4 and have the three-dimensional scanner 2 generate a bright line image. Furthermore, the detector 5 is disposed separately from the imaging unit 3 and the processing unit 4, and the measuring operator can bring the detector 5 to the vicinity of the workpiece W located away from the imaging unit 3 and the processing unit 4 and use the detector 5 to specify measurement points.

[0032] (Configuration of camera unit 3)

[0033] The camera unit 3 is a unit that captures images of multiple scanner markers (described later) set in the 3D scanner 2 to generate a scanner marker image including the multiple scanner markers, and captures images of multiple detector markers (described later) set in the detector 5 to generate a detector marker image including the multiple detector markers. The scanner marker image including the scanner markers is generated when measurement is performed using the 3D scanner 2, and the detector marker image including the detector markers is generated when measurement is performed using the detector 5.

[0034] like Figure 2As shown, the camera unit 3 includes a base 30 and a movable camera unit 3A. The movable camera unit 3A moves its field of view so that the 3D scanner 2 is within the field of view, and captures images of scanner marks to generate a mark image including the scanner marks in order to measure the position and orientation of the 3D scanner 2. The movable camera unit 3A includes a movable stage 31 supported by the base 30 and a scanner camera 32 fixed to the upper part of the movable stage 31. The movable stage 31 includes a stage drive unit 31a. The stage drive unit 31a incorporates an actuator such as a motor and is configured to rotate the movable stage 31 about a left-right axis and a vertical axis. Furthermore, the scanner camera 32 rotates about a vertical axis by rotating the movable stage 31 about a vertical axis, and the scanner camera 32 rotates about a left-right axis by rotating the movable stage 31 about a left-right axis. As a result, the field of view of the scanner camera 32 can be moved (by moving the camera's field of view). Figure 1 and Figure 2 The dashed line A in the diagram schematically indicates the tracking of scanner markers, allowing the 3D scanner 2 (i.e., the multiple scanner markers provided in the 3D scanner 2) to enter the field of view of the scanner camera 32. Similarly, the detector markers of the detector 5 can also be tracked. The stage drive unit 31a is controlled by the main control unit 33 provided in the camera unit 3.

[0035] At the lower part of the movable stage 31, a plurality of light-emitting elements 31b are arranged at predetermined intervals on a two-dimensional plane, and the light-emitting elements 31b are switched between an on state and an off state by a lighting control unit 31c. Furthermore, the arrangement information of each of the light-emitting elements 31b is pre-stored in the camera unit 3. Note that components other than light-emitting elements used as markers can be used. The lighting control unit 31c is controlled by the main control unit 33. On the other hand, the base 30 is provided with a reference camera 34 for capturing images of the movable camera unit 3A. The reference camera 34 captures images of the light-emitting elements 31b activated by the lighting control unit 31c. The reference camera 34 captures images of the plurality of light-emitting elements 31b arranged in the movable camera unit 3A and generates an image including the light-emitting elements 31b.

[0036] The camera unit 3 is equipped with a camera image processing unit 35. The camera image processing unit 35 includes image processing circuitry and controls the scanner camera 32 to perform image capture at predetermined intervals. The scanner camera 32 captures images of the scanner markers on the 3D scanner 2 to generate a scanner marker image including the scanner markers. Furthermore, the scanner camera 32 captures images of the detector markers on the detector 5 to generate a detector marker image including the detector markers.

[0037] Examples of image processing circuits include graphics processing units (GPUs), field-programmable gate arrays (FPGAs), and digital signal processors (DSPs). The camera image processing unit 35 receives input from scanner marker images or detector marker images captured by the scanner camera 32, and from an image of the light source 31b captured by the reference camera 34.

[0038] The camera image processing unit 35 processes the scanner mark image captured by the scanner camera 32 to generate center position information of the scanner mark. Specifically, the camera image processing unit 35 performs a process to extract the center of the scanner mark from the scanner mark image. Then, it generates center position information of the scanner mark based on the extraction result. Furthermore, based on the center position information of the scanner mark obtained as a result of the center extraction process, the camera image processing unit 35 generates position and orientation information of the scanner mark relative to the movable camera unit 3A.

[0039] Furthermore, as in the case of scanner markers, the camera image processing unit 35 processes the image of the detector marker captured by the scanner camera 32 to generate center position information of the detector marker. Based on the center position information of the detector marker, position and orientation information of the detector marker relative to the movable camera unit 3A is generated.

[0040] The camera unit 3 includes a first wireless communication unit 36 ​​controlled by the main control unit 33. The first wireless communication unit 36 ​​is a communication module or the like configured to communicate with devices other than the camera unit 3. In this example, the camera unit 3 communicates with the 3D scanner 2 and the detector 5 via the first wireless communication unit 36, thereby enabling it to send and receive, for example, various types of data such as image data captured by the scanner camera 32 and various signals such as synchronization signals.

[0041] The first wireless communication unit 36, for example, sends synchronization signals to the 3D scanner 2 and the detector 5, and receives measurement information (edge ​​data) generated by the scanner image processing unit 147, which will be described later. Furthermore, the center position information of the scanner markers as measurement information generated by the camera image processing unit 35, and the center position information of the detector markers as measurement information generated by the camera image processing unit 35, are sent to the processing unit 4. The first wireless communication unit 36 ​​includes an optical communication interface 36a and a radio communication interface 36b. The optical communication interface 36a is configured to perform optical communication using visible or invisible light, and may be configured, for example, by an infrared communication interface. The radio communication interface 36b may be configured, for example, to construct a wireless LAN, or may be a part capable of short-range digital wireless communication using radio waves, such as Bluetooth communication.

[0042] The camera unit 3 also includes a communication unit 37 controlled by the main control unit 33. The communication unit 37 is a communication module or similar device configured to communicate with the processing unit 4. The camera unit 3 communicates with the processing unit 4 via the communication unit 37, thereby enabling it to send and receive various types of data (such as image data and various signals). The communication via the communication unit 37 can be wired or wireless.

[0043] The camera unit 3 includes a trigger generation unit 38 for generating identification information for identifying the timing of synchronous execution based on a measurement command. For example, when a measurement operator performs a predetermined measurement start operation, the main control unit 33 of the camera unit 3 receives the measurement start operation. Upon receiving the measurement start operation, the main control unit 33 causes the trigger generation unit 38 to generate a trigger as the aforementioned identification information. This trigger is sent to the 3D scanner 2, for example, via the optical communication interface 36a of the first wireless communication unit 36. When using the detector 5, the trigger is sent to the detector 5 via the optical communication interface 36a.

[0044] In response to the generation trigger, the main control unit 33 simultaneously executes the illumination of the scanner mark of the 3D scanner 2, the imaging of the scanner mark of the 3D scanner 2 by the movable camera unit 3A, the illumination of the light source 31b of the movable stage 31, and the imaging of the light source 31b by the reference camera 34. The main control unit 33 simultaneously executes the illumination of the scanner mark of the 3D scanner 2, the imaging by the movable camera unit 3A, and the imaging by the reference camera 34.

[0045] The radio communication interface 36b of the first wireless communication unit 36 ​​transmits, in relation to each other, the center position information of the scanner mark generated by the camera image processing unit 35 and the identification information corresponding to the center position information of the scanner mark generated by the trigger generation unit 38. For example, the center position information of the scanner mark is associated with identification information used to distinguish the center position information of the scanner mark from the center position information of other scanner marks. Therefore, the desired center position information of the scanner mark can be specified based on the identification information.

[0046] (Configuration of Processing Unit 4)

[0047] The processing unit 4 is a three-dimensional data generation unit that receives the position and orientation of multiple scanner marks obtained by processing the scanner mark image generated by the camera unit 3, receives the edge data of the bright line image obtained by processing the bright line image generated by the three-dimensional scanner 2, and measures the three-dimensional shape of the workpiece W based on the received position and orientation of the scanner marks and the edge data.

[0048] An example of a technique for measuring three-dimensional shapes will be described. Since the plurality of light emitters 31b of the imaging unit 3 are mounted on a movable stage 31 to which the scanner camera 32 is fixed, the positional relationship of the plurality of light emitters 31b relative to the scanner camera 32 is known. When the scanner camera 32 is moved by the stage drive unit 31a, the scanner camera 32 moves within the range where the reference camera 34 can capture images of the light emitters 31b. The position and orientation of the three-dimensional scanner 2 relative to the scanner camera 32 are determined based on the scanner mark image of the three-dimensional scanner 2 captured by the scanner camera 32.

[0049] Furthermore, the reference camera 34 similarly determines the position and orientation of the scanner camera 32 relative to the reference camera 34 based on images obtained by capturing multiple light-emitting objects 31b. Specifically, the camera image processing unit 35 processes the images of the light-emitting objects 31b generated by the reference camera 34 to generate position and orientation information of the scanner camera 32 relative to the reference camera 34.

[0050] The position and orientation of the 3D scanner 2 relative to the reference camera 34 are determined based on the position and orientation of the 3D scanner 2 relative to the scanner camera 32 and the position and orientation of the scanner camera 32 relative to the reference camera 34, and the coordinates of the measurement points are obtained, making 3D coordinate measurement (i.e. 3D shape measurement) possible.

[0051] Since the three-dimensional measurement program or application for realizing the function of the three-dimensional measuring device 1 is installed in the processing unit 4, the processing unit 4 can be used as the three-dimensional measuring device 1 and can execute the three-dimensional measurement method according to the present invention. The three-dimensional measurement method is a method for measuring the three-dimensional shape of the workpiece W and is executed by the processing unit 4, which is a computer. The three-dimensional measurement program for enabling the computer to execute the three-dimensional measurement method can be recorded in the storage medium 1000. The storage medium 1000 can be, for example, an optical disc such as a CD-ROM or DVD-ROM, or a semiconductor memory such as a memory card.

[0052] The processing unit 4 can be set separately from the camera unit 3, or it can be integrated with the camera unit 3. In addition, a part of the processing unit 4 can be incorporated into the camera unit 3, or a part of the camera unit 3 can be incorporated into the processing unit 4.

[0053] like Figure 2 As shown, the processing unit 4 includes a control unit 40, a monitor 41, and an operation input unit 42. The monitor 41 is composed of a liquid crystal display or an organic EL display configured to display various images and user interfaces, etc.

[0054] The operation input unit 42 is a component for users to perform various input operations. The operation input unit 42 may include, for example, a keyboard or a mouse.

[0055] The control unit 40 includes a control unit 43, a display control unit 44, a storage unit 45, and a second wireless communication unit 46. The display control unit 44 is a component used to control the monitor 41 based on signals output from the control unit 43, and to cause the monitor 41 to display various images and user interfaces, etc. The control unit 43 acquires user operations on the user interface based on signals output from the operation input unit 42.

[0056] Storage unit 45 can be a ROM, a solid-state drive, or a hard disk drive, etc. Storage unit 45 stores the arrangement information of each scanner marker in the marker block set in the 3D scanner 2. The arrangement information of the marker block and each scanner marker includes the distance between the marker blocks and information used to indicate the relative positional relationship of the self-illuminating markers set in each marker block, etc.

[0057] Furthermore, the second wireless communication unit 46 of the processing unit 4 is controlled by the control unit 43. The second wireless communication unit 46 is a communication module or the like configured to communicate with the first wireless communication unit 36 ​​of the camera unit 3. The second wireless communication unit 46 includes a radio communication interface 46a. The radio communication interface 46a of the second wireless communication unit 46 receives edge data, center position information of scanner markers, and center position information of detector markers as measurement information transmitted via the radio communication interface 36b of the first wireless communication unit 36 ​​of the camera unit 3.

[0058] (Configuration of 3D Scanner 2)

[0059] The 3D scanner 2 is configured to allow the measuring operator to measure the shape of the workpiece W while holding and freely moving the 3D scanner 2 with one or both hands, and the 3D scanner 2 is a handheld and portable scanner. In this embodiment, as... Figure 3 As shown, the front, back, left, right, top, and bottom of the 3D scanner 2 are defined. That is, when the operator holds the 3D scanner 2 by hand, the side located on the right is called "right," and the side located on the left is called "left." The "front" of the 3D scanner 2 is the side opposite to the workpiece W, and the "back" of the 3D scanner 2 is the side opposite to the side opposite to the workpiece W. The "top" of the 3D scanner 2 is the upper side when the gripping part 112 (described later) is held in a natural posture, as defined, and the "bottom" of the 3D scanner 2 is the lower side when the gripping part 112 is held in a natural posture, as defined. However, since the three-dimensional shape of the workpiece W can be measured while the 3D scanner 2 is held and moved by hand as described above, the 3D scanner 2 can have an upside-down orientation or a posture with its top side on the right or left, or its back side can be located on top or bottom.

[0060] The 3D scanner 2 includes a scanner body 20, a first marker block 21, a second marker block 22, a third marker block 23, and a fourth marker block 24. Each of the first marker blocks 21 to the fourth marker block 24 has self-illuminating scanner marks 21a, 22a, 23a, and 24a facing multiple directions. Each of the scanner marks 21a, 22a, 23a, and 24a includes, for example, a light-emitting diode (LED).

[0061] The scanner body 20 includes a scanner unit 60. The scanner unit 60 includes two first scanner light sources 62, a second scanner light source 63, a first scanner camera unit 64, a second scanner camera unit 65, and a texture camera 66. The two first scanner light sources 62 are multi-line light sources that each emits multiple linear beams of light in the measurement direction (forward), and are arranged such that the light emission surface is opposite to the workpiece W during measurement. The light emitted by the first scanner light sources 62 can be referred to as multi-line light, and the multi-line light is included in the pattern light.

[0062] The second scanner light source 63 is a single-line light source for emitting a linear beam of light in the measurement direction (forward), and is arranged such that the light emitting surface is opposite to the workpiece W during measurement. The light emitted by the second scanner light source 63 can be referred to as single-line light, and this single-line light is also included in the pattern light.

[0063] The first scanner light source 62 and the second scanner light source 63 each include a laser light source for emitting laser light, but the type of light source is not particularly limited. Furthermore, in this example, a total of three scanner light sources 62 and 63 are provided, but the invention is not limited to this, and one or more scanner light sources may be provided. Further, the type of pattern light is not particularly limited, and the scanner light source can emit pattern light other than multi-line light and single-line light.

[0064] The first scanner camera unit 64 and the second scanner camera unit 65 include, for example, light-receiving elements such as CMOS sensors and optical systems for forming an image of externally incident light on the light-receiving surface of the light-receiving elements. The first scanner camera unit 64 is attached to a portion spaced upward from the scanner light sources 62 and 63. The second scanner camera unit 65 is attached to a portion spaced downward from the scanner light sources 62 and 63. The first scanner camera unit 64 and the second scanner camera unit 65 are arranged such that their optical axes are respectively oriented towards the illumination direction of the pattern light beams from the scanner light sources 62 and 63, and thus can capture an image of the pattern light beams emitted from the scanner light sources 62 and 63 in the measurement direction and generate a bright line image including the pattern light.

[0065] The distance between the optical axes of the first scanner camera unit 64 and the second scanner camera unit 65 is known. Corresponding points between corresponding images generated by simultaneously capturing patterned light emitted from the first scanner light source 62 or the second scanner light source 63 by the first scanner camera unit 64 and the second scanner camera unit 65 are obtained, and the three-dimensional coordinates of the corresponding points can be obtained using a stereo measurement method. The stereo measurement method can be a passive stereo measurement using the first scanner camera unit 64 and the second scanner camera unit 65, or an active stereo measurement using one scanner camera unit.

[0066] The texture camera 66 includes, for example, a light-receiving element such as a CMOS sensor capable of acquiring color images, and an optical system for forming an image of externally incident light on the light-receiving surface of the light-receiving element. The texture camera 66 captures an image of the workpiece W to generate a texture image.

[0067] like Figure 4 As shown, the scanner body 20 includes a display unit (scanner display unit) 113 configured to display measurement results obtained by the scanner unit 60 and an operation unit 114 configured to operate the scanner unit 60. The display unit 113 is composed of a liquid crystal display or an organic EL display, etc. Furthermore, the display surface faces the measurement operator so that the 3D scanner 2 can be moved while viewing the display content of the display unit 113.

[0068] The display unit 113 also has a touch panel 113a on its display side, which is capable of touch operation. The operation unit 114 includes, for example, multiple operation buttons (including a measurement start button and a measurement stop button, etc.) and is arranged below the display unit 113. The touch panel 113a may also be part of the operation unit.

[0069] The 3D scanner 2 includes a display control unit 140, a marker illumination control unit 141, a scanner control unit 142, and a storage unit 143. The display control unit 140 controls the display unit 113 based on signals output from the scanner control unit 142, and causes the display unit 113 to display various images and user interfaces, etc. The scanner control unit 142 acquires user operations on the display unit 113 based on signals output from the touch panel 113a.

[0070] The marker illumination control unit 141 controls the scanner markers. The marker illumination control unit 141 switches the scanner markers 21a, 22a, 23a, and 24a between on and off states. The marker illumination control unit 141 is controlled by the scanner control unit 142. The storage unit 143 can temporarily store control programs and images captured by the scanner unit 60, etc.

[0071] The 3D scanner 2 includes a third wireless communication unit 144 controlled by a scanner control unit 142. The third wireless communication unit 144 is a communication module or similar device configured to communicate with devices other than the 3D scanner 2. The third wireless communication unit 144 is configured to transmit edge data generated by the scanner image processing unit 147 to a first wireless communication unit 36 ​​of the camera unit 3, and to receive synchronization signals transmitted from the first wireless communication unit 36. The third wireless communication unit 144 includes an optical communication interface 144a and a radio communication interface 144b.

[0072] The 3D scanner 2 includes a motion sensor 145. The motion sensor 145 includes sensors for detecting the acceleration and angular velocity of the 3D scanner 2, and the detected values ​​are output to the scanner control unit 142 for various types of computational processing. For example, the values ​​output from the motion sensor 145 can be used to obtain an initial solution for the pose of the 3D scanner 2 (i.e., the poses of the first marker block 21 to the fourth marker block 24), thereby improving matching accuracy and processing speed during pose calculation. Processing using the values ​​output from the motion sensor 145 can be performed by the camera unit 3 or the processing unit 4.

[0073] The 3D scanner 2 includes a scanner light source control unit 146 and a scanner image processing unit 147. The scanner light source control unit 146 controls a first scanner light source 62 and a second scanner light source 63. The first scanner light source 62 and the second scanner light source 63 are switched between an on and off state by the scanner light source control unit 146. The scanner light source control unit 146 is controlled by the scanner control unit 142. Furthermore, the scanner image processing unit 147 controls a first scanner camera unit 64, a second scanner camera unit 65, and a texture camera 66 to perform image capture at predetermined times. Images captured by the first scanner camera unit 64, the second scanner camera unit 65, and the texture camera 66 are input to the scanner image processing unit 147. The scanner image processing unit 147 performs various types of image processing (such as extracting edge data from the input image).

[0074] That is, the scanner image processing unit 147 generates edge data by performing edge extraction processing on the bright line image generated by the first scanner camera unit 64 or the second scanner camera unit 65. When the first scanner light source 62 emits multi-line light, the first scanner camera unit 64 and the second scanner camera unit 65 generate multi-line images. The scanner image processing unit 147 processes the multi-line images to generate edge data.

[0075] The third wireless communication unit 144 transmits edge data generated by the scanner image processing unit 147 and identification information corresponding to the edge data generated by the trigger generation unit 38 in a mutually related manner. That is, the edge data and the identification information used to distinguish the edge data from other edge data are linked to each other. Therefore, the desired edge data can be specified based on the identification information.

[0076] Furthermore, when a trigger generated by the trigger generation unit 38 of the imaging unit 3 is sent to the 3D scanner 2, the 3D scanner 2 receives the trigger as a synchronization signal via the optical communication interface 144a of the third wireless communication unit 144. Upon receiving the trigger, the scanner light source control unit 146 emits pattern light from the first scanner light source 62 or the second scanner light source 63, the scanner image processing unit 147 performs imaging by the first scanner imaging unit 64 and the second scanner imaging unit 65, and the marker illumination control unit 141 causes the scanner markers 21a, 22a, 23a, and 24a to emit light. The emission of pattern light from the first scanner light source 62 or the second scanner light source 63, the imaging by the first scanner imaging unit 64 and the second scanner imaging unit 65, and the illumination of the scanner markers 21a, 22a, 23a, and 24a are synchronized with each other.

[0077] The 3D scanner 2 transmits the edge data generated by the scanner image processing unit 147 to the camera unit 3 via the radio communication interface 144b of the third wireless communication unit 144. The camera unit 3 receives the edge data via the radio communication interface 36b of the first wireless communication unit 36, and transmits the received edge data and the center position information of the scanner mark generated by the camera image processing unit 35 to the second wireless communication unit 46 of the processing unit 4.

[0078] The 3D scanner 2 includes an indicator light 148 and a communication control unit 149. The indicator light 148 displays the operating status of the 3D scanner 2 and is controlled by the scanner control unit 142. The communication control unit 149 is a part that performs processing for communication, such as image data.

[0079] (Contact detector)

[0080] The contact detector 5 is a handheld or portable detector similar to the 3D scanner 2. For example... Figure 1 As shown, the detector 5 includes a detector body 120 and a stylus 121 protruding from the detector body 120. A contact point 121a, configured to contact the workpiece W, is provided at the front end of the stylus 121. The contact point 121a has, for example, a spherical shape. The contact point 121a is the portion configured to specify the position of a measurement point on the workpiece W and various designated points, etc. Furthermore, the detector body 120 has a grip portion 5A in its middle portion in the longitudinal direction, and the measuring operator can hold the grip portion 5A with one hand and move the detector 5 or change its orientation during measurement.

[0081] The detector body 120 is provided with a plurality of detector marks 5B spaced apart from each other. For example, the plurality of detector marks 5B are spaced apart from each other at one end of the detector body 120 in the longitudinal direction, and the plurality of detector marks 5B are also spaced apart from each other at the other end of the detector body 120 in the longitudinal direction.

[0082] Figure 5 The circuit configuration of detector 5 is shown. Although in Figure 5 Only one detector mark 5B is shown, but multiple detector marks 5B are actually provided. The detector camera 122 is located near the stylus 121. The detector 5 includes a display unit 123a composed of a liquid crystal display or an organic EL display, a touch panel 123b operated by touch, and a display control unit 123c. Furthermore, an operation unit 124 including multiple buttons is located near the display unit 123a. The detector 5 also includes a detector control unit 125, a storage unit 126, a detector mark illumination control unit 127, a fourth wireless communication unit 128, and a motion sensor 129. In addition, the detector 5 includes a battery 5C used as a power source.

[0083] The display control unit 123c controls the display unit 123a based on signals output from the detector control unit 125, and causes the display unit 123a to display various images and user interfaces, etc. The detector control unit 125 obtains user operations performed on the display unit 123a based on signals output from the touch panel 123b.

[0084] The detector marker illumination control unit 127 controls the detector marker 5B. The detector marker illumination control unit 127 switches the detector marker 5B between an on and off state. The detector marker illumination control unit 127 is controlled by the detector control unit 125. Control programs and the like can be stored in the storage unit 126.

[0085] Similar to the first wireless communication unit 36 ​​of the camera unit 3, the fourth wireless communication unit 128 includes an optical communication interface 128a and a radio communication interface 128b. The optical communication interface 128a is the part that receives triggers sent via the optical communication interface 36a of the camera unit 3. When a trigger is received, the detector mark illumination control unit 127 activates the detector mark 5B. As a result, the camera unit 3's imaging of the detector mark and the illumination of the detector mark 5B can be synchronized. The radio communication interface 128b may have a different radio communication system than the radio communication system of the radio communication interface 144b of the 3D scanner 2, and for example, when the radio communication interface 144b of the 3D scanner 2 establishes a wireless LAN, the radio communication interface 128b of the fourth wireless communication unit 128 may be configured to perform Bluetooth communication, etc., with a communication speed lower than that of a wireless LAN.

[0086] The motion sensor 129 includes sensors for detecting the acceleration and angular velocity of the detector 5, and the detected values ​​are output to the detector control unit 125 and used for various types of computational processing, such as the posture calculation of the detector 5, similar to the posture calculation of the 3D scanner 2.

[0087] (Processing content by the processing unit)

[0088] Processing unit 4 includes a 3D coordinate generation unit 43a, a 3D data generation unit 43b, a setting unit 43c, and a measurement unit 43d. The 3D coordinate generation unit 43a sequentially generates the 3D coordinates of a reference workpiece and the 3D coordinates of a measured workpiece based on images including patterned light generated by the first scanner camera unit 64 and the second scanner camera unit 65. Furthermore, the 3D data generation unit 43b generates reference 3D shape data of the reference workpiece and the 3D shape data of the measured workpiece based on the 3D coordinates sequentially generated by the 3D coordinate generation unit 43a. For example, if invalid points are included in the 3D coordinates generated by the 3D coordinate generation unit 43a, the 3D data generation unit 43b can convert the remaining final points after the invalid points are removed by the data editing unit 43h, which will be described later, into point cloud data or mesh data.

[0089] Specifically, when the first scanner camera unit 64 and the second scanner camera unit 65 of the 3D scanner 2 have captured images of the reference workpiece, if the processing unit 4 receives the center position information of the scanner mark generated by the camera image processing unit 35 via the second wireless communication unit 46, the 3D coordinate generation unit 43a sequentially generates the 3D coordinates of the reference workpiece. Furthermore, the 3D data generation unit 43b acquires the edge data generated by the scanner image processing unit 147, the center position information of the scanner mark generated by the camera image processing unit 35, and the position and pose information of the scanner camera 32. The 3D data generation unit 43b generates point cloud data or mesh data indicating the 3D shape of the reference workpiece based on the edge data, the center position information of the scanner mark, and the position and pose information of the scanner camera 32. Point cloud data or mesh data can be generated similarly for the measured workpiece.

[0090] Here, edge data is calculated for each multiline image in the multiline images generated by the first scanner camera unit 64 and the second scanner camera unit 65. Edge data is calculated by specifying changes in brightness values ​​for each Y-coordinate of the multiline image and performing arithmetic processing (such as differentiation) on these changes in brightness values. In other words, edge data is data indicating the position (X-coordinate) of the bright lines in each Y-coordinate.

[0091] Furthermore, the center position information of the self-illuminating marks 21a, 22a, 23a, and 24a is generated using the following method. First, the camera image processing unit 35 obtains the arrangement information of the self-illuminating marks 21a, 22a, 23a, and 24a from the storage unit 143 of the 3D scanner 2. Then, based on the arrangement information of the self-illuminating marks 21a, 22a, 23a, and 24a obtained from the storage unit 143 of the 3D scanner 2, and the relative three-dimensional position information between the marks included in the mark image generated by the camera image processing unit 35, the camera image processing unit 35 calculates any position of the image of each mark 21a, 22a, 23a, and 24a captured by the camera unit 3 when the relative position or posture of the 3D scanner 2 relative to the camera unit 3 changes, and matches the calculated positions of each mark 21a, 22a, 23a, and 24a with the mark positions on the image. Then, the relative position and pose of the 3D scanner 2 relative to the camera unit 3 are calculated to minimize the error between the calculated positions of each of the markers 21a, 22a, 23a, and 24a and the marker positions on the image, and this relative position and pose are generated as the center position information of each of the self-illuminating markers 21a, 22a, 23a, and 24a. That is, the camera image processing unit 35 virtually changes the arrangement information of each of the self-illuminating markers 21a, 22a, 23a, and 24a obtained from the storage unit 143 of the 3D scanner 2 by virtually changing the position and pose of the 3D scanner 2, calculates the position and pose that match the marker image generated by the camera image processing unit 35, and generates the center position information of each of the self-illuminating markers 21a, 22a, 23a, and 24a. This position and pose information calculation process can be referred to as bundle adjustment.

[0092] The center position information of the self-illuminating marks 21a, 22a, 23a, and 24a calculated here uses the scanner camera 32 as a reference. In this respect, the camera image processing unit 35 calculates the position and pose information of the 3D scanner 2 using the scanner camera 34 as a reference, based on the position and pose information of the scanner camera 32 using the reference camera 34 and the position and pose information of the 3D scanner 2 using the scanner camera 32 as a reference, thereby generating the center position information of the self-illuminating marks 21a, 22a, 23a, and 24a using the reference camera 34 as a reference.

[0093] When performing imaging, the 3D coordinate generation unit 43a receives edge data generated by the scanner image processing unit 147, recognition information corresponding to the edge data, center position information of each scanner mark generated by the camera image processing unit 35, and recognition information corresponding to the center position information of each scanner mark. The 3D coordinate generation unit 43a can generate a point cloud indicating the 3D shape of the workpiece W based on the edge data, the recognition information corresponding to the edge data, the center position information of each self-illuminating mark, the recognition information corresponding to the center position information of each self-illuminating mark, and the calibration data of the 3D scanner 2 stored in the storage unit 45 of the processing unit 4.

[0094] In this example, such as Figure 2 As shown, the camera unit 3 includes a memory 39a and an association unit 39b. The memory 39a sequentially accumulates edge data generated by the scanner image processing unit 147, and the association unit 39b associates the edge data with the center position information of the scanner mark based on the recognition information. For example, when sequentially measuring multiple workpieces W or sequentially measuring different parts of the same workpiece W, the scanner image processing unit 147 generates multiple edge data. While associating the generated multiple edge data with mutually different recognition information, the multiple edge data are transmitted from the third wireless communication unit 144 of the 3D scanner 2 to the camera unit 3. While associating the multiple edge data transmitted from the third wireless communication unit 144 of the 3D scanner 2 with recognition information, the multiple edge data are accumulated in the memory 39a of the camera unit 3.

[0095] When the 3D coordinate generation unit 43a generates a point cloud indicating a 3D shape, the association unit 39b specifies the center position information of the scanner mark to be sent to the 3D coordinate generation unit 43a. The association unit 39b specifies edge data with recognition information associated with the specified center position information of the scanner mark from multiple edge data accumulated in the memory 39a. Thereafter, the association unit 39b associates the specified edge data with the center position information of the scanner mark. The communication unit 37 of the imaging unit 3 sends the edge data specified by the association unit 39b and the center position information of the scanner mark to the 3D coordinate generation unit 43a in association with each other.

[0096] During measurement via detector 5, the operator brings contact point 121a into contact with the point to be measured (measurement point) on workpiece W, and then operates operation unit 124. Trigger generation unit 38 then generates a trigger to synchronously execute the emission of detector mark 5B, the imaging of detector mark 5B, the emission of light from light source 31b, and the imaging of light source 31b by reference camera 34. Camera image processing unit 35 generates center position information of detector mark 5B. Processing unit 4 receives center position information of detector mark 5B generated by camera image processing unit 35 via second wireless communication unit 46. Three-dimensional coordinate generation unit 43a generates three-dimensional coordinates of the measurement point indicated by contact point 121a based on the center position information of detector mark 5B generated by camera image processing unit 35 and the position and orientation information of scanner camera 32.

[0097] (Settings file creation)

[0098] When creating the setup file, the coordinate system is first established using the contact detector 5. In this embodiment, for example, the reference coordinate system can be defined using any part of the reference workpiece as the reference. In this case, similar to the design drawing of the reference workpiece, a reference coordinate system is set using any part of the reference workpiece as the reference, and the coordinates of the measurement position are calculated based on the reference coordinate system. When setting the reference coordinate system, regardless of the position and orientation of the reference workpiece, the reference coordinate system can be set using any part of the reference workpiece as the reference.

[0099] When the user initiates the setup process, the display control unit 44 generates... Figure 6 The user interface screen 100 is shown and displayed on the monitor 41. The user interface screen 100 includes a coordinate system setting button 101, an image display area 102, a measurement item setting area 103, and a measurement element setting area 104. The image display area 102 displays the set reference coordinate system and various images. The measurement item setting area 103 includes, for example, user-selectable measurement items such as "distance" for measuring the distance between two geometric elements and "angle" for measuring the angle between two geometric elements. The measurement element setting area 104 includes user-selectable measurement elements such as "plane," "line," "point," and "circle."

[0100] When the coordinate system setting button 101 on the user interface screen 100 is detected to have been operated, the control unit 43 operates in coordinate system setting mode. In coordinate system setting mode, the coordinate system generation unit 43i included in the control unit 43 generates a reference workpiece coordinate system based on the positions of multiple measurement points indicated by the contact detector 5 on the reference workpiece.

[0101] In this embodiment, to establish a reference coordinate system, the coordinate system generation unit 43i sequentially sets a plane, a line, and a point. In the coordinate system setting mode, the user designates a surface (e.g., the upper surface) of the reference workpiece as a "plane". When a "plane" is designated, the contact detector 5 sequentially designates four points separated from each other on a surface of the reference workpiece. When three points are designated, the plane passing through these three points is set as the "plane", and the position of the plane is calculated. This plane is, for example, defined as the XY plane.

[0102] Furthermore, to specify a "straight line" after specifying a plane, it is sufficient to sequentially specify two points separated from each other on one surface of the reference workpiece using the contact detector 5. Similarly, to specify a "point," it is sufficient to sequentially specify points on one surface of the reference workpiece using the contact detector 5. There are no particular limitations on the method used to set up the reference coordinate system, and for example, the reference coordinate system can be set up by setting three planes, including the point where the origin is to be set, as measurement planes.

[0103] Once the reference coordinate system is set, the reference workpiece is scanned by the non-contact 3D scanner 2. Then, the 3D coordinate generation unit 43a sequentially generates the 3D coordinates of the reference workpiece based on the image including patterned light generated by the scanner's camera units 64 and 65. While generating the 3D coordinates of the reference workpiece, the 3D data generation unit 43b generates reference 3D shape data of the reference workpiece based on the 3D coordinates sequentially generated by the 3D coordinate generation unit 43a. While the 3D data generation unit 43b generates the reference 3D shape data, the display control unit 44 generates display data for displaying the reference 3D shape data on the monitor (display unit) 41. On the monitor 41, for example... Figure 7 As shown, the reference three-dimensional shape data is displayed in the image display area 102 of the user interface screen 100.

[0104] After acquiring mesh data in this way as the 3D shape of the reference workpiece, the desired mesh data is obtained by partially deleting (removing) unnecessary data from the acquired mesh data. That is, for example, in... Figure 7The user interface screen 100 shown is equipped with a data editing button 105, and when the user operates the data editing button 105, data editing can be performed by the data editing unit 43h included in the control unit 43. For example, the data editing unit 43h deletes data of the background portion other than the reference workpiece, or deletes data of unnecessary portions of the reference workpiece. The data of unnecessary portions may include invalid points. To delete data, it is sufficient for the user to specify the area to be deleted, and only the data of the area specified by the user is deleted, and the necessary mesh data is generated by the three-dimensional data generation unit 43b. After the mesh data is generated by the three-dimensional data generation unit 43b, the deletion of unnecessary points can be performed.

[0105] The setting unit 43c is the part that sets the measurement elements for the reference three-dimensional shape data generated by the three-dimensional data generation unit 43b and displayed on the monitor 41. The setting of the measurement elements can be achieved through... Figure 7 The measurement item setting area 103 and measurement element setting area 104 of the user interface screen 100 shown are used for this purpose. For example, as Figure 8 As shown, when it is desired to set a "plane" as a measurement element for the reference three-dimensional shape data displayed on the monitor 41, the "plane" in the measurement element setting area 104 is selected. Then, the setting unit 43c sets the "plane" as the measurement element.

[0106] The measurement unit 43d is the part that performs measurements based on the measurement elements set by the setting unit 43c. The measurement unit 43d measures "distance" and "angle" selected in the measurement item setting area 103.

[0107] When the measurement elements are set by the setting unit 43c, a setting file is generated. In this setting file, the reference 3D shape data generated by the 3D data generation unit 43b and the measurement elements set by the setting unit 43c are associated with each other. This setting file is stored in the storage unit 45 as a measurement reproduction template. The storage unit 45 may store only one measurement reproduction template or multiple measurement reproduction templates.

[0108] When setting up measurement elements, the measurement elements can include not only planes extracted from the scan results, but also comparisons with CAD data used as a reference.

[0109] Processing unit 4 includes a receiving unit 43e, a reading unit 43f, and a registration unit 43g. The receiving unit 43e receives the selection of a measurement reproduction template from the measurement reproduction templates stored in the storage unit 45. For example, with the list of measurement reproduction templates stored in the storage unit 45 displayed on the monitor 41, the user can select a desired measurement reproduction template by operating the operation input unit 42. When the receiving unit 43e receives this selection operation from the user, it can specify the selected measurement reproduction template.

[0110] The reading unit 43f is used to read from the storage unit 45 the reference three-dimensional shape data and measurement elements included in a measurement reproduction template received by the receiving unit 43e. The registration unit 43g is used to register the reference three-dimensional shape data and measurement elements read by the reading unit 43f as reference three-dimensional shape data and reference measurement elements, respectively.

[0111] To reproduce the measurement of the measurement element during the measurement process described later, the viewing direction, click position, and operation history when creating the setup file are stored in the setup file. For example, receiving unit 43e receives the input of the extraction reference point when extracting a geometric element as a measurement element on monitor 41, as well as the selection of the type of geometric element to be extracted. The user inputs the extraction reference point and selects the type of geometric element. The type of geometric element is, for example, a plane or a line. Storage unit 45 stores the extraction reference point received by receiving unit 43e, the display posture or viewing direction of the measured workpiece on monitor 41 when the extraction reference point is received, and the type of geometric element as a measurement element.

[0112] (Measurement Processing)

[0113] After the setup file is stored as a measurement reproduction template in storage unit 45, the user performs measurement processing. Measurement processing can be performed according to... Figure 9 The process is as shown in the flowchart. In step SA1 after the start, alignment with the measurement settings is performed by the contact detector 5. For example, as described above, a plane, a line, and a point are sequentially set by the contact detector 5. In this way, the coordinate system generation unit 43i can generate a measurement workpiece coordinate system based on multiple measurement points indicated by the contact detector 5 on the measurement workpiece W. Figure 10 A user interface screen 150 for measurement is shown, which shows the state of the coordinate system of the workpiece being measured, and this user interface screen 150 for measurement is displayed on the monitor 41.

[0114] like Figure 2As shown, the control unit 43 includes an alignment unit 43j. After the coordinate system generation unit 43i generates the coordinate system of the measuring workpiece, the alignment unit 43j of the control unit 43 aligns the reference workpiece and the measuring workpiece based on the reference workpiece coordinate system and the measuring workpiece coordinate system generated by the coordinate system generation unit 43i. The alignment method using the contact detector 5 is not limited to the method of creating and referencing elements such as planes and lines to create a coordinate system, and other methods can be used.

[0115] Figure 11 The user interface screen 150 for measurement is displayed on monitor 41 when alignment between the reference workpiece and the measuring workpiece is completed. When alignment is performed using the contact detector 5, the grid data in the setup file and the position of the measuring workpiece are consistent.

[0116] During this alignment, the mesh data of the reference workpiece (i.e., the reference 3D shape data) is displayed as the scan result of the setup file. Figure 11 The user interface screen 150 used for measurement is shown here. Reference measurement elements can also be displayed on the user interface screen 150 used for measurement. When the user scans the workpiece using the 3D scanner 2, the grid data of the reference workpiece serves as a guide.

[0117] exist Figure 9 In step SA1, with the grid data of the setup file used as a guide displayed on monitor 41, the process proceeds to step SA2, where the scanning of the workpiece by the 3D scanner 2 begins. Here, although the CAD data used as a reference can be displayed on monitor 41 as a guide, there are situations where displaying only the CAD data makes it difficult to show which part of the workpiece will be measured by scanning. In such cases, as in this embodiment, by displaying the grid data of the setup file on monitor 41 as a guide instead of the CAD data, not only the shape of the workpiece but also the specific parts to be measured can be presented to the user, allowing the user to scan the measurement parts more appropriately.

[0118] Figure 12 The user interface screen 150 for measurement displayed on monitor 41 when the grid data of the setup file is displayed as a wizard on monitor 41 is shown. Figure 12 The lines indicated by reference numeral L in the figure represent multi-line light emitted from the first scanner light source 62 of the 3D scanner 2.

[0119] like Figure 12As shown, when a user scans and measures a workpiece using the 3D scanner 2, the 3D coordinate generation unit 43a sequentially generates the 3D coordinates of the workpiece based on images including patterned light generated by the scanner's camera units 64 and 65. Then, the display control unit 44 sequentially generates display data, in which the 3D coordinates of the workpiece sequentially generated by the 3D coordinate generation unit 43a are superimposed on a 3D shape based on reference 3D shape data registered by the registration unit 43g. Since the image based on the generated display data is incorporated into the user interface screen 150 for measurement and displayed on the monitor 41, the user can efficiently scan while simultaneously checking whether the necessary area has been scanned in real time by viewing the monitor 41. The user can also determine whether the measurement area has been scanned. When the measurement area has not yet been scanned, the user can move the 3D scanner 2 to allow the measurement area to be scanned.

[0120] The wireless communication unit 144 of the 3D scanner 2 sequentially receives display data in which the three-dimensional coordinates of the measured workpiece are superimposed on a three-dimensional shape based on reference three-dimensional shape data sequentially generated by the display control unit 44 of the processing unit 4. The scanner display unit 113 of the 3D scanner 2 displays the display data received by the wireless communication unit 144. In this way, due to... Figure 12 The user interface screen 150 for measurement shown is displayed on the scanner display unit 113 of the 3D scanner 2 held by the user, so the user can know whether the necessary range has been scanned in real time simply by looking at the hand.

[0121] In step SA3, the three-dimensional data generation unit 43b generates three-dimensional shape data of the workpiece based on the three-dimensional coordinates of the workpiece obtained in step SA2. Figure 13 The state of the scan completed by the 3D scanner 2 is shown, and the 3D shape data of the workpiece is displayed on the monitor 41.

[0122] In step SA4, the mesh data of the measuring workpiece that is a certain distance away from the mesh data (reference 3D shape data) in the setting file is deleted. Step SA4 is performed by the data editing unit 43h. With the alignment unit 43j aligned with the reference 3D shape data and the 3D shape data of the measuring workpiece, the data editing unit 43h partially removes 3D shape data from the 3D shape data of the measuring workpiece based on the distance between the reference 3D shape data and the 3D shape data of the measuring workpiece. For example, when the 3D shape data of the measuring workpiece is separated from the reference 3D shape data by a predetermined distance or more, only the 3D shape data of the measuring workpiece that is separated from the reference 3D shape data by a predetermined distance or more is removed. The "predetermined distance" can be set by the user.

[0123] In step SA5, the user determines whether the necessary parts of the workpiece to be measured have been scanned by the 3D scanner 2. If the necessary parts of the workpiece to be measured have not been scanned, the process proceeds to step SA2, and the user additionally scans the workpiece to be measured using the 3D scanner 2. If the necessary parts of the workpiece to be measured have been scanned, the process proceeds to step SA6, and the measurement unit 43d performs the measurement based on the reference measurement elements registered by the registration unit 43g and the 3D shape data of the workpiece to be measured. That is, the set measurement elements can be measured by obtaining mesh data that reproduces both position and shape from the setup file.

[0124] The measuring unit 32d obtains the display posture or viewing direction stored as measurement elements in the storage unit 45 from the setup file. Based on the obtained display posture or viewing direction, the measuring unit 32d updates the position and posture of the measured workpiece's three-dimensional data. In the updated position and posture state, it extracts geometric elements based on the extraction reference points and geometric element types included in the setup file, and performs various types of measurements based on the extracted geometric elements (see...). Figure 15 ).

[0125] The control unit 43 includes a synthesis unit 43k. The synthesis unit 43k is used to acquire three-dimensional coordinates with different parameters and synthesize the acquired multiple three-dimensional coordinates to generate three-dimensional shape data. An example of such parameters is, for instance, the resolution of the density of measurement points. When the synthesis unit 43k synthesizes the three-dimensional coordinates acquired with multiple parameters to generate three-dimensional shape data, the point clouds acquired with each parameter and the mesh data obtained by synthesizing the point clouds acquired with each parameter are associated with the three-dimensional shape data. When the reference three-dimensional shape data is displayed on the monitor 41, the display control unit 44 projects the point clouds acquired with each parameter onto the mesh data to display the data acquired with each parameter in a recognizable manner. Furthermore, not only resolution, but also parameters such as exposure time or laser type can be used, and multiple types of point clouds with different settings can be stored in the storage unit 45. Note that the laser type here refers to the setting of whether multi-line light or single-line light is used as the pattern light. Then, when the point clouds acquired with different settings are projected onto the mesh data, the areas acquired with each setting can be displayed on the mesh data in a recognizable manner.

[0126] (Other embodiments)

[0127] The above embodiments are merely illustrative in all respects and should not be construed as limiting. Furthermore, all modifications and alterations falling within the equivalent scope of the claims are within the scope of this invention.

[0128] Examples of display formats for the three-dimensional shape of a reference workpiece can include opaque, semi-transparent, and wireframe. Depending on the workpiece being measured, these formats can be switched between. In the opaque display format, the back of the workpiece is not visible, allowing for differentiation between the front and back sides.

[0129] In addition to displaying the three-dimensional shape of the reference workpiece, the user can also be shown, for example, whether the measurement by the 3D scanner 2 has obtained a sufficient number of point clouds through color or transparency. For example, by darkening the color or reducing the transparency on the user interface screen 150 used for measurement as the number of point clouds increases, the user can determine the end timing of the scan while viewing the color or transparency displayed on the monitor 41 or scanner display unit 113.

[0130] In addition, scanning settings include, for example, resolution and exposure time. Different 3D wizards can be displayed on the user interface screen 150 used for measurement, for each resolution or exposure time.

[0131] Furthermore, the color and display status of the mesh data used to indicate the 3D shape of the reference workpiece can be changed in the scanning settings to show the user which settings are preferred for scanning. For example, as Figure 16 Flowcharts and Figure 17 As shown in the conceptual diagram, not only information from the mesh data but also information from the point cloud during the scan can be used to represent the scan setup. That is, in step SB1, the mesh data and point cloud are stored and thus read separately, and both are aligned in the same way. In step SB2, the point cloud and mesh are associated with each other. In this example, the point cloud is projected onto the mesh based on information related to its position coordinates and normals. In step SB3, the settings of the projected point cloud are determined according to... Figure 17 The settings shown (A and B) are used to change the display color of the grid data.

[0132] Furthermore, the scanning progress of the 3D scanner 2 can be presented to the user by comparing it with the grid data in the settings file, or it can be done automatically. Figure 18In step SC1 of the flowchart shown, the user uses the 3D scanner 2 to scan and measure the workpiece to obtain a point cloud. In step SC2, the control unit 43 determines whether the distance between the acquired point cloud and the grid data in the setup file is equal to or less than a certain value. When the distance between the acquired point cloud and the grid data in the setup file is equal to or less than the certain value, the process proceeds to step SC3, and the point cloud is added as the point cloud measured in step SC1. When the distance between the acquired point cloud and the grid data in the setup file is not equal to or less than the certain value, the process proceeds to step SC4, and the point cloud measured in step SC1 is deleted. At this time, a value indicating the percentage of the grid data in the setup file scanned by the 3D scanner 2 is presented to the user. If the 3D scanner 2 can scan the grid data in the setup file at a certain percentage or more, the control unit 43 can automatically stop scanning. Furthermore, when deleting point clouds, point clouds can be deleted in real time during the scanning of the 3D scanner 2, rather than deleting the grid data of the measured workpiece that is far from the grid data in the setup file after scanning.

[0133] As described above, the present invention can be used to measure the three-dimensional shape of various workpieces.

Claims

1. A three-dimensional measuring device for measuring the three-dimensional shape of a workpiece, the three-dimensional measuring device comprising: A 3D scanner includes a scanner light source for emitting patterned light and a scanner camera unit for capturing the patterned light emitted by the scanner light source and generating an image including the patterned light; A three-dimensional coordinate generation unit sequentially generates the three-dimensional coordinates of a reference workpiece based on an image including the patterned light generated by the scanner's camera unit. A three-dimensional data generation unit generates reference three-dimensional shape data of the reference workpiece based on the three-dimensional coordinates sequentially generated by the three-dimensional coordinate generation unit. The display control unit generates display data for causing the display unit to display the reference three-dimensional shape data generated by the three-dimensional data generation unit; The setting unit sets measurement elements for reference three-dimensional shape data generated by the three-dimensional data generation unit and displayed on the display unit; The measurement unit performs measurements based on the measurement elements set by the setting unit; The storage unit stores setting files as measurement reproduction templates. Each setting file is obtained by linking the reference three-dimensional shape data generated by the three-dimensional data generation unit to the measurement elements set by the setting unit. A receiving unit that receives a selection of one of the measurement reproduction templates stored in the storage unit; The reading unit reads the reference three-dimensional shape data and the measurement elements included in a template received by the receiving unit; as well as The registration unit registers the reference three-dimensional shape data and the measurement elements read by the reading unit as reference three-dimensional shape data and reference measurement elements. The three-dimensional coordinate generation unit sequentially generates the three-dimensional coordinates of the workpiece based on the image including the patterned light generated by the scanner's camera unit. The display control unit sequentially generates display data. In this display data, the three-dimensional coordinates of the measured workpiece, generated sequentially by the three-dimensional coordinate generation unit, are superimposed on a three-dimensional shape based on the reference three-dimensional shape data registered by the registration unit. The three-dimensional data generation unit generates three-dimensional shape data of the measured workpiece based on its three-dimensional coordinates, and The measuring unit performs measurements based on the reference measuring elements registered by the registration unit and the three-dimensional shape data of the workpiece.

2. The three-dimensional measuring device according to claim 1 further includes a data editing unit, wherein the data editing unit partially removes three-dimensional shape data from the reference three-dimensional shape data generated by the three-dimensional data generation unit.

3. The three-dimensional measuring device according to claim 2, wherein, The data editing unit partially removes three-dimensional shape data from the three-dimensional shape data of the measured workpiece based on the distance between the reference three-dimensional shape data registered by the registration unit and the three-dimensional shape data of the measured workpiece.

4. The three-dimensional measuring device according to claim 1, further comprising: A contact-type detector that indicates the location of the measurement point; as well as The coordinate system generation unit generates a coordinate system based on the positions of multiple measurement points indicated by the contact detector. The coordinate system generation unit generates a reference workpiece coordinate system based on multiple measurement points indicated by the contact detector on the reference workpiece, and generates a measurement workpiece coordinate system based on multiple measurement points indicated by the contact detector on the measurement workpiece.

5. The three-dimensional measuring device according to claim 4, further comprising: The alignment unit aligns the reference workpiece and the measuring workpiece based on the reference workpiece coordinate system and the measuring workpiece coordinate system generated by the coordinate system generation unit. as well as The data editing unit, under the condition that the alignment unit is aligned with the reference three-dimensional shape data and the three-dimensional shape data of the workpiece being measured, partially removes three-dimensional shape data from the three-dimensional shape data of the workpiece being measured based on the distance between the reference three-dimensional shape data and the three-dimensional shape data of the workpiece being measured as registered by the registration unit.

6. The three-dimensional measuring device according to claim 1, wherein, The receiving unit receives the input of the extraction reference point and the selection of the type of geometric element to be extracted, and The storage unit stores the extraction reference point received by the receiving unit, the display posture of the measured workpiece on the display unit when the extraction reference point is received, and the type of the geometric element, as the measuring element.

7. The three-dimensional measuring device according to claim 6, wherein, The measurement unit updates the position and orientation of the three-dimensional shape data of the workpiece based on the display orientation stored as the measurement element. In the updated position and orientation, the geometric element is extracted based on the extraction reference point and the type of the geometric element, and the measurement is performed based on the extracted geometric element.

8. The three-dimensional measuring device according to claim 1 further includes a synthesis unit, wherein the synthesis unit acquires multiple three-dimensional coordinates with different parameters and synthesizes the acquired multiple three-dimensional coordinates to generate a three-dimensional shape data. in, The three-dimensional shape data is associated with point clouds obtained with the parameters and mesh data obtained by synthesizing point clouds obtained with the parameters.

9. The three-dimensional measuring device according to claim 8, wherein, When displaying the reference three-dimensional shape data, the display control unit projects the point clouds obtained with the parameters onto the grid data, thereby displaying the data obtained with the parameters in a recognizable manner.

10. The three-dimensional measuring device according to claim 1, wherein, The 3D scanner includes a communication unit and a scanner display unit. The communication unit sequentially receives the display data, in which the three-dimensional coordinates of the measured workpiece are superimposed on a three-dimensional shape based on the reference three-dimensional shape data, which is sequentially generated by the display control unit. The scanner display unit displays the display data received by the communication unit.

11. A three-dimensional measurement method for measuring the three-dimensional shape of a workpiece, the three-dimensional measurement method comprising: To generate an image including the patterned light emitted by the scanner light source, an image is captured. The three-dimensional coordinates of the reference workpiece are sequentially generated based on the generated image including the patterned light; The reference three-dimensional shape data of the reference workpiece is generated based on the sequentially generated three-dimensional coordinates; The generated reference three-dimensional shape data is displayed on the display unit, and measurement elements are set for the reference three-dimensional shape data displayed on the display unit; The storage configuration file serves as a measurement reproduction template, and each configuration file is obtained by associating the generated reference 3D shape data with the configured measurement elements; Receive the selection of one of the stored measurement reproduction templates; Read the reference 3D shape data and measurement elements included in a received template; Register the read baseline 3D shape data and the read measurement elements as reference 3D shape data and reference measurement elements; Display data is generated sequentially, in which the three-dimensional coordinates of the measured workpiece are superimposed on a three-dimensional shape based on the registered reference three-dimensional shape data; The three-dimensional shape data of the workpiece is generated based on its three-dimensional coordinates. as well as The workpiece is measured based on the registered reference measurement elements and the three-dimensional shape data of the workpiece.

12. A storage medium storing a three-dimensional measurement program, the three-dimensional measurement program being used to cause a computer to execute a three-dimensional measurement method for measuring the three-dimensional shape of a workpiece, the three-dimensional measurement program causing the computer to perform: To generate an image including the patterned light emitted by the scanner light source, an image is captured. The three-dimensional coordinates of the reference workpiece are sequentially generated based on the generated image including the patterned light; The reference three-dimensional shape data of the reference workpiece is generated based on the sequentially generated three-dimensional coordinates; The generated reference three-dimensional shape data is displayed on the display unit, and measurement elements are set for the reference three-dimensional shape data displayed on the display unit; The storage configuration file serves as a measurement reproduction template, and each configuration file is obtained by associating the generated reference 3D shape data with the configured measurement elements; Receive the selection of one of the stored measurement reproduction templates; Read the reference 3D shape data and measurement elements included in a received template; Register the read baseline 3D shape data and the read measurement elements as reference 3D shape data and reference measurement elements; Display data is generated sequentially, in which the three-dimensional coordinates of the measured workpiece are superimposed on a three-dimensional shape based on the registered reference three-dimensional shape data; The three-dimensional shape data of the workpiece is generated based on its three-dimensional coordinates. as well as The workpiece is measured based on the registered reference measurement elements and the three-dimensional shape data of the workpiece.

Images