A storage medium on which an information processing device, an information processing method, and an information processing program are stored.

Abstract

This system reduces the burden on users during analysis by automatically setting the reference coordinate system from three-dimensional data. [Solution] The information processing device includes a data acquisition unit 291 that acquires three-dimensional data of a workpiece, an extraction unit 292 that extracts geometric elements from the three-dimensional data of the workpiece acquired by the data acquisition unit 291, a type identification unit 294 that identifies the type of geometric element extracted by the extraction unit 292, a coordinate system setting unit 295 that identifies a rule from a pre-stored set of rules based on the type of geometric element identified by the type identification unit 294 and sets a reference coordinate system based on the identified rule and the geometric element, and an analysis unit 296 that performs analysis of the three-dimensional data of the workpiece based on the reference coordinate system set by the coordinate system setting unit 295.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a storage medium storing an information processing program for analyzing three-dimensional data of a workpiece.

Background Art

[0002] For example, Patent Document 1 discloses a three-dimensional scanner that scans a workpiece placed on a stage to generate three-dimensional data. This type of three-dimensional scanner is mainly configured to irradiate structured illumination light onto the workpiece on the stage, image and analyze the distortion of the illumination light by a camera, and thereby measure the three-dimensional shape of the workpiece.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, there are cases where three-dimensional data obtained by a three-dimensional scanner such as Patent Document 1 is analyzed. In order to analyze three-dimensional data, a reference coordinate system (reference coordinate system) is required. However, the three-dimensional data obtained by the three-dimensional scanner is displayed in a device coordinate system that is different from the reference coordinate system that the user wants to analyze. Therefore, the user needs to separately set the reference coordinate system.

[0005] That is, when determining the reference coordinate system, a reference plane, cylinder, cone, etc. are specified from within the workpiece, and the normal line of the plane, the central axis of the cylinder, the central axis of the cone, etc. are used as reference axes, and it is determined based on this reference axis. After the user determines the reference axis, the user cuts a cross-section along the reference axis or projects it onto the reference plane to two-dimensionalize it, measures and analyzes the width, angle, etc. of each part of the workpiece.

[0006] If there is an error in the reference axis, the analysis results will naturally be inaccurate, so setting an accurate reference axis as a pre-processing step for analysis is essential. However, setting the reference axis must be done for each 3D data set to be analyzed, which is particularly cumbersome when repeatedly analyzing workpieces with similar shapes. Furthermore, for beginners in analysis, the concept of setting a reference axis itself is difficult to understand, and it is not a simple task. In addition, setting the reference axis becomes an obstacle when trying to automate repetitive measurement tasks.

[0007] This disclosure is made in view of the above points, and its purpose is to reduce the burden on users during analysis by enabling the automatic setting of a reference coordinate system from three-dimensional data during analysis. [Means for solving the problem]

[0008] To achieve the above objective, one aspect of this disclosure may be based on an information processing device that analyzes three-dimensional data of a workpiece for which a predetermined coordinate system has been set. The information processing device includes: a storage unit that pre-stores a set of rules that associates a plurality of types of geometric elements used to create a coordinate system with information of a coordinate system created based on each type of geometric element; a data acquisition unit that acquires three-dimensional data of a workpiece; an extraction unit that extracts geometric elements from the three-dimensional data of the workpiece acquired by the data acquisition unit; a type identification unit that identifies the type of geometric element extracted by the extraction unit; a coordinate system setting unit that identifies a rule from the set of rules stored in the storage unit based on the type of geometric element identified by the type identification unit and sets a reference coordinate system based on the identified rule and the geometric element extracted by the extraction unit; and an analysis unit that analyzes the three-dimensional data of the workpiece based on the reference coordinate system set by the coordinate system setting unit.

[0009] In this configuration, when the extraction unit extracts geometric elements from the three-dimensional data acquired by the data acquisition unit, the type of the extracted geometric elements is identified by the type identification unit. Based on the identified type of geometric element, a rule is identified that associates the type of geometric element with coordinate system information. Based on that rule and the geometric element, the coordinate system setting unit automatically sets the reference coordinate system, so the user does not need to set the reference coordinate system during analysis. Furthermore, since the reference coordinate system set in this way utilizes the geometric elements contained in the three-dimensional data, it is highly accurate and suitable for analysis.

[0010] Another aspect of this disclosure may also be based on an information processing method for analyzing three-dimensional data of a workpiece for which a predetermined coordinate system has been set. In this information processing method, a set of rules that associates multiple types of geometric elements used to create a coordinate system with information of the coordinate system created based on each type of geometric element is stored in a memory unit in advance. Three-dimensional data of the workpiece is acquired, geometric elements are extracted from the acquired three-dimensional data of the workpiece, the type of the extracted geometric element is identified, a rule is identified from the set of rules stored in the memory unit based on the identified type of geometric element, a reference coordinate system is set based on the identified rule and the extracted geometric element, and the three-dimensional data of the workpiece is analyzed based on the set reference coordinate system.

[0011] In yet another aspect of this disclosure, a storage medium can be provided which stores an information processing program that causes a computer to perform information processing to analyze three-dimensional data of a workpiece for which a predetermined coordinate system has been set. For example, the computer may have a storage unit that has a set of rules pre-stored in which a set of rules is associated with a plurality of types of geometric elements used to create a coordinate system and information of a coordinate system created based on each type of geometric element. The information processing program can then be provided which causes the computer to perform the following processes: acquiring three-dimensional data of a workpiece; extracting geometric elements from the acquired three-dimensional data of the workpiece; identifying the type of the extracted geometric elements; identifying a rule from the set of rules stored in the storage unit based on the identified type of geometric element, setting a reference coordinate system based on the identified rule and the extracted geometric elements; and analyzing the three-dimensional data of the workpiece based on the set reference coordinate system.

[0012] Furthermore, the information processing device may include: a storage unit that stores a model that identifies a combination of analysis menus to be used for analyzing three-dimensional data from a plurality of pre-prepared analysis menus based on the shape of the three-dimensional data of the workpiece; a candidate identification unit that identifies candidate combinations of analysis menus to be used for analyzing three-dimensional data from a plurality of pre-prepared analysis menus based on the shape of the three-dimensional data acquired by the data acquisition unit and the model stored in the storage unit; a display control unit that displays the combination of analysis menus identified by the candidate identification unit on a display unit; and an analysis unit that performs shape analysis of the workpiece based on the analysis menus identified by the candidate identification unit. [Effects of the Invention]

[0013] As explained above, the system can automatically set the reference coordinate system for analysis from the three-dimensional data of the workpiece, thereby reducing the burden on the user during analysis. [Brief explanation of the drawing]

[0014] [Figure 1] Figure 1 shows the overall configuration of a three-dimensional scanner according to an embodiment of the present invention. [Figure 2] Figure 2 is a block diagram of the three-dimensional scanner. [Figure 3] Figure 3 is a side view of the measurement unit and the pedestal unit. [Figure 4] Figure 4 is a block diagram of the measurement unit. [Figure 5] Figure 5 is a diagram showing a configuration example of the module. [Figure 6] Figure 6 is a flowchart showing an example of the setting process of the reference coordinate system. [Figure 7] Figure 7 is a diagram for explaining the process of extracting a plane from the three-dimensional data of the workpiece. [Figure 8] Figure 8 is a diagram for explaining the estimation method of the first reference axis and the first provisional origin. [[ID=2,2]] [Figure 9] Figure 9 is a diagram for explaining the estimation method of the second reference axis. [Figure 10] Figure 10 is a diagram for explaining the method of estimating the second reference axis based on the second plane. [Figure 11] Figure 11 is a diagram for explaining the estimation method of the third reference axis. [Figure 12] Figure 12 is a diagram for explaining the estimation method of the second provisional origin. [Figure 13] Figure 13 is a diagram for explaining the estimation method of the third plane. [Figure 14] Figure 14 is a diagram for explaining the method of setting the origin. [Figure 15] Figure 15 is a diagram showing an example of extracting a solid shape as geometric elements. [Figure 16] Figure 16 is a diagram for explaining the method of setting the reference coordinate system based on the extracted cylinder. [Figure 17] Figure 17 is a diagram for explaining the method of extracting a symmetric plane and setting a reference axis. [Figure 18] Figure 18 is a flowchart in the case of setting the reference coordinate system based on a plurality of types of geometric elements. [Figure 19] Figure 19 is a diagram showing an example of a screen displayed when proposing measurement items. [[ID=SS]] [Figure 20]Figure 20 shows another example of the screen displayed when suggesting measurement items. [Figure 21] Figure 21 is the equivalent of Figure 20 when the cursor is in the geometric tolerance menu. [Figure 22] Figure 22 shows an example of the screen displayed when cross-sectional measurement and three-dimensional measurement are selected. [Figure 23] Figure 23 shows an example of a cross-sectional measurement results screen. [Figure 24] Figure 24 shows an example of the second results screen for cross-sectional measurement. [Figure 25] Figure 25 shows an example of a results screen for three-dimensional measurement. [Figure 26] Figure 26 shows an example of cross-sectional measurement performed on CAD data. [Modes for carrying out the invention]

[0015] Embodiments of the present invention will be described in detail below with reference to the drawings. The following description of preferred embodiments is essentially illustrative and is not intended to limit the present invention, its applications, or its uses.

[0016] Figure 1 shows the overall configuration of a three-dimensional scanner 1 as an information processing device according to an embodiment of the present invention. The three-dimensional scanner 1 is configured to acquire three-dimensional data by measuring the shape of a workpiece (object to be measured) W, convert it into mesh data of the workpiece W and output it, and is also configured to analyze the three-dimensional data of the workpiece W. Furthermore, the three-dimensional scanner 1 can also convert the mesh data of the workpiece W into CAD data and output it, or convert the mesh data into surface data and output it.

[0017] In the following description, when measuring the shape of a workpiece W, coordinate information of the workpiece W surface is obtained by irradiating the workpiece W with a predetermined pattern of measurement light and using the signal obtained from the reflected light reflected from the surface of the workpiece W. For example, as the predetermined pattern of measurement light, a measurement method using triangulation with a fringe projection image obtained from the reflected light, which is projected onto the workpiece W, can be used. However, in this invention, the principle and configuration for obtaining the coordinate information of the workpiece W are not limited to this, and other methods can also be applied.

[0018] The three-dimensional scanner 1 comprises a measuring unit 100 for measuring the shape of a workpiece W, a base unit 600 on which the workpiece W can be placed, a controller 200, a light source unit 300, and a display unit 400. The controller 200 may be incorporated into the measuring unit 100, the light source unit 300 may be incorporated into the measuring unit 100, or the display unit 400 may be incorporated into the measuring unit 100. Furthermore, the controller 200 and the light source unit 300 may be integrated, or the controller 200 and the display unit 400 may be integrated.

[0019] In this embodiment, an example is described in which the information processing device is configured with a three-dimensional scanner 1. However, the information processing device may also be configured with, for example, only a controller 200, or with a controller 200 and a display unit 400. In other words, the measuring unit 100, the base unit 600, and the light source unit 300 can each be provided as needed.

[0020] The three-dimensional scanner 1 uses a light source unit 300 to provide structured illumination to the workpiece W, captures a fringe projection image, generates a depth image containing coordinate information, and can measure the three-dimensional dimensions and shape of the workpiece W based on this image. Measurement using such fringe projection has the advantage of shortening measurement time because it allows for three-dimensional measurement without moving the workpiece W or optical systems such as lenses in the Z direction (height direction).

[0021] Figure 2 shows a block diagram of the three-dimensional scanner 1. As shown in this figure, the measurement unit 100 includes a pattern light projection unit (first projection unit) 110 that projects a pattern light for measurement onto the workpiece W, a light receiving unit 120, a measurement control unit 150, and an illumination light output unit 130. The projection unit 110 is the part that irradiates the workpiece W, which is placed on the mounting unit 140 (described later), with measurement light having a predetermined pattern. Placing the workpiece W on the mounting unit 140 is the same as arranging the workpiece W on the mounting unit 140.

[0022] The light-receiving unit 120 is fixed in an inclined position with respect to the mounting surface 142 of the rotating stage 143, which will be described later. The light-receiving unit 120 receives the measurement light that is irradiated by the light-emitting unit 110 and reflected by the workpiece W. When the light-receiving unit 120 receives the measurement light as reflected light from the workpiece W, it generates and outputs a first measurement reception signal that represents the amount of measurement light received. The light-receiving unit 120 can generate an observation image for observing the overall shape of the workpiece W by imaging the workpiece W placed on the mounting unit 140. In this example, there is an illumination light output unit 130, but uniform light may be irradiated onto the workpiece W from the light-emitting unit 110. In this case, the light-emitting unit 110 is a component that irradiates the workpiece W with measurement light and uniform light at different timings. The light-receiving unit 120 can also receive the uniform light irradiated from the light-emitting unit 110 and output a second reception signal for texture acquisition. For example, it is also possible to irradiate the measurement light source with uniform light of the same wavelength as the measurement light and output a received signal containing uniaxial color information. Although not shown in the diagram, it is also possible to prepare a first camera and a second camera that are calibrated, acquire the shape with the first camera and acquire texture information with the second camera. The texture information includes color information and brightness information of the workpiece W.

[0023] The light-receiving unit 120 according to this embodiment includes a high-magnification light-receiving unit and a low-magnification light-receiving unit. The high-magnification light-receiving unit is the part of the workpiece W that can be imaged at a magnified view compared to the low-magnification light-receiving unit. On the other hand, the low-magnification light-receiving unit is a light-receiving unit with a wider field of view compared to the high-magnification light-receiving unit.

[0024] The base unit 600 comprises a base plate 602, a mounting unit 140, and a movement control unit (stage control unit) 144. The mounting unit 140 is supported on the base plate 602 of the base unit 600. The movement control unit 144 controls the movement and rotation of the rotating stage 143 on which the workpiece W is placed. The movement control unit 144 may be provided on the base unit 600 side or on the controller 200 side.

[0025] The light source unit 300 is connected to the measurement unit 100. The light source unit 300 is the part that generates measurement light and supplies it to the measurement unit 100. The controller 200 is the part that controls the measurement unit 100 and the like. The display unit 400 is connected to the controller 200 and is configured to display the image generated by the measurement unit 100, and to allow necessary settings, inputs, selections, etc.

[0026] The mounting section 140 has a rotating stage 143 with a mounting surface 142 on its upper surface on which the workpiece W is placed. As shown in Figure 4, in the apparatus coordinate system, two mutually orthogonal directions within the mounting surface 142 of the rotating stage 143 are defined as the X direction and the Y direction, and are indicated by arrows X and Y, respectively. The direction perpendicular to the mounting surface 142 of the mounting section 140 is defined as the Z direction and is indicated by arrow Z. The direction of rotation around an axis parallel to the Z direction is defined as the θ direction and is indicated by arrow θ.

[0027] The mounting section 140 includes a rotating stage 143 that rotates the mounting surface 142 around an axis extending in the Z direction, and a translational stage 141 that moves the mounting surface 142 horizontally (in the X and Y directions). The translational stage 141 has an X-direction movement mechanism and a Y-direction movement mechanism. The rotating stage 143 has a θ-direction rotation mechanism. The mounting section 140 may also include a fixing member (clamp) for fixing the workpiece W to the mounting surface 142. Furthermore, the mounting section 140 may include a tilt stage having a mechanism that can rotate around an axis parallel to the mounting surface 142.

[0028] The movement control unit 144 controls the rotational movement of the rotating stage 143 and the parallel movement of the translational stage 141 according to the measurement conditions set by the measurement condition setting unit 261, which will be described later. The movement control unit 144 also controls the movement of the mounting unit 140 by the mounting movement unit based on the measurement area set by the measurement condition setting unit 261, which will be described later.

[0029] The controller 200 includes a CPU (Central Processing Unit) 210, ROM (Read-Only Memory) 220, working memory 230, storage device (storage unit) 240, and an operation unit 250, etc. For example, a PC (Personal Computer) can be used as the controller 200.

[0030] The configuration of the measurement unit 100 is shown in the block diagram of Figure 4. The measurement unit 100 comprises a light-emitting unit 110, a light-receiving unit 120, an illumination light output unit 130, a measurement control unit 150, and a main body case 101 housing these components. The light-emitting unit 110 includes a measurement light source 111, a pattern generation unit 112, and a plurality of lenses 113, 114, and 115. The light-receiving unit 120 includes a camera 121 and a plurality of lenses 122 and 123. When performing measurements at different magnifications by providing multiple light-receiving units, a light-receiving unit 120a including a low-magnification camera 121 and low-magnification lenses, and a light-receiving unit 120b including a high-magnification camera 121 and high-magnification lenses may be installed. Note that the configuration is not limited to this one; the magnification may be varied by switching between multiple lenses for a single camera 121, or by providing a zoom lens for a single camera 121.

[0031] The light-emitting unit 110 is positioned diagonally above the mounting unit 140. In the example shown in Figure 4, the measurement unit 100 includes two light-emitting units 110, but the measurement unit 100 may include multiple light-emitting units 110. Here, a first measurement light-emitting unit 110A (right side in Figure 4) capable of irradiating the workpiece W with a first measurement light ML1 from a first direction, and a second measurement light-emitting unit 110B (left side in Figure 4) capable of irradiating the workpiece W with a second measurement light ML2 from a second direction different from the first direction are provided. The first measurement light-emitting unit 110A and the second measurement light-emitting unit 110B are arranged symmetrically with the optical axis of the light-receiving unit 120 as the center of symmetry. Although not shown, it is also possible to have three or more light-emitting units 110, or to move the light-emitting unit 110 and the mounting unit 140 relative to each other to project light onto the workpiece W in different directions, even while using a common light-emitting unit 110. In the above example, multiple light-emitting units 110 are provided and the light is received by a common light-receiving unit 120. However, conversely, multiple light-receiving units 120 may be provided to receive light from a common light-emitting unit 110. Furthermore, in this example, the irradiation angle of the illumination light emitted by the light-emitting unit 110 with respect to the Z direction is fixed, but this can also be made variable.

[0032] Each first measurement light projection unit 110A and second measurement light projection unit 110B is equipped with a first measurement light source and a second measurement light source, respectively, as measurement light sources 111. These measurement light sources 111 are, for example, halogen lamps that emit white light. The measurement light source 111 may also be a light source that emits monochromatic light, such as a blue LED (light-emitting diode) or organic EL that emits blue light. The light emitted from the measurement light source 111 (hereinafter referred to as "measurement light") is appropriately focused by the lens 113 and then incident on the pattern generation unit 112.

[0033] The relative positions of the light-receiving unit 120, light-emitting units 110A and 110B, and the light-receiving unit 120 are determined such that the central axes of the light-emitting units 110A and 110B intersect at a position where the arrangement of the workpiece W on the mounting unit 140 and the depth of field of the light-emitting and light-receiving units 110 and 120 are appropriate. Furthermore, since the center of the rotation axis in the θ direction coincides with the central axis of the light-receiving unit 120, when the mounting unit 140 rotates in the θ direction, the workpiece W rotates within the field of view around the rotation axis without moving out of the field of view.

[0034] The pattern generation unit 112 reflects the light emitted from the measurement light source 111 so that it projects measurement light onto the workpiece W. The measurement light incident on the pattern generation unit 112 is converted into a preset pattern and a preset intensity (brightness) and emitted. The measurement light emitted by the pattern generation unit 112 is converted by a plurality of lenses 114 and 115 into light with a diameter larger than the observation and measurement field of view of the light receiving unit 120, and then irradiated onto the workpiece W on the mounting unit 140.

[0035] The pattern generation unit 112 is a component that can switch between a projection state in which measurement light is projected onto the workpiece W and a non-projection state in which measurement light is not projected onto the workpiece W. For example, a DMD (Digital Micromirror Device) can be used for such a pattern generation unit 112. A pattern generation unit 112 using a DMD can be controlled by the measurement control unit 150 to switch between a reflection state in which the measurement light is reflected onto the optical path as the projection state and a light-shielding state in which the measurement light is blocked as the non-projection state.

[0036] A DMD is an element in which numerous micromirrors (tiny mirror surfaces) are arranged on a plane. Each micromirror can be individually switched ON or OFF by the measurement control unit 150, so a desired projection pattern can be constructed by combining the ON and OFF states of numerous micromirrors. This makes it possible to generate the pattern necessary for triangulation and measure the shape of the workpiece W. In this way, the DMD functions as a projection pattern optical system that projects a periodic projection pattern for measurement onto the workpiece W during measurement. Furthermore, the DMD has excellent response speed and offers the advantage of being able to operate at high speeds compared to shutters and the like.

[0037] In the above example, an example using a DMD for the pattern generation unit 112 was described, but the present invention is not limited to a DMD for the pattern generation unit 112, and other materials can be used. For example, an LCOS (Liquid Crystal on Silicon: reflective liquid crystal element) may be used as the pattern generation unit 112. Alternatively, a transmissive material may be used instead of a reflective material to adjust the amount of light transmitted for measurement. In this case, the pattern generation unit 112 is placed on the optical path of the measurement light, and a light projection state that transmits the measurement light and a light shielding state that blocks the measurement light are switched. For example, an LCD (liquid crystal display) can be used for such a pattern generation unit 112. Alternatively, the pattern generation unit 112 may be configured using a projection method using multiple line LEDs, a projection method using multiple optical paths, an optical scanner method composed of a laser and a galvanometer mirror, an AFI (Accordion fringe interferometry) method that uses interference fringes generated by superimposing beams divided by a beam splitter, or a projection method using a physical grid composed of a piezo stage and a high-resolution encoder and a moving mechanism.

[0038] The light-receiving unit 120 is positioned above the mounting unit 140. The measurement light reflected upward from the workpiece W towards the mounting unit 140 is collected and imaged by the multiple lenses 122 and 123 of the light-receiving unit 120, and then received by the camera 121.

[0039] Camera 121 is a CCD (charge-coupled device) camera, for example, including an image sensor 121a. Image sensor 121a is, for example, a monochrome CCD (charge-coupled device). Image sensor 121a may be other image sensors such as a CMOS (complementary metal-oxide-semiconductor) image sensor. Color image sensors require each pixel to correspond to the reception of red, green, and blue light, resulting in lower measurement resolution compared to monochrome image sensors, and sensitivity is reduced because each pixel requires a color filter. Therefore, in this embodiment, a monochrome CCD is used as the image sensor, and a color image is acquired by illuminating the workpiece W with illumination corresponding to RGB in a time-division manner using the illumination light output unit 130, which will be described later. With this configuration, a color image of the object to be measured can be acquired without reducing the measurement accuracy. Illumination light output unit 130 is an example of a second light projection unit that irradiates the workpiece W with illumination light. The illumination light can be uniform light.

[0040] A color image sensor may also be used as the image sensor 121a. In this case, although the measurement accuracy and sensitivity will be lower compared to a monochrome image sensor, it will no longer be necessary to irradiate the image sensor with illumination corresponding to RGB in a time-division manner from the illumination light output unit 130. A color image can be acquired simply by irradiating with white light, thus simplifying the illumination optical system. Each pixel of the image sensor 121a outputs an analog electrical signal corresponding to the amount of light received (hereinafter referred to as the "received light signal") to the measurement control unit 150.

[0041] The measurement control unit 150 is equipped with an A / D converter (analog-to-digital converter) and a FIFO (First In First Out) memory (not shown). The light received signal output from the camera 121 is sampled at a constant sampling period and converted into a digital signal by the A / D converter of the measurement control unit 150, based on control by the light source unit 300. The digital signals output from the A / D converter are sequentially stored in the FIFO memory. The digital signals stored in the FIFO memory are sequentially transferred to the controller 200 as pixel data.

[0042] The control unit 250 of the controller 200 may include, for example, a keyboard or a pointing device. Examples of pointing devices include a mouse or a joystick.

[0043] The ROM 220 of the controller 200 stores system programs and the like. The working memory 230 of the controller 200 consists of, for example, RAM (Random Access Memory) and is used for processing various types of data. The storage device 240 consists of a solid-state drive, a hard disk drive, and the like. The storage device 240 stores information processing programs that cause the controller (computer) 200 to perform the setting and analysis processes of the reference coordinate system, as described later.

[0044] Furthermore, the storage device 240 is used to store various data such as pixel data (image data), setting information, measurement conditions, and coordinate system setting rules provided by the measurement control unit 150. Measurement conditions include, for example, the settings of the light-emitting unit 110 (pattern frequency, pattern type) and the type of light-receiving unit 120 (low-magnification light-receiving unit, high-magnification light-receiving unit), which are set by the scanner module 260 described later when measuring the shape of the workpiece W. In addition, the storage device 240 can also store brightness information, coordinate information, and attribute information for each pixel that makes up the measurement image.

[0045] The CPU210 is a control circuit or control element that processes given signals and data, performs various calculations, and outputs the calculation results. In this specification, CPU refers to an element or circuit that performs calculations, and is used to mean not limited to processors such as CPUs, MPUs, GPUs, and TPUs for general-purpose PCs, regardless of their name, but also including processors such as FPGAs, ASICs, LSIs, microcontrollers, and chipsets such as SoCs.

[0046] The CPU 210 generates image data based on pixel data provided by the measurement control unit 150. The CPU 210 also performs various processing on the generated image data using the working memory 230. For example, based on the light-receiving signal output from the light-receiving unit 120, the CPU 210 generates measurement data representing the three-dimensional shape of the workpiece W contained within the field of view of the light-receiving unit 120 at a specific position on the mounting unit 140. The measurement data is the image itself acquired by the light-receiving unit 120. For example, when measuring the shape of the workpiece W using a phase-shift method, multiple images constitute one set of measurement data. The measurement data may also be point cloud data, which is a collection of points having three-dimensional position information. Measurement data of the workpiece W can be obtained using this point cloud data. Point cloud data is data represented by a collection of multiple points having three-dimensional coordinates.

[0047] The movement control unit 144 determines, based on measurement data of at least a portion of the workpiece W, whether to perform only the rotation of the rotating stage 143 or both the rotation of the rotating stage 143 and the translation of the translation stage 141. This facilitates three-dimensional measurement by automatically determining the imaging range according to the external shape of the workpiece W without the user having to be aware of it. The movement control unit 144 can also control the rotation of the rotating stage 143 after moving the translation stage 141 in the XY direction and then stopping the movement in the XY direction, thereby acquiring the shape around the workpiece W. Furthermore, scanning can also be performed by moving and rotating the workpiece W relative to the measuring unit 100 while the measuring unit 100 is fixed.

[0048] The display unit 400 is a component for displaying stripe projection images acquired by the measurement unit 100, depth images generated based on the stripe projection images, texture images captured by the measurement unit 100, various user interface screens, etc. The display unit 400 is composed of, for example, an LCD panel or an organic EL (electroluminescent) panel. Furthermore, by using a touch panel in the display unit 400, it can also be used in conjunction with the operation unit 250. The display unit 400 can also display images generated by the light receiving unit 120.

[0049] The light source unit 300 includes a control board 310 and an observation illumination light source 320. A CPU (not shown) is mounted on the control board 310. The CPU of the control board 310 controls the light-emitting unit 110, the light-receiving unit 120, and the measurement control unit 150 based on commands from the CPU 210 of the controller 200. Note that this configuration is just one example, and other configurations are possible. For example, the light-emitting unit 110 and the light-receiving unit 120 could be controlled by the measurement control unit 150, or the light-emitting unit 110 and the light-receiving unit 120 could be controlled by the controller 200, thus omitting the control board. Alternatively, a power supply circuit for driving the measurement unit 100 can be provided in this light source unit 300.

[0050] The observation illumination light source 320 includes, for example, three LEDs that emit red, green, and blue light. By controlling the brightness of the light emitted from each LED, the observation illumination light source 320 can generate light of any color. The illumination light IL generated from the observation illumination light source 320 is output from the illumination light output unit 130 of the measurement unit 100 through a light guide member (light guide). In addition to LEDs, other light sources such as semiconductor lasers (LDs), halogen lights, and HIDs can also be used as appropriate for the observation illumination light source. In particular, if a color imaging sensor is used as the image sensor, a white light source can be used for the observation illumination light source.

[0051] The illumination light IL output from the illumination light output unit 130 illuminates the workpiece W by switching between red, green, and blue light in a time-division manner. This allows the texture images captured by these RGB lights to be combined to obtain a color texture image, which can then be displayed on the display unit 400.

[0052] An information processing program, including a three-dimensional measurement program and applications for implementing the coordinate system setting and analysis functions of the three-dimensional scanner 1, is installed on the controller 200. This allows the information processing method according to the present invention to be executed using the three-dimensional scanner 1. The information processing method is a method for analyzing three-dimensional data of a workpiece W with a predetermined coordinate system set, and is specifically executed by a computer in the controller 200. An information processing program for causing the computer to execute the information processing method can be stored in a storage medium 1000. The storage medium 1000 may be, for example, an optical disc such as a CD-ROM or DVD-ROM, or a semiconductor memory such as a memory card.

[0053] In the controller 200 on which the information processing program is installed, the CPU 210, ROM 220, working memory 230, storage device 240, etc., constitute the scanner module 260, conversion module 270, integration module 280, and analysis module 290 shown in Figure 5. In this embodiment, the system is divided into four modules: scanner module 260, conversion module 270, integration module 280, and analysis module 290. However, any two or more of these modules 260, 270, 280, and 290 may be integrated to form a single module. Furthermore, parts of each module 260, 270, 280, and 290 may be incorporated into other modules. In other words, the configuration example shown in Figure 5 is just one example and is not limited to the configuration example shown in Figure 5.

[0054] The scanner module 260 acquires image data of the workpiece W by measuring its shape and creates mesh data of the workpiece W based on that image data. The conversion module 270 converts the mesh data created by the scanner module 260 into CAD data. CAD data is three-dimensional shape information composed of analytical surfaces and freeform surfaces, and includes surface data, solid data, and data used for design. Surface data is data of shape surfaces composed of freeform surfaces and analytical surfaces, such as side data and planar data of a cylinder.

[0055] The integration module 280 is responsible for transmitting signals and data from the scanner module 260 to the conversion module 270 and the analysis module 290, and transmitting signals and data from the conversion module 270 to the scanner module 260. In this example, a module is a unit capable of executing multiple arithmetic processes, and can also be called a functional unit, functional block, etc.

[0056] The scanner module 260 includes, for example, a measurement condition setting unit 261, a scanner control unit 262, a point cloud acquisition unit 263a, a mesh data generation unit 263b, a scanner output unit 264, etc. The measurement condition setting unit 261 is the part that sets the measurement conditions for the shape of the workpiece. The scanner control unit 262 is the part that controls the measurement unit 100 according to the measurement conditions set in the measurement condition setting unit 261 to generate image data and acquires measurement data for the workpiece W based on the generated image data.

[0057] The point cloud acquisition unit 263a is responsible for acquiring point cloud data of the workpiece W based on the image data of the workpiece W acquired by the scanner control unit 262. The mesh data generation unit 263b is responsible for acquiring the point cloud data acquired by the point cloud acquisition unit 263a, processing the acquired point cloud data, and converting it into mesh data.

[0058] The scanner output unit 264 is the part that outputs the mesh data created by the mesh data generation unit 263b and additional data to the conversion module 270. The additional data is, for example, data that includes at least one of the measurement conditions and data calculated from the measurement data of the workpiece W.

[0059] The scanner module 260 controls the measurement unit 100 and generates three-dimensional data along with the various conditions under which the shape of the workpiece W was measured (measurement model, measurement magnification, resolution, etc.) and the raw data (e.g., image data) at the time of measurement. The three-dimensional data is mesh data containing multiple polygons and can also be called polygon data. A polygon is data composed of information that identifies multiple points and information that shows the polygonal surface formed by connecting those points. For example, it can consist of information that identifies three points and information that shows the triangular surface formed by connecting those three points. Mesh data and polygon data can also be defined as data represented by a collection of multiple polygons.

[0060] The conversion module 270 converts mesh data into CAD data and determines the conversion process based on measurement conditions and raw data. Specifically, the conversion module 270 includes, for example, a data input unit 271, a processing parameter determination unit 272, a CAD conversion unit 273, and a CAD output unit 274. The data input unit 271 is the part that receives mesh data output from the scanner output unit 264 and additional data. The processing parameter determination unit 272 is the part that determines the processing parameters for converting mesh data into CAD data according to the additional data received by the data input unit 271. The CAD conversion unit 273 is the part that converts mesh data into CAD data according to the processing parameters determined by the processing parameter determination unit 272. The CAD output unit 274 is the part that outputs the CAD data converted by the CAD conversion unit 273.

[0061] The analysis module 290 of the three-dimensional scanner 1 is responsible for setting a reference coordinate system for user analysis as the coordinate system for the three-dimensional data of the workpiece W generated by the scanner module 260, and for performing analysis of the three-dimensional data of the workpiece W based on the set reference coordinate system. In other words, the coordinate system of the three-dimensional data of the workpiece W generated by the scanner module 260 is the device coordinate system, and this device coordinate system is different from the reference coordinate system that the user uses as a reference when analyzing the three-dimensional data, for example, a coordinate system based on the lens origin of the scanner head that constitutes the measuring unit 100, or a coordinate system based on the rotation center of the rotating stage 143. In the three-dimensional scanner 1 of this embodiment, the analysis module 290 has a setting function that automatically sets a reference coordinate system for the three-dimensional data of the workpiece W, even if the user does not set a reference coordinate system for the three-dimensional data of the workpiece W.

[0062] To enable automatic setting of the reference coordinate system, the analysis module 290 is equipped with a data acquisition unit 291, an extraction unit 292, a symmetry plane determination unit 293, a type identification unit 294, a coordinate system setting unit 295, an analysis unit 296, a candidate identification unit 297, a model creation unit 298, and a preliminary analysis unit 299. In addition, the analysis module 290 is equipped with an analysis unit 296 that performs analysis of the three-dimensional data of the workpiece W after the automatic setting of the reference coordinate system.

[0063] As will be explained in detail later, when creating a reference coordinate system for analysis, one or more geometric elements contained in the three-dimensional data of the workpiece W are used. The geometric elements used when creating the reference coordinate system include, for example, at least one of a surface and a solid shape. The solid shape includes, for example, a cylinder, a cone, etc. Thus, it is possible to use multiple different types of geometric elements to create a coordinate system, and the types of geometric elements include the aforementioned surfaces, solid shapes (cylinders, cones), etc. Furthermore, the method and rules for creating the coordinate system differ for each different type of geometric element. Corresponding to this, the storage device 240 of this embodiment stores a set of rules that associates the types of multiple geometric elements used to create the coordinate system with the information of the coordinate system created based on each type of geometric element. The coordinate system information includes, for example, the method and rules for creating the coordinate system. For example, if the geometric element is a surface, the method and rules for creating the coordinate system based on the surface are associated and the first rule is determined. Furthermore, if the geometric element is a three-dimensional shape, the method and rules for creating the coordinate system based on the three-dimensional shape are associated with the second rule, and this is used to determine the second rule. In such cases, the memory device 240 pre-stores a rule set that includes both the first rule and the second rule.

[0064] In addition, if the geometric element is a cylinder, a method and rules for creating a coordinate system based on the cylinder may be associated with a second rule for the cylinder, and if the geometric element is a cone, a method and rules for creating a coordinate system based on the cone may be associated with a second rule for the cone. In such cases, the memory device 240 pre-stores a rule set that includes the second rule for the cylinder and the second rule for the cone.

[0065] Figure 6 is a flowchart showing the process for setting the reference coordinate system. This flowchart is started automatically, for example, when an execution command is received from the user, or without an execution command being received, before the three-dimensional data of the workpiece W is analyzed, and it shows the case when a plane is used to create the coordinate system. Before this flowchart is started, the process of storing the rule set in the storage device 240 is completed.

[0066] In step SA1 after the start, the data acquisition unit 291 acquires three-dimensional data of the workpiece W. The three-dimensional data of the workpiece W acquired by the data acquisition unit 291 is shown in Figure 7, and this three-dimensional data may be mesh data or CAD data.

[0067] In step SA2, the extraction unit 292 extracts geometric elements (in this example, the first plane) from the three-dimensional data of the workpiece W acquired by the data acquisition unit 291. At this time, the extraction unit 292 can extract planes using methods such as RANSAC (Random sample consensus), and by using such methods, it is possible to identify not only planes but also cylinders, spheres, cones, etc.

[0068] Step SA3 determines whether the extraction of the first plane in Step SA2 was successful. Workpiece W may contain multiple planes. In this case, the extraction unit 292 identifies multiple planes as geometric element candidates from the three-dimensional data of workpiece W using a method such as RANSAC. The extraction unit 292 calculates the size of the identified multiple planes and extracts one plane from among the identified multiple planes based on the size of each plane. Specifically, it extracts the largest plane among the identified multiple planes and designates that largest plane as the first plane A1, as shown in Figure 7 (2). If the extraction of the first plane A1 is successful, Step SA3 is determined to be YES and the process proceeds to Step SA4. On the other hand, if the extraction of the first plane A1 fails, Step SA3 is determined to be NO. If the extraction of the first plane A1 fails, it is not possible to create a reference axis based on geometric elements, so this flow is terminated.

[0069] If step SA3 is determined to be YES, the type identification unit 294 identifies the type of geometric element extracted by the extraction unit 292. In this example, since the first plane A1 was extracted by the extraction unit 292, the type identification unit 294 identifies the type of geometric element extracted by the extraction unit 292 as "plane".

[0070] When the type identification unit 294 identifies the type of geometric element extracted by the extraction unit 292, the coordinate system setting unit 295 identifies a rule corresponding to the type of geometric element extracted by the extraction unit 292 from the rule set stored in the storage device 240 based on the type of geometric element identified by the type identification unit 294. In this example, since the geometric element extracted by the extraction unit 292 is a plane, the coordinate system setting unit 295 identifies and reads the rule (first rule) to which the method and rules for creating a coordinate system created based on a plane are associated. Based on the identified rule and the geometric element extracted by the extraction unit 292, the coordinate system setting unit 295 sets the reference coordinate system as described in the following flow.

[0071] In step SA4, the coordinate system setting unit 295 estimates the first reference axis, the second provisional reference axis, and the provisional origin. Specifically, as shown in Figure 8, the normal vector of the first plane A1 is set as the first reference axis B1, and the centroid of the point cloud constituting the first plane A1 is set as the first provisional origin C1. Also, as shown in Figure 9, the first principal component (the direction in which the distribution of points is maximized) obtained when principal component analysis is performed on the point cloud constituting the first plane A1 is set as the second provisional reference axis B2. This allows setting a reference axis based on the maximum plane included in the three-dimensional data of the workpiece W, but since the second reference axis B2 is not set along any reference plane, it is insufficient for setting a cross section. Therefore, the process proceeds to step SA5, where a second plane orthogonal to the first plane A1 is extracted. As shown in (3) of Figure 7, since the extraction unit 292 uses the RANSAC method, multiple planes A2a and A2b can be identified as candidates for the second plane. Although not shown, there are cases where three or more planes are identified.

[0072] Step SA6 determines whether the extraction of the second plane in Step SA5 was successful. The RANSAC method is an algorithm that can identify multiple plane candidates and select the one that fits best. Therefore, if there are many planes as candidates for the second plane, the restriction that they must be orthogonal to the first plane A1 can be applied to easily narrow down the candidates to planes A2a and A2b. As shown in Figure 7 (4), the extraction unit 292 extracts the larger of the two planes A2a and A2b that are orthogonal to the first plane A1, A2a, as the second plane. If there are three or more planes orthogonal to the first plane A1, the largest plane is extracted as the second plane.

[0073] If the extraction of the second plane A2a is successful, step SA6 is judged as YES and the process proceeds to step SA7. Conversely, if the extraction of the second plane A2a fails, step SA6 is judged as NO. If the extraction of the second plane A2a fails, it is not possible to create a reference axis based on geometric elements, so this flow is terminated.

[0074] In step SA7, the coordinate system setting unit 295 estimates the first reference axis, the second reference axis, and the second provisional origin. Specifically, as shown in Figure 10, the normal vector of the first plane A1 is set as the first reference axis B1, and the normal vector of the second plane A2a is set as the second reference axis B2. Once the first reference axis B1 and the second reference axis B2 are determined, the third reference axis B3 can be calculated by taking the cross product of the first reference axis B1 and the second reference axis B2 (shown in Figure 11). Note that the first reference axis B1 and the second reference axis B2 are not perfectly orthogonal, and there may be some error in the angle between the first reference axis B1 and the second reference axis B2. To eliminate this error, the coordinate system setting unit 295 performs the following calculations.

[0075] (Third reference axis) = (First reference axis) x (Second reference axis) (Second reference axis) = (Third reference axis) x (First reference axis) By having the coordinate system setting unit 295 perform this calculation, the second reference axis B2 can be corrected to be precisely orthogonal to the first reference axis B1. At this time, as shown in Figure 12, the coordinate system setting unit 295 can make the first provisional origin C1 the second provisional origin C2 by projecting it onto the intersection line L of the first plane A1 and the second plane A2a.

[0076] In step SA8, the extraction unit 292 extracts a third plane A3 that is orthogonal to the first plane A1 and the second plane A2a, as shown in Figure 13. In step SA9, it is determined whether the extraction of the third plane in step SA8 was successful. If there are many planes listed as candidates for the third plane, the restriction that it must be orthogonal to the first plane A1 and the second plane A2a makes it easy to narrow down the candidates to the third plane A3.

[0077] In step SA10, the coordinate system setting unit 295 sets the origin. As shown in Figure 14, the coordinate system setting unit 295 calculates the point where the first plane A1, the second plane A2a, and the third plane A3 intersect, and sets the point where the first plane A1, the second plane A2a, and the third plane A3 intersect as the origin C of the reference coordinate system. In other words, the coordinate system setting unit 295 estimates the first reference axis B1, the second reference axis B2, and the third reference axis B3 based on the rules stored in the memory device 240 and the planes extracted by the extraction unit 292, and sets the reference coordinate system from the estimated first reference axis B1, second reference axis B2, and third reference axis B3. If the extraction of each plane fails, a provisional origin and provisional reference axes can be used as the final estimated result to set the reference coordinate system.

[0078] The example described above shows that the extraction unit 292 extracts a plane as a geometric element from three-dimensional data. However, the geometric elements extracted from three-dimensional data are not limited to planes; they may also be three-dimensional shapes. For example, if the workpiece W to be measured is a pin as shown in Figure 15, the pin includes a cylindrical part Wa and a conical part Wb as three-dimensional shapes. In this case, the extraction unit 292 extracts the cylindrical part Wa and the conical part Wb as geometric elements from the three-dimensional data. Note that only one of the cylindrical part Wa or the conical part Wb may be extracted. Figure 16 shows the case where the extraction unit 292 extracts the cylindrical part Wa, but the same applies when the conical part Wb is extracted.

[0079] The coordinate system setting unit 295 estimates the first reference axis B1 as shown in (1) of Figure 16. Specifically, the coordinate system setting unit 295 first calculates the axis of the cylindrical part Wa. The coordinate system setting unit 295 sets the calculated axis of the cylindrical part Wa as the first reference axis B1. Similarly, when the cone part Wb is extracted, the coordinate system setting unit 295 calculates the axis of the cone part Wb and sets the calculated axis of the cone part Wb as the first reference axis B1.

[0080] Subsequently, the coordinate system setting unit 295 calculates the centroid D of the point cloud constituting the cylindrical part Wa, as shown in (2) of Figure 16. The coordinate system setting unit 295 projects the centroid D onto the first reference axis B1 and sets this as the first provisional origin C1. In this case, since the centroid D is the centroid of the point cloud constituting the cylindrical part Wa, it does not necessarily coincide with the centroid of the extracted cylindrical part Wa. Similarly, when the cone part Wb is extracted, the coordinate system setting unit 295 calculates the centroid D of the point cloud constituting the cone part Wb and projects the centroid D onto the first reference axis B1 and sets this as the first provisional origin C1.

[0081] Next, the coordinate system setting unit 295 selects point E from the point cloud constituting the cylindrical part Wa, as shown in (3) of Figure 16. After selecting point E, as shown in (4), the line passing through point E selected in (3) and the first reference axis B1, and perpendicular to the first reference axis B1, is defined as the second reference axis B2. In this way, because the cylindrical part Wa has an axially symmetric shape, it is difficult to uniquely determine the second provisional reference axis. Therefore, one point is selected from the point cloud constituting the cylindrical part Wa, and the perpendicular line passing through that point and the first reference axis B1 is defined as the second provisional reference axis. The second reference axis B2 can be determined in the same way for the cone part Wb.

[0082] As shown in (5) of Figure 16, the coordinate system setting unit 295 can determine the third reference axis B3 by taking the cross product of the first reference axis B1 and the second reference axis B2. The third reference axis B3 can be determined in the same way for the cone Wb.

[0083] The analysis unit 296 is the part that performs analysis of the three-dimensional data of the workpiece W based on the reference coordinate system set by the coordinate system setting unit 295. For example, the analysis unit 296 can use a plane intersecting the first reference axis set by the coordinate system setting unit 295 as a cross-section of the three-dimensional data of the workpiece W and perform dimensional analysis of that cross-section, or use a plane intersecting the second reference axis as a cross-section of the three-dimensional data of the workpiece W and perform dimensional analysis of that cross-section, or use a plane intersecting the third reference axis as a cross-section of the three-dimensional data of the workpiece W and perform dimensional analysis of that cross-section, etc.

[0084] Furthermore, the analysis unit 296 can determine the position of the cutting surface based on the center of gravity of the workpiece W. In addition, the analysis unit 296 can accept adjustments to the position of the cutting surface by the user. For example, the user can adjust the position of the cutting surface by operating the operation unit 250, and the analysis unit 296 detects the user's operation of the operation unit 250 and adjusts the position of the cutting surface accordingly. This allows the analysis to be performed using a cutting surface at the user's desired position.

[0085] The analysis unit 296 can also project the three-dimensional data of the workpiece W onto a reference plane defined by multiple reference axes set by the coordinate system setting unit 295, and perform dimensional analysis of the projected shape obtained by the projection. For example, it can analyze dimensions such as the width of the projected shape and the angles of two sides of the projected shape.

[0086] (A form with a plane of symmetry as the reference plane) Many industrial products are symmetrical. For example, the workpiece W shown in Figure 17 is a product that has a plane of symmetry F that passes through the center in the width direction and extends in the vertical direction. For a workpiece W that has a plane of symmetry F, the plane of symmetry F may be used as the reference plane. That is, the plane of symmetry determination unit 293 shown in Figure 5 is the part that determines the symmetry of the three-dimensional data of the workpiece W acquired by the data acquisition unit 291. When the three-dimensional data of the workpiece W is acquired by the data acquisition unit 291, the plane of symmetry determination unit 293 determines whether or not the three-dimensional data of the workpiece W has symmetry. If the plane of symmetry determination unit 293 determines that there is symmetry, it identifies the plane of symmetry F.

[0087] Furthermore, a workpiece W with the shape shown in Figure 15 is also a symmetrical workpiece. In the case of the workpiece W shown in Figure 15, the planes extending radially through the axes of the cylindrical portion Wa and the conical portion Wb become the planes of symmetry. There are countless such planes on the workpiece W shown in Figure 15, but the symmetry plane determination unit 293 identifies any one of these planes as the plane of symmetry.

[0088] The method for detecting the plane of symmetry is not particularly limited; for example, 3DSymm:Robust and Accurate 3D Reflection Symmetry Detection can be used. Furthermore, by replacing the method for determining the reference plane with plane fitting, the flow for estimating the reference axis (shown in Figure 6) can be applied almost directly. Instead of using points constituting the plane when setting the second provisional reference axis, all points can be projected onto the symmetry reference plane.

[0089] If the symmetry plane determination unit 293 determines that the three-dimensional data has symmetry, the extraction unit 292 extracts a symmetry plane as a reference plane. The coordinate system setting unit 295 sets the reference axes based on the symmetry plane extracted as the reference plane. By using the symmetry plane F, it becomes possible to appropriately set the reference plane even if there are no clear planes or cylinders in the three-dimensional data.

[0090] (Creation of a reference coordinate system by combining planes, solid shapes, and planes of symmetry) As described above, in this embodiment, a reference coordinate system can be automatically created using the planes, cylinders, cones, and planes of symmetry included in the three-dimensional data of the workpiece W. However, it is not limited to this, and a reference coordinate system can also be automatically created using any two or more combinations of the planes, cylinders, cones, and planes of symmetry. For example, a combination of a cylinder and a plane, or a combination of a plane of symmetry and a plane. It is also possible to simultaneously detect these multiple geometric elements and select the one that fits the most points to determine the reference axis.

[0091] On the other hand, it is also possible to accept user-specified planes and limit the process to those specified by the user, with the first reference axis being identified using a plane and the second reference axis being identified using a cylinder.

[0092] One way to easily implement such combinations of multiple geometric elements is to create a rule set, such as a table or tree, that shows multiple rules, and to pre-store this rule set in the memory device 240. For example, when two types of rules are set, one for geometric elements being planes or planes of symmetry (where the determining element of the reference axis is a plane) and another for geometric elements being cylinders or cones (where the determining element of the reference axis is an axis), a table is constructed using the rule applied to estimate the first reference axis (the first rule), the rule applied to estimate the second reference axis (the second rule), and the rule applied to estimate the third reference axis (the third rule) in order to estimate the first, second, and third reference axes.

[0093] Specifically, to show examples of combinations in the order of the first rule - second rule - third rule, a total of eight tables are created in advance: face-face-face, face-face-axis, face-axis-face, face-axis-axis, axis-face-face, axis-axis-face-axis, and axis-axis-axis. "Face" is the rule for geometric elements that are planes or planes of symmetry, and "axis" is the rule for geometric elements that are cylinders or cones. By creating such rule sets in advance, the reference axis and origin can be easily determined in accordance with the geometric elements extracted by the extraction unit 292.

[0094] Figure 18 is a flowchart for setting a reference coordinate system based on multiple types of geometric elements. In step SB1 after the start, the data acquisition unit 291 acquires three-dimensional data of the workpiece W. In step SB2, the extraction unit 292 extracts geometric elements from the three-dimensional data of the workpiece W acquired by the data acquisition unit 291.

[0095] Step SB3 determines whether the extraction of geometric elements in Step SB2 was successful. If the extraction of geometric elements fails, the result in Step SB3 is NO. If the extraction of geometric elements fails, it is not possible to create a reference axis based on the geometric elements, so this flow is terminated.

[0096] In step SB2, if a plane, cylinder, cone, or plane of symmetry is extracted as a geometric element, the extraction unit 292 determines YES in step SB3. In step SB4, the type identification unit 294 determines whether the extracted geometric element is a plane or plane of symmetry, or a cylinder or cone. The coordinate system setting unit 295 then determines which rule in the rule set to apply as the first rule to estimate the reference axis. If the extracted geometric element is a plane or plane of symmetry, the "plane" rule is applied as the first rule; if the extracted geometric element is a cylinder or cone, the "axis" rule is applied as the first rule. After determining the rule to apply, the coordinate system setting unit 295 estimates the first reference axis and the provisional origin according to the determined rule.

[0097] In step SB5, the extraction unit 292 extracts the next geometric element (second geometric element) from the three-dimensional data of the workpiece W according to the first rule applied in step SB4. If a plane, cylinder, cone, or plane of symmetry is extracted as a geometric element in step SB5, the extraction unit 292 determines YES in step SB6. In step SB7, the type identification unit 294 determines whether the extracted geometric element is a plane or plane of symmetry, or a cylinder or cone. The coordinate system setting unit 295 then determines which rule in the table to apply as the second rule to estimate the reference axis. If the extracted geometric element is a plane or plane of symmetry, the "plane" rule is applied as the second rule; if the extracted geometric element is a cylinder or cone, the "axis" rule is applied as the second rule. After determining the rule to apply, the coordinate system setting unit 295 estimates the second reference axis and the provisional origin according to the determined rule.

[0098] In step SB8, the extraction unit 292 extracts the next geometric element (the third geometric element) from the three-dimensional data of the workpiece W according to the second rule applied in step SB7. If a plane, cylinder, cone, or plane of symmetry is extracted as the geometric element in step SB8, the extraction unit 292 determines YES in step SB9. In step SB9, the type identification unit 294 determines whether the extracted geometric element is a plane or plane of symmetry, or a cylinder or cone. The coordinate system setting unit 295 then determines which rule in the table to apply as the third rule to estimate the reference axis. If the extracted geometric element is a plane or plane of symmetry, the "plane" rule is applied as the third rule; if the extracted geometric element is a cylinder or cone, the "axis" rule is applied as the third rule. After determining the rule to apply, the coordinate system setting unit 295 estimates the third reference axis and the origin according to the determined rule and sets the reference coordinate system.

[0099] As described above, when the extraction unit 292 extracts geometric elements from the three-dimensional data of the workpiece W, it can extract multiple candidate reference planes. Furthermore, when the extraction unit 292 extracts geometric elements from the three-dimensional data of the workpiece W, it can also extract multiple candidate solid shapes. When the extraction unit 292 extracts multiple candidate reference planes and multiple candidate solid shapes, it calculates the degree of agreement between the point cloud constituting each candidate reference plane and the candidate reference plane, and the degree of agreement between the point cloud constituting each candidate solid shape and the candidate solid shape. From the extracted multiple candidate reference planes and multiple candidate solid shapes, the extraction unit 292 can extract geometric elements to be used for creating a coordinate system based on the degree of agreement between the point cloud constituting each candidate reference plane and the candidate reference plane, and the degree of agreement between the point cloud constituting each candidate solid shape and the candidate solid shape.

[0100] (Measurement item suggestion function) The 3D scanner 1 in this embodiment is not essential, but it has a measurement item suggestion function. While the 3D scanner 1 can acquire 3D data of the workpiece W, there are many types of 3D measurement items, such as cross-sectional measurement, 3D measurement, planar measurement, thickness measurement, and geometric tolerance. With so many measurement items, inexperienced users often don't know which ones to use. On the other hand, experienced users can generally understand which measurement items are commonly used just by looking at the workpiece W. The measurement item suggestion function assists inexperienced users by suggesting items similar to those selected by experienced users. Measurement items are also referred to as analysis menus.

[0101] There are several methods for implementing the measurement item suggestion function, but the two main methods are described below.

[0102] Method 1 involves creating training and test data by combining acquired three-dimensional data with pairs of recommended values ​​for each measurement item to be performed on the acquired three-dimensional data, and then constructing a rule-based algorithm or a machine learning model trained on the training data.

[0103] Method 2 involves creating data that links the acquired three-dimensional data with the type of workpiece W included in the acquired three-dimensional data (e.g., whether it is sheet metal or a machined part), classifying these data using an algorithm similar to Method 1 (rule-based or machine learning model), and calculating the recommendation level for measurement items from those items.

[0104] The difference between Method 1 and Method 2 lies in the labels assigned to the training data. Method 1 uses recommendation scores for each measurement item set by experts, while Method 2 uses product types as industrial products. Since product types in Method 2 are relatively easy to determine, the cost of building training data for Method 2 is lower. On the other hand, if the correspondence between product types and measurement items is fixed, and the probability of belonging to the same item is constant, then the recommendation score for the measurement item will also be fixed. For example, if the probability of being sheet metal is constant, then the recommendation score for thickness measurement will not change regardless of the type of sheet metal.

[0105] Next, we will describe classification methods. When three-dimensional data is acquired, feature vectors are calculated first. These can be obtained using machine learning methods such as PointNet, rule-based methods such as VFH (Viewpoint Feature Histogram), or by listing physical quantities that humans can understand, such as volume and surface area. Of course, these can be combined.

[0106] These features are used as input to create a classifier using rule-based or machine learning algorithms. If the features are a series of physical quantities that humans can understand, such as volume or surface area, then classification rules can be determined using heuristics. For example, a small volume / surface area value indicates a thin object; principal component analysis can be performed, and if only the first principal component is large, it indicates a rod shape; if both the first and second principal components are large, it indicates a plate shape; and if both the first and third principal components are large, it indicates a complex shape. These methods do not require training data, making them effective when only a small amount of data is available.

[0107] Machine learning can be used when a sufficient amount of training data is available. Representative classifiers that can be used include, for example, SVM (Support Vector Machine), Random Forest, and DNN (Deep Neural Network). The output of these classifiers is the probability of belonging to that classification result (for example, a probability of 0.6 for sheet metal, a probability of 0.4 for flat plate, etc.). In the case of Method 1 mentioned above, the output is the probability that an expert would choose that measurement item, so the output can be used directly as a recommendation score.

[0108] On the other hand, in the case of Method 2, the classifier output is the probability of belonging to each type of object, so it cannot be used directly as a score for the measurement item. For example, if the thickness is to be measured for pressed sheet metal or flat plate, the classifier's belonging probabilities are P(pressed) and P(flat plate), P (thickness measurement) = max[P (pressed), P (flat plate)] It can be calculated as follows.

[0109] If the object is not cylindrical and is block-shaped, when proposing cross-sectional measurement, P(cross-sectional measurement) can be calculated as min[1-P(cylinder), P(block)].

[0110] Logical operations can be implemented by replacing AND with the min operation, NOT with the difference from 1, and OR with the max operation. Of course, this can also be achieved by treating AND as multiplication, OR as addition, and NOT as the difference from 1, and clipping the results within the range of 0 and 1.

[0111] After calculating the recommendation score in this way, the items are sorted in descending order of recommendation score, and when an item is selected, a description of that measurement item is displayed. This allows users to select the measurement items they are most likely to use simply by looking at each item from top to bottom. It is also possible to automatically perform measurements on a certain number of top-ranking items, or items with a recommendation score above a certain value, and for the user to select the measurement items they want to perform while viewing the results on their own 3D data. In this case, it is necessary to automatically determine the reference axis, so it is best to set the reference coordinate system using the coordinate system setting unit 295 described above. Also, since performing multiple measurements takes time, it is possible to automatically perform measurements on pre-decimated preview 3D data and display the results.

[0112] The above model identifies a combination of analysis menus to be used for analyzing three-dimensional data from a pre-prepared set of analysis menus, based on the shape of the three-dimensional data of the workpiece W. This model is stored in the storage device 240 shown in Figure 2, and includes rule-based classifications such as classification lists and correspondence tables, as well as machine learning classifications. For example, the storage device 240 can store a classification list that shows the classification targets for classifying three-dimensional data based on shape, and a correspondence table that associates each classification target included in the classification list with the analysis menu used for analyzing the three-dimensional data.

[0113] The analysis module 290 has a model creation unit 298. The model creation unit 298 receives input of multiple training 3D data and training data to which variety / shape classification is associated for each training 3D data, and creates a model that classifies variety / shape based on the input training data. When the model creation unit 298 creates a model, the storage device 240 can store the model created by the model creation unit 298 and a correspondence table to which variety / shape and analysis menu are associated.

[0114] Furthermore, the model creation unit 298 can also receive input of multiple training 3D data sets and training data to which analysis menus are associated with each training 3D data set, and create a model to estimate which analysis menu to be used for analyzing the 3D data of workpiece W from among multiple pre-prepared analysis menus based on the input training data. In this case, the storage device 240 can store the model for estimating which analysis menu to be used for analyzing the 3D data of workpiece W from among multiple pre-prepared analysis menus.

[0115] As shown in Figure 5, the analysis module 290 has a candidate identification unit 297. The candidate identification unit 297 acquires the shape of the three-dimensional data acquired by the data acquisition unit 291 and the model stored in the storage device 240. Based on the shape of the three-dimensional data acquired by the data acquisition unit 291 and the model stored in the storage device 240, the candidate identification unit 297 identifies candidate combinations of analysis menus to be used for analyzing the three-dimensional data from among a plurality of pre-prepared analysis menus. If a classification list is stored in the storage device 240, the candidate identification unit 297 can calculate the similarity of the three-dimensional data acquired by the data acquisition unit 291 for each classification target included in the classification list stored in the storage device 240. Then, based on the calculated similarity of the classification targets and the correspondence table stored in the storage device 240, the candidate identification unit 297 calculates a recommendation score for each of the plurality of analysis menus. The candidate identification unit 297 can also identify the analysis menu to be used for analysis from among the analysis menus based on the recommendation score. For example, analysis menus with a recommendation level above a certain threshold, or analysis menus with a high recommendation level, are suggested to the user as the analysis menu to be used.

[0116] Furthermore, if a model has been created in the model creation unit 298, the candidate identification unit 297 can classify the three-dimensional data acquired by the data acquisition unit 291 based on the model created in the model creation unit 298 and stored in the storage device 240, and then identify the analysis menu to be used for analyzing the three-dimensional data based on the classified type / shape and the correspondence table stored in the storage device 240.

[0117] The display control unit 255 shown in Figure 2 is the part that displays the combination of analysis menus identified by the candidate identification unit 297 on the display unit 400. Figures 19 and 20 show the suggestion screen 700 that the display control unit 255 displays on the display unit 400. The suggestion screen 700 is provided with an analysis menu display area 701, a measurement result display area 702, an execution button 703 to be operated when performing a measurement, and a cancel button 704 to be operated when canceling a measurement.

[0118] The analysis menu display area 701 displays a list of analysis menu combinations identified by the candidate identification unit 297. The recommendation level of each analysis menu is displayed numerically (Figure 19) or graphically (Figure 20) in the analysis menu display area 701, with higher numerical values ​​indicating a higher recommendation level. In other words, the analysis menu display area 701 displays multiple analysis menus and information indicating the recommendation level of each analysis menu. In addition, an explanatory diagram or preview screen 705 corresponding to each analysis menu may be displayed on the proposal screen 700. The explanatory diagram or preview screen 705 may be switched to the explanatory diagram or preview screen corresponding to the analysis menu indicated by the cursor, for example, based on the cursor position (see Figure 21). Figure 21 shows the case where the cursor position is on the geometric tolerance menu, and the explanatory diagram corresponding to the geometric tolerance menu is displayed on the screen.

[0119] Furthermore, checkboxes are provided for each analysis menu, allowing the user to select which analysis menu to use for measurement from a given combination of analysis menus. In other words, the analysis unit 296 is configured to accept user selections of analysis menus.

[0120] When the execute button 703 is pressed, the measurement is performed according to the selected analysis menu, and the measurement results are displayed in the measurement result display area 702. On the other hand, when the cancel button 704 is pressed, the measurement is canceled without being performed.

[0121] If the execute button 703 is operated while multiple analysis menus are selected, measurements based on each selected analysis menu are performed, and multiple measurement results corresponding to each analysis menu are displayed. The multiple measurement results may be displayed in tab format, each in a different window, or multiple measurement results may be arranged side by side on a single results screen. Figure 22 shows the case where cross-sectional measurement and three-dimensional measurement are selected as analysis menus, and measurements are performed by the analysis unit 296. In this way, a first measurement results screen and a second measurement results screen may be displayed. For example, the first measurement results screen may show the results of the three-dimensional measurement, and the second measurement results screen may show the results of the cross-sectional measurement.

[0122] In the cross-sectional measurement results screen 800 shown in Figure 23, three-dimensional data may be displayed in the first result display area 801 based on the coordinate system set by the coordinate system setting unit 295. In the first result display area 801, the three-dimensional data may be displayed in three-dimensional space. Alternatively, the three-dimensional data may be displayed in the second result display area 802 based on the coordinate system set by the coordinate system setting unit 295. In the second result display area 802, the three-dimensional data may be displayed as a two-dimensional image viewed from a predetermined direction. Furthermore, as shown in Figure 24, the analysis unit 296 may cut the three-dimensional data with a plane intersecting the first axis of the coordinate system set by the coordinate system setting unit 295, and the cross-sectional shape obtained by cutting the three-dimensional data may be displayed in the third result display area 803. Also, if cross-sectional measurement is selected as the analysis menu and the measurement is performed by the analysis unit 296, a second result screen 810 for cross-sectional measurement may be displayed in addition to the cross-sectional measurement results screen 800. The second result screen 810 for cross-sectional measurement may include a first result display area 801, a second result display area 802, and a third result display area 803, similar to the cross-sectional measurement result screen 800. In addition, the third result display area 803 may display the cross-sectional shape obtained by cutting the three-dimensional data with a plane that intersects the second axis of the coordinate system set by the coordinate system setting unit 295, and the analysis unit 296 cutting the three-dimensional data.

[0123] In the three-dimensional measurement results screen shown in Figure 25, the three-dimensional data is displayed based on the coordinate system set by the coordinate system setting unit 295. In addition to the three-dimensional data, the analysis results from the analysis unit 296, such as the dimensions of geometric elements extracted by the extraction unit 292 and the distances between geometric elements, may also be superimposed on the three-dimensional data and displayed.

[0124] Furthermore, the analysis module 290 includes a preliminary analysis unit 299 (shown in Figure 5) that reduces the resolution of the three-dimensional data acquired by the data acquisition unit 291 and performs a preliminary analysis based on the combination of the reduced-resolution three-dimensional data and the analysis menu identified by the candidate identification unit 297. In this case, the display control unit 255 can display the preliminary analysis results from the preliminary analysis unit 299 in a preview format on the display unit 400 or the measurement result display area 702.

[0125] Furthermore, the analysis module 290 can read CAD data stored in the storage device 240 as a reference model, and the analysis unit 296 can perform the analysis and compare it with the three-dimensional data of the workpiece W. Figure 26 shows an example of performing cross-sectional measurement on CAD data. For the CAD data, a reference coordinate system is set by the extraction unit 292, the type identification unit 294, and the coordinate system setting unit 295, similar to the flow shown in Figure 6. The analysis unit 296 can perform dimensional analysis on the cross-section of the CAD data using a plane intersecting the first reference axis set by the coordinate system setting unit 295, or on a plane intersecting the second reference axis, or on a plane intersecting the third reference axis. Figure 23 shows an example of performing cross-sectional measurement on the three-dimensional data of the workpiece W. By comparing the results of cross-sectional measurement of the CAD data and the three-dimensional data of the workpiece W, it is possible to compare the design values ​​of the CAD data with the measured values ​​of the workpiece W. The analysis module 290 may automatically perform the above comparison based on receiving an instruction to read the CAD model, and the display control unit 255 may display the comparison results on the display unit 400. Alternatively, the analysis module 290 may automatically perform the above comparison based on receiving an instruction to start the comparison, and the display control unit 255 may display the comparison results on the display unit 400.

[0126] Furthermore, the analysis unit 296 can align the read-out CAD data and the three-dimensional data of the workpiece W based on shape and coordinate system information. For the data in which the three-dimensional data of the workpiece W and the CAD data have been aligned, the extraction unit 292, the type identification unit 294, and the coordinate system setting unit 295 set a reference coordinate system, similar to the flow shown in Figure 6. The analysis unit 296 can perform dimensional analysis on the cross-section of the data in which the three-dimensional data of the workpiece W and the CAD data have been aligned, using the plane intersecting the first reference axis set by the coordinate system setting unit 295 as the cross-section of the data in which the three-dimensional data of the workpiece W and the CAD data have been alignedsectional measurement results of the aligned CAD data and the three-dimensional data of the workpiece W. The analysis module 290 may automatically perform the above comparison based on receiving an instruction to read the CAD model, and the display control unit 255 may display the comparison results on the display unit 400. Alternatively, the analysis module 290 may automatically perform the above comparison based on receiving an instruction to start the comparison, and the display control unit 255 may display the comparison results on the display unit 400.

[0127] The embodiments described above are in all respects merely illustrative and should not be interpreted restrictively. Furthermore, any modifications or changes that fall within the equivalent scope of the claims are all within the scope of the present invention. [Industrial applicability]

[0128] As described above, the present invention can be used when analyzing three-dimensional data of a workpiece. [Explanation of Symbols]

[0129] 1. Three-dimensional scanner (information processing device) 240 Storage device (storage unit) 291 Data Acquisition Unit 292 Extraction part 293 Symmetry plane determination unit 294 Category Identification Section 295 Coordinate System Setting Section 296 Analysis Department

Claims

1. An information processing device for analyzing three-dimensional data of a workpiece with a predetermined coordinate system, A memory unit that pre-stores a rule set that associates the types of geometric elements used to create a coordinate system with the coordinate system information created based on each type of geometric element, A data acquisition unit that acquires three-dimensional data of the workpiece, An extraction unit extracts geometric elements from the three-dimensional data of the workpiece acquired by the data acquisition unit, A type identification unit that identifies the type of geometric element extracted by the extraction unit, A coordinate system setting unit that identifies a rule from the rule set stored in the storage unit based on the type of geometric element identified by the type identification unit, and sets a reference coordinate system based on the identified rule and the geometric element extracted by the extraction unit, An analysis unit that performs analysis of the three-dimensional data of the workpiece based on the reference coordinate system set by the coordinate system setting unit, Equipped with, information processing device.

2. In the information processing apparatus according to claim 1, The memory unit stores rules that associate a surface as a type of geometric element with information of a coordinate system created based on the surface. The extraction unit extracts faces as geometric elements from the three-dimensional data, The coordinate system setting unit is an information processing device that estimates a reference axis based on the rule and the plane extracted by the extraction unit, and sets a reference coordinate system from the estimated reference axis.

3. In the information processing apparatus according to claim 2, The extraction unit is an information processing device that identifies multiple faces as candidate geometric elements and extracts one face from among the identified multiple faces based on the size of each face.

4. In the information processing apparatus according to claim 2, The memory unit stores rules that associate the three-dimensional shape as a type of geometric element with the coordinate system information created based on the three-dimensional shape. The extraction unit extracts the three-dimensional shape as geometric elements from the three-dimensional data, The coordinate system setting unit estimates a reference axis based on the rule and the three-dimensional shape extracted by the extraction unit, and sets a reference coordinate system from the estimated reference axis, and is an information processing device.

5. In the information processing apparatus according to claim 1, The system further includes a symmetry plane determination unit that determines the symmetry of the three-dimensional data of the workpiece acquired by the data acquisition unit, If the symmetry plane determination unit determines that the three-dimensional data has symmetry, the extraction unit extracts the symmetry plane as the reference plane. The coordinate system setting unit is an information processing device that sets reference axes based on a plane of symmetry extracted as a reference plane.

6. In the information processing apparatus according to claim 1, The extraction unit extracts a plurality of candidate reference planes and a plurality of candidate three-dimensional shapes, and extracts geometric elements from the extracted plurality of candidate reference planes and a plurality of candidate three-dimensional shapes based on the degree of agreement between the point cloud constituting each candidate reference plane and the candidate reference plane, and the degree of agreement between the point cloud constituting each candidate three-dimensional shape and the candidate three-dimensional shape.

7. In the information processing apparatus according to claim 2, The extraction unit extracts planes as geometric elements from the three-dimensional data, The coordinate system setting unit sets a first reference axis based on the normal direction of the plane extracted by the extraction unit, and sets an origin based on the centroid of the point cloud constituting the plane extracted by the extraction unit, an information processing device.

8. In the information processing apparatus according to claim 7, The coordinate system setting unit is an information processing device that sets a second reference axis based on the distribution of point clouds constituting the plane extracted by the extraction unit.

9. In the information processing apparatus according to claim 4, The extraction unit extracts at least one of a cylinder and a cone as a three-dimensional shape from the three-dimensional data as a geometric element. The coordinate system setting unit identifies a first reference axis based on the three-dimensional shape extracted by the extraction unit, and sets an origin based on the centroid of the point cloud constituting the three-dimensional shape extracted by the extraction unit, and is an information processing device.

10. In the information processing apparatus according to claim 1, The analysis unit is an information processing device that uses a plane intersecting a reference axis set by the coordinate system setting unit as the cross-section of the three-dimensional data of the workpiece, and performs dimensional analysis of the said cross-section.

11. In the information processing apparatus according to claim 10, The analysis unit is an information processing device that determines the position of the cross-section based on the center of gravity of the workpiece.

12. In the information processing apparatus according to claim 11, The analysis unit is an information processing device that accepts adjustments to the position of the cross-section by the user.

13. In the information processing apparatus according to claim 1, The coordinate system setting unit sets a plurality of reference axes based on the identified rule and the geometric elements extracted by the extraction unit, The analysis unit is an information processing device that projects three-dimensional data of the workpiece onto a reference plane determined by a plurality of reference axes set by the coordinate system setting unit, and performs dimensional analysis of the projected shape obtained by the projection.

14. An information processing method for analyzing three-dimensional data of a workpiece with a predetermined coordinate system, A set of rules that associates the types of geometric elements used to create a coordinate system with the coordinate system information created based on each type of geometric element is stored in the memory beforehand. Acquire 3D data of the workpiece, Geometric elements are extracted from the acquired 3D data of the workpiece. Identify the type of geometric element extracted, Based on the type of geometric element identified, one rule is identified from the rule set stored in the memory unit, and a reference coordinate system is set based on the identified rule and the extracted geometric element. An information processing method for analyzing three-dimensional data of a workpiece based on a set reference coordinate system.

15. A storage medium containing an information processing program that causes a computer to perform information processing to analyze three-dimensional data of a workpiece with a predetermined coordinate system set, The computer has a memory unit that stores in advance a set of rules that associates the types of geometric elements used to create a coordinate system with the coordinate system information created based on each type of geometric element. The process of acquiring 3D data of the workpiece, The process involves extracting geometric elements from the acquired 3D data of the workpiece, A process to identify the type of extracted geometric element, A process of identifying a rule from the rule set stored in the memory unit based on the type of geometric element identified, and setting a reference coordinate system based on the identified rule and the extracted geometric element, A storage medium containing an information processing program that performs a process to analyze the three-dimensional data of the workpiece based on a set reference coordinate system.

Images