Reverse Engineering System

Abstract

To enable easy reverse engineering of only a part of the components constituting a workpiece or only a part of the workpiece.SOLUTION: A reverse engineering system 1 includes: an analysis module 290 having an element extraction section 292 that extracts a plurality of geometric elements from acquired mesh data and a calculation section 293 that calculates dimensions between the plurality of geometric elements; and a conversion module 270 that performs CAD conversion on the mesh data to generate CAD data. The analysis module 290 has a setting section 294 that sets a conversion area for the execution of CAD conversion in the mesh data. The conversion module 294 has a CAD conversion section 273 that converts the mesh data extracted on the basis of the conversion area into CAD data and a CAD output section 274 that outputs the CAD data generated by the CAD conversion section 273.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present disclosure relates to a reverse engineering system that acquires the shape of an object, converts it into design data, and utilizes it in manufacturing.

Background Art

[0002] In recent years, reverse engineering has been used in various aspects of manufacturing. For example, the shape of an existing product is scanned and converted into CAD data for next-generation product development and shape analysis on CAD / CAE, or the shape of a model or mock-up in product design is scanned and converted into CAD data for reflection in product design, or the shape of a mating part is scanned and converted into CAD data for designing the product serving as the mating source, or the shape of a prototype is scanned and converted into CAD data for leading to an improved design. Such applications exist in reverse engineering and are showing a tendency to expand.

[0003] Non-Patent Document 1 discloses that when scanning an existing shape with a laser distance scanner and converting the dense point cloud generated by this scanner into a geometric model, the surface is specified from the point cloud.

Prior Art Documents

Non-Patent Documents

[0004]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Conventional software that enables reverse engineering combines pre-processing steps such as feature extraction and hole filling as part of the CAD data output preparation process, with the conversion process to CAD data, into a single package. Therefore, it is assumed that the entire 3D model of the workpiece is being processed.

[0006] By the way, there are times when you want to reverse engineer only some of the multiple parts that make up a workpiece, or only a part of the workpiece. In such cases, you want to partially edit the 3D data of the entire workpiece. However, conventional software assumes that it processes the entire 3D data of the workpiece, so if you want to reverse engineer only a part of it, you need to delete the mesh data of the unnecessary areas, which is time-consuming.

[0007] This disclosure has been made in view of the foregoing, and its purpose is to enable easy reverse engineering of only some of the components constituting a workpiece, or only a portion of a workpiece. [Means for solving the problem]

[0008] To achieve the above objective, this embodiment may assume a reverse engineering system that converts mesh data obtained by measuring the shape of a workpiece into CAD data and outputs it. The reverse engineering system includes an analysis module having a data acquisition unit that acquires mesh data, an element extraction unit that extracts a plurality of geometric elements from the acquired mesh data, and a calculation unit that calculates the dimensions between the plurality of extracted geometric elements, and a conversion module that converts the mesh data into CAD data to create CAD data. The analysis module further includes a setting unit that sets a conversion region in which CAD conversion is performed from the mesh data acquired by the data acquisition unit, a data extraction unit that extracts mesh data to be output to the conversion module based on the conversion region set by the setting unit, and a conversion data output unit that outputs the mesh data extracted by the data extraction unit to the conversion module. The conversion module also includes a data input unit that receives input of mesh data output from the conversion data output unit, a CAD conversion unit that converts the mesh data input to the data input unit into CAD data, and a CAD output unit that outputs the CAD data created by the CAD conversion unit.

[0009] With this configuration, when multiple geometric elements are extracted from the mesh data of a workpiece, it becomes possible to measure the dimensions between the extracted geometric elements, and it also becomes possible to set a conversion region within the mesh data to perform CAD conversion. For example, it becomes possible to set only some of the parts that make up the workpiece as the conversion region, or to set only a part of the workpiece as the conversion region. Once the conversion region is set, the mesh data is extracted based on the conversion region and converted into CAD data, so when you want to reverse engineer only a part of the workpiece, the process of deleting the mesh data of unnecessary areas is unnecessary, reducing the effort required from the user.

[0010] In another embodiment, by outputting the mesh data of the conversion region to the conversion module, mesh data with a smaller data capacity than the mesh data acquired by the data acquisition unit can be output to the conversion module. This reduces the time required for conversion compared to when the CAD conversion unit converts all of the mesh data acquired by the data acquisition unit into CAD data.

[0011] Furthermore, when extracting mesh data based on the conversion region, one might consider dividing the mesh data. When mesh data is divided, a hole region without mesh data, i.e., a loop without polygons, is formed at the dividing surface. In this case, by creating new mesh data within the loop and correcting the loop, it is possible to prevent the loss of part of the converted CAD data.

[0012] Furthermore, it is conceivable that some of the mesh data acquired by the data acquisition unit may need to be approximated by geometric elements such as cylinders or planes. In this case, the user specifies the approximation region to be approximated by a geometric element, and also specifies what kind of geometric element the approximation region should be approximated by. The approximation processing unit of the analysis module can then approximate the mesh data included in the approximation region with the specified geometric element.

[0013] Geometric element data can be obtained by approximating the mesh data included in the approximation region with geometric elements. In this case, the geometric element data and the mesh data of the conversion region may be mixed and output to the conversion module. That is, the parts that are necessary when converted to CAD data can be output as mesh data so that they can be adjusted and edited during the conversion to CAD data, while the parts that are not so important when converted to CAD data can be approximated and output as geometric elements.

[0014] The CAD conversion unit can convert mesh data in the conversion region into CAD data as surface data of a freeform surface, and can also convert mesh data included in the approximation region into CAD data as surface data of an analyzed surface.

[0015] Multiple conversion modules may be provided. For example, if there is a first conversion module and a second conversion module, the setting unit can set a first conversion area for performing the first CAD conversion and a second conversion area for performing the second CAD conversion from the mesh data acquired by the data acquisition unit. The data extraction unit can extract the first mesh data based on the first conversion area and extract the second mesh data based on the second conversion area. The conversion data output unit can output the first mesh data to the first conversion module and the second mesh data to the second conversion module, so, for example, multiple CAD data conversion processes can be executed simultaneously, or the analysis module can be operated while the CAD data conversion process is being executed.

[0016] Furthermore, if the first conversion module is configured to convert the first mesh data into CAD data, the second conversion module can convert the second mesh data into CAD data regardless of the operating status of the first conversion module. In other words, multiple conversion modules can independently perform the CAD data conversion process.

[0017] Furthermore, the conversion data output unit can create an output file that associates the mesh data acquired by the data acquisition unit with the coordinate system used to display the mesh data on the display unit, and can also output this file to the conversion module. In this case, by displaying the mesh data contained in the output file on the display unit based on the coordinate system contained in the output file, the display state of the mesh data in the analysis module can be reproduced on the display unit.

[0018] Furthermore, the conversion data output unit can perform a coordinate transformation on the mesh data based on the coordinate system used to display the mesh data on the display unit, and output the transformed mesh data to the conversion module. In this case, by displaying the transformed mesh data on the display unit, the display unit can reproduce the display state of the mesh data in the analysis module.

[0019] In addition, on the display unit, in addition to the mesh data, a measurement setting area for accepting the specification of the shape of geometric elements and the specification of the dimension calculation method can be displayed. Furthermore, a conversion area specification area for accepting the specification of the conversion area for executing CAD conversion and an output instruction icon for accepting an instruction for outputting the mesh data to the conversion module can also be displayed. Therefore, each module can be clearly shown on the user interface.

[0020] In addition, on the display unit, a CAD conversion target data display area and a conversion instruction icon for accepting a conversion instruction for converting the mesh data displayed in the CAD conversion target data display area into CAD data can also be displayed.

Advantages of the Invention

[0021] As described above, a plurality of geometric elements are extracted from the mesh data, and the mesh data extracted based on the conversion area set as the area for executing CAD conversion among the mesh data is converted into CAD data. Therefore, only some of the parts constituting the workpiece or only a part of the workpiece can be easily reverse-engineered.

Brief Description of the Drawings

[0022] [Figure 1] It is a diagram showing the overall configuration of a reverse engineering system according to an embodiment of the present invention. [Figure 2] It is a block diagram of a reverse engineering system. [Figure 3] It is a side view of the measurement unit and the pedestal unit. [Figure 4] It is a block diagram of the measurement unit. [Figure 5] It is a diagram showing a configuration example of a module. [Figure 6] It is a diagram showing a configuration example of a module according to Modification 1. [Figure 7] It is a diagram showing a configuration example of a module according to Modification 2. [Figure 8] This diagram shows the first half of a flowchart illustrating the operation of a reverse engineering system. [Figure 9] This diagram shows the latter half of a flowchart illustrating the operation of a reverse engineering system. [Figure 10] This figure shows an example of a parameter table used when setting processing parameters. [Figure 11] This figure shows an example of a dataset's data structure. [Figure 12] This diagram shows each data point included in the measurement data. [Figure 13] This figure shows an example of a user interface screen for specifying a conversion area. [Figure 14] This figure shows an example of a user interface screen for surface conversion. [Figure 15] This figure shows an example of a user interface screen for mesh editing. [Figure 16] This figure shows an example of a user interface screen for displaying results. [Figure 17] This block shows the details of the analysis module. [Figure 18] This is a flowchart showing an example of the processing procedure for the analysis module. [Figure 19] This is a flowchart showing an example of a measurement processing procedure. [Figure 20] This flowchart shows an example of the procedure for extracting geometric elements. [Figure 21] This flowchart shows an example of a mesh selection process. [Figure 22] This is a flowchart showing an example of a dimensional measurement process. [Figure 23] This flowchart shows an example of a CAD conversion instruction procedure. [Figure 24] This flowchart shows an example of the procedure for specifying the target for CAD conversion. [Figure 25] A flowchart illustrating an example of the mesh editing procedure. [Figure 26]This flowchart shows an example of the procedure for converting data to CAD data. [Figure 27] This figure shows an example of a user interface screen for creating analytical surfaces. [Figure 28] This diagram shows the procedure for creating the analytical surface. [Figure 29] This figure shows an example of a window for selecting target elements. [Figure 30] This diagram illustrates a configuration with multiple conversion modules. [Figure 31] This is a functional block diagram for generating CAD data that reflects shape attributes. [Figure 32] This flowchart shows an example of the procedure for generating CAD data that reflects shape attributes. [Figure 33] This flowchart shows an example of the procedure for specifying a user area. [Figure 34] This flowchart shows an example of the procedure for specifying the user area when a preview display is available. [Figure 35] This flowchart shows an example of the procedure for displaying a preview. [Figure 36] This flowchart shows an example of the mesh selection process. [Figure 37] This figure shows the procedure for combining user-specified areas and automatically generated areas, and provides an example of the combining process. [Figure 38] This figure shows an example of a user interface screen for specifying a region. [Figure 39] This flowchart shows an example of the processing procedure when adding additional information. [Figure 40] This figure shows an example of a color selection window. [Figure 41] This figure shows an example of the structure of measurement data with additional information. [Figure 42] This flowchart shows an example of the correction process procedure. [Figure 43] This flowchart shows an example of the procedure for extracting the correction reference plane. [Figure 44]This flowchart shows an example of the procedure for extracting candidate areas for correction. [Figure 45] This flowchart shows an example of the procedure for identifying hole locations. [Figure 46] This flowchart shows an example of the procedure for replacing mesh data. [Figure 47] This figure shows an example of area selection. [Figure 48] This is a diagram showing an example of a fill-in-the-blank. [Figure 49] This flowchart shows an example of the procedure for assigning a coordinate system. [Figure 50] This flowchart shows an example of the procedure for selecting a coordinate system. [Figure 51] This flowchart shows an example of the procedure for creating a coordinate system. [Figure 52] This flowchart shows an example of the procedure for selecting the conversion region. [Figure 53] This diagram shows the screen used to set the reference plane, reference axis, and origin. [Figure 54] This figure shows an example of a user interface screen that displays measurement data and reference data. [Figure 55] This figure shows an example of a user interface screen that indicates the state after the alignment of measurement data and reference data has been completed. [Figure 56] This diagram shows the first half of a flowchart illustrating the operation of a reverse engineering system related to a modification. [Figure 57] This figure shows the latter half of a flowchart illustrating the operation of a reverse engineering system related to a modified example. [Figure 58] This figure shows an example of a user interface screen for displaying conversion results. [Figure 59] This figure shows an example of the manual conversion settings screen. [Figure 60] This figure shows an example of a display screen when a spherical region is converted into CAD data as a single surface data. [Figure 61]This figure shows an example of a user interface screen for surface transformation when multiple geometric elements are selected. [Figure 62] This figure shows an example of a user interface screen for displaying conversion results when surface data and 3D elements are combined and displayed. [Figure 63] This figure shows an example of a user interface screen for displaying conversion results when only 3D elements are shown. [Figure 64] This figure shows an example of the user interface screen when data synthesis is initiated. [Figure 65] This figure shows an example of a composite data selection window. [Figure 66] This figure shows an example of a user interface screen for acquiring measurement data for a workpiece whose orientation has been changed. [Modes for carrying out the invention]

[0023] Embodiments of the present invention will be described in detail below with reference to the drawings. The following description of preferred embodiments is essentially illustrative and is not intended to limit the present invention, its applications, or its uses.

[0024] Figure 1 shows the overall configuration of a reverse engineering system 1 according to an embodiment of the present invention. The reverse engineering system 1 is a system that converts mesh data of a workpiece (object to be measured) W, obtained by measuring the shape of the workpiece W, into CAD data and outputs it. The reverse engineering system 1 is used, for example, to acquire CAD data of an existing product and perform next-generation development or shape analysis on CAD / CAE, to reflect the shape of a model or mock in product design into product design, to design a product to be fitted based on the shape of a mating part, or to perform improvement design based on the shape of a prototype. Therefore, the workpiece W is, for example, an existing product, a model, a mock, a prototype, etc.

[0025] Furthermore, the reverse engineering system 1 is a device capable of converting the mesh data of the workpiece W into surface data and outputting it. By converting the mesh data of the workpiece W into surface data and outputting it, it can support the user's reverse engineering process and reverse engineering work, so the reverse engineering system 1 can also be called a reverse engineering support device 1.

[0026] In the following description, when measuring the shape of a workpiece W, coordinate information of the workpiece W surface is obtained by irradiating the workpiece W with a predetermined pattern of measurement light and using the signal obtained from the reflected light reflected from the surface of the workpiece W. For example, as the predetermined pattern of measurement light, a measurement method using triangulation with a fringe projection image obtained from the reflected light, which is projected onto the workpiece W, can be used. However, in this invention, the principle and configuration for obtaining the coordinate information of the workpiece W are not limited to this, and other methods can also be applied.

[0027] The reverse engineering system 1 comprises a measuring unit 100, a base unit 600, a controller 200, a light source unit 300, and a display unit 400. The reverse engineering system 1 uses the light source unit 300 to perform structured illumination on the workpiece W, captures a fringe projection image, generates a depth image containing coordinate information, and can measure the three-dimensional dimensions and shape of the workpiece W based on this image. Measurement using such fringe projection has the advantage of shortening measurement time because three-dimensional measurement can be performed without moving the workpiece W or optical systems such as lenses in the Z direction (height direction).

[0028] Figure 2 shows a block diagram of a reverse engineering system 1 according to an embodiment of the present invention. As shown in this figure, the measurement unit 100 includes a light-emitting unit 110, a light-receiving unit 120, a measurement control unit 150, and an illumination light output unit 130. The light-emitting unit 110 is the part that irradiates the workpiece W, which is placed on the mounting unit 140 (described later), with measurement light having a predetermined pattern. The light-receiving unit 120 is fixed in an inclined position with respect to the mounting surface 142 (described later). The light-receiving unit 120 receives the measurement light irradiated by the light-emitting unit 110 and reflected by the workpiece W. When the light-receiving unit 120 receives reflected light from the workpiece W, it outputs a light-receiving signal representing the amount of light received. The light-receiving unit 120 can generate an observation image for observing the overall shape of the workpiece W by imaging the workpiece W placed on the mounting unit 140.

[0029] The light-receiving unit 120 according to this embodiment includes a high-magnification light-receiving unit and a low-magnification light-receiving unit. The high-magnification light-receiving unit is the part of the workpiece W that can be imaged at a magnified view compared to the low-magnification light-receiving unit. On the other hand, the low-magnification light-receiving unit is a light-receiving unit with a wider field of view compared to the high-magnification light-receiving unit.

[0030] The base portion 600 comprises a base plate 602, a mounting portion 140, and a movement control unit 144. The mounting portion 140 is supported on the base plate 602 of the base portion 600. The movement control unit 144 is a component that moves the mounting portion 140. The movement control unit 144 may be provided on the base portion 600 side or on the controller 200 side.

[0031] The light source unit 300 is connected to the measurement unit 100. The light source unit 300 is the part that generates measurement light and supplies it to the measurement unit 100. The controller 200 is the part that controls the measurement unit 100 and the like. The display unit 400 is connected to the controller 200 and is configured to display the image generated by the measurement unit 100, and to allow necessary settings, inputs, selections, etc.

[0032] The mounting section 140 has a mounting surface 142 on which the workpiece W is placed. As shown in Figure 4, two mutually orthogonal directions within the mounting surface 142 are defined as the X direction and the Y direction, and are indicated by arrows X and Y, respectively. The direction perpendicular to the mounting surface 142 of the mounting section 140 is defined as the Z direction, and is indicated by arrow Z. The direction of rotation around an axis parallel to the Z direction is defined as the θ direction, and is indicated by arrow θ.

[0033] The mounting section 140 includes a rotating stage 143 that rotates the mounting surface 142 around an axis extending in the Z direction, and a translational stage 141 that moves the mounting surface 142 horizontally (in the X and Y directions). The translational stage 141 has an X-direction movement mechanism and a Y-direction movement mechanism. The rotating stage 143 has a θ-direction rotation mechanism. The mounting section 140 may also include a fixing member (clamp) for fixing the workpiece W to the mounting surface 142. Furthermore, the mounting section 140 may include a tilt stage having a mechanism that can rotate around an axis parallel to the mounting surface 142.

[0034] The movement control unit 144 controls the rotational movement of the rotating stage 143 and the parallel movement of the translational stage 141 according to the measurement conditions set by the measurement condition setting unit 261, which will be described later. The movement control unit 144 also controls the movement of the mounting unit 140 by the mounting movement unit based on the measurement area set by the measurement condition setting unit 261, which will be described later.

[0035] The controller 200 includes a CPU (Central Processing Unit) 210, ROM (Read-Only Memory) 220, working memory 230, storage device (storage unit) 240, and an operation unit 250, etc. A PC (Personal Computer) or the like can be used as the controller 200.

[0036] The configuration of the measurement unit 100 is shown in the block diagram of Figure 4. The measurement unit 100 includes a light-emitting unit 110, a light-receiving unit 120, an illumination light output unit 130, a measurement control unit 150, and a main body case 101 that houses these components. The light-emitting unit 110 includes a measurement light source 111, a pattern generation unit 112, and a plurality of lenses 113, 114, and 115. The light-receiving unit 120 includes a camera 121 and a plurality of lenses 122 and 123. When performing measurements at different magnifications by providing multiple light-receiving units, a light-receiving unit 120a including a low-magnification camera 121 and low-magnification lenses, and a light-receiving unit 120b including a high-magnification camera 121 and high-magnification lenses may be installed. Note that the configuration is not limited to this one; the magnification may be varied by switching between multiple lenses for a single camera 121, or by providing a zoom lens for a single camera 121.

[0037] The light-emitting unit 110 is positioned diagonally above the mounting unit 140. In the example shown in Figure 4, the measurement unit 100 includes two light-emitting units 110, but the measurement unit 100 may include multiple light-emitting units 110. Here, a first measurement light-emitting unit 110A (right side in Figure 4) is provided, which can irradiate the object WK to be measured with a first measurement light ML1 from a first direction, and a second measurement light-emitting unit 110B (left side in Figure 4) is provided, which can irradiate the object WK to be measured with a second measurement light ML2 from a second direction different from the first direction. The first measurement light-emitting unit 110A and the second measurement light-emitting unit 110B are arranged symmetrically with the optical axis of the light-receiving unit 120 as the center of symmetry. Although not shown, it is also possible to have three or more light-emitting units 110, or to move the light-emitting unit 110 and the mounting unit 140 relative to each other to project light onto the workpiece W in different directions, even while using a common light-emitting unit 110. In the above example, multiple light-emitting units 110 are provided and the light is received by a common light-receiving unit 120. However, conversely, multiple light-receiving units 120 may be provided to receive light from a common light-emitting unit 110. Furthermore, in this example, the irradiation angle of the illumination light emitted by the light-emitting unit 110 with respect to the Z direction is fixed, but this can also be made variable.

[0038] Each first measurement light projection unit 110A and second measurement light projection unit 110B is equipped with a first measurement light source and a second measurement light source, respectively, as measurement light sources 111. These measurement light sources 111 are, for example, halogen lamps that emit white light. The measurement light source 111 may also be a light source that emits monochromatic light, such as a blue LED (light-emitting diode) or organic EL that emits blue light. The light emitted from the measurement light source 111 (hereinafter referred to as "measurement light") is appropriately focused by the lens 113 and then incident on the pattern generation unit 112.

[0039] The relative positions of the light-receiving unit 120, light-emitting units 110A and 110B, and the light-receiving unit 120 are determined such that the central axes of the light-emitting units 110A and 110B intersect at a position where the arrangement of the workpiece W on the mounting unit 140 and the depth of field of the light-emitting and light-receiving units 110 and 120 are appropriate. Furthermore, since the center of the rotation axis in the θ direction coincides with the central axis of the light-receiving unit 120, when the mounting unit 140 rotates in the θ direction, the workpiece W rotates within the field of view around the rotation axis without moving out of the field of view.

[0040] The pattern generation unit 112 reflects the light emitted from the measurement light source 111 so that it projects measurement light onto the workpiece W. The measurement light incident on the pattern generation unit 112 is converted into a preset pattern and a preset intensity (brightness) and emitted. The measurement light emitted by the pattern generation unit 112 is converted by a plurality of lenses 114 and 115 into light with a diameter larger than the observation and measurement field of view of the light receiving unit 120, and then irradiated onto the workpiece W on the mounting unit 140.

[0041] The pattern generation unit 112 is a component that can switch between a projection state in which measurement light is projected onto the workpiece W and a non-projection state in which measurement light is not projected onto the workpiece W. For example, a DMD (Digital Micromirror Device) can be used for such a pattern generation unit 112. A pattern generation unit 112 using a DMD can be controlled by the measurement control unit 150 to switch between a reflection state in which the measurement light is reflected onto the optical path as the projection state and a light-shielding state in which the measurement light is blocked as the non-projection state.

[0042] A DMD is an element in which numerous micromirrors (tiny mirror surfaces) are arranged on a plane. Each micromirror can be individually switched ON or OFF by the measurement control unit 150, so a desired projection pattern can be constructed by combining the ON and OFF states of numerous micromirrors. This makes it possible to generate the pattern necessary for triangulation and measure the shape of the workpiece W. In this way, the DMD functions as a projection pattern optical system that projects a periodic projection pattern for measurement onto the workpiece W during measurement. Furthermore, the DMD has excellent response speed and offers the advantage of being able to operate at high speeds compared to shutters and the like.

[0043] In the above example, an example using a DMD for the pattern generation unit 112 was described, but the present invention is not limited to a DMD for the pattern generation unit 112, and other materials can be used. For example, an LCOS (Liquid Crystal on Silicon: reflective liquid crystal element) may be used as the pattern generation unit 112. Alternatively, a transmissive material may be used instead of a reflective material to adjust the amount of light transmitted for measurement. In this case, the pattern generation unit 112 is placed on the optical path of the measurement light, and a light projection state that transmits the measurement light and a light shielding state that blocks the measurement light are switched. For example, an LCD (liquid crystal display) can be used for such a pattern generation unit 112. Alternatively, the pattern generation unit 112 may be configured using a projection method using multiple line LEDs, a projection method using multiple optical paths, an optical scanner method composed of a laser and a galvanometer mirror, an AFI (Accordion fringe interferometry) method that uses interference fringes generated by superimposing beams divided by a beam splitter, or a projection method using a physical grid composed of a piezo stage and a high-resolution encoder and a moving mechanism.

[0044] The light-receiving unit 120 is positioned above the mounting unit 140. The measurement light reflected upward from the workpiece W towards the mounting unit 140 is collected and imaged by the multiple lenses 122 and 123 of the light-receiving unit 120, and then received by the camera 121.

[0045] Camera 121 is a CCD (charge-coupled device) camera, for example, including an image sensor 121a. Image sensor 121a is, for example, a monochrome CCD (charge-coupled device). Image sensor 121a may be other image sensors such as a CMOS (complementary metal-oxide-semiconductor) image sensor. Color image sensors require each pixel to correspond to the reception of red, green, and blue light, resulting in lower measurement resolution compared to monochrome image sensors, and sensitivity is reduced because each pixel requires a color filter. Therefore, in this embodiment, a monochrome CCD is used as the image sensor, and a color image is acquired by illuminating the illumination light output unit 130 (described later) with illumination corresponding to RGB in a time-division manner and capturing the image. With this configuration, a color image of the object to be measured can be acquired without reducing the measurement accuracy.

[0046] A color image sensor may also be used as the image sensor 121a. In this case, although the measurement accuracy and sensitivity will be lower compared to a monochrome image sensor, it will no longer be necessary to irradiate the image sensor with illumination corresponding to RGB in a time-division manner from the illumination light output unit 130. A color image can be acquired simply by irradiating with white light, thus simplifying the illumination optical system. Each pixel of the image sensor 121a outputs an analog electrical signal corresponding to the amount of light received (hereinafter referred to as the "received light signal") to the measurement control unit 150.

[0047] The measurement control unit 150 is equipped with an A / D converter (analog-to-digital converter) and a FIFO (First In First Out) memory (not shown). The light received signal output from the camera 121 is sampled at a constant sampling period and converted into a digital signal by the A / D converter of the measurement control unit 150, based on control by the light source unit 300. The digital signals output from the A / D converter are sequentially stored in the FIFO memory. The digital signals stored in the FIFO memory are sequentially transferred to the controller 200 as pixel data.

[0048] The control unit 250 of the controller 200 may include, for example, a keyboard or a pointing device. Examples of pointing devices include a mouse or a joystick.

[0049] The ROM 220 of the controller 200 stores system programs and the like. The working memory 230 of the controller 200 consists of, for example, RAM (Random Access Memory) and is used for processing various data. The storage device 240 consists of a solid-state drive, a hard disk drive, etc. The storage device 240 stores a reverse engineering program. The storage device 240 is also used to store various data such as pixel data (image data), setting information, and measurement conditions provided by the measurement control unit 150. Measurement conditions include, for example, the settings of the light-emitting unit 110 (pattern frequency, pattern type) and the type of light-receiving unit 120 (low-magnification light-receiving unit, high-magnification light-receiving unit), which are set by the scanner module 260 described later when measuring the shape of the workpiece W. Furthermore, the storage device 240 can also store brightness information, coordinate information, and attribute information for each pixel that makes up the measurement image.

[0050] The CPU210 is a control circuit or control element that processes given signals and data, performs various calculations, and outputs the calculation results. In this specification, CPU refers to an element or circuit that performs calculations, and is used to mean not limited to processors such as CPUs, MPUs, GPUs, and TPUs for general-purpose PCs, regardless of their name, but also including processors such as FPGAs, ASICs, LSIs, microcontrollers, and chipsets such as SoCs.

[0051] The CPU 210 generates image data based on pixel data provided by the measurement control unit 150. The CPU 210 also performs various processing on the generated image data using the working memory 230. For example, based on the light-receiving signal output from the light-receiving unit 120, the CPU 210 generates measurement data representing the three-dimensional shape of the workpiece W contained within the field of view of the light-receiving unit 120 at a specific position on the mounting unit 140. The measurement data is the image itself acquired by the light-receiving unit 120. For example, when measuring the shape of the workpiece W using a phase-shift method, multiple images constitute one set of measurement data. The measurement data may also be point cloud data, which is a collection of points having three-dimensional position information. Measurement data of the workpiece W can be obtained using this point cloud data. Point cloud data is data represented by a collection of multiple points having three-dimensional coordinates.

[0052] When measuring the shape of a workpiece W, if measurement data for at least a portion of the workpiece W is acquired as described above, the movement control unit 144 moves the mounting unit 140 to acquire measurement data for other parts of the workpiece W located around the acquired portion of measurement data, and this process is repeated to generate measurement data for other parts of the workpiece W at that position. By combining the multiple obtained measurement data, composite measurement data including the overall shape of the workpiece W can be generated. The mode in which composite measurement data is generated can be called the linking mode, and by selecting the linking mode, a wider field of view image can be obtained compared to when imaging with a single field of view.

[0053] The movement control unit 144 determines, based on measurement data of at least a portion of the workpiece W, whether to perform only the rotation of the rotating stage 143 or both the rotation of the rotating stage 143 and the translation of the translation stage 141. This facilitates three-dimensional measurement by automatically determining the imaging range according to the external shape of the workpiece W without the user having to be aware of it. Furthermore, the movement control unit 144 can control the rotation of the rotating stage 143 after moving the translation stage 141 in the XY direction and then stopping its movement in the XY direction, thereby also acquiring the shape of the area around the workpiece W.

[0054] The display unit 400 is a component for displaying stripe projection images acquired by the measurement unit 100, depth images generated based on the stripe projection images, texture images captured by the measurement unit 100, various user interface screens, etc. The display unit 400 is composed of, for example, an LCD panel or an organic EL (electroluminescent) panel. Furthermore, by using a touch panel in the display unit 400, it can also be used in conjunction with the operation unit 250. The display unit 400 can also display images generated by the light receiving unit 120.

[0055] The light source unit 300 includes a control board 310 and an observation illumination light source 320. A CPU (not shown) is mounted on the control board 310. The CPU of the control board 310 controls the light-emitting unit 110, the light-receiving unit 120, and the measurement control unit 150 based on commands from the CPU 210 of the controller 200. Note that this configuration is just one example, and other configurations are possible. For example, the light-emitting unit 110 and the light-receiving unit 120 could be controlled by the measurement control unit 150, or the light-emitting unit 110 and the light-receiving unit 120 could be controlled by the controller 200, thus omitting the control board. Alternatively, a power supply circuit for driving the measurement unit 100 can be provided in this light source unit 300.

[0056] The observation illumination light source 320 includes, for example, three LEDs that emit red, green, and blue light. By controlling the brightness of the light emitted from each LED, the observation illumination light source 320 can generate light of any color. The illumination light IL generated from the observation illumination light source 320 is output from the illumination light output unit 130 of the measurement unit 100 through a light guide member (light guide). In addition to LEDs, other light sources such as semiconductor lasers (LDs), halogen lights, and HIDs can also be used as appropriate for the observation illumination light source. In particular, if a color imaging sensor is used as the image sensor, a white light source can be used for the observation illumination light source.

[0057] The illumination light IL output from the illumination light output unit 130 illuminates the workpiece W by switching between red, green, and blue light in a time-division manner. This allows the texture images captured by these RGB lights to be combined to obtain a color texture image, which can then be displayed on the display unit 400.

[0058] The controller 200, consisting of a CPU 210, ROM 220, working memory 230, storage device 240, etc., comprises the scanner module 260, conversion module 270, integration module 280, and analysis module 290 shown in Figure 5. The detailed operation of these modules 260, 270, and 290 will be described later, but will be explained based on the flowcharts shown in Figures 8 and 9.

[0059] In general terms, the scanner module 260 acquires image data of the workpiece W by measuring its shape and creates mesh data of the workpiece W based on that image data. The conversion module 270 converts the mesh data created by the scanner module 260 into CAD data. CAD data is three-dimensional shape information composed of analytical surfaces and freeform surfaces, and includes surface data, solid data, and data used for design. Surface data is data of shape surfaces composed of freeform surfaces and analytical surfaces, such as side data and planar data of a cylinder.

[0060] The analysis module 290 is responsible for acquiring mesh data, extracting multiple geometric elements from the acquired mesh data, and calculating the dimensions between the extracted geometric elements.

[0061] The integration module 280 is responsible for transmitting signals and data from the scanner module 260 to the conversion module 270 and the analysis module 290, and transmitting signals and data from the conversion module 270 to the scanner module 260. In this example, a module is a unit capable of executing multiple arithmetic processes, and can also be called a functional unit, functional block, etc.

[0062] As will be described in detail later, the scanner module 260 includes, for example, a measurement condition setting unit 261, a scanner control unit 262, a mesh data creation unit 263, a scanner output unit 264, etc. The measurement condition setting unit 261 is the part that sets the measurement conditions for the shape of the workpiece. The scanner control unit 262 is the part that controls the measurement unit 100 according to the measurement conditions set in the measurement condition setting unit 261 to generate image data and acquires measurement data of the workpiece W based on the generated image data. The mesh data creation unit 263 is the part that creates mesh data based on the image data of the workpiece W acquired by the scanner control unit 262. The scanner output unit 264 is the part that outputs the mesh data created by the mesh data creation unit 263 and additional data to the conversion module 270. The additional data is data that includes at least one of the measurement conditions and data calculated from the measurement data of the workpiece W.

[0063] In other words, the scanner module 260 controls the measurement unit 100 and generates 3D data along with the various conditions under which the shape measurement of the workpiece W was performed (measurement model, measurement magnification, resolution, etc.) and the raw data (e.g., image data) at the time of measurement. The 3D data is mesh data containing multiple polygons and can also be called polygon data. A polygon is data composed of information that identifies multiple points and information that shows the polygonal surface formed by connecting those points. For example, it can be composed of information that identifies three points and information that shows the triangular surface formed by connecting those three points. Mesh data and polygon data can also be defined as data represented by a collection of multiple polygons.

[0064] The conversion module 270 converts mesh data into CAD data and determines the conversion process based on measurement conditions and raw data. Specifically, the conversion module 270 includes, for example, a data input unit 271, a processing parameter determination unit 272, a CAD conversion unit 273, a CAD output unit 274, a correction unit 275, etc. The data input unit 271 is the part that receives mesh data output from the scanner output unit 264 and additional data. The processing parameter determination unit 272 is the part that determines the processing parameters when converting mesh data into CAD data according to the additional data received by the data input unit 271, and is configured to determine processing parameters based on a parameter table as shown in Figure 10. The CAD conversion unit 273 is the part that converts mesh data into CAD data according to the processing parameters determined by the processing parameter determination unit 272. The CAD output unit 274 is the part that outputs the CAD data converted by the CAD conversion unit 273. In addition, the mesh data information can be stored in the form shown in Figures 11 and 12. This will be described later.

[0065] The conversion module 270 can determine conversion parameters, conversion algorithms and their combinations, processing order, and number of iterations in the CAD conversion process. For example, it can change the search radius when calculating curvature from polygons based on measurement conditions such as magnification. Depending on the model of the measurement unit 100, it can also determine whether or not to implement a smoothing filter during CAD conversion, and determine weights in polygon processing according to the image brightness data in the raw data.

[0066] The analysis module 290 includes, for example, a data acquisition unit 291 for acquiring mesh data, an element extraction unit 292 for extracting multiple geometric elements from the acquired mesh data, a calculation unit 293 for calculating the dimensions between the extracted multiple geometric elements, a setting unit 294, a data extraction unit 295, a converted data output unit 296, an approximation processing unit 297, and the like.

[0067] The analysis module 290 further includes a setting unit 294 for setting a conversion area from the mesh data acquired by the data acquisition unit 291 to perform CAD conversion, a data extraction unit 295 for extracting mesh data to be output to the conversion module 270 based on the conversion area set by the setting unit 294, and a scanner output unit (conversion data output unit) 264 for outputting the mesh data extracted by the data extraction unit 295 to the conversion module 270. The conversion data output unit may be replaced by the scanner output unit 264 of the scanner module 260, or it may be provided separately from the scanner output unit 264.

[0068] This reverse engineering system 1 is an all-in-one system that includes all of the measurement, processing, and conversion modules 260 and 270, and data transmission and reception between each module 260 and 270 can be automated.

[0069] To automate data transmission and reception between modules 260 and 270, an integration module 280 is provided to integrate these modules 260 and 270. This allows each module 260 and 270 to proceed with the processing specified by the integration module 280. The role of the integration module 280 can also be consolidated into one of the modules 260 or 270 that perform measurement, processing determination, CAD conversion, etc. Furthermore, the functions of the analysis module 290 can be integrated into the scanner module 260.

[0070] Figure 6 shows an example of the module configuration according to Modification 1. In this Modification 1, the processing determination module 270A is composed of a data input unit 271 and a processing parameter determination unit 272, and the processing execution module 270B is composed of a CAD conversion unit 273 and a CAD output unit 274. The conversion module 270 is composed of the processing determination module 270A and the processing execution module 270B. In the case of Modification 1, all data transmission and reception are performed via the integration module 280.

[0071] Figure 7 shows an example of the module configuration according to Modification 2. In this Modification 2, the integration module is omitted, and the scanner module 260, the processing decision module 270A, and the processing execution module 270B are configured in series.

[0072] (Operation of the reverse engineering system) Figures 8 and 9 are flowcharts illustrating the operation of the reverse engineering system 1. In step SA1 after startup, the main application, the analysis application, is launched on the controller 200. In step SA2, the scanner module 260 is launched. In step SA3, the measurement magnification (low or high) is selected. Steps SA1 to SA3 are user operations. User operations can be performed using the operation unit 250, etc., and the operating status of the operation unit 250 is detected by the CPU 210 and reflected in each control.

[0073] In step SA4, the scanner module 260 determines whether the measurement magnification selected in step SA3 is low or high. If the measurement magnification selected in step SA3 is high, the process proceeds to step SA5, and the light receiving unit 120 is switched to high magnification. On the other hand, if the measurement magnification selected in step SA3 is low, the process proceeds to step SA6, and the light receiving unit 120 is switched to low magnification. Steps SA5 and SA6 are performed by the scanner module 260.

[0074] In step SA7, exposure conditions such as illumination brightness, exposure time, and gain are adjusted. In step SA8, the measurement mode (resolution / number of multiple shots) is selected. In step SA9, another measurement mode (color on / off, measurement range) is selected. In step SA10, the number of measurement shots is specified. In step SA11, the measurement execution button is pressed. Steps SA7 to SA11 are user operations; the resolution setting, number of multiple shots setting, color on / off setting, and measurement range setting are all accepted by the measurement condition setting unit 261. The resolution is the resolution of the measurement unit (imaging unit) 100.

[0075] Step SA12 is when the scanner module 260 performs the measurement. That is, when the scanner module 260 detects that the measurement execution button has been pressed, the scanner control unit 262 controls the measurement unit 100 according to the measurement conditions set in the measurement condition setting unit 261 to generate image data, and acquires measurement data of the workpiece W based on the generated image data. Once the scanner control unit 262 has acquired the measurement data of the workpiece W, the mesh data creation unit 263 creates mesh data based on that measurement data. At this time, confidence information for each polygon vertex is also created.

[0076] In step SA13, the measurement results are displayed on the display unit 400. In step SA14, it is determined whether the measurement is complete or not. If it is not complete, the process returns to step SA3; if it is complete, the process proceeds to step SA15. In step SA15, the measurement result synthesis is performed. In step SA16, the measurement result synthesis performed in step SA15 is displayed on the display unit 400.

[0077] In step SA17, it is determined again whether the measurement is complete or not. If it is not complete, the process returns to step SA3; if it is complete, the process proceeds to step SA18 to execute the measurement completion process.

[0078] In step SA19, the scanner output unit 264 outputs the mesh data created by the mesh data creation unit 263 and the additional data to the conversion module 270. As a result, the measurement data, along with information on the magnification, resolution, number of multiple scans, presence or absence of color, and measurement range at the time the measurement data was acquired, are transferred to the main application as first additional data, while information on the reliability of each polygon vertex, etc., is transferred to the main application as second additional data. Furthermore, an image of the workpiece W based on the mesh data is displayed on the display unit 400.

[0079] In step SA20, the coordinate system is set and selected, and then in step SA21, the conversion area to be converted to CAD data is specified or selected. Next, in step SA22, the user starts the conversion module 270 on the controller 200. When the conversion module 270 is started, the mesh data output from the scanner output unit 264 and the input of additional data are received by the data input unit 271. In this embodiment, the coordinate system and conversion area are determined in the main application (analysis module 290), but the coordinate system and conversion area may also be determined in the conversion module 270.

[0080] In other words, the conversion module 270 generates a user interface screen 700 for specifying a conversion area, as shown in Figure 13, and displays it on the display unit 400. The user interface screen 700 for specifying a conversion area includes an image display area 701 for displaying an image of the workpiece W and a conversion area specification area 702. The conversion area specification area 702 includes a full area specification button 702a, a first area specification button 702b for specifying an area enclosed by the operation unit 250, a second area specification button 702c for specifying one of two areas divided by a predetermined plane, a third area specification button 702d for specifying an area enclosed by a polygon drawn by the operation unit 250, and a release button 702e for releasing the specification. When the full area specification button 702a is operated, the entire area of ​​the workpiece W is specified. When the first area specification button 702b, the second area specification button 702c, and the third area specification button 702d are operated, a portion of the area of ​​the workpiece W is specified. Furthermore, the user interface screen 700 for specifying the conversion area is provided with an output instruction button (conversion instruction button) 704. When the output instruction button 704 is operated, mesh data is output to the conversion module 270.

[0081] Next, the process proceeds to step SB1 of the flowchart shown in Figure 9. In step SB1, it is determined whether the conversion area to be converted to CAD data is the entire area of ​​workpiece W or only a part of the area (partial area) of workpiece W. If it is the entire area, the process proceeds to step SB2, where the entire polygon of the mesh data output from the scanner output unit 264 is extracted. If it is a partial area, the process proceeds to step SB3, where only the specified area of ​​polygon from the mesh data output from the scanner output unit 264 is extracted.

[0082] In step SB4, the user selects a conversion mode. There are two conversion modes: a fully automatic mode that requires no user settings or adjustments, and a manual mode that requires user settings and adjustments. Step SB5 determines whether the user selected the fully automatic mode or the manual mode. If it is the manual mode, the process proceeds to step SB6, where, as shown in Figure 14, the surface conversion GUI (graphical user interface) is launched, and the conversion module 270 displays the surface conversion user interface screen 710 on the display unit 400. The surface conversion GUI can also be called a GUI for converting to CAD data.

[0083] The user interface screen 710 for surface conversion includes an image display area 711 for displaying an image of the workpiece W and a detailed settings area 712. The detailed settings area 712 includes a pre-processing setting area 712a for setting the processing to be performed before surface conversion, an output format selection area 712b, and a conversion method setting area 712c. In the output format selection area 712b, users can choose between STEP files and IGES files. Note that the selectable output formats are not limited to these; CAD native formats, Parasolid formats, etc., may also be available.

[0084] In step SB7, before converting the mesh data to surface data (CAD data), it is determined whether or not to perform preprocessing such as hole filling on the mesh data. Here, preprocessing includes hole filling, removal of unnecessary parts, healing, and remeshing. Hole filling involves identifying open edges (edges without adjacent polygons) on the mesh data, identifying loops composed of consecutive open edges, and then correcting the loops by creating new mesh data within those loops. Specifically, the correction unit 275 of the conversion module 270 corrects the loops by creating new mesh data within loops that do not contain polygons if the mesh data received by the data input unit 271 contains such loops. In short, in step SB7, the loops are filled with polygons to generate data without holes. This makes it possible to convert completely closed mesh data with no holes into CAD data, so the converted data can also be solid data. Remeshing is a process that makes the triangles of the mesh closer to equilateral triangles.

[0085] If preprocessing is required for the mesh data, proceed to step SB8 and perform the preprocessing yourself. This process is automated and should be performed at the appropriate time during the reverse engineering process.

[0086] In step SB8, the conversion module 270 generates a mesh editing user interface screen 720 as shown in Figure 15 and displays it on the display unit 400. The mesh editing user interface screen 720 includes an image display area 721 where an image based on mesh data is displayed, and an operation area 722. The operation area 722 includes selection operation buttons 722a for selecting a portion of the mesh data displayed in the image display area 721, and delete buttons 722b for deleting the selected area. The user operates the selection operation buttons 722a and delete buttons 722b to perform hole filling, unnecessary part removal, healing, remeshing, etc.

[0087] Step SB9 determines whether or not to perform surface area specification. If YES, proceed to step SB10; otherwise, proceed to step SB11. Also, if it is determined in step SB7 that mesh preprocessing should not be performed, proceed to step SB11.

[0088] After specifying the surface area (the area to be converted to CAD data) in step SB10, step SB12 determines whether the surface area specification is complete. If the surface area specification is not complete, the process returns to step SB10; if the surface area specification is complete, the process proceeds to step SB13.

[0089] Step SB14 determines whether the magnification of the light-receiving unit 120 is low or high. If it is low, the process proceeds to step SB15 to select low-magnification parameters, a low-magnification algorithm, and a low-magnification processing flow. If it is high-magnification, the process proceeds to step SB16 to select high-magnification parameters, a high-magnification algorithm, and a high-magnification processing flow. Steps SB14 to SB16 are executed by the processing parameter determination unit 272, and by going through steps SB14 to SB16, the processing parameters for converting mesh data to CAD data can be determined according to the additional data received in step SA19.

[0090] For example, the side length of a single polygon may differ by a factor of three between low and high magnification. In this case, it becomes possible to set conditions according to the magnification, such as using a parameter set in which the surrounding search conditions from each vertex of the polygon are reduced to 1 / 3. Also, depending on the model of the measurement unit 100, there is a risk that unique noise may be generated depending on the magnification, in which case the smoothing intensity may be changed according to the magnification as one of the processing parameters.

[0091] The resolution selected in step SA8 shown in Figure 8 can be associated with the smoothing kernel size as one of the processing parameters. The number of multiple exposures selected in step SA8 can be associated with the smoothing processing intensity as one of the processing parameters. The presence or absence of color selected in step SA9 can be associated with the CAD surface color processing judgment based on surface color information as one of the processing parameters. The measurement range selected in step SA9 can be associated with the downsampling processing intensity as one of the processing parameters. These processing parameters differ for low magnification and high magnification, and are automatically set to the appropriate processing parameters after processing through steps SB14 to SB16.

[0092] By referencing the brightness of each vertex from the raw data and confidence parameters linked to shape information (polygon vertices), it is possible to determine noise shapes that do not actually exist on the surface of the workpiece W. By ignoring the noise shapes from the conversion region and estimating the surface from the surrounding shape, it becomes possible to convert the workpiece W into CAD data that is closer to the actual object.

[0093] The measurement conditions for measuring the shape of the workpiece W include the magnification, field of view (single field or wide field), exposure conditions (exposure time, gain, illumination brightness), resolution setting (low-resolution measurement, standard measurement, high-resolution measurement), and whether or not color is used.

[0094] If the process proceeds from step SB15 to step SB17, the surface conversion is performed by applying low-multiplier parameters, a low-multiplier algorithm, and a low-multiplier processing flow. If the process proceeds from step SB16 to step SB17, the surface conversion is performed by applying high-multiplier parameters, a high-multiplier algorithm, and a high-multiplier processing flow. In other words, the CAD conversion unit 273 converts the mesh data into CAD data according to the processing parameters determined by the processing parameter determination unit 272.

[0095] More specifically, the measurement conditions for the workpiece W are transferred from the scanner module 260 to the conversion module 270 along with the shape information of the workpiece W. The processing parameter determination unit 272 determines the processing parameters based on a parameter table as shown in Figure 10. The parameter table associates items of additional data, such as measurement conditions, with processing parameters corresponding to each item, and is stored in the storage device 240. Therefore, the processing parameter determination unit 272 can determine the processing parameters based on the items of additional data and the parameter table.

[0096] In this example, resolution information is assigned as an item in the parameter table's supplementary data. Therefore, the parameter table associates resolution information with processing parameters corresponding to that resolution information.

[0097] For example, in the case of magnification, the spatial resolution is determined according to the optical magnification of the lens and the pixel pitch of the image sensor 121a, so the resolution of the shape information changes due to the change in magnification. Therefore, in the calculation processing performed by the CAD conversion unit 273, the appropriate values ​​for, for example, the peripheral search radius for curvature calculation and the intensity when applying the filter will differ. As shown in Figure 10, by creating a table of each processing parameter for each measurement condition in advance, the processing parameters to be applied by the CAD conversion algorithm can be automatically determined based on the magnification condition of the light receiving unit 120.

[0098] As another example, in the case of a field of view, there are conditions under which the sampling of measurement data can be made coarser based on the tendency for the size of the workpiece W to increase. Furthermore, by multiplying some of the settings of the processing parameters set at a certain magnification by a coefficient, it is possible to set processing parameters that ignore fine details for larger workpieces W. In the example shown in Figure 10, at low magnification and a wide field of view, the micro-hole detection size can be set to 0.2 and the shape detection ratio to 2.0, and the total micro-hole detection size can be set to 0.2 × 2.0 = 0.4.

[0099] On the other hand, independently of the determination of processing parameters through steps SB14 to SB16, the CAD conversion unit 273 can also determine processing parameters for conversion to CAD data using additional data (e.g., confidence level, validity flag, color information, etc.) that is calculated from the image data of the workpiece W and is attached to the shape information. Specifically, it is possible to apply certain rules to the conversion to CAD data based on data calculated from the image data of the workpiece W, such as adopting data as a component of CAD data only if the confidence level is above a predetermined value, or adopting data as a component of CAD data only if the validity flag is at a predetermined value. In this reverse engineering system 1, the CAD conversion unit 273 can grasp not only the shape information of the workpiece W, but also information individually attached to polygons and vertices included in the mesh data, based on predetermined rules. An example of the data structure of the dataset is shown in Figure 11.

[0100] For example, if there is an array setting table that defines the rule that vertex information is assigned as data 1, normal information as data 2, and confidence level as data 3, then by interpreting the dataset based on this, the data structure shown in Figure 11 can be reproduced. This array setting table is stored in the storage device 240. The arrangement setting table includes measurement conditions as additional data (first additional data) assigned to the mesh data, as well as vertex information that makes up the mesh data, and further includes additional data (second additional data) assigned to each piece of vertex information. The second additional data includes vertex confidence level, validity flag, texture information, etc. The arrangement setting table defines the array of the first additional data, vertex information, and second additional data.

[0101] The scanner output unit 264 can identify the arrangement of vertex information constituting the mesh data and the first and second additional data based on the arrangement setting table shown in Figure 11. The scanner output unit 264 outputs a dataset containing the vertex information constituting the mesh data and the first and second additional data arranged according to the identified arrangement to the conversion module 270.

[0102] The data input unit 271 receives the data set output from the scanner output unit 264, and the processing parameter determination unit 272 identifies the vertex information constituting the mesh data, as well as the first additional data and the second additional data, from the data set received by the data input unit 271, based on the array setting table shown in Figure 11. The processing parameter determination unit 272 determines the processing parameters for converting the mesh data to CAD data based on the first additional data identified based on the array setting table stored in the storage device 240. In other words, by storing the array setting table, the same rules can be shared between the scanner module 260 and the conversion module 270, so that the additional data output from the scanner module 260 can be interpreted by the conversion module 270. Furthermore, by determining the processing parameters for converting the mesh data to CAD data based on the first additional data, the parameters to be applied to the entire mesh data can be determined in advance, thereby shortening the processing time.

[0103] Furthermore, the processing parameter determination unit 272 can also determine whether or not to use each vertex information constituting the mesh data for conversion to CAD data, based on second additional data identified based on the array setting table stored in the storage device 240. That is, if the second additional data includes the reliability of the vertices, it can determine whether or not to use each vertex information for conversion to CAD data based on the reliability, and for example, if the reliability is below a predetermined level, the vertex information can be not used for conversion to CAD data.

[0104] In the array table shown in Figure 11, the second additional data may be data assigned to each polygon piece of information. In this case, the array table defines the arrangement of the first additional data, the polygon information, and the second additional data assigned to each polygon piece of information. The scanner output unit 264 identifies the arrangement of the polygon information and the first and second additional data based on the array setting table, and outputs a dataset containing the polygon information, the first and second additional data arranged according to the identified arrangement to the conversion module 270. The data input unit 271 accepts input of a dataset in which the arrangement of the polygon information and the first and second additional data has been identified. The processing parameter determination unit 272 then identifies the polygon information and the first and second additional data from the dataset received by the data input unit 271 based on the array setting table, and determines the processing parameters for converting the mesh data to CAD data based on the identified first additional data. In this case as well, based on the identified second piece of additional data, it may be decided whether or not to use the information of each polygon constituting the mesh data for conversion to CAD data.

[0105] More specifically, as shown in Figure 11, based on the reliability information attached to the shape information, the processing can be changed for each part of the shape, for example, by "determining the processing weight of each vertex / polygon based on the reliability r1 value (including cases where reliability = weight)" or by "using only polygons with valid flag f1 set to 1 for processing". In this case, a reliability-weight setting table is kept in advance. Furthermore, by keeping an array setting table of attached information in advance according to the model of the measurement unit 100, the CAD conversion unit 273 can grasp even if the data format differs for each model, and the conversion to CAD data is performed appropriately.

[0106] Furthermore, as shown in Figure 12, the measurement data may include a condition header consisting of the measurement mode, measurement range, measurement model ID, probe type (when the workpiece W is measured with a probe), etc.

[0107] In step SB18, the conversion module 270 generates a user interface screen 730 for displaying results, as shown in Figure 16, and displays it on the display unit 400. The user interface screen 730 for displaying results includes a mesh data display area 731 that displays an image of the workpiece W based on the mesh data, and a converted image display area 732 that displays an image of the workpiece W after surface conversion. The mesh data display area 731 and the converted image display area 732 are displayed simultaneously on the display unit 400 and are arranged side by side. This allows the user to easily compare the image before surface conversion with the image after surface conversion.

[0108] Step SB19 determines whether the surface conversion is complete. If the surface conversion is not complete, the process returns to step SB6 and restarts the surface conversion GUI. If the surface conversion is complete, the process proceeds to step SB20 to execute the surface conversion completion process. In step SB21, the CAD output unit 274 outputs the CAD data converted by the CAD conversion unit 273, and the output CAD data is saved to the storage device 240. The save operation is performed by operating the CAD file save button 733 on the results display user interface screen 730 shown in Figure 16. Also, if the settings file save button 734 on the results display user interface screen 730 is operated, the settings file is saved to the storage device 240.

[0109] The output CAD data can be imported into 3D CAD software and used for reverse engineering. For example, it is possible to create products based on the CAD data, or to create molds for molding products.

[0110] (Mesh data synthesis) The reverse engineering system 1 is configured to synthesize multiple mesh data sets with different resolutions. Specifically, the mesh data creation unit 263 creates a first mesh data set based on measurement data acquired at a first resolution, and a second mesh data set based on measurement data acquired at a second resolution, which is higher than the first resolution. Subsequently, the mesh data creation unit 263 synthesizes the first mesh data set with a relatively smaller resolution and the second mesh data set with a relatively larger resolution to create a combined mesh data set. The difference between the first and second resolutions can be set arbitrarily.

[0111] The processing parameter determination unit 272 identifies a first mesh data created from the composite mesh data created by the mesh data creation unit 263, based on measurement data acquired at a first resolution with a relatively small resolution, and determines the processing parameters for converting the identified first mesh data into CAD data based on a parameter table as shown in Figure 10.

[0112] Furthermore, the processing parameter determination unit 272 identifies a second mesh data created from the composite mesh data created by the mesh data creation unit 263, which is based on measurement data acquired at a second resolution with a relatively higher resolution. The unit then determines the processing parameters for converting the identified second mesh data into CAD data based on a parameter table as shown in Figure 10.

[0113] Subsequently, the CAD conversion unit 273 converts the composite mesh data into CAD data by applying the processing parameters determined by the processing parameter determination unit 272 to each of the first mesh data and the second mesh data created by the mesh data creation unit 263. Then, the CAD output unit 274 outputs the CAD data converted by the CAD conversion unit 273.

[0114] The mesh data creation unit 263 is configured to create composite mesh data in which resolution information is assigned as second additional data, indicating whether the measurement data for each vertex or polygon is based on the first or second resolution. In other words, there may be cases where the resolution at which each vertex or polygon is acquired is a relatively small first resolution, and cases where it is a relatively large second resolution. When both the first and second resolutions exist, it is necessary to be able to determine which resolution the measurement data for each vertex or polygon is based on, and in this case, creating composite mesh data with resolution information assigned makes this determination easier.

[0115] (Details of each module) Figure 17 is a block diagram showing the details of the analysis module 290. The data acquisition unit 291 acquires the mesh data created by the mesh data creation unit 263 (shown in Figure 5) of the scanner module 260. This is referred to as mesh data 1. Mesh data 1 is input to the element extraction unit 292, the setting unit 294, and the approximation processing unit 297, as well as to the first display control unit 198.

[0116] The element extraction unit 292 extracts multiple geometric elements such as circles and planes from the mesh data 1. The calculation unit 293 calculates dimensions using the multiple geometric elements extracted by the element extraction unit 292. The setting unit 294 allows the user to set the area (conversion area) that they want to convert to CAD data from the mesh data 1. Based on the conversion area set in the setting unit 294, the data extraction unit 295 extracts partially extracted mesh data 2. Mesh data 2 is data corresponding to the conversion area and has a smaller data size than mesh data 1. This mesh data 2 is input to the conversion data output unit 296. The mesh data 2 input to the conversion data output unit 296 is output to the conversion module 270. In other words, by outputting mesh data 2 corresponding to the conversion area that corresponds to a part of the workpiece W to the conversion module 270, only a portion of the data necessary for conversion to CAD data is passed to the conversion module 270, reducing the data size and consequently shortening the conversion time to CAD data.

[0117] For shapes that do not require precise geometry when converting to CAD data, the approximation processing unit 297 of the analysis module 290 can create approximated analysis surfaces (e.g., planes, cylinders, freeform surfaces, etc.). The set of created analysis surfaces is output from the conversion data output unit 296 to the conversion module 270.

[0118] The approximation processing unit 297 is configured to accept the specification of an approximation region to approximate a geometric element and the geometric element to approximate the approximation region. The approximation processing unit 297 approximates the mesh data included in the accepted approximation region with the specified geometric element.

[0119] The conversion module 270 receives mesh data and analysis surface data from the analysis module 290 and performs a conversion process to CAD data. While the conversion module 270 is performing the computationally intensive conversion process to CAD data, the analysis module 290 can continue processing. The analysis module 290 also has a first display control unit 298 for displaying the mesh data acquired by the data acquisition unit 291 on the display unit 400.

[0120] Figure 18 is a flowchart showing the processing procedure of the analysis module 290. In step SC1 after startup, mesh data 1 (shown in Figure 17) is acquired. In step SC2, the user specifies the processing content (e.g., measurement processing, CAD conversion processing). If measurement processing is specified, the process proceeds to step SC3 to execute the measurement processing. If CAD conversion processing is specified, the process proceeds to step SC4 to execute the CAD conversion processing. In step SC5, it is determined whether the analysis application has been terminated. If it has been terminated, this process is terminated; otherwise, the process returns to step SC2. The CAD data conversion process in step SC4 generates an analysis surface from the mesh data, and therefore the processing time is often considerably longer than the measurement processing in step SC3. After a CAD data conversion instruction is given, the analysis module 290 can execute measurement processing or another CAD data conversion instruction without waiting for the CAD data conversion process to complete.

[0121] Figure 19 is a flowchart showing an example of the measurement processing procedure. When measurement is started in step SC3 in Figure 18, the process proceeds to step SD1 in Figure 19, where the element extraction unit 292 shown in Figure 17 extracts multiple geometric elements (e.g., planes, cylinders, spheres, etc.) from the mesh data. Then, the process proceeds to step SD2, where the calculation unit 293 shown in Figure 17 calculates the dimensions between the multiple geometric elements extracted in step SD1. For example, if the geometric elements are two planes, the distance between the planes can be calculated, and if the geometric elements are a plane and a cylinder, the angle between the normal of the plane and the axis of the cylinder can be calculated.

[0122] An example of the geometric element extraction process in step SD1 of Figure 19 will be explained based on the flowchart shown in Figure 20. In step SE1 after the start, the selection of the type of geometric element is accepted. In step SE2, mesh data is selected. In step SE3, it is determined whether the confirmation button has been pressed. If the confirmation button has not been pressed, the process proceeds to step SE2; however, if the confirmation button has been pressed, the process proceeds to step SE4 for error checking. If an error is detected in step SE4, the process proceeds to step SE2. If there are no errors, the geometric element extraction process is terminated.

[0123] Figure 21 is a flowchart showing an example of the mesh selection process procedure in step SE2 of Figure 20. In step SF1 after the start, the selection method is specified, and the type of the specified selection method is determined. If the type of selection method is automatic shape selection, the process proceeds to step SF2, where the mesh is selected by the automatic shape selection process. For example, if the user double-clicks using the operation unit 250 on the 3D display screen, the range where the geometric shape selected by the area type continues will be automatically selected.

[0124] Furthermore, if the selection method is one that encloses an area with a polygon, the process proceeds to step SF3 to select the mesh by enclosing it with a polygon. For example, if the user operates the control unit 250 on the 3D display screen to specify a polygon, the area enclosed by the polygon becomes selected.

[0125] Furthermore, if the selection method is a brush painting process, the process proceeds to step SF4 to select the mesh by painting with a brush. For example, if the user moves the mouse while holding down the mouse button on the control unit 250 in the 3D display screen, the movement path becomes the selected state. Here, different brush thicknesses and shapes may be specified. Note that the selection method is not limited to those described above.

[0126] Figure 22 is a flowchart showing an example of the dimensional measurement processing procedure in step SD2 of Figure 19. In step SG1 after the start, the mode is specified and the specified mode is determined. If it is 3D measurement mode, proceed to step SG2; if it is geometric tolerance measurement mode, proceed to step SG3; if it is 3D comparison measurement mode, proceed to step SG4; if it is cross-sectional measurement mode, proceed to step SG5; if it is cross-sectional comparison measurement mode, proceed to step SG6; if it is thickness measurement mode, proceed to step SG7; and if it is planar measurement mode, proceed to step SG8. This is merely an example, and the types of measurement modes are not limited to those listed above.

[0127] Figure 23 shows the CAD conversion instruction procedure by the analysis module 290. In step SH1 after the start, it is determined whether or not a CAD conversion instruction has been given. If a CAD conversion instruction has been given, the process proceeds to step SH2 and the CAD conversion instruction is executed. If a CAD conversion instruction has not been given, step SH3 accepts the specification of the conversion method and determines the specified conversion method. If area specification is selected in step SH3, that is, if the mesh data is to be converted to CAD data that is faithful to the shape, the process proceeds to step SH4 and accepts the specification of the target for CAD conversion from the mesh data. In this case, the process proceeds to step SH6 and accepts mesh editing. On the other hand, if geometric element extraction is selected in step SH3, geometric elements are extracted from the mesh data. The geometric elements extracted here are extracted by approximating a part of the mesh data, and the geometric element extraction process can also be called an approximation process. In this case, the process proceeds to step SH5 and the mesh data approximation process is executed. For example, the geometric element extraction process in step SD1 of Figure 19 is executed, and the extracted geometric elements are approximated to the corresponding geometric shape. After that, in step SH7, they are registered as targets for CAD conversion. After going through step SH7, return to step SH1. Note that multiple conversion areas can also be registered.

[0128] Figure 24 is a flowchart showing the procedure for specifying the CAD conversion target in step SH4 of Figure 23. In step SI1 after the start, a user interface screen 700 for specifying the conversion area, such as the one shown in Figure 13, is displayed on the display unit 400, and the user determines whether or not there are any conversion areas remaining in the mesh data that need to be specified, based on the image displayed in the image display area 701. If there are no conversion areas remaining that need to be specified, the process proceeds to step SI2, where the mesh data extraction process is performed and then the process ends. On the other hand, if there are conversion areas remaining that need to be specified, the process proceeds to step SI3, where the method for selecting the conversion areas is accepted. The selection method can be any method, such as polygon selection, geometric shapes such as planes or cylinders, or color specification, and can be performed by operating the all-specify button 702a, first-area-specify button 702b, second-area-specify button 702c, third-area-specify button 702d, etc., as shown in Figure 13. The selected conversion area is displayed in the image display area 701 with a different color than before selection, making it easy for the user to understand which area has been designated as the conversion area.

[0129] At this time, a coordinate system for displaying the mesh data on the display unit 400 may be selected. That is, the setting unit 294 shown in Figure 5 is configured to be able to set the coordinate system included in the display conditions for displaying the mesh data on the display unit 400. The setting of the display conditions when displaying the mesh data on the display unit 400 is accepted by the setting unit 294 which constitutes the display condition setting unit. The display conditions set by the setting unit 294 are stored in the storage device 240. Based on the display conditions stored in the storage device 240, the first display control unit 298 can reproduce the display state set by the setting unit 294 and display it on the display unit 400 by specifying the coordinate system for displaying the mesh data newly acquired by the data acquisition unit 291 on the display unit 400.

[0130] As an example of how to set display conditions, the user interface screen 700 for specifying the conversion area is provided with a coordinate system selection area 703. The coordinate system selection area 703 is composed of, for example, a pull-down menu, and allows the user to select from among the device coordinate system, the coordinate system of the base or head that constitutes the workpiece W, and the coordinate system of the smallest rectangular parallelepiped circumscribed around the workpiece W (circumscribed smallest rectangular parallelepiped coordinate system). Once the selected coordinate system is set in the setting unit 294, the first display control unit 298 assigns the set coordinate system to the mesh data as coordinates for display and displays the mesh data on the display unit 400.

[0131] Furthermore, the first display control unit 298 displays on the display unit 400 a mesh data display area for displaying mesh data (for example, the image display area 701 shown in Figure 13), a measurement setting area (the user interface screen 740 for creating analysis curves shown in Figure 27) that accepts the specification of the shape of the geometric elements extracted by the element extraction unit 292 and the specification of the method for calculating the dimensions calculated by the calculation unit 293, a conversion area specification area (the area where buttons 702a to 702e shown in Figure 13 are displayed) that accepts the specification of a conversion area for executing CAD conversion set by the setting unit 294, and an output instruction icon (the output instruction button 704 shown in Figure 13) that accepts instructions for outputting mesh data to the conversion module 270.

[0132] The conversion data output unit 296 creates an output file that associates the mesh data with the coordinate system set in the setting unit 294. The conversion data output unit 296 outputs the created output file to the conversion module 270.

[0133] The conversion module 270 has a second display control unit 276 for displaying the mesh data received as input by the data input unit 271 on the display unit 400. The data input unit 271 is configured to be able to read the output file output by the conversion data output unit 296. The second display control unit 276 is configured to reproduce the display state of the mesh data in the analysis module 290 on the display unit 400 by displaying the mesh data contained in the output file on the display unit 400 based on the coordinate system contained in the output file.

[0134] The second display control unit 276 can display on the display unit 400 a CAD conversion target data display area (for example, the image display area 711 shown in Figure 14) that displays mesh data received by the data input unit 271, and a conversion instruction icon (the conversion instruction button 714 shown in Figure 14) that accepts a conversion instruction to convert the mesh data displayed in the CAD conversion target data display area into CAD data.

[0135] When the second display control unit 276 detects that the output instruction button 704 shown in Figure 13 has been operated, it generates a first conversion window (corresponding to the surface conversion user interface screen 710 shown in Figure 14) which includes a CAD conversion target data display area and a conversion instruction icon, and displays it on the display unit 400. When the second display control unit 276 detects that the output instruction button 704 has been operated, it displays a second conversion window which includes a CAD conversion target data display area and a conversion instruction icon on the display unit 400 in addition to the first conversion window. In other words, if the output instruction of the analysis module 290 is operated multiple times, the conversion module 270 will be activated the same number of times.

[0136] The conversion data output unit 296 may be configured to perform a coordinate transformation on the mesh data based on the coordinate system set in the setting unit 294, and output the transformed mesh data to the conversion module 270. In this case, the data input unit 271 is configured to read the transformed mesh data output by the conversion data output unit 296. The second display control unit 276 is configured to reproduce the display state of the mesh data in the analysis module on the display unit 400 by displaying the transformed mesh data on the display unit 400.

[0137] When converting mesh data directly to CAD data, you can select the mesh data of the area you want to convert using a mouse or other means. Note that partially selected mesh data will have gaps at the boundaries of the selected area. Since CAD data should ideally be complete, you may want to perform mesh editing (e.g., filling holes, deleting, healing) at this point.

[0138] Step SI4 performs mesh selection using the selection method specified in Step SI3. Step SI5 determines whether the confirm button has been pressed; if not, proceeds to Step SI4, while if not, proceeds to Step SI6. Step SI6 performs error checking. If an error is detected in Step SI7, proceeds to Step SI4. If there are no errors, the selected mesh is registered. Once this process is complete and the user executes a conversion command, the conversion module 270 starts and the CAD data conversion process begins.

[0139] Figure 25 is a flowchart of the mesh editing procedure. After starting, step SJ1 determines whether or not mesh editing is complete. If mesh editing is complete, this process ends; otherwise, the process proceeds to step SJ2. In step SJ2, the type of mesh editing is specified, and it is determined whether the type is deletion or hole filling. If it is deletion, the process proceeds to step SJ3, where the desired mesh is selected by the operation unit 250 and the deletion process is executed. If it is hole filling, the process proceeds to step SJ4, where the missing open edges of the mesh are selected and the hole filling process is performed. In step SJ5, mesh healing is performed, and then the process returns to step SJ1.

[0140] Mesh healing is an automatic process performed on mesh data that has undergone mesh deletion in step SJ3 or hole filling in step SJ4. If an invalid mesh is generated as a result of mesh editing, it will result in a conversion error during CAD data conversion, so this healing process is performed before CAD data conversion. Specifically, the healing process is a process to repair invalid meshes, such as self-intersection removal, non-manifold mesh removal, and minuscule mesh removal. Self-intersection refers to meshes that intersect each other in space without any connection relationship. A non-manifold mesh is a mesh in which three or more polygons share one edge. A minuscule mesh is, for example, a mesh with a small total number of meshes in polygon data with connection relationships (e.g., 100 polygons or less).

[0141] Figure 26 is a flowchart showing the procedure for converting to CAD data. In step SK1 after the start, it is determined whether the data to be converted is mesh data or an analytical surface. If it is mesh data, the process proceeds to step SK2, where the CAD conversion unit 273 converts the mesh data of the conversion area set in the setting unit 294 into CAD data as freeform surface data. If it is an analytical surface, the process proceeds to step SK3, where the CAD conversion unit 273 converts it into a data format that can be handled by CAD software, such as a plane, cylinder, or curved surface. In other words, the CAD conversion unit 273 can convert the mesh data included in the geometric elements extracted in step SH5 of Figure 23 into CAD data as surface data of an analytical surface.

[0142] In step SK4, the CAD data converted in step SK2 and the CAD data converted in step SK3 are set as output targets. In step SK5, it is determined whether all conversion targets have been converted. If there are still targets to be converted, the process returns to step SK1; otherwise, it proceeds to step SK6. In step SK6, if there are multiple output targets set in step SK4, the multiple CAD data are merged into one.

[0143] Step SK6 accepts two types of data formats for combining: part files and assembly files. A part file is a file where all models are treated as a single rigid body. An assembly file is a file where all models are treated as separate models. With an assembly file, individual models can be moved separately within the CAD software.

[0144] The details of creating the analysis surface will be explained in detail. The analysis module 290 generates the user interface screen 740 for creating the analysis surface shown in Figure 27 and displays it on the display unit 400. The user interface screen 740 for creating the analysis surface is provided with an element display area 741 where selectable basic measurement elements are displayed. Basic measurement elements include, for example, a plane, cylinder, cone, sphere, and point. The user interface screen 740 for creating the analysis surface can also display selectable special measurement elements.

[0145] Figure 28 shows the procedure for creating an analytical surface, which can be performed by the approximation processing unit 297 shown in Figure 17. In Figure 28, the symbol 750 indicates the user interface screen displayed on the display unit 400. In this example, a workpiece W having two cylindrical sections is displayed. When the user clicks point P1 on one of the cylindrical sections with the mouse on the operation unit 250, the cylindrical section containing point P1 is selected and its color changes (the color-changed area is indicated by diagonal lines). This operation specifies the approximation region to approximate the geometric element and is accepted by the approximation processing unit 297. Subsequently, when the user clicks point P2 on the other cylindrical section with the mouse on the operation unit 250, the cylindrical section containing point P2 is selected and its color changes. This operation is also accepted by the approximation processing unit 297. The approximation processing unit 297 also accepts the specification of a cylinder as the geometric element to approximate the approximation region. After that, by operating the confirmation button, the state in which the two cylindrical sections are selected is confirmed, and the selected parts are approximated by cylinders. The approximate geometric shapes are not limited to cylinders; for example, it is also possible to approximate them with planes, spheres, cones, etc.

[0146] By displaying a window like the one shown in Figure 29 on the display unit 400, it becomes possible to select and display 3D elements and target elements individually. In other words, the conversion data output unit 296 outputs to the conversion module 270 a mixture of the mesh data of the conversion region set in the setting unit 294 and the geometric element data created by approximating the mesh data included in the approximation region with geometric elements.

[0147] In this example, after selecting the conversion area, the recipe can be saved and reproduced. When saving the polygon selection command history, all 3D operations performed by the operation unit 250 up to the polygon selection are saved. When reproducing the polygon selection command history, the same polygon selection command can be executed on new mesh data (mesh data with some modifications) to select mesh data located in almost the same position. In other words, by reproducing the operations performed by the user during mesh extraction based on the line of sight information and polygon commands saved in the recipe, mesh data can be extracted accurately.

[0148] Furthermore, it is possible to save selected polygons and select new polygons that are a certain distance from the selected polygons. In this case, saving saves the selected polygons as they are, taking the coordinate system into consideration. Reproduction involves loading the selected polygons into the same coordinate system as the new polygon data (partially modified polygon data), and selecting the region of the new polygon data that is a certain distance from the selected polygons, thereby selecting mesh data that is in approximately the same position. In other words, it is possible to extract nearby regions from previously extracted mesh data. For example, it is possible to expand previously extracted mesh data by a predetermined amount and extract the mesh data contained within the expanded region. Note that the pose of the past mesh data and the pose of the newly acquired mesh data are approximately the same, but they do not exactly match, so nearby regions can be identified by expanding the previously extracted mesh data.

[0149] The recipes for saving and reproducing the polygon selection command history include the following as coordinate systems (transformation matrices relative to the measurement data coordinate system): reference plane, planar elements (planes, parallel planes, etc.), reference axis, axis elements (cylinders, cones, surface intersection lines, etc.), origin, and point elements (points, intersections of faces and axes, etc.). The recipes for saving and reproducing the polygon selection command history also include the following as 3D line of sight information: camera (viewpoint) position (x, y, z), camera orientation (roll, pitch, yaw), and line of sight direction vector. The recipes for saving and reproducing the polygon selection command history also include the following as 3D view volume information: projection transformation method (perspective projection or parallel projection or...), and viewport settings (left, right, bottom, top, near, far). The recipes for saving and reproducing the polygon selection command history include the following as part of the polygon selection command history: Polygon Selection 1 (double-click selection), double-click coordinates on the projection plane (x, y); Polygon Selection 2 (line division), coordinates of the first point on the projection plane (x, y), first point of the line segment, coordinates of the second point on the projection plane (x, y), second point of the line segment, coordinates of the third point on the projection plane (x, y), selected area; Polygon Selection 3 (inside polygon), coordinates of the first point on the projection plane (x, y), coordinates of the second point on the projection plane (x, y), etc.

[0150] The recipe for saving selected polygons and selecting new polygons within a certain distance from selected polygons includes the coordinate system (transformation matrix for the measurement data coordinate system), which includes the reference plane, planar elements (planes, parallel planes, etc.), reference axis, axis elements (cylinders, cones, surface intersection lines, etc.), origin, and point elements (points, intersections of surfaces and axes, etc.). The recipe for saving selected polygons and selecting new polygons within a certain distance from selected polygons also includes the selected polygons, search range (distance), etc. Alternatively, the recipe may not be saved, and user input may be accepted during reproduction. In addition to the history of polygon selection commands, the history of mesh preprocessing may also be included in the recipe.

[0151] (Multiple conversion modules) Figure 30 shows a configuration with multiple conversion modules. That is, the reverse engineering system 1 may have a first conversion module 270C and a second conversion module 270D. The setting unit 294 (shown in Figure 5) sets a first conversion region for performing a first CAD conversion and a second conversion region for performing a second CAD conversion from the mesh data acquired by the data acquisition unit 291. The data extraction unit 295 is configured to extract first mesh data based on the first conversion region set by the setting unit 294, and is also configured to extract second mesh data based on the second conversion region set by the setting unit 294. The conversion data output unit 296 outputs the first mesh data extracted by the data extraction unit 295 to the first conversion module 270C and outputs the second mesh data extracted by the data extraction unit 295 to the second conversion module 270D.

[0152] For example, the first conversion module 270C converts the first mesh data (data constituting the cylindrical part in Figure 11) into the first CAD data. Meanwhile, the second conversion module 270D converts the second mesh data (data constituting the cubic part in Figure 11) into the second CAD data, regardless of the operating status of the first conversion module 270C. In other words, even while the first conversion module 270C is performing the CAD data conversion process, the second conversion module 270D can convert other mesh data into CAD data, allowing the first and second mesh data to be converted into CAD data in parallel, thus reducing processing time. While the conversion to CAD data is in progress, a window indicating that processing is in progress may be generated and displayed on the display unit 400. Alternatively, a cancel button may be provided to allow the conversion process to be canceled during the conversion to CAD data, and the conversion process to CAD data may be canceled by operating the cancel button.

[0153] (Function to reflect design intent) The reverse engineering system 1 according to this embodiment has a function that allows the design intent to be actively reflected when creating surface data. To explain the overall overview, as shown in Figure 31, first, the area specification user interface is activated and the mesh data is displayed in 3D on the display unit 400. On the 3D display, the user sets an arbitrary area and the shape attributes (information of the analysis surface) of that area. That is, the shape attribute setting unit 277 shown in Figure 5 sets shape attributes for the polygons included in the mesh data acquired by the data acquisition unit 291. Shape attributes can be input by the user. User input of shape attributes is received by the input reception unit 271a.

[0154] Furthermore, the region identification unit 278 shown in Figure 5 identifies regions included in the mesh data acquired by the data acquisition unit 291 based on the shape attributes set by the shape attribute setting unit 277. User operations for identifying regions by the region identification unit 278 are received by the input reception unit 271a. When the input reception unit 271a receives user operations for identifying regions, the region identification unit 278 identifies the regions based on the user operations received by the input reception unit 271a. The input reception unit 271a is also configured to accept the specification of shape attributes for the regions identified by the region identification unit 278.

[0155] A region is a collection of polygons that are part of a single area of ​​mesh data (polygon data) where mathematical parameters such as the direction and curvature of the normals are similar. In the following explanation, the area identified by the region identification unit 278 corresponds to a region.

[0156] Furthermore, as will be described in detail later, the data conversion unit 279 shown in Figure 5 converts polygons included in the mesh data acquired by the data acquisition unit 291, specifically those included in the regions identified by the region identification unit 278, into surface data based on the shape attributes set by the shape attribute setting unit 277. Here, the data conversion unit 279 may also convert the polygons included in the regions identified by the region identification unit 278 into surface data of the analyzed curved surface based on the shape attributes set by the shape attribute setting unit 277.

[0157] Furthermore, the data conversion unit 279 converts polygons that are not included in the region identified by the region identification unit 278, from among the multiple polygons included in the mesh data acquired by the data acquisition unit 291, into freeform surface data. The CAD output unit 274, acting as a data output unit, outputs the surface data converted by the data conversion unit 279 based on the shape attributes and the freeform surface data converted by the data conversion unit as CAD data. Here, when the data conversion unit 279 converts polygons included in the region identified by the region identification unit 278 into analytical surface data, it can output data containing a mixture of analytical surface data and freeform surface data as CAD data.

[0158] Surface data for freeform surfaces is difficult to position on design drawings and has low machinability for measurement data, but it has high conformability to measurement data. On the other hand, surface data for analytical surfaces allows for axis acquisition, surface drafting, and the application of constraints during positioning, resulting in high machinability for measurement data, but because it is mathematically represented data, it has low conformability to measurement data. By mixing these two types of surface data, it is possible to output freeform surfaces in areas where conformability to measurement data is important, and analytical surfaces in areas where machinability of measurement data is important, thereby achieving a balance between machinability and conformability.

[0159] The mapping unit 271b shown in Figure 5 is responsible for creating mapping data that maps the shape attributes of a region identified by the region identification unit 278 to the polygons contained within that region. When the mapping unit 271b creates mapping data, it is stored in the storage device 240. The data conversion unit 279 reads the mapping data from the storage device 240 when converting to surface data. After reading the mapping data, the data conversion unit 279 identifies multiple polygons that have been assigned a single shape attribute based on the mapping data stored in the storage device 240, using the mesh data acquired by the data acquisition unit 291. The data conversion unit 279 can then convert the identified multiple polygons into a single surface data based on the shape attributes associated with those polygons.

[0160] The region identification unit 278 can also identify multiple regions. That is, the region identification unit 278 can be configured to sequentially identify a first region and a second region from the mesh data acquired by the data acquisition unit 291. In this case, the mapping unit 271b associates a first shape attribute with the polygons included in the first region identified by the region identification unit 278 and determines whether or not there is an overlapping area between the first region and the second region. If the mapping unit 271b determines that there is an overlapping area between the first region and the second region, it can change the first shape attribute already associated with the overlapping area to the second shape attribute by associating a second shape attribute with the overlapping area.

[0161] Furthermore, if the mapping unit 271b determines that there is an overlapping area between the first region and the second region, it can also maintain the first shape attribute that has already been mapped to the overlapping area by mapping the second shape attribute to the area of ​​the second region that is not the overlapping area.

[0162] When multiple shape attributes are associated with a region, the data conversion unit 279 generates surface data corresponding to each of the multiple shape attributes. In this case, multiple surface data will exist for overlapping areas, resulting in invalid data. When there is an overlapping area between the first and second regions, associating the first shape attribute with the overlapping area prevents the generation of invalid data and improves the practicality of defining higher-priority shapes over a wider area. Furthermore, when there is an overlapping area between the first and second regions, associating the second shape attribute with the overlapping area prevents the generation of invalid data and improves the user experience by allowing easy modification of the region's range later on.

[0163] The segment processing unit 271c shown in Figure 5 is responsible for dividing the mesh data acquired by the data acquisition unit 291 into multiple segments. First, the segment processing unit 271c acquires the mesh data acquired by the data acquisition unit 291 and calculates geometric characteristic values ​​of the polygons, such as polygon area, edge length, curvature, and normal vector. Then, it generates segments based on the calculated geometric characteristic values ​​of the polygons.

[0164] At this time, the segment processing unit 271c determines whether or not shape attributes are associated with multiple polygons included in the mesh data acquired by the data acquisition unit 291. The segment processing unit 271c is configured to segment the mesh data so that geometric elements corresponding to the shape attributes are extracted from the multiple polygons that have been identified as having associated shape attributes, and to segment the multiple polygons that have been identified as not having associated shape attributes into multiple segments based on the geometric characteristic values ​​of each polygon. In this case, the data conversion unit 279 converts each segment segmented by the segment processing unit 271c into surface data.

[0165] Once the user has specified the region, the conversion module 270 is activated. For regions not specified by the user, it automatically generates subdivision regions of the curved surface and combines them with the user-specified regions to generate segment information, which is a set of regions that cover all of the mesh data. The segment information contains a list of regions and information on the shape attributes of the regions set in the shape attribute setting unit 277.

[0166] In the surface generation process, the surface and trim lines of individual regions are calculated from mesh data and segment information to generate trimmed surfaces. Topological information, which is the adjacency information between trimmed surfaces, is also generated. In the CAD data generation process, Brep (Boundary Representation) data is generated from the trimmed surfaces, their shape attributes, and topology information. A unique CAD file (STEP file) is generated from the Brep data.

[0167] The confirmation display user interface displays the Brep data in 3D for final confirmation. If the generated Brep data is as intended, a CAD file is generated and the process ends. If the Brep data is not as intended, the user returns to the area selection user interface to restart the process.

[0168] The following provides a concrete example. Figure 32 is a flowchart showing the procedure for generating CAD data that reflects the shape attributes set in the shape attribute setting unit 277. Step SL1 is the step of reading the saved area when starting by reading a saved area. Step SL2 determines whether or not to specify a user area. If a user area is specified, the process proceeds to step SL3 to specify the user area, and then to step SL4. Specifying a user area is a user operation to identify a region and is accepted by the input reception unit 271a.

[0169] If no user area is specified, the process proceeds directly to step SL4. In step SL4, it is determined whether or not to save the user-specified area (user-specified area) to the storage device 240, etc. If the user-specified area is to be saved to the storage device 240, etc., the process proceeds to step SL5, where the user-specified area is saved to the storage device 240, etc., and then the process proceeds to step SL6. If the user-specified area is not to be saved to the storage device 240, etc., the process proceeds directly to step SL6.

[0170] Step SL6 performs automatic region generation. For example, the region identification unit 278 identifies a region based on the user region specification operation received by the input reception unit 271a. At this time, if the input reception unit 271a has received an input operation for shape attributes by the user, the region is identified based on the shape attributes entered by the user.

[0171] There are several methods for automatic region generation. One method involves generating regions independently of user-specified regions, which are then combined with the user-specified regions. Another method involves marking user-specified regions as specified regions and then performing automatic region generation on the remaining regions. Shape attributes may also be automatically set by the shape attribute setting unit 277 during automatic region generation. In step SL7, the automatically generated regions and user-specified regions are combined. Step SL7 is unnecessary in the cases of the other methods described above.

[0172] Step SL8 generates a surface for each region. Step SL9 performs boundary healing and interpolation on the boundaries between adjacent surfaces. Step SL10 generates a Brep structure, and Step SL11 outputs a CAD file. Step SL12 saves the user-specified region to the storage device 240, etc.

[0173] Figure 33 is a flowchart showing the procedure for specifying a user region. In step SM1 after the start, it is determined whether there are any regions that need to be specified. If there are no regions that need to be specified, the process ends. If there are regions that need to be specified, the process proceeds to step SM2, where mesh selection is performed. In step SM3, it is determined whether the confirm button has been pressed. If the confirm button has not been pressed, the process returns to step SM2; if the confirm button has been pressed, the process proceeds to step SM4. In step SM4, an error check is performed. If there is an error, the process returns to step SM2; if there is no error, the process proceeds to step SM5. In step SM5, the region type and the selected mesh are registered.

[0174] Figure 34 is a flowchart showing the procedure for specifying the user area when a preview display is available. Steps SN1 and SN2 are the same as steps SM1 and SM2 in Figure 33. In step SN3, it is determined whether or not the preview button has been pressed. If the preview button has not been pressed, the process returns to step SN2. If the preview button has been pressed, the process proceeds to step SN4, where a preview screen that can be viewed by the user is generated and displayed on the display unit 400. Steps SN5 to SN7 are the same as steps SM3 to SM5 in Figure 33.

[0175] Figure 35 is a flowchart detailing step SN4 of Figure 34. In step SP1, a curved surface is calculated from the mesh selected by the user according to the type of region. In step SP2, the curved surface calculated in step SP1 and the mesh selected by the user are superimposed and displayed on the display unit 400. The curved surface calculated in step SP1 and the mesh selected by the user are displayed in a distinguishable form. A distinguishable form is a form in which the curved surface calculated in step SP1 and the mesh selected by the user are displayed in different colors, or a form in which one is made semi-transparent so that the other is visible, etc.

[0176] Figure 36 is a flowchart showing the procedure for mesh selection processing. Step SQ1 determines whether or not to perform automatic geometric shape determination. Automatic geometric shape determination means that the type of geometric shape (plane, cylinder, cone, sphere, torus, or freeform surface) is automatically determined based on the mesh shape of the area specified by the user and set as an attribute. For example, if any of the plane button, cylinder button, cone button, or sphere button in the detailed setting area 712 of the surface conversion user interface screen 710 shown in Figure 14 is operated, the type of geometric shape will be set to the geometric shape specified by the selected button.

[0177] When a button in the detailed settings area 712 is operated, the area selection user interface screen 760 shown in Figure 38 is generated and displayed on the display unit 400. The area selection user interface screen 760 includes an image display area 761 that displays an image of the workpiece W based on mesh data, and a detailed settings area 762. The detailed settings area 762 also displays the set geometric elements, and in this example, a plane is set. When the user double-clicks on the area they want to specify using the operation unit 250 on the image display area 761, the shape currently being selected is automatically extracted and selected. If a plane is set, an area that can be considered a plane continuously from the double-clicked location is automatically selected as a plane. When specifying by enclosing with the mouse, the shape enclosed by the mouse pointer is selected.

[0178] If automatic determination is not performed, the process proceeds to step SQ2 to select the geometric shape, and then to step SQ3 to perform the mesh selection operation. The mesh selection operation is as described in the flowchart shown in Figure 21. User operations for identifying a region by the region identification unit 278 include, for example, specifying a point or specifying an area to enclose, and these operations are received by the input reception unit 271a. When a point is specified, the region identification unit 278 identifies the region based on the characteristics of the surface containing the point received by the input reception unit 271a. When an area to enclose is specified, the region identification unit 278 identifies the polygons included in the area to enclose as a region. When a point or a specified area is specified as a user operation for identifying a region by the region identification unit 278, the storage device 240 stores the coordinate information of the point or specified area in association with the display conditions for displaying the mesh data on the display unit 400. As described above, the second display control unit 276 can reproduce the display state of the mesh data in the analysis module 290 on the display unit 400. The region identification unit 278 can also identify a region based on the mesh data whose display state has been reproduced and the coordinate information stored in the storage device 240. Subsequently, in step SQ4, attribute settings other than geometric shape (e.g., color specification) are performed.

[0179] If automatic detection is enabled, proceed to step SQ5 to perform the mesh selection operation. In step SQ6, automatic geometric shape detection is performed, and then in step SQ7, attribute settings other than geometric shape (e.g., color specification) are performed.

[0180] Figure 37 is a flowchart showing the procedure for combining user-specified regions and automatically generated regions. The numbers in each step correspond to the numbers in the figure on the right. In step SR1, the region IDs of the user-specified regions are reassigned so as not to overlap with the region IDs of the automatically generated regions. In step SR2, the user-specified regions are overwritten with the automatically generated regions. In step SR3, independent region IDs are assigned to the regions that have been divided and become detached by the user-specified regions. In this way, the combined region can be obtained.

[0181] In the case of a boundary between freeform surfaces, first, it is determined whether healing of the boundary connection is necessary. That is, points are placed at regular intervals on the boundary curve. The average and maximum distances between these points and each surface are evaluated, and if they are greater than a certain threshold, it is determined that healing is necessary, and both surfaces are regenerated. In this process, the point cloud used to fit the surfaces uses the point cloud belonging to the region of the surfaces and the point cloud generated from the boundary curve. Also, the weight of the point cloud derived from the boundary curve is made greater than that of the point cloud derived from the region in the calculation. Next, it is determined again whether healing is necessary using the method described above. If the criteria are not met, the weight of the point cloud derived from the boundary is increased and the calculation is performed again.

[0182] In the case of inter-primitive boundaries, first, the boundary line is projected onto the primitive to create a new boundary line. Alternatively, the original vertices are projected onto the primitive before calculation when generating the boundary line. Then, it is determined whether healing of the boundary connection is necessary. That is, points are placed at regular intervals along the boundary curve. The average and maximum distances between these points and the freeform surface are evaluated, and if they are greater than a certain value, it is determined that healing is necessary, and the freeform surface is regenerated. In this case, the point cloud used to fit the surface uses the point cloud belonging to the region of the surface and the point cloud generated from the boundary curve. Also, the weight of the point cloud derived from the boundary curve is made larger than that of the point cloud derived from the region in the calculation. Next, it is determined again whether healing is necessary using the method described above. If the criteria are not met, the weight of the point cloud derived from the boundary line is increased and the calculation is performed again.

[0183] There may be cases where the user-specified region and the user-specified analysis surface are inconsistent. For example, if the user-specified region was a plane but specified a cylindrical surface, or if a stepped surface was combined into a single region and specified as a plane, an error message should be displayed during region specification to prompt the user to re-specify the region.

[0184] Furthermore, even if the domain specification and the analytical surface are consistent during domain specification, problems may arise when the surface is actually generated. For example, if there is a gap between adjacent freeform surfaces, or between freeform surfaces and analytical surfaces, or if they intersect, a healing process is performed to resolve the problem by deforming the freeform surface.

[0185] When analytical surfaces are adjacent, healing is possible with freeform surfaces by deforming the surface to connect adjacent regions with a boundary line, but this is not possible with analytical surfaces. If the analytical surfaces are continuous within the tolerance range, there is no problem, but if they are not continuous, an error will occur. If an error occurs, check the problem in the confirmation user interface and resolve it by re-specifying the region by inserting a freeform surface between the analytical surfaces.

[0186] For example, if a user performs an operation to identify a region by enclosing a predetermined area, the region identification unit 278 calculates an approximate plane based on the multiple polygons included in the enclosed predetermined area. The region identification unit 278 also obtains the normal of the calculated approximate plane and the positions and normals of the polygons included in the predetermined area. Based on the obtained normal of the approximate plane and the positions and normals of the polygons included in the predetermined area, the region identification unit 278 can also determine whether the predetermined area can be approximated as a plane.

[0187] The shape attribute setting unit 277 can also calculate the radius of curvature of a polygon based on the polygon's curvature. Based on the calculated radius of curvature of the polygon and the direction of the polygon's normal, the shape attribute setting unit 277 can calculate the center point of the curvature circle of the polygon, and set shape attributes for the polygons included in the mesh data acquired by the data acquisition unit 291 based on the positional relationship of the center points of the curvature circles calculated for multiple polygons constituting the mesh data.

[0188] Error handling is also possible when specifying geometric shapes. Specifically, for regions where a geometric shape is set as an attribute, such as a plane or cylindrical surface, it is determined whether the shape of the region selected by the user matches the attribute. For example, assuming that the region specified by the user is a geometric shape set as an attribute, the analytical surface of the geometric shape is estimated using the least squares method or similar. After estimating the analytical surface, the shortest distance between the estimated analytical surface and each vertex of the region specified by the user is calculated. After calculating the shortest distance, the maximum and average values ​​of these shortest distances are calculated, and an error is generated if the maximum or average value exceeds a predetermined threshold. Before calculating the maximum and average values, preprocessing such as removing the top 0.5% of the shortest distances may be performed to remove outliers. Outlier removal can prevent situations where region specification becomes difficult due to shape noise.

[0189] Error handling is also possible when specifying a freeform surface. Specifically, since it is difficult to generate a region with an overhang as a single surface for a freeform surface, the presence or absence of an overhang is determined. For example, principal component analysis is performed on the vertex coordinates of the user-specified region to find the direction of the first principal component. After finding the direction of the first principal component, the angle that each normal vector of each vertex in the user-specified region makes with the direction of the first principal component is calculated. If any of these angles exceed a predetermined threshold, an error is generated. The predetermined threshold is, for example, around 60° to 80°. For freeform surfaces, if an overhang is detected, there are two methods: having the user re-specify the region, or automatically dividing the region to eliminate the overhang. For the latter, for example, after finding the vertex with the maximum principal curvature within the user-specified region, the closest vertex is searched for in both the positive and negative directions of the principal curvature vector from that vertex. For each found vertex, the process of searching for the closest vertex in both the positive and negative directions of the principal curvature vector is repeated until the edge of the region is reached. This results in a sequence of vertices that divides the region. The region is divided into two parts based on the vertex sequence, overhangs are detected in each region, and if overhangs are found, the region is recursively divided using the method described above.

[0190] (Adding supplementary information to surface data) This reverse engineering system 1 is configured to allow additional information, not directly related to the three-dimensional shape, to be added to the surface data. This additional information represents the surface characteristics of the workpiece W, and includes, for example, color information indicating the color of the workpiece W's surface, reflection information indicating the degree of light reflection on the workpiece W's surface, and diffusion information indicating the degree of light diffusion on the workpiece W's surface.

[0191] As explained based on the block diagram shown in Figure 5, the input receiving unit 271a is configured to accept the specification of color information for each shape attribute as additional information. The additional information identification unit 271d is the part that identifies the additional information to be added to the surface data converted by the data conversion unit 279. For example, the additional information identification unit 271d can identify one piece of additional information to be assigned to a region based on the additional information associated with multiple polygons included in the region identified by the region identification unit 278. Examples of the one piece of additional information identified here include the representative color and representative brightness of the multiple polygons included in the region.

[0192] The additional information assignment unit 271e is the part that assigns additional information, as determined by the additional information identification unit 271d, to the surface data converted by the data conversion unit 279. When the additional information identification unit 271d determines which additional information to assign to a region, the additional information assignment unit 271e assigns the determined additional information to the region. Once the additional information is assigned to the surface data, the CAD output unit 274 outputs CAD data with the additional information assigned to the surface data. When color information is assigned as additional information, the additional information identification unit 271d determines a representative color to assign to the region based on the color information associated with the multiple polygons included in the region as additional information. The additional information assignment unit 271e then assigns the determined representative color to the region.

[0193] Furthermore, the data conversion unit 279 is configured to determine whether or not shape attributes are assigned to multiple polygons included in the mesh data acquired by the data acquisition unit 291, and then convert the multiple polygons identified as having shape attributes assigned into surface data of an analyzed curved surface based on the shape attributes, while converting the multiple polygons identified as not having shape attributes assigned into surface data of a free-form curved surface. When the conversion to surface data of an analyzed curved surface and surface data of a free-form curved surface is performed, the additional information identification unit 271d identifies color information specified for each shape attribute as additional information to be added to the surface data of the analyzed curved surface, while identifying predetermined color information as additional information to be added to the surface data of a free-form curved surface.

[0194] The mapping unit 271b is configured to associate polygons included in the region identified by the region identification unit 278 with additional information identified by the additional information identification unit 271d. The associated polygons and additional information can also be stored in the storage device 240 or the like. Furthermore, the mapping unit 271b is configured to associate polygons included in the region identified by the region identification unit 278 with additional information identified by the additional information identification unit 271d with shape attributes received by the input receiving unit 271a. The associated polygons, additional information, and shape attributes can also be stored in the storage device 240 or the like.

[0195] Furthermore, when the input receiving unit 271a receives input of color information for each shape attribute as additional information, the mapping unit 271b is configured to associate the color information corresponding to the shape attribute of a region with the polygons contained in the region, based on the received shape attribute and the color information for each shape attribute, thereby enabling the same color information to be associated with regions with the same shape attribute.

[0196] The procedure for adding information to surface data will be explained below based on the flowchart shown in Figure 39. In step SR1 after the start, the conversion module 270 is activated. In step SR2, the conversion module 270 generates a user interface screen 710 for surface conversion as a region specification GUI, for example as shown in Figure 14, and displays it on the display unit 400. In step SR3, the user selects an attribute, and the input receiving unit 271a accepts the selected attribute. The detailed setting area 712 of the user interface screen 710 for surface conversion is provided with a shape type selection area 712d, and by operating this shape type selection area 712d, it is possible to specify geometric shapes such as planes and cylinders. Also in step SR3, the user specifies a color, and the input receiving unit 271a accepts the color specification. When accepting a color specification, the conversion module 270 generates a color specification window 770 as shown in Figure 40 and displays it on the display unit 400. The color selection window 770 displays multiple color samples with different hues and saturations, allowing the user to select their desired color.

[0197] In step SR4, the user specifies a region based on the shape specified in step SR3. In step SR5, it is determined whether or not to change the attributes. If the user changes the attributes, the process proceeds to step SR6; otherwise, it proceeds to step SR7. In step SR6, the user re-selects the attributes. Step SR6 can be performed using the same procedure as in step SR3.

[0198] Step SR7 determines whether the setup is complete or not. If it is not complete, the process returns to step SR3; if it is complete, it proceeds to step SR8. In step SR8, the segment processing unit 271c divides the mesh data into multiple segments. Areas specified by the user are excluded from the division.

[0199] Step SR9 determines whether processing of all polygons in the mesh data is complete. If not, proceed to step SR8; if complete, proceed to step SR10. Step SR10 executes the face generation process. After that, proceed to step SR11. Step SR11 determines whether attributes exist. If attributes exist, proceed to step SR12; if no attributes exist, proceed to step SR13. Step SR12 determines the type of attribute. If the attribute is a shape, proceed to step SR14; if the attribute is a color, proceed to step SR15.

[0200] In step SR14, the additional information identification unit 271d identifies a color based on the shape-color correspondence table. The shape-color correspondence table is a table that identifies a color assigned to each shape, for example, a table that identifies a plane as blue and a cylinder as green. This table may be created by the user. In step SR15, a curved surface is generated from polygons within the area specified by the user. In step SR16, the additional information assignment unit 271e assigns color information to the curved surface generated in step SR15.

[0201] If polygon color is detected in step SR13, the process proceeds to step SR17 to determine a representative color from the polygon colors within each segment area. The representative color determined here is an example of additional information. If polygon color is detected in step SR13, the process proceeds to step SR18 to generate a surface from the polygons within the area specified by the user.

[0202] The process then proceeds to step SR19 to determine whether processing of the entire area is complete. If processing of the entire area is not complete, the process returns to step SR10. If processing of the entire area is complete, the process proceeds to step SR20, where the CAD output unit 274 outputs the CAD data with the added information, and the process terminates.

[0203] Figure 41 shows an example of the structure of measurement data that includes texture information as additional information. The texture information includes a texture image.

[0204] As explained above, once the region included in the mesh data obtained by measuring the shape of the workpiece W is identified, the polygons included in the identified region are converted into surface data. If additional information such as color, reflection, or color deviation is to be added to the converted surface data, the additional information is identified, and the identified additional information is automatically added to the surface data. Then, CAD data with the added information is output, and this additional information can be used as useful information during reverse engineering.

[0205] Furthermore, since the polygons included in the region identified by the region identification unit 278 can be associated with the additional information identified by the additional information identification unit 271d and their shape attributes, regions with the same shape attributes can be automatically assigned the same additional information.

[0206] Furthermore, if the additional information is color deviation, it may be difficult to add color information that precisely represents the difference and convert it into CAD data. In such cases, a representative color may be determined, and the additional information may be added to the region based on that determined representative color. Also, if the additional information is a color texture, the texture may be added as is, or a representative color may be assigned to each region.

[0207] (Mesh data correction function) This reverse engineering system 1 has a correction function that corrects the mesh data obtained by measuring the workpiece W, and this correction function enables the execution of the correction process. As explained based on the block diagram shown in Figure 5, when the correction process is started, the input receiving unit 271a is configured to accept user input to extract a correction reference surface that will serve as the basis for the correction process from the mesh data obtained by the data acquisition unit 291. The data used to identify the correction reference surface includes, for example, the polygons and vertices that make up the correction reference surface, as well as the type of surface to be analyzed (plane, cylinder, etc.) and the coefficients of the mathematical formula, and this data is stored.

[0208] The reference surface extraction unit 271f extracts a corrected reference surface from the mesh data acquired by the data acquisition unit 291 based on the user input received by the input reception unit 271a. The correction unit 275 corrects the mesh data acquired by the data acquisition unit 291 based on the corrected reference surface extracted by the reference surface extraction unit 271f. The correction unit 275 can also generate a geometric polygon that represents the ideal shape of the corrected reference surface based on the corrected reference surface. In this case, polygons included in the corrected reference surface are deleted and replaced with geometric polygons. Here, a geometric polygon is a geometric element such as a cylinder or a plane that has been tessellated (triangulated into a polygon).

[0209] An example of correction by the correction unit 275 is to correct the mesh data acquired by the data acquisition unit 291 by detecting the discrepancy between the mesh data acquired by the data acquisition unit 291 and the correction reference surface, removing the mesh data in the area where the discrepancy was detected, and creating new mesh data in the area where the mesh data was removed. For example, the correction unit 275 is configured to correct the mesh data acquired by the data acquisition unit 291 by detecting the distance between the mesh data acquired by the data acquisition unit 291 and the correction reference surface, removing the mesh data in the area where the detected distance is greater than or equal to a predetermined threshold, and creating new mesh data in the area where the mesh data was removed.

[0210] Furthermore, the correction unit 275 may detect the amount of normal change between the mesh data acquired by the data acquisition unit 291 and the correction reference surface. After detecting the amount of normal change between the mesh data and the correction reference surface, the correction unit 275 can also correct the mesh data acquired by the data acquisition unit 291 by removing the mesh data in the region where the detected amount of normal change is greater than or equal to a predetermined threshold, and creating new mesh data in the portion where the mesh data was removed.

[0211] The input receiving unit 271a can also accept the designation of unnecessary regions included in the mesh data acquired by the data acquisition unit 291. Unnecessary regions include, for example, holes or protrusions. In this case, the correction unit 275 corrects the mesh data acquired by the data acquisition unit 291 by removing the mesh data in the regions included in the unnecessary regions accepted by the input receiving unit 271a and in which there is a discrepancy between the mesh data acquired by the data acquisition unit 291 and the correction reference surface, and by creating new mesh data in the parts from which the mesh data was removed.

[0212] When the correction unit 275 performs mesh data correction processing, it first determines whether or not mesh data is included in the area where the input receiving unit 271a has received user input for extracting the correction reference surface. If the correction unit 275 determines that mesh data is not included in the area received by the input receiving unit 271a, it can cancel the correction of the mesh data acquired by the data acquisition unit 291.

[0213] The correction unit 275 may also correct the mesh data acquired by the data acquisition unit 291 by removing mesh data from the area extending continuously from the point received as an unnecessary area by the input reception unit 271a to the correction reference surface, and creating new mesh data in the area from which the mesh data was removed.

[0214] The input receiving unit 271a can also accept input for multiple correction reference surfaces. Specifically, the input receiving unit 271a can accept a first user input for extracting a first surface as the first correction reference surface, a second user input for extracting a second surface that is spaced apart from the first surface as the second correction reference surface, and the specification of a through-hole region existing between the first surface and the second surface.

[0215] When the first and second user inputs are received, the reference surface extraction unit 271f extracts a first correction reference surface from the mesh data acquired by the data acquisition unit 291 based on the first user input, and extracts a second correction reference surface from the mesh data acquired by the data acquisition unit 291 based on the second user input. Subsequently, the correction unit 275 detects the discrepancies between the mesh data acquired by the data acquisition unit 291 and the first and second correction reference surfaces, respectively. Then, the correction unit 275 corrects the mesh data acquired by the data acquisition unit 291 by removing the mesh data of the region included in the through-hole region received by the input reception unit 271a and where a discrepancy exists, and by creating new mesh data in the portion where the mesh data was removed.

[0216] The correction unit 275 may also correct the mesh data acquired by the data acquisition unit 291 by removing mesh data from the area extending continuously from the point received as an unnecessary area by the input reception unit 271a to the first correction reference plane and the second correction reference plane, and creating new mesh data in the area from which the mesh data was removed.

[0217] The following describes a specific example of the correction process based on a flowchart. In step S101 after the start of the flowchart shown in Figure 42, the correction reference surface is extracted. Specifically, as shown in the flowchart in Figure 43, geometric elements are extracted in step SU1. The method for extracting geometric elements is as shown in the flowchart in Figure 20. In step SU2, the discrepancy between the mesh data and the correction reference surface is detected, and polygons are selected from the discrepancy with the correction reference surface. In step SU3, the type of geometric element and the selected mesh data are registered. In step SU4, it is determined whether or not there are penetrating polygons. If there are penetrating polygons, the process returns to step SU1; however, if there are no penetrating polygons, the extraction of the correction reference surface is terminated. Note that the geometric elements used to determine the correction reference surface include not only planes, but also cylinders, cones, spheres, tori, etc.

[0218] Next, the process proceeds to step S102 in Figure 42. In step S102, candidate areas for correction are extracted. Specifically, as shown in the flowchart in Figure 44, step SO1 detects the inner loop of the polygon constituting the correction reference surface. In step SO2, the area inside the inner loop detected in step SO1 is set as the candidate area for correction. In step SO3, it is determined whether all boundaries have been adjusted. If all boundaries have been adjusted, the process ends; otherwise, the process proceeds to step SO4. In step SO4, the range of the candidate area for correction is expanded based on density. In step SO5, it is determined whether the correction reference surface area has been divided. If the correction reference surface area has been divided, the process proceeds to step SO6; otherwise, the process returns to step SO3. In step SO6, the candidate area for correction is restored to its original state. Data for identifying the candidate area for correction includes boundary edges connected to the correction reference surface, and polygons and vertices connected to the correction reference surface polygon, and these are retained.

[0219] Next, the process proceeds to step S103 in Figure 42. In step S103, the hole locations are identified. Specifically, as shown in the flowchart in Figure 45, step S111 determines whether the hole location identification is automatic or manual. If it is manual, the process proceeds to step S112; if it is automatic, the process proceeds to step S113. In step S113, the geometric inner circumference loop is detected, and then in step S114, all inner circumferences are identified as holes to be corrected, and the process ends. Data used to identify the hole locations includes, for example, the boundary edges of the candidate correction area, and the polygons and vertices of the candidate correction area, and this data is stored.

[0220] In step S112, polygons or edges are selected. In step S115, an error check is performed; if there are no errors, the process proceeds to step S116, but if there are errors, it returns to step S112. In step S116, it is determined whether all have been selected or not; if not all have been selected, the process returns to step S112; if all have been selected, the process proceeds to step S117. In step S117, the specified hole location is identified as the hole to be corrected, and this process ends.

[0221] Next, proceed to step S104 in Figure 42. In step S104, delete the polygons that are candidates for correction. In step S105, perform polygon healing. Specifically, repair the corrupted mesh by removing self-intersections, non-manifold meshes, and minute meshes. In step S106, fill the specified holes. Then, in step S107, perform polygon healing again. Polygon healing should be performed as needed.

[0222] Figure 46 is a flowchart showing the procedure for deleting polygons included in the correction reference plane and replacing the deleted parts with geometric polygons. In step S121, the correction reference plane is extracted. This is the same as step S101 in Figure 42. In step S122, geometric polygons are generated. In step S123, the polygons of the correction reference plane are deleted. In step S124, the correction reference plane is replaced with geometric polygons. In step S125, a process is executed to connect the boundary between the polygons outside the correction reference plane and the replacement polygons replaced in step S124.

[0223] Figure 47 shows an example of area selection. FIG. 47A shows the case where the planar portion W1 of the workpiece W is specified using the operation unit 250. When the planar portion W1 is specified using the operation unit 250, the planar area continuous from the specified point is automatically selected and displayed on the display unit 400. Specifically, the color of the planar area is changed before and after selection. In addition, cylindrical areas, spherical areas, conical areas, etc., other than planar areas can be specified in the same way.

[0224] Figure 47B shows the case where the four holes W2 to W5 of the workpiece W are specified as deletion candidate areas using the operation unit 250. When hole W2 is specified, hole W2 is displayed on the display unit 400 in a specified state. Similarly, each time holes W3, W4, and W5 are specified in order, the display on the display unit 400 is updated, making it easy to see that they have been selected. Specifically, the color of the selected hole changes before and after the selection operation. In addition, if the holes penetrate the workpiece W, it is possible to automatically specify the holes that open on the back side.

[0225] Figure 48 shows an example of hole filling. FIG. 48A shows the case of filling holes W2 to W5 on the plane of the workpiece W. After specifying the plane and holes W2 to W5, the display unit 400 can display the state with holes W2 to W5 filled. FIG. 48B shows the case where the plane of the workpiece W is replaced with a planar polygon. The display unit 400 can display the state with the planar polygon replacement.

[0226] In addition to changing the polygons obtained by measuring the workpiece W into modified polygons, the measured polygons can also be given a status of "convert to planar form after CAD data conversion," and then replaced with planar elements after CAD data conversion. For example, for polygon data containing noise, geometric element (plane, cylinder, etc.) ID information can be assigned to the vertices or faces of the selected area, and after surface conversion, the ID information can be referenced to correct the polygon data into a perfect plane or cylinder.

[0227] The following are specific examples of hole filling. If the correction reference surface is a plane, it can be filled by either replacing it with a plane polygon or by positional continuity filling, which may be user-specified. If the correction reference surface is a cylinder, it can be filled by either replacing it with a cylindrical polygon or by curvature continuity filling. If the correction reference surface is a cone, it can be filled by either replacing it with a cone polygon or by curvature continuity filling. If the correction reference surface is a sphere, it can be filled by either replacing it with a spherical polygon or by curvature continuity filling. If the correction reference surface is a torus (ring surface), it can be filled by either replacing it with a torus polygon or by curvature continuity filling. If the correction reference surface is a freeform surface, it can be filled by either positional continuity filling, tangent continuity filling, or curvature continuity filling.

[0228] As an example of error handling in the event of an error, if a geometric shape does not exist at the location specified by the user, the geometric shape extraction is canceled. For example, if the user performs cylinder extraction on a plane, it will be canceled. Also, when specifying a hole position, if a polygon does not exist at the location specified by the user, or if the orientation (normal) of the selected polygon is facing backward, the polygon will not be selected. That is, the correction unit 275 determines the orientation of the polygons in the area where the input reception unit 271a has received user input for extracting the correction reference surface. If the correction unit 275 determines that the orientation of the polygons in that area is facing backward, it stops correcting the mesh data acquired by the data acquisition unit 291. Here, the determination of whether the orientation of a polygon is facing backward can be performed, for example, by determining whether the orientation of the normal of the correction reference surface and the orientation of the normal of the polygon are at or above a predetermined angle. The correction unit 275 may also cancel geometric shape extraction based on continuity with the inner loop. That is, if a polygon that is discontinuous with the inner loop is selected, the polygon will not be selected. This prevents the entire polygon from being selected by selecting the polygon on the back side due to a missing polygon.

[0229] During the fill-in process, if, after deleting surface polygons, another polygon protrudes from the back side, the user can be warned of the interference with the back side polygon and given the option to delete the back side polygon or cancel the polygon replacement. Furthermore, problems may occur when actually generating geometric polygons during polygon replacement. For example, if adjacent polygons intersect, a healing process is performed to resolve the problem by deforming the polygons at the boundary of the corrected polygon.

[0230] In other words, when a user views mesh data, they can sometimes distinguish between normal areas that do not require hole-filling and incomplete areas that do require hole-filling. Conventional hole-filling and smoothing functions correct incomplete areas by specifying them, but in this embodiment, the user can input for extracting the correction reference surface so that the normal areas are extracted as the correction reference surface. As a result, normal areas are automatically extracted from the mesh data as the correction reference surface. By using the extracted correction reference surface as a reference, it becomes possible to identify incomplete areas in the mesh data. By automatically correcting these incomplete areas based on the normal areas extracted as the correction reference surface by the correction unit 275, hole-filling is performed in a short time without relying on human judgment and with good finished quality.

[0231] (CAD conversion function that takes coordinate systems into account) This reverse engineering system 1 has a function to convert data to be converted into CAD data, to which a certain coordinate system has been assigned. As explained based on the block diagram shown in Figure 5, the input receiving unit 271a can accept the selection of data to be converted into CAD from the mesh data acquired by the data acquisition unit 291, and is configured to accept the selection of the shape of the geometric elements to be extracted from the mesh data acquired by the data acquisition unit 291, as well as the selection of points on the mesh data. Furthermore, the input receiving unit 271a can also accept the selection of the area of ​​the mesh data to be converted into CAD from the mesh data acquired by the data acquisition unit 291.

[0232] The data identification unit 271g identifies data to be converted to CAD from the mesh data based on the data selected by the input reception unit 271a. Specifically, the data identification unit 271g identifies geometric elements extracted based on the shape of the geometric elements and the specification of points received by the input reception unit 271a as data to be converted to CAD. When the input reception unit 271a accepts a region selection, the data identification unit 271g identifies the mesh data contained in the region as data to be converted to CAD based on the region selection accepted by the input reception unit 271a. Here, when accepting a region selection in order to identify data to be converted to CAD, for example, the data to be converted to CAD may be identified by accepting user input enclosing a predetermined region, thereby identifying the mesh data contained in that region as data to be converted to CAD, or by accepting user input of a line and a point, the mesh data existing on one side of the line may be identified as data to be converted to CAD based on the positional relationship between the line and the point.

[0233] The coordinate system assignment unit 271h assigns a coordinate system to the data to be converted into CAD, which has been identified by the data identification unit 271g. At this time, a user interface screen 700 for specifying a conversion area, which has a coordinate system selection area 703 as shown in Figure 13, can be displayed on the display unit 400. This allows the user to select an area and set the coordinate system associated with that area. The image display area 701 in the user interface screen 700 for specifying a conversion area is a mesh data display area that displays mesh data. The coordinate system selection area 703 is a coordinate system assignment area where coordinate system assignment operations can be performed. The area where buttons 702a to 702e are displayed is a data extraction setting area where settings for extracting data to be converted into CAD can be performed.

[0234] The CAD conversion unit 273 converts the data to be converted into CAD data, to which a coordinate system has been assigned by the coordinate system assignment unit 271h. The coordinate system assignment unit 271h is configured to assign a coordinate system to the data to be converted into CAD data, which has been identified by the data identification unit 271g, a coordinate system different from the coordinate system used for display on the display unit 400. The coordinate system of the CAD data output as a result of reverse engineering and the coordinate system of the data to be converted into CAD data displayed on the display unit 400 do not necessarily match. By specifying a coordinate system before conversion to CAD data, the user can select a coordinate system suitable for the design, independently of the coordinate system used for display on the display unit 400.

[0235] The input receiving unit 271a may be configured to accept multiple data selections. That is, the input receiving unit 271a can accept the selection of a first CAD conversion target data and the selection of a second CAD conversion target data. In this case, the data identification unit 271g identifies the first CAD conversion target data from the mesh data based on the first CAD conversion target data selected by the input receiving unit 271a, and identifies the second CAD conversion target data from the mesh data based on the second CAD conversion target data. Then, the coordinate system assignment unit 271h assigns a first coordinate system to the first CAD conversion target data identified by the data identification unit 271h, and assigns a second coordinate system to the second CAD conversion target data identified by the data identification unit 271h. The coordinate system of each CAD data output as a result of reverse engineering does not necessarily match the coordinate system of the CAD conversion target data displayed on the display unit 400. For each piece of data to be converted to CAD data, the user can specify a coordinate system before conversion to CAD data, allowing them to select a coordinate system suitable for the design, independently of the coordinate system used for display on the display unit 400.

[0236] The coordinate system assigned by the coordinate system assignment unit 271h is stored in the storage device 240. The storage device 240 stores the first coordinate system assigned to the first CAD conversion target data and the relative position and orientation of the first coordinate system with respect to the global coordinate system, in association with each other. Furthermore, the storage device 240 stores the second coordinate system assigned to the second CAD conversion target data and the relative position and orientation of the second coordinate system with respect to the global coordinate system, in association with each other.

[0237] The CAD conversion unit 273 converts the data to be converted into first CAD data based on the relative position and orientation of the first coordinate system with respect to the global coordinate system. The CAD conversion unit 273 also converts the data to be converted into second CAD data based on the relative position and orientation of the second coordinate system with respect to the global coordinate system. The CAD conversion unit 273 then generates assembled CAD data in which the first CAD data and the second CAD data are assembled. In this assembled CAD data, the relative position and orientation with respect to the global coordinate system are assigned to the first CAD data and the second CAD data included in the assembly. Therefore, the assembled CAD data is generated while maintaining the relative position and orientation of the data to be converted into first CAD data and the data to be converted into second CAD data selected from the mesh data acquired by the data acquisition unit 291.

[0238] If the relative position and orientation of the first coordinate system with respect to the global coordinate system are not associated with the relative position and orientation of the second coordinate system with respect to the global coordinate system, the CAD data generated by the CAD conversion unit 273 will be displayed on the display unit with the origins of each coordinate system overlapping. As a result, the original positional relationship between the first CAD data and the second CAD data will be lost. However, by associating the relative position and orientation of the first coordinate system with respect to the global coordinate system with respect to the global coordinate system, as described above, the original positional relationship between the first CAD data and the second CAD data can be preserved while converting to CAD data.

[0239] The following describes a specific example of the process based on the flowchart. In step S131 after the start of the flowchart shown in Figure 49, the user selects a coordinate system. The coordinate system selection operation is shown in Figure 50, and in step S141, it is determined whether the coordinate system selection is automatic or manual. If it is manual, the process proceeds to step S142; if it is automatic, the process proceeds to step S145. In step S142, it is determined whether or not to create a coordinate system. If a coordinate system is to be created, the process proceeds to step S143; if a coordinate system is not to be created, the process proceeds to step S144. In step S143, the coordinate system creation unit 271i shown in Figure 5 executes the coordinate system creation process in manual mode (first mode). The coordinate system creation process is shown in Figure 51, and it creates geometric elements such as a plane, cylinder, cone, and sphere that will be the basis of the coordinate system, and then creates a coordinate system by combining these geometric elements.

[0240] Specifically, in step S151 of Figure 51, geometric elements are extracted. In step S152, the user selects a reference plane. At this time, a reference plane selection screen 780, as shown in FIG. 53A of Figure 53, can be displayed on the display unit 400. Here, the reference plane selection screen 780 is an example of a coordinate system creation screen. The reference plane can be, for example, a machined surface or the XY plane.

[0241] In step S153, the user selects a reference axis. At this time, a reference axis selection screen 790, as shown in FIG. 53B of Figure 53, can be displayed on the display unit 400. Here, the reference axis selection screen 790 is an example of a coordinate system creation screen. The reference axis can be, for example, the normal of the cross-sectional reference plane, or the X-axis.

[0242] In step S154, the user selects the origin. At this time, an origin selection screen 800, as shown in FIG. 53C of Figure 53, can be displayed on the display unit 400. Here, the origin selection screen 800 is an example of a coordinate system creation screen. As the origin, for example, the starting point of the shaft holder can be used, or any point can be arbitrarily selected. Then, in step S155, the coordinate system creation unit 271i stores the created coordinate system in the storage device 240.

[0243] As described above, once the coordinate system creation unit 271i completes the creation of the coordinate system by accepting the selection of the reference plane, reference axis, and origin on the coordinate system creation screen, it proceeds to step S144 in Figure 50. In step S144, the user selects an arbitrary coordinate system from the created coordinate systems stored in the storage device 240 and then terminates. When selecting a coordinate system, the user is provided with a user interface that allows the user to select any one coordinate system from among multiple coordinate systems, as shown in the coordinate system selection area 703 of the user interface screen 700 for specifying the transformation area in Figure 13. The input receiving unit 271a then accepts the selection of any one coordinate system from among multiple coordinate systems. If the desired coordinate system is not available in the coordinate system selection area 703, the input receiving unit 271a can accept the operation input of the coordinate system creation button 705. When the input receiving unit 271a accepts the operation input of the coordinate system creation button 705, the display control unit 276 displays the coordinate system creation screen on the display unit 400, and the coordinate system creation unit 271i can create a new coordinate system.

[0244] Also, in step S145 that is determined to be automatic and proceeds in step S141, the coordinate system creation unit 271i executes automatic setting of the coordinate system and ends. That is, the coordinate system creation unit 271i creates a coordinate system in the automatic mode (the second mode). Here, the coordinate system creation unit 271i creates a coordinate system based on the bounding box of the data to be CAD-converted. That is, when calculating the simple coordinate system, the coordinate system creation unit 271i calculates a bounding box that is the smallest rectangle or rectangular parallelepiped circumscribing the data to be CAD-converted, and determines the XYZ axes of the coordinate system from the orientation of the bounding box. The origin is set at the center of the bounding box or the center or vertex of the plane constituting the bounding box. Note that the origin may be set by the user. For example, selection of one vertex from the eight vertices of the bounding box may be accepted, and the selected vertex may be used as the origin.

[0245] After that, after proceeding to step S132 in FIG. 49 to select a conversion area, it proceeds to step S133 to convert it into CAD data.

[0246] Also, as shown in FIG. 52, it may be determined whether there is an area that needs to be specified in step S161. If there is no remaining area to be specified, it ends. If there is a remaining area, it proceeds to step S162. In step S162, the selection method of the area is specified. The selection methods include polygon selection, geometric shapes such as a plane or a cylinder, color specification, etc.

[0247] In step S163, mesh selection is executed. After that, in step S164, it is determined whether the confirmation button has been pressed. If the confirmation button has not been pressed, it returns to step S163. If the confirmation button has been pressed, it proceeds to step S165. In step S165, an error determination is made. If there is an error, it returns to step S163. If there is no error, it proceeds to step S166 to register the selected mesh.

[0248] FIG. 54 shows a user interface screen 810 capable of displaying measurement data and reference data. On this user interface screen 810, there are provided a measurement data display area 811 for displaying measurement data, a reference data display area 812 for displaying reference data serving as an alignment reference, and a preview area 813. In the preview area 813, at least one of the measurement data and the reference data is displayed. It is also possible to display both the measurement data and the reference data in the preview area 813. In this case, the input reception unit 271a receives the designation of corresponding surfaces with the measurement data and the reference data. Then, based on the transformation matrix of the coordinate system of the measurement data and the coordinate system of the reference data calculated based on the designation of the corresponding surfaces received by the input reception unit 271a, the display control unit can display the composite data obtained by performing the alignment process between the measurement data and the reference data.

[0249] FIG. 55 shows a user interface screen 820 showing a state where the alignment of the measurement data and the reference data is completed. On this user interface screen 820, there is provided a display area 821 for displaying the data after alignment. In the display area 821, a coordinate system 821a of the CAD data is shown. Also, as shown in FIG. 55, 3D comparison measurement may be possible using the data after alignment.

[0250] Data specifying a reference plane, a reference axis, and an origin are held as data of a coordinate system (transformation matrix with respect to the measurement data coordinate system). The reference plane includes plane elements (such as a plane, a parallel plane, etc.), the reference axis includes axis elements (such as a cylinder, a cone, an intersection of surfaces, etc.), and the origin includes point elements (such as a point, an intersection of a surface and an axis, etc.). Further, information on the selected polygon and vertices is also held as a transformation area.

[0251] [[ID=eleven]] It is also possible to register different local coordinate systems for each transformation region, resulting in CAD data transformations with one global coordinate system and multiple local coordinate systems + selected regions. When specifying the global coordinate system as data for CAD data transformation, it includes a reference plane containing planar elements, a reference axis containing axis elements, and an origin containing point elements. Furthermore, transformation region 1 includes the local coordinate system (coordinate transformation matrix relative to the global coordinate system) and the selected region (selected polygons or vertices), and transformation region 2 contains similar information. The same applies to transformation regions 3, 4, ...

[0252] (A modified example to enable fully automated CAD conversion) Figures 56 and 57 are flowcharts showing modified examples for enabling fully automated CAD conversion. In step S201 after the start in Figure 56, the analysis application, which is the main application, is launched on the controller 200. In step S201, it is determined whether or not to perform fully automated CAD conversion. If fully automated CAD conversion is not performed, the process proceeds to step S203 as a normal measurement; if fully automated CAD conversion is performed, the process proceeds to step S204. In steps S203 and S204, measurements are performed using the measurement module. In the measurement in step S204, multiple measurement data obtained by changing the relative positional relationship between the workpiece W and the measurement unit 100 are combined, and after combining the multiple measurement data, the process may proceed to data saving in step S206 and automatic execution of surface conversion in step S219. In step S205, the results are checked in the analysis module 290. In step S206, the data is saved in the analysis module 290, and the process proceeds to the surface conversion process in step S219.

[0253] In step S207, it is determined whether or not to add data. If data is to be added, the process proceeds to step S208 for measurement, and then to step S209 for data synthesis. If data is not to be added, the process proceeds to step S210 to determine whether or not to create geometric elements. If geometric elements are to be created, the process proceeds to step S211 for extraction of the geometric elements; if geometric elements are not to be created, the process proceeds to step S212 for setting and selecting the coordinate system. In step S213, the transformation region is selected.

[0254] In step S214 of Figure 59, it is determined whether the transformation region is the entire region or a partial region. If the transformation region is a partial region, the process proceeds to step S215 to extract the specified region from the polygon. If the transformation region is the entire region, the process proceeds to step S216 to extract the entire polygon.

[0255] Next, the process proceeds to step S217, where the user selects a conversion mode. In step S218, it is determined whether the mode selected by the user is fully automatic mode or manual mode. In fully automatic mode, the process proceeds to step S219, where the surface conversion is automatically executed. In the automatic execution of the surface conversion in step S219, pre-processing such as filling holes or removing unnecessary parts may be performed based on the irregularities contained in the surface data. That is, the correction unit 275 determines whether or not the surface data contains irregularities and whether or not the mesh data received by the data input unit 271 contains loops without polygons, and can perform pre-processing based on the determination result. In manual mode, the process proceeds to step S220, where the conversion module 270 displays the surface conversion user interface screen 710 shown in Figure 14 on the display unit 400.

[0256] In step S221, it is determined whether or not to perform processing such as hole filling on the mesh data. If processing such as hole filling is to be performed, the process proceeds to step S222 to execute each process, and then the process proceeds to step S223 to determine whether or not to specify the surface area. On the other hand, if mesh preprocessing is not performed, the process proceeds to step S224, and if the surface conversion button is pressed, the process proceeds to step S228 to execute the surface conversion.

[0257] If step S223 is NO, the process proceeds to step S224; otherwise, it proceeds to step S225 to perform surface area specification. In step S226, it is determined whether the specification is complete or not. If the specification is not complete, the process returns to step S225; however, if the specification is complete, the process proceeds to step S227, and if the surface conversion button is pressed, it proceeds to step S228 to perform the surface conversion.

[0258] In step S229, the surface conversion result is displayed on the display unit 400. This allows the user to confirm the surface conversion result. In step S230, it is determined whether the conversion is complete or not. If it is not complete, that is, if the user wants to change some settings and redo the CAD conversion, they can return to the manual settings. For example, as shown in Figure 58, the user interface screen 830 for displaying the conversion result is provided with a pre-conversion data display area 831 that displays the data before conversion, a post-conversion data display area 832 that displays the CAD data after conversion, and a display setting area 833. By displaying the pre-conversion data and the post-conversion CAD data side by side, the user can compare the pre-conversion data and the post-conversion CAD data to determine whether or not it is necessary to change the settings.

[0259] Furthermore, by toggling the checkbox 833a in the display setting area 833, it is possible to switch whether or not to add contour lines indicating the boundaries of surface data to the CAD data. Adding contour lines to the CAD data makes it easier to understand whether the CAD conversion is being performed as intended by the design. In addition, radio buttons 833b may be displayed in the display setting area 833 to select whether to display a combination of surface data and 3D elements, or to display either surface data or 3D data. Switching the display format will be described later. If it is determined that the process is not completed in step S230, the process proceeds to step S231 in Figure 57 to determine whether data addition and conversion reset are necessary. If conversion reset is necessary, the process returns to step S220 and it is determined that fill-in processing, etc., will be performed in step S221. If conversion reset is required, a user interface screen 840 for resetting, as shown in Figure 59, is displayed on the display unit 400. The user interface screen 840 for resetting includes a converted data display area 841 that displays the converted CAD data and a settings change display area 842. This allows users to find the part they want to modify within the conversion results and then change the settings. In the settings change display area 842, it is possible to perform operations such as filling in holes, removing unnecessary parts, and specifying surface areas.

[0260] Specifically, the surface conversion results displayed on the display unit 400 may not reflect the user's design intent. For example, when the spherical region 834 in the pre-conversion data display area 831 in Figure 58 is converted to CAD data, it is converted into CAD data divided into multiple surfaces, as shown in the spherical region 835 in the post-conversion data display area 832. In such cases, when the operation input of the icon 833c for performing manual conversion is detected, the system transitions to the manual conversion settings screen 836 shown in Figure 59. On the manual conversion settings screen 836 shown in Figure 59, settings for pre-processing such as data deletion and hole filling, as well as shape settings, can be made. Data deletion and hole filling are the same as in step SB8 in Figure 9. In addition, in the shape settings, by selecting a predetermined shape such as a plane, cylinder, freeform surface, sphere, or cone, and specifying the area on the mesh data to which the selected shape will be applied, the desired shape can be extracted from the mesh data. In other words, the mesh data can be converted to CAD data while reflecting the user's design intent. For example, for the spherical region 834 shown in Figure 58, the shape "sphere" is specified in the manual conversion settings screen shown in Figure 59, and the surface conversion is performed again. In this case, the surface conversion shown in step S228 of Figure 57 is performed again, and the surface conversion result shown in Figure 60 is displayed on the display unit 400. As shown in the spherical region 837 of Figure 60, by selecting the shape "sphere" in the manual conversion settings screen, the spherical region 835 of Figure 58 is converted into CAD data as a single surface data, allowing the user's design intent to be reflected.

[0261] Furthermore, the switching of display modes will be explained using Figures 61 to 63. Figure 61 is an example of the user interface screen 710 for surface conversion displayed on the display unit 400. In Figure 61, two cylinders, one cone, and one sphere are selected as geometric elements. Figure 62 shows the mesh data containing the geometric elements extracted in Figure 61 converted to CAD data. Figure 62 is an example of the user interface screen 830 for displaying conversion results. As described above, the user interface screen 830 for displaying conversion results in Figure 62 is provided with a display setting area 833, and by switching the selection of the radio button 833b, it is possible to switch between displaying a combination of surface data and 3D elements, or displaying either surface data or 3D data. Figure 62 shows the case where a combination of surface data and 3D elements is displayed. Figure 63 shows the case where only 3D elements are displayed. Here, "3D elements" refers to the geometric elements extracted from the mesh data converted to CAD data. By switching the display mode on the display unit 400 in this way, it is possible to extract only the 3D elements and check and save the data, or to check and save data that combines surface data and 3D elements. This allows the user to selectively save the desired area or part shape as CAD data.

[0262] On the other hand, if additional data is required, the process returns to step S205 and determines in step S207 that data addition is necessary. After re-conversion, the user can confirm the conversion results by displaying a user interface screen 830 for displaying the conversion results on the display unit 400, as shown in Figure 58. If completion is determined, the surface conversion is terminated.

[0263] When you want to add data after viewing the surface conversion result, you can return to the already launched analysis module 290. When data synthesis is launched in the analysis module 290, a user interface screen 850 for data synthesis is displayed on the display unit 400 as shown in FIG. 64. It is possible to display a composite data selection window 860 shown in FIG. 65 from the user interface screen 850. In the composite data selection window 860, as a method for selecting composite data, one of a method of acquiring data from the actual machine (measurement unit 100) and a method of reading from a file stored in the storage device 240 or the like can be selected.

[0264] FIG. 66 shows a user interface screen 870 for acquiring measurement data of the workpiece W with its orientation changed. In this example, it shows the measurement data on the back side of the workpiece W. Thus, it is possible to synthesize data when the workpiece W is measured from different directions. The orientation of the workpiece W can be set arbitrarily.

[0265] The above-described embodiments are merely illustrative in all respects and should not be construed in a limiting sense. Further, modifications and changes belonging to the equivalent scope of the claims are all within the scope of the present invention.

Industrial Applicability

[0266] As described above, the present invention can be used for reverse engineering that acquires the shape of an actual object and converts it into design data.

Explanation of Reference Numerals

Claims

1. A reverse engineering system that converts mesh data obtained by measuring the shape of a workpiece into CAD data and outputs it, An analysis module having a data acquisition unit that acquires mesh data, an element extraction unit that extracts multiple geometric elements from the acquired mesh data, and a calculation unit that calculates the dimensions between the extracted multiple geometric elements, The system includes a conversion module that converts the mesh data output from the analysis module into CAD data to create CAD data, The analysis module further includes: a setting unit for setting a conversion region in which CAD conversion is performed from the mesh data acquired by the data acquisition unit; a data extraction unit for extracting the portion of the entire mesh data acquired by the data acquisition unit that is included in the conversion region, based on the conversion region set by the setting unit, as mesh data to be output to the conversion module; and a conversion data output unit for outputting the mesh data extracted by the data extraction unit to the conversion module. The reverse engineering system is characterized by comprising: a data input unit that receives input of mesh data extracted by the data extraction unit and output from the converted data output unit; a CAD conversion unit that converts the mesh data extracted by the data extraction unit and input to the data input unit into CAD data; and a CAD output unit that outputs the CAD data created by the CAD conversion unit.

2. In the reverse engineering system according to claim 1, The reverse engineering system is characterized in that the conversion data output unit is configured to output mesh data with a smaller data capacity to the conversion module than the mesh data acquired by the data acquisition unit, by outputting the mesh data of the conversion region set by the setting unit from the mesh data acquired by the data acquisition unit to the conversion module.

3. In the reverse engineering system according to claim 1, The reverse engineering system is characterized in that the conversion module further includes a correction unit that corrects the loop by creating new mesh data within the loop when the mesh data received by the data input unit includes a loop in which no polygons exist.

4. In the reverse engineering system according to claim 1, The aforementioned analysis module is a reverse engineering system characterized by further having an approximation processing unit that accepts the specification of an approximation region that approximates a geometric element and a geometric element that approximates the approximation region, and approximates the mesh data included in the approximation region with the specified geometric element.

5. In the reverse engineering system according to claim 4, The reverse engineering system is characterized in that the conversion data output unit outputs to the conversion module a mixture of mesh data of the conversion region set by the setting unit and geometric element data created by approximating the mesh data included in the approximation region with the geometric elements.

6. In the reverse engineering system according to claim 4, The reverse engineering system is characterized in that the CAD conversion unit is configured to convert mesh data of the conversion region set by the setting unit into CAD data as surface data of a free-form surface, and to convert mesh data included in the approximation region into CAD data as surface data of an analyzed surface.

7. In the reverse engineering system according to claim 1, The aforementioned conversion module includes a first conversion module and a second conversion module. The setting unit is configured to allow setting a first conversion region for performing a first CAD conversion and a second conversion region for performing a second CAD conversion from the mesh data acquired by the data acquisition unit. The data extraction unit is configured to extract first mesh data based on the first conversion region set by the setting unit, and is configured to extract second mesh data based on the second conversion region set by the setting unit. The reverse engineering system is characterized in that the conversion data output unit is configured to output the first mesh data extracted by the data extraction unit to the first conversion module, and is configured to output the second mesh data extracted by the data extraction unit to the second conversion module.

8. In the reverse engineering system according to claim 7, The first conversion module converts the first mesh data into CAD data. A reverse engineering system characterized in that the second conversion module converts the second mesh data into CAD data regardless of the operating status of the first conversion module.

9. In the reverse engineering system according to claim 1, The analysis module further includes a first display control unit for displaying the mesh data acquired by the data acquisition unit on the display unit. The setting unit is configured to be able to set a coordinate system for displaying the mesh data on the display unit. The first display control unit causes the mesh data to be displayed on the display unit based on the coordinate system set by the setting unit. The conversion data output unit is configured to create an output file that associates the mesh data with the coordinate system, and to output the created output file to the conversion module. The conversion module further includes a second display control unit for displaying the mesh data received by the data input unit on the display unit. The data input unit is configured to be able to read the output file output by the converted data output unit. The reverse engineering system is characterized in that the second display control unit is configured to reproduce the display state of the mesh data in the analysis module on the display unit by displaying the mesh data contained in the output file on the display unit based on the coordinate system contained in the output file.

10. In the reverse engineering system according to claim 1, The analysis module further includes a first display control unit for displaying the mesh data acquired by the data acquisition unit on the display unit. The setting unit is configured to be able to set a coordinate system for displaying the mesh data on the display unit. The first display control unit causes the mesh data to be displayed on the display unit based on the coordinate system set in the setting unit. The conversion data output unit is configured to perform a coordinate transformation on the mesh data based on the coordinate system and output the transformed mesh data to the conversion module. The conversion module further includes a second display control unit for displaying the mesh data received by the data input unit on the display unit. The data input unit is configured to read the mesh data after the coordinate transformation output by the transformation data output unit. The reverse engineering system is characterized in that the second display control unit is configured to reproduce the display state of the mesh data in the analysis module on the display unit by displaying the mesh data after the coordinate transformation on the display unit.

11. In the reverse engineering system according to claim 9, The first display control unit is: A mesh data display area for displaying the aforementioned mesh data, A measurement setting area that accepts the specification of the shape of the geometric elements extracted by the element extraction unit and the specification of the method for calculating the dimensions calculated by the calculation unit, A conversion area specification area that accepts the specification of a conversion area to be executed by the setting unit, A reverse engineering system characterized by displaying an output instruction icon on the display unit that accepts instructions for outputting the mesh data to a conversion module.

12. In the reverse engineering system according to claim 11, The second display control unit, A CAD conversion target data display area that displays the mesh data received by the data input unit, A reverse engineering system characterized by displaying a conversion instruction icon on the display unit that accepts a conversion instruction to convert mesh data displayed in the CAD conversion target data display area into CAD data.

13. In the reverse engineering system according to claim 12, The reverse engineering system is characterized in that, when the second display control unit detects that the output instruction icon has been operated, it causes the display unit to display a first conversion window including the CAD conversion target data display area and the conversion instruction icon.

14. In the reverse engineering system according to claim 13, The reverse engineering system is characterized in that, when the second display control unit detects that the output instruction icon has been operated, it displays a second conversion window, which includes the CAD conversion target data display area and the conversion instruction icon, on the display unit in addition to the first conversion window.

Images