Three-dimensional shape measurement system

Abstract

To provide a three-dimensional shape measurement system that can easily remove unnecessary data from a picked-up image of an object even when a position and a size of the object change.SOLUTION: A three-dimensional shape measurement system 10 comprises a camera 32 that picks up an image of an object 40 to acquire a basic image 42, and a controller 12. The controller 12 stores in advance three-dimensional shape data of a plurality of known structures, identifies one or more structures from among the plurality of known structures as one or more reference structures 52, calculates one or more extruded areas 56 obtained by extruding the one or more reference structures 52 in a predetermined extrusion direction on the basis of the three-dimensional shape data, on the basis of the one or more extruded areas 56 and the basic image 42, creates an image 44 for operation obtained by partially trimming the basic image 42, and measures a shape of the object 40 on the basis of the image 44 for operation.SELECTED DRAWING: Figure 1

Description

Technical Field

[0004] , , , , , , , , ,

[0005] , , , , , ,

[0001] This specification discloses a three-dimensional shape measurement system that measures the shape of an object based on an image of the object taken.

Background Art

[0002] Conventionally, a technique for measuring the shape of an object based on an image of the object taken has been known.

[0003] For example, Patent Document 1 discloses a technique for irradiating a pattern light onto an object placed on a stage, imaging the object with a camera, and generating point cloud data representing the three-dimensional shape of the object based on the obtained image. Here, usually, an image taken with a camera also includes an unnecessary area other than the object. When generating point cloud data based on an image including such an unnecessary area, not only does the amount of calculation increase, but there is also a possibility that the shape of the object is measured incorrectly. Therefore, in Patent Document 1, region information for discriminating between an effective region and an invalid region is stored in advance for the space on the stage, and only the point cloud data corresponding to the effective region is generated. By adopting such a configuration, the amount of calculation required for measuring the shape of the object can be reduced.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, the technology described in Patent Document 1 can only be used when the size of the object and its position relative to the camera are always within a certain range, because the effective area is constant. On the other hand, some three-dimensional shape measurement systems are used when the position and size of the object are changed as needed. For example, a three-dimensional shape measurement system may be incorporated into a machine tool. In this case, the three-dimensional shape measurement system measures the shape of workpieces, tools, etc., which are the objects. In machine tools, various types of products are processed. Each time the type of product to be manufactured is changed, the position and size of the workpieces, tools, etc. change significantly. Also, even during the processing of a single workpiece, the position of the workpieces, tools, etc. changes according to the progress of processing. In such cases, the technology described in Patent Document 1 cannot be used, and it was difficult to remove unnecessary areas from the image captured of the object (workpiece, etc.).

[0006] Therefore, this specification discloses a three-dimensional shape measurement system that can easily remove unnecessary data from images of an object, even when the object's position or size changes. [Means for solving the problem]

[0007] The three-dimensional shape measurement system disclosed herein comprises a camera that images an object and acquires a base image, and a controller that measures the shape of the object, wherein the controller pre-stores three-dimensional shape data of a plurality of known structures, identifies one or more structures from the plurality of known structures as one or more reference structures, calculates one or more extrusion regions by extruding each of the one or more reference structures in a predetermined extrusion direction based on the three-dimensional shape data, generates a calculation image by partially cropping the base image based on the one or more extrusion regions and the base image, and measures the shape of the object based on the calculation image.

[0008] In this case, the one or more reference structures may include structures that move or change their orientation in conjunction with the object.

[0009] Furthermore, the controller is configured to instruct the camera to acquire a plurality of base images of the object under different imaging conditions, and to generate the calculation image corresponding to each of the plurality of base images. The controller may also be configured not to change the one or more reference structures and the one or more extrusion regions used to generate the calculation image, even if the imaging conditions of the base images are changed.

[0010] Furthermore, the imaging direction of the base image may always be parallel to or perpendicular to one of the one or more extrusion directions.

[0011] The controller may be configured such that, when there is a single reference structure, it identifies the extruded region of the reference structure as the target region, and when there are multiple reference structures, it identifies the AND region or OR region of the extruded region of each of the multiple reference structures as the target region, and generates the calculation image by trimming the portion other than the target region from the base image.

[0012] Furthermore, the controller may be configured to identify the target region as the logical OR region of a plurality of extrusion regions whose extrusion directions are parallel to each other, and the logical AND region of a plurality of extrusion regions whose extrusion directions are orthogonal to each other.

[0013] Furthermore, the one or more reference structures may include an object mounting device to which the object is attached, and the direction in which the object is attached to the object mounting device may be defined as the pushing direction of the object mounting device.

[0014] In this case, the controller may be configured to generate three-dimensional shape data of the object based on the calculation image, and to pre-store attributes corresponding to the characteristics of the object to be attached to the object mounting device in association with the object mounting device, and when the object mounting device is selected as one of the reference structures, to assign the attributes associated with the object mounting device to the three-dimensional shape data of the object. [Effects of the Invention]

[0015] According to the three-dimensional shape measurement system disclosed herein, since a calculation image is generated based on the extrusion region of a reference structure, unnecessary data can be easily removed from the image of the object even when the position or size of the object changes. [Brief explanation of the drawing]

[0016] [Figure 1] This is a block diagram showing the configuration of a three-dimensional shape measurement system. [Figure 2] This is a block diagram showing the configuration of a three-dimensional shape measurement system. [Figure 3] This diagram shows the process of generating model data for an object. [Figure 4] This is a diagram showing an example of the object. [Figure 5] This figure shows the first basic image obtained when the object is photographed from the direction of arrow A. [Figure 6] This diagram shows the process of calculating the first calculated image from the first base image. [Figure 7] This diagram shows the process of calculating the second calculated image from the second base image. [Figure 8A] This figure shows an example of a structural table for a machining center. [Figure 8B] This figure shows an example of a structural table for a turret lathe. [Figure 9] This figure shows how calculation images are generated when there are multiple reference structures. [Figure 10]It is a flowchart showing the flow of a process for generating model data of an object. [Figure 11] It is a flowchart showing the flow of a process for generating a calculation image.

Embodiments for Carrying Out the Invention

[0017] Hereinafter, the configuration of the three-dimensional shape measurement system 10 will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the three-dimensional shape measurement system 10. In the example of FIG. 1, the three-dimensional shape measurement system 10 is used in combination with a machine tool 80.

[0018] The machine tool 80 performs predetermined processing on the workpiece 90 in order to manufacture a product. The type of such a machine tool 80 is not particularly limited. For example, the machine tool 80 may be a press device that presses the workpiece 90, a painting device that paints the workpiece 90, or a cutting machine that performs cutting processing on the workpiece 90. In FIG. 1, the machine tool 80 is a machining center having a translatable table 86. This machine tool 80 includes a spindle head 94 and a table 86.

[0019] Such a machine tool 80 includes a numerical control device 82. The numerical control device 82 analyzes a machining program (also referred to as an “NC program”) and generates numerical information composed of numbers and codes, such as a tool path for the workpiece and the steps of operations required for machining. The numerical control device 82 controls the drive of the machine tool 80 based on this numerical information. Such a numerical control device 82 is physically a computer including a processor and a memory. The machine tool 80 further includes an operation panel 84 that presents information to the operator and receives commands from the operator. The numerical control device 82 can communicate with the controller 12 of the three-dimensional shape measurement system 10 by wired communication or wireless communication.

[0020] The three-dimensional shape measurement system 10 generates point cloud data 46 and three-dimensional shape data 48 (e.g., CAD data) that represent the shape of the object 40. Hereinafter, the three-dimensional shape data will be abbreviated as "model data". Here, the object 40 is not particularly limited as long as it is an item that can be imaged by the imaging unit 30 described later. Therefore, the object 40 may be, for example, a workpiece 90, a tool 92, a jig, etc. The model data 48 of the object 40 is sent to the numerical control device 82 and used for various simulations and judgments. For example, if the object 40 is a workpiece 90, the numerical control device 82 performs interference checks between the tool 92 and the workpiece 90, generates a path for the tool 92, determines the machining accuracy, determines whether the shape of the object 40 matches a reference shape, and determines whether the object 40 is in a predetermined position, etc., based on the model data 48 of the workpiece 90. Furthermore, if the object 40 is a tool 92, the numerical control device 82 performs tasks such as determining wear on the tool 92, determining the type of tool 92, and determining the quality of the tool 92's mounting condition, based on the model data 48 of the tool 92.

[0021] The three-dimensional shape measurement system 10 includes a controller 12 and an imaging unit 30. The imaging unit 30 images the object 40 and acquires a base image 42 according to imaging conditions 70 transmitted from the controller 12. The imaging conditions 70 are the conditions for acquiring the base image 42, and include, for example, the imaging direction, imaging position, imaging magnification, etc. The imaging unit 30 transmits the base image 42 to the controller 12.

[0022] The controller 12 controls the imaging process by the imaging unit 30 and generates point cloud data 46 and model data 48 of the object 40 based on the base image 42 in order to understand the shape of the object 40. The generated model data 48 is transmitted to the numerical control device 82. As will be explained in detail later, the controller 12 also obtains machining information 66 from the numerical control device 82. The machining information 66 includes the control status of known structures provided on the machine tool 80. That is, the machine tool 80 includes multiple structures whose shapes are known, such as the table 86 and the spindle head 94. Some of these known structures change position and orientation as machining progresses. The machining information 66 includes information on the position and orientation of these known structures.

[0023] Next, the configuration of the three-dimensional shape measurement system 10 will be described in more detail. Figure 2 is a block diagram showing the configuration of the three-dimensional shape measurement system 10. As shown in Figure 2 and as described above, the three-dimensional shape measurement system 10 includes a controller 12 and an imaging unit 30.

[0024] The controller 12 is physically a computer comprising a processor 14, memory 16, communication interface 22, and user interface device 24. This "computer" also includes microcontrollers that integrate a computer system into a single integrated circuit. Furthermore, the processor 14 refers to a processor in a broad sense, including general-purpose processors (e.g., CPU: Central Processing Unit, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.).

[0025] The memory 16 is a device for storing various types of data and includes both a main memory that the processor 14 accesses directly and an auxiliary memory that the processor 14 accesses via an input / output channel. The main memory includes, for example, semiconductor memory. The auxiliary memory includes, for example, semiconductor memory or magnetic memory.

[0026] Memory 16 stores the model DB 18 and the structure table 20. The model DB 18 is a database that stores model data for each of several known structures. Known structures are not particularly limited as long as their shape is known in advance. Therefore, known structures may include, for example, at least one of a table 86, a spindle head 94, a vise 88, a cover, and a fixed base. Also, if the machine tool 80 is a lathe or a multi-tasking machine, known structures may include, in place of or in addition to the above, at least one of a work spindle, a turret, a tool post, and a tailstock. The controller 12 may generate the model data for these known structures, or another computer (e.g., a numerical control device 82) may generate it. Model data generated by another computer is transmitted to the controller 12 via the communication I / F 22 and registered in the model DB 18. The structure table 20 is a table 86 that records information for each of several known structures. The information recorded in this structure table 20 will be described in detail later.

[0027] The communication interface 22 transmits and receives data to and from other external electronic devices via wired or wireless communication. For example, the communication interface 22 communicates with the numerical control unit 82 and the imaging unit 30. The user interface (UI) device 24 presents various information to the operator and receives instructions from the operator. Such a UI device 24 includes, for example, output devices such as a display and a speaker, and input devices such as a keyboard, mouse, microphone, and touch panel. In this example, the UI device 24 is described as a component of the controller 12, but some or all of the UI device 24 may be configured as a completely separate device from the controller 12. For example, some functions of the UI device 24 may be implemented as a user interface of an information terminal (e.g., a smartphone) owned by the operator and capable of communicating with the controller 12. Also, the controller 12 does not need to be a single computer, but may be configured by combining multiple physically separated computers.

[0028] The imaging unit 30 images the object 40 located inside the machining chamber of the machine tool 80. As shown in Figure 2, the imaging unit 30 includes a camera 32 and a light source 34. The camera 32 images the object 40 and acquires a base image 42. The camera 32 transmits the base image 42 to the controller 12. There may be one camera 32 or multiple cameras. The light source 34 illuminates the object 40. The light source 34 may be a light that simply illuminates the object 40, or it may be a projector that illuminates the object 40 with a predetermined pattern of light.

[0029] Next, the generation of model data 48 of the object 40 by the three-dimensional shape measurement system 10 will be explained with reference to Figure 3. When generating model data 48 of the object 40, the controller 12 drives the imaging unit 30 to acquire a basic image 42 of the object 40. At this time, imaging is performed multiple times by changing the relative position and angle of the object 40 with respect to the camera 32 so that a basic image 42 is obtained for some or all of the surfaces of the object 40. In order to change the relative position and angle of the object 40 with respect to the camera 32, the object 40 may be moved, the camera 32 may be moved, or both the object and the camera 32 may be moved. The imaging unit 30 may also have a dedicated movable mechanism. The imaging unit 30 may also be attached to a movable part of the machine tool 80 (for example, the spindle head 94 in a machining center, the tool post in a lathe, etc.). In this case, the relative position and angle of the imaging unit 30 with respect to the object 40 can be changed by moving the movable part.

[0030] The controller 12 partially crops the obtained base image 42 to generate a calculation image 44. The upper part of Figure 3 shows two calculation images 44a and 44b extracted from the two base images 42. The procedure for generating this calculation image 44 will be described later. Next, the controller 12 generates point cloud data 46 of the object 40 based on the calculation image 44. In the example in Figure 3, the controller 12 obtains a calculation image 44a of the object 40 taken from a first direction, and a calculation image 44b of the object 40 taken from a second direction different from the first direction. The controller 12 generates point cloud data 46a and 46b of the object 40 from these two calculation images 44a and 44b, respectively. The controller 12 combines the multiple point cloud data 46a and 46b, and then generates model data 48 based on the combined point cloud data 46.

[0031] The point cloud data 46 is calculated using the light section method, phase shift method, trigonometry, stereometry method, or a combination thereof. When using the light section method or phase shift method, the light source 34 illuminates the object 40 with patterned light suitable for these methods. When using the trigonometry method, the imaging unit 30 images a single object 40 in parallel using two or more cameras 32 that are spaced apart from each other.

[0032] Furthermore, the procedure for generating model data 48 from point cloud data 46 can utilize prior art, so a detailed explanation is omitted here. Also, in the above example, model data 48 is generated after combining multiple point cloud data 46a and 46b, but this order can be reversed. For example, surface data may be generated based on each of the multiple point cloud data 46a and 46b, and then the multiple surface data may be combined to generate model data 48.

[0033] As mentioned above, in this example, the base image 42 is partially cropped to generate a calculation image 44, and point cloud data 46 is generated based on this calculation image 44. The reason for generating the calculation image 44 in this way is to prevent erroneous measurements and to reduce the amount of computation. In other words, the base image 42 acquired by the camera 32 usually contains many images other than the object 40. If point cloud data 46 is calculated directly from such a base image 42, there is a possibility that the shape of the object 40 will be measured incorrectly. Also, in this case, it is necessary to calculate points in areas unrelated to the object 40, which increases the amount of computation. Therefore, in this example, a calculation image 44 is generated by cropping the unnecessary parts of the base image 42.

[0034] The generation of this calculation image 44 will be explained below. In this example, the calculation image 44 is generated using model data of a known structure. Figures 4 to 7 are schematic diagrams showing the generation process of the calculation image 44.

[0035] In this example, a specific known structure is designated as the reference structure 52, and a calculation image 44 is generated based on the extrusion region 56 obtained by extruding this reference structure 52 in a predetermined extrusion direction. That is, as described above, the machine tool 80 is equipped with many known structures such as a table 86 and a vise 88. When measuring the shape of the object 40, the operator designates one or more of these known structures as the reference structure 52. The controller 12 calculates the extrusion region 56 obtained by extruding the reference structure 52 in a predetermined extrusion direction. Then, the obtained extrusion region 56 is superimposed on the base image 42, the area inside the extrusion region 56 is left as the target region 60, and the area outside the extrusion region 56 is trimmed as an unnecessary area. The image obtained by this trimming becomes the calculation image 44.

[0036] For example, as shown in Figure 4, a vise 88 is fixed on a table 86, and a workpiece 90, which is the object 40, is attached to the vise 88. The table 86 translates as machining progresses. This movement of the table 86 is controlled by a numerical control device 82. The vise 88 is manually fixed to the table 86. The relative position of the vise 88 with respect to the table 86 is measured in advance by a sensor (not shown) provided on the machine tool 80. The machining information 66 sent from the numerical control device 82 to the controller 12 includes information on the position and orientation of the table 86, and information on the fixing position of the vise 88.

[0037] Consider the case shown in Figure 4, where the vise 88 is selected as the reference structure 52 and the direction of arrow A is identified as the extrusion direction. In this case, the controller 12 instructs the imaging unit 30 to use a direction parallel to or perpendicular to the extrusion direction as the imaging direction. Figure 5 shows the first base image 42a obtained when the workpiece 90, which is the object 40, is imaged from the direction of arrow A. As is clear from Figure 5, in this case, the first base image 42a includes many images other than the workpiece 90. In order to improve the resolution of the base image 42, or to use a wider area than the field of view of one shot as the base image, the object 40 may be imaged multiple times from the same direction but from different positions, and the resulting images may be combined and treated as a single base image 42.

[0038] The controller 12 calculates an extruded region 56 by extruding the reference structure 52, which is a vise 88, in the extrusion direction (i.e., the direction of arrow A) in order to remove unnecessary parts from the first base image 42a. Then, it superimposes the obtained extruded region 56 onto the first base image 42a. Figure 6 shows the first base image 42a and the extruded region 56 superimposed. The controller 12 identifies the area inside the extruded region 56 as the target region 60 and the area outside the extruded region 56 as the unnecessary region. In Figure 6, the area with cross-hatching is the unnecessary region. The controller 12 calculates an image in which only the target region 60 is extracted from the first base image 42a, in other words, an image in which the unnecessary region has been trimmed from the first base image 42a, as the first calculation image 44a. Subsequently, it generates point cloud data 46 of the object 40 based on this first calculation image 44a.

[0039] The same processing is applied to another base image 42 obtained by imaging the workpiece 90 from a different direction. For example, Figure 7 shows a second base image 42b obtained when the workpiece 90 is imaged from the direction of arrow B. In this case as well, the controller 12 calculates the extrusion region 56 obtained by extruding the vise 88 in the extrusion direction (i.e., the direction of arrow A), and superimposes the obtained extrusion region 56 onto the second base image 42b. Then, it identifies the area inside the extrusion region 56 as the target region 60 and the area outside the extrusion region 56 as the unnecessary region. In Figure 7, the area with cross-hatching is the unnecessary region. The controller 12 calculates an image in which only the target region 60 is extracted from the second base image 42b as the second calculation image 44b. Subsequently, the controller 12 generates point cloud data 46 based on this second calculation image 44b.

[0040] As is clear from the above explanation, in this example, a calculation image 44 is calculated by partially cropping the base image 42, and point cloud data 46 is generated based on this calculation image 44. This reduces the amount of computation required to calculate the point cloud data 46 and also reduces the risk of mismeasuring objects other than the target object 40 as the target object 40.

[0041] By the way, when trimming unnecessary parts of the base image 42, other techniques can also be considered. For example, in Patent Document 1, the region to be extracted from the base image 42 as a calculation image 44 is stored in advance. However, in the case of the technique in Patent Document 1, since the stored region is immutable, there is a problem that the calculation image 44 cannot be properly extracted if the position or size of the object 40 changes significantly. On the other hand, in this example, since the operator specifies the reference structure 52 according to the object 40, the target region 60 can be properly set even if the position or size of the object 40 changes significantly.

[0042] Another possible technique involves the operator observing the base image 42 and using an input device such as a mouse to specify a rectangular area that will become the target region 60. In the following, this technique in which the operator directly specifies the target region 60 will be referred to as the "direct specification method." With the direct specification method, the target region 60 can be appropriately set even if the position or size of the object 40 changes significantly. However, with the direct specification method, the operator must specify the target region 60 for each of the multiple base images 42, which is time-consuming. For example, consider the case where the first calculation image 44a and the second calculation image 44b in Figures 6 and 7 are specified using the direct specification method. In this case, the operator must specify the target region 60 twice, which increases the operator's workload.

[0043] On the other hand, in the case of a technique that extracts the extrusion region 56 of the reference structure 52 as the target region 60, as in this example, the effort required of the operator to calculate multiple calculation images 44 from multiple base images 42 can be reduced. For example, consider the case of calculating the first calculation image 44a and the second calculation image 44b in Figures 6 and 7. In this example, the operator only needs to specify the reference structure 52 (i.e., the vise 88) and the extrusion direction (i.e., the direction of arrow A) at the beginning, and even if the imaging direction changes, the appropriate calculation image 44 can be calculated. As a result, according to this example, the effort required of the operator can be significantly reduced compared to the direct specification method.

[0044] Furthermore, in this example, a structure that moves together with the object 40 is selected as the reference structure 52. That is, in the example in Figure 4, the vise 88 to which the workpiece 90 is attached is selected as the reference structure 52. Therefore, even if the position and orientation of the workpiece 90 (i.e., the object 40) changes due to changes in the position and orientation of the table 86, the relative position of the vise 88 (i.e., the reference structure 52) to the workpiece 90 does not change. In this case, even if the position and orientation of the workpiece 90 change, the calculation image 44 can be extracted appropriately.

[0045] In particular, with machine tools 80, the shape of a single workpiece 90 may be measured multiple times depending on the progress of machining. For example, the shape of a single workpiece 90 may be measured before the start of machining, after rough machining is completed, and after finish machining is completed. At this time, the position of the workpiece 90 may change depending on the timing. If a structure that moves together with the object 40 is selected as the reference structure 52, even if the position of the workpiece 90 changes, the calculation image 44 can be appropriately extracted without having to specify the reference structure 52 again. As a result, the operator's workload can be greatly reduced.

[0046] Next, we will explain how to determine the extrusion direction. As repeatedly explained, in this example, the calculation image 44 is extracted based on the extrusion region 56 obtained by extruding the reference structure 52 in the extrusion direction. Here, the extrusion direction may be specified by the operator or may be predetermined. For example, in the case of an object mounting device to which an object 40 is attached, the mounting direction of the object 40 may be predetermined as the extrusion direction. Here, if the object 40 is a workpiece, the vise 88, table 86, and workpiece spindle, etc., correspond to the object mounting device. For example, since the workpiece 90 is attached to the vise 88 from direction A, direction A may be predetermined as the extrusion direction of the vise 88. Similarly, in a lathe, since the workpiece 90 is attached to the workpiece spindle from the direction of the workpiece rotation axis, the direction of the workpiece rotation axis may be predetermined as the extrusion direction of the workpiece spindle.

[0047] Furthermore, if the object 40 is a tool 92, the spindle head 94 in a machining center or the turret in a turret lathe are examples of object mounting devices. However, a turret usually has multiple tool mounting points. In other words, there are multiple mounting directions for the object 40 (tool) to the turret. In such cases, the controller 12 may present the operator with multiple mounting directions as candidates for the extrusion direction. The operator selects one extrusion direction from among the multiple candidates.

[0048] Furthermore, in another form, the operator may specify the extrusion direction. The procedure for specifying the extrusion direction is not particularly limited. For example, the operator may specify the coordinates of two points, and the controller 12 may identify the direction parallel to the line passing through these two points as the extrusion direction. Alternatively, in another form, the operator may specify a particular plane, and the controller 12 may identify the direction perpendicular to the specified plane as the extrusion direction.

[0049] The structure table 20 records known structures and their corresponding extrusion directions. Figure 8A shows an example of the structure table 20 for a machining center, and Figure 8B shows an example of the structure table 20 for a turret lathe. When a reference structure 52 is specified by the operator, the controller 12 compares the reference structure 52 with the structure table 20 and determines the extrusion direction of the reference structure 52. If a single direction is recorded as the extrusion direction of the reference structure 52 in the structure table 20, the controller 12 automatically determines that single direction as the extrusion direction. If multiple directions are recorded as extrusion directions in the structure table 20, the controller 12 presents these multiple directions to the operator as candidates for the extrusion direction. Furthermore, if "operator specified" is recorded as the extrusion direction in the structure table 20, the controller 12 prompts the operator to specify the extrusion direction.

[0050] Next, the designation of the reference structure 52 will be explained. As mentioned above, in this example, the operator designates one or more reference structures 52. The form of designation of this reference structure 52 is not particularly limited. Therefore, for the designation of the reference structure 52, images of known structures (e.g., illustrations or CG images) may be displayed on the display of the UI device 24. For example, when the controller 12 receives a designation of a reference structure 52 from the operator, it may acquire an image of the object 40 and superimpose images of known structures that can be designated as reference structures 52 onto this image and display them on the display. In this case, the operator only needs to select the image of the known structure they want to use as the reference structure 52. With this configuration, the operator can visually determine a known structure suitable for the reference structure 52. Of course, other designation procedures may be adopted instead of this designation procedure. For example, the operator may input the file path of the model data file of the known structure they want to use as the reference structure 52.

[0051] Furthermore, the specified reference structure 52 is not limited to one, but may be multiple. If multiple reference structures 52 are selected, the controller 12 takes the logical OR or logical AND of the multiple extrusion regions 56 and identifies the region obtained by this logical operation as the target region 60.

[0052] More specifically, if the extrusion regions 56 of the multiple reference structures 52 are parallel to each other, the controller 12 identifies the logical OR region of the multiple extrusion regions 56 as the target region 60. If the extrusion regions 56 of the multiple reference structures 52 are not parallel to each other, the controller 12 identifies the logical AND region of the multiple extrusion regions 56 as the target region 60.

[0053] For example, consider the case shown in Figure 9, where a workpiece 90 is placed on a table 86, and several covers 96a to 96d are arranged to the side of the workpiece 90. The extrusion direction of the table 86 is the Z direction (i.e., perpendicular to the plane of the paper), and the extrusion direction of the covers 96a to 96d is the Y direction (i.e., vertical direction of the plane of the paper). In this case, suppose the table 86, cover 96b, and cover 96c are selected as reference structures 52 in order to measure the shape of the workpiece 90.

[0054] In this case, the controller 12 groups multiple reference structures 52 based on their extrusion direction such that reference structures 52 whose extrusion directions are parallel to each other belong to the same group. Next, the controller 12 calculates the logical OR of the extrusion regions 56 of the reference structures 52 belonging to the same group. Subsequently, the controller 12 calculates the logical AND of the extrusion regions 56 of one or more groups and sets the resulting region as the target region 60.

[0055] In the example in Figure 9, the controller 12 groups the two covers 96b and 96c into a first group and the table 86 into a second group. Next, the controller 12 calculates the logical OR region 57 of the extrusion region 56b of cover 96b and the extrusion region 56c of cover 96c. Then, the controller 12 calculates the logical AND region of this logical OR region 57 and the extrusion region 56t of table 86 as the target region 60. In the lower part of Figure 9, the area that is not hatched is the target region 60. In this example, the controller 12 automatically determines whether to calculate a logical OR or a logical AND based on the extrusion direction, but the operator may make this determination. Therefore, if the operator wishes, the logical OR of two extrusion regions 56 with different extrusion directions may be calculated.

[0056] Incidentally, the reference structure 52 includes an object installation device to which the object 40 is attached. Attribute information to be assigned to the model data 48 of the object 40 may be associated with this object installation device in advance, or according to the operator's instructions.

[0057] In other words, the three-dimensional shape measurement system 10 generates model data 48 of the object 40. This model data 48 is used for simulations such as interference checks. In order to perform such simulations properly, it is necessary that the model data 48 of the object 40 is pre-assigned various attribute information, such as whether or not it is machinable. However, it was cumbersome for the operator to set this attribute information each time the model data 48 was generated.

[0058] Therefore, attribute information may be pre-associated with the object mounting device. When an object mounting device is selected as a reference structure 52, the controller 12 automatically adds attribute information to the model data 48 generated using the reference structure 52 (i.e., the object mounting device). Such attribute information may be pre-recorded in, for example, the structure table 20. In the examples of Figures 8A and 8B, whether or not machining is possible is recorded as attribute information. Object mounting devices to which the workpiece 90 is attached, such as the table 86 and the vise 88, are recorded as "possible" for machining. Therefore, when the table 86 is selected as the reference structure 52, the controller 12 automatically adds attribute information indicating "possible" for machining to the model data 48 generated using the reference structure 52 (table 86). Also, in the example of Figure 8, object mounting devices to which the tool 92 is attached, such as the spindle head 94 and the turret, are recorded as "not possible" for machining. Therefore, when the spindle head 94 is selected as the reference structure 52, the controller 12 automatically adds the attribute information "cannot be cut" to the model data 48 generated using the reference structure 52 (spindle head 94). This configuration further reduces the operator's workload. Note that this attribute information does not need to be pre-recorded in the structure table 20; the operator may specify it.

[0059] Next, the process flow for generating model data 48 of the object 40 will be explained with reference to Figures 10 and 11. When it is desired to generate three-dimensional shape data 48 of the object 40, the controller 12 determines the reference structure 52 and the extrusion direction (S10). The reference structure 52 is specified by the operator. The specified reference structure 52 may be one or more. The extrusion direction may be automatically determined by the controller 12 or specified by the operator.

[0060] Next, the controller 12 determines the imaging conditions 70 for the object 40 based on the determined extrusion direction (S12). Here, the controller 12 determines one imaging condition 70 for each imaging direction. Therefore, when imaging the object 40 from five imaging directions, the controller 12 determines five imaging conditions 70. Also, usually there are multiple imaging directions, and therefore multiple imaging conditions 70. That is, in order to grasp the overall shape of the object 40, it is necessary to image the object 40 multiple times while changing the imaging direction. Therefore, in step S12, the controller 12 determines multiple imaging conditions 70 with different imaging directions. The controller 12 also determines directions parallel to and perpendicular to the extrusion direction as imaging directions.

[0061] The imaging conditions 70 include at least the imaging direction, and may further include at least one of the imaging position, imaging magnification, and number of imaging cycles. As described above, the reference structure 52 may be imaged multiple times for a single imaging direction. For example, the reference structure 52 may be imaged multiple times while translating the camera 32, keeping the optical axis of the camera 32 parallel to direction A. The obtained multiple images may then be combined to form a single base image 42. Therefore, the controller 12 may determine the imaging position, imaging magnification, and number of imaging cycles as imaging conditions 70 for each imaging direction so that an image of the entire reference structure 52 can be obtained. If the reference structure 52 is immobile and its position is fixed, the position of the reference structure 52 is recorded in the model DB 18. On the other hand, if the position of the reference structure 52 changes, for example, if the reference structure 52 is the spindle head 94, the position of the reference structure 52 is determined based on the machining information 66 sent from the numerical control device 82.

[0062] Next, the controller 12 sets the parameter i to its initial value of "1" (S14). Then, the controller 12 instructs the imaging unit 30 to perform imaging based on the i-th imaging condition 70 and acquires the i-th base image 42 (S16). As repeatedly stated, the i-th base image 42 may be acquired in a single imaging, or it may be generated by combining two or more images. Once the i-th base image 42 is obtained, the controller 12 uses the reference structure 52 and extrusion direction determined in step S10 to extract the i-th calculation image 44 from the base image 42 (S18).

[0063] Specifically, as shown in Figure 11, the controller 12 identifies the position and shape of the reference structure 52 in machine coordinates based on the model data of the reference structure 52 and the machining information 66 (S30). Next, based on the imaging conditions 70, the position of the reference structure 52 is converted from machine coordinates to camera coordinates (S32).

[0064] If the camera coordinates of the reference structure 52 can be obtained, the controller 12 identifies the extrusion region 56 in the base image 42 for each of one or more reference structures 52 (S34). Further, the controller 12 identifies the target region 60 based on the calculated one or more extrusion regions 56 (S36). Specifically, when the extrusion region 56 is single, the controller 12 identifies the extrusion region 56 as the target region 60. On the other hand, when there are a plurality of extrusion regions 56, the controller 12 obtains the logical sum or logical product of the plurality of extrusion regions 56 based on the extrusion direction, and identifies the region obtained by the logical operation as the target region 60.

[0065] If the target region 60 can be identified, the controller 12 extracts only the target region 60 from the base image 42 as the calculation image 44 (S38). If the calculation image 44 can be extracted, the process proceeds to step S20 in FIG. 10. In step S20, the controller 12 calculates the point cloud data 46 of the object 40 from the i-th calculation image 44. The calculated point cloud data 46 is temporarily stored in the memory 16 together with the imaging condition 70.

[0066] Next, the controller 12 compares the parameter i with the numerical value imax (S22). Note that imax is the imaging condition 70 specified in step S12, and thus the number of imaging directions. Imax is usually a natural number of 2 or more. When i < imax, after incrementing the parameter i (S24), steps S16 to S22 are repeated again. On the other hand, when i ≧ imax, the controller 12 synthesizes the imax pieces of point cloud data 46 temporarily stored in the memory 16 (S26), and generates the model data 48 of the object 40 (S28).

[0067] As is clear from the above explanation, in this example, even if the imaging direction of the base image 42 is changed, the reference structure 52 and the extrusion region 56 used to generate the calculation image 44 are not changed. Therefore, in step S10, the operator only needs to specify the reference structure 52, or both the reference structure 52 and the extrusion direction, thereby reducing the effort required of the operator to generate multiple calculation images 44.

[0068] However, the configurations described so far are all examples, and other configurations may be modified as appropriate, as long as one or more reference structures 52 are extruded in the corresponding extrusion direction to calculate one or more extrusion regions 56, and a calculation image 44 is generated by partially trimming the base image 42 based on these one or more extrusion regions 56. For example, a reference structure 52 may be specified each time the imaging direction changes. Also, in the above example, the inside of the extrusion region 56 is identified as the target region 60. However, the inside of the extrusion region 56 may be used as the trimming region, and the outside of the extrusion region 56 may be identified as the target region 60.

[0069] Furthermore, the three-dimensional shape measuring system 10 may be used in combination with other devices, not just the machine tool 80. The three-dimensional shape measuring system 10 may also be used independently, without being combined with any other device. In the description above, the three-dimensional shape measuring system 10 has been described as a separate device independent of the machine tool 80. However, the three-dimensional shape measuring system 10 may be integrated with the machine tool 80. For example, the numerical control device 82 and the operation panel 84 of the machine tool 80 may constitute the controller 12 of the three-dimensional shape measuring system 10, and the imaging unit 30 may be attached to a holding device provided on the machine tool 80 instead of a tool 92 or workpiece 90. [Explanation of Symbols]

[0070] 10 Three-dimensional shape measurement system, 12 Controller, 14 Processor, 16 Memory, 18 Model DB, 20 Structure table, 22 Communication I / F, 24 UI device, 30 Imaging unit, 32 Camera, 34 Light source, 40 Object, 42 Base image, 44 Image for calculation, 46 Point cloud data, 48 Three-dimensional shape data (model data), 52 Reference structure, 56 Extrusion region, 57 Logical OR region, 60 Target region, 66 Machining information, 70 Imaging conditions, 80 Machine tool, 82 Numerical control device, 84 Operation panel, 86 Table, 88 Vice, 90 Workpiece, 92 Tool, 94 Spindle head, 96 Cover.

Claims

1. A camera that captures images of an object and acquires a basic image, A controller for measuring the shape of the object, The controller is equipped with, The three-dimensional shape data of multiple known structures is stored in advance. If the operator identifies one or more structures from among the multiple known structures as one or more reference structures, Based on the three-dimensional shape data, one or more extrusion regions are calculated by extruding each of the one or more reference structures in a predetermined extrusion direction. Based on the one or more extrusion regions and the base image, a calculation image is generated by partially cropping the base image. Based on the calculation image, the shape of the object is measured. A three-dimensional shape measurement system characterized by being configured as follows.

2. A three-dimensional shape measuring system according to claim 1, A three-dimensional shape measuring system characterized in that the one or more reference structures include structures that move or change orientation in conjunction with the object.

3. A three-dimensional shape measuring system according to claim 1, The controller is configured to instruct the camera to acquire multiple base images of the object under different imaging conditions, and to generate the calculation image corresponding to each of the multiple base images. The controller is configured so as not to change the one or more reference structures and the one or more extrusion regions used to generate the calculation image, even if the imaging conditions of the base image are changed. A three-dimensional shape measurement system characterized by the following features.

4. A three-dimensional shape measuring system according to claim 1, A three-dimensional shape measurement system characterized in that the imaging direction of the base image is always parallel to or perpendicular to one of the one or more extrusion directions.

5. A three-dimensional shape measuring system according to claim 1, The aforementioned controller, If there is only one reference structure, the extruded region of the reference structure is identified as the target region. If there are multiple reference structures, the AND or OR region of the extrusion area of ​​each of the multiple reference structures is identified as the target region. The part of the base image other than the target region is trimmed to generate the calculation image. A three-dimensional shape measurement system characterized by being configured in such a way.

6. A three-dimensional shape measuring system according to claim 5, The three-dimensional shape measuring system is characterized in that the controller is configured to identify the target region as the logical OR region of a plurality of extrusion regions whose extrusion directions are parallel to each other, and the logical AND region of a plurality of extrusion regions whose extrusion directions are orthogonal to each other.

7. A three-dimensional shape measuring system according to claim 1, The one or more reference structures include an object installation device to which the object is attached, The direction in which the object is attached to the object mounting device is defined as the pushing direction of the object mounting device. A three-dimensional shape measurement system characterized by the following features.

8. A three-dimensional shape measuring system according to claim 7, The controller is configured to generate three-dimensional shape data of the object based on the calculation image, The aforementioned controller, The attributes corresponding to the characteristics of the object to be attached to the object mounting device are stored in advance in association with the object mounting device, When the object installation device is selected as one of the reference structures, attributes associated with the object installation device are assigned to the three-dimensional shape data of the object. A three-dimensional shape measurement system characterized by being configured in such a way.

Images