Three-dimensional shape measurement system and machine tool system

Abstract

To provide a three-dimensional shape measurement system capable of appropriately measuring shapes of various types of objects by an easier procedure than ever.SOLUTION: A three-dimensional shape measurement system 12 includes: an imaging unit 50; a storage device 40 for storing imaging conditions for each of plural combinations of quality of a material and a surface texture of an object 100 in imaging for measurement as condition information 42; and a measurement controller 30 for controlling the drive of the imaging unit 50. The measurement controller 30 identifies the quality of the material and the surface texture of the object 100, identifies the imaging conditions corresponding to the identified quality of the material and surface texture of the object 100 on the basis of the condition information 42, and makes the imaging unit 50 perform the imaging for the measurement on the identified imaging conditions so as to measure a shape of the object 100 on the basis of an obtained image 60 for the measurement.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] This specification discloses a three-dimensional shape measurement system that measures the three-dimensional shape of an object based on image data obtained by imaging the object, and a machine tool system equipped with the same. [Background technology]

[0002] In recent years, a technology has been proposed for measuring the three-dimensional shape of an object based on an image obtained by capturing an image of the object. For example, Patent Document 1 discloses a processing device including a light irradiation unit that irradiates a line light onto the object and an imaging unit that captures an image of the line light reflected by the object. The machine tool of Patent Document 1 measures the three-dimensional shape of the surface of the object based on the image captured of the line light. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-69486 A Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, in order to measure the shape of an object based on an image, it is necessary to set the imaging conditions appropriately so that the surface shape of the object can be grasped from the image. For example, when light is strongly reflected from the object, whiteout occurs in the image due to halation. In the places where such whiteout occurs, the surface shape of the object cannot be grasped, and as a result, the shape of the object cannot be measured. Also, when the amount of light irradiated to the object is insufficient and the brightness of the image is low, the surface shape of the object cannot be grasped and the shape of the object cannot be measured.

[0005] Therefore, in order to measure the shape of an object based on an image, it is necessary to appropriately set imaging conditions such as the gain of the camera. Here, imaging conditions suitable for shape measurement differ depending on the material and surface properties of the object. For example, an object made of metal is more likely to reflect light and to cause whiteout than an object made of resin or the like. In addition, even if the material is the same, an object with a small surface roughness is more likely to reflect light than an object with a large surface roughness. Therefore, in order to appropriately perform shape measurement, it is necessary to change the imaging conditions depending on the material and surface properties of the object.

[0006] Conventionally, such imaging conditions were often set by the operator, but it was difficult for the operator to specify appropriate imaging conditions. As a result, it was difficult for conventional 3D shape measurement systems to appropriately measure the shapes of various types of objects in a simple procedure.

[0007] Therefore, this specification discloses a three-dimensional shape measurement system and a machine tool system that can appropriately measure the shapes of various types of objects using simpler procedures. [Means for solving the problem]

[0008] The three-dimensional shape measurement system disclosed in this specification comprises an imaging unit having one or more cameras that images an object, a storage device that stores, as condition information, imaging conditions for measurement imaging of the object to measure the shape of the object for each of a plurality of combinations of the object's material and surface properties, and a measurement controller that controls the operation of the imaging unit, wherein the measurement controller identifies the material and surface properties of the object, identifies the imaging conditions corresponding to the identified material and surface properties of the object based on the condition information, causes the imaging unit to perform the measurement imaging under the identified imaging conditions, and measures the shape of the object based on the obtained measurement image.

[0009] In this case, the measurement controller may cause the imaging unit to image the object prior to the measurement imaging, and identify at least one of the material and surface properties of the object based on a preliminary image thus obtained.

[0010] In addition, the storage device may further store a learning model that takes the preliminary image as input and outputs at least one of the material and surface properties of the object, and the measurement controller may identify at least one of the material and surface properties of the object based on the learning model.

[0011] In addition, the imaging unit may be provided on a machine tool, and image a workpiece being machined by the machine tool as the target object, and the measurement controller may identify at least one of the material and surface properties of the target object based on at least one of a machining program for the workpiece and a machining drawing for the workpiece.

[0012] The measurement controller may also acquire information indicating at least one of the machining program and the machining drawing from a numerical control device of the machine tool.

[0013] At least one of the material and the surface texture of the object may be specified by an operator.

[0014] In addition, the imaging unit may further have one or more light sources that irradiate imaging light onto the object, and the measurement controller may perform basic imaging of the object one or more times to obtain one of the measurement images, and the imaging conditions may include at least one of the number of times the basic imaging is performed to obtain one of the measurement images, the shutter speed of the camera in each of the one or more basic imaging, the gain of the camera in each of the one or more basic imaging, and the luminance distribution of the imaging light in each of the one or more basic imaging.

[0015] The measurement controller may generate point cloud data of the object based on the measurement images, and generate three-dimensional data of the object from the point cloud data.

[0016] Furthermore, the machine tool system disclosed in this specification comprises the above-mentioned three-dimensional shape measurement system and a machine tool that performs machining on the object, and is characterized in that the machine tool performs at least one of the following based on the three-dimensional data generated by the measurement controller: checking for interference between a tool and the object, generating a path for the tool, determining the machining accuracy, determining whether the shape of the object matches a reference shape, and determining whether the object is located at a predetermined position. Effect of the Invention

[0017] According to the technology disclosed in this specification, the shapes of various types of objects can be appropriately measured with simpler procedures. [Brief description of the drawings]

[0018] [Figure 1] FIG. 1 is an image diagram showing a configuration of a machine tool system. [Diagram 2] FIG. 1 is a block diagram showing a configuration of a shape measurement system. [Diagram 3] 1 is an image diagram showing the principle of three-dimensional shape measurement of an object by a shape measurement system. [Figure 4] FIG. 1 is an image diagram showing how an HDR image is generated. [Diagram 5] FIG. 11 is a diagram illustrating an example of condition information. [Figure 6] FIG. 1 is a diagram showing an example of a processing drawing. [Figure 7] FIG. 4 is a diagram showing an example of a machining program. [Figure 8] 11 is a diagram showing an example of a correspondence relationship between a value of a common variable and a material and a surface property. FIG. [Figure 9] FIG. 13 is a diagram showing an example of a correspondence relationship between a sequence name and a material and a surface property. [Figure 10]FIG. 11 is a diagram showing another example of a machining program. [Figure 11] 10 is a flowchart showing a process flow of the shape measurement system. [Figure 12] 10 is a flowchart showing a process flow of the shape measurement system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, the configuration of a three-dimensional shape measurement system 12 (hereinafter abbreviated as "shape measurement system 12") and a machine tool system 10 incorporating the shape measurement system 12 will be described with reference to the drawings. FIG. 1 is a conceptual diagram showing the configuration of the machine tool system 10.

[0020] As shown in FIG. 1, a machine tool system 10 includes a machine tool 14 and a shape measurement system 12. The machine tool 14 performs a predetermined process on a workpiece to manufacture a processed product. The type of the machine tool 14 is not particularly limited, and for example, the machine tool 14 may be a metal processing machine that performs metal processing (e.g., cutting, pressing, etc.) on a metal workpiece. In FIG. 1, the machine tool 14 is a lathe having a spindle 14a and a tool rest 14b. In the following description, the workpiece handled by the machine tool 14 is treated as an object 100 for shape measurement. However, the object 100 for shape measurement is not limited to a workpiece and may be other members, such as a tool or a jig.

[0021] The machine tool 14 has a numerical control device 20. The numerical control device 20 is a device that analyzes a machining program 66 (also called an "NC program") and commands the tool path for the workpiece, the steps of the work required for machining, etc., by using numerical information consisting of numerical values ​​and codes to operate the machine tool 14. The numerical control device 20 checks for interference between the tool and the workpiece, generates a tool path, judges the machining accuracy, judges whether the shape of the object 100 matches a reference shape, judges whether the object 100 is at a predetermined position, etc., based on three-dimensional data 69 (e.g., CAD data, etc.) that indicates the shape of the workpiece (i.e., the object 100). The numerical control device 20 is physically a computer having a processor and a memory. The machine tool 14 is further provided with an operation panel 22 that presents information to the operator and receives commands from the operator.

[0022] Furthermore, the numerical control device 20 stores data necessary for machining the target object 100, such as a machining program 66 and a machining drawing 64. The numerical control device 20 can communicate with the measurement controller 30 by wired communication or wireless communication, and transmits the machining program 66, etc. to the measurement controller 30 as necessary. In this example, the numerical control device 20 stores the machining program 66, etc., but the numerical control device 20 may store the machining program 66, etc. in another external storage device that can communicate with the measurement controller 30. For example, a cloud data server or the like may store the machining program 66, etc., and transmit the machining program 66, etc. to the measurement controller 30.

[0023] The shape measurement system 12 measures the shape of an object 100, and includes a measurement controller 30 and an imaging unit 50. The imaging unit 50 captures an image of the object 100. An image captured by the imaging unit 50 is transmitted to the measurement controller 30 as a measurement image 60 or a preliminary image 62, which will be described later.

[0024] The measurement controller 30 specifies the imaging conditions 56 for capturing the measurement image 60, and instructs the imaging unit 50 to capture the image under the specified imaging conditions 56. In the following, the symbols for the imaging conditions will be omitted. The measurement controller 30 also generates point cloud data 74 of the object 100 based on the measurement image 60 sent from the imaging unit 50, and generates three-dimensional data 69 indicating the three-dimensional shape of the object 100 based on the point cloud data 74. The generated three-dimensional data 69 is transmitted to the numerical control device 20.

[0025] Next, a more detailed description will be given of the configuration of the shape measurement system 12. Fig. 2 is a block diagram showing the configuration of the shape measurement system 12. As shown in Fig. 2 and as described above, the shape measurement system 12 has a measurement controller 30 and an imaging unit 50.

[0026] The measurement controller 30 is physically a computer having a processor 32, a memory 34, a communication I / F 36, a UI device 38, and a storage device 40. The "computer" also includes a microcontroller in which a computer system is incorporated into a single integrated circuit. The processor 32 refers to a processor in a broad sense, and includes a general-purpose processor (e.g., a CPU: Central Processing Unit, etc.) and a dedicated processor (e.g., a GPU: Graphics Processing Unit, an ASIC: Application Specific Integrated Circuit, an FPGA: Field Programmable Gate Array, a programmable logic device, etc.). The operation of the measurement controller 30 described below may be performed not only by one processor, but also by the cooperation of multiple processors located at physically separate locations. Similarly, the memory 34 does not need to be a single physical element, and may be composed of multiple memories located at physically separate locations. The memory 34 may also include, for example, a semiconductor memory (e.g., a RAM, a ROM, etc.).

[0027] The communication I / F 36 transmits and receives data to and from other external electronic devices through wired or wireless communication. The communication I / F 36 communicates with, for example, the numerical control device 20 and the imaging unit 50. The UI device 38 presents various information to the operator and receives instructions from the operator. The UI device 38 has, for example, an output device such as a display and a speaker, and an input device such as a keyboard, a mouse, a microphone, and a touch panel. In this example, the UI device 38 is described as a component of the measurement controller 30, but a part or all of the UI device 38 may be configured as a device completely different from the measurement controller 30. For example, some functions of the UI device 38 may be realized by a user interface of an information terminal (for example, a smartphone) owned by the operator and capable of communicating with the measurement controller 30.

[0028] The storage device 40 stores various data required for shape measurement, and is physically configured, for example, as a hard disk drive, a solid state drive, or a combination of these. In this example, the storage device 40 is described as a component of the measurement controller 30, but a part or all of the storage device 40 may be configured as a separate device completely different from the measurement controller 30. For example, the storage device 40 may be a storage device provided in a cloud server capable of communicating with the measurement controller 30.

[0029] The storage device 40 stores condition information 42, reference data 44, and a learning model 45. The condition information 42 is data in which the imaging conditions for measurement imaging are recorded separately for each of a plurality of combinations of the material and surface properties of the object 100. The specific content of this condition information 42 will be described later. The reference data 44 is a plurality of image data to be compared with a preliminary image 62, which will be described later, in order to identify the material and surface properties of the object 100. The specific content of this reference data 44 will also be described later. The learning model 45 will also be described later.

[0030] The imaging unit 50 captures an image of the object 100 present in the machining chamber of the machine tool 14. As shown in Fig. 2, the imaging unit 50 has a camera 52 and a projector 54. The camera 52 captures an image of the object 100 and generates image data. The image data obtained by the camera 52 is sent to the measurement controller 30. Furthermore, the imaging conditions for capturing an image of the object 100, such as the number of times images are captured and the shutter speed, are instructed by the measurement controller 30.

[0031] The projector 54 functions as the light source 53 that irradiates the object 100 with imaging light. The projector 54 (light source 53) in this example can freely change the luminance distribution of the imaging light that irradiates the object 100. The luminance distribution of the imaging light is instructed by the measurement controller 30.

[0032] Next, the principle of three-dimensional shape measurement of the object 100 by the shape measurement system 12 will be described with reference to FIG. 3. When measuring the three-dimensional shape of the object 100, the measurement controller 30 drives the imaging unit 50 to obtain a measurement image 60. At this time, the relative attitude of the object 100 with respect to the camera 52 is changed and imaging is performed multiple times so that the measurement image 60 can be obtained for all surfaces of the object 100. In order to change the relative attitude of the object 100 with respect to the camera 52, the object 100 may be moved or the camera 52 may be moved. For example, the attitude of the object 100 with respect to the camera 52 may be changed by driving the main shaft 14a that holds the object 100.

[0033] The measurement controller 30 generates point cloud data 74 of the object 100 based on the obtained measurement image 60. The point cloud data 74 is also called a "point cloud" and is data that represents the surface shape of the object 100 with a plurality of points having X, Y, and Z coordinate values. Such point cloud data 74 can be generated using, for example, a stereo method, a phase shift method, a light section method, or the like. The stereo method is a technique in which the camera 52 is moved or two cameras 52 are used to capture the object 100 from different positions, corresponding points are searched from the parallax images, and the height is obtained by triangulation. The phase shift method is a technique in which a sine wave is projected onto the object 100 by the projector 54 while shifting the phase, and the phase is obtained from the luminance change of the image obtained by capturing the image multiple times. The light section method is a technique in which a line light is projected onto the object 100, the camera 52 captures the image, and the shape of the object 100 is measured based on the obtained image. These techniques are all conventionally known, and therefore detailed description thereof will be omitted here. When the point cloud data 74 is obtained, the measurement controller 30 generates three-dimensional data 69 of the object 100 based on the point cloud data 74. The generated three-dimensional data 69 is transmitted to the numerical control device 20 or a computer designated by the operator as necessary.

[0034] Incidentally, in order to properly generate the point cloud data 74, the object 100 must be properly captured in the measurement image 60. However, when the object 100 is simply captured by the camera 52, blackout or whiteout may occur. Blackout is a phenomenon in which a dark part of an image becomes completely black, and whiteout is a phenomenon in which a bright part of an image becomes completely white. Regardless of whether blackout or whiteout occurs, the measurement controller 30 cannot properly determine the shape of the object 100. Therefore, in this example, in order to obtain one measurement image 60, the object 100 is captured once or more under a predetermined shooting condition. Hereinafter, the imaging performed to obtain this one measurement image 60 is called "basic imaging", and the image obtained by the basic imaging is called "basic image 70". When basic imaging is performed only once to obtain one measurement image 60, the measurement controller 30 treats the obtained one basic image 70 as the measurement image 60. When basic imaging is performed two or more times, the measurement controller 30 generates a high dynamic range image 72 (hereinafter referred to as “HDR image 72 ”) from two or more basic images 70 , and handles this HDR image 72 as the image 60 for measurement.

[0035] The HDR image 72 is an image obtained by synthesizing two or more basic images 70 obtained by capturing an image of the object 100 two or more times under different capturing conditions so as to have a high luminance range. FIG. 4 is a diagram showing how the HDR image 72 is generated. In the example of FIG. 4, the object 100 is illuminated by a light (not shown) located on the rear side, and has a protrusion 102 on the front side. Also, in FIG. 4, the first basic image 70f is an image captured by adjusting the exposure to the protrusion 102, and the second basic image 70s is an image captured by adjusting the exposure to the edge 104 on the rear side. In this case, in the first basic image 70f, the entire image is bright, so that the shape of the protrusion 102 can be grasped, but the shape of the edge 104 on the rear side cannot be grasped due to whiteout. In the second basic image 70s, the entire image is dark, so that the shape of the edge 104 on the rear side can be grasped, but the shape of the protrusion 102 cannot be grasped due to blackout.

[0036] The HDR image 72 is obtained by synthesizing such a plurality of basic images 70 so as to increase the luminance range. According to the example of Fig. 4, data around the protrusion 102 of the first basic image 70f and data around the back edge 104 of the second basic image 70s are synthesized to generate one HDR image 72. Based on such an HDR image 72, the shape of the object 100 can be appropriately grasped.

[0037] Incidentally, the imaging conditions (e.g., the number of basic imaging times, the shutter speed of the camera 52, the luminance distribution of the imaging light, etc.) required to obtain an appropriate measurement image 60 differ depending on the material and surface properties of the object 100. For example, aluminum generally has a whitish silver color compared to steel materials such as S45C, and is therefore more likely to reflect light and cause whiteout. Also, even if the material is the same, an object 100 with a rough surface roughness is less likely to reflect light and cause whiteout than an object 100 with a fine surface roughness. Therefore, in order to obtain an appropriate HDR image 72, it is necessary to change the imaging conditions depending on the material and surface properties of the object 100.

[0038] Therefore, the measurement controller 30 of this example stores in advance appropriate imaging conditions for each of a plurality of combinations of the material and surface texture of the object 100 as condition information 42. Then, when measuring the shape of the object 100, the material and surface texture of the object 100 are identified, imaging conditions corresponding to the identified material and surface texture are identified, and the object 100 is imaged under the identified imaging conditions. The identification of the imaging conditions will be described below.

[0039] FIG. 5 is a diagram showing an example of the condition information 42 stored in the storage device 40. As shown in FIG. 5, in the condition information 42, the imaging conditions are managed by dividing them into a plurality of combinations of the material and surface properties of the object 100. In this example, the surface properties are expressed by the stages of processing applied to the object 100, that is, "unprocessed," "rough processing," "semi-finished," "finished," "mirror finish," and the like. However, the expressions of the surface properties given here are merely examples, and the surface properties may be expressed in other forms. For example, the surface properties may be expressed by the surface roughness, diffuse reflectance, surface treatment state, and the like of the object 100.

[0040] In the example of FIG. 5, the imaging conditions include the number of basic imaging operations performed to obtain one measurement image 60, and the shutter speed, camera gain, and luminance distribution of imaging light for each basic imaging operation. However, the parameters listed here are merely examples and may be changed as appropriate. For example, the imaging conditions may include only the number of basic imaging operations and the camera gain, and may not include the shutter speed and luminance distribution of imaging light. The values ​​recorded in the condition information 42 are obtained in advance by experiment or simulation. Furthermore, the operator may add imaging conditions for new materials and surface properties, or modify recorded imaging conditions, as necessary.

[0041] The measurement controller 30 identifies the material and surface texture of the object 100, and compares it with the condition information 42 to identify the imaging conditions corresponding to the material and surface texture of the object 100. For example, if the material of the object 100 is "SS400" and the surface texture of the object 100 is "unfinished", the measurement controller 30 performs basic imaging of the object 100 once to measure the shape of the object 100. At this time, the shutter speed of the camera 52 is "20", the camera gain is "2", and the luminance distribution of the imaging light is "R, G, B = 164, 128, 128". Also, if the material of the object 100 is "SS400" and the surface texture of the object 100 is "finished", the measurement controller 30 performs basic imaging of the object 100 twice to measure the shape of the object 100. In the first basic imaging, the shutter speed of the camera 52 is "10", the camera gain is "1", and the luminance distribution of the imaging light is "R, G, B = 64, 64, 96". In the second basic imaging, the shutter speed of the camera 52 is "12", the camera gain is "1", and the luminance distribution of the imaging light is "R, G, B = 128, 128, 128".

[0042] Next, the specification of the material and surface texture of the object 100 will be described. In this example, the measurement controller 30 specifies at least one of the material and surface texture of the object 100 based on at least one of the processing drawing 64 and the processing program 66 in which the processing contents of the object 100 are recorded. Furthermore, if the material and surface texture cannot be specified by the processing drawing 64 and the processing program 66, the measurement controller 30 captures the object 100 under standard conditions to obtain a preliminary image 62, analyzes the preliminary image 62, and specifies at least one of the material and surface texture of the object 100. Furthermore, if the material and surface texture cannot be specified by the preliminary image 62, the measurement controller 30 requests the operator to input the material and surface texture of the object 100. This will be described in order below.

[0043] First, a technique for identifying at least one of the material and the surface texture based on at least one of the machining drawing 64 and the machining program 66 will be described. When identifying the material and the surface texture, the measurement controller 30 acquires the machining drawing 64 and the machining program 66 of the object 100. FIG. 6 is a diagram showing an example of the machining drawing 64. As shown in FIG. 6, the machining drawing 64 usually includes the product name and the material of the object 100 in addition to a diagram showing the shape of the object 100 after machining is completed. The measurement controller 30 performs image analysis processing such as OCR on the machining drawing 64 to extract text described in the machining drawing 64. Then, the measurement controller 30 identifies at least one of the material and the surface texture of the object 100 based on the extracted text. For example, if there is text indicating a specific material in the extracted text, the measurement controller 30 determines that the object 100 is made of the specific material. Similarly, if there is text indicating a specific surface texture in the extracted text, the measurement controller 30 determines that the object 100 is made of the specific surface texture. Therefore, for example, if the extracted text contains the text "S45C" and "roughly machined", the measurement controller 30 determines that the material of the object 100 is "S45C" and the surface property of the object 100 is "roughly machined".

[0044] In another embodiment, the measurement controller 30 may identify at least one of the material and the surface properties of the target object 100 based on the machining program 66. The machining program 66 describes commands for causing the machine tool 14 to perform an intended operation, and is also called an "NC program." When identifying the material, etc. based on the machining program 66, the operator embeds information indicating the material, etc. in the machining program 66 according to predetermined rules.

[0045] For example, in the machining program 66, the part enclosed in parentheses usually indicates a comment, and the comment is not used for controlling the machine tool 14. Therefore, when a text indicating at least one of the material and the surface texture is described as a comment in the machining program 66, the comment may be specified as at least one of the material and the surface texture of the object 100. For example, in the case of the machining program 66 shown in FIG. 7, "S45C, ROUGH" is described as a comment in the third line. Among these, "S45C" is a text indicating "S45C", which is a type of material, and "ROUGH" is a text indicating "rough machining", which is a type of surface texture. Therefore, in this case, the measurement controller 30 specifies the material of the object 100 as "S45C" and the surface texture of the object 100 as "rough machining".

[0046] As another embodiment, at least one of the material and the surface texture may be represented by the value of a variable or command used in the machining program 66. In this case, the measurement controller 30 stores in advance the correspondence between the value of the variable or command and the material and the surface texture. FIG. 8 shows an example of data showing such a correspondence. The table 80a on the left side of FIG. 8 is a material correspondence table 80a that records the correspondence between the value of the common variable "VC101" and the material, and the table 80b on the right side is a surface texture correspondence table 80b that records the correspondence between the value of the common variable "VC102" and the surface texture. The measurement controller 30 stores such correspondence tables 80a and 80b in the storage device 40 in advance. In the example of FIG. 8, when the value of the common variable "VC101" is "10", it represents the material "S45C", and when the value of the common variable "VC102" is "20", it represents the surface texture "rough machining". Therefore, if the machining program 66 contains the statements "VC101=10;" and "VC102=20;", the measurement controller 30 identifies the material of the object 100 as "S45C" and the surface property as "roughly machined".

[0047] Also, instead of using variables, a dedicated command code may be created and the material or surface texture may be assigned to the value of the command code. For example, "MP" may be created as a code for commanding the material, and if the MP value is "1", it may be set to "S45C", if the MP value is "2", it may be set to "SS400", etc. Similarly, "MQ" may be created as a code for commanding the surface texture, and if the MQ value is "0", it may be set to "unmachined", if the MQ value is "1", it may be set to "rough machining", etc. In this case, if the machining program 66 contains the description "MP 2; MQ 0;", the measurement controller 30 will determine that the material is "SS400" and the surface texture is "unmachined".

[0048] Alternatively, at least one of the material and the surface property may be represented using a sequence name (also called a "sequence number") written in the machining program 66. The sequence name indicates the relative position of a block (line) in the machining program 66 and is written at the beginning of any line. Normally, the sequence name is written as "N + "alphanumeric characters". For example, if there is a line "N100;" in the machining program 66, that line represents the sequence name "N100".

[0049] When the sequence name indicates at least one of the material and the surface texture, the measurement controller 30 stores in advance a correspondence between the sequence name and at least one of the material and the surface texture. FIG. 9 is a diagram showing an example of such a correspondence. In the case of FIG. 9, the second character of the sequence name indicates a shape measurement command, the third character indicates the material, and the fourth character indicates the surface texture. In this case, when the second character of the sequence name is "Z", the measurement controller 30 determines that the third character of the sequence name indicates the material and the fourth character indicates the surface texture. In addition, in the example of FIG. 9, the third character "S" indicates the material "S45C", and the fourth character "A" indicates the surface texture "rough machining". Therefore, in this case, when the processing program 66 contains the description "NZSA;", the measurement controller 30 specifies the material of the object 100 as "S45C" and the surface texture as "rough machining".

[0050] As another embodiment, the surface texture may be specified based on the tool specified in the machining program 66. In this case, the measurement controller 30 stores in advance the correspondence between the tool number and the surface texture. For example, the measurement controller 30 stores in advance the tool number and the corresponding machining content (and thus the surface texture), such as tool number "01" being an end mill for rough machining and tool number "02" being an end mill for semi-finishing. In this case, if there is a description of "T01; M06;" in the machining program 66, the measurement controller 30 specifies the surface texture of the target object 100 as "rough machining".

[0051] As another embodiment, the surface texture may be specified based on the machining parameters specified in the machining program 66. In this case, the measurement controller 30 stores in advance the correspondence between the numerical range of the machining parameters and the surface texture. In this case, the machining parameters include, for example, the spindle speed, the cutting feed rate, the cutting depth, and the like. The measurement controller 30 stores the numerical range of the spindle speed, the numerical range of the cutting feed rate, and the numerical range of the cutting depth for each type of surface texture. Then, the measurement controller 30 may specify the surface texture based on the command values ​​of these machining parameters described in the machining program 66. For example, consider the case where there is a machining program 66 shown in FIG. 10. In the machining program 66, S is a spindle speed, F is a cutting feed rate, and PQ is a variable indicating the cutting depth defined by the operator. Also, M3 is a spindle normal rotation command, and G01 is a cutting feed command. Therefore, the machining program 66 in Fig. 10 indicates that the machining is performed at a spindle speed of 4000 rpm, a cutting feed rate of 100 mm / min, and cutting 2 mm from a position of 100 mm in the Z direction. The measurement controller 30 checks these values ​​against the numerical ranges of parameters stored in advance to identify the surface texture.

[0052] Alternatively, a file describing the material and surface texture may be created, and an access command to this file may be written in the machining program 66. For example, "MR" may be set as a command code indicating access to a file at a specified file path, and the material and surface texture may be written in text or binary format in the file. If "MR;" is written in the machining program 66, the measurement controller 30 will access the file at the specified file path and identify the material and surface texture.

[0053] Also, when the material and surface texture cannot be identified from the processing drawing 64 and the processing program 66, the material and surface texture of the object 100 may be identified based on the captured image of the object 100. That is, in this case, the measurement controller 30 stores a group of images of objects having various materials and surface textures as reference data 44 in advance. Also, when it is desired to identify the material and surface texture of the object 100, the measurement controller 30 drives the imaging unit 50 to capture the object 100 and obtain a preliminary image 62. The imaging conditions of this preliminary image 62 are standard conditions. In other words, the imaging conditions of the preliminary image 62 are not changed according to the material and surface texture of the object 100.

[0054] The measurement controller 30 compares the preliminary image 62 with the image group of the reference data 44, and identifies an image similar to the preliminary image 62. The measurement controller 30 may then identify the material and surface texture of the target object 100 based on the material and surface texture of the article depicted in the identified image. Note that, for example, a technique such as pattern matching may be used in comparing the preliminary image 62 with the image group. Furthermore, if the material and surface texture cannot be identified from any of the processing drawing 64, the processing program 66, and the preliminary image 62, the measurement controller 30 may request the operator to input the material and surface texture.

[0055] In another embodiment, the measurement controller 30 may use the learning model 45 to identify the material and surface properties of the target object 100. In this case, the measurement controller 30 stores the learning model 45, which has been generated in advance by machine learning, in the storage device 40 (see FIG. 2). The learning model 45 can be generated by supervised learning, in which images of an object whose material and surface properties are known (for example, a group of images of the reference data 44 labeled with information on the material and surface properties) are input to a learning device as teacher data, and the learning device learns the correlation between the images and the material and surface properties. A method for generating the learning model 45 by supervised learning is publicly known, and therefore a detailed description thereof will be omitted here.

[0056] The measurement controller 30 can obtain the corresponding material and surface texture as an output by inputting the preliminary image 62 into the learning model 45. The output material and surface texture may be modified by an input from the operator. In another embodiment, the learning model 45 may output a plurality of combinations of material and surface texture for one preliminary image 62. In this case, the measurement controller 30 presents the obtained plurality of combinations to the operator in descending order of likelihood. Then, the combination selected and determined by the operator's input may be specified as the material and surface texture of the object 100. In this case, the learning model 45 may be updated by creating teacher data using the preliminary image 62 as input data and labeling the combination of material and surface texture modified or selected by the operator, and re-learning by the learning device. With such a configuration, the accuracy of estimating the material and surface texture can be improved. In the above description, both the material and the surface texture are specified, but the configuration may be such that only one of the material and the surface texture is specified.

[0057] If the material and surface properties of the object 100 can be identified, as described above, the measurement controller 30 checks the identified material and surface properties against the condition information 42 to identify the imaging conditions for capturing the measurement image 60. Then, if a trigger for shape measurement of the object 100 is turned ON, the shape measurement system 12 executes measurement imaging to capture the object 100 under the identified imaging conditions. Here, the trigger for shape measurement may be an operation instruction from an operator. For example, when the operator operates the operation panel 22 of the numerical control device 20 or the UI device 38 of the measurement controller 30 to instruct "start shape measurement," measurement imaging of the object 100 may be executed.

[0058] As another embodiment, a code serving as a trigger may be written in the machining program 66. For example, a dedicated M code or G code for instructing the execution of three-dimensional shape measurement may be written in the machining program 66, and when it is time for the machine tool 14 to execute this dedicated M code or G code, the shape measurement system 12 may execute imaging for measurement.

[0059] Next, the process flow of the shape measurement system 12 will be described with reference to Fig. 11 and Fig. 12. When the shape measurement system 12 is started, first, the entire system is initialized (S10). Specifically, the measurement controller 30 initializes parameter settings of the camera 52 and the projector 54.

[0060] Once the initialization is complete, the measurement controller 30 identifies the material and surface properties of the object 100 (S12 to S24). Specifically, the measurement controller 30 first communicates with the numerical control device 20 and attempts to acquire the machining drawing 64 and the machining program 66 (S12). If the machining drawing 64 and the machining program 66 have been acquired (Yes in S12), the measurement controller 30 analyzes the machining drawing 64 and the machining program 66 to identify the material and surface properties of the object 100 (S16).

[0061] On the other hand, if the processing drawing 64 and the processing program 66 cannot be acquired (No in S12), the measurement controller 30 captures an image of the object 100 with the imaging unit 50 and acquires a preliminary image 62 (S20). Next, the measurement controller 30 compares the acquired preliminary image 62 with the reference data 44 or inputs it into the learning model 45 to identify the material and surface properties of the object 100 (S22).

[0062] If the material and surface texture are obtained based on the processing drawing 64 or the preliminary image 62 (Yes in S18), the process proceeds to step S26. On the other hand, if at least one of the material and surface texture is not obtained (No in S18), the measurement controller 30 requests the operator to input at least one of the material and surface texture, thereby identifying the material and surface texture (S24).

[0063] If the material and surface texture can be specified, the measurement controller 30 checks the specified material and surface texture against the condition information 42. If the condition corresponding to the specified material, etc. is recorded in the condition information 42 (Yes in S26), the measurement controller 30 sets the corresponding condition as the imaging condition (S28). On the other hand, if the condition corresponding to the specified material, etc. is not recorded in the condition information 42 (No in S26), the measurement controller 30 requests the operator to newly register the imaging condition corresponding to the specified material, etc. In response to this request, if the operator newly registers the imaging condition (Yes in S30), the measurement controller 30 accepts this new registration (S32) and returns to step S26. On the other hand, if the operator does not newly register (No in S30), the measurement controller 30 requests the operator to input the imaging condition. Then, the condition input by the operator in response to this request is set as the imaging condition (S34).

[0064] Once the setting of the imaging conditions is completed, the measurement controller 30 monitors whether or not the trigger for measuring the shape of the target object 100 is ON (S36). The trigger for measuring the shape is an instruction from an operator or a specific code in the machining program 66, as described above. Therefore, for example, the measurement controller 30 may monitor whether or not a measurement instruction has been issued from an operator. In addition, the measurement controller 30 may periodically acquire the number of lines currently being executed in the machining program 66 from the numerical control device 20 as the progress of machining, and monitor whether or not a code instructing measurement has been executed based on this number of lines being executed.

[0065] When the shape measurement trigger is turned ON (Yes in S36), the measurement controller 30 drives the imaging unit 50 to capture an image of the object 100 under the set imaging conditions and obtains a measurement image 60. Furthermore, the measurement controller 30 generates point cloud data 74 of the object 100 based on the obtained measurement image 60 (S38).

[0066] On the other hand, if the trigger for the shape measurement is not ON (No in S36) and the shape measurement system 12 is not OFF (No in S40), the measurement controller 30 further judges whether or not the imaging conditions need to be changed (S42). That is, if the object 100 itself is changed or another type of cutting is performed on the object 100 after the imaging conditions are set, the material and surface properties of the object 100 are likely to have changed. Therefore, the measurement controller 30 judges that the imaging conditions need to be changed when the code describing the machining program 66 in the numerical control device 20, which commands a tool change or a workpiece change, is executed, or when the machining program 66 executed by the numerical control device 20 itself is changed. If it is judged that the imaging conditions need to be changed (Yes in S42), the measurement controller 30 returns to step S12. On the other hand, if the imaging conditions do not need to be changed (No in S42), the measurement controller 30 returns to step S36.

[0067] As is clear from the above description, in this example, the imaging conditions for each of a plurality of combinations of the material and surface texture of the object 100 are stored as the condition information 42. Furthermore, the measurement controller 30 identifies the material and surface texture of the object 100, identifies imaging conditions corresponding to the identified material and surface texture of the object 100 based on the condition information 42, and performs imaging of the object 100 under the identified imaging conditions. With this configuration, even if the material and surface texture of the object 100 are changed, the object 100 can be imaged under appropriate conditions. As a result, the shape of the object 100 can be appropriately grasped, and therefore the shape of the object 100 can be appropriately measured with a simpler procedure.

[0068] In addition, the shape measurement system 12 stores imaging conditions for each of a plurality of combinations of the material and surface texture of the object 100 as condition information 42 in advance, and other configurations may be changed as appropriate as long as the imaging conditions of the object 100 are specified based on the condition information 42. For example, in the above description, in order to specify the material and surface texture of the object 100, the processing drawing 64 and the processing program 66 are used preferentially, then the preliminary image 62 is used, and finally, the operator is asked to input information. However, the procedure for specifying the material and surface texture may be changed as appropriate. Therefore, for example, the operator may always be asked to input information. In another embodiment, the material and surface texture may be specified always by using the preliminary image 62 without using the processing drawing 64 and the processing program 66.

[0069] In the above description, depending on the material and surface properties of the object 100, the HDR image 72 is generated and treated as the measurement image 60. However, if the shape of the object 100 can be grasped, the HDR image 72 does not need to be generated at all. In the above description, the measurement controller 30 and the numerical control device 20 are separate computers independent of each other, but the measurement controller 30 may be incorporated into the numerical control device 20. In other words, the numerical control device 20 may function as the measurement controller 30. The number of each component may be changed as appropriate. For example, the shape measurement system 12 may have a plurality of measurement controllers 30 and a plurality of imaging units 50. One machine tool system 10 may have a plurality of shape measurement systems 12. In the above description, the projector 54 is used as the light source 53. However, the light source 53 may be another device as long as it can irradiate the object 100 with light suitable for imaging, that is, light for imaging. For example, the light source 53 may be a laser light source that irradiates pattern light such as line light or point cloud light as imaging light. In addition, in the description so far, the shape measurement system 12 is incorporated in the machine tool system 10 having the machine tool 14. However, the shape measurement system 12 may be incorporated in a system unrelated to the machine tool 14, or may be provided independently without being incorporated in another system. [Explanation of symbols]

[0070] 10 machine tool system, 12 shape measurement system, 14 machine tool, 20 numerical control device, 22 operation panel, 30 measurement controller, 32 processor, 34 memory, 36 communication I / F, 38 UI device, 40 storage device, 42 condition information, 44 reference data, 45 learning model, 50 imaging unit, 52 camera, 53 light source, 54 projector, 56 imaging conditions, 60 measurement image, 62 preliminary image, 64 processing drawing, 66 processing program, 69 three-dimensional data, 70, 70f, 70s basic image, 72 HDR image, 74 point cloud data, 80a material correspondence table, 80b surface property correspondence table, 100 object, 102 protrusion, 104 edge.

Claims

1. An imaging unit having one or more cameras for imaging an object; a storage device that stores, as condition information, imaging conditions for measurement imaging of the object for measuring the shape of the object for each of a plurality of combinations of the material and surface property of the object; a measurement controller for controlling the driving of the imaging unit; The measurement controller includes: identifying a material and a surface property of the object, identifying the imaging conditions corresponding to the identified material and surface property of the object based on the condition information, causing the imaging unit to perform the measurement imaging under the identified imaging conditions, and measuring the shape of the object based on the obtained measurement image. A three-dimensional shape measuring system comprising:

2. The three-dimensional shape measurement system according to claim 1, The three-dimensional shape measurement system is characterized in that the measurement controller causes the imaging unit to image the object prior to the measurement imaging, and identifies at least one of the material and surface properties of the object based on the preliminary image obtained thereby.

3. 3. The three-dimensional shape measuring system according to claim 2, The storage device further stores a learning model that receives the preliminary image as an input and outputs at least one of the material and the surface texture of the object, The measurement controller identifies at least one of a material and a surface property of the object based on the learning model. A three-dimensional shape measuring system comprising:

4. 4. The three-dimensional shape measurement system according to claim 1, The imaging unit is provided in a machine tool and captures an image of a workpiece being machined by the machine tool as the target object; The measurement controller identifies at least one of a material and a surface property of the object based on at least one of a machining program for the workpiece and a machining drawing for the workpiece. A three-dimensional shape measuring system comprising:

5. The three-dimensional shape measuring system according to claim 4, the measurement controller acquires information indicating at least one of the machining program and the machining drawing from a numerical control device of the machine tool; A three-dimensional shape measuring system comprising:

6. 6. The three-dimensional shape measurement system according to claim 1, 2. A three-dimensional shape measuring system, wherein at least one of the material and the surface property of the object is specified by an operator.

7. 7. The three-dimensional shape measurement system according to claim 1, The imaging unit further includes one or more light sources that irradiate the object with imaging light, the measurement controller executes basic imaging of the object one or more times to obtain one of the measurement images; The imaging conditions include at least one of the number of times the basic imaging is performed to obtain one of the measurement images, the shutter speed of the camera in each of the basic imaging performed one or more times, the gain of the camera in each of the basic imaging performed one or more times, and the luminance distribution of the imaging light in each of the basic imaging performed one or more times. A three-dimensional shape measuring system comprising:

8. The three-dimensional shape measurement system according to any one of claims 1 to 7, the measurement controller generates point cloud data of the object based on the measurement images, and generates three-dimensional data of the object from the point cloud data. A three-dimensional shape measuring system comprising:

9. The three-dimensional shape measuring system according to claim 8 ; A machine tool that performs machining on the object; Equipped with The machine tool performs at least one of the following, based on the three-dimensional data generated by the measurement controller: checking for interference between a tool and the object; generating a path for the tool; determining machining accuracy; determining whether the shape of the object coincides with a reference shape; and determining whether the object is located at a predetermined position. A machine tool system comprising:

Images