PLACEMENT IDENTIFICATION DEVICE

Abstract

A placement identification device (10) comprises: a camera (14) that captures an image of a target; a UI device (18); and a controller (30) designed to identify the placement of a measurement target.The controller (30) is designed to: store a reference image (60) in advance, which represents a form of reference target; identify a preliminary placement of the measurement target based on a position identification image (70), which is a captured image of the measurement target; preliminary adjustment of the position and angle of the reference image (60) based on the preliminary placement and generate an overlap image (80), which is an image in which the reference image (60) overlaps with the position identification image (70); receive a user command to fine-tune the position and angle of the reference image (60); and identify an actual placement of the measurement target based on the reference image (60) after the fine-tuning.

Description

CROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims priority to Japanese Patent Application No. 2023-184247, filed on October 26, 2023, which is incorporated herein by reference in its entirety, including the specification, claims, drawings, and abstract. TECHNICAL FIELD

[0002] The present disclosure relates to a placement identifying device that identifies a placement of a measurement target set in a predetermined range. BACKGROUND

[0003] In the prior art, there has been a need to identify the placement of a target located at an arbitrary location. For example, when machining a workpiece with a machine tool, an operator places the workpiece, which is a machining target, into a machining chamber. During this process, a position and angle of the workpiece may deviate from a predefined reference position and angle. If the placement of the workpiece deviates from the reference, there is a possibility that a moving part (such as a tool) of the machine tool may unexpectedly interfere with the workpiece.

[0004] In view of this, in the prior art, the operator measures the workpiece's positioning before machining by the machine tool. The measurement is performed, for example, by manually or automatically manipulating a touch probe attached to the moving part of the machine tool. However, such a measurement using the touch probe has the problem that the measurement requires a long time.

[0005] Furthermore, there have been some proposals for a technique for detecting the placement of the workpiece by capturing an image of a specified workpiece and analyzing the captured image of the workpiece. For example, JP 2010-078513 A discloses a technique in which a pattern matching process is applied to an image of a workpiece captured by a camera to calculate an approximate position of the workpiece, and then a more accurate position is calculated using a characteristic region of the workpiece.

[0006] JP H7-110217 A discloses a technique in which a captured image of a workpiece is analyzed to identify a position of the center of gravity of the workpiece and a characteristic point of the workpiece, and an angle of the workpiece is determined based on an angle between the characteristic point and the center of gravity position.

[0007] However, the technique using pattern matching, as in JP 2010-078513 A, requires matching the entire pattern image to the entire captured image of the workpiece while changing the position and angle, and checking the degree of matching. In this case, the problem is that the required calculation time is very long.

[0008] In the technique of JP H7-110217 A, the captured image of the workpiece is analyzed and the angle of the workpiece is calculated. However, due to fluctuations in the external environment such as the degree of illumination, an error may occur in the angle of the workpiece obtained by such image analysis. For this reason, the technique of JP H7-110217 A tends to easily cause an error in the calculated position and angle of the workpiece. If the machine tool is operated based on position and angle data containing errors, there is a possibility of damage to the workpiece and / or the tool.

[0009] In view of the foregoing, the present disclosure discloses a placement identifying device that can identify a placement of a target more accurately and in a shorter period of time. SUMMARY

[0010] According to one aspect of the present disclosure, there is provided a placement identification device comprising: a camera that captures an image of a target placed in a predetermined area; a UI device that presents information to a user and receives an operation command from the user; and a controller configured to identify a placement of a measurement target, which is a target set at an arbitrary placement, based on an image captured by the camera, the controller configured to: store in advance a reference image representing a shape of a reference target, which is a target set at a known placement; acquire a position identification image by causing the camera to capture an image of the measurement target;Identifying a preliminary placement of the measurement target based on the position identification image; preliminary adjusting a position and angle of the reference image based on the preliminary placement so that the reference target shown in the reference image overlaps the measurement target, and generating an overlap image that is an image in which the reference image overlaps the position identification image; presenting the overlap image to the user and receiving a command to finely adjust the position and angle of the reference image from the user; and identifying a true placement of the measurement target based on the position and angle of the reference image after the fine adjustment.

[0011] In this case, the target may include a reference point and one or more characteristic parts, and the controller may be further configured to define a position of the target relative to the reference point and to define an angle of the target relative to the one or more characteristic parts.

[0012] Further, the reference point may be a center of gravity of the target, each of the one or more characteristic parts may be a shaped part that is present in an external shape of the target and that can be distinguished from a surrounding thereof, and the controller may be further configured to define the angle of the target by a direction angle of each of the one or more characteristic parts when viewed from the reference point or another characteristic part.

[0013] The reference image may be transparent to an extent that the position identification image is visible in the overlap image.

[0014] In this case, the reference image may be a mask image in which portions other than the one or more characteristic parts in the reference target are masked.

[0015] The controller may be further configured to identify a deviation amount between each of the one or more characteristic parts of the measurement target and a respective one of the one or more characteristic parts of the reference target in the overlap image, and then, if the deviation amount is less than a predefined allowable value, identify the true placement of the measurement target based on the position and angle of the reference image after the preliminary adjustment, without presenting the overlap image to the user.

[0016] The controller may be further configured to store a size of the characteristic part and a relative position of the characteristic part with respect to the reference point in the reference target as characteristic information; identify a search range of the characteristic part of the measurement target in the position identification image based on the characteristic information; and output a target mismatch error when no shape corresponding to the characteristic information is found in the search range of the position identification image.

[0017] The one or more characteristic parts may be an external shape of the target, and the controller may be further configured to define the angle of the target by an inclination angle of the external shape of the target.

[0018] The controller may be further configured to calculate the preliminary placement of the measurement target based on an image obtained by binarizing the position identification image.

[0019] The specified area may be a machining chamber of a machine tool and the target may be a workpiece that is fixed in the machining chamber and that is to be machined by the machine tool.

[0020] According to one aspect of a placement identification device of the present disclosure, fine adjustment is performed by the user after a reference image is preliminarily adjusted. With this configuration, the placement of the workpiece can be identified quickly and accurately. Brief description of the drawings

[0021] One or more embodiments of the present disclosure are described with reference to the following figures, in which: Fig. 1 is a schematic diagram showing a structure of a placement identification device; Fig. 2 a schematic representation of a reference workpiece and a workpiece to be measured; Fig. 3 is a schematic diagram showing the extraction of a center of gravity and a characteristic part; Fig. 4 is a schematic diagram showing a position identification image and a negative-positive inversion image; Fig. 5 is a diagram showing an example of a reference image; Fig. 6 is a diagram showing a case example of a registration screen of the reference image; Fig. 7 is a diagram showing an example of an overlap image; Fig. 8 is a flowchart showing a first half of a procedure for machining a workpiece to be measured by a machine tool; Fig. 9 is a flowchart showing a second half of the procedure for machining the workpiece to be measured by the machine tool; and Fig. 10 is a diagram showing an example of another workpiece. DESCRIPTION OF EMBODIMENTS

[0022] Now, the structure of a placement identification device 10 will be described with reference to the drawings. Fig. 1 is a schematic diagram showing the structure of the placement identification device 10. The placement identification device 10 identifies a placement of a target set within a predetermined range. In the present embodiment, a "placement" includes a "position" of a target in a two-dimensional plane and may also include an "angle" of the target in the two-dimensional plane. Hereinafter, a placement identification device 10 configured in combination with a machine tool 100 will be exemplified for description. In the placement identification device 10, the "predetermined range" is a machining chamber 102 of the machine tool 100, and the "target" is a workpiece W machined by the machine tool 100. However, the placement identification device 10 described herein is merely exemplary.Thus, the placement identification device 10 is not limited to combination with the machine tool 100 and can alternatively be combined with other devices. Alternatively, the placement identification device 10 can be used as a standalone device without being combined with any other device. Therefore, the "target" is not limited to the workpiece W and can alternatively be another element.

[0023] First, the machine tool 100 will be briefly described. The machine tool 100 performs machining of the workpiece W according to an NC program commanded by a user. For the purposes of the following description, a machining center with a spindle head 104 will be presented as an example.

[0024] In the machining chamber 102 of the machine tool 100, a table 106 and the spindle head 104 for holding a tool are provided. After a human or a robot places the workpiece W on the table 106, the human or robot fixes the workpiece W to the table 106 using a magnetic chuck, a special fixing device, or the like. An NC device 110 of the machine tool 100 operates a movable part (for example, the spindle head 104) according to an NC program specified by the user to machine the workpiece W.

[0025] At this time, the NC program is usually generated under the assumption that the workpiece W is placed at a reference position and angle defined in advance. However, in reality, a slight placement error is caused in the workpiece W placed in the machining chamber 102. If the machine tool 100 is operated without correcting such a placement error, there is a possibility that the movable part of the machine tool 100 may interfere with the workpiece W or that the precision of the workpiece W (i.e., a product) achieved after machining is completed may be reduced.

[0026] To prevent such interference and precision degradation, the user typically measures the placement error of the workpiece W before starting machining and registers the placement error as an offset amount in the NC device 110. The NC device 110 controls the movement of the moving part according to the input offset amount. With this configuration, interference between the moving part and the workpiece W is prevented, and the precision of the product can be maintained at an appropriate level.

[0027] However, in the prior art, there was a problem that measuring the placement error of the workpiece W requires time and effort. For example, to measure the position and angle of the workpiece W, a configuration in which the user operates a touch probe manually or automatically can be considered. However, measurement using the touch probe requires time and effort. In addition, there have been some proposals for a technique for identifying the position and angle of the workpiece W based on a captured image of the workpiece W. However, in the prior art method, calculating the position and angle requires some time or the accuracy was low.

[0028] The placement identification device 10 identifies the position and angle, that is, the placement, of the workpiece W. In the following description, directions parallel to a mounting surface of the workpiece W are referred to as the X direction and the Y direction, and a direction perpendicular to the mounting surface is referred to as the Z direction. Furthermore, the position and angle of the workpiece W to be identified by the placement identification device 10 of the present embodiment are, respectively, a position in an XY plane of the workpiece W and a rotation angle of the workpiece W about an axis parallel to the Z direction.

[0029] The placement identification device 10 includes an imaging unit 12, a UI device 18, and a controller 30. The imaging unit 12 captures an image of a target located in the processing chamber 102; that is, the workpiece W. The imaging unit 12 includes, for example, a camera 14 and an illumination device 16 that illuminates the workpiece W. The camera 14 can be fixedly located in the processing chamber 102 or attached to a position adjustment device such as a pan head and an XY stage. When a position and orientation of the camera 14 can be changed, the position and orientation of the camera 14 are detected by a sensor and sent to the controller 30. The controller 30 transforms the coordinate system between a camera coordinate system and a machine coordinate system of the machine tool 100 based on the position and orientation of the camera 14.The number of cameras 14 is not limited to one and alternatively multiple cameras 14 may be provided.

[0030] In the present embodiment, the camera 14 is positioned facing the mounting surface of the workpiece W (i.e., an upper surface of the table 106) and has an optical axis approximately perpendicular to the mounting surface. The camera 14 can communicate with the controller 30 via a wire or wirelessly. The camera 14 operates according to a control signal sent from the controller 30. Furthermore, the camera 14 sends data of a captured image to the controller 30.

[0031] The illumination device 16 illuminates the workpiece W. The illumination device 16 can be fixedly placed in the processing chamber 102 or arranged in a state where its position and orientation can be changed. Furthermore, the number of illumination devices 16 is not limited to one, and alternatively, multiple illumination devices 16 can be provided. Similar to the camera 14, the illumination device 16 can communicate with the controller 30 via a wire or wirelessly. In the illumination device 16, the light quantity, color temperature, and irradiation direction are changed according to a control signal sent from the controller 30.

[0032] The UI device 18 includes an output device 20 that presents information to the user and an input device 22 that receives an operation command from the user. In the present embodiment, the output device 20 includes a display that displays an overlap image 80 or the like, to be described later. In addition, the input device 22 includes, for example, a keyboard, a touch panel, a mouse, a microphone, a barcode scanner, or the like. The output device 20 and the input device 22 may be provided independently of the machine tool 100 or integrated into the machine tool 100. For example, an operation panel provided on the machine tool 100 and a display of the operation panel may be used as the input device 22 and the output device 20 of the placement identification device 10.The input device 22 and the output device 20 can each communicate with the controller 30 via a wire or wirelessly.

[0033] The controller 30 identifies the placement (i.e., the position and angle) of the workpiece W based on an image of the workpiece W captured by the camera 14. Physically, the controller 30 is a computer having a processor 32 and a memory 34. The controller 30 may communicate with the NC device 110 via a wire or wirelessly. Fig. 1 shows the controller 30 as a single computer independent of the NC device 110. Alternatively, however, the controller 30 may be formed by combining multiple computers that are physically separated from each other. Alternatively, the NC device 110 may function as part or all of the controller 30. In either case, the controller 30 identifies the placement of the workpiece W, which is the target to be measured. The controller 30 calculates a difference between the identified placement and a reference placement defined in advance as an offset amount and sends the offset amount to the NC device 110. The NC device 110 causes the movable part of the machine tool 100 to operate in accordance with the received offset amount.

[0034] Next, the identification of the placement of the workpiece W by the placement identification device 10 will be described. An upper part of Fig. Figure 2 is a schematic representation of a workpiece W with a known placement and a lower part of Fig. Figure 2 is a schematic diagram of a workpiece W, which is the target to be measured, with an arbitrary placement. Hereinafter, the workpiece W with the known placement is referred to as the "reference workpiece Wr," and the workpiece W, which is the target to be measured, is referred to as the "workpiece to be measured Wm." Further, reference symbols of elements related to the reference workpiece Wr are shown with a subscript "r," and reference symbols of elements related to the workpiece Wm to be measured are shown with a subscript "m." Furthermore, when the reference workpiece Wr and the workpiece Wm to be measured are not distinguished, the subscripts "r" and "m" are omitted, and the workpiece is simply referred to as "workpiece W." This notation applies similarly to other elements.

[0035] In the present embodiment, workpiece information is set for each type of workpiece W. The workpiece information can be roughly divided into machining information and placement information. The machining information includes a product number of the workpiece W, identification information of a machining process of the workpiece W, and identification information of an NC program to be applied to the workpiece W.

[0036] The placement information is information representing an ideal placement of the workpiece W and is information representing the placement of the reference workpiece Wr. The placement information includes a position of a reference point of the reference workpiece Wr, characteristic information, and a reference image 60, which will be described later. In the present embodiment, a center of gravity PG of the workpiece W is treated as a reference point. Furthermore, in the present embodiment, a characteristic shaped part (for example, a protrusion, a hole, or the like) of the workpiece W is treated as a characteristic part 50. Hereinafter, the number of the characteristic part 50 is described as one for convenience of description, but alternatively, the number of the characteristic parts 50 may be more than one. In the example configuration of Fig. 2, a projection in the shape of a circular column protruding from a surface of the workpiece W in the Z direction is defined as the characteristic part 50.

[0037] Thus, in the present embodiment, the placement information includes a position of a center of gravity PGr of the reference workpiece Wr, a relative position of a characteristic part 50r with respect to the center of gravity PGr, and a size of the characteristic part 50r. The relative position of the characteristic part 50 with respect to the center of gravity PG is represented by a direction angle of the characteristic part 50 when viewed from the center of gravity PG (hereinafter also referred to as "characteristic part angle A") and a distance from the center of gravity PG to the characteristic part 50 (hereinafter also referred to as "characteristic part distance L"). Hereinafter, a size of the characteristic part 50 is referred to as "characteristic part size C." The coordinate system used in determining the angle is a linear orthogonal coordinate system (ie, a Cartesian coordinate system).

[0038] When the placement of the workpiece Wm to be measured is to be identified, the controller 30 controls the imaging unit 12 to acquire a position identification image 70 that captures an image of the workpiece Wm to be measured. The controller 30 then analyzes the position identification image 70 to identify a position of a center of gravity PGm of the workpiece Wm to be measured and an angle of the characteristic part Am. When the image of the workpiece Wm to be measured is ideally captured, a position error between the center of gravity PGm of the workpiece Wm to be measured and the center of gravity PGr of the reference workpiece Wr, and an angular error between the angle of the characteristic part Am of the workpiece Wm to be measured and an angle of the characteristic part Ar of the reference workpiece Wr become the offset amount that represents the placement error of the workpiece Wm to be measured.

[0039] However, since various disturbances occur in the imaging environment of the workpiece Wm to be measured, it is difficult to ideally capture the image of the workpiece Wm to be measured. As a result, the position of the center of gravity PGm and the angle of the characteristic part Am, which are obtained by simple image analysis, usually contain a certain degree of error in many cases. This point will now be discussed with reference to Fig. 3 to 5. First, a method for extracting the center of gravity PGm and the characteristic part 50m from the position identification image 70 will be described with reference to Fig. 3 briefly described. Fig. Figure 3 is a schematic diagram showing the extraction of the center of gravity PGm and the characteristic part 50m.

[0040] When the center of gravity PGm and the characteristic part 50m of the workpiece Wm to be measured are to be extracted, the controller 30 first applies a binarization process to the position identification image 70 obtained by capturing an image of the workpiece Wm to be measured. Specifically, the controller 30 converts the position identification image 70 to grayscale and applies a binarization process with an arbitrary threshold. The grayscale conversion is performed, for example, by an NTSC weighted average method. The controller 30 may apply other processes such as image enlargement and reduction, edge enhancement, or the like before the binarization process, as needed.

[0041] After the binarization process, the controller 30 performs a negative-positive inversion on the binarized image. An upper part of Fig. Figure 3 is a schematic representation of the image with negative-positive inversion. In the upper part of Fig. 3, for the sake of simplicity of representation of reference symbols or the like, a black section is not filled and grey hatching is applied instead.

[0042] Then, the controller 30 applies blob processing to the negative-positive inversion image. In practice, the blob processing is applied to the negative-positive inversion image, but a middle part and a lower part of Fig. 3 shows images without the negative-positive inversion to facilitate viewing.

[0043] A blob is a collective collection of pixels with the same concentration. The controller 30 extracts the blob from the image using negative-positive inversion. State-of-the-art methods can be used for blob processing, and therefore, blob processing will not be described in detail here. In any case, multiple blobs are extracted through blob processing.

[0044] For each of the multiple blobs to be extracted, the controller 30 identifies a quadrilateral of the minimum size containing the blob. The controller 30 identifies a quadrilateral of the maximum size among the multiple quadrilaterals identified as “workpiece-containing quadrilateral Rw” that contains an outer shape line of the workpiece Wm to be measured. The controller 30 calculates a center position of the workpiece-containing quadrilateral Rw as the center of gravity PGm of the workpiece Wm to be measured. The middle part of Fig. Figure 3 shows the identification of the workpiece-containing quadrilateral Rw and the center of gravity PGm.

[0045] The controller 30 also identifies a quadrilateral containing the blob corresponding to the characteristic part 50m as “characteristic part containing quadrilateral Rc”.

[0046] To identify the quadrilateral Rc containing the characteristic part, the controller 30 uses characteristic information of the reference workpiece Wr (i.e., the angle of the characteristic part Ar, a distance of the characteristic part Lr, and a size of the characteristic part Cr of the reference workpiece Wr). Based on the angle of the characteristic part Ar and the distance of the characteristic part Lr of the reference workpiece Wr, the controller 30 identifies an approximate area where the characteristic part 50m of the workpiece Wm to be measured exists as a search area S (see the middle part of Fig. 3). The controller 30 identifies, among the blobs present in this search area S, a blob having an angle, a distance, and a size similar to the angle of the characteristic part Ar, the distance of the characteristic part Lr, and the size of the characteristic part Cr of the reference workpiece Wr as the characteristic part 50m of the workpiece Wm to be measured.

[0047] If no suitable blob exists in the search area S, it can be considered that a workpiece W other than the workpiece W intended by the user has been placed. In this case, the controller 30 notifies the user of an error indicating a target mismatch and completes the placement identification process.

[0048] Once the controller 30 has identified the blob of the characteristic part 50m, the controller 30 determines a quadrilateral of the minimum size containing the blob as the quadrilateral containing the characteristic part Rc (see the lower part of Fig. 3). The controller 30 then calculates a direction angle of the center of the quadrilateral containing the characteristic part Rc, when viewed from the center of gravity PGm, as the angle of the characteristic part Am of the characteristic part 50m.

[0049] When an ideal position identification image 70 is acquired, the center of gravity PGm and the angle of the characteristic part Am of the workpiece to be measured Wm thus calculated become the information indicating the placement of the workpiece Wm to be measured. That is, an amount of positional deviation between the center of gravity PGr and the center of gravity PGm and an amount of angular deviation between the angle of the characteristic part Ar and the angle of the characteristic part Am become the offset amount of the workpiece to be measured Wm.

[0050] As apparent from the above description, in the present embodiment, the center of gravity PGm and the characteristic part 50m of the workpiece Wm to be measured are extracted based on the binarized image of the position identification image 70. However, errors due to vibrations of the camera 14 and the illumination device 16, or the like, may be included in the binarized image of the position identification image 70. If the position of the center of gravity PGm and the angle of the characteristic part Am were calculated based on the binarized image containing such errors, the position of the center of gravity PGm and the angle of the characteristic part Am finally obtained would also contain errors. In addition, there is a slight individual difference in the workpiece Wm to be measured, and the workpiece Wm to be measured does not completely conform to the shape of the reference workpiece Wr.It is not possible to compensate for such a slight individual difference through image analysis alone.

[0051] For example, consider a case where, as described in an upper part of Fig. 4, due to vibrations of the camera 14 and the illumination device 16, or the like, a strong shadow or reflection occurs at one end of the workpiece Wm to be measured and the characteristic part 50m. In this case, the workpiece Wm to be measured actually has an edge shown with a two-dot chain line, but due to the influences of shadow and reflection, an edge shown with a solid line would be extracted when the binarization process is applied. A lower part of Fig. Figure 4 shows a negative-positive inversion image generated based on the binarized image containing these errors. When the center of gravity PGm and the characteristic part angle Am are determined based on the binarized image with errors, errors will naturally also occur in the center of gravity PGm and the angle of the characteristic part Am. Even if an image of the workpiece Wm to be measured is ideally acquired, a small individual difference in the shape of the workpiece Wm to be measured cannot be adequately handled by image analysis alone. As a result, an appropriate offset cannot be set, possibly leading to interference between the moving part and the workpiece W and a reduction in product accuracy.

[0052] In view of this, in the present embodiment, the reference image 60 is prepared in advance, and the errors are corrected using the reference image 60. This process will now be described in detail.

[0053] Fig. 5 is a diagram showing an example of the reference image 60. The reference image 60 is an image representing a shape of the reference workpiece Wr, specifically, an external shape and the characteristic part 50r of the reference workpiece Wr. As described, the reference workpiece Wr is a workpiece with a known location. The reference image 60 is a mask image in which parts other than the characteristic part 50r are masked. Therefore, in the portions of the reference image 60 except for the characteristic part 50r, the shape of the reference workpiece Wr is eliminated. Fig. 5, the center of gravity PGr is shown for explanation purposes, but in the actual reference image 60, the center of gravity PGr is not shown.

[0054] As will be described in detail later, the reference image 60 is overlapped with the position identification image 70 to thereby form the overlap image 80. The reference image 60 is transparent enough to allow the shape of the workpiece Wm to be measured, which is depicted in the position identification image 70, to be visible.

[0055] The reference image 60 is prepared for each type of the workpiece W. Further, a position of the center of gravity PGr, the characteristic information (ie, the angle of the characteristic part Ar, the distance of the characteristic part Lr, and the size of the characteristic part Cr), and the machining information (ie, the NC program or the like) are associated with the reference image 60.

[0056] Such a reference image 60 can be generated, for example, based on an image acquired by capturing an image of an actually existing reference workpiece Wr. In this case, the user first fixes the workpiece W on the table 106 in the processing chamber 102 and precisely measures the position and angle of the workpiece W. After the user fixes the workpiece W in the processing chamber 102, the user can measure the position and angle of the workpiece W using a sensor such as a contact probe, for example. Alternatively, the user can fix the workpiece W on a pallet outside the processing chamber 102 and measure the position of the workpiece W on the pallet. In this case, the position and angle of the workpiece W in the processing chamber 102 are known by the user fixing the pallet in the processing chamber 102 without deviations in position or angle.When the position and angle in the processing chamber 102 become known, the workpiece W can be treated as a reference workpiece Wr.

[0057] When the reference workpiece Wr is loaded into the processing chamber 102, the controller 30 controls the imaging unit 12 and acquires a captured image of the reference workpiece Wr. For example, the controller 30 may capture images of the reference workpiece Wr multiple times while changing the imaging conditions (e.g., the driving conditions of the camera 14 and the illumination device 16). In this case, multiple captured images may be combined into one image to enable more accurate extraction of the external shape and the characteristic part 50r of the reference workpiece Wr through image analysis. Alternatively, the number of imaging operations may be one if an image from which the external shape and the characteristic part 50r of the reference workpiece Wr can be extracted can be acquired with a single imaging.In any case, when a suitable captured image is acquired, the captured image is processed to generate the reference image 60.

[0058] For example, using any drawing application (e.g., Microsoft Paint, where "Microsoft" is a registered trademark), the user can blacken portions of the captured image of the reference workpiece Wr except for the characteristic part 50r and make the blackened image transparent to generate a mask image as the reference image 60. The degree of transparency of the reference image 60 can be uniform or different between the characteristic part 50r and the other portions.

[0059] The controller 30 also extracts the centroid PGr and the characteristic part 50r of the reference workpiece Wr from the captured image of the reference workpiece Wr. More specifically, the controller 30 applies the grayscale conversion, the binarization process, the negative-positive inversion process, and the blob processing to the captured image of the reference workpiece Wr. The controller 30 then extracts a quadrilateral containing the largest blob as the workpiece-containing quadrilateral Rw. The controller 30 calculates a center point of the workpiece-containing quadrilateral Rw as the centroid PGr of the reference workpiece Wr.

[0060] The controller 30 also presents the image after blob processing to the user and prompts the user to select a blob to be used as the characteristic part 50r. When the user selects a specific blob, the controller 30 extracts a containing quadrilateral of the selected blob as the characteristic part containing quadrilateral Rc. The controller 30 then calculates the center point of the characteristic part containing quadrilateral Rc as the position of the characteristic part 50r and calculates the characteristic information based on the position; that is, the angle of the characteristic part Ar, the distance of the characteristic part Lr, and the size of the characteristic part Cr of the reference workpiece Wr. The controller 30 stores these obtained values ​​in correlation with the reference image 60 in the memory 34.

[0061] As another configuration, the reference image 60 may be generated not based on the captured image of the reference workpiece Wr, but based on CAD data or a design drawing of the reference workpiece Wr. In this case, the controller 30 may generate a schematic plan view of the reference workpiece Wr based on the CAD data or the design drawing of the reference workpiece Wr and process the plan view to generate the reference image 60. Furthermore, the controller 30 may calculate the position of the center of gravity PGr, the angle of the characteristic part Ar, the distance of the characteristic part Lr, and the size of the characteristic part Cr of the reference workpiece Wr based on the plan view.

[0062] Fig. 6 is a diagram showing a case example of a registration screen 112 of the reference image 60. The registration screen 112 is displayed on the display of the UI device 18. In the example configuration of Fig. 6, the user presses an image load button 130 to select a pre-generated reference image 60. The selected reference image 60 is displayed on the registration screen 112. In addition, in the example configuration of Fig. 6, a product number of the workpiece W, a machining process number of the machining process applied to the workpiece W, and identification information of the NC program applied to the workpiece W are defined as machining information 132. Such machining information 132 can be manually input by the user or retrieved by reading the identification information (e.g., a barcode) affixed to the workpiece W using the input device 22 (e.g., a barcode scanner). Furthermore, characteristic information 134 linked to the reference image 60 is displayed on the registration screen 112. Fig. 6 shows a case with two characteristic parts 50r. If the displayed information does not have a problem, the user presses an OK button 136 to register the reference image 60 in correlation with the characteristic information and the processing information.

[0063] Next, the principle of identifying the placement of the workpiece Wm to be measured using such a reference image 60 will be described. As described above, when the placement of the workpiece Wm to be measured is to be identified, the controller 30 captures an image of the workpiece Wm to be measured and acquires the position identification image 70. The controller 30 further calculates the position of the center of gravity PGm and the angle of the characteristic part Am of the workpiece Wm to be measured based on images acquired by binarizing and negative-positive inverting the position identification image 70. However, as described above, the center of gravity PGm and the angle of the characteristic part Am usually contain errors. To easily correct the errors, the controller 30 generates the overlap image 80 in which the reference image 60 overlaps the position identification image 70. Fig. 7 is a diagram showing an example of the overlap image 80.

[0064] When generating the overlap image 80, the controller 30 preliminarily adjusts the reference image 60 based on a position error of the center of gravity PG and an angular error of the characteristic part 50 between the reference workpiece Wr and the workpiece to be measured Wm. That is, the controller 30 preliminarily adjusts the position of the reference image 60 so that the center of gravity PGr of the reference workpiece Wr coincides with a preliminary center of gravity PGm of the workpiece to be measured Wm, which is determined from the image of the position identification image 70 with negative-positive inversion. Furthermore, the controller 30 preliminarily adjusts the angle of the reference image 60 so that the characteristic partial angle Ar of the reference workpiece Wr coincides with a preliminary characteristic partial angle Am of the workpiece to be measured Wm, which is determined from the binarized image of the position identification image 70.The controller 30 then generates the overlap image 80 by overlapping the reference image 60 after the preliminary adjustment with the position identification image 70.

[0065] The generated overlap image 80 is presented to the user. Here, the position identification image 70 is a pre-binarization image and thus has sufficient gradation. Therefore, the user can understand the shape of the reference workpiece Wr even if a strong shadow or reflection occurs in the reference workpiece Wr depicted in the position identification image 70.

[0066] That is, the user can understand the shapes and positions of the workpiece Wm to be measured and the reference image 60 by observing the overlap image 80. Specifically, the reference image 60 in the present embodiment is a mask image in which parts other than the characteristic part 50r are masked and has a certain degree of transparency. For this reason, the user can clearly understand the shapes of parts of the workpiece Wm to be measured that overlap the reference workpiece Wr. As a result, the user can easily recognize deviations in the position and angle of the reference image 60 with respect to the workpiece Wm to be measured. After the controller 30 presents the overlap image 80 to the user, the controller 30 prompts the user to finely adjust the position and angle of the reference image 60.

[0067] The user finely adjusts the position and angle of the reference image 60 while observing the overlap image 80 displayed on the display so that the reference image 60 accurately overlaps the workpiece Wm to be measured. Specifically, the user operates the input device 22, such as the keyboard, to specify a movement amount and a rotation amount of the reference image 60. The controller 30 continuously reflects the correction amount input by the user in the overlap image 80. When the reference image 60 accurately overlaps the workpiece Wm to be measured through the fine adjustment, the user commands a calculation of the position and angle of the workpiece Wm to be measured.

[0068] Upon receiving this command, the controller 30 calculates the position of the center of gravity PGr and the angle of the characteristic part Ar of the reference image 60 after fine adjustment and adopts these values ​​as the true position of the center of gravity PGm and the true angle of the characteristic part Am of the workpiece to be measured Wm. The controller 30 then calculates an offset amount of the workpiece to be measured Wm based on the actual position of the center of gravity PGm and the actual angle of the characteristic part Am and sends the offset amount to the NC device 110.

[0069] In this way, since the user understands the actual shape of the workpiece Wm to be measured and finely adjusts the position and angle of the reference image 60, the position and angle of the workpiece Wm to be measured can be more accurately identified. Furthermore, by preliminarily adjusting the position and angle of the reference image 60 before the user fine-adjusts, the amount of fine adjustment by the user can be reduced, and the effort and time required for fine-adjustment can be significantly reduced. Furthermore, as described above, the reference image 60 is a mask image in which parts other than the characteristic part 50r are masked and which has a certain degree of transparency. Therefore, the user can appropriately recognize the shape of the workpiece Wm to be measured even on the overlap image 80 and thus easily fine-adjust the reference image 60.

[0070] Alternatively, a configuration may be used in which, when the amount of positional deviation between the characteristic part 50r in the reference image 60 and the characteristic part 50m in the position identification image 70 in the phase after the preliminary adjustment is less than or equal to a predefined allowable value, the controller determines the true position and the true angle of the workpiece Wm to be measured without requesting the fine adjustment. For example, the controller 30 identifies, as shown in Fig. 7, a center point Or of the characteristic part 50r and a center point Om of the characteristic part 50m in the phase after the preliminary adjustment, and determines a distance between the two points Or and Om. The controller 30 can then prompt the user for fine adjustment only when the determined distance is greater than or equal to the predefined allowable value.

[0071] Next, a procedure for machining the workpiece Wm to be measured by the machine tool 100 will be described with reference to Fig. 8 and Fig. 9. When the workpiece Wm to be measured is to be machined by the machine tool 100, the user starts up and initializes the machine tool 100 and the placement identification device 10 (S10). During initialization, communication is established between the controller 30 and the NC device 110, and between the controller 30 and the imaging unit 12. Furthermore, the controller 30 acquires a transformation parameter between the camera coordinate system and the machine coordinate system. For example, the controller 30 acquires an image of a reference area with a known position and size (for example, a marker attached to the origin of the machine coordinate system) using the camera 14 and identifies the transformation parameter of the coordinate systems based on the position and size of the reference area in the acquired image.

[0072] The user then places the workpiece Wm to be measured on the table 106 in the processing chamber 102 and fixes it (S12). In this process, the positioning of the workpiece Wm to be measured does not need to be precise, and positioning by eye is sufficient.

[0073] Then, the user inputs machining information corresponding to the workpiece Wm to be measured (S14). The machining information includes, for example, the product number of the workpiece to be measured, a process number of a process to be executed next, the NC program name, or the like. The machining information can be manually input by the user using the keyboard or the like. Alternatively, identification information (e.g., a barcode) of the machining information can be affixed in advance to the workpiece Wm to be measured or the fixture, and the controller 30 can read the identification information.

[0074] The controller 30 checks the presence or absence of a reference image 60 that correlates with the input processing information (S16). If no corresponding reference image 60 exists, the controller 30 prompts the user to register a reference image 60. In this case, the user calls the Fig. 6 and registers the reference image 60 and the characteristic information via the registration screen 112 (S18).

[0075] After the reference image 60 is registered, the controller 30 begins imaging the workpiece Wm to be measured (S20 to S26). Specifically, the controller 30 adjusts imaging conditions (S20). The imaging conditions include a condition related to the camera 14 and a condition related to the illumination device 16. The condition related to the camera 14 is, for example, a position, shutter speed, automatic white balance, or the like of the camera 14. The condition related to the illumination 16 is, for example, a position, brightness, color temperature, or the like of the illumination device 16. These imaging conditions are set and registered in advance. When the adjustment of the imaging conditions is completed, the controller 30 controls the imaging unit 12 to acquire an image of the workpiece Wm to be measured (S24). The acquired image is temporarily stored in the memory 34.Then, the controller 30 repeats imaging of the workpiece Wm to be measured while changing the imaging conditions until a required number of images are captured (S20 ~ S26).

[0076] When the required number of images has been acquired, the controller 30 combines these multiple images to generate a position identification image 70 (S28). In this case, the controller 30 extracts from each of the multiple acquired images a portion in which an image of the workpiece Wm to be measured is clearly captured, and combines the extracted portions. For example, when a first acquired image is acquired in a state in which the illumination device 16 (see Fig. 1) is applied from a left side of the workpiece Wm to be measured, and a second captured image taken in a state where the illumination device 16 is applied from a right side of the workpiece Wm to be measured are to be combined, a left half of the first captured image and a right half of the second captured image are combined. By employing such a configuration, the entire workpiece Wm to be measured becomes bright, and shading due to unevenness on the workpiece Wm to be measured becomes clearer, and thus an image that can be easily analyzed can be obtained. If a suitable image can be obtained, the number of imaging operations may be one, in which case, obviously, the image combining process (S28) is skipped.

[0077] Next, the controller 30 extracts the center of gravity PGm and the characteristic part 50m of the workpiece Wm to be measured based on the obtained position identification image 70, and calculates a preliminary position of the center of gravity PGm and a preliminary angle of the characteristic part Am (S30). Then, the controller 30 preliminarily adjusts the position and angle of the reference image 60 based on the obtained preliminary position of the center of gravity PGm and the preliminary angle of the characteristic part Am (S32).

[0078] The controller 30 overlaps the reference image 60 after the preliminary adjustment with the position identification image 70 to generate the overlap image 80 (S34). The controller 30 then judges whether fine adjustment of the reference image 60 is required (S36). For example, if a command is received from the user indicating that fine adjustment is not required, the controller 30 proceeds to step S39. Alternatively, the controller 30 may judge whether fine adjustment is required based on the positional deviation amount between the center of the characteristic part 50m of the workpiece Wm to be measured and the center of the characteristic part 50r of the reference workpiece Wr in the overlap image 80. If the positional deviation amount is less than or equal to an allowable value defined in advance, the controller 30 judges that fine adjustment is not required and proceeds to step S39.

[0079] On the other hand, if the controller 30 judges that fine adjustment is necessary, the controller 30 displays the overlap image 80 on the display and prompts the user to fine-adjust the reference image 60. If the user commands fine adjustment of the position and angle of the reference image 60 in response to this request, the controller 30 corrects the position and angle of the reference image 60 according to the command and regenerates the overlap image 80 (S38, S34). The newly generated overlap image 80 is displayed on the display.

[0080] The user or the controller 30 judges whether further fine adjustment is required based on the newly generated overlap image 80 (S36). Finally, if it is decided that no further fine adjustment is required, the controller 30 assumes the center of gravity PGr and the angle of the characteristic part Ar of the reference workpiece Wr contained in the reference image 60 in the overlap image 80 as the true center of gravity PGm and the true angle of the characteristic part Am of the workpiece Wm to be measured, and calculates the placement of the workpiece Wm to be measured (S39).

[0081] Then, the controller 30 calculates the offset amount based on the calculated true position and angle of the workpiece Wm to be measured, and registers the offset amount in the NC device 110 (S40). The NC device 110 controls the movable part of the machine tool 100 taking into account the registered offset amount and executes the machining of the workpiece Wm to be measured (S42). When all machining of the workpiece W to be machined according to the current machining information is completed (Yes in S44), the procedure is completed.

[0082] As apparent from the above description, according to the technique of the present embodiment, an accurate position and angle of the workpiece Wm to be measured are identified after the workpiece Wm to be measured is set and before machining of the workpiece Wm to be measured is started. This allows the offset amount to be accurately calculated and interference between the movable part and the workpiece Wm to be measured can be effectively prevented. Furthermore, the position and angle of the workpiece Wm to be measured are adjusted through a combination of image analysis and fine adjustment by the user. This allows the position and angle to be accurately calculated while reducing the user's effort and time.

[0083] The above-described structure is merely exemplary, and as long as the structure described in claim 1 is provided, the other structures can be appropriately modified. For example, in the above description, the number of characteristic parts 50 is described as one, but alternatively, the number of characteristic parts 50 set on a target may be more than one. Furthermore, a reference point serving as a reference for the position of the target is not limited to the center of gravity PG and may be another point. For example, the reference point may be the center O of a characteristic part 50 or a corner of the outer shape of the workpiece W.The angle A of the characteristic part is not limited to the direction angle of the characteristic part 50 when viewed from the center of gravity PG as described above, and may alternatively be the direction angle of a characteristic part 50 when viewed from another characteristic part 50.

[0084] Furthermore, in the above description, the characteristic part 50 is described as a molded part within the outer shape of the workpiece W. However, depending on the workpiece W, there may be cases where an upper surface of the workpiece W is flat and there is no characteristic molded part. For example, in the case where there is no specific characteristic part 50 within the workpiece W, as shown in Fig.10, the outer shape of the workpiece W itself may be treated as the characteristic part 50. In this case, the angle of the workpiece W is defined by an inclination angle of the outer shape of the workpiece W. Therefore, for example, an angle of an edge of the outer shape of the workpiece W or an inclination angle of the minimum-sized quadrilateral containing the workpiece W may be treated as the angle of the characteristic part A. Moreover, in the above description, the position and angle of the workpiece W are determined as placement. However, in the case of a workpiece W having a circular shape and no characteristic asperities, the position alone may be determined as placement. List of reference symbols

[0085] 10 Placement identifying device, 12 Imaging unit, 14 Camera, 16 Illumination device, 18 UI device, 20 Output device, 22 Input device, 30 Controller, 32 Processor, 34 Memory, 50 Characteristic part, 60 Reference image, 70 Position identifying image, 80 Overlap image, 100 Machine tool, 102 Machining chamber, 104 Spindle head, 106 Table, 110 NC device, 112 Registration screen, 130 Image load button, A Angle of characteristic part, C Size of characteristic part, L Distance of characteristic part, PG Center of gravity, W Workpiece. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] JP 2023-184247

[0001] JP 2010-078513 A [0005, 0007] JP H7-110217 A [0006, 0008]

Claims

[1] Placement identification device comprising: a camera that captures an image of a target placed within a predetermined area; a UI device that presents information to a user and receives an operating command from the user; and a controller configured to identify a placement of a measurement target, which is a target set at an arbitrary location, based on an image captured by the camera, The controller is designed to: pre-storing a reference image representing a shape of a reference target, which is a target fixed at a known location; Acquiring a position identification image by causing the camera to take an image of the measurement target; Identifying a preliminary placement of the measurement target based on the position identification image; preliminarily adjusting a position and an angle of the reference image based on the preliminarily placement so that the reference target shown in the reference image overlaps the measurement target, and generating an overlap image which is an image in which the reference image overlaps the position identification image; Presenting the overlap image to the user and receiving a command to fine-adjust the position and angle of the reference image from the user; and Identify a true placement of the measurement target based on the position and angle of the reference image after fine adjustment. [2] Placement identification device according to claim 1, wherein the target has a reference point and one or more characteristic parts, and the controller is further configured to define a position of the target relative to the reference point and to define an angle of the target relative to the one or more characteristic parts. [3] Placement identification device according to claim 2, wherein the reference point is a center of gravity of the target, each of the one or more characteristic parts is a shaped part that is present in an external shape of the target and that can be distinguished from an environment thereof, and the controller is further configured to define the angle of the target by a directional angle of each of the one or more characteristic parts when viewed from the reference point or another characteristic part. [4] Placement identification device according to claim 3, wherein the reference image is transparent to an extent that the position identification image is visible in the overlap image. [5] Placement identification device according to claim 4, wherein the reference image is a mask image in which parts other than the one or more characteristic parts in the reference target are masked. [6] Placement identification device according to one of claims 2 to 5, wherein the controller is further configured to identify a deviation amount between each of the one or more characteristic parts of the measurement target and a respective one of the one or more characteristic parts of the reference target in the overlap image, and then, if the deviation amount is less than a predefined allowable value, to identify the true placement of the measurement target based on the position and angle of the reference image after the preliminary adjustment, without presenting the overlap image to the user. [7] Placement identification device according to one of claims 2 to 5, wherein the controller is further configured to: Storing a size of the characteristic part and a relative position of the characteristic part with respect to the reference point in the reference target as characteristic information; Identifying a search area of ​​the characteristic part of the measurement target in the position identification image based on the characteristic information; and Outputting a target mismatch error when no shape matching the characteristic information is found in the search area of ​​the position identification image. [8] Placement identification device according to claim 2, wherein the one or more characteristic parts are an outer shape of the target and the controller is further configured to define the angle of the target by an inclination angle of the outer shape of the target. [9] Placement identification device according to one of claims 1 to 5, wherein the controller is further configured to calculate the preliminary placement of the measurement target based on an image obtained by binarizing the position identification image. [10] A placement identification device according to any one of claims 1 to 5, wherein the predetermined area is a machining chamber of a machine tool and the target is a workpiece that is fixed in the machining chamber and that is to be machined by the machine tool.

Images