Arrangement identification device

Abstract

To provide an arrangement identification device that can identify arrangement of an object more accurately and in a shorter time.SOLUTION: An arrangement identification device 10 includes a camera 14 for capturing an image of an object, a UI device 18, and a controller 30 configured to identify an arrangement of a measurement object. The controller 30 pre-stores a reference image 60 representing a shape of a reference object, identifies, on the basis of a position-identified image 70 which is a captured image of the measurement object, a tentative arrangement of the measurement object, preliminarily adjusts, on the basis of the tentative arrangement, a position and an angle of the reference image 60, generates a superimposed image 80 in which the reference image 60 is superimposed on the position-identified image 70, receives an instruction for fine-tuning the position and the angle of the reference image 60 from a user, and determines, on the basis of the fine-tuned reference image 60, a true arrangement of the measurement object.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] This specification discloses a configuration identification device that identifies the configuration of a measurement object set in a predetermined area. [Background technology]

[0002] Conventionally, there has been a demand for identifying the position of an object that has been set in an arbitrary position. For example, when machining a workpiece with a machine tool, an operator sets the workpiece, which is the object to be machined, in a machining chamber. At this time, the position and angle of the workpiece may deviate from a predefined reference position and reference angle. If the position of the workpiece deviates from the reference, there is a risk that a moving part of the machine tool (e.g., a tool, etc.) may unexpectedly interfere with the workpiece.

[0003] Conventionally, an operator would measure the layout of a workpiece before machining using a machine tool. This measurement is performed, for example, by manually or automatically operating a touch probe attached to a moving part of the machine tool. However, such measurements using a touch probe have the problem of being time-consuming.

[0004] Also, some have proposed a technique for detecting the arrangement of a workpiece by capturing an image of the set workpiece and analyzing the captured image of the workpiece. For example, Patent Document 1 discloses a technique for calculating the rough position of the workpiece by performing a pattern matching process on the captured image of the workpiece captured by a camera, and then calculating a more accurate position by using characteristic parts of the workpiece.

[0005] Furthermore, Patent Document 2 discloses a technique for analyzing a captured image of a workpiece, identifying the position of the center of gravity of the workpiece and a feature point of the workpiece, and determining the angle of the workpiece from the angle between the feature point and the position of the center of gravity. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2010-078513 A [Patent Document 2] Japanese Patent Application Publication No. 7-110217 Summary of the Invention [Problem to be solved by the invention]

[0007] However, in the case of a technique using pattern matching as in Patent Document 1, it is necessary to check the degree of match by fitting the pattern image to the captured image of the workpiece in a brute force manner while changing the position and angle, which causes a problem of long calculation times.

[0008] Furthermore, in the technology described in Patent Document 2, the captured image of the workpiece is analyzed to calculate the angle of the workpiece. However, the angle of the workpiece obtained by such image analysis is subject to errors due to fluctuations in the external environment, such as the lighting conditions. Therefore, in the technology of Patent Document 2, errors are likely to occur in the calculated position and angle of the workpiece. If a machine tool is operated based on position and angle data that includes errors, there is a risk of damage to the workpiece or tool.

[0009] Therefore, this specification discloses a location identification device that can identify the location of an object more accurately and in a shorter time. [Means for solving the problem]

[0010] The arrangement identification device disclosed in the present specification includes a camera that captures an image of an object that is placed in a predetermined area, a UI device that presents information to a user and accepts operation instructions from the user, and a controller configured to identify the arrangement of a measurement object, which is an object set in an arbitrary arrangement, based on an image captured by the camera, wherein the controller is configured to pre-store a reference image that represents a shape of a reference object, which is an object set in a known arrangement, cause the camera to capture the measurement object to obtain a position identification image, identify a tentative arrangement of the measurement object based on the position identification image, pre-adjust a position and angle of the reference image based on the tentative arrangement so that the reference object represented in the reference image overlaps the measurement object, generate a superimposed image that is an image in which the reference image is superimposed on the position identification image, present the superimposed image to the user, and accept an instruction from the user to fine-tune the position and angle of the reference image, and identify the true arrangement of the measurement object based on the position and angle of the reference image after the fine adjustment.

[0011] In this case, the object may have a reference point and one or more features, and the controller may be configured to determine a position of the object with respect to the reference point, and to determine an angle of the object with respect to the one or more features.

[0012] Furthermore, the reference point may be a center of gravity of the object, and each of the one or more features may be within the outline of the object and be a geometric feature distinguishable from its surroundings, and the controller may be configured to determine an angle of the object in terms of a directional angle of each of the one or more features relative to the reference point or another feature.

[0013] The reference image may have a transparency that allows the position specifying image to be visible in the superimposed image.

[0014] In this case, the reference image may be a mask image in which parts of the reference object other than the one or more characteristic parts are masked.

[0015] The controller may also be configured to determine an amount of misalignment between each of the one or more characteristic features of the measurement object and each of the one or more characteristic features of the reference object in the superimposed image, and if the amount of misalignment is less than a specified tolerance, determine the true location of the measurement object based on the position and angle of the reference image after the preliminary adjustment, without presenting the superimposed image to the user.

[0016] The controller may also be configured to store the size of the characteristic feature in the reference object and the relative position of the characteristic feature with respect to the reference point as characteristic information, identify a search range for the characteristic feature of the measurement object in the position identification image based on the characteristic information, and output an object mismatch error if a shape matching the characteristic information is not found within the search range of the position identification image.

[0017] The one or more features may be a contour of the object, and the controller may be configured to define the angle of the object in terms of an inclination angle of the contour of the object.

[0018] The controller may be configured to calculate a tentative position of the measurement object based on a binarized image of the position specifying image.

[0019] The specified area may be a machining chamber of a machine tool, and the object may be a workpiece that is fixed to the machining chamber and machined by the machine tool. Effect of the Invention

[0020] According to the location identification device disclosed in this specification, after the reference image is preliminarily adjusted, the user can fine-tune it, thereby enabling the location of the workpiece to be identified accurately in a short time. [Brief description of the drawings]

[0021] [Figure 1] FIG. 2 is a schematic diagram showing a configuration of a placement identification device. [Diagram 2] FIG. 2 is a schematic diagram of a reference work and a measurement work. [Diagram 3] FIG. 13 is a schematic diagram showing how centers of gravity and characteristic portions are extracted. [Figure 4] FIG. 13 is a schematic diagram showing a position-specific image and a negative-positive inverted image. [Diagram 5] FIG. 13 is a diagram illustrating an example of a reference image. [Figure 6] FIG. 13 is a diagram showing an example of a reference image registration screen. [Figure 7] FIG. 13 is a diagram showing an example of a superimposed image. [Figure 8] 11 is a flowchart showing the first half of a procedure for machining a measurement workpiece by a machine tool. [Figure 9] 13 is a flowchart showing the second half of the procedure for machining a measurement workpiece by a machine tool. [Figure 10] FIG. 13 is a diagram showing an example of another workpiece; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] The configuration of the layout identification device 10 will be described below with reference to the drawings. FIG. 1 is a schematic diagram showing the configuration of the layout identification device 10. This layout identification device 10 identifies the layout of an object set in a predetermined area. Here, in this example, the "layout" includes the "position" of the object in a two-dimensional plane, and may further include the "angle" of the object in the two-dimensional plane. Below, the layout identification device 10 configured in combination with a machine tool 100 will be described as an example. In this layout identification device 10, the "predetermined area" is the machining chamber 102 of the machine tool 100, and the "object" is the workpiece W machined by the machine tool 100. However, the layout identification device 10 described here is only an example. Therefore, it may be used in combination with other devices, not limited to the machine tool 100. Moreover, the layout identification device 10 may be used alone without being combined with other devices. Therefore, the "object" is not limited to the workpiece W, and may be other members.

[0023] First, a brief description will be given of the machine tool 100. The machine tool 100 performs machining on a workpiece W in accordance with an NC program instructed by a user. In the following, a machining center having a spindle head 104 will be described as an example.

[0024] A machining chamber 102 of the machine tool 100 is provided with a table 106 and a spindle head 104 that holds a tool. A person or a robot sets a workpiece W on the table 106 and then fixes the workpiece W to the table 106 using a magnetic chuck, a dedicated fixing jig, or the like. In order to machine the workpiece W, an NC device 110 of the machine tool 100 operates a movable part (e.g., the spindle head 104, etc.) according to an NC program specified by a user.

[0025] Here, an NC program is usually created on the assumption that the workpiece W is set at a predefined reference position and reference angle. However, in reality, some positioning error occurs in the workpiece W set in the machining chamber 102. If machine tool 100 is operated without correcting such positioning error, there is a risk that the moving parts of machine tool 100 will interfere with the workpiece W, or that the accuracy of the workpiece W (i.e., the product) obtained after machining is completed will decrease.

[0026] To prevent such interference and accuracy degradation, the user usually measures the positioning error of the workpiece W before starting processing and registers the positioning error as an offset amount in the NC device 110. The NC device 110 controls the movement of the movable part taking into account the input offset amount. This prevents interference between the movable part and the workpiece W and allows the accuracy of the product to be appropriately maintained.

[0027] However, conventionally, there has been a problem that measuring the placement error of the workpiece W takes time and effort. For example, in order to measure the position and angle of the workpiece W, a user may operate a touch probe manually or automatically. However, measurement using a touch probe takes time and effort. Some have proposed a technology for identifying the position and angle of the workpiece W based on an image of the workpiece W. However, with conventional technology, it takes time to calculate the position and angle, and the accuracy is low.

[0028] The layout identification device 10 identifies the position and angle, i.e., the layout, of the workpiece W. In the following, the directions parallel to the placement surface of the workpiece W are defined as the X and Y directions, and the direction perpendicular to the placement surface is defined as the Z direction. The position and angle of the workpiece W targeted by the layout identification device 10 in this example are the position of the workpiece W in the XY plane and the rotation angle of the workpiece W around an axis parallel to the Z direction.

[0029] The arrangement identification device 10 has an imaging unit 12, a UI device 18, and a controller 30. The imaging unit 12 captures an image of an object set in a processing room 102, that is, a workpiece W. The imaging unit 12 has, for example, a camera 14 and a light 16 for illuminating the workpiece W. The camera 14 may be fixedly disposed in the processing room 102, or may be attached to a position adjustment device such as a pan head or an XY table. When the position and attitude of the camera 14 are changeable, the position and attitude of the camera 14 are detected by a sensor and transmitted to the controller 30. The controller 30 converts between the camera coordinate system and the machine coordinate system of the machine tool 100 based on the position and attitude of the camera 14. The number of cameras 14 is not limited to one, and multiple cameras may be provided.

[0030] In this example, the camera 14 is disposed facing the surface on which the workpiece W is placed (i.e., the upper surface of the table 106), and has an optical axis that is approximately perpendicular to the surface. The camera 14 is capable of communicating with the controller 30 via a wire or wirelessly. The camera 14 operates according to a control signal transmitted from the controller 30. The camera 14 also transmits captured image data to the controller 30.

[0031] The lighting 16 illuminates the workpiece W. The lighting 16 may be fixedly disposed in the processing chamber 102, or may be disposed in a state in which the position and attitude can be changed. The lighting 16 is not limited to being one, and a plurality of lightings 16 may be provided. The lighting 16, like the camera 14, can communicate with the controller 30 wirelessly or by wire. The lighting 16 changes the light amount, color temperature, and irradiation direction according to a control signal transmitted from the controller 30.

[0032] The UI device 18 has an output device 20 that presents information to a user, and an input device 22 that accepts an operation instruction from the user. In this example, the output device 20 includes a display that displays a superimposed image 80, which will be described later, and the like. The input device 22 includes, for example, a keyboard, a touch panel, a mouse, a microphone, a barcode scanner, and the like. The output device 20 and the input device 22 may be provided independently of the machine tool 100, or may be incorporated into the machine tool 100. For example, a control panel and a display of the control panel provided in the machine tool 100 may be used as the input device 22 and the output device 20 of the layout identification device 10. Both the input device 22 and the output device 20 are capable of communicating with the controller 30 by wire or wirelessly.

[0033] The controller 30 specifies the arrangement (i.e., the position and angle) of the workpiece W based on the image of the workpiece W captured by the camera 14. The controller 30 is physically a computer having a processor 32 and a memory 34. The controller 30 can communicate with the NC device 110 by wire or wirelessly. In FIG. 1, the controller 30 is illustrated as a single computer independent of the NC device 110. However, the controller 30 may be configured by combining a plurality of physically separated computers. The NC device 110 may function as a part or the whole of the controller 30. In any case, the controller 30 specifies the arrangement of the workpiece W to be measured. Then, the controller 30 calculates the difference between the specified arrangement and a reference arrangement defined in advance as an offset amount, and transmits the offset amount to the NC device 110. The NC device 110 operates the movable part of the machine tool 100 taking into account the received offset amount.

[0034] Next, the determination of the arrangement of the workpiece W by the arrangement determination device 10 will be described. The upper part of FIG. 2 is a schematic diagram of a workpiece W whose arrangement is known, and the lower part of FIG. 2 is a schematic diagram of a workpiece W to be measured whose arrangement is arbitrary. Hereinafter, the workpiece W whose arrangement is known will be referred to as a "reference workpiece Wr", and the workpiece W to be measured will be referred to as a "measurement workpiece Wm". Furthermore, the reference numerals of elements related to the reference workpiece Wr will be given the suffix "r", and the reference numerals of elements related to the measurement workpiece Wm will be given the suffix "m". Furthermore, when there is no need to distinguish between the reference workpiece Wr and the measurement workpiece Wm, the suffixes r and m will be omitted and they will simply be referred to as "workpiece W". The same applies to other elements.

[0035] In this example, workpiece information is set for each type of workpiece W. The workpiece information is broadly divided into processing information and placement information. The processing information includes the product number of the workpiece W, identification information of the processing process of the workpiece W, and identification information of the NC program applied to the workpiece W.

[0036] The position information is information that represents the ideal position of the workpiece W, and is information that represents the position of the reference workpiece Wr. This position information includes the position of the reference point of the reference workpiece Wr, feature information, and a reference image 60 described later. In this example, the center of gravity PG of the workpiece W is treated as the reference point. In this example, a characteristic shape part (e.g., a protrusion, a hole, etc.) of the workpiece W is treated as a characteristic part 50. In the following, for simplicity of explanation, there is one characteristic part 50, but there may be multiple characteristic parts 50. In the example of FIG. 2, a cylindrical protrusion protruding from the surface of the workpiece W in the Z direction is treated as the characteristic part 50.

[0037] Therefore, in this example, the arrangement information includes the position of the center of gravity PGr of the reference work Wr, the relative position of the characteristic part 50r with respect to the center of gravity PGr, and the size of the characteristic part 50r. The relative position of the characteristic part 50 with respect to the center of gravity PG is expressed by the directional angle of the characteristic part 50 seen from the center of gravity PG (hereinafter referred to as "characteristic part angle A") and the distance from the center of gravity PG to the characteristic part 50 (hereinafter referred to as "characteristic part distance L"). In addition, the size of the characteristic part 50 will be referred to as "characteristic part size C" below. The coordinate system used to determine the angle is a linear orthogonal coordinate system (i.e., a Cartesian coordinate system).

[0038] When identifying the position of the measurement workpiece Wm, the controller 30 controls the imaging unit 12 to obtain a position identifying image 70 in which the measurement workpiece Wm is imaged. The controller 30 then analyzes the position identifying image 70 to identify the position of the center of gravity PGm of the measurement workpiece Wm and the characteristic part angle Am. When the measurement workpiece Wm can be ideally imaged, the position error between the center of gravity PGm of the measurement workpiece Wm and the center of gravity PGr of the reference workpiece Wr, and the angle error between the characteristic part angle Am of the measurement workpiece Wm and the characteristic part angle Ar of the reference workpiece Wr become offset amounts that represent the position error of the measurement workpiece Wm.

[0039] However, since various disturbances occur in the imaging environment of the measurement workpiece Wm, it is difficult to ideally image the measurement workpiece Wm. As a result, the position of the center of gravity PGm and the characteristic part angle Am obtained by simple image analysis usually contain a certain degree of error. This will be explained with reference to Figs. 3 to 5. First, the procedure for extracting the center of gravity PGm and the characteristic part 50m from the position identification image 70 will be briefly explained with reference to Fig. 3. Fig. 3 is a schematic diagram showing the extraction of the center of gravity PGm and the characteristic part 50m.

[0040] When extracting the center of gravity PGm and the characteristic portion 50m of the measurement workpiece Wm, the controller 30 first performs binarization processing on the position identification image 70 obtained by imaging the measurement workpiece Wm. Specifically, the controller 30 converts the position identification image 70 to grayscale and then performs binarization processing using an arbitrary threshold value. The grayscale conversion is performed, for example, by the NTSC weighted average method. Furthermore, the controller 30 may perform processing such as enlarging / reducing the image and emphasizing edges before this binarization processing, as necessary.

[0041] After the binarization process, the controller 30 performs negative-positive inversion of the binarized image. The upper part of Fig. 3 is a schematic diagram of a negative-positive inversion image. Note that in the upper part of Fig. 3, in order to make the symbols and the like easier to see, the black parts are not filled in but are hatched in gray.

[0042] Next, the controller 30 performs blob processing on the negative-positive inverted image. Note that this blob processing is normally performed on the negative-positive inverted image. However, in the middle and lower parts of FIG. 3, images that are not negative-positive inverted are shown for ease of viewing.

[0043] A blob is a group of pixels having the same density. The controller 30 extracts blobs from the negative-positive inverted image. This blob processing can be performed using conventional techniques, so a detailed description is omitted here. In any case, multiple blobs are extracted by the blob processing.

[0044] The controller 30 identifies the smallest rectangle that contains each of the extracted blobs. The controller 30 identifies the largest rectangle among the identified rectangles as a "workpiece containing rectangle Rw" that contains the outline of the measurement workpiece Wm. The controller 30 then calculates the center position of this workpiece containing rectangle Rw as the center of gravity PGm of the measurement workpiece Wm. The middle part of FIG. 3 shows how the workpiece containing rectangle Rw and center of gravity PGm are identified.

[0045] Furthermore, the controller 30 identifies a rectangle that contains the blob corresponding to the feature 50m as a "feature containing rectangle Rc."

[0046] To identify the feature-containing rectangle Rc, the controller 30 uses feature information of the reference work Wr (i.e., the feature angle Ar, feature distance Lr, and feature size Cr of the reference work Wr). Based on the feature angle Ar and feature distance Lr of the reference work Wr, the controller 30 identifies an approximate range in which the feature 50m of the measurement work Wm exists as a search range S (see the middle part of FIG. 3). Of the blobs existing in this search range S, the controller 30 identifies blobs having angles, distances, and sizes similar to the feature angle Ar, feature distance Lr, and feature size Cr of the reference work Wr as the feature 50m of the measurement work Wm.

[0047] If no appropriate blob is present within the search range S, it is considered that a work W different from the work W intended by the user has been placed. In this case, the controller 30 notifies the user of an error indicating a mismatch of the target objects, and ends the placement identification process.

[0048] If the controller 30 can identify the blob of the feature portion 50m, it determines the smallest rectangle that contains the blob as the feature portion containing rectangle Rc (see the lower part of FIG. 3).The controller 30 then calculates the directional angle of the center of the feature portion containing rectangle Rc from the center of gravity PGm as the feature portion angle Am of the feature portion 50m.

[0049] When an ideal position specifying image 70 is obtained, the center of gravity PGm and the characteristic part angle Am of the measurement work Wm calculated in this manner become information indicating the arrangement of the measurement work Wm. In other words, the positional deviation amount between the center of gravity PGr and the center of gravity PGm, and the angle deviation amount between the characteristic part angle Ar and the characteristic part angle Am become the offset amount of the measurement work Wm.

[0050] As is clear from the above description, in this example, the center of gravity PGm and the characteristic part 50m of the measurement workpiece Wm are extracted based on the binary image of the position identification image 70. However, the binary image of the position identification image 70 is likely to contain errors due to the shaking of the camera 14 and the lighting 16. If the position of the center of gravity PGm and the characteristic part angle Am are calculated based on the binary image containing such errors, the position of the center of gravity PGm and the characteristic part angle Am finally obtained will also contain errors. In addition, there are slight individual differences in the measurement workpiece Wm, and it does not completely match the shape of the reference workpiece Wr. Image analysis alone cannot compensate for such subtle individual differences.

[0051] For example, as shown in the upper part of FIG. 4, a case where strong shadows or reflections are generated at the ends of the measurement workpiece Wm or the characteristic part 50m due to the shaking of the camera 14 or the lighting 16, etc., is considered. In this case, the measurement workpiece Wm originally has edges shown by two-dot chain lines, but due to the influence of shadows and reflections, edges shown by solid lines are extracted when binarization processing is performed. The lower part of FIG. 4 shows a negative-positive inverted image generated based on such a binarized image containing errors. If the center of gravity PGm and the characteristic part angle Am are calculated based on a binarized image containing errors, naturally, errors will also occur in the center of gravity PGm and the characteristic part angle Am. In addition, even if the measurement workpiece Wm can be ideally imaged, if there are slight individual differences in the shape of the measurement workpiece Wm, the individual differences cannot be appropriately processed by image analysis alone. As a result, an appropriate offset amount cannot be set, which may lead to interference between the moving part and the workpiece W and a decrease in the accuracy of the product.

[0052] Therefore, in this embodiment, a reference image 60 is prepared in advance, and the error is eliminated by using this reference image 60. This will be described in detail below.

[0053] FIG. 5 is a diagram showing an example of a reference image 60. The reference image 60 is an image showing the shape of the reference workpiece Wr, in particular the outer shape and characteristic portion 50r of the reference workpiece Wr. As described above, the reference workpiece Wr is a workpiece whose arrangement is known. The reference image 60 is a masked image in which parts other than the characteristic portion 50r are masked. Therefore, the shape of the reference workpiece Wr disappears in parts of the reference image 60 other than the characteristic portion 50r. Note that, although the center of gravity PGr is shown in FIG. 5 for the sake of explanation, the center of gravity PGr is not shown in the actual reference image 60.

[0054] As will be described in detail later, the reference image 60 is superimposed on the position specifying image 70 to form a superimposed image 80. The reference image 60 has transparency to such an extent that the shape of the measurement workpiece Wm shown in the position specifying image 70 can be visually recognized in the superimposed image 80.

[0055] A reference image 60 is prepared for each type of workpiece W. In addition, the reference image 60 is linked to the position of the center of gravity PGr, feature information (i.e., feature angle Ar, feature distance Lr, feature size Cr), and processing information (i.e., NC program, etc.).

[0056] Such a reference image 60 may be generated based on an image obtained by capturing an image of a reference workpiece Wr that actually exists. In this case, the user first fixes the workpiece W to the table 106 of the processing chamber 102 and precisely measures the position and angle of the workpiece W. For example, the user may measure the position and angle of the workpiece W using a sensor such as a touch probe after fixing the workpiece W in the processing chamber 102. The user may also fix the workpiece W to a pallet outside the processing chamber 102 and measure the position of the workpiece W on the pallet. In this case, the user fixes the pallet to the processing chamber 102 without shifting the position and angle, so that the position and angle of the workpiece W in the processing chamber 102 become known. Since the position and angle in the processing chamber 102 are known, the workpiece W can be treated as a reference workpiece Wr.

[0057] When the reference workpiece Wr is set in the processing chamber 102, the controller 30 drives the imaging unit 12 to capture an image of the reference workpiece Wr. For example, the controller 30 may capture an image of the reference workpiece Wr multiple times while changing the imaging conditions (e.g., the driving conditions of the camera 14 and the lighting 16, etc.). In this case, the captured images may be synthesized to capture one image so that the outer shape and characteristic parts 50r of the reference workpiece Wr can be accurately extracted by image analysis. Also, if an image from which the outer shape and characteristic parts 50r of the reference workpiece Wr can be extracted can be obtained by one imaging, the number of imaging operations may be one. In any case, once an appropriate captured image is obtained, the captured image is processed to generate a reference image 60.

[0058] For example, the user uses any drawing application (such as "Microsoft Paint," "Microsoft" is a registered trademark) to fill in the image of the reference work Wr except for the characteristic portion 50r, and further generates a mask image in which the filled-in image is made transparent as the reference image 60. The transmittance of the reference image 60 may be uniform or may be different between the characteristic portion 50r and the other portions.

[0059] The controller 30 also extracts the center of gravity PGr and the characteristic portion 50r of the reference workpiece Wr from the captured image of the reference workpiece Wr. Specifically, the controller 30 performs grayscale conversion, binarization, negative-positive inversion, and blob processing on the captured image of the reference workpiece Wr. The controller 30 then extracts a rectangle that contains the largest blob as a workpiece-containing rectangle Rw. The controller 30 calculates the center of the workpiece-containing rectangle Rw as the center of gravity PGr of the reference workpiece Wr.

[0060] The controller 30 also presents the image after blob processing to the user, and asks the user to select a blob to be used as the feature 50r. If the user selects an arbitrary blob, the controller 30 extracts the inclusive rectangle of the selected blob as the feature inclusive rectangle Rc. The controller 30 then calculates the center of the feature inclusive rectangle Rc as the position of the feature 50r, and calculates the feature information, i.e., the feature angle Ar, feature distance Lr, and feature size Cr of the reference workpiece Wr, based on this position. The controller 30 then associates these obtained values ​​with the reference image 60 and stores them in the memory 34.

[0061] As another embodiment, the reference image 60 may be generated based on CAD data or a design drawing of the reference work Wr, instead of the captured image of the reference work Wr. In this case, the controller 30 may generate a schematic plan view of the reference work Wr based on the CAD data or the design drawing of the reference work Wr, and process this plan view to generate the reference image 60. The controller 30 may also calculate the position of the center of gravity PGr of the reference work Wr, the characteristic part angle Ar, the characteristic part distance Lr, and the characteristic part size Cr, based on the plan view.

[0062] FIG. 6 is a diagram showing an example of a registration screen 112 of the reference image 60. This registration screen 112 is displayed on the display of the UI device 18. In the example of FIG. 6, the user presses the image reading button 130 to select the reference image 60 generated in advance. The selected reference image 60 is displayed on the registration screen 112. In the example of FIG. 6, the product number of the workpiece W, the number of the processing step performed on the workpiece W, and the identification information of the NC program applied to the workpiece W are set as the processing information 132. Such processing information 132 may be manually input by the user, or may be called by reading the identification information (e.g., barcode, etc.) attached to the workpiece W using the input device 22 (e.g., barcode scanner, etc.). In addition, the registration screen 112 displays feature information 134 linked to the reference image 60. Note that FIG. 6 shows a case where the feature portion 50r is two. If the user is satisfied with the displayed information, he or she presses the OK button 136 to register the reference image 60 in association with the feature information and processing information.

[0063] Next, the principle of identifying the position of the measurement workpiece Wm using the reference image 60 will be described. As described above, when identifying the position of the measurement workpiece Wm, the controller 30 captures the measurement workpiece Wm to obtain a position identification image 70. Furthermore, based on an image obtained by binarizing and inverting the negative-positive of the position identification image 70, the position of the center of gravity PGm of the measurement workpiece Wm and the characteristic part angle Am are calculated. However, as described above, the center of gravity PGm and the characteristic part angle Am usually contain errors. In order to easily correct this error, the controller 30 generates a superimposed image 80, which is an image in which the reference image 60 is superimposed on the position identification image 70. FIG. 7 is a diagram showing an example of the superimposed image 80.

[0064] When generating the superimposed image 80, the controller 30 preliminarily adjusts the reference image 60 based on the position error of the center of gravity PG between the reference work Wr and the measurement work Wm and the angle error of the characteristic portion 50. That is, the controller 30 preliminarily adjusts the position of the reference image 60 so that the center of gravity PGr of the reference work Wr coincides with the tentative center of gravity PGm of the measurement work Wm obtained from the negative-positive inverted image of the position specifying image 70. The controller 30 also preliminarily adjusts the angle of the reference image 60 so that the characteristic portion angle Ar of the reference work Wr coincides with the tentative characteristic portion angle Am of the measurement work Wm obtained from the binarized image of the position specifying image 70. Then, the controller 30 generates the superimposed image 80 by superimposing the reference image 60 after the preliminarily adjustment on the position specifying image 70.

[0065] The generated superimposed image 80 is presented to the user. Here, the position identification image 70 is an image before binarization, and is an image with sufficient gradation. Therefore, even if the reference workpiece Wr shown in the position identification image 70 has strong shadows or reflections, the user can grasp the shape of the reference workpiece Wr.

[0066] That is, by looking at the superimposed image 80, the user can grasp the shape and position of each of the measurement workpiece Wm and the reference image 60. In particular, in this example, the reference image 60 is a masked image in which parts other than the characteristic part 50r are masked and has transparency. Therefore, the user can clearly grasp the shape of the part of the measurement workpiece Wm that overlaps with the reference workpiece Wr. As a result, the user can easily recognize the position and angle deviation of the reference image 60 relative to the measurement workpiece Wm. The controller 30 presents such a superimposed image 80 to the user and then asks the user to fine-tune the position and angle of the reference image 60.

[0067] While viewing the superimposed image 80 displayed on the display, the user fine-tunes the position and angle of the reference image 60 so that the reference image 60 accurately overlaps the measurement workpiece Wm. Specifically, the user operates the input device 22, such as a keyboard, to specify the amount of movement and rotation of the reference image 60. The controller 30 reflects the amount of correction input by the user in the superimposed image 80 as needed. Once the reference image 60 accurately overlaps the measurement workpiece Wm through fine adjustment, the user instructs calculation of the position and angle of the measurement workpiece Wm.

[0068] When such an instruction is received, the controller 30 calculates the position of the center of gravity PGr and the characteristic part angle Ar of the reference image 60 after fine adjustment, and regards these as the position of the true center of gravity PGm and the true characteristic part angle Am of the measurement workpiece Wm. Then, the controller 30 calculates the offset amount of the measurement workpiece Wm based on the position of the true center of gravity PGm and the true characteristic part angle Am, and transmits the offset amount to the NC device 110.

[0069] In this way, a user who knows the actual shape of the measurement workpiece Wm can more accurately specify the position and angle of the measurement workpiece Wm by fine-tuning the position and image of the reference image 60. In addition, by pre-adjusting the position and angle of the reference image 60 before the user's fine adjustment, the amount of fine adjustment by the user can be reduced, and the effort and time required for fine adjustment can be significantly reduced. Furthermore, as described above, the reference image 60 is a mask image in which parts other than the characteristic part 50r are masked and has transparency. Therefore, the user can properly recognize the shape of the measurement workpiece Wm even in the superimposed image 80, and can easily fine-tune the reference image 60.

[0070] Note that, in the stage after the preliminary adjustment, if the amount of positional misalignment between the characteristic portion 50r in the reference image 60 and the characteristic portion 50m in the position identification image 70 is equal to or less than a specified tolerance, the true position and angle of the measurement workpiece Wm may be obtained without requesting fine adjustment. For example, as shown in Fig. 7, in the stage after the preliminary adjustment, the controller 30 respectively identifies the center point Or of the characteristic portion 50r and the center point Om of the characteristic portion 50m, and obtains the distance between the two points Or and Om. The controller 30 may request the user to perform fine adjustment only if this distance is equal to or greater than a specified tolerance.

[0071] Next, a procedure for machining the measurement workpiece Wm by the machine tool 100 will be described with reference to Figs. 8 and 9. When machining the measurement workpiece Wm by the machine tool 100, the user starts up and initializes the machine tool 100 and the arrangement identification device 10 (S10). In the initialization, communication is established between the controller 30 and the NC device 110, and between the controller 30 and the imaging unit 12. The controller 30 also acquires conversion parameters between the camera coordinate system and the machine coordinate system. For example, the controller 30 captures an image of a reference portion (e.g., a mark affixed to the origin of the machine coordinate system) whose position and size are known, using the camera 14, and identifies the conversion parameters of the coordinate system based on the position and size of the reference portion in the image obtained.

[0072] Next, the user places and fixes the measurement workpiece Wm on the table 106 in the processing chamber 102 (S12). At this time, the positioning of the measurement workpiece Wm does not need to be performed precisely, and visual positioning is sufficient.

[0073] Thereafter, the user inputs processing information corresponding to the measurement workpiece Wm (S14). The processing information includes, for example, the product number of the measurement workpiece Wm, the number of the process to be executed, the name of the NC program, etc. The processing information may be input manually by the user by operating a keyboard or the like. Also, identification information (e.g., a barcode, etc.) of the processing information may be attached in advance to the measurement workpiece Wm or the fixing jig, and this identification information may be read by the controller 30.

[0074] The controller 30 checks whether or not there is a reference image 60 associated with the input processing information (S16). If there is no corresponding reference image 60, the controller 30 prompts the user to register the reference image 60. In this case, the user calls up the registration screen 112 shown in Fig. 6 and registers the reference image 60 and feature information through the registration screen 112 (S18).

[0075] Once the reference image 60 is registered, the controller 30 starts capturing an image of the measurement workpiece Wm (S20 to S26). Specifically, the controller 30 adjusts the image capturing conditions (S20). The image capturing conditions include conditions related to the camera 14 and conditions related to the lighting 16. The conditions related to the camera 14 are, for example, the position of the camera 14, the shutter speed, auto white balance, etc. Furthermore, the conditions related to the lighting 16 are the position, brightness, color temperature, etc. of the lighting 16. These image capturing conditions are set and registered in advance. Once the adjustment of the image capturing conditions is completed, the controller 30 drives the imaging unit 12 to capture an image of the measurement workpiece Wm (S24). The obtained image is temporarily stored in the memory 34. Thereafter, the controller 30 repeats capturing an image of the measurement workpiece Wm while changing the image capturing conditions until the required number of images are obtained (S20 to S26).

[0076] When the required number of images are obtained, the controller 30 combines the images to generate one position-specific image 70 (S28). In this case, the controller 30 extracts only the parts of the measurement workpiece Wm that are well-viewed from each of the multiple captured images and combines them. For example, when combining a first captured image obtained with the illumination 16 (see FIG. 1) shining from the left side of the measurement workpiece Wm and a second captured image obtained with the illumination 16 shining from the right side of the measurement workpiece Wm, the left half of the first captured image and the right half of the second captured image are combined. With this configuration, the entire measurement workpiece Wm becomes bright, and the shading of the unevenness of the measurement workpiece Wm becomes clear, resulting in an image that is easy to analyze. Note that if an appropriate image is obtained, the number of times of imaging may be one, and in this case, the image combination process (S28) is naturally omitted.

[0077] Next, the controller 30 extracts the center of gravity PGm and the characteristic portion 50m of the measurement workpiece Wm based on the obtained position specifying image 70, and calculates the position of the tentative center of gravity PGm and the tentative characteristic portion angle Am (S30). Next, the controller 30 preliminarily adjusts the position and angle of the reference image 60 based on the obtained position of the tentative center of gravity PGm and the tentative characteristic portion angle Am (S32).

[0078] The controller 30 overlays the reference image 60 after the preliminary adjustment on the position identification image 70 to generate a superimposed image 80 (S34). Next, the controller 30 judges whether fine adjustment of the reference image 60 is necessary (S36). For example, when the controller 30 receives an instruction from the user that fine adjustment is not necessary, the controller 30 proceeds to step S39. The controller 30 may also judge whether fine adjustment is necessary based on the amount of positional deviation between the center of the characteristic portion 50m of the measurement workpiece Wm and the center of the characteristic portion 50r of the reference workpiece Wr in the superimposed image 80. When the amount of positional deviation is equal to or less than a predetermined allowable value, the controller 30 judges that fine adjustment is not necessary and proceeds to step S39.

[0079] On the other hand, if it is determined that fine adjustment is necessary, the controller 30 displays the superimposed image 80 on the display and prompts the user to fine-tune the reference image 60. If the user instructs fine adjustment of the position and angle of the reference image 60 in response to this request, the controller 30 corrects the position and angle of the reference image 60 in response to the instruction and regenerates the superimposed image 80 (S38, S34). The regenerated superimposed image 80 is displayed on the display.

[0080] The user or the controller 30 determines whether further fine adjustment is necessary based on the regenerated superimposed image 80 (S36). Then, if it is ultimately determined that fine adjustment is unnecessary, the controller 30 regards the center of gravity PGr and the characteristic part angle Ar of the reference work Wr included in the reference image 60 in the superimposed image 80 as the true center of gravity PGm and the true characteristic part angle Am of the measurement work Wm, and calculates the position of the measurement work Wm (S39).

[0081] Thereafter, the controller 30 calculates an offset amount based on the calculated true position and true angle of the measurement workpiece Wm, and registers this in the NC device 110 (S40). The NC device 110 controls the movable part of the machine tool 100 while reflecting the registered offset amount, and executes machining of the measurement workpiece Wm (S42). Then, when all machining of the workpiece W to be machined with the current machining information is completed (Yes in S44), the process ends.

[0082] As is clear from the above description, according to the technology of this example, after the measurement workpiece Wm is set, the exact position and angle of the measurement workpiece Wm are identified before starting machining of the measurement workpiece Wm. As a result, the offset amount can be accurately calculated, and interference between the movable part and the measurement workpiece Wm can be effectively prevented. In addition, the position and angle of the measurement workpiece Wm are determined by combining image analysis and fine adjustment by the user. As a result, the position and angle can be accurately calculated while reducing the user's effort and time.

[0083] The configuration described above is an example, and other configurations may be changed as appropriate as long as the configuration described in claim 1 is included. For example, in the above description, the characteristic portion 50 is described as one, but the characteristic portion 50 set in one object may be multiple. In addition, the reference point that serves as the reference for the position of the object is not limited to the center of gravity PG, but may be another point. For example, the reference point may be the center point O of one characteristic portion 50, or may be a corner of the outer shape of the workpiece W. In addition, in the above description, the characteristic portion angle A is not limited to the directional angle of the characteristic portion 50 as viewed from the center of gravity PG, but may be the directional angle of another characteristic portion 50 as viewed from one characteristic portion 50.

[0084] In addition, in the above description, the characteristic portion 50 is a shape portion inside the outer shape of the workpiece W. However, in some workpieces W, the upper surface of the workpiece W is flat and there are no characteristic shape portions. For example, as shown in FIG. 10, when there is no particular characteristic portion 50 inside the workpiece W, the outer shape of the workpiece W itself may be treated as the characteristic portion 50. In this case, the angle of the workpiece W is defined by the inclination angle of the outer shape of the workpiece W. Therefore, for example, the angle of the edge of the outer shape of the workpiece W or the inclination angle of the smallest rectangle that contains the workpiece W may be treated as the characteristic portion angle A. In addition, in the above description, the position and angle of the workpiece W are obtained as the arrangement. However, in the case of a workpiece W that is circular and does not have a characteristic unevenness, only the position may be obtained as the arrangement. [Explanation of symbols]

[0085] 10 Layout identification device, 12 Imaging unit, 14 Camera, 16 Lighting, 18 UI device, 20 Output device, 22 Input device, 30 Controller, 32 Processor, 34 Memory, 50 Feature part, 60 Reference image, 70 Location identification image, 80 Overlaid image, 100 Machine tool, 102 Machining room, 104 Spindle head, 106 Table, 110 NC device, 112 Registration screen, 130 Image loading button, A Feature part angle, C Feature part size, L Feature part distance, PG Center of gravity, W Workpiece.

Claims

1. A camera that captures an image of an object placed in a predetermined area; a UI device that presents information to a user and receives operation instructions from the user; A controller configured to identify the arrangement of a measurement object, which is an object set in an arbitrary arrangement, based on an image captured by the camera; Equipped with The controller: A reference image representing the shape of a reference object which is an object set in a known arrangement is stored in advance; The camera captures an image of the measurement object to obtain a position-specific image; Identifying a tentative location of the measurement object based on the position identification image; preliminarily adjusting a position and an angle of the reference image based on the provisional arrangement so that the reference object shown in the reference image overlaps the measurement object, and then generating a superimposed image which is an image in which the reference image is superimposed on the position specifying image; presenting the superimposed image to the user and receiving an instruction from the user to fine-tune the position and angle of the reference image; determining a true location of the measurement object based on the position and angle of the reference image after the fine adjustment; The arrangement identification device is characterized by being configured as follows.

2. The arrangement identification device according to claim 1 , the object has a reference point and one or more features; the controller is configured to define a position of the object relative to the reference point and define an angle of the object relative to the one or more features. A configuration identification device.

3. The arrangement identification device according to claim 2, the reference point is the center of gravity of the object, Each of the one or more features is within the outline of the object and has a shape that is distinguishable from its surroundings; the controller is configured to define an angle of the object in terms of an orientation angle of each of the one or more features relative to the reference point or other features; A configuration identification device.

4. The arrangement identification device according to claim 3, The location specifying device, wherein the reference image has a transparency to such an extent that the location specifying image can be seen in the superimposed image.

5. The arrangement identification device according to claim 4, The layout identification device, wherein the reference image is a mask image in which portions of the reference object other than the one or more characteristic portions are masked.

6. The arrangement identification device according to any one of claims 2 to 5, the controller is configured to determine an amount of misalignment between each of the one or more characteristic features of the measurement object and each of the one or more characteristic features of the reference object in the superimposed image, and if the amount of misalignment is less than a specified tolerance, to determine a true arrangement of the measurement object based on a position and angle of the reference image after the preliminary adjustment, without presenting the superimposed image to the user.

7. The arrangement identification device according to any one of claims 2 to 5, The controller: storing a size of the feature of the reference object and a relative position of the feature with respect to the reference point as feature information; specifying a search range for the characteristic portion of the measurement object in the position specifying image based on the characteristic information; If a shape matching the feature information is not found within the search range of the position-specified image, an error indicating an object mismatch is output. The arrangement identification device is characterized by being configured as follows.

8. The arrangement identification device according to claim 2, the one or more features are an external shape of the object; The controller is configured to define an angle of the object in terms of a tilt angle of a contour of the object. A configuration identification device.

9. The arrangement identification device according to any one of claims 1 to 5, The arrangement specifying device, wherein the controller is configured to calculate a tentative arrangement of the measurement object based on a binarized image of the position specifying image.

10. The arrangement identification device according to any one of claims 1 to 5, the predetermined area is a machining room of a machine tool, The object is a workpiece fixed in the machining chamber and machined by the machine tool. A configuration identification device.

Images