Method and apparatus for detecting objects with calibration function

Abstract

To provide a method and a device for detecting a target object having a calibration function that can maintain an accuracy of detecting a target object regardless of change of outside environments including temperatures.SOLUTION: The method for detecting a target object includes the steps of: calculating a determined value; measuring a first target object by a first positional information generation device; measuring a second target object by a second positional information generation device; calculating the position of the second target object based on the first positional information generation device, on the basis of the measured value of the first target object obtained by the first positional information generation device, the measured value of the second target object obtained by the second positional information generation device, and the determined value; and calibrating the determined value before a third target object is measured by the second positional information generation device.SELECTED DRAWING: Figure 9

Description

Technical Field

[0001] The present invention relates to a method and a detection device for detecting an object having a calibration function.

Background Art

[0002] Conventionally, a technique of welding one work while gripping another work with an arm robot is known. In this case, a dedicated positioning mechanism (including a positioning mold or jig) is used to position (detect the position and orientation) of the arm robot and thus the work gripped by the arm robot. However, since a separate positioning mechanism has to be provided, the equipment cost is high and a large installation space for the equipment is required. Moreover, since the positioning mechanism is required for each vehicle model, when there are many vehicle models, the number of changeovers is large and the total changeover time becomes long. Also, the same problem occurs every time a new vehicle model is launched.

[0003] Patent Document 1 describes a method for positioning at least one object in a final posture by an industrial robot in a space. In this method, in order to position the object, a first industrial robot, a first optical imaging device, and at least one second optical imaging device are used. The first industrial robot can be position-adjusted to a predetermined positioning. The first optical imaging device is calibrated in a three-dimensional space coordinate system and is positioned in a known direction at a known first position. The second optical imaging device is calibrated in a space coordinate system and is positioned in a known direction at a known second position.

[0004] The first optical imaging device has a first camera, a first drive unit, and a first angle measurement unit. The first camera is optically calibrated to capture an image within a predetermined first field of view. The first drive unit is a component for orienting the first camera to adjust the position of the first field of view. The first angle measurement unit is calibrated in a space coordinate system for detecting the angular direction of the first camera and obtaining the first field of view in the space coordinate system.

[0005] The second optical imaging device comprises a second camera, a second drive unit, and a second angle measuring unit. The second camera is optically calibrated to capture an image within a predetermined second field of view. The second drive unit is a component for orienting the second camera to adjust the position of the second field of view. The second angle measuring unit is calibrated in a spatial coordinate system to detect the angular direction of the second camera and determine the second field of view in the spatial coordinate system. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Patent No. 5290324 [Overview of the project] [Problems that the invention aims to solve]

[0007] However, Patent Document 1 lacked a function to calibrate against changes in the external environment (such as temperature), making it difficult to maintain the initially set accuracy, thus leaving room for improvement.

[0008] This invention was made based on the above-mentioned concerns, and aims to provide an object detection method and detection device that have a calibration function that can maintain the detection accuracy of an object regardless of changes in the external environment (such as temperature). [Means for solving the problem]

[0009] The object detection method having a calibration function in this embodiment is: This is the origin position and orientation of the second position information generation device, based on the position correction marker. A step of calculating a quantitative value, and the first object using the first position information generation device The position correction marker as The steps to measure, The aforementioned The second object is generated by the second location information generator. Shape marker The steps of measuring and the first position information generating device The aforementioned position correction marker The measured values ​​and the second position information generating device The aforementioned shape markerThe measured values ​​and, The position and orientation of the origin of the second position information generating device, with reference to the position correction marker. Based on the aforementioned quantitative values, the first position information generation device is used as a reference. The aforementioned shaped marker The steps include calculating the position of the third object and measuring the third object with the second position information generating device, The position and orientation of the origin of the second position information generating device, with reference to the position correction marker. The step of calibrating the aforementioned quantitative value, and In the step of calibrating the quantitative value, the quantitative value, which is the position and orientation of the origin of the second position information generator relative to the position correction marker, is calibrated by calculating the measurement value of the position correction marker by the first position information generator and the measurement value of the shape marker by the second position information generator, based on the position of the shape marker relative to the first position information generator. It is characterized by the following: [Effects of the Invention]

[0010] According to the present invention, it is possible to provide an object detection method and detection apparatus having a calibration function that can maintain the detection accuracy of an object regardless of changes in the external environment (such as temperature). [Brief explanation of the drawing]

[0011] [Figure 1] This figure shows an example of how to represent the information of each point in a point cloud. [Figure 2] This figure shows an example of how the origin can be represented in a 4x4 matrix. [Figure 3] This figure shows an example of origin calculation using a 4x4 matrix. [Figure 4] This figure shows a first example of the configuration of a welding apparatus for vehicle parts to which the object detection method and detection device having calibration function of this embodiment are applied. [Figure 5] This figure shows a second example of the configuration of a welding apparatus for vehicle parts to which the object detection method and detection device having a calibration function of this embodiment are applied. [Figure 6] This figure shows a third example of the configuration of a welding apparatus for vehicle parts to which the object detection method and detection device having a calibration function of this embodiment are applied. [Figure 7] This figure shows the first example of a shape marker. [Figure 8] This figure shows a second example of a shape marker. [Figure 9] This figure shows an example of a position calibration process between a second position information generation device and a position correction marker. [Modes for carrying out the invention]

[0012] First, definitions of terms used in this specification are provided.

[0013] The "position information generation device (3D scanner)" in this specification is a device for acquiring shape information of a workpiece (e.g., upper sash, pillar sash, door frame, bracket, etc.), its gripping device (e.g., robot arm, etc.), and other targets (e.g., markers or fixtures provided near the robot arm) as point cloud data. Each point information of the point cloud data can be read as "the position and orientation of the object". Each point information (the position and orientation of the object) of the point cloud data can be represented by numerical values of coordinates X, Y, Z with the 3D scanner origin as the origin and its normal directions I, J, K. In this specification, the "3D scanner" may be referred to as the "second position information generation device".

[0014] The "position information generation device (laser tracker)" in this specification is a device that can calculate, for example, three-dimensional information (position information and angles) of a target based on its own coordinate system. The "position information generation device (laser tracker)" acquires three-dimensional position information of the target, for example, by irradiating laser light and having the laser light reflected from the target return to the light source. The three-dimensional information (position information and angles) of the target can be read as "the position and orientation of the object". Since the measurement range of the "position information generation device (laser tracker)" is wide, if there is a "position information generation device (laser tracker)" and a target, or something like a touch probe with a marker described later, the dimensions, accuracy, origin position, etc. between each facility can be calculated with high accuracy. In this specification, the "position information generation device (laser tracker)" may be referred to as the "first position information generation device".

[0015] As used in this specification, a "marker" is, for example, a type of target for acquiring position coordinates and angles using a first position information generation device (laser tracker) or a second position information generation device (3D scanner). The position coordinates and angles of the "marker" are calculated with the first position information generation device (laser tracker) or the second position information generation device (3D scanner) as the origin. These can be represented by coordinates X, Y, Z and angles Rx, Ry, Rz. The angles Rx, Ry, Rz can each be represented by 4×4 matrix data (matrix data) using unit vectors (I, J, K) for the posture change from the reference X-axis, reference Y-axis, and reference Z-axis, respectively, and the movement amounts (X, Y, Z) from the reference point, as will be described later. The "marker" is, for example, attached to a gripping part at the tip of a robotic arm and is used to obtain, as a numerical value, the operating accuracy of the robotic arm (the absolute position of the gripping part in three-dimensional space) with the first position information generation device (laser tracker) or the second position information generation device (3D scanner) as the origin. In addition, it is widely used in other types such as a touch probe with a marker for measuring a target workpiece by contact as described later, or a type attached to a 3D scanner for non-contact measurement of the target workpiece. It is used for placement at the origin of the first position information generation device (laser tracker) or the second position information generation device (3D scanner) in any measurement information.

[0016] As used in this specification, a "touch probe with a marker" is, for example, a device that can calculate the position of a target workpiece in three-dimensional space with the first position information generation device (laser tracker) or the second position information generation device (3D scanner) as the origin by touching the tip of the probe to the target workpiece for measurement while the target part of the "touch probe with a marker" is locked on by the first position information generation device (laser tracker) or the second position information generation device (3D scanner).

[0017] In this specification, a "3D scanner with a marker" refers to, for example, a 3D scanner with a marker that has been locked onto the target area of ​​the "3D scanner with a marker" using a first position information generation device (laser tracker). By 3D scanning the target workpiece, it is possible to calculate the origin of the first position information generation device (laser tracker) from the marker origin coordinates / angle from the origin of the first position information generation device (laser tracker) and the origin coordinates / angle of the second position information generation device (3D scanner) from the marker origin (described later). The relationship between the marker origin position and the origin position of the second position information generation device (3D scanner) is determined at the time of installation by calculating the position information of a reference block using the first position information generation device (laser tracker) and a marker-equipped touch probe, and then matching (calibrating) the position information of the reference block imaged by the second position information generation device (3D scanner) to calculate the origin / angle information of the second position information generation device (3D scanner). Typically, the second position information generator (3D scanner) has the characteristic of decreasing accuracy when the imaging range is widened, making it unsuitable for imaging wide areas, and there are limitations to using the second position information generator (3D scanner) alone for the operation of the entire facility as described later. For this reason, it is conceivable to use the first position information generator (laser tracker) to widen the imaging range and measurement range. However, as described later, in the object detection method and detection device of this embodiment, by appropriately combining the first position information generator (laser tracker) and the second position information generator (3D scanner), the first position information generator (laser tracker) can be used only for initial setup (calibration), and then the final position correction can be performed using the second position information generator (3D scanner) and shape marker. For this reason, for example, it becomes unnecessary to install a dedicated first position information generator (laser tracker) in each of multiple manufacturing lines, and the first position information generator (laser tracker) can be reused in multiple manufacturing lines.

[0018] In this specification, "point cloud data" refers to three-dimensional information of a workpiece (e.g., upper sash, column sash, door frame, bracket, etc.) obtained by 3D scanning (imaging the workpiece) using equipment such as a 3D scanner (camera). Furthermore, the information of each point in the point cloud data can be represented by numerical values ​​of coordinates X, Y, Z and their normal directions I, J, K (Figure 1), for example. In the example in Figure 1, point cloud 1, point cloud 2, point cloud 3, point cloud 4, point cloud 5, and point cloud** are drawn.

[0019] In this specification, "reference data" means, for example, data that serves as the design basis for a workpiece (e.g., upper sash, column sash, door frame, bracket, etc.) (e.g., master work data, design value data, raw data, CAD data).

[0020] In this specification, "shape-matched point cloud data" refers to point cloud data that has been adjusted to match the "reference data" described above.

[0021] In this specification, the "movement matrix" and "inverse matrix" are represented, for example, by 4x4 matrix data (matrix data). For example, as shown in Figure 2, the "movement matrix" and "inverse matrix" are represented as information about the origin or the position from a reference origin to a target origin. The first column from the left shows the unit vector (I, J, K) of the attitude change from the reference X-axis, the second column from the left shows the unit vector (I, J, K) of the attitude change from the reference Y-axis, the third column from the left shows the unit vector (I, J, K) of the attitude change from the reference Z-axis, and the fourth column from the left shows the amount of movement (X, Y, Z) from the reference point.

[0022] Using a matrix, it is possible to calculate the amount of movement for each coordinate using a matrix, as shown in Figure 3. By performing this matrix calculation, the position and angle quantities for each coordinate system can be calculated. The movement when aligning the point cloud data with the reference data, as described above, can also be represented by 4x4 matrix data. Furthermore, it is possible to output 4x4 matrix data representing the movement when aligning the reference data with the point cloud data by performing an inverse matrix transformation on this 4x4 matrix data. In Figure 3, the origin reference, origin A as seen from the origin reference, origin A as seen from origin B, origin B as seen from the origin reference, and the inverse matrix of origin B are each shown as 4x4 matrix data.

[0023] Figures 4 and 5 show first and second examples of the configuration of a welding apparatus 1 for vehicle parts to which the object detection method and detection device having a calibration function of this embodiment are applied.

[0024] The welding apparatus 1 has a fixing jig (for example, a jig for the tact system) 10 located in the center of the apparatus. The fixing jig 10 supports a door frame 11 in a fixed state, for example, by welding the end of the upper sash (front main frame) and the end of the vertical sash (front vertical frame) in an aligned state. The relative positional relationship between the fixing jig 10 and the door frame 11 may change each time the workpiece is attached or detached, so the repeatable positioning function required of a normal jig is unnecessary (because the workpiece position of the door frame 11 is determined by the inverse matrix described later). For this reason, the fixing jig 10 may fix any position on the door frame 11. For the sake of drawing convenience, Figure 4 shows the state in which the fixing jig 10 supports the door frame 11, and Figure 5 shows the state in which the fixing jig 10 does not support the door frame 11 (the door frame 11 is omitted in the drawing).

[0025] The welding apparatus 1 has a robot arm 20 and a robot arm 30 located on the left and right sides of the fixed jig 10 in the figure, respectively. The robot arm 20 is composed of, for example, an arm robot having multiple axes (e.g., 6 axes), and moves (moves) to align a lock bracket (shown as a callout in Figure 4) gripped by a gripping part 21 at the tip of the robot arm 20 to a predetermined part of the door frame 11. A position correction marker 22 is also provided near the gripping part 21 at the tip of the robot arm 20. The position correction marker 22 has, for example, a reflector that can be tracked by a first position information generation device (laser tracker) 70, which will be described later. The robot arm 30 is composed of, for example, an arm robot having multiple axes (e.g., 6 axes), and moves (moves) to align a hinge bracket (shown as a callout in Figure 4) gripped by a gripping part 31 at the tip of the robot arm 30 to a predetermined part of the door frame 11. Furthermore, a position correction marker 32 is provided near the gripping portion 31 at the tip of the robot arm 30. The position correction marker 32 has a reflector that can be tracked by, for example, the first position information generation device (laser tracker) 70, which will be described later.

[0026] The relative positional relationship between the robot arm 20 and the lock bracket (shown by a callout in Figure 4), and the relative positional relationship between the robot arm 30 and the hinge bracket (shown by a callout in Figure 4) may change each time a workpiece is attached or detached. Therefore, the repeatable positioning function required for a typical jig is unnecessary (because the workpiece gripping positions of the lock bracket and hinge bracket are determined by the inverse matrix described later). Thus, the robot arm 20 may grip any position on the lock bracket, and the robot arm 30 may grip any position on the hinge bracket. If the relative positional accuracy of the fixing jig 10 and the door frame 11, the relative positional accuracy of the robot arm 20 and the lock bracket, and the relative positional accuracy of the robot arm 30 and the hinge bracket are guaranteed, and the absolute positional accuracy of the robot spatial coordinates is guaranteed, then high-precision positioning is possible with a device (for example, a machine tool such as a machining center).

[0027] The welding apparatus 1 has a welding robot 40 positioned offset in the depth direction between the fixing jig 10 and the robot arm 20, and a welding robot 50 positioned offset in the depth direction between the fixing jig 10 and the robot arm 30. The welding robot 40 is composed of, for example, an arm robot having multiple axes (e.g., 6 axes), and uses a welding nozzle 41 provided at its tip to weld the door frame 11 supported by the fixing jig 10 and the lock bracket gripped by the gripping part 21 of the robot arm 20 in a predetermined alignment state. The welding robot 50 is composed of, for example, an arm robot having multiple axes (e.g., 6 axes), and uses a welding nozzle 51 provided at its tip to weld the door frame 11 supported by the fixing jig 10 and the hinge bracket gripped by the gripping part 31 of the robot arm 30 in a predetermined alignment state.

[0028] The welding apparatus 1 has a 3D scanner support arm 60 positioned behind the fixing jig 10 and sandwiched between the welding robot 40 and the welding robot 50. The 3D scanner support arm 60 is composed of, for example, an arm robot having multiple axes (e.g., 6 axes), and a second position information generation device (3D scanner) 61 and a position correction marker 62 are provided at the tip of the arm. The position correction marker 62 has, for example, a reflector that can be tracked by a first position information generation device (laser tracker) 70, which will be described later.

[0029] The welding apparatus 1 may be used in combination with a first position information generating device (laser tracker) 70. The first position information generating device (laser tracker) 70 targets each component of the welding apparatus 1 (for example, the door frame 11 supported by the fixing jig 10, the lock bracket gripped by the robot arm 20, the hinge bracket gripped by the robot arm 30, the position correction marker 22 for the robot arm 20, the position correction marker 32 for the robot arm 30, the 3D scanner 61, and the position correction marker 62 for the 3D scanner 61) and calculates its three-dimensional information (position information and angle) based on its own coordinate system.

[0030] As shown in Figures 4 and 5, a position calibration reference base is provided to the side of the welding robot 40 (between it and the 3D scanner support arm 60), and a shape marker (fixed object, metal block) 80 is provided above this position calibration reference base. The specific configuration and function of the shape marker 80 will be described in detail later. The shape marker 80 works in cooperation with the first position information generation device (laser tracker) 70 and the second position information generation device (3D scanner) 61 to perform calibration functions for object detection (detection of position and orientation). Furthermore, a position correction marker (fifth object) 42 is provided on the position calibration reference base, positioned slightly below the shape marker 80.

[0031] In this embodiment, a marker-equipped 3D scanner 61 is used to scan (image) the door frame 11 supported by the fixing jig 10, the lock bracket gripped by the robot arm 20, and the hinge bracket gripped by the robot arm 30, thereby acquiring point cloud data (three-dimensional information) of these objects. In other words, the marker-equipped 3D scanner 61 functions as a "point cloud data acquisition unit". By acquiring the point cloud with the marker-equipped 3D scanner 61 virtually locked on by the first position information generation device (laser tracker) 70, the point cloud data acquired by the marker-equipped 3D scanner 61 is calculated with the first position information generation device (laser tracker) 70 as the origin (calculated as a fictitious origin associated with the virtual lock-on of the first position information generation device (laser tracker) 70). This point cloud data is represented, for example, by the coordinates X, Y, Z of each point and their normal directions I, J, K, and the origin of the point cloud data is represented by 4x4 matrix data (details will be described later).

[0032] Furthermore, when acquiring point cloud data, instead of 3D scanning the entire door frame 11 supported by the fixed jig 10, the lock bracket gripped by the robot arm 20, and the hinge bracket gripped by the robot arm 30, it is also possible to 3D scan only a part of the door frame 11 supported by the fixed jig 10, the lock bracket gripped by the robot arm 20, and the hinge bracket gripped by the robot arm 30, for example, only the part that serves as a reference for product evaluation. This 3D scanning can be done by acquiring point cloud data while moving a 2D scanner, by positioning a 3D scanner and scanning, or by installing fixed 3D scanners in multiple locations and scanning. As will be described later, when aligning point cloud data with reference data, it is possible to set any number of arbitrary positions on the reference data as references and perform alignment. In addition, a priority can be set for each of these references, so that alignment of the more important parts can be prioritized. This makes it possible to control the alignment method between the reference data and the point cloud data. In this embodiment, if the error amount of each reference part is large, there is a possibility that the welded parts of the lock bracket gripped by the robot arm 20 and the hinge bracket gripped by the robot arm 30 will interfere with each other with the door frame 11. Therefore, by setting the priority of the welded parts higher than that of the other reference parts, it is possible to prioritize positioning by matching the shape of the welded parts.

[0033] Furthermore, in this embodiment, reference data (e.g., CAD data) that serves as the design basis for the door frame 11 supported by the fixing jig 10, the lock bracket gripped by the robot arm 20, and the hinge bracket gripped by the robot arm 30 is prepared in advance. This reference data includes the origin (e.g., the vehicle origin, in this case the vehicle origin = the position information generation device origin) and can also be data representing the alignment state of the door frame 11 supported by the fixing jig 10, the lock bracket gripped by the robot arm 20, and the hinge bracket gripped by the robot arm 30. Then, in this embodiment, shape-aligned point cloud data is calculated by aligning the point cloud data with the reference data including the origin (vehicle origin) using a control unit which will be described later. The amount of movement when the point cloud data is moved to the reference data, and the origins of the reference data and shape-aligned point cloud data are represented by 4x4 matrix data (details will be described later).

[0034] The origin coordinates of the shape-aligned point cloud data are equivalent to moving the origin of the point cloud data acquired at the origin of the position information generation device. By performing an inverse matrix transformation on the 4x4 matrix data (matrix) used when aligning the point cloud data to this reference data, it is equivalent to aligning the reference data to the point cloud data, thus assigning an origin to the point cloud data (the state of the shape-aligned point cloud data, which was represented by the origin of the reference data, is now represented based on the point cloud data). This method makes it possible to assign CAD origin information to point cloud data that does not have an origin.

[0035] The result of moving the point cloud data of the door frame 11 supported by the fixing jig 10, the lock bracket gripped by the robot arm 20, and the hinge bracket gripped by the robot arm 30 to match the CAD data (reference data) can be represented (i.e., equivalent) by moving the origin. By converting the matrix to an inverse matrix, the amount of movement required to match the CAD data (reference data) to the point cloud data can be determined. Matching the point cloud data to the CAD data (reference data) corresponds to the positive matrix, and matching the CAD data (reference data) to the point cloud data corresponds to the inverse matrix (which corresponds to returning to the origin from the perspective of the positive matrix). In other words, from the perspective of the origin, the state in which the CAD data (reference data) and the point cloud data were aligned has been moved so that the CAD coordinates match the point cloud data. For example, if the actual workpiece placement of a portion of the door frame 11 supported by the fixing jig 10, the lock bracket gripped by the robot arm 20, and the hinge bracket gripped by the robot arm 30 can be reproduced in the CAD coordinate system, then the positions of that portion become known, and calculations between the origins can be performed by calculating the matrix of the other parts of the door frame 11 supported by the fixing jig 10, the lock bracket gripped by the robot arm 20, and the hinge bracket gripped by the robot arm 30 (in this case, inverse matrix transformation is unnecessary).

[0036] The first position information generator (laser tracker) 70 assists in the alignment (micro-movement) of the lock bracket gripped by the robot arm 20 and the hinge bracket gripped by the robot arm 30 by constantly monitoring (tracking) the position of the position correction markers 22 and 32. The robot arms 20 and 30 are pre-calibrated by the first position information generator (laser tracker) 70 and the position correction markers 22 and 32, improving their absolute accuracy in spatial coordinates.

[0037] When measuring the position correction markers 22 and 32 attached to the robot arms 20 and 30, the vehicle origin of the lock bracket gripped by robot arm 20 and the hinge bracket gripped by robot arm 30 is replaced with the marker, allowing the first position information generation device (laser tracker) 70 to constantly monitor the three-dimensional coordinates. Furthermore, the difference between the marker target position and the current position is used to correct the final position, thereby forcibly improving the spatial coordinate accuracy of the robot.

[0038] Figure 6 shows a third example of the configuration of a welding apparatus 1 for vehicle parts to which the object detection method and detection device having a calibration function of this embodiment are applied. Figure 6 illustrates functional components for performing motion (movement) control of robot arms 20 and 30 (gripping parts 21 and 31) based on the point cloud data, reference data, and shape matching point cloud data described above. Each functional component constitutes a part of the control unit 90, which is composed of a CPU (Central Processing Unit).

[0039] The control unit 90 includes a point cloud data acquisition unit 91, a reference data acquisition unit 92, a movement matrix calculation unit 93, an inverse matrix calculation unit 94, a target movement matrix calculation unit 95, and a robot arm control unit 96.

[0040] The point cloud data acquisition unit 91 acquires (receives input) point cloud data (three-dimensional information) of the door frame 11 supported by the fixing jig 10, the lock bracket gripped by the robot arm 20, and the hinge bracket gripped by the robot arm 30, using the first position information generation device (laser tracker) 70 and the second position information generation device (3D scanner) 61. This point cloud data may be point cloud data with the first position information generation device (laser tracker) 70 as the origin (fictitious origin).

[0041] The reference data acquisition unit 92 acquires (stores) reference data (e.g., CAD data) that serves as the design standard for the door frame 11 supported by the fixing jig 10, the lock bracket gripped by the robot arm 20, and the hinge bracket gripped by the robot arm 30. This reference data includes the origin (e.g., the vehicle origin, in this case the vehicle origin = the position information generation device origin) and can also be data on the alignment state of the door frame 11 supported by the fixing jig 10, the lock bracket gripped by the robot arm 20, and the hinge bracket gripped by the robot arm 30.

[0042] The movement matrix calculation unit 93 calculates a movement matrix for aligning point cloud data to reference data representing the alignment state of the door frame 11 supported by the fixed jig 10, the lock bracket gripped by the robot arm 20, and the hinge bracket gripped by the robot arm 30. This movement matrix calculation yields shape-aligned point cloud data obtained by aligning the point cloud data to the reference data based on the movement matrix.

[0043] The inverse matrix calculation unit 94 performs an inverse matrix transformation on the 4x4 matrix data (matrix) used to align the point cloud data with the reference data obtained by the movement matrix calculation unit 93. This transformation adds an origin to the aforementioned point cloud data (the state of the shape-aligned point cloud data, which was represented by the origin of the reference data, is now represented by the point cloud data as the reference). This method enables the addition of CAD origin information to point cloud data that does not have an origin.

[0044] The target movement matrix calculation unit 95 can use an inverse matrix to calculate the amount of movement from the origin of the lock bracket and the hinge bracket to the origin of the other, starting from a state in which an origin (vehicle origin) has been assigned to each of the door frame 11 supported by the fixed jig 10, the lock bracket gripped by the robot arm 20, and the hinge bracket gripped by the robot arm 30.

[0045] The robot arm control unit 96 positions the lock bracket and hinge bracket on the door frame 11 by moving the lock bracket with the robot arm 20 and the hinge bracket with the robot arm 30, based on at least one of the movement matrix and the inverse matrix. More specifically, the robot arm control unit 96 makes small movements of each axis (6 axes) of the robot arms 20 and 30, for example, until the lock bracket and hinge bracket, which are to be positioned, reach the target position from their initial or current position. The command value for the amount of this small movement is the difference between the current position and the target position of the position correction markers 22 and 32 as seen from the first position information generation device (laser tracker) 70. By setting the numerical value of this small movement amount by coefficient adjustment and the number of movements, positioning that does not depend on the accuracy of the robot is made possible. In other words, in this embodiment, the positioning of the workpieces (lock bracket and hinge bracket) is performed in two stages: (1) movement of the workpiece (gripping part) based on at least one of the movement matrix and the inverse matrix (for example, movement from one origin to the other origin based on the origins added to the lock bracket and hinge bracket by the inverse matrix), and (2) movement of the lock bracket and hinge bracket based on the position correction markers 22 and 32 (corrected positioning by the position correction markers 22 and 32, which corrects the operational error in (1)). With the lock bracket and hinge bracket positioned by the robot arm control unit 96, the welding parts of both are welded by the welding robots 40 and 50.

[0046] The functions of the control unit 90 can also be described as follows: The control unit 90 acquires three-dimensional data (e.g., point cloud data) and relative position data (e.g., reference data) of the hand and the workpiece. The control unit 90 performs spatial positioning processing to best fit the acquired three-dimensional data (e.g., point cloud data) and relative position data (e.g., reference data). The control unit 90 outputs coordinate values ​​for the movement of the workpiece by the hand. The control unit 90 performs motion (movement) control of the hand and the workpiece by 6-axis calculation based on robot inverse kinematics.

[0047] The positioning process in this embodiment is performed, for example, by the following processing steps. First, the point cloud data of the lock bracket and hinge bracket are aligned with CAD data (reference data), and the amount of movement is output as a 4x4 matrix data (matrix data), which is a movement matrix. This provides shape-matched point cloud data for the lock bracket and hinge bracket, with the point cloud data aligned with the reference data based on the movement matrix. Next, the point cloud data of the door frame 11 is aligned with CAD data (reference data), and the amount of movement is output as a 4x4 matrix data (matrix data), which is a movement matrix. This provides shape-matched point cloud data for the door frame 11, with the point cloud data aligned with the reference data based on the movement matrix. Finally, an origin is assigned to the point cloud data by performing an inverse matrix operation (inverse matrix operation) from the calculated movement matrix. In other words, the origin of one is aligned with the origin of the other. Alternatively, the origin of the reference data (vehicle origin) may be moved along with the shape-matching point cloud data, and the moved origins of the lock bracket, hinge bracket, and door frame 11 may be aligned. Furthermore, coordinate values ​​(e.g., CAD coordinates) are assigned to the point cloud data of the lock bracket, hinge bracket, and door frame 11 using the calculated inverse matrix. Then, the amount of movement of the point cloud data is calculated in a coordinate system based on the origin (vehicle origin), and the lock bracket and hinge bracket are positioned on the door frame 11 by moving the robot arms 20 and 30 that grip the lock bracket and hinge bracket based on the calculated amount of movement. In this embodiment, since the door frame 11 is fixed and the lock bracket and hinge bracket are movable, the amount of movement corresponds to the amount of movement of the lock bracket and hinge bracket relative to the door frame 11 (the amount of movement from the CAD coordinates of the lock bracket and hinge bracket to the CAD coordinates of the door frame 11). Finally, based on the calculated amount of movement of the lock bracket and hinge bracket relative to the door frame 11, the lock bracket and hinge bracket are moved relative to the door frame 11, thereby positioning them relative to the door frame 11.

[0048] The process for aligning point cloud data to reference data in this embodiment includes, for example, a forced numerical movement step that moves the point cloud information by a forced numerical value (quantitative value), a best-fit step for overall shape alignment that aligns the entire point cloud with the overall CAD shape, and a reference alignment step that performs detailed alignment according to a reference part, for example, a part that serves as the basis for product evaluation.

[0049] Incidentally, for example, during the initial setup (calibration) of the welding machine 1, the position and orientation of the position correction marker 62 of the second position information generation device (3D scanner) 61 are measured each time by the first position information generation device (laser tracker) 70. Also, as mentioned above, the 3D scanner 61 is supported at the tip of the 3D scanner support arm 60, and the point cloud data obtained by the second position information generation device (3D scanner) 61 is used to replace the origin data of the first position information generation device (laser tracker) 70.

[0050] However, during operation, the second position information generation device (3D scanner) 61 and the surrounding metal heat up (for example, to 20°C) and undergo thermal expansion, causing variations in the positional accuracy of the 3D scanner 61. As a result, the initial origin of the 3D scanner 61 and the origin of the 3D scanner 61 after heating up become misaligned, which may worsen the accuracy of the alignment of the object (for example, the lock bracket, hinge bracket, and door frame 11). Consequently, if a workpiece that has been continuously 3D scanned in the same location is converted to a laser tracker origin, a problem arises where the position and orientation of the acquired point cloud data vary over time (due to heating).

[0051] Therefore, in this embodiment, focusing on the fact that if the distance from the first position information generation device (laser tracker) 70 to the shape marker 80 can be calculated, then the dimension from the second position information generation device (3D scanner) 61 to the position correction marker 62 can be calculated, a function to calibrate the positions of both is added by remeasuring the dimension between the 3D scanner 61 and the position correction marker 62 before actually measuring the workpiece. By periodically calibrating the origin position of the 3D scanner 61, which changes depending on environmental conditions such as temperature, for example, once per cycle, it becomes possible to maintain high-precision detection of position and orientation in response to environmental conditions such as temperature.

[0052] Figures 7A to 7D show a first example of a shape marker (fixed object, metal block) 80. Figures 7A to 7D show that the shape marker 80 includes the shape of a triangular pyramid. The shape marker 80 has a base portion 82 that forms a right triangle shape when viewed from the plane of reference numeral 83a (for example, when viewed from a direction perpendicular to the plane of reference numeral 83a), three side portions 83 extending from the center of the right triangle shape in plan view, and a top portion 84 that is a right triangle shape when viewed from the plane of reference numeral 83a (for example, when viewed from a direction perpendicular to the plane of reference numeral 83a), obtained by cutting out the aggregate portion of the three side portions 83. The base portion 82 and the top portion 84 have similar right triangle shapes when viewed from a plan view. A part of the plane 83a may be provided to identify which coordinate axis is pointing where during coordinate extraction, which will be described later. Alternatively, any means of identifying the orientation may be used, such as color-coding some of the faces to identify the coordinate axes. In other words, including a triangular pyramidal shape also includes shapes having a vertex portion 84, as shown in Figures 7A to 7D.

[0053] Figure 8 shows a second example of a shape marker (fixed object, metal block) 80. In Figure 8, the shape marker 80 includes the shape of three spheres 85. The position of an object can be determined by one of the three spheres 85, and the orientation of the object can be determined by all three. By connecting the points defined by the three spheres 85, the position and orientation of the object in spatial coordinates (3D coordinates) based on mutually orthogonal X, Y, and Z axes can be determined. The arrangement of the three spheres may be such that each has different dimensions in order to identify which coordinate axis is pointing where during coordinate extraction, which will be described later. Alternatively, any means of identifying the orientation may be used, such as color-coding some of the faces to identify the coordinate axes.

[0054] The shapes of the shape markers 80 shown in Figures 7A to 7D and Figure 8 are merely examples, and various design modifications are possible. The shape marker 80 only needs to be able to determine the position and orientation of an object in spatial coordinates (3D coordinates) based on mutually orthogonal X, Y, and Z axes. For example, for the markers including the triangular pyramid shape shown in Figures 7A to 7D, the orientation of the shape marker 80 can be determined by calculating the intersection line of the mean planes of the point clouds 83a and 83b as the X-axis, calculating the intersection line of the mean planes of the point clouds 83b and 83c as the Y-axis, and calculating the remaining Z-axis from the relationship between the X and Y axes. If the shape marker 80 rotates during this process, it may become impossible to determine the orientation of the X, Y, and Z axes from a calculation perspective. Therefore, by defining a shape difference that identifies the orientation of the marker, such as making that part the Z-axis direction, the orientation of the shape marker 80 can be accurately determined even if it rotates. The position can be calculated from the intersection of the mean planes of the point clouds 83a, 83b, and 83c. In this case, the position information can also be used by projecting this intersection onto the top surface 84. A characteristic of 3D scanning is that point cloud information obtained perpendicular to the surface is more accurate than that obtained when scanning an inclined surface. Therefore, by including this process, it is possible to reduce positional errors. In the case of the spheres in Figure 8, instead of evenly arranging the spheres, by creating differences in the dimensions between the three spheres 85a, 85b, and 85c, it is possible to determine the direction of the X, Y, or Z axis and thus determine the orientation of the shape marker 80. Furthermore, the three sphere spacing dimensions of 85a, 85b, and 85c offer flexibility, allowing for various design modifications.

[0055] The shape marker 80 may also be provided on the tip side of the robot arms 20 and 30 that grip the workpiece (e.g., lock bracket, hinge bracket). In this case, the shape marker 80 may be provided in place of / in addition to the position correction markers 22 and 32 of the robot arms 20 and 30.

[0056] Figure 9 shows an example of a position calibration process between the second position information generation device (3D scanner) 61 and the position correction marker 62. The position calibration process in Figure 9 is for calculating and calibrating quantitative values, which are the position and orientation of the origin of the second position information generation device (3D scanner) 61, based on at least one of the second position information generation device (3D scanner) 61 and the position correction marker 62. In this specification, the position and orientation of the second position information generation device (3D scanner) 61 refer to values ​​(parameters) based on the origin of the second position information generation device (3D scanner) 61.Calculate the quantitative values ​​(default values) at the time of initial setup (C).Measure the position and orientation of the position correction marker 62 as the first object using the first position information generation device (laser tracker) 70 (D).Measure the position and orientation of the shape marker 80 provided on the position calibration reference base as the second object using the second position information generation device (3D scanner) 61 (B). Based on the measurement value of the position correction marker 62 as the first object by the first position information generation device (laser tracker) 70 (D), the measurement value of the shape marker 80 provided on the position calibration reference base as the second object by the second position information generation device (3D scanner) 61 (B), and the quantitative value (default value) at the time of initial setup (C), the position of the shape marker 80 provided on the position calibration reference base as the second object is calculated with reference to the first position information generation device (laser tracker) 70 (A). Then, before measuring the third object (for example, the door frame 11 that aligns the lock bracket and hinge bracket that the robot arms 20 and 30 grip) using the second position information generation device (3D scanner) 61, the quantitative values, which are the origin position and orientation of the second position information generation device (3D scanner) 61 with reference to the position correction marker 62, are calibrated.Specifically, the quantitative values ​​(C) which represent the position and orientation of the origin of the second position information generation device (3D scanner) 61 relative to the position correction marker 62 are calibrated by calculating the measured value (D) of the position correction marker 62 as the first object by the first position information generation device (laser tracker) 70 and the measured value (B) of the shape marker 80 as the second object by the second position information generation device (3D scanner) 61 relative to the position correction marker 62, from the position (A) of the shape marker 80 as the second object relative to the first position information generation device (laser tracker) 70, respectively. At this time, the data captured by the second position information generation device (3D scanner) 61 appears as a point cloud at the origin of the second position information generation device (3D scanner) 61, and the position and orientation of the second position information generation device (3D scanner) 61 relative to the origin of the first position information generation device (laser tracker) 70 can be determined by aligning the coordinates extracted from the point cloud.

[0057] The above processing steps for calculating and / or calibrating quantitative values ​​may include the following steps 1 to 4. (1) A first step (D) in which the first position information generating device (laser tracker) 70 measures at least one of the second position information generating device (3D scanner) 61 and the position correction marker 62. (2) The second step (A) involves measuring the shape marker 80 as the fourth object with the first position information generating device (laser tracker) 70. (3) The third step (B) involves measuring the shape marker 80 as the fourth object with the second position information generating device (3D scanner) 61. (4) A fourth step (C) in which quantitative values ​​are the position and orientation of the origin of the second position information generating device (3D scanner) 61, based on the measurement results of the first step (D), the second step (A), and the third step (B), with respect to the position correction marker 62.

[0058] In the above explanation, the case in which the shape marker 80 serves as both the second and fourth object is illustrated. However, a separate shape marker may be provided in addition to the shape marker 80 located on the position calibration reference stand, and these may be used as the second and fourth objects. Furthermore, a fifth object (marker), i.e., a position correction marker 42, may be provided as a shape marker for calibrating the positional deviation of the second position information generation device (3D scanner) 61 to ensure the position and orientation of the fourth object, and a step of measuring the fifth object may be added. Since the position correction marker 62, which serves as the first object, may be subject to positional deviation due to temperature, in order to ensure accuracy, a fifth object (marker) whose position has been measured in advance may be interposed during calibration measurements, and quantitative values ​​which are the position and orientation of the origin of the second position information generation device (3D scanner) 61 based on the position correction marker 62 may be calculated and / or calibrated.

[0059] Alternatively, quantitative values ​​representing the position and orientation of the second position information generating device (3D scanner) 61 relative to the position correction marker 62 may be stored in a table as predetermined values ​​that differ depending on the temperature of the second position information generating device (3D scanner) 61. Then, by referring to this table, quantitative values ​​representing the position and orientation of the second position information generating device (3D scanner) 61 relative to the position correction marker 62 may be calculated and / or calibrated based on the temperature measurement of the second position information generating device (3D scanner) 61. For example, quantitative values ​​(calibration values) corresponding to the measured temperature may be calculated by referring to a table that stores quantitative values ​​(calibration values) for each predetermined temperature range, or quantitative values ​​(calibration values) may be calculated by substituting the measured temperature into a calculation formula that takes temperature as an input parameter.

[0060] The quantitative value calibration process described above is performed at predetermined timings when environmental changes such as temperature are a concern (for example, the calibration process is performed before each 3D scan measurement cycle). This allows for calibration of the position and orientation between the 3D scanner and the marker, thereby absorbing variations caused by environmental influences (temperature).

[0061] As described above, the object detection method of this embodiment includes the steps of: calculating a quantitative value; measuring a first object with a first position information generating device; measuring a second object with a second position information generating device; calculating the position of the second object relative to the first position information generating device based on the measurement value of the first object by the first position information generating device, the measurement value of the second object by the second position information generating device, and the quantitative value; and calibrating the quantitative value before measuring a third object with the second position information generating device. This makes it possible to maintain the accuracy of object detection regardless of changes in the external environment (such as temperature). By calibrating the position and orientation between the 3D scanner and the marker, variations due to environmental influences (temperature) can be absorbed. [Explanation of symbols]

[0062] 1. Welding equipment 10 Fixing fixtures 11. Door frame (third object) 20 Robot Arms 21 Gripping part 22 Position correction marker 30 Robot Arms 31 Gripping part 32 Position correction marker 40 Welding robots 41 Welding Nozzle 42 Position correction marker (fifth object) 50 Welding Robots 51 Welding Nozzle 60 3D scanner support arm 61. Second location information generation device (3D scanner) 62 Position correction marker (first object) 70. First position information generating device (laser tracker) 80 Shape markers (second object, fourth object, fixed object, metal block) 82 Base 83 Side part 84 Top part 85 Three balls 90 Control Unit 91 Point cloud data acquisition unit 92 Reference Data Acquisition Unit 93 Movement Matrix Calculation Unit 94 Inverse Matrix Calculation Unit 95 Target movement matrix calculation unit 96 Robot Arm Control Unit

Claims

1. A step of calculating quantitative values ​​which are the position and orientation of the origin of a second position information generating device based on a position correction marker, A step of measuring the position correction marker as a first object with a first position information generating device, The steps include measuring the shape marker as a second object using the second position information generating device, A step of calculating the position of the shape marker relative to the first position information generating device based on the measurement value of the position correction marker by the first position information generating device, the measurement value of the shape marker by the second position information generating device, and the quantitative value which is the position and orientation of the origin of the second position information generating device relative to the position correction marker, Before measuring the third object with the second position information generating device, the quantitative value, which is the position and orientation of the origin of the second position information generating device based on the position correction marker, is calibrated. It has, In the step of calibrating the quantitative value, the quantitative value, which is the position and orientation of the origin of the second position information generator relative to the position correction marker, is calibrated by calculating the measurement value of the position correction marker by the first position information generator and the measurement value of the shape marker by the second position information generator, based on the position of the shape marker relative to the first position information generator. A method for detecting an object having a calibration function, characterized by the features described above.

2. The processing step for calculating and / or calibrating the quantitative value is: The first step is to measure the position correction marker with the first position information generation device, The second step involves measuring the fourth object with the first position information generating device, The third step is to measure the fourth object with the second position information generating device, A fourth step in which, based on the measurement results of the first, second, and third steps, the quantitative value which is the position and orientation of the origin of the second position information generating device with respect to the position correction marker is calculated and / or calibrated, A method for detecting an object having a calibration function as described in claim 1, characterized by having the above.

3. In addition to the shape marker as the second object, a shape marker as the fourth object is provided, In addition to the position correction marker as the first object, a position correction marker as a fifth object is provided. The method includes a step of measuring the position correction marker, which is the fifth object, in order to ensure the position and orientation of the shape marker, which is the fourth object, A method for detecting an object having a calibration function as described in claim 2.

4. The quantitative value is a predetermined value that varies depending on the temperature of the second location information generating device, and the quantitative value is calculated and / or calibrated based on the temperature measurement of the second location information generating device. A method for detecting an object having a calibration function as described in claim 1.

5. The first position information generating device is a laser tracker, The second location information generating device is a 3D scanner. A method for detecting an object having a calibration function as described in any one of claims 1 to 4.

6. A device for calculating quantitative values ​​which are the position and orientation of the origin of a second position information generation device based on a position correction marker, A first position information generation device that measures the position correction marker as the first object, A second position information generation device for measuring a shape marker as a second object, A device for calculating the position of the shape marker relative to the first position information generation device, based on the measurement value of the position correction marker by the first position information generation device, the measurement value of the shape marker by the second position information generation device, and the quantitative value which is the position and orientation of the origin of the second position information generation device relative to the position correction marker, Before measuring the third object with the second position information generating device, a device is provided to calibrate the quantitative value, which is the position and orientation of the origin of the second position information generating device, based on the position correction marker, It has, The apparatus for calibrating the quantitative value calibrates the quantitative value, which is the position and orientation of the origin of the second position information generating apparatus with respect to the position correction marker, by calculating the measurement value of the position correction marker by the first position information generating apparatus and the measurement value of the shape marker by the second position information generating apparatus from the position of the shape marker with respect to the first position information generating apparatus. A detection device for objects having a calibration function, characterized by the above.

Images