Manufacturing overhead photogrammetric guidance system

Abstract

A manufacturing system includes: a stability target; a set of rails; robots including photogrammetric targets mounted on the heads of the robots, the robots moving along the set of tracks, and each robot including a plasma cutting tool or camera mounted on one of the heads; and a photogrammetric system. The photogrammetric system includes: a support frame; an overhead camera mounted on the support frame above the robot and a workpiece to be processed by one of the robots; and a control module. The control module: determines, via the overhead camera, the position of the stability target and the photogrammetric target; determining the positions of the head of the overhead camera and the robot based on the positions of the stability target and the photogrammetry target; and control the robot to create and inspect features on the workpiece based on the position of the overhead camera and the robot head.

Description

[0001] introduction

[0002] The information provided in this section is for the purpose of generally presenting the background of the present disclosure. To the extent described in this section, the work of the presently named inventors, and aspects that may not qualify as prior art at the time of filing, are neither explicitly nor implicitly admitted to be prior art to the present disclosure. Technical Field

[0003] The present disclosure relates to the manufacturing of products, and more particularly, to robotic-based feature formation and inspection of workpieces of products. Background Art

[0004] Materials such as aluminum, steel, plastic, etc. are used during vehicle manufacturing. Vehicle components (or parts) are manufactured using, for example, stamping, pressing, molding, and cutting machines. Examples of such components include vehicle frames, doors, hoods, roofs, trunk lids, etc.

[0005] Vehicles are often assembled along an assembly line. Components of the vehicle and the vehicle itself may be assembled at stations along the assembly line. This may include forming features and attaching and / or connecting components using various techniques such as cutting, welding, fastening and / or bonding processes. Detail operations may also be performed including cleaning, applying corrosion and / or scratch protection layers, painting, etc. Summary of the invention

[0006] A manufacturing system is provided and includes: a stability target; a set of tracks; a robot including photogrammetric targets respectively mounted on robot heads, the robots being configured to move along the set of tracks and each robot including a cutting tool and a camera mounted on a respective one of the heads; and a photogrammetric system. The photogrammetric system includes: a support frame; an overhead camera mounted on the support frame above the robot and a workpiece to be processed by at least one of the robots; and at least one control module. The control module is configured to: determine, via the overhead camera, a position of the stability target and a position of the photogrammetric target; determine a position of the overhead camera and a position of the robot head based on the position of the stability target and the position of the photogrammetric target; and control the robot to create and inspect at least one feature on the workpiece based on the position of the overhead camera and the position of the robot head.

[0007] In other features, the robots include: at least one working robot configured to form or process the at least one feature; and at least one measuring robot configured to inspect the at least one feature.

[0008] In other features, each stability target is located in a field of view of at least two of the overhead cameras.

[0009] In other features, each photogrammetric target is located in a field of view of at least two of the overhead cameras.

[0010] In other features, at least two transmitters of each stability target are located in the field of view of at least two of the overhead cameras.

[0011] In other features, at least one of the following: an emitter for at least one of the photogrammetric targets is located in the field of view of at least two of the overhead cameras; and at least two emitters for each photogrammetric target are located in the field of view of at least two of the overhead cameras.

[0012] Among other features, each feature of the workpiece to be at least one of machined or inspected is within a predetermined range of two of the overhead cameras.

[0013] In other features, each stability target and each photogrammetry target are at an angle within a predetermined range relative to at least two of the overhead cameras.

[0014] In other features, the at least one control module is configured to determine a position of the stability target and a position of the photogrammetry target based on one or more reference coordinate systems.

[0015] In other features, the at least one control module is configured to determine a position of the overhead camera and a position of the robot head based on one or more reference coordinate systems.

[0016] In other features, a manufacturing system design method is provided, and the method includes: defining a workpiece family, the workpiece family including the dimensions of the workpieces in the workpiece family to be processed; determining the feature location and workpiece orientation of the workpiece feature to be created or processed based on the dimensions of the workpiece; determining the position of a plasma cutting tool of a working robot and a corresponding first photogrammetric target at each pre-position of the plasma cutting tool and at each valid working position of the plasma cutting tool associated with creating or processing the feature based on the feature location; determining the position of a second photogrammetric target on a measurement robot to determine the position of a measurement robot camera at each pre-position and each measurement position of the feature; and determining the number of overhead cameras of an overhead photogrammetric guidance system and the positions of the overhead cameras based on a first parameter. The method also includes: determining the size of an overhead structure holding the overhead cameras based on a second parameter; determining the number of stability targets and the positions of the stability targets based on a third parameter; and creating and inspecting at least one feature on one of the workpieces based on i) the position of the plasma cutting tool, ii) the position of the measurement robot camera, iii) the size of the overhead structure, iv) the position and orientation of the overhead camera, and v) the position of the stability target.

[0017] In other features, determining the position of the measurement robot camera at each pre-position and each measurement position includes determining each pre-position and each measurement position of i) a workpiece reference, ii) machined and measured features, and iii) measured but not machined features.

[0018] Among other features, the first parameters include i) a size of the workpiece, ii) a feature location, iii) a location of a first photogrammetric target, iv) a location of a second photogrammetric target, and v) a location of a measurement robot camera.

[0019] In other features, the second parameters include i) positions of the first and second photogrammetric targets, ii) positions of the surveying robot cameras, iii) a number of overhead cameras, and iv) positions and orientations of the overhead cameras.

[0020] In other features, the third parameter includes i) the number of overhead cameras, and ii) the locations of the overhead cameras.

[0021] Among other features, at least one of machining and inspecting a feature of the one of the workpieces includes: calibrating an overhead photogrammetric guidance system; calibrating a work robot and a measuring robot; moving one of the workpieces into a cell with an overhead photogrammetric guidance system to create or machine the feature of the one of the workpieces; orienting the one of the workpieces and determining a reference position of the one of the workpieces; directing a plasma cutting tool to machine the feature of the one of the workpieces; and directing a measuring robot camera to inspect the feature on the one of the workpieces.

[0022] In other features, calibrating the overhead photogrammetric guidance system includes determining a position of an overhead camera relative to a stability target.

[0023] In other features, calibrating the working robot and the measuring robot includes determining positions of the first photogrammetric target and the second photogrammetric target relative to the overhead camera.

[0024] In other features, the plasma cutting tool is guided based on a known position of the first photogrammetric target relative to the overhead camera.

[0025] In other features, the surveying robot camera is guided based on a known position of a second photogrammetric target relative to the overhead camera.

[0026] The present invention provides the following technical solutions:

[0027] 1. A manufacturing system comprising:

[0028] Multiple stability objectives;

[0029] A set of tracks;

[0030] a plurality of robots including a plurality of photogrammetric targets respectively mounted on heads of the plurality of robots, the plurality of robots being configured to move along the set of tracks, and each of the plurality of robots including a plasma cutting tool and a camera mounted on a corresponding one of the heads; and

[0031] Photogrammetry system, including

[0032] Support frame,

[0033] a plurality of overhead cameras mounted on a support frame above the plurality of robots and a workpiece to be processed by at least one of the plurality of robots, and

[0034] At least one control module configured to

[0035] determining, via the plurality of overhead cameras, positions of the plurality of stability targets and positions of the plurality of photogrammetric targets,

[0036] determining positions of the plurality of overhead cameras and positions of heads of the plurality of robots based on positions of the plurality of stability targets and positions of the plurality of photogrammetric targets, and

[0037] Based on the positions of the plurality of overhead cameras and the positions of the heads of the plurality of robots, the plurality of robots are controlled to create and inspect at least one feature on the workpiece.

[0038] 2. The manufacturing system according to technical solution 1, wherein the plurality of robots include:

[0039] at least one working robot configured to form or process the at least one feature; and

[0040] At least one measurement robot is configured to inspect the at least one feature.

[0041] 3. A manufacturing system according to technical solution 1, wherein each of the multiple stability targets is located in the field of view of at least two of the multiple overhead cameras.

[0042] 4. A manufacturing system according to technical solution 1, wherein each of the multiple photogrammetric targets is located in the field of view of at least two of the multiple overhead cameras.

[0043] 5. A manufacturing system according to technical solution 1, wherein at least two transmitters of each of the multiple stability targets are located in the field of view of at least two of the multiple overhead cameras.

[0044] 6. The manufacturing system according to technical solution 1, wherein at least one of the following is true:

[0045] An emitter of at least one of the plurality of photogrammetric targets is located within a field of view of at least two of the plurality of overhead cameras; and

[0046] At least two transmitters of each of the plurality of photogrammetric targets are located in a field of view of at least two of the plurality of overhead cameras.

[0047] 7. A manufacturing system according to technical solution 1, wherein each feature of a workpiece to be processed or inspected is within a predetermined range of two overhead cameras among the multiple overhead cameras.

[0048] 8. A manufacturing system according to technical solution 1, wherein an angle between each of the multiple stability targets and each of the multiple photogrammetry targets relative to at least two of the multiple overhead cameras is within a predetermined range.

[0049] 9. A manufacturing system according to technical solution 1, wherein the at least one control module is configured to determine the positions of the multiple stability targets and the positions of the multiple photogrammetry targets based on one or more reference coordinate systems.

[0050] 10. A manufacturing system according to technical solution 1, wherein the at least one control module is configured to determine the positions of the multiple overhead cameras and the positions of the heads of the multiple robots based on one or more reference coordinate systems.

[0051] 11. A manufacturing system design method, comprising:

[0052] defining a workpiece family, including the dimensions of the workpieces of the workpiece family to be machined;

[0053] Determining feature locations and workpiece orientations of features of the workpiece to be created or machined based on the dimensions of the workpiece;

[0054] determining, based on the feature location, a position of a plasma cutting tool of a working robot and a corresponding first photogrammetric target at each pre-position of the plasma cutting tool and at each valid working position of the plasma cutting tool associated with creating or machining the feature;

[0055] determining a position of a second photogrammetric target on the surveying robot to determine a position of the surveying robot camera at each pre-position and each survey position of the feature;

[0056] determining a number of overhead cameras of the overhead photogrammetric guidance system and positions of the overhead cameras based on the first plurality of parameters;

[0057] determining a dimension of an elevated structure to hold the elevated camera based on a second plurality of parameters;

[0058] determining a number of the stability targets and locations of the stability targets based on a third plurality of parameters; and

[0059] At least one feature on one of the workpieces is created and inspected based on i) a position of a plasma cutting tool, ii) a position of a measurement robot camera, iii) a size of an overhead structure, iv) a position and orientation of the overhead camera, and v) a position of a stability target.

[0060] 12. A method according to technical solution 11, wherein determining the position of the measuring robot camera at each pre-positioning and each measuring position includes determining i) a reference of the workpiece, ii) processed and measured features among the features, and iii) each pre-positioning and each measuring position of measured but unprocessed features among the features.

[0061] 13. A method according to technical solution 11, wherein the first plurality of parameters include i) the size of the workpiece, ii) the feature position, iii) the position of the first photogrammetric target, iv) the position of the second photogrammetric target, and v) the position of the measuring robot camera.

[0062] 14. A method according to technical solution 11, wherein the second plurality of parameters include i) the positions of the first photogrammetric target and the second photogrammetric target, ii) the position of the measuring robot camera, iii) the number of the overhead cameras, and iv) the position and orientation of the overhead cameras.

[0063] 15. The method according to technical solution 11, wherein the third plurality of parameters include i) the number of the overhead cameras, and ii) the positions of the overhead cameras.

[0064] 16. The method according to technical solution 11, wherein at least one of processing and inspecting the feature of the one of the workpieces comprises:

[0065] Calibrate the overhead photogrammetric guidance system:

[0066] calibrating the working robot and the measuring robot;

[0067] moving one of the workpieces into a cell having an overhead photogrammetric guidance system to create or machine features of the one of the workpieces;

[0068] orienting said one of said workpieces and determining a reference position of said one of said workpieces;

[0069] directing the plasma cutting tool to machine a feature of the one of the workpieces; and

[0070] The measurement robot camera is directed to inspect a feature on the one of the workpieces.

[0071] 17. A method according to technical solution 16, wherein calibrating the overhead photogrammetric guidance system includes determining the position of the overhead camera relative to the stability target.

[0072] 18. A method according to technical solution 11, wherein calibrating the working robot and the measuring robot includes determining the positions of the first photogrammetric target and the second photogrammetric target relative to the overhead camera.

[0073] 19. A method according to technical solution 11, wherein the plasma cutting tool is guided based on the known position of the first photogrammetric target relative to the overhead camera.

[0074] 20. A method according to technical solution 11, wherein the surveying robot camera is guided based on the known position of the second photogrammetric target relative to the overhead camera.

[0075] Further areas of applicability of the present disclosure will become apparent from the detailed description, claims and drawings.The detailed description and specific examples are intended for illustrative purposes only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] The present disclosure will be more fully understood from the detailed description and accompanying drawings, in which:

[0077] Figure 1 is a perspective view of an example unit of a manufacturing system including an overhead photogrammetric guidance system according to the present disclosure;

[0078] Figure 2 yes Figure 1 A side view of a unit;

[0079] Figure 3 yes Figure 1 A top view of a unit;

[0080] Figure 4 is a side view of a support frame for an overhead camera for an overhead photogrammetric guidance system according to the present disclosure, illustrating overlapping fields of view (FOV) of two overhead cameras;

[0081] Figure 5is a perspective view of an example stability target according to the present disclosure;

[0082] Figure 6 is a perspective view of an example photogrammetric target according to the present disclosure;

[0083] Figure 7 yes Figure 1 Functional block diagram of the manufacturing system

[0084] Figure 8 illustrates a manufacturing method according to the present disclosure; and

[0085] Fig. 9 A method of designing a photogrammetric guidance system according to the present disclosure is illustrated.

[0086] In the drawings, reference numerals may be repeated to identify similar and / or identical elements. DETAILED DESCRIPTION

[0087] Some products are produced in small batches. For example, certain large heavy trucks may have low production quantities such that the trucks and / or portions thereof are manufactured in one or more cells. A cell refers to a three-dimensional (3D) space (or room) in which at least a portion of a vehicle is located during manufacturing. The portion of the vehicle does not move while in the cell unless it is redirected. A cell is not typically part of an assembly line. At least a portion of a vehicle is formed and / or assembled in a cell. For example, the structural frame of a heavy truck may be moved into a cell and machined to create features on the structural frame. Features may include holes, slots, grooves, etc.

[0088] The unit may include a working robot and an inspection robot. The working robot may have a tool attached to the robot head for cutting, drilling, coating and / or performing other feature forming and / or modification operations. The inspection robot may include a camera attached to the robot head for inspecting the created features. The accuracy of the working and inspection robots may be limited. For example, the robot may position the tip of the tool or inspection camera within ±2 mm of the target positioning. This accuracy may be negatively affected due to changes in temperature, wear and tear of the robot components over time, etc. In addition, the longer the distance between the base of the robot and the head of the robot and / or the more joints and segments (or links) the robot has, the greater the potential error in positioning the head of the robot.

[0089] The examples disclosed herein provide an overhead photogrammetric guidance system for improving the positioning accuracy of a robot head and the positioning accuracy of items attached to the robot head (such as tools and cameras). The tools may include cutting tools such as plasma torches (or plasma cutting tools), welding tools, fastening tools, etc. A corresponding method for designing an overhead photogrammetric guidance system is also disclosed. The overhead photogrammetric guidance system reduces the robot head positioning error to less than ±1 mm, so that the tip of the tool and / or the inspection camera can be positioned within ±1 mm of the target position. In one embodiment, the accuracy is such that the positioning error is less than ±0.5 mm.

[0090] Figure 1-3 A cell 100 of a manufacturing system 102 is shown that includes an overhead photogrammetric guidance system 104. Cell 100 refers to a three-dimensional (3D) volume in which at least a portion of a product is processed, the product being referred to as a workpiece. In the example shown, workpiece 106 is shown and is a large truck frame. Cell 100 may be used to manufacture a product, such as a vehicle or a portion thereof. Cell 100 may be one station in a series of stations used to manufacture a product.

[0091] The manufacturing system 102 includes a support frame 110, a work robot 112, an inspection (or measurement) robot 114, a track set 116, and a cart 118 on which a workpiece 106 is mounted. The workpiece 106 is moved into the cell 100 via the cart 118 and slides between the track sets 116 to be centered within the cell 100, as shown. The cart 118 can move along a track 300 (labeled 301) in the floor, which is linear and extends longitudinally and parallel to the track set 116. The cart 118 includes a trunnion system 120 that can rotate the workpiece 106 around the longitudinal axis of the workpiece 106. The trunnion system 120 may include a motor for rotating the workpiece 106. The cart 118 may have a motor for moving the cart 118 relative to the track set 116, or the cart 118 may be manually moved relative to the track set 116.

[0092] The support frame 110 includes legs 121 and crossbars 123, 127 and 129. The legs 121 are attached to feet 131, which are fastened to the floor of the unit. The legs 121 and crossbars 123, 127 and 129 can be hollow and filled with damping materials, such as sand, particles and / or other damping materials. In one embodiment, the legs 121 and crossbars 123, 127 and 129 are formed of steel, but can also be formed of other materials. In one embodiment, the support frame 110 is constructed to be heavy and rigidly mounted to the floor. The feet 131 can be fastened and / or adhered to the floor. The support frame 110 is configured to prevent the movement of the overhead camera and minimize and / or prevent the vibration of the overhead camera. In one embodiment, the floor is made into a single concrete slab that spans the entire space below the workpiece 106, the robot 112, 114, the track group 116 and the support frame 110. This minimizes: relative movement between the stability targets 302 mounted on the floor; twisting of the support frame 110; and ensures greater overall accuracy of the overhead photogrammetric guidance system 104. In one embodiment, the concrete slab has a minimum thickness of 8 inches (200 mm) of reinforced concrete with no cutouts or expansion joints.

[0093] The working robot 112 performs processing on the workpiece 106. This may include performing operations such as cutting, drilling, welding, fastening and / or other operations. The working robot 112 has a tool (e.g., any tool mentioned herein) mounted on the head 122 of the robot 112. As an example, the working robot 112 is shown as having a welding torch 124 mounted on the head 122 of the robot 112. The inspection robot 114 has a camera 126 mounted on the head 128 of the robot 114. The camera 126 is used to inspect features created by the working robot 112 on the workpiece 106, such as holes, slots, welds, installed fasteners and / or other objects. In the example shown, a working robot and an inspection robot are mounted on a corresponding pair of tracks. As shown, each track group 116 includes a pair of tracks. The robots 112 and 114 have a base 133 that moves longitudinally along the track group 116. The robots 112 and 114 are configured so that the head 128 moves with 6 degrees of freedom (DOF) around the corresponding axis. Each of the robots 112, 114 may include 6 joints and 6 links, with the head of the robot connected to the 6th link. The track set 116 provides a seventh axis of movement for each head 128.

[0094] The overhead photogrammetric guidance system 104 is used for workpiece alignment and guidance of the robots 112, 114. The overhead photogrammetric guidance system 104 includes an overhead camera, which includes a top mounted camera 136 and an end hanging camera 138. Top mounted means connected to the top crossbar. The overhead camera can be manually mounted in a fixed position, or can be moved via a motor, represented by box 140. The top mounted camera 136 can be mounted to and moved along a track 142 extending longitudinally along the crossbar 129, which extends from the front end of the unit 100 to the rear end of the unit 100. The track 142 extends parallel to the crossbar 129 and can be offset from the crossbar 129. The end hanging camera 138 can be mounted to and moved along a track 146, which is suspended from the crossbar 127 via a vertically extending coupling member 150. The horizontal and vertical position of the overhead camera can be adjusted to provide a line of sight between the overhead camera and features such as holes, body mounts, slots, and other features of various sizes. Crossbar 127 extends laterally across unit 100 perpendicular to crossbar 129. Track 146 extends parallel to crossbar 127.

[0095] The overhead cameras 136, 138 detect the position of a stability (or calibration) target 302 and a photogrammetry target (or navigation target) 160. The stability target 302 can be mounted on the floor of the cell 100 and is located in a fixed position within the cell 100. Although six stability targets are shown, any number of stability targets can be used. The number of stability targets used depends on the number of overhead cameras and the FOV of the overhead cameras. In one embodiment, there is at least one stability target in the FOV of each overhead camera. In another embodiment, there are at least two stability targets in the FOV of each overhead camera. The photogrammetry target 160 is mounted to the head 122, 128 of the robot 112, 114. Examples of targets 302, 160 are shown in FIG. Figure 5-6 1. Overhead cameras 136, 138 monitor the movement of photogrammetric target 160 and are used to track the position of tools (e.g., welding torch 124) and cameras (e.g., camera 126) mounted on robots 112, 114. Camera 126 is used to detect fiducials on workpiece 106 and track and inspect features and surfaces of workpiece 106. Overhead cameras 136, 138 can also be used to determine the position of features and surfaces of workpiece 106.

[0096] Figure 4 A method for using an overhead photogrammetric guidance system such as Figure 1-3 The support frame 400 of the overhead camera 402 of the photogrammetric guidance system 104 is used as an alternative to Figure 1-3Another example support frame of the support frame 110. The overhead camera 402 is similar to Figure 1-3 136. The support frame 400 includes vertical legs 404, crossbars 406, and crossbars 407 (one shown). Angled support legs 408 are connected to the vertical legs 404. Angled support plates 410 are connected to the upper portions of the vertical legs 404 and crossbars 407. Angled support plates may also be included and attached to lateral crossbars that Figure 4 4, but is arranged similarly to the angled support plate 410 and attached to a support plate similar to Figure 1 and 3 The cross bar 127 is a laterally extending cross bar.

[0097] exist Figure 4 , an example overlapping FOV 420 of two overhead cameras 402 is shown. The photogrammetric guidance system is configured so that at any time, at least two of the overhead cameras can see each stability target and each photogrammetric target in the corresponding unit. In other words, each target is seen by at least two of the overhead cameras at any time during the manufacturing process. These targets can be seen by different overhead cameras in the overhead cameras. This allows the position of each stability target and photogrammetric target to be determined at any time. As an example, the photogrammetric system can use triangulation to determine the position of each of the stability target and the photogrammetric target. An example target 422 is shown, and it can represent a photogrammetric target. Each camera has a FOV angle, where for two cameras in the camera 402, two are designated as 430, 432. An example FOV angle 434 for the target 422 is shown, and the FOV angle refers to the angle between two lines 436 extending from the center of the camera and intersecting at a point on the target 422, where the target 422 is in the overlapping FOV 420. In one embodiment, the minimum angle is 35° and the maximum angle is 90°. In one embodiment, each overhead camera 402 and other overhead cameras ( Figure 4 not shown) (such as Figure 1-3 The FOV of the suspended overhead camera 138 is 45-90°. In one embodiment, the photogrammetry system is configured such that at any time, the distance between each overhead camera 402 and at least one of the targets is 1.5-6.0 meters (m).

[0098] Figure 5A stability target 500 is shown having a body 502 having emitters (or artifacts) 504. The body 502 can be of any shape and size and include any number of emitters. In one embodiment, the emitters 504 include light emitting diodes (LEDs). The body 502 includes two or more emitters. In one embodiment, the body 502 has 4 emitters (e.g., 4 LEDs). The emitters 504 can be arranged in a row or linear pattern as shown, or can be arranged in a non-linear pattern. Although shown in the same plane extending parallel to the top surface 506 of the body 502, the emitters 504 can be offset from each other so that the emitters 504 are not in the same plane extending parallel to the top surface 506. The emitters 504 can be arranged in a variety of different patterns. In one embodiment, each emitter 504 has a ±60° conical emission pattern that can be visible by multiple overhead cameras. In one embodiment, the body 502 is formed of carbon fiber and / or other materials with minimal dimensional changes due to temperature changes.

[0099] Figure 6 A photogrammetric target 600 is shown having a body 602 having an emitter (or artifact) 604. The body 602 can be of any shape and size and include any number of emitters. In one embodiment, the emitter 604 includes an LED. The body 602 includes two or more emitters. In one embodiment, the body 602 has 5 emitters (e.g., 5 LEDs). The emitters 604 can be arranged in a row or linear pattern, or can be arranged in a non-linear pattern. Although shown in two planes extending parallel to the top surfaces 606, 608 of the body 602, the emitters 604 can be in the same plane extending parallel to the top surface of the body 602, or can be in more than two planes extending parallel to more than two top surfaces. The emitters 604 can be arranged in a variety of different patterns. In one embodiment, each emitter 604 has a ±60° conical emission pattern that can be visible by multiple overhead cameras. In one embodiment, the body 602 is formed of carbon fiber and / or other materials with minimal dimensional changes due to temperature changes.

[0100] In the example shown, the body 602 includes a base portion 610 and a tapered portion 612 integrally formed with the base portion. Three of the emitters 604 are mounted on the base portion 610 and are located in a first plane extending parallel to a first top surface 614. Two of the emitters 604 are mounted on the tapered portion 612 and are located in a second plane that is offset from the first plane and extends parallel to a second top surface 616. The tapered portion 612 has tapered sides 618 extending from the surface 614 to the surface 616. Although the body 602 is shown as a "top hat" shape, the body 602 can be a shape different from that shown.

[0101] Figure 5-6 The target is 500, 600 and Figure 1-3 Other similar goals can be obtained from the corresponding robots (e.g. Figure 1-3 robots 112, 114) and / or from one or more system control modules (e.g. Figure 7 ) receives power. The LED may be continuously on during the manufacturing process, or may be turned on and off at a predetermined frequency and / or have a corresponding duty cycle. One or more system control modules may select the frequency, duty cycle, intensity, etc. of the target's transmitter. As an example, the transmitters of the stability target and the photogrammetry target may each have a corresponding color, frequency, duty cycle, intensity, and / or other distinguishing parameters compared to other targets in the stability target and the photogrammetry target. In one embodiment, the transmitters of the targets have the same color and are continuously on during the manufacturing process (e.g., a cutting process to form one or more features of a workpiece).

[0102] Figure 7 A manufacturing system 700 is shown, comprising: an overhead photogrammetric guidance system 701 including a support frame 702; a work and inspection robot 704; one or more system control modules 706; a cart 708 on which a workpiece 710 is mounted; a stability target 712; and a photogrammetric target 714 mounted on the robot 704. The manufacturing system 700 may be similar to Figure 1-3 The support frame 702 can be configured similar to the manufacturing system 102. Figure 1-4 The support frame 702 may be configured to be one of the support frames 110 and 400. The support frame 702 may include an overhead camera 720, a camera actuator 722 including a motor 724, and one or more reference coordinate systems 726 associated with a first coordinate system of a corresponding overhead photogrammetric guidance system.

[0103] The robots 704 may each include a joint motor 730, a link 732, a joint 734, a head 735, one or more reference coordinate systems 736 associated with a second coordinate system, one or more tools and / or cameras 738, a base 740, a track motor 742, and a robot control module 744. The joint motor 730 is located at the joint 734 and moves the link 732 to move the head 735. The joint motor 730 is controlled by the robot control module 744. The one or more tools and / or cameras 738 are mounted on the head 735. The first of the links 732 is connected to the base 740, and the base 740 moves along a track (e.g., Figure 1 Track motor 742 can move base 740 relative to the track. Alternatively, a motor separate from one of the robots 704 can be used, for example, to rotate a ball screw in a track assembly that includes the track to move the robot relative to the track.

[0104] Each robot 704 includes one or more photogrammetric targets 714, which may include a transmitter 746. In one embodiment, each photogrammetric target 714 is associated with Figure 6 The photogrammetric target 600 is similarly configured. Each robot 704 can include any number of photogrammetric targets 714.

[0105] The cart 708 may include a trunnion system 750, a workpiece table 752, and one or more reference coordinate systems associated with the third coordinate system. The workpiece table 752 holds the workpiece 710. The trunnion system 750 may include a motor 754 for rotating the workpiece 710 on the workpiece table 752. The cart 708 may also include one or more translation motors 756 to move the cart 708 relative to the support frame 702.

[0106] Workpiece 710 may include datum 760, feature 762, and one or more reference coordinate systems 764 associated with a third coordinate system. Datum 760 may include holes, slots, and / or other reference points and / or items of workpiece 710. Feature 762 refers to an item machined on workpiece 710, such as a hole, slot, fastener, weld joint, groove, channel, etc.

[0107] Each stability target 712 may include a transmitter, such as transmitter 770. In one embodiment, each stability target 712 is associated with Figure 5 The stability target 500 is configured similarly.

[0108] One or more system control modules 706 can control the overhead camera 720, motor 724, joint motor, tool and / or camera 738, track motor 742, robot control module 744, transmitter 746, motor 754 and transmitter 770. One or more system control modules 706 coordinate the timing, positioning, location, opening and closing time, frequency, duty cycle, intensity, speed, etc. of these items. Positioning and / or position can be determined relative to a coordinate system. Positions determined relative to second and third coordinate systems can be converted to or related to a first coordinate system, which can be referred to as an integral unit (or master) coordinate system of the corresponding unit. Reference coordinate systems 726, 736, 755, 764 can refer to reference points on the support frame 702, robot 704, cart 708 and workpiece 710. Reference coordinate systems 726, 736, 755, 764 may be used to determine the position of the head 735, the tool and / or camera 738, and the corresponding surfaces of the feature 762 and / or workpiece 710 relative to the support frame 702, the cart 708, the overhead camera 720, the stability target 712, the photogrammetry target 714, and / or other items in the unit. One or more of the reference coordinate systems 726, 736, 755, 764 may be virtual reference coordinate systems. For example, one of the reference coordinate systems 726 may be a virtual reference coordinate system, and the position of the workpiece 710 and portions thereof may be determined relative to the virtual reference coordinate system. This may include X, Y, Z, roll, pitch, and yaw values ​​for each datum of the workpiece. A best fit method may be used to align the workpiece to the first (or overall unit) coordinate system.

[0109] The reference coordinate systems 726, 736, 755, 764 can be placed anywhere in the cell and on the items described above. One of the reference coordinate systems 764 can be located in front of the workpiece 710. Each of the reference coordinate systems 726, 736, 755, 764 can have an x-axis pointing horizontally from the front to the back of the cell, a y-axis pointing laterally across the cell perpendicular to the x-axis, and a z-axis pointing vertically and perpendicular to the x and y axes.

[0110] The control module is configured to determine a 6-degree-of-freedom pose (DOF) of the workpiece 710 and a target position of a hole to be cut in the workpiece 710 based on the data from the inspection robot and the position of the identifier.

[0111] In one embodiment, one or more cutting (or working) robots have an identifier on or near the end effector of the cutting robot. The control module determines the positioning of the end effector based on the identifier. This allows the control module to accurately determine and control the positioning of the end effector to cut a hole in a corresponding workpiece (e.g., a vehicle frame) at a target location. This provides manufacturing flexibility and quality for producing medium to small volume workpieces to be processed using an inspection and cutting station.

[0112] The location of the overhead camera 720, stability target 712, and photogrammetry target 714 is flexible. The positioning of the photogrammetry target 714 and stability target 712 can be changed to provide a line of sight between the benchmark 760 and the measurement robot camera (or inspection camera) 738, and to provide a line of sight between i) the transmitters of the photogrammetry target 714 and stability target 712 and ii) the overhead camera 720. In one embodiment, the overhead camera 720 includes 6-8 cameras. The cameras include top-mounted cameras and hanging (or end) cameras. Top mounting means connecting to the top crossbar. The top-mounted camera can be installed above the crossbar, vertically flush with the crossbar, or below it.

[0113] Figure 8 A manufacturing method is shown. Although mainly for Figure 7 The method is described for a manufacturing system, but it can also be applied to Figure 1-3 manufacturing system.

[0114] At 800, one or more control modules 706 or other control modules design an overhead photogrammetry guidance system (e.g., Figure 1-3 Overhead photogrammetry guidance system 104 or Figure 7 Overhead photogrammetry guidance system 701). Fig. 9 8 shows a design method that can be implemented as part of operation 800. The design method includes determining i) the size and / or support frame of the overhead structure for the overhead camera (e.g., Figure 1 and 7 The dimensions of the overhead structure and the number and location of stability targets are used to construct an overhead photogrammetric guidance system. The overhead photogrammetric guidance system includes a support frame, an overhead camera, and one or more control modules 706.

[0115] At 802 , the one or more control modules 706 calibrate the one or more control modules 706 based on at least one reference coordinate system and the detected position of the stability target to determine the position of the overhead camera in the unit. The at least one reference coordinate system may include the reference coordinate system 736 .

[0116] At 804, one or more control modules 706 calibrate the work and inspection robot using the overhead camera based on at least one reference coordinate system to determine the position of the robot's photogrammetric target relative to the overhead camera. The emission pattern of the target's emitter is detected and can be compared to a known pattern of the target to determine the position and / or orientation of the target. The at least one reference coordinate system can include reference coordinate system 736.

[0117] At 806 , the workpiece is moved into the cell. This can be done manually or by one or more control modules 706 .

[0118] At 808, if the workpiece is not in the correct orientation for the first (or next) feature, one or more control modules 706 orient the workpiece and determine the position of the reference on the workpiece to determine the position of the workpiece relative to one or more coordinate systems, robots, and overhead cameras. The coordinate system may include any of the first, second, and third coordinate systems of the support frame, robot, and cart. At the beginning of processing and after each reorientation of the workpiece, the position of the reference is measured. The reference is captured using an overhead photogrammetry guidance system to determine the accurate (six DOF pose) of the workpiece relative to the reference coordinate system. This is called reference alignment, where the workpiece position is mapped into the overhead photogrammetry guidance system. The overhead photogrammetry guidance system determines the offset of the robot to accurately determine the position of the feature.

[0119] At 810, one or more control modules 706 guide tools on one or more work robots to one or more surfaces of a workpiece to be processed. Before forming and / or processing features, the position of the surface is measured. The head of the work robot can be roughly moved to a position close to the feature to be processed (pre-positioning), and then the overhead photogrammetric guidance system can be used to accurately determine the position of the working end or tip of the tool on the workpiece. The precise position determination of the end or tip can be within ±1 mm or less of the target position.

[0120] At 812 , the one or more control modules 706 process on the workpiece, including optionally directing tools on one or more working robots to create one or more features on the workpiece.

[0121] Each pre-position is a position where the robot stops just before processing (e.g., cutting with a torch or measuring with a camera) on a workpiece. The pre-position is close to the actual starting position of the work (e.g., cutting or measuring) operation. The pre-position is far enough from the workpiece so that there is no collision with the workpiece. Once in the pre-position, one or more control modules 706 determine and / or calculate the position of the tool or camera on the robot head. Based on this information, one or more control modules 706 determine the X, Y, Z, pitch, roll, and roll offsets to correct the robot positioning and improve accuracy. Depending on whether the robot moves the tool or camera, the work position or measurement position is the correct position where the work (e.g., cutting) or measurement should start or end. The work and measurement positions are specific for each feature and each coordinate system. The work position and measurement position are determined based on i) the actual measurement value of the workpiece reference, and ii) the measurement value of the feature surface (e.g., hole surface) to be made (e.g., cut) with the tool.

[0122] At 814, the one or more control modules 706 determine whether there is another feature to create and / or surface to machine. If so, operation 808 is performed, otherwise operation 816 is performed.

[0123] At 816, the one or more control modules 706 direct the camera of the one or more inspection robots to the location of the feature and inspect the feature via the inspection robot camera. Measure the feature. Measure the size and location of the feature.

[0124] At 818, the one or more control modules 706 determine whether there is another feature to check. If so, operation 816 is performed, otherwise the method can end.

[0125] Fig. 9 A method for designing a photogrammetric guidance system is shown. At 900, one or more control modules 706 define a family of workpieces, including the sizes of the workpieces of the family to be processed. A family of workpieces may refer to a plurality of different workpieces having different sizes, shapes, characteristics, etc. Workpieces of a particular family may be of the same type and / or model, but with different sizes and / or number of features. In one embodiment and when the family of parts is, for example, a family of truck frames, the maximum length, width, and height dimensions of the workpiece are defined.

[0126] At 902, one or more control modules 706 determine feature locations and workpiece orientations for the feature locations for variants in a family of workpieces. This can be based on the size of the workpiece. As an example, the feature location can refer to the location of a feature on the workpiece to be located. In one embodiment and for a family of workpieces including truck frames, a cutting location and an orientation tool for the cutting location are determined for a variant truck frame in the family.

[0127] At 904, one or more control modules 706 determine the position of the tool of the work robot and the corresponding photogrammetric target at each pre-position and at each valid work tool position. In one embodiment and for a family of workpieces including truck frames, each pre-position and each cutting position of each tool for each workpiece of the family of workpieces is determined. This is done for each feature of each workpiece to be machined and is based on the location of the feature. The work position is determined based on i) actual measurements of the workpiece datum, and ii) measurements of the feature surface (e.g., hole surface) to be made (e.g., cut) with the tool.

[0128] At 906, one or more control modules 706 determine the position of the photogrammetric target on the measurement robot to determine the position of the measurement robot camera at each pre-position and each measurement position for fiducials, machined and measured features, and measured but unmachined features. The pre-position and measurement positions may be fiducials, machined features, and unmachined features of a workpiece of a family of workpieces. In one embodiment and for a family of workpieces including truck frames, each position of the robot camera (for inspection) and the associated photogrammetric target is determined at each pre-position and at each measurement position, including the position of fiducials, cut features, and only measured features.

[0129] At 908, one or more control modules 706 determine i) the number of photogrammetric cameras, and ii) the position and orientation of the photogrammetric cameras. This is done so that each photogrammetric target position and orientation is visible and within an acceptable FOV of at least two overhead cameras during the entire manufacturing process of forming and / or machining features of a particular series of workpieces of interest. At any time during the manufacturing process, at least a portion of a photogrammetric target having a predetermined pattern of emitters (e.g., two or more emitters) can be seen by at least two of the overhead cameras. In one embodiment, 2-5 emitters can be seen by at least two overhead cameras during the manufacturing process. The number of overhead cameras and the corresponding positions can be based on a first parameter, such as the size of the workpiece, the location of the feature, the location of the photogrammetric target, and the location of the measurement robot camera.

[0130] At 910, the one or more control modules 706 determine the dimensions of the overhead structure and / or support frame to maintain the overhead camera in a position defined based on the information determined in the above operations. The dimensions of the overhead structure and / or support frame can be determined based on second parameters, such as the location of the photogrammetric target, the location of the surveying robot camera, the number of overhead cameras, the location and orientation of the overhead cameras, the number of photogrammetric cameras, and the location of the photogrammetric cameras.

[0131] At 912, the one or more control modules 706 determine the number and location of the stability targets. This may be determined so that each photogrammetric camera can see at least two stability targets. The number and location of the stability targets are determined so that at any time during the manufacturing process, at least a portion of the stability targets having a predetermined pattern of emitters are visible to at least two of the overhead cameras. The number of stability targets and the location of the stability targets may be determined based on a third parameter, such as the number of overhead cameras and the location of the overhead cameras.

[0132] In execution Fig. 9 After the method, Fig. 9 Several parameters determined during the method can be found in Figure 8 These parameters include: the working position of the tool during use, which can remain unchanged as long as the type and model of the workpiece remains unchanged; the position of the robot camera, which remains unchanged as long as the type and model of the workpiece remains unchanged; the geometry of the overhead structure supporting the overhead camera, which remains unchanged; the position and orientation of the photogrammetric camera, which remains fixed during the manufacture of the workpiece until the workpiece series is changed; and the position of the stability target (or artifact), which remains unchanged as long as the position of the overhead photogrammetric camera does not change.

[0133] The above examples include the design and use of an overhead photogrammetric guidance system that provides accurate positioning of the robot head for the corresponding work tools and inspection cameras mounted on the robot head. The head or the last link of the robot has tools and inspection cameras to provide accurate guidance and positioning. The inspection cameras are used for quality inspection. Each unit includes an overhead photogrammetric structure that supports multiple cameras that provide a vision system that provides feedback based on which the tools and inspection cameras can be accurately placed and accurately moved. The examples disclosed herein can be applied to frame cutting applications and other applications.

[0134] The above examples provide an overhead photogrammetric guidance system that includes a minimum number of overhead cameras and targets to accurately determine the position of workpieces in a cell and the position of references and features of workpieces in the cell. A minimum number of targets are included for real-time accurate position determination of tools and cameras on the robot head, and for guiding the movement of tools and cameras. This is based on accurate detection of reference positions before guided movement. The inclusion of a minimum number of overhead cameras, targets, and inspection cameras allows high-speed detection and mapping of the positions of workpieces and parts thereof in a cell, as well as inspection of features of the workpieces. These positions are 6 DOF positions within the cell. Based on the size and position of the workpiece surface relative to the cell's global coordinate system, the tools and cameras mounted on the head can be moved to nominal positions. These examples are flexible for different products (or workpieces) by adjusting the position, orientation, and / or number of overhead cameras. These examples are fixtureless, so that the workpiece is not held by a workpiece-specific fixture, but by a cart capable of holding different types and models of workpieces.

[0135] The foregoing description is merely illustrative in nature and is in no way intended to limit the present disclosure, its application or use. The broad teachings of the present disclosure can be implemented in a variety of forms. Therefore, although the present disclosure includes specific examples, the true scope of the present disclosure should not be so limited, because after studying the drawings, the specification and the appended claims, other modifications will become apparent. It should be understood that one or more steps in the method can be performed in different orders (or simultaneously) without changing the principles of the present disclosure. In addition, although each embodiment is described above as having certain features, any one or more of those features described with respect to any embodiment of the present disclosure can be implemented in any other embodiment and / or combined with the features of any other embodiment, even if the combination is not explicitly described. In other words, the described embodiments are not mutually exclusive, and the replacement of one or more embodiments with each other is still within the scope of the present disclosure.

[0136] Various terms are used to describe the spatial and functional relationships between elements (e.g., between modules, circuit elements, semiconductor layers, etc.), including "connected," "engaged," "coupled," "adjacent," "next to," "on top of," "above," "below," and "disposed." Unless explicitly described as "direct," when describing the relationship between a first and a second element in the above disclosure, the relationship may be a direct relationship in which there are no other intervening elements between the first and second elements, but may also be an indirect relationship in which there are one or more intervening elements (either spatially or functionally) between the first and second elements. As used herein, the phrase "at least one of A, B, and C" should be interpreted to mean a logical "A or B or C" using a non-exclusive logical "or," and should not be interpreted to mean "at least one of A, at least one of B, and at least one of C."

[0137] In a diagram, the direction of the arrow, as indicated by the arrow, generally represents the flow of information (such as data or instructions) of interest to the diagram. For example, when element A and element B exchange various information, but the information transmitted from element A to element B is relevant to the diagram, the arrow may point from element A to element B. This unidirectional arrow does not imply that no other information is transmitted from element B to element A. In addition, for information sent from element A to element B, element B may send a request for the information or a receipt confirmation to element A.

[0138] In this application, including the following definitions, the term "module" or the term "controller" may be replaced by the term "circuit". The term "module" may refer to, be a part of, or include: an application-specific integrated circuit (ASIC); a digital, analog, or mixed analog / digital discrete circuit; a digital, analog, or mixed analog / digital integrated circuit; a combinational logic circuit; a field programmable gate array (FPGA); a processor circuit (shared, dedicated, or group) that executes code; a memory circuit (shared, dedicated, or group) that stores code executed by the processor circuit; other suitable hardware components that provide the described functionality; or a combination of some or all of the above, such as in a system on a chip.

[0139] The module may include one or more interface circuits. In some examples, the interface circuit may include a wired or wireless interface connected to a local area network (LAN), the Internet, a wide area network (WAN), or a combination thereof. The functions of any given module of the present disclosure may be distributed in multiple modules connected via the interface circuit. For example, multiple modules may allow load balancing. In another example, a server (also referred to as a remote or cloud) module may perform some functions on behalf of a client module.

[0140] As described above, the term "code" may include software, firmware, and / or microcode, and may refer to a program, a routine, a function, a class, a data structure, and / or an object. The term "shared processor circuit" encompasses a single processor circuit that executes some or all of the code from multiple modules. The term "group processor circuit" encompasses a processor circuit that executes some or all of the code from one or more modules in combination with an additional processor circuit. References to "multiple processor circuits" encompass multiple processor circuits on discrete dies, multiple processor circuits on a single die, multiple cores of a single processor circuit, multiple threads of a single processor circuit, or a combination of the above. The term "shared memory circuit" encompasses a single memory circuit that stores some or all of the code from multiple modules. The term "group memory circuit" encompasses a memory circuit that stores some or all of the code from one or more modules in combination with additional memory.

[0141] The term "memory circuit" is a subset of the term computer-readable medium. The term "computer-readable medium" as used herein does not cover transient electrical or electromagnetic signals propagated through a medium (such as on a carrier wave); therefore, the term "computer-readable medium" may be considered to be tangible and non-transitory. Non-limiting examples of non-transitory tangible computer-readable media are non-volatile memory circuits (such as flash memory circuits, erasable programmable read-only memory circuits, or shielded read-only memory circuits), volatile memory circuits (such as static random access memory circuits or dynamic random access memory circuits), magnetic storage media (such as analog or digital magnetic tape or hard disk drives), and optical storage media (such as CDs, DVDs, or Blu-ray discs).

[0142] The apparatus and methods described in this application may be implemented in part or in whole by a special purpose computer created by configuring a general purpose computer to perform one or more specific functions contained in a computer program. The functional blocks, flow chart components and other elements described above serve as software specifications, which can be translated into a computer program by the routine work of a skilled technician or programmer.

[0143] The computer program includes processor executable instructions stored on at least one non-transitory tangible computer readable medium. The computer program may also include or rely on stored data. The computer program may include a basic input / output system (BIOS) that interacts with the hardware of the special-purpose computer, device drivers that interact with specific devices of the special-purpose computer, one or more operating systems, user applications, background services, background applications, etc.

[0144] A computer program may include: (i) descriptive text to be parsed, such as HTML (Hypertext Markup Language), XML (Extensible Markup Language), or JSON (JavaScript Object Notation) (ii) assembly code, (iii) object code generated by a compiler from source code, (iv) source code executed by an interpreter, (v) source code compiled and executed by a just-in-time compiler, etc. By way of example only, source code may be written using syntax from the following languages: C, C++, C#, Objective-C, Swift, Haskell, Go, SQL, R, Lisp, Fortran, Perl, Pascal, Curl, OCaml, HTML5 (Hypertext Markup Language Fifth Revision), Ada, ASP (Active Server Pages), PHP (PHP: Hypertext Preprocessor), Scala, Eiffel, Smalltalk, Erlang, Ruby, Visual Lua, MATLAB, SIMULINK, and

Claims

1. A manufacturing system comprising: Multiple stability objectives; A set of tracks; a plurality of robots including a plurality of photogrammetric targets respectively mounted on heads of the plurality of robots, the plurality of robots being configured to move along the set of tracks, and each of the plurality of robots including a plasma cutting tool and a camera mounted on a corresponding one of the heads; and Photogrammetry system, including Support frame, a plurality of overhead cameras mounted on a support frame above the plurality of robots and a workpiece to be processed by at least one of the plurality of robots, and At least one control module configured to determining, via the plurality of overhead cameras, positions of the plurality of stability targets and positions of the plurality of photogrammetric targets, determining positions of the plurality of overhead cameras and positions of heads of the plurality of robots based on positions of the plurality of stability targets and positions of the plurality of photogrammetric targets, and Based on the positions of the plurality of overhead cameras and the positions of the heads of the plurality of robots, the plurality of robots are controlled to create and inspect at least one feature on the workpiece.

2. The manufacturing system according to claim 1, wherein: The plurality of robots include: at least one working robot configured to form or process the at least one feature; and At least one measurement robot is configured to inspect the at least one feature.

3. The manufacturing system according to claim 1, wherein: Each of the plurality of stability targets is located in a field of view of at least two of the plurality of overhead cameras.

4. The manufacturing system according to claim 1, wherein: Each of the plurality of photogrammetric targets is located in a field of view of at least two of the plurality of overhead cameras.

5. The manufacturing system according to claim 1, wherein: At least two transmitters of each of the plurality of stability targets are located in a field of view of at least two of the plurality of overhead cameras.

6. The manufacturing system according to claim 1, wherein: At least one of the following: An emitter of at least one of the plurality of photogrammetric targets is located within a field of view of at least two of the plurality of overhead cameras; and At least two transmitters of each of the plurality of photogrammetric targets are located in a field of view of at least two of the plurality of overhead cameras.

7. The manufacturing system according to claim 1, wherein: Each feature of the workpiece, one of to be processed or to be inspected, is within a predetermined range of two of the plurality of overhead cameras.

8. The manufacturing system according to claim 1, wherein: An angle between each of the plurality of stability targets and each of the plurality of photogrammetry targets relative to at least two of the plurality of overhead cameras is within a predetermined range.

9. The manufacturing system according to claim 1, wherein: The at least one control module is configured to determine positions of the plurality of stability targets and positions of the plurality of photogrammetric targets based on one or more reference coordinate systems.

10. The manufacturing system according to claim 1, wherein: The at least one control module is configured to determine positions of the plurality of overhead cameras and positions of heads of the plurality of robots based on one or more reference coordinate systems.

Images