Robot-assisted manufacturing system

Abstract

A manufacturing system comprises: stability targets; a set of rails; robots with photogrammetry targets mounted on their heads, the robots moving along the set of rails, each robot having a plasma cutting tool or camera mounted on one of its heads; and a photogrammetry system. The photogrammetry system comprises: a stand; overhead cameras mounted on the stand above the robots and a workpiece to be processed by one of the robots; and a control module.The control module uses the overhead cameras to determine the locations of the stability targets and the photogrammetry targets; based on the locations of the stability targets and the photogrammetry targets, it determines the locations of the overhead cameras and the robot heads; and controls the robots to create and inspect a feature on the workpiece based on the locations of the overhead cameras and the robot heads.

Description

[0001] The present disclosure relates to the manufacture of products and, more particularly, to the robotic formation of features and inspection of workpieces of the products.

[0002] Materials such as aluminum, steel, plastic, etc. are used in the manufacture of a vehicle. Vehicle components (or parts) are manufactured using stamping, pressing, forming, and cutting machines. Examples of such parts include vehicle frames, doors, hoods, roofs, trunk lids, etc.

[0003] Vehicles are often assembled on an assembly line. A vehicle's components and the vehicle itself may be assembled at stations along the line. This may include molding features and attaching and / or joining components using various techniques such as cutting, welding, fastening, and / or bonding. Detail work such as cleaning, applying corrosion and / or scratch protection coatings, painting, etc., may also be performed.

[0004] DE 10 2021 114 598 B4 discloses a manufacturing system comprising a robot, a remotely arranged vision system for detecting actual positions of the robot by means of the vision system, and a controller which determines a repositioning signal for the robot by comparing the actual positions with desired positions of the robot.

[0005] Further prior art is described in DE 10 2019 114 070 B4, DE 10 2015 204 796 A1, DE 20 2004 017 881 U1, DE 100 26 711 B4 and FR 2 905 888 B1.

[0006] The object of the invention is to create a robot-assisted manufacturing system with which it is possible to compensate for the occurrence of positioning errors, which can be attributed, for example, to temperature fluctuations or wear of robot components, even over large distances or with a large number of joints and segments of the robot.

[0007] The object is solved by the subject matter of claim 1. Advantageous developments of the invention are described in the subclaims.

[0008] A manufacturing system is described comprising: stability targets; a group of rails; robots having photogrammetry targets each attached to the heads of the robots, the robots being configured to move along the group of rails, each of the robots including a cutting tool and a camera attached to a respective one of the heads; and a photogrammetry system. The photogrammetry system includes: a pedestal; overhead cameras mounted on the pedestal above the robots and a workpiece to be machined by at least one of the robots; and at least one control module.The control module is configured to: determine the locations of the stability targets and the locations of the photogrammetry targets via the overhead cameras; determine the locations of the overhead cameras and the locations of the robot heads based on the locations of the stability targets and the locations of the photogrammetry targets; and control the robots to create and inspect at least one of the features on the workpiece based on the locations of the overhead cameras and the locations of the robot heads.

[0009] In further features, the robots comprise: at least one worker robot configured to form or machine the at least one feature; and at least one metrology robot configured to inspect the at least one feature.

[0010] In other features, each of the stability targets is within the field of view of at least two of the overhead cameras.

[0011] In other features, each of the photogrammetry targets is within the field of view of at least two of the overhead cameras.

[0012] For other features, at least two transmitters of each of the stability targets are within the field of view of at least two of the overhead cameras.

[0013] In further features, at least one of the photogrammetry targets is located within the field of view of at least two of the overhead cameras; and at least two transmitters of each of the photogrammetry targets are located within the field of view of at least two of the overhead cameras.

[0014] In other features, each feature of the workpiece to be machined or inspected is located within a predetermined range of two of the overhead cameras.

[0015] In other features, an intersection angle of each of the stability targets and each of the photogrammetry targets with respect to at least two of the overhead cameras is within a predetermined range.

[0016] In other features, the at least one control module is configured to determine the locations of the stability targets and the photogrammetry targets based on one or more reference coordinate systems.

[0017] In other features, the at least one control module is configured to determine the locations of the overhead cameras and the locations of the heads of the robots based on one or more reference coordinate systems.

[0018] In other features, a purely exemplary method of designing a manufacturing system is provided, comprising: defining a workpiece family with dimensions of the workpieces of the workpiece family to be machined; determining feature locations and workpiece orientations for features of the workpieces to be created or machined based on the dimensions of the workpieces; determining the positions of a plasma cutting tool of a work robot and a corresponding first photogrammetry target at each pre-position of the plasma cutting tool and at each active work position of the plasma cutting tool associated with creating or machining the features based on the feature positions; determining the locations of a second photogrammetry target on a metrology robot to determine the locations of a metrology robot camera at each pre-position and at each measurement position of the features;and determining the number of overhead cameras of an overhead photogrammetry guidance system and the locations of the overhead cameras based on first parameters. The exemplary method further includes: determining dimensions of an overhead structure supporting the overhead cameras based on second parameters; determining a number of stability targets and the locations of the stability targets based on third parameters; and generating and inspecting at least one of the features on one of the workpieces based on i) the positions of the plasma cutting tool, ii) the locations of the metrology robot cameras, iii) the dimensions of the overhead structure, iv) the locations and orientations of the overhead cameras, and v) the locations of the stability targets.

[0019] In other features, determining the locations of the measuring robot camera at each pre-position and at each measuring position includes determining at each pre-position and each measuring position i) reference points of the workpieces, ii) machined and measured features, and iii) measured but not machined features.

[0020] The first parameters include, in other features, i) the dimensions of the workpieces, ii) the positions of the features, iii) the locations of the first photogrammetry target, iv) the second photogrammetry target, and v) the locations of the measuring robot camera.

[0021] The second parameters include, in other features, i) the locations of the first and second photogrammetry targets, ii) the locations of the measuring robot camera, iii) the number of overhead cameras, and iv) the locations and orientations of the overhead cameras.

[0022] The third parameters include, in other features, i) the number of overhead cameras and ii) the locations of the overhead cameras.

[0023] In other features, machining and / or inspecting the features of one of the workpieces comprises: calibrating the overhead photogrammetry guidance system; calibrating the work robot and the measuring robot; moving one of the workpieces into a cell equipped with the overhead photogrammetry guidance system to create or machine the features of the one of the workpieces; aligning the one of the workpieces and determining the locations of datum points of the one of the workpieces; guiding the plasma cutting tool to machine features of the one of the workpieces; and guiding the camera of the measuring robot to inspect the features on the one of the workpieces.

[0024] In other features, calibrating the overhead photogrammetry guidance system involves determining the locations of the overhead cameras relative to the stability targets.

[0025] Calibrating the work robot and the measuring robot also includes, among other features, determining the locations of the first photogrammetry target and the second photogrammetry target relative to the overhead cameras.

[0026] In other features, the plasma cutting tool is guided based on known locations of the first photogrammetry target relative to the overhead cameras.

[0027] In other features, the camera of the measuring robot is guided based on known locations of the second photogrammetry target relative to the overhead cameras.

[0028] This disclosure will become more fully apparent from the detailed description and the accompanying drawings, in which: Fig. 1 is a perspective view of an exemplary cell of a manufacturing system having an overhead photogrammetry guidance system according to the present disclosure; Fig. 2 is a side view of the cell of Fig. 1; Fig. 3 is a top view of the cell of Fig. 1; Fig. 4 is a side view of an overhead camera stand of an overhead photogrammetry guidance system showing an overlapping field-of-view (FOV) of two overhead cameras according to the present disclosure; Fig. 5 is a perspective view of an example of a stability target according to the present disclosure; Fig. 6 is a perspective view of an exemplary photogrammetry target according to the present disclosure; Fig. 7 is a functional block diagram of the manufacturing system of Fig. 1; Fig. 8 shows a manufacturing method according to the present disclosure; and Fig. 9 shows a method of designing a photogrammetry guidance system according to the present disclosure.

[0029] Some products are manufactured in small quantities. For example, certain large heavy-duty trucks may have low production runs, so the trucks and / or parts thereof are manufactured in one or more cells. A cell is a three-dimensional (3D) space (or room) in which at least part of a vehicle is located during manufacturing. The part of the vehicle is not moved within the cell unless reorientated. A cell is not normally part of an assembly line. At least part of the vehicle is formed and / or assembled in the cell. For example, a structural frame of a heavy-duty truck may be brought into a cell and machined to create features on the structural frame. The features may be holes, slots, grooves, etc.

[0030] A cell may include worker robots and inspection robots. Tools for cutting, drilling, coating, and / or performing other work to form and / or modify features may be attached to the heads of the worker robots. The inspection robots may be equipped with cameras mounted on the heads of the robots to inspect the created features. The accuracy of the worker and inspection robots may be limited. For example, a robot may position the tip of a tool or an inspection camera to within ± 2 millimeters of a target position. This accuracy can be affected by temperature fluctuations, wear of robot components over time, etc.The greater the distance between a robot's base and its head and / or the more joints and segments (or links) the robot has, the greater the potential for error in the positioning of the robot's head.

[0031] The examples disclosed herein provide an overhead photogrammetry guidance system for improved positioning accuracy of robot heads and objects attached to the robot heads, such as tools and cameras. The tools may include cutting tools such as plasma torches (or plasma cutting tools), welding tools, fastening tools, etc. A corresponding method for designing the overhead photogrammetry guidance system is also disclosed. The overhead photogrammetry guidance system reduces the error in positioning the robot head to less than ± 1 mm, so that the tip of a tool and / or an inspection camera can be positioned with an accuracy of ± 1 mm to a desired position. In one embodiment, the accuracy is such that the positioning error is less than ± 0.5 mm.

[0032] Fig. 1-3 show a cell 100 of a manufacturing system 102 with an overhead photogrammetry guidance system 104. The cell 100 refers to a three-dimensional (3D) volume in which at least a portion of a product is machined, the product being referred to as a workpiece. In the example shown, the workpiece 106 is a large truck frame. The cell 100 may be used for manufacturing a product, e.g., a vehicle, or a portion thereof. The cell 100 may be one station in a series of stations used to manufacture the product.

[0033] The manufacturing system 102 includes a stand 110, work robot 112, inspection (or measuring) robot 114, rail assemblies 116, and a carriage 118 on which the workpiece 106 is mounted. The workpiece 106 is moved into the cell 100 via the carriage 118 and slid between the rail assemblies 116 to be centered in the cell 100, as shown. The carriage 118 can move along a track 300 in the floor (denoted by 301) that is linear and extends longitudinally and parallel to the rail assemblies 116. The carriage 118 includes a pivot system 120 capable of rotating the workpiece 106 about a longitudinal axis of the workpiece 106. The pivot system 120 can include a motor for rotating the workpiece 106. The carriage 118 may have motors to move the carriage 118 relative to the rail groups 116, or the carriage 118 may be manually moved relative to the rail groups 116.

[0034] The stand 110 includes legs 121 and crossbars 123, 127, and 129. The legs 121 are attached to feet 131 mounted on the floor of the cell. The legs 121 and the crossbars 123, 127, and 129 may be hollow and filled with a dampening material such as sand, pellets, and / or other dampening material. In one embodiment, the legs 121 and the crossbars 123, 127, and 129 are made of steel, but may be made of other materials. In one embodiment, the stand 110 is designed to be heavy and firmly planted on the floor. The feet 131 may be attached and / or glued to the floor. The stand 110 is designed to prevent movement of overhead cameras and to minimize and / or prevent vibration of overhead cameras.In one embodiment, the floor is constructed as a single concrete slab spanning the entire space beneath the workpiece 106, the robots 112, 114, the rail assemblies 116, and the column 110. This minimizes: relative movement between the stability targets 302 mounted on the floor; deformation of the column 110; and ensures greater overall accuracy of the overhead photogrammetry guidance system 104. In one embodiment, the concrete slab has a minimum thickness of 8 inches (200 mm) of reinforced concrete with no cuts or expansion joints therein.

[0035] The work robots 112 process the workpiece 106. This may include performing work such as cutting, drilling, welding, fastening, and / or other work. The work robots 112 are equipped with tools (e.g., any of the tools mentioned here) attached to heads 122 of the robots 112. As an example, the work robots 112 are shown with torches 124 mounted on the heads 122 of the robots 112. The inspection robots 114 have cameras 126 attached to the heads 128 of the robots 114. The cameras 126 are used to inspect features created by the work robots 112 on the workpiece 106, such as holes, slots, welds, installed fasteners, and / or other objects, etc. In the example shown, a work robot and an inspection robot are mounted on a respective pair of rails. Each of the rail groups 116 includes a pair of rails, as shown.The robots 112, 114 have bases 133 that move longitudinally along the rail assemblies 116. The robots 112, 114 are configured so that the heads 128 have six degrees of freedom (DOF) of movement about the respective axes. Each of the robots 112, 114 can include six joints and six linkages, with the head of the robot connected to the sixth linkage. The rail assemblies 116 provide a seventh axis of movement for each of the heads 128.

[0036] The overhead photogrammetry guidance system 104 is used for workpiece alignment and guidance of the robots 112, 114. The overhead photogrammetry guidance system 104 includes the overhead cameras, which include top-mounted cameras 136 and end-hanging cameras 138. Top-mounted means they are connected to an upper crossbar. The overhead cameras can be manually mounted in fixed positions or moved via motors represented by boxes 140. The top-mounted cameras 136 can be attached to and move along rails 142 that extend longitudinally along the crossbar 129, which extends from a front end of the cell 100 to a rear end of the cell 100. The rails 142 run parallel to the crossbar 129 and can be offset relative to it.The end-hanging cameras 138 can be attached to rails 146, which hang from and move along the crossbars 127 via vertically extending coupling elements 150. The horizontal and vertical positions of the overhead cameras can be adjusted to create lines of sight between the overhead cameras and features such as holes, body mounts, slots, and other features of various dimensions. The crossbars 127 extend laterally across the cell 100, perpendicular to the crossbar 129. The rails 146 run parallel to the crossbars 127.

[0037] The overhead cameras 136, 138 capture the positions of stability targets (or calibration targets) 302 and the photogrammetry targets (or navigation targets) 160. The stability targets 302 may be mounted on the floor of the cell 100 and located at fixed locations within the cell 100. Although six stability targets are illustrated, any number of stability targets may be used. The number of stability targets used depends on the number of overhead cameras and the fields of view of the overhead cameras. In one embodiment, at least one stability target is within the field of view of each overhead camera. In another embodiment, at least two stability targets are within the field of view of each overhead camera. The photogrammetry targets 160 are attached to the heads 122, 128 of the robots 112, 114. Examples of the targets 302, 160 are shown in Fig. 5-6. The overhead cameras 136, 138 monitor the movement of the photogrammetry targets 160 and are used to track the positions of the tools (e.g., the torch 124) and cameras (e.g., the cameras 126) mounted on the robots 112, 114. The cameras 126 are used to detect reference points on the workpiece 106 and to track and inspect features and surfaces of the workpiece 106. The overhead cameras 136, 138 can also be used to determine the location or position of features and surfaces of the workpiece 106.

[0038] Fig. Figure 4 shows a stand 400 for overhead cameras 402 of an overhead photogrammetry guidance system such as the photogrammetry guidance system 104 of Fig. 1 - 3. The stand 400 is another example of a stand that replaces the stand 110 of Fig. 1 - 3. The overhead cameras 402 are similar to the overhead cameras 136 in Fig. 1 - 3. The stand 400 includes vertical legs 404, a crossbar 406, and crossbars 407 (one is shown). Angled support legs 408 are connected to the vertical legs 404. Angled support plates 410 are connected to upper portions of the vertical legs 404 and the crossbars 407. There may also be angled support plates attached to side crossbars that are Fig. 4 are not shown, but are arranged similarly to the angled support plates 410 and are attached to laterally extending crossbars, similarly to the crossbars 127 of the Fig. 1 and Fig. 3.

[0039] In Fig. 4 shows an example of an overlapping FOV 420 from two of the overhead cameras 402. The photogrammetry guidance system is configured so that at any given time, at least two of the overhead cameras can see each stability target and each photogrammetry target in the corresponding cell. In other words, each target is seen by at least two of the overhead cameras at any given time during a manufacturing process. The targets can be seen by different overhead cameras. This is done so that the locations of the stability targets and the photogrammetry targets can be determined at any given time. The photogrammetry system can, for example, use triangulation to determine the locations of all stability targets and photogrammetry targets. An example target 422 is shown and can represent a photogrammetry target.Each of the cameras, two of which are designated 430, 432 for two of the cameras 402, has a field of view angle. An example field of view intersection angle 434 for the target 422 is shown and refers to an angle between two lines 436 extending from the centers of the cameras and intersecting at a point on the target 422, with the target 422 located in the overlapping FOV 420. In one embodiment, the minimum intersection angle is 35° and the maximum intersection angle is 90°. In one embodiment, the field of view of each of the overhead cameras 402 and other overhead cameras (in . Fig. 4 not shown), such as the hanging overhead cameras 138 in Fig. 1-3, 45-90°. In one embodiment, the photogrammetry system is configured such that at any given time, the distance between each of the overhead cameras 402 and at least one of the targets is 1.5 to 6.0 meters (m).

[0040] Fig. 5 shows a stability target 500 having a housing 502 and emitters (or artifacts) 504. The housing 502 can have any shape and size and can contain any number of emitters. In one embodiment, the emitters 504 comprise light-emitting diodes (LEDs). The housing 502 contains two or more emitters. In one embodiment, the housing 502 has four emitters (e.g., four LEDs). The emitters 504 can be arranged in a row or linear pattern, as shown, or they can be arranged in a non-linear pattern. Although shown in a plane extending parallel to a top surface 506 of the housing 502, the emitters 504 can be offset from one another such that the emitters 504 do not lie in a plane extending parallel to the top surface 506. The emitters 504 can be arranged in various patterns.In one embodiment, each of the emitters 504 has a ±60° conical emission pattern that can be viewed by multiple overhead cameras. In one embodiment, the housing 502 is made of carbon fiber and / or another material whose dimensions change minimally with temperature fluctuations.

[0041] Fig. 6 shows a photogrammetry target 600 having a housing 602 and emitters (or artifacts) 604. The housing 602 can have any shape and size and can contain any number of emitters. In one embodiment, the emitters 604 comprise LEDs. The housing 602 contains two or more emitters. In one embodiment, the housing 602 has four emitters (e.g., four LEDs). The emitters 604 can be arranged in a row or a linear pattern, or in a non-linear pattern. Although shown in two planes extending parallel to the top surfaces 606, 608 of the housing 602, the emitters 604 can lie in the same plane extending parallel to a top surface of the housing 602, or they can lie in more than two planes extending parallel to more than two top surfaces. The emitters 604 can be arranged in different patterns.In one embodiment, each of the emitters 604 has a ±60° conical emission pattern that can be viewed by multiple overhead cameras. In one embodiment, the housing 602 is made of carbon fiber and / or another material whose dimensions change minimally with temperature fluctuations.

[0042] In the example shown, the housing 602 includes a base portion 610 and a tapered portion 612 formed integrally with the base portion. Three of the transmitters 604 are mounted on the base portion 610 and in a first plane extending parallel to first top surfaces 614. Two of the transmitters 604 are mounted on the tapered portion 612 and in a second plane offset from the first plane and extending parallel to a second top surface 616. The tapered portion 612 has sloped sides 618 extending from the surfaces 614 to the surface 616. Although the housing 602 is shown in the shape of a hat, the housing 602 may be shaped differently than shown.

[0043] The targets 500, 600 of the Fig. 5 - 6 and the other similar targets of the Fig. 1 - 3 can be controlled by the corresponding robots (e.g. robots 112, 114 of the Fig. 1 - 3) and / or one or more system control modules, such as the one in Fig. 7. The LEDs may be continuously illuminated or may be turned on and off at specific frequencies and / or have corresponding duty cycles during the manufacturing process. The one or more system control modules may select the frequencies, duty cycles, intensities, etc., of the targets' emitters. For example, the stability targets and the photogrammetry targets' emitters may each have a specific color, frequency, duty cycle, intensity, and / or other distinguishing parameters compared to the other stability targets and photogrammetry targets. In one embodiment, the target emitters have the same color and are continuously turned on during a manufacturing process (e.g., a cutting process to create one or more features of a workpiece).

[0044] Fig. 7 shows a manufacturing system 700 comprising: an overhead photogrammetry guidance system 701 with a stand 702; work and inspection robots 704; one or more system control modules 706; a cart 708 on which a workpiece 710 is mounted; stability targets 712; and photogrammetry targets 714 mounted on the robots 704. The manufacturing system 700 may be configured similarly to the manufacturing system 102 in Fig. 1 - 3. The stand 702 may be configured similarly to one of the stands 110 and 400 in Fig. 1 - 4. The stand 702 may contain overhead cameras 720, camera actuators 722 with motors 724, and one or more reference coordinate frames 726 connected to a first coordinate system of the corresponding overhead photogrammetry guidance system.

[0045] The robots 704 may each include joint motors 730, linkages 732, joints 734, a head 735, one or more reference coordinate frames 736 connected to a second coordinate system, one or more tools and / or cameras 738, a base 740, a rail motor 742, and a robot control module 744. The joint motors 730 are located at the joints 734 and move the linkages 732 to move the head 735. The joint motors 730 are controlled by the robot control module 744. The one or more tools and / or cameras 738 are mounted on the head 735. A first of the linkages 732 is connected to the base 740, which extends along rails (e.g., rails of one of the rail groups 116 of Fig. 1). The rail motor 742 can move the base 740 relative to the rails. Alternatively, a separate motor from one of the robots 704 can be used, for example, to rotate a ball screw in a rail assembly that includes the rails to move the robot relative to the rails.

[0046] Each of the robots 704 includes one or more photogrammetry targets 714, which may include transmitters 746. In one embodiment, each of the photogrammetry targets 714 is configured similarly to the photogrammetry target 600 in Fig. 6. Each of the robots 704 may contain any number of photogrammetry targets 714.

[0047] The carriage 708 may include a pivot system 750, a workpiece support 752, and one or more reference coordinate frames connected to a third coordinate system. The workpiece support 752 holds the workpiece 710. The pivot system 750 may include a motor 754 for rotating the workpiece 710 on the workpiece support 752. The carriage 708 may also include one or more translation motors 756 for moving the carriage 708 relative to the support 702.

[0048] The workpiece 710 may include datum points 760, features 762, and one or more reference coordinate frames 764 associated with the third coordinate system. The datum points 760 may include holes, slots, and / or other reference points and / or features of the workpiece 710. The features 762 refer to features of the workpiece 710 that are being machined, such as holes, slots, fasteners, welds, grooves, channels, etc.

[0049] Each of the stability targets 712 may include transmitters, e.g., transmitters 770. In one embodiment, the stability targets 712 are each configured similarly to the stability target 500 in Fig. 5.

[0050] The one or more system control modules 706 may control the overhead cameras 720, the motors 724, the articulated motors, the tools and / or cameras 738, the track motor 742, the robot control module 744, the transmitters 746, the motors 754, and the transmitters 770. The one or more system control modules 706 coordinate the timing, positioning, locations, on / off times, frequencies, duty cycles, intensities, speeds, etc., of these elements. The positioning and / or locations may be determined relative to the coordinate systems. The locations determined relative to the second and third coordinate systems may be converted to or referenced to the first coordinate system, which may be referred to as the overall cell coordinate system (or master coordinate system) of the corresponding cell.The reference coordinate frames 726, 736, 755, 764 may refer to reference points on the stand 702, the robots 704, the carriage 708, and the workpiece 710. The reference coordinate frames 726, 736, 755, 764 may be used to determine the positions of the head 735, the tools and / or cameras 738, and the features 762 and / or the corresponding surfaces of the workpiece 710 relative to the stand 702, the carriage 708, the overhead cameras 720, the stability targets 712, the photogrammetry targets 714, and / or other elements in the cell. One or more of the reference coordinate frames 726, 736, 755, 764 may be virtual reference coordinate frames. For example, one of the reference coordinate frames 726 may be a virtual reference coordinate frame, and the position of the workpiece 710 and portions thereof may be determined relative to the virtual reference coordinate frame.This can include X, Y, Z, roll, pitch, and yaw values ​​for each reference point of the workpiece. A best-fit method can be used to align the workpiece to the first coordinate system (or that of the entire cell).

[0051] The reference coordinate frames 726, 736, 755, 764 can be placed anywhere in the cell and on the aforementioned elements. One of the reference coordinate frames 764 can be located on a front side of the workpiece 710. Each of the reference coordinate frames 726, 736, 755, 764 can have an x-axis pointing horizontally from the front to the back of the cell, a y-axis extending laterally through the cell perpendicular to the x-axis, and a z-axis extending vertically and perpendicular to the x- and y-axes.

[0052] A control module is configured to determine the position of the workpiece 710 with six degrees of freedom (DOF) and target locations at which holes are to be cut into the workpiece 710 based on the data from the inspection robots and the locations of the identifiers.

[0053] In one embodiment, one or more cutting robots (or worker robots) have identifiers on or near the end effectors of the cutting robot(s). The control module determines the position(s) of the end effector(s) based on the identifiers. In this way, the position of the end effector(s) can be precisely determined and controlled by the control module to cut the holes in the corresponding workpiece (e.g., the vehicle frame) at the target positions. This ensures flexibility and quality in the production of medium- to low-volume workpieces to be processed with the inspection and cutting station.

[0054] The locations of the overhead cameras 720, the stability targets 712, and the photogrammetry targets 714 are flexible. The positions of the photogrammetry targets 714 and the stability targets 712 can change to establish lines of sight between reference points 760 and the measurement robot cameras (or inspection cameras) 738, and to establish lines of sight between i) transmitters of the photogrammetry targets 714 and the stability targets 712, and ii) the overhead cameras 720. In one embodiment, the overhead cameras 720 comprise 6-8 cameras. The cameras include top-mounted cameras and pendant (or end-mounted) cameras. Top-mounted means they are connected to a top crossbar. The top-mounted cameras can be mounted above, vertically flush with, or below the crossbar.

[0055] Fig. Figure 8 shows a manufacturing process. Although the process is primarily related to the manufacturing system of Fig. 7, the method is also applicable to the manufacturing system of Fig. 1 - 3 applicable.

[0056] At 800, the one or more control modules 706 or other control modules design an overhead photogrammetry guidance system (e.g., the overhead photogrammetry guidance system 104 of Fig. 1 - 3 or the overhead photogrammetry guidance system 701 from Fig. 7). In Fig. 9, a design method is shown that may be implemented as part of operation 800. The design method includes determining i) the dimensions of an overhead structure for overhead cameras and / or the dimensions of a stand (e.g., stand 110, 702 of Fig. 1 and Fig. 7) and ii) the number and locations of the stability targets. For example, the cell can be 15-20 feet long, 10-15 feet wide, and 10-25 feet high to accommodate a workpiece that is 10-15 feet long, 5-8 feet wide, and 1-3 feet high. The dimensions of the overhead structure, as well as the number and locations of the stability targets, are used to construct the overhead photogrammetry guidance system. The overhead photogrammetry guidance system includes a pedestal, overhead cameras, and one or more control modules 706.

[0057] At 802, the one or more control modules 706 calibrate the one or more control modules 706 to determine the locations of the overhead cameras in the cell based on at least one reference coordinate frame and detected locations of stability targets. The at least one reference coordinate frame may include reference coordinate frame 736.

[0058] At 804, the one or more control modules 706 calibrate the work and inspection robots based on at least one reference coordinate frame using the overhead cameras to determine the positions of the robots' photogrammetry targets relative to the overhead cameras. The emission patterns of the targets' emitters are captured and can be compared to known patterns for the targets to determine the locations and / or orientations of the targets. The at least one reference coordinate frame can include reference coordinate frame 736.

[0059] At 806, a workpiece is moved into the cell. This can be done manually or by the one or more control modules 706.

[0060] At 808, the one or more control modules 706 align the workpiece if it is not in the correct orientation for first (or next) features and determines the positions of datum points on the workpiece to determine a position of the workpiece relative to one or more coordinate systems, the robots, and the overhead cameras. The coordinate systems may include any of the first, second, and third coordinate systems of the stand, the robots, and the carriage. At the beginning of a process and after each workpiece reorientation, the positions of the datum points are measured. The datum points are acquired with the overhead photogrammetry guidance system to determine an accurate (six DOF) attitude of the workpiece relative to the reference coordinate frames. This is referred to as zero-point alignment, or datum point alignment, where the position of the workpiece is mapped into the overhead photogrammetry guidance system.The overhead photogrammetry guidance system determines the offsets of the robots to precisely locate features.

[0061] At 810, the one or more control modules 706 guide tools of one or more work robots to one or more surfaces of the workpiece to be machined. Before forming and / or machining a feature, the positions of the surfaces are measured. The heads of the work robots can be roughly moved close to the features to be machined (pre-positions), and then the overhead photogrammetry guidance system can be used to precisely position the working ends or tips of the tools on the workpiece. The exact positioning of the ends or tips can be within ± 1 mm or less of the target positions.

[0062] At 812, the one or more control modules 706 machine the workpiece, including optionally guiding the tools on the one or more work robots to create one or more features on the workpiece.

[0063] Each pre-position is the position at which the robot stops shortly before processing the workpiece (e.g., cutting with a torch or measuring with a camera). A pre-position is close to an actual start position for the work operation (e.g., cutting or measuring). The pre-position is far enough from the workpiece so that no collision with the workpiece occurs. Once the pre-position is assumed, the one or more control modules 706 determine and / or calculate the position of the tool or camera on the head of the robot. Based on this information, the one or more control modules 706 determine the X, Y, Z, pitch, yaw, and roll offsets to correct the robot position and improve accuracy. The work position or measurement position, depending on whether the robot is moving a tool or a camera, is the corrected position at which the processing (e.g.,Cutting) or measurement should begin or end. The working and measuring positions are specific for each feature and each frame. The working and measuring positions are determined based on i) current measurements of datum points on the workpiece and ii) measurements of the feature surfaces (e.g., hole surfaces) to be produced (e.g., cut) with the tool.

[0064] At 814, the one or more control modules 706 determine whether there is another feature to create and / or another surface to machine. If so, operation 808 is performed; otherwise, operation 816 is performed.

[0065] At 816, the one or more control modules 706 guide the cameras of one or more inspection robots to the positions of the features and inspect the features via the cameras of the inspection robots. The features are measured. The dimensions and positions of the features are measured.

[0066] At 818, the one or more control modules 706 determines whether there is another feature to be checked. If so, operation 816 may be performed; otherwise, the method may be terminated.

[0067] Fig. 9 shows a method for designing a photogrammetry guidance system. At 900, the one or more control modules 706 define a workpiece family with the dimensions of the workpieces of the family to be machined. A family of workpieces may refer to several different workpieces with different sizes, shapes, properties, etc. The workpieces of a particular family may be of the same type and / or model, but have different dimensions and / or a different number of features. In one embodiment, and if the part family is, for example, a truck frame family, the maximum length, width, and height dimensions of the workpieces are defined.

[0068] At 902, the one or more control modules 706 determine the feature locations and workpiece orientations for the feature locations for variants in the workpiece family. This may be based on the dimensions of the workpieces. For example, the positions of the features may refer to the locations on a workpiece where the features are to be placed. In one embodiment, and for a workpiece family that includes truck frames, cutting locations and alignment tools for the cutting locations are determined for the variants of the truck frames in the family.

[0069] At 904, the one or more control modules 706 determine the positions of the tools of the work robots and the corresponding photogrammetry targets in each pre-position and in each active work tool position. In one embodiment, and for a workpiece family that includes vehicle frames, each pre-position and each cutting position of each tool is determined for each workpiece in the workpiece family. This is done for each feature of each workpiece to be machined and is based on the positions of the features. The work positions are determined based on i) current measurements of reference points of the workpiece and ii) measurements of the feature surfaces (e.g., hole surfaces) to be produced (e.g., cut) with the tool.

[0070] At 906, the one or more control modules 706 determine the positions of the photogrammetry targets on the measuring robots to determine the positions of the measuring robot cameras at each pre-position and at each measurement position for datum points, machined and measured features, and measured but not machined features. The pre-positions and measurement positions can be datum points of the workpieces of the workpiece family, machined features, and unmachined features. In one embodiment, and for a workpiece family that includes truck frames, each position of the robot cameras (used for inspection) and the associated photogrammetry targets is determined at each pre-position and in each measurement position, including the positions of datum points, cutting features, and features that are only being measured.

[0071] At 908, the one or more control modules 706 determine i) the number of photogrammetry cameras and ii) the locations and orientations of the photogrammetry cameras. This is done so that each of the photogrammetry target locations and orientations is visible throughout an entire manufacturing process of forming and / or machining features for a workpiece of the respective family and is within acceptable FOVs of at least two overhead cameras. At least a portion of a photogrammetry target with a predefined pattern of emitters (e.g., two or more emitters) is visible at any time during the manufacturing process by at least two of the overhead cameras. In one embodiment, 2-5 emitters are visible during the manufacturing process by at least two overhead cameras. The number of overhead cameras and the corresponding locations can be based on initial parameters, such asthe dimensions of the workpieces, the positions of the features, the locations of the photogrammetry targets and the locations of the measuring robot cameras.

[0072] At 910, the one or more control modules 706 determine the dimensions of the overhead structure and / or the stand to support the overhead cameras at the locations determined based on the information determined in the above operations. The dimensions of the overhead structure and / or the stand may be determined based on second parameters such as the locations of the photogrammetry targets, the locations of the measurement robot cameras, the number of overhead cameras, the locations and orientations of the overhead cameras, the number of photogrammetry cameras, and the locations of the photogrammetry cameras.

[0073] At 912, the one or more control modules 706 determine the number and locations of the stability targets. This can be determined such that each photogrammetry camera can see at least two stability targets. The number and locations of the stability targets are determined such that at least a portion of the stability targets with a predefined pattern of transmitters are visible to at least two of the overhead cameras at any time during the manufacturing process. The number of stability targets and the locations of the stability targets can be determined based on third parameters such as the number of overhead cameras and the locations of the overhead cameras.

[0074] Following the implementation of the procedure by Fig. 9 may be some of the information provided during the procedure by Fig. 9 parameters determined during the procedure of Fig.8. These parameters include: the working locations of the tools during use, which can remain the same as long as the type and model of the workpiece remain the same; the locations of the robot cameras, which remain the same as long as the type and model of the workpiece remain the same; the geometry of the overhead structure supporting the overhead cameras, which remains the same; the locations and orientations of the photogrammetry cameras, which remain fixed during workpiece production until the workpiece family changes; and the location of the stability targets (or artifacts), which remains constant as long as the locations of the overhead photogrammetry cameras are not changed.

[0075] The examples described above include the development and deployment of an overhead photogrammetry guidance system that enables precise positioning of the robot heads for precise positioning of the corresponding work tools and the inspection cameras mounted on the robot heads. The heads or final links of the robots are equipped with tools and inspection cameras to enable precise guidance and positioning. The inspection cameras are used for quality control checks. Each cell has an overhead photogrammetry structure that supports multiple cameras and provides an image processing system that provides feedback based on which the tools and inspection cameras can be precisely placed and moved. The examples disclosed herein are suitable for frame cutting applications and other applications.

[0076] The examples described above provide an overhead photogrammetry guidance system that includes a minimum number of overhead cameras and targets to accurately determine the position of a workpiece within a cell and the positions of datum points and features of the workpiece within the cell. A minimum number of targets is used for precise real-time positioning of tools and cameras on the robot heads and to control the movement of the tools and cameras. This is based on the accurate determination of the positions of the datum points prior to the guided movement. The inclusion of a minimum number of overhead cameras, targets, and inspection cameras enables high-speed capture and mapping of the positions of workpieces and their parts within a cell, as well as the inspection of workpiece features. The locations are six DOF locations within the cell.The tools and cameras mounted on the heads can be moved to target positions based on the dimensions and position of the workpiece surfaces relative to the cell's overall coordinate system. The examples are flexible for different products (or workpieces) by adjusting the locations, orientations, and / or number of overhead cameras. The examples are shown without fixtures, meaning the workpieces are not held by workpiece-specific fixtures, but rather by a carriage that can accommodate various types and models of workpieces.

[0077] In this application, including the definitions below, the term "module" or the term "controller" may be replaced by the term "circuit". The term "module" may refer to, be a part of, or contain: an application-specific integrated circuit (ASIC); a digital, analog, or mixed analog / digital discrete circuit; a digital, analog, or mixed analog / digital integrated circuit; a combinational logic circuit; a field-programmable gate array (FPGA); processor circuitry (common, dedicated, or group) executing code; memory circuitry (common, dedicated, or group) storing code executed by the processor circuitry; other suitable hardware components providing the described functionality; or a combination of some or all of the above, e.g., in a system-on-chip.

[0078] The module may include one or more interface circuits. In some examples, the interface circuits may include wired or wireless interfaces connected to a local area network (LAN), the Internet, a wide area network (WAN), or combinations thereof. The functionality of any module of the present disclosure may be distributed among multiple modules connected via interface circuits. For example, multiple modules may enable load balancing. In another example, a server module (also referred to as a remote or cloud module) may perform some functions on behalf of a client module.

[0079] The term code, as used above, may include software, firmware, and / or microcode and may refer to programs, routines, functions, classes, data structures, and / or objects. The term "shared processor circuit" includes a single processor circuit that executes some or all of the code from multiple modules. The term "group processor circuit" includes a processor circuit that, in combination with other processor circuits, executes some or all of the code from one or more modules. References to multiple processor circuits include multiple processor circuits on discrete chips, multiple processor circuits on a single chip, multiple cores of a single processor circuit, multiple threads of a single processor circuit, or a combination of the above.The term "shared memory circuit" refers to a single memory circuit that stores some or all of the code from multiple modules. The term "group memory circuit" refers to a memory circuit that, in combination with other memories, stores some or all of the code from one or more modules.

[0080] The term "memory circuit" is a subset of the term "computer-readable medium." The term "computer-readable medium," as used herein, does not include transitory electrical or electromagnetic signals that propagate through a medium (e.g., on a carrier wave); the term "computer-readable medium" can therefore be considered tangible / material and non-transitory. Non-limiting examples of a non-transitory, tangible, computer-readable medium include non-volatile memory circuits (e.g., a flash memory circuit, an erasable, programmable read-only memory circuit, or a mask read-only memory circuit), volatile memory circuits (e.g., a static random access memory circuit or a dynamic random access memory circuit), magnetic storage media (e.g., an analog or digital magnetic tape or a hard disk drive), and optical storage media (e.g.,a CD, a DVD or a Blu-ray Disc).

[0081] The devices and methods described in this application may be implemented partially or entirely by a special-purpose computer formed by configuring a general-purpose computer to perform one or more specific functions embodied in computer programs. The functional blocks, flowchart components, and other elements described above serve as software specifications that can be translated into computer programs through the routine work of a skilled technician or programmer.

[0082] The computer programs contain processor-executable instructions stored on at least one non-transitory, tangible, computer-readable medium. The computer programs may also contain or access stored data. The computer programs may include a basic input / output system (BIOS) that interacts with the hardware of the special-purpose computer, device drivers that interact with specific devices of the special-purpose computer, one or more operating systems, user applications, background services, background applications, etc.

[0083] The computer programs may contain: (i) descriptive text to be parsed, e.g. HTML (Hypertext Markup Language), XML (Extensible Markup Language) or JSON (JavaScript Object Notation) (ii) assembly code, (iii) object code generated from the source code by a compiler, (iv) source code for execution by an interpreter, (v) source code for compilation and execution by a just-in-time compiler, etc. The source code may only contain, for example, the syntax of languages ​​such as C, C++, C#, Objective-C, Swift, Haskell, Go, SQL, R, Lisp, Java®, Fortran, Perl, Pascal, Curl, OCaml, Javascript®, HTML5 (Hypertext Markup Language 5th revision), Ada, ASP (Active Server Pages), PHP (PHP: Hypertext Preprocessor), Scala, Eiffel, Smalltalk, Erlang, Ruby, Flash®, Visual Basic®, Lua, MATLAB, SIMULINK and Python®. Figure caption for Fig. 8: Y Yes N No

Claims

[1] Manufacturing system (102) comprising: a variety of stability targets (302); a group of rails (116); a plurality of robots (112, 114) comprising a plurality of photogrammetry targets (160) each mounted on heads (122, 128) of the plurality of robots (112, 114), the plurality of robots (112, 114) being configured to move along the set of rails (116), and each of the plurality of robots (112, 114) comprising a plasma cutting tool and a camera (126) mounted on a respective one of the heads (122, 128); and a photogrammetry system (104) comprising: a stand (110), a plurality of overhead cameras (136, 138) mounted on the stand (110) above the plurality of robots (112, 114) and a workpiece (106) to be processed by at least one of the plurality of robots (112, 114), and at least one control module configured to: Determining the locations of the plurality of stability targets (302) and the locations of the plurality of photogrammetry targets (160) by means of the plurality of overhead cameras (136, 138), Determining the locations of the plurality of overhead cameras (136, 138) and the locations of the heads (122, 128) of the plurality of robots (112, 114) based on the locations of the plurality of stability targets (302) and the locations of the plurality of photogrammetry targets (160), and Controlling the plurality of robots (112, 114) to generate and inspect at least one feature on the workpiece (106) based on the locations of the plurality of overhead cameras (136, 138) and the locations of the heads (122, 128) of the plurality of robots (112, 114). [2] The manufacturing system (102) of claim 1, wherein the plurality of robots comprises: at least one work robot (112) configured to form or machine the at least one feature; and at least one measuring robot (114) configured to inspect the at least one feature. [3] The manufacturing system (102) of claim 1, wherein each of the plurality of stability targets (302) is located within the field of view of at least two of the plurality of overhead cameras (136, 138). [4] The manufacturing system (102) of claim 1, wherein each of the plurality of photogrammetry targets (160) is located within the field of view of at least two of the plurality of overhead cameras (136, 138). [5] The manufacturing system (102) of claim 1, wherein at least two transmitters (504) from each of the plurality of stability targets (302; 500) are located within the field of view of at least two of the plurality of overhead cameras (136, 138). [6] Manufacturing system (102) according to claim 1, wherein: Transmitters (604) of at least one of the plurality of photogrammetry targets (160; 600) are located in the field of view of at least two of the plurality of overhead cameras (136, 138); and / or at least two transmitters (604) from each of the plurality of photogrammetry targets (160; 600) are located in the field of view of at least two of the plurality of overhead cameras (136, 138). [7] The manufacturing system (102) of claim 1, wherein each feature of the workpiece (106) to be machined or inspected is located within a predetermined range of two of the plurality of overhead cameras (136, 138). [8] The manufacturing system (102) of claim 1, wherein an intersection angle (434) of each of the plurality of stability targets (302) and each of the plurality of photogrammetry targets (160; 422) with respect to at least two of the plurality of overhead cameras (136, 138; 402) is within a predetermined range. [9] The manufacturing system (102) of claim 1, wherein the at least one control module is configured to determine the locations of the plurality of stability targets (302) and the locations of the plurality of photogrammetry targets (160) based on one or more reference coordinate frames. [10] The manufacturing system (102) of claim 1, wherein the at least one control module is configured to determine the locations of the plurality of overhead cameras (136, 138) and the locations of the heads (122, 128) of the plurality of robots (112, 114) based on one or more reference coordinate frames.

Images