Robot-assisted manufacturing system

Abstract

A manufacturing system comprises: stability targets; a set of rails; robots with photogrammetry targets mounted on their heads, the robots moving along the set of rails, each robot having a plasma cutting tool or camera mounted on one of its heads; and a photogrammetry system. The photogrammetry system comprises: a stand; overhead cameras mounted on the stand above the robots and a workpiece to be processed by one of the robots; and a control module.The control module uses the overhead cameras to determine the locations of the stability targets and the photogrammetry targets; based on the locations of the stability targets and the photogrammetry targets, it determines the locations of the overhead cameras and the robot heads; and controls the robots to create and inspect a feature on the workpiece based on the locations of the overhead cameras and the robot heads.

Description

INTRODUCTION

[0001] The information in this section serves to present the general context of the disclosure. Works of the inventors mentioned herein, insofar as they are described in this section, as well as aspects of the description that may not have been prior art at the time of filing, are neither expressly nor implicitly admitted as prior art against the present disclosure.

[0002] The present disclosure relates to the manufacture or production of products and in particular to the robot-assisted formation of features and the testing of product workpieces.

[0003] Materials such as aluminum, steel, and plastic are used in vehicle manufacturing. Vehicle components (or parts) are manufactured using tools such as stamping, pressing, forming, and cutting machines. Examples of such parts include vehicle frames, doors, hoods, roofs, trunk lids, and so on.

[0004] Vehicles are often assembled on an assembly line. The components of a vehicle, and the vehicle itself, can be assembled at stations along the line. This can include shaping features and attaching and / or joining components using various techniques such as cutting, welding, fastening, and / or gluing. Detail work such as cleaning, applying corrosion and / or scratch protection coatings, painting, etc., can also be carried out. SUMMARY

[0005] A manufacturing system is provided and comprises: stability targets; a group of rails; robots with photogrammetry targets, each attached to the robot heads, the robots configured to move along the group of rails, each robot containing a cutting tool and a camera attached to one of its heads; and a photogrammetry system. The photogrammetry system comprises: a stand; overhead cameras mounted on the stand above the robots and a workpiece to be machined by at least one of the robots; and at least one control module.The control module is configured to: determine the locations of the stability targets and the photogrammetry targets via the overhead cameras; determine the locations of the overhead cameras and the robot heads based on the locations of the stability targets and the photogrammetry targets; and control the robots to generate and check or inspect at least one of the features on the workpiece based on the locations of the overhead cameras and the robot heads.

[0006] Further features include: at least one working robot configured to shape or process the at least one feature; and at least one measuring robot configured to inspect the at least one feature.

[0007] In other features, each of the stability targets is located within the field of view of at least two of the overhead cameras.

[0008] In other features, each of the photogrammetry targets is located within the field of view of at least two of the overhead cameras.

[0009] For other features, at least two transmitters of each of the stability targets are within the field of view of at least two of the overhead cameras.

[0010] Further features include at least one of the photogrammetry targets being within the field of view of at least two of the overhead cameras; and at least two transmitters of each of the photogrammetry targets being within the field of view of at least two of the overhead cameras.

[0011] In other features, each feature of the workpiece to be processed or inspected is located within a predefined area of ​​two of the overhead cameras.

[0012] In other features, the intersection angle of each of the stability targets and each of the photogrammetry targets with respect to at least two of the overhead cameras lies within a specified range.

[0013] In other features, at least one control module is configured to determine the locations of the stability targets and the photogrammetry targets based on one or more reference coordinate systems.

[0014] In other features, at least one control module is configured to determine the locations of the overhead cameras and the locations of the robot heads based on one or more reference coordinate systems.

[0015] Other features provide a method for designing a manufacturing system, which includes: defining a workpiece family with dimensions of the workpieces to be machined within the workpiece family; determining feature positions and workpiece orientations for features of the workpieces to be created or machined based on the workpiece dimensions; determining the positions of a plasma cutting tool on a work robot and a corresponding first photogrammetry target at each pre-position of the plasma cutting tool and in each active working position of the plasma cutting tool associated with the creation or machining of the features, based on the feature positions; determining the locations of a second photogrammetry target on a measuring robot to determine the locations of a measuring robot camera at each pre-position and at each measuring position of the features;and determining the number of overhead cameras of an overhead photogrammetry guidance system and the locations of the overhead cameras based on first parameters. The procedure further includes: determining the dimensions of an overhead structure holding the overhead cameras based on second parameters; determining a number of stability targets and the locations of the stability targets based on third parameters; and generating and inspecting at least one of the features on one of the workpieces based on i) the positions of the plasma cutting tool, ii) the locations of the measuring robot cameras, iii) the dimensions of the overhead structure, iv) the locations and orientations of the overhead cameras, and v) the locations of the stability targets.

[0016] In other features, determining the locations of the measuring robot camera at each pre-position and at each measuring position includes determining each pre-position and each measuring position of i) reference points of the workpieces, ii) machined and measured features, and iii) measured but not machined features.

[0017] The first parameters include, among other features, i) the dimensions of the workpieces, ii) the positions of the features, iii) the locations of the first photogrammetry target, iv) the second photogrammetry target, and v) the locations of the measuring robot camera.

[0018] The second parameters include, among other features, i) the locations of the first and second photogrammetry targets, ii) the locations of the measuring robot camera, iii) the number of overhead cameras, and iv) the locations and orientations of the overhead cameras.

[0019] The third parameters include, among other features, i) the number of overhead cameras and ii) the locations of the overhead cameras.

[0020] In other features, the processing and / or inspection of the features of one of the workpieces includes: calibrating the overhead photogrammetry guidance system; calibrating the work robot and the measuring robot; moving one of the workpieces into a cell equipped with the overhead photogrammetry guidance system to create or process the features of one of the workpieces; aligning one of the workpieces and determining the locations of reference points of one of the workpieces; guiding the plasma cutting tool to process features of one of the workpieces; and guiding the camera of the measuring robot to inspect the features on one of the workpieces.

[0021] Other features of calibrating the overhead photogrammetry guidance system include determining the locations of the overhead cameras relative to the stability targets.

[0022] Calibrating the work robot and the measuring robot also includes, in other features, determining the locations of the first photogrammetry target and the second photogrammetry target relative to the overhead cameras.

[0023] In other features, the plasma cutting tool is guided based on known locations of the first photogrammetry target relative to the overhead cameras.

[0024] In other features, the camera of the measuring robot is guided based on known locations of the second photogrammetry target relative to the overhead cameras.

[0025] Further applications of this disclosure will become apparent from the detailed description, the claims, and the drawings. The detailed description and specific examples serve only for illustration and are not intended to limit the scope of this disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The present disclosure will become more fully apparent from the detailed description and the accompanying drawings, whereby the following applies: Fig. Figure 1 is a perspective view of an exemplary cell of a manufacturing system with an overhead photogrammetry guidance system according to the present disclosure; Fig. 2 is a side view of the cell of Fig. 1; Fig. 3 is a top view of the cell of Fig. 1; Fig. Figure 4 is a side view of a stand for overhead cameras of an overhead photogrammetry guidance system, showing an overlapping field of view (FOV) of two overhead cameras according to the present disclosure; Fig. Figure 5 is a perspective view of an example of a stability target according to the present disclosure; Fig. Figure 6 is a perspective view of an exemplary photogrammetry target according to the present disclosure; Fig. Figure 7 is a functional block diagram of the manufacturing system of Fig. 1; Fig. Figure 8 shows a manufacturing process according to the present disclosure; and Fig. Figure 9 shows a method for designing a photogrammetry guidance system according to the present disclosure.

[0027] Reference numbers can be reused in the drawings to designate similar and / or identical elements. DETAILED DESCRIPTION

[0028] Some products are manufactured in small batches. Certain large heavy-duty trucks, for example, may have low production volumes, so the trucks and / or parts thereof are manufactured in one or more cells. A cell is a three-dimensional (3D) space (or room) in which at least one part of a vehicle is located during manufacturing. The part of the vehicle is not moved within the cell unless it is reoriented. A cell is not typically part of an assembly line. At least one part of the vehicle is formed and / or assembled within the cell. For example, a structural frame of a heavy-duty truck might be placed in a cell and machined to create features on the frame. These features could be holes, slots, grooves, and so on.

[0029] A cell can include work robots and inspection robots. The heads of the work robots can be equipped with tools for cutting, drilling, coating, and / or performing other tasks to create and / or modify features. The inspection robots can be equipped with cameras mounted on their heads to inspect the created features. The accuracy of the work and inspection robots may be limited. For example, a robot might position the tip of a tool or an inspection camera within ±2 millimeters of a target position. This accuracy can be affected by temperature fluctuations, wear and tear of the robot components over time, and other factors.The greater the distance between the base of a robot and the head of a robot, and / or the more joints and segments (or connections) the robot has, the greater the potential errors in the positioning of the robot head.

[0030] The examples disclosed herein provide an overhead photogrammetry guidance system for improved positioning accuracy of robot heads and objects attached to the robot heads, such as tools and cameras. Tools can include cutting tools such as plasma torches (or plasma cutting tools), welding tools, fastening tools, etc. A corresponding method for designing the overhead photogrammetry guidance system is also disclosed. The overhead photogrammetry guidance system reduces the positioning error of the robot head to less than ±1 mm, enabling the tip of a tool and / or an inspection camera to be positioned with an accuracy of ±1 mm relative to a target position. In one embodiment, the accuracy is such that the positioning error is less than ±0.5 mm.

[0031] Fig. Figures 1-3 show a cell 100 of a manufacturing system 102 with an overhead photogrammetry guidance system 104. The cell 100 refers to a three-dimensional (3D) volume in which at least a part of a product is machined, the product being referred to as the workpiece. In the example shown, the workpiece 106 is a large truck frame. The cell 100 can be used for the production of a product, e.g., a vehicle, or a part thereof. The cell 100 can be one station in a series of stations used to manufacture the product.

[0032] The manufacturing system 102 comprises a stand 110, a worker robot 112, an inspection robot (or measuring robot) 114, rail assemblies 116, and a carriage 118 on which the workpiece 106 is mounted. The workpiece 106 is moved into the cell 100 via the carriage 118 and pushed between the rail assemblies 116 to be centered in the cell 100 as shown. The carriage 118 can move along a linear track 300 in the floor (designated 301), which extends longitudinally and parallel to the rail assemblies 116. The carriage 118 includes a pivot system 120 capable of rotating the workpiece 106 about a longitudinal axis of the workpiece 106. The pivot system 120 may include a motor for rotating the workpiece 106. Car 118 can be equipped with motors to move car 118 relative to rail groups 116, or car 118 can be moved manually relative to rail groups 116.

[0033] The stand 110 comprises legs 121 and crossbars 123, 127, and 129. The legs 121 are attached to feet 131, which are mounted on the floor of the cell. The legs 121 and the crossbars 123, 127, and 129 can be hollow and filled with a damping material such as sand, pellets, and / or other damping material. In one embodiment, the legs 121 and the crossbars 123, 127, and 129 are made of steel, but they can also be made of other materials. In one embodiment, the stand 110 is designed to be heavy and stable on the ground. The feet 131 can be fixed and / or attached to the ground. The stand 110 is designed to prevent the movement of overhead cameras and to minimize and / or prevent the vibration of overhead cameras.In one embodiment, the floor is manufactured as a single concrete slab spanning the entire space beneath the workpiece 106, the robots 112 and 114, the rail assemblies 116, and the stand 110. This minimizes: relative movement between the stability targets 302 mounted on the floor; deformation of the stand 110; and ensures higher overall accuracy of the overhead photogrammetry guidance system 104. In one embodiment, the concrete slab has a minimum thickness of 8 inches (200 mm) and is made of reinforced concrete without any cuts or expansion joints.

[0034] The work robots 112 process the workpiece 106. This can include performing tasks such as cutting, drilling, welding, fastening, and / or other operations. The work robots 112 are equipped with tools (e.g., one of the tools mentioned here) attached to heads 122 of the robots 112. As an example, the work robots 112 are shown with torches 124 mounted on the heads 122 of the robots 112. The inspection robots 114 have cameras 126 attached to the heads 128 of the robots 114. The cameras 126 are used to inspect features created on the workpiece 106 by the work robots 112, such as holes, slots, welds, installed fasteners, and / or other objects, etc. In the example shown, a work robot and an inspection robot are mounted on separate pairs of rails. Each of the rail groups 116 comprises one pair of rails, as shown.Robots 112 and 114 have bases 133 that move longitudinally along rail assemblies 116. Robots 112 and 114 are configured such that their heads 128 have six degrees of freedom (DOF) about their respective axes. Each robot 112 and 114 can have six joints and six connections, with the robot head being connected to the sixth connection. The rail assemblies 116 provide a seventh axis of movement for each head 128.

[0035] The overhead photogrammetry guidance system 104 is used for workpiece alignment and guidance of the robots 112 and 114. The overhead photogrammetry guidance system 104 comprises the overhead cameras, which include top-mounted cameras 136 and end-hanging cameras 138. Top-mounted means that they are connected to an upper crossbar. The overhead cameras can be manually attached to fixed positions or moved by motors represented by boxes 140. The top-mounted cameras 136 can be attached to and move along rails 142 that extend longitudinally along the crossbar 129, which runs from a front end of cell 100 to a rear end of cell 100. The rails 142 run parallel to the crossbar 129 and can be offset relative to it.The cameras 138, suspended at the ends, can be attached to rails 146, which are suspended from the crossbars 127 via vertically extending coupling elements 150 and move along them. The horizontal and vertical positions of the overhead cameras can be adjusted to create lines of sight between the overhead cameras and features such as holes, body mounts, slots, and other features of various dimensions. The crossbars 127 extend laterally across the cell 100, perpendicular to the crossbar 129. The rails 146 run parallel to the crossbars 127.

[0036] The overhead cameras 136, 138 detect the positions of stability targets (or calibration targets) 302 and the photogrammetry targets (or navigation targets) 160. The stability targets 302 can be mounted on the floor of cell 100 and are located at fixed points within cell 100.

[0037] Although six stability targets are shown, any number of stability targets can be used. The number of stability targets used depends on the number of overhead cameras and their fields of view. In one embodiment, at least one stability target is located within the field of view of each overhead camera. In another embodiment, at least two stability targets are located within the field of view of each overhead camera. The photogrammetry targets 160 are attached to the heads 122, 128 of the robots 112, 114. Examples of the targets 302, 160 are shown in Fig. Figures 5-6 illustrate this. The overhead cameras 136 and 138 monitor the movement of the photogrammetry targets 160 and are used to track the positions of the tools (e.g., the torches 124) and cameras (e.g., the cameras 126) mounted on the robots 112 and 114. The cameras 126 are used to detect reference points on the workpiece 106 and to track and inspect features and surfaces of the workpiece 106. The overhead cameras 136 and 138 can also be used to determine the location or position of features and surfaces of the workpiece 106.

[0038] Fig. Figure 4 shows a stand 400 for overhead cameras 402 of an overhead photogrammetry guidance system such as the photogrammetry guidance system 104 of the Fig. 1 - 3. The stand 400 is another example of a stand that replaces the stand 110. Fig. 1-3 can be used. The 402 overhead cameras are similar to the 136 overhead cameras in Fig. 1-3. The stand 400 comprises vertical legs 404, a crossbar 406, and crossbars 407 (one is shown). Angled support legs 408 are connected to the vertical legs 404. Angled support plates 410 are connected to the upper sections of the vertical legs 404 and the crossbars 407. There may also be angled support plates attached to lateral crossbars that are in Fig. 4 are not shown, but are arranged similarly to the angled support plates 410 and attached to laterally extending crossbars, similar to the crossbars 127 of the Fig. 1 and Fig. 3.

[0039] In Fig. Figure 4 shows an example of an overlapping FOV of 420 from two of the overhead cameras 402. The photogrammetry guidance system is configured so that at any given time, at least two of the overhead cameras can see each stability target and each photogrammetry target in the corresponding cell. In other words, each target is seen by at least two of the overhead cameras at any given time during a manufacturing process. The targets can be seen by different overhead cameras. This is done so that the locations of the stability targets and the photogrammetry targets can be determined at any given time. The photogrammetry system can, for example, use triangulation to determine the locations of all stability targets and photogrammetry targets. An example target 422 is shown and can represent a photogrammetry target.Each of the cameras, two of which are designated 430 and 432 for two of the cameras 402, has a field of view angle. An example of a field of view angle 434 for the target 422 is shown and refers to an angle between two lines 436 extending from the centers of the cameras and intersecting at a point on the target 422, the target 422 being in the overlapping FOV 420. In one embodiment, the minimum field of view angle is 35° and the maximum field of view angle is 90°. In one embodiment, the field of view of each of the overhead cameras 402 and other overhead cameras (in ) is . Fig. 4 not shown), such as the hanging overhead cameras 138 in Fig. 1 - 3, 45 - 90°. In one embodiment, the photogrammetry system is configured such that at any given time the distance between each of the overhead cameras 402 and at least one of the targets is 1.5 to 6.0 meters (m).

[0040] Fig. Figure 5 shows a stability target 500 with a housing 502 and transmitters (or artifacts) 504. The housing 502 can have any shape and size and contain any number of transmitters. In one embodiment, the transmitters 504 comprise light-emitting diodes (LEDs). The housing 502 contains two or more transmitters. In one embodiment, the housing 502 has four transmitters (e.g., four LEDs). The transmitters 504 can be arranged in a row or a linear pattern, as shown, or they can be arranged in a non-linear pattern. Although shown in a plane extending parallel to a top surface 506 of the housing 502, the transmitters 504 can be offset from one another so that the transmitters 504 do not lie in a plane extending parallel to the top surface 506. The transmitters 504 can be arranged in various patterns.In one embodiment, each of the transmitters 504 has a ± 60° conically shaped emission pattern that can be seen by several overhead cameras. In another embodiment, the housing 502 is made of carbon fiber and / or another material whose dimensions change only minimally with temperature fluctuations.

[0041] Fig. Figure 6 shows a photogrammetry target 600 with a housing 602 and transmitters (or artifacts) 604. The housing 602 can have any shape and size and contain any number of transmitters. In one embodiment, the transmitters 604 comprise LEDs. The housing 602 contains two or more transmitters. In one embodiment, the housing 602 has four transmitters (e.g., four LEDs). The transmitters 604 can be arranged in a row or in a linear pattern, or in a non-linear pattern. Although shown in two planes extending parallel to the top surfaces 606, 608 of the housing 602, the transmitters 604 can lie in the same plane extending parallel to one top surface of the housing 602, or they can lie in more than two planes extending parallel to more than two top surfaces. The transmitters 604 can be arranged in various patterns.In one embodiment, each of the transmitters 604 has a ± 60° conically shaped emission pattern that can be seen by several overhead cameras. In another embodiment, the housing 602 is made of carbon fiber and / or another material whose dimensions change only minimally with temperature fluctuations.

[0042] In the example shown, the housing 602 comprises a base part 610 and a tapered part 612, which is formed integrally with the base part. Three of the transmitters 604 are mounted on the base part 610 in a first plane extending parallel to first top surfaces 614. Two of the transmitters 604 are mounted on the tapered part 612 in a second plane offset from the first plane and extending parallel to a second top surface 616. The tapered part 612 has sloping sides 618 extending from surfaces 614 to surface 616. Although the housing 602 is depicted in the form of a hat, it can also have a different shape than shown.

[0043] The targets 500, 600 of the Fig. 5 - 6 and the other similar targets of the Fig. 1 - 3 can be done by the corresponding robots (e.g., robots 112, 114 of the Fig. 1 - 3) and / or from one or more control panel modules, such as the one in Fig. The LEDs shown in Figure 7 are powered by energy. They can remain illuminated continuously during the manufacturing process or be switched on and off at specific frequencies and / or have corresponding duty cycles. The one or more system control modules can select the frequencies, duty cycles, intensities, etc., of the target transmitters. For example, the transmitters of the stability targets and the photogrammetry targets can each have a specific color, frequency, duty cycle, intensity, and / or other distinguishing parameters compared to the other stability targets and photogrammetry targets. In one embodiment, the target transmitters have the same color and are continuously switched on during a manufacturing process (e.g., a cutting process to produce one or more features of a workpiece).

[0044] Fig. Figure 7 shows a manufacturing system 700 comprising: an overhead photogrammetry guidance system 701 with a stand 702; a work and inspection robot 704; one or more system control modules 706; a carriage 708 on which a workpiece 710 is mounted; stability targets 712; and photogrammetry targets 714 mounted on the robots 704. The manufacturing system 700 can be configured similarly to the manufacturing system 102 in Figure 7. Fig. 1 - 3. The 702 stand can be configured similarly to one of the 110 and 400 stands in Fig. 1 - 4. The stand 702 can contain overhead cameras 720, camera actuators 722 with motors 724 and one or more reference coordinate frames 726 connected to a first coordinate system of the corresponding overhead photogrammetry guidance system.

[0045] The robots 704 can each comprise articulating motors 730, joints 732, joints 734, a head 735, one or more reference coordinate frames 736 connected to a second coordinate system, one or more tools and / or cameras 738, a base 740, a rail motor 742, and a robot control module 744. The articulating motors 730 are located on the joints 734 and move the joints 732 to move the head 735. The articulating motors 730 are controlled by the robot control module 744. The one or more tools and / or cameras 738 are mounted on the head 735. One of the joints 732 is connected to the base 740, which extends along rails (e.g., rails of one of the rail groups 116). Fig. 1) moves. The rail motor 742 can move the base 740 relative to the rails. Alternatively, a motor separate from one of the robots 704 can be used to, for example, rotate a ball screw in a rail arrangement that includes the rails, in order to move the robot relative to the rails.

[0046] Each of the robots 704 contains one or more photogrammetry targets 714, which may contain transmitters 746. In one embodiment, each of the photogrammetry targets 714 is configured similarly to the photogrammetry target 600 in Fig. 6. Each of the robots 704 can contain any number of photogrammetry targets 714.

[0047] The carriage 708 can include a pivot system 750, a workpiece stand 752, and one or more reference coordinate frames connected to a third coordinate system. The workpiece stand 752 holds the workpiece 710. The pivot system 750 can include a motor 754 for rotating the workpiece 710 on the workpiece stand 752. The carriage 708 can also include one or more translational motors 756 for moving the carriage 708 relative to the stand 702.

[0048] The workpiece 710 can contain datum points 760, features 762, and one or more reference coordinate frames 764 connected to the third coordinate system. The datum points 760 can include holes, slots, and / or other reference points and / or elements of the workpiece 710. The features 762 refer to elements of the workpiece 710 that are being machined, such as holes, slots, fasteners, welds, grooves, channels, etc.

[0049] Each of the stability targets 712 can contain transmitters, e.g., the transmitters 770. In one embodiment, the stability targets 712 are each configured similarly to the stability target 500 in Fig. 5.

[0050] The one or more 706 system control modules can control the 720 overhead cameras, 724 motors, articulated motors, 738 tools and / or cameras, 742 rail motor, 744 robot control module, 746 transmitters, 754 motors, and 770 transmitters. The one or more 706 system control modules coordinate the timing, positioning, locations, on / off times, frequencies, duty cycles, intensities, speeds, etc., of these elements. The positioning and / or locations can be determined relative to the coordinate systems. The locations determined relative to the second and third coordinate systems can be converted to or referenced to the first coordinate system, which can be referred to as the overall cell coordinate system (or principal coordinate system) of the corresponding cell.The reference coordinate frames 726, 736, 755, and 764 can refer to reference points on the stand 702, the robots 704, the carriage 708, and the workpiece 710. These reference coordinate frames can be used to determine the positions of the head 735, the tools and / or cameras 738, and the features 762 and / or the corresponding surfaces of the workpiece 710 relative to the stand 702, the carriage 708, the overhead cameras 720, the stability targets 712, the photogrammetry targets 714, and / or other elements in the cell. One or more of the reference coordinate frames 726, 736, 755, and 764 can be virtual reference coordinate frames. For example, one of the reference coordinate frames 726 can be a virtual reference coordinate frame, and the position of the workpiece 710 and parts thereof can be determined relative to the virtual reference coordinate frame.This can include X, Y, Z, roll, pitch, and yaw values ​​for each reference point of the workpiece. A best-fit method can be used to align the workpiece to the first coordinate system (or that of the entire cell).

[0051] The reference coordinate frames 726, 736, 755, and 764 can be placed anywhere within the cell and on the elements mentioned above. One of the reference coordinate frames 764 can be located on the front face of the workpiece 710. Each of the reference coordinate frames 726, 736, 755, and 764 can have an x-axis that points horizontally from the front to the back of the cell, a y-axis that runs laterally through the cell perpendicular to the x-axis, and a z-axis that runs vertically and perpendicular to both the x- and y-axes.

[0052] A control module is configured to determine the position of the workpiece 710 with six degrees of freedom (DOF) and target locations where holes are to be cut into the workpiece 710, based on data from the inspection robots and the locations of the identifiers.

[0053] In one embodiment, one or more cutting robots (or work robots) have identifiers on or near the end effectors of the cutting robot(s). The control module determines the position(s) of the end effector(s) based on the identifiers. In this way, the position of the end effector(s) can be precisely determined and controlled by the control module to cut the holes in the corresponding workpiece (e.g., the vehicle frame) at the target positions. This ensures flexibility and quality in the production of medium to low batch sizes of workpieces to be processed by the inspection and cutting station.

[0054] The locations of the overhead cameras 720, the stability targets 712, and the photogrammetry targets 714 are flexible. The positions of the photogrammetry targets 714 and the stability targets 712 can be changed to establish lines of sight between reference points 760 and the measuring robot cameras (or inspection cameras) 738, and to establish lines of sight between i) transmitters of the photogrammetry targets 714 and the stability targets 712, and ii) the overhead cameras 720. In one embodiment, the overhead cameras 720 comprise 6 to 8 cameras. The cameras include top-mounted cameras and hanging cameras (or end cameras). Top-mounted means that they are connected to an upper crossbar. The top-mounted cameras can be mounted above, vertically flush with, or below the crossbar.

[0055] Fig. Figure 8 shows a manufacturing process. Although the process is primarily related to the manufacturing system of Fig. As described in section 7, the process can also be applied to the manufacturing system of Fig. Applicable to 1-3.

[0056] At 800, one or more control modules 706 or other control modules design an overhead photogrammetry guidance system (e.g., the overhead photogrammetry guidance system 104 from Fig. 1 - 3 or the 701 overhead photogrammetry guidance system from Fig. 7). In Fig. Figure 9 presents a design procedure that can be implemented as part of Procedure 800. The design procedure includes determining i) the dimensions of an overhead structure for overhead cameras and / or the dimensions of a stand (e.g., stand 110, 702 from Fig. 1 and Fig. 7) and ii) the number and locations of the stability targets. For example, the cell may be 15–20 feet long, 10–15 feet wide, and 10–25 feet high to accommodate a workpiece that is 10–15 feet long, 5–8 feet wide, and 1–3 feet high. The dimensions of the overhead structure, as well as the number and locations of the stability targets, are used to configure the overhead photogrammetry guidance system. The overhead photogrammetry guidance system includes a stand, overhead cameras, and one or more 706 control modules.

[0057] At 802, the one or more control modules 706 calibrate themselves, based on at least one reference coordinate frame and the detected locations of stability targets, to determine the locations of the overhead cameras in the cell. The at least one reference coordinate frame can include the reference coordinate frame 736.

[0058] At 804, the one or more control modules 706 calibrate the work and inspection robots based on at least one reference coordinate frame using the overhead cameras to determine the positions of the robots' photogrammetry targets relative to the overhead cameras. The emission patterns of the targets' transmitters are captured and can be compared with known patterns for the targets to determine the locations and / or orientations of the targets. The at least one reference coordinate frame can include the reference coordinate frame 736.

[0059] At step 806, a workpiece is moved into the cell. This can be done manually or by one or more control modules 706.

[0060] In the 808, one or more control modules (706) align the workpiece if it is not in the correct orientation for first (or next) features and determine the positions of reference points on the workpiece to establish its position relative to one or more coordinate systems, the robots, and the overhead cameras. The coordinate systems can include any of the first, second, and third coordinate systems of the stand, the robots, and the carriage. At the start of a process and after each workpiece realignment, the reference point positions are measured. The reference points are captured by the overhead photogrammetry guidance system to establish an accurate (six degrees of freedom) position of the workpiece relative to the reference coordinate frames. This is referred to as zero-point alignment or reference point alignment, where the workpiece position is mapped to the overhead photogrammetry guidance system.The overhead photogrammetry guidance system determines the offsets or displacement of the robots in order to precisely locate features.

[0061] In the 810 system, one or more control modules guide the tools of one or more robotic workers to one or more surfaces of the workpiece. Before forming and / or machining a feature, the positions of the surfaces are measured. The robotic heads can be roughly moved close to the features to be machined (pre-positions), and then the overhead photogrammetry guidance system can be used to precisely position the working ends or tips of the tools on the workpiece. The exact positioning of the ends or tips can be within ±1 mm or less of the target positions.

[0062] In 812, one or more control modules 706 process the workpiece, including the optional guidance of the tools on the one or more working robots to create one or more features on the workpiece.

[0063] Each pre-position is the position at which the robot stops just before processing the workpiece (e.g., cutting with a torch or measuring with a camera). A pre-position is located close to the actual starting position for the operation (e.g., cutting or measuring). The pre-position is far enough away from the workpiece to prevent any collision. Once the pre-position is established, the one or more 706 control modules determine and / or calculate the position of the tool or camera on the robot head. Based on this information, the one or more 706 control modules determine the X, Y, Z, pitch, yaw, and roll offsets to correct the robot position and improve accuracy. The working position or measuring position, depending on whether the robot is moving a tool or a camera, is the corrected position at which the operation (e.g., cutting or measuring) begins.The working and measuring positions are specific to each feature and frame, determining when cutting or measuring should begin or end. They are based on i) current measurements of reference points on the workpiece and ii) measurements of the feature surfaces (e.g., hole surfaces) to be produced (e.g., cut) with the tool.

[0064] At step 814, one or more control modules (706) determine whether there is another feature to create and / or another surface to process. If so, step 808 is executed; otherwise, step 816 is executed.

[0065] In the 816 system, one or more control modules (706) guide the cameras of one or more inspection robots to the positions of the features and inspect the features using the cameras of the inspection robots. The features are measured. The dimensions and positions of the features are measured.

[0066] At step 818, one or more control modules 706 determine whether there is another characteristic to be checked. If so, step 816 can be carried out; otherwise, the procedure can be terminated.

[0067] Fig. Figure 9 shows a method for designing a photogrammetry guidance system. In Figure 900, the one or more control modules 706 define a workpiece family with the dimensions of the workpieces to be machined within the family. A workpiece family can refer to several different workpieces with varying sizes, shapes, properties, etc. The workpieces in a given family can be of the same type and / or model but have different dimensions and / or a different number of features. In one embodiment, and if the part family is, for example, a truck frame family, the maximum length, width, and height dimensions of the workpieces are defined.

[0068] In 902, one or more control modules 706 determine the feature locations and the workpiece orientations for the feature locations for variants in the workpiece family. This can be based on the dimensions of the workpieces. For example, the positions of the features can refer to the locations on a workpiece where the features are to be arranged. In one embodiment and for a workpiece family that includes truck frames, cutting points and alignment tools for the cutting points are determined for the variants of the truck frames in the family.

[0069] In the 904, the one or more control modules 706 determine the positions of the work robot's tools and the corresponding photogrammetry targets in each pre-position and in each active working tool position. In one embodiment and for a workpiece family that includes vehicle frames, each pre-position and each cutting position of each tool is determined for each workpiece in the workpiece family. This is done for each feature of each workpiece to be machined and is based on the positions of the features. The working positions are determined based on i) current measurements of reference points of the workpiece and ii) measurements of the feature surfaces (e.g., hole surfaces) to be produced (e.g., cut) with the tool.

[0070] In the 906, the one or more control modules 706 determine the positions of the photogrammetry targets on the measuring robots in order to determine the positions of the measuring robot cameras at each pre-position and at each measuring position for reference points, machined and measured features, and measured but not machined features. The pre-positions and measuring positions can be reference points of the workpieces of the workpiece family, machined features, and unmachined features. In one embodiment and for a workpiece family that includes truck frames, each position of the robot cameras (used for inspection) and the associated photogrammetry targets is determined at each pre-position and at each measuring position, including the positions of reference points, cutting features, and features that are only measured.

[0071] In 908, the one or more control modules 706 determine i) the number of photogrammetry cameras and ii) the locations and orientations of the photogrammetry cameras. This is done such that each of the photogrammetry target locations and orientations is visible during the entire manufacturing process of forming and / or machining features for a workpiece of the family in question and is within acceptable fields of view (FOV) of at least two overhead cameras. At least a portion of a photogrammetry target with a predefined pattern of transmitters (e.g., two or more transmitters) is visible to at least two of the overhead cameras at any given time during the manufacturing process. In one embodiment, 2 to 5 transmitters are visible to at least two overhead cameras during the manufacturing process. The number of overhead cameras and their corresponding locations can be based on initial parameters, such as...the dimensions of the workpieces, the positions of the features, the locations of the photogrammetry targets and the locations of the measuring robot cameras.

[0072] At 910, the one or more control module(s) 706 determine the dimensions of the overhead structure and / or stand to hold the overhead cameras at the locations determined based on the information gathered in the operations above. The dimensions of the overhead structure and / or stand can be determined based on secondary parameters such as the locations of the photogrammetry targets, the locations of the measuring robot cameras, the number of overhead cameras, the locations and orientations of the overhead cameras, and the number and locations of the photogrammetry cameras.

[0073] In the 912 system, one or more control modules (706) determine the number and locations of the stability targets. This can be determined so that each photogrammetry camera can see at least two stability targets. The number and locations of the stability targets are determined such that at least some of the stability targets, with a predefined pattern of transmitters, are visible to at least two of the overhead cameras at any given time during the manufacturing process. The number and locations of the stability targets can be determined based on third-party parameters, such as the number and locations of the overhead cameras.

[0074] Following the execution of the procedure of Fig. 9. Some of the [unclear] during the procedure of Fig. 9 parameters determined during the procedure of Fig.8. These parameters include: the tool positions during operation, which can remain the same as long as the workpiece type and model remain the same; the robot camera positions, which remain the same as long as the workpiece type and model remain the same; the geometry of the overhead structure supporting the overhead cameras, which remains the same; the positions and orientations of the photogrammetry cameras, which remain fixed during workpiece manufacturing until the workpiece family changes; and the position of the stability targets (or artifacts), which remains constant as long as the positions of the overhead photogrammetry cameras are not changed.

[0075] The examples described above involve the development and use of an overhead photogrammetry guidance system that enables precise positioning of the robot heads for the corresponding work tools and the inspection cameras mounted on the robot heads. The robot heads or terminals are equipped with tools and inspection cameras to facilitate accurate guidance and positioning. The inspection cameras are used for quality control. Each cell has an overhead photogrammetry structure that supports multiple cameras and provides an image processing system that delivers feedback, enabling the precise placement and movement of the tools and inspection cameras. The examples disclosed here are suitable for frame cutting applications and other uses.

[0076] The examples described above provide an overhead photogrammetry guidance system that includes a minimal number of overhead cameras and targets to accurately determine the position of a workpiece within a cell, as well as the positions of reference points and features on the workpiece within the cell. This minimal number of targets enables precise, real-time positioning of tools and cameras mounted on the robot heads and the control of their movement. This is based on the accurate determination of reference point positions prior to the guided movement. The inclusion of a minimum number of overhead cameras, targets, and inspection cameras allows for high-speed acquisition and mapping of the positions of workpieces and their components within a cell, as well as the inspection of workpiece features. The system utilizes six depth-of-field (DOF) locations within the cell.The tools and cameras mounted on the heads can be moved to target positions based on the dimensions and surface orientation of the workpieces relative to the cell's overall coordinate system. The examples are flexible for different products (or workpieces) by adjusting the locations, orientations, and / or number of overhead cameras. The examples are shown without clamping devices; that is, the workpieces are not held by workpiece-specific fixtures but by a cart that can accommodate various types and models of workpieces.

[0077] The foregoing description is merely explanatory and is not intended to limit the disclosure, its application, or use. The comprehensive teachings of the disclosure can be implemented in a multitude of forms. Although this disclosure contains certain examples, the true scope of the disclosure should therefore not be so limited, since other modifications are evident upon study of the drawings, the description, and the following claims. It is understood that one or more steps within a process may be carried out in a different order (or simultaneously) without altering the principles of the present disclosure.Although each of the embodiments above is described with specific features, any one or more of these features described in relation to any embodiment of the disclosure can be implemented in one of the other embodiments and / or combined with features of another embodiment, even if this combination is not explicitly described. In other words, the described embodiments are not mutually exclusive, and permutations of one or more embodiments with each other remain within the scope of this disclosure.

[0078] Spatial and functional relationships between elements (e.g., between modules, circuit elements, semiconductor layers, etc.) are described using various terms, such as "connected," "interlocking," "coupled," "adjacent," "next to," "on," "above," "below," and "arranged." If a relationship between a first and a second element is not explicitly described as "direct" in the above disclosure, this relationship may be a direct relationship in which no other intervening elements exist between the first and the second element, or it may be an indirect relationship in which one or more intervening elements (either spatial or functional) exist between the first and the second element.As used herein, the phrase “at least one of A, B and C” should be interpreted as a logical (A OR B OR C) using a non-exclusive logical OR and not as “at least one of A, at least one of B and at least one of C”.

[0079] In the diagrams, the direction of an arrow, as indicated by the arrowhead, generally shows the flow of information (e.g., data or instructions) that is relevant to the illustration. For example, if Element A and Element B exchange a variety of information, but the information transferred from Element A to Element B is relevant to the illustration, the arrow may point from Element A to Element B. This unidirectional arrow does not imply that no further information is transferred from Element B to Element A. Furthermore, Element B may send requests for or acknowledgments of information to Element A in return for information sent from Element A to Element B.

[0080] In this application, including the definitions below, the term "module" or the term "controller" may be replaced by the term "circuit". The term "module" may refer to, be part of, or include: an application-specific integrated circuit (ASIC); a digital, analog, or mixed analog / digital discrete circuit; a digital, analog, or mixed analog / digital integrated circuit; a combinational logic circuit; a field-programmable gate array (FPGA); a processor circuit (common, dedicated, or group) that executes code; a memory circuit (common, dedicated, or group) that stores the code executed by the processor circuit; other suitable hardware components that provide the described functionality; or a combination of some or all of the above, e.g., in a system-on-a-chip.

[0081] The module may contain one or more interface circuits. In some examples, the interface circuits may include wired or wireless interfaces connected to a local area network (LAN), the internet, a wide area network (WAN), or combinations thereof. The functionality of any module of this disclosure may be distributed across multiple modules connected via interface circuits. For example, multiple modules may enable load balancing. In another example, a server module (also called a remote or cloud module) may perform some functions on behalf of a client module.

[0082] The term "code," as used above, can include software, firmware, and / or microcode, and can refer to programs, routines, functions, classes, data structures, and / or objects. The term "shared processor circuit" refers to a single processor circuit that executes some or all of the code of multiple modules. The term "group processor circuit" refers to a processor circuit that, in combination with other processor circuits, executes some or all of the code of one or more modules. References to "multiple processor circuits" include multiple processor circuits on discrete chips, multiple processor circuits on a single chip, multiple cores of a single processor circuit, multiple threads of a single processor circuit, or a combination of the above.The term "shared memory circuit" refers to a single memory circuit that stores some or all of the code from multiple modules. The term "group memory circuit" refers to a memory circuit that, in combination with other memory devices, stores some or all of the code from one or more modules.

[0083] The term "memory circuit" is a subset of the term "computer-readable medium." The term "computer-readable medium," as used here, does not include transitory electrical or electromagnetic signals that propagate through a medium (e.g., on a carrier wave); the term "computer-readable medium" can therefore be considered tangible / material and non-transient. Non-restrictive examples of a non-transient, tangible, computer-readable medium are non-volatile memory circuits (e.g., a flash memory circuit, a erasable programmable read-only memory circuit, or a mask read-only memory circuit), volatile memory circuits (e.g., a static random-access memory circuit or a dynamic random-access memory circuit), magnetic storage media (e.g., an analog or digital magnetic tape or a hard disk drive), and optical storage media (e.g.,a CD, a DVD or a Blu-ray Disc).

[0084] The devices and methods described in this application can be implemented partially or completely by a specialized computer formed by configuring a general-purpose computer to perform one or more specific functions embodied in computer programs. The functional blocks, flowchart components, and other elements described above serve as software specifications that can be translated into computer programs through the routine work of an experienced technician or programmer.

[0085] The computer programs contain processor-executable instructions stored on at least one non-transient, tangible, machine-readable medium. The computer programs may also contain or access stored data. The computer programs may include a basic input / output system (BIOS) that interacts with the hardware of the specialized computer, device drivers that interact with specific devices of the specialized computer, one or more operating systems, user applications, background services, background applications, etc.

[0086] The computer programs can contain: (i) descriptive text to be parsed, e.g., HTML (Hypertext Markup Language), XML (Extensible Markup Language), or JSON (JavaScript Object Notation); (ii) assembly code; (iii) object code generated from the source code by a compiler; (iv) source code for execution by an interpreter; (v) source code for compilation and execution by a just-in-time compiler, etc. The source code can, for example, use the syntax of languages ​​such as C, C++, C#, Objective-C, Swift, Haskell, Go, SQL, R, Lisp, Java®, Fortran, Perl, Pascal, Curl, OCaml, Javascript®, HTML5 (Hypertext Markup Language 5th revision), Ada, ASP (Active Server Pages), PHP

[0087] (PHP: Hypertext Preprocessor), Scala, Eiffel, Smalltalk, Erlang, Ruby, Flash®, Visual Basic®, Lua, MATLAB, SIMULINK and Python®. Figure caption for Fig. 8: Yes No

Claims

[1] Manufacturing system comprising: a variety of stability targets; a group of rails; a plurality of robots comprising a plurality of photogrammetry targets each mounted on heads of the plurality of robots, the plurality of robots being configured to move along the set of rails, and each of the plurality of robots comprising a plasma cutting tool and a camera mounted on a respective one of the heads; and a photogrammetry system that includes: a stand, a plurality of overhead cameras mounted on the stand above the plurality of robots and a workpiece to be machined by at least one of the plurality of robots, and at least one control module configured to: Determining the locations of the plurality of stability targets and the locations of the plurality of photogrammetry targets using the plurality of overhead cameras, Determining the locations of the plurality of overhead cameras and the locations of the heads of the plurality of robots based on the locations of the plurality of stability targets and the locations of the plurality of photogrammetry targets, and Controlling the plurality of robots to generate and inspect at least one feature on the workpiece based on the locations of the plurality of overhead cameras and the locations of the heads of the plurality of robots. [2] The manufacturing system of claim 1, wherein the plurality of robots comprises: at least one work robot configured to form or machine the at least one feature; and at least one measuring robot configured to inspect the at least one feature. [3] The manufacturing system of claim 1, wherein each of the plurality of stability targets is within the field of view of at least two of the plurality of overhead cameras. [4] The manufacturing system of claim 1, wherein each of the plurality of photogrammetry targets is within the field of view of at least two of the plurality of overhead cameras. [5] The manufacturing system of claim 1, wherein at least two transmitters from each of the plurality of stability targets are located within the field of view of at least two of the plurality of overhead cameras. [6] Manufacturing system according to claim 1, wherein: Transmitters of at least one of the plurality of photogrammetry targets are located in the field of view of at least two of the plurality of overhead cameras; and / or at least two transmitters from each of the plurality of photogrammetry targets are within the field of view of at least two of the plurality of overhead cameras. [7] The manufacturing system of claim 1, wherein each feature of the workpiece to be machined or inspected is within a predetermined range of two of the plurality of overhead cameras. [8] The manufacturing system of claim 1, wherein an intersection angle of each of the plurality of stability targets and each of the plurality of photogrammetry targets with respect to at least two of the plurality of overhead cameras is within a predetermined range. [9] The manufacturing system of claim 1, wherein the at least one control module is configured to determine the locations of the plurality of stability targets and the locations of the plurality of photogrammetry targets based on one or more reference coordinate frames. [10] The manufacturing system of claim 1, wherein the at least one control module is configured to determine the locations of the plurality of overhead cameras and the locations of the heads of the plurality of robots based on one or more reference coordinate frames.

Images