Systems, methods, and apparatus for arc welding processes and quality monitoring

Abstract

Systems, methods, and apparatuses of a welding system are disclosed and include a first stage of a scanning apparatus to scan a set of welding components to generate a three-dimensional (3D) profile of a welding target, wherein the 3D profile captures a matching defect of the set of welding components' butt joint when performing a welding operation; and a second stage of a monitoring apparatus to monitor the welding operation and generate high-resolution measured data of the welding operation; wherein the first stage further includes the monitoring apparatus to determine a welding plan based on the 3D profile and adjust the welding plan to accommodate a predicted deformation based on the 3D profile and a sensed deformation as the welding operation proceeds; wherein the second stage further includes a plurality of sensors to sense a set of components associated with the welding operation to generate high-resolution measured data.

Description

TECHNICAL FIELD

[0001] The technical field is generally related to welding processes, and more particularly to systems, methods, apparatuses for monitoring an arc welding (AW) process and predicting welding quality. BACKGROUND

[0002] Automotive factories are becoming increasingly common with automated collaborative processes, and there is a need to integrate more advanced manufacturing processes to ensure the quality of the products produced by the factory. To meet these goals, robotic welding is becoming a viable automated tool that can be used in the assembly process in the automotive industry, but there are still problems such as imperfect welds in the production line.

[0003] There is a need for better data analysis of the welding process to detect irregularities in the welded parts, such as incorrect welding position, weak welds, etc., to improve the overall quality and efficiency of the welded production products. For example, in-vehicle assembly, welding errors can delay vehicle production and increase production costs.

[0004] It is desirable to provide a more advanced analysis during the welding process to overcome the inefficiencies present in the welding process in the current production line by providing a multi-stage welding process consisting of multiple stages in welding, for use in a production line that includes stages involving part positioning, welding monitoring, process monitoring, and post-weld inspection to improve welding quality.

[0005] Further features and characteristics of the present application, as well as the structure of, and advantages of using the same, will become apparent from the subsequent detailed description and the appended claims, taken in conjunction with the accompanying drawings, and the foregoing technical field and background. SUMMARY

[0006] A system, method, and apparatus for a welding process to ensure welding quality are disclosed.

[0007] In at least one example embodiment, a welding system is provided.

[0008] The welding system includes at least a first stage of a scanning device for scanning a workpiece comprising a set of welding components to generate a three-dimensional (3D) profile of a welding target, wherein the 3D profile captures a matching defect caused by an abutment of the set of welding components when performing a welding operation for joining the set of welding components; and a second stage of a monitoring device for monitoring the welding operation and generating high resolution measured data of the welding operation; wherein the first stage further includes the monitoring device determining a welding plan based on the 3D profile of the welding target, and adjusting the welding plan while the welding operation is in progress to accommodate a predicted deformation of a shape of the welding based on the 3D profile of the target welding, and to accommodate a sensed deformation caused by a combination of forces caused by a welding between the set of welding components and a clamping force exerted on the workpiece during the welding operation; wherein the second stage further includes a plurality of sensors configured to sense a set of components associated with the welding operation to generate high resolution measured data from direct and indirect sensing of measurements of the set of components associated with the welding operation compared to low resolution measured data provided by a welding controller.

[0009] In at least one example embodiment, the plurality of sensors in the second stage providing high resolution measured data includes at least one or more of a set of sensors to provide direct measurements of the welding operation of a high resolution current sensor, a high resolution voltage monitoring sensor, and a high resolution flow sensor.

[0010] In at least one example embodiment, the plurality of sensors in the second stage providing high resolution measured data includes at least one or more of a set of sensors to provide indirect measurements of the welding operation of a microphone, a vibrometer, a plasma sensor, an ultraviolet sensor, a strain gauge sensor, a welding reaction force sensor, an electromagnetic spectrometer, a wire feed sensor, and an infrared camera.

[0011] In at least one example embodiment, the welding system in the first stage further includes a welding wire sensor capable of determining a welding component position prior to a welding operation by a touch action that impinges on at least one welding component of the set of welding components, wherein the welding component is subjected to a clamping force that holds each welding component of the set of welding components together, and measures a change in shape of the welding component position by touch to determine a deformation of a shape of the welding component; and wherein the second stage further includes the sensor including the welding wire, wherein the welding wire is configured as a consumable electrode that advances to the welding component to deposit a weld bead forming a weld segment of the weld bead between the set of welding components, and is further configured as a wire feed sensor to provide a measured melting rate of the consumable electrode in the welding operation.

[0012] In at least one example embodiment, the welding system includes a microphone configured to monitor ultrasonic frequencies and audible range frequencies of noise associated with the welding operation to determine whether the weld segment is up to par; and a strain gauge sensor configured to measure a set of measurements exhibited by the weldment during the welding operation to determine a warpage of the weldment and whether the warpage of the weldment exceeds a compliance level at which the strain gauge sensor is attached to the weldment.

[0013] In at least one example embodiment, the welding system includes a weld reaction force sensor configured to determine a strength of the weld segment in opposition to a clamping force applied to the weldment.

[0014] In at least one example embodiment, the welding system includes a third stage of the welding system including a process monitoring module configured to qualify the weld segment formed by the welding operation based on a welding monitoring rule applied to a result of a calculation of a function using the measured weldment gap plan.

[0015] In at least one example embodiment, the welding system includes the monitoring module configured to combine high resolution data and low resolution data from the plurality of sensors and the welding controller to determine a trajectory of a robotic device implemented in the welding operation.

[0016] In at least one example embodiment, the welding system includes a fourth stage of the welding system including a post-weld inspection module to perform an automated inspection based on a plan of adjustments of the weld segment and the set of weldments and by fusing together high resolution data and low resolution data for quality analysis to determine whether the weld joint is stable and whether the weld segment is up to par, wherein the quality analysis uses a rule-based schema and a classification algorithm that receives input of the fused high resolution data and low resolution data to qualify and classify the weld joint.

[0017] In at least one example embodiment, a method for monitoring a welding operation is provided.

[0018] The method comprises at least: configuring a welding system by a first stage of a scanning device for scanning a workpiece comprising a set of welding components to generate a three-dimensional (3D) profile of a welding target, wherein the 3D profile captures a matching defect caused by an abutment of the set of welding components when performing a welding operation for joining the set of welding components; and configuring the welding system by a second stage of a monitoring device for monitoring the welding operation and for generating high-resolution measured data of the welding operation; configuring the monitoring device by the first stage for determining a welding plan based on the 3D profile of the welding target, and for adjusting the welding plan while the welding operation is in progress for adapting a predicted deformation of a shape of the welding based on the 3D profile of the welding target, and for adapting a sensed deformation caused by a combination of forces caused by the welding between the set of welding components and a clamping force applied on the workpiece during the welding operation; and configuring a plurality of sensors in the second stage for sensing a set of components associated with the welding operation to generate high-resolution measured data from direct and indirect sensing of measurements of the set of components associated with the welding operation compared to low-resolution measured data provided by a welding controller.

[0019] In at least one example embodiment, the method comprises configuring the plurality of sensors in the second stage for providing the high-resolution measured data by at least one or more of a set of sensors providing direct measurements of the welding operation, the set of sensors comprising a high-resolution current sensor, a high-resolution voltage monitoring sensor, and a high-resolution flow sensor.

[0020] In at least one example embodiment, the method comprises configuring the plurality of sensors in the second stage for providing the high-resolution measured data by at least one or more of the plurality of sensors providing indirect measurements of the welding operation, the plurality of sensors comprising a microphone, a vibrometer, a plasma sensor, an ultraviolet sensor, a strain gauge sensor, a welding reaction force sensor, an electromagnetic spectrometer, a wire feed sensor, and an infrared camera.

[0021] In at least one example embodiment, the method includes configuring a first stage of sensors including a wire for implementing a strike to at least one of the set of weld parts by a contact action for determining a weld part position prior to a welding operation, wherein the weld parts are subjected to a clamping force that holds each of the set of weld parts together, and for measuring a change in shape of the weld part position by a tactile to determine a deformation of a shape of the weld part; and configuring by a second stage of sensors including a welding wire, wherein the welding wire is configured as a consumable electrode that advances to the weld parts to deposit a weld bead to form a weld segment between the set of weld parts, and further configured as a wire feed sensor for providing a measured melt rate of the consumable electrode in the welding operation.

[0022] In at least one example embodiment, the method includes configuring a microphone for monitoring ultrasonic frequencies and audible range frequencies of noise associated with a welding operation to determine whether a weld segment is in compliance; and configuring the strain gauge sensor for measuring a set of measurements presented by the weld parts during the welding operation to determine a warp of the weld parts and whether the warp of the weld parts exceeds a compliance level, wherein the strain gauge sensor is attached to the weld parts.

[0023] In at least one example embodiment, the method includes configuring a third stage of a welding system including a process monitoring module configured for qualifying a weld segment formed by a welding operation based on a welding monitoring rule applied to a calculated result of a function using measured weld part gap plans.

[0024] In at least one example embodiment, the method includes configuring the process monitoring module to combine high resolution data and low resolution data from the plurality of sensors and the welding controller to determine a trajectory of a robotic device implemented in the welding operation.

[0025] In at least one example embodiment, the method includes configuring a fourth stage of a welding system including a post weld inspection module to perform an automated inspection based on a weld segment and adjusted weld plans of the set of weld parts, and to fuse high resolution data and low resolution data together for a quality analysis to determine whether a weld joint is stable. And the weld segment is in compliance, wherein the quality analysis uses a rule-based pattern and a classification algorithm that receives an input of the fused high resolution data and low resolution data to qualify and classify the weld joint.

[0026] In at least one example embodiment, a welding device is provided.

[0027] The welding apparatus comprises at least a welding unit configured to, in a first stage of a scanning apparatus, scan a workpiece comprising a set of welding components to generate a three-dimensional (3D) profile of a welding target, wherein the 3D profile captures a matching defect caused by an abutment of the set of welding components when performing a welding operation for joining the set of welding components; and in a second stage comprising a monitoring apparatus, monitor the welding operation and generate high-resolution measured data of the welding operation; wherein the first stage further comprises the monitoring apparatus determining a welding plan based on the 3D profile of the welding target and adjusting the welding plan while the welding operation is in progress to accommodate a predicted deformation of a shape of the welding based on the 3D profile of the target welding and a sensed deformation caused by a combination of forces caused by the welding between the set of welding components and a clamping force exerted on the workpiece during the welding operation; wherein the second stage further comprises a plurality of sensors configured to sense a set of components associated with the welding operation to generate high-resolution measured data from direct and indirect sensing of measurements of the set of components associated with the welding operation compared to low-resolution measured data provided by a welding controller.

[0028] In at least one example embodiment, the welding apparatus comprises a welding unit further configured to, in a second stage, measure, by a plurality of sensors, high-resolution measured data by at least one or more high-resolution sensors of a set of high-resolution sensors comprising a high-resolution current sensor, a high-resolution voltage monitor, and a high-resolution flow sensor to provide direct measurements of a welding operation; and in the second stage, measure, by the plurality of sensors, the high-resolution measured data of at least one or more of the plurality of sensors to provide indirect measurements of the welding operation comprising a microphone, a vibrometer, a plasma sensor, an ultraviolet sensor, a strain gauge sensor, a welding reaction force sensor, an electromagnetic spectrometer, a wire feed sensor, and an infrared camera.

[0029] In at least one example embodiment, the welding device includes wherein the welding unit is further configured to, in the first stage, configure the sensor including the welding wire to enable determination of a weldment part position of the weldment part prior to the welding operation by a contact action of the welding wire impinging on at least one weldment part of the set of weldment parts, wherein the weldment part is subjected to a clamping force that holds each weldment part of the set of weldment parts together, and by a tactile measurement of a change in shape of the weldment part position to determine a deformation of a shape of the weldment part; and in the second stage, configure the sensor including the welding wire as a consumable electrode that advances to the weldment part to deposit a weld bead to form the weld segment of the weld bead between the set of weldment parts, and the sensor is further configured as a wire feed sensor to provide a measured melt rate of the consumable electrode in the welding operation. BRIEF DESCRIPTION OF DRAWINGS

[0030] In the following exemplary embodiments will be described with reference to the following drawings, wherein like reference numerals refer to like elements, and wherein:

[0031] Figure 1 An exemplary diagram of a welding system for monitoring a welding operation, processing data from multiple sensors in each stage of a welding process, and determining welding eligibility according to exemplary embodiments is shown;

[0032] Figure 2 Multiple stages of a welding process and multiple sets of sensors in each stage that generate high resolution data and low resolution data about a welding operation according to exemplary embodiments are shown;

[0033] Figure 3 An exemplary diagram of receiving data from multiple sensors with a broad set of sensing modalities that are used in series to assess welding quality of a welding system according to exemplary embodiments is illustrated; and

[0034] Figure 4 An exemplary flowchart of a welding operation of a welding system 100 according to exemplary embodiments is shown. DETAILED DESCRIPTION

[0035] The following detailed description is merely exemplary in nature and is not intended to limit the application and use. Furthermore, there is no intention to be bound by any expressed or implied theory presented in the preceding technical field, background, brief summary or the following detailed description.

[0036] Embodiments of the disclosure can be described herein in terms of functional and / or logical block components and various processing steps. It should be appreciated that such block components can be realized by any number of hardware, software, and / or firmware components configured to perform the specified functions. For example, an embodiment of the disclosure can employ various integrated circuit components, e.g., memory elements, digital signal processing elements, logic elements, look-up tables, or the like, which can carry out a variety of functions under the control of one or more microprocessors or other control devices. In addition, those skilled in the art will appreciate that embodiments of the disclosure can be practiced with any number of systems including systems not specifically depicted herein, including any number of systems that can integrate with other systems.

[0037] The present disclosure describes systems, methods, and apparatuses that implement an adaptive welding process that implements a monitoring system with more than one set of possible threshold limits (or quality determination rules) that can be applied to a welding operation based in part on a selected set of welding plans or welding conditions.

[0038] The present disclosure describes systems, methods, and apparatuses that implement an adaptive welding planning process for a welding operation by implementing a scanning operation of a workpiece to generate a three-dimensional (3D) profile of a weld target (i.e., a joint where two parts to be welded or welded parts are brought together and joined, bonded, mated together, etc.) and generating a 3D weld profile for a target weld based on an appropriate welding plan based on the geometry of the weld parts. In an embodiment, based on the 3D weld profile, a monitoring system is implemented to monitor or predict deformations expected during the welding operation in real-time as the welding is performed and adjustments are made to the welding plan as the welding is performed to prevent or optimize the welding operation (i.e., minimize anomalies in the weld) to attempt to correct any defects that are deemed likely to occur.

[0039] Figure 1 An exemplary diagram of a welding system 100 is shown in accordance with an embodiment for monitoring a welding operation (for GMAW or other types of welding systems, including arc brazing, laser brazing, hybrid laser-arc welding, etc.), processing data from multiple sensors in each stage of the welding process, and determining weld eligibility. In Figure 1 In particular, the welding system 100 includes an edge computer system 5 configured with interfaces to process data from multiple sensors in first, second, third, and fourth stages of a welding process (e.g., a GMAW welding process) and determine weld eligibility. Figure 2The edge computer system 5 receives input from sensors in the welding environment (described below). The edge computer system 5 can be hosted locally at the server 15 or remotely in the cloud. In the depicted embodiment, the edge computer system 5 can be connected to the server 15, the signal repository and database 10, and the display 20 as well as the user’s mobile device 30. The edge computer system 5 can be a self-configuring processor system in communication with the server 15 and include elements of a communication gateway controller, a block data processor communicatively coupled to internal or external memory, internal storage, an inter-processor bus, and optional storage disks. In various embodiments, the edge computer system 5 performs the actions and other functions related to the welding operations further described below. The block data processor performs computational and control functions attributed to algorithms programmed for the edge computer system 5, which can include various types of modules or multiple modules, a single integrated circuit such as a micro-module, or any suitable number of integrated circuit devices and / or circuit boards working in cooperation to perform the operations, tasks, and functions described by manipulating electrical signals representing data bits at memory locations in system memory and other processing of signals. Figures 2-4 The block data processor performs computational and control functions attributed to algorithms programmed for the edge computer system 5, which can include various types of modules or multiple modules, a single integrated circuit such as a micro-module, or any suitable number of integrated circuit devices and / or circuit boards working in cooperation to perform the operations, tasks, and functions described by manipulating electrical signals representing data bits at memory locations in system memory and other processing of signals.

[0040] During operation, the block data processor loads and executes one or more programs, algorithms, and rules embodied as instructions and applications contained within the internal memory of the edge computer system 5 (i.e., machine learning algorithms) and thus controls the general operation of the control system of the communication gateway controller to perform a plurality of functions in each stage of the welding process. In performing the processes described herein, the block data processor loads and executes at least one program.

[0041] For example, in an embodiment, a machine learning (ML) algorithm can be executed by the edge computer system 5 to monitor, make position determinations, and perform other quality-oriented processing steps in one or more stages of the welding operation. In embodiments, the ML algorithm, when executed, can receive inputs of low resolution data and high resolution data from a plurality of sensors. The ML algorithm can be configured to automatically receive quality inputs from three-dimensional laser scan measurements of completed welds to automatically assess weld quality (e.g., weld dislocation, excessive weld expulsion). Implementation of such ongoing processes and training in combination with different outputs from a plurality of sensed modalities (i.e., acoustic, electromagnetic visual force, and emission) can provide an assessment of overall weld quality. For example, data from different sensed modalities can be correlated by the ML algorithm or other applications of the edge computer system 5 with weld quality sensors to detect excessive expulsion based on feature analysis and sound amplitude levels.

[0042] In one embodiment, the edge computer system 5 is programmed with an intelligent (functional) algorithm to implement welding monitoring rules based on a set of constraints configured for the welding operation, which are derived from the algorithm based on the identified weld segments, the measured gap widths of a set of welded components, and the welding plan.

[0043] In one embodiment, the edge computing system 5 is programmed to communicate with an interface to receive sensor data scanned by a 3D camera and can implement functional algorithms to measure the deformation and strain of the welded component in real time during the welding operation. The measurements performed can provide a basis for determining or estimating the deformation and residual stress that may be caused by the welding operation and clamping forces.

[0044] In one embodiment, the edge computer system 5 is programmed to fuse or combine datasets of high-resolution and low-resolution data sensed during the welding operation and from the welding controller to determine the robot trajectory.

[0045] In one embodiment, the edge computing system 5 is programmed to combine pre-inspection, during-inspection, and post-inspection data from multiple sensors as input to a classification algorithm that can use rule-based decision processes or Bayesian estimation to determine quality, defining the probability of having a pass or fail weld.

[0046] In one embodiment, the edge computer system 5 is programmed to receive data from sensors, including welding wires configured in multiple modes, to sense the position of the weld joint and estimate the size of the gap between the substrates, and to combine this information with a classification algorithm to evaluate the quality of the weld, predict weld planning, and make other relevant weld operation decisions.

[0047] In one embodiment, the edge computing system 5 is programmed to implement the scanning device ( Figure 2 The scanning operation is performed by a scanning device 225, which may be a laser line scanner, and is instructed to generate a three-dimensional (3D) profile of the welding target using an algorithm executed by the edge computer system 5. The welding target is a seam formed by placing two parts or weld components together and joining them based on their geometry. In the case of joining, defects may exist due to inaccurate geometric matching between the two weld components, which may result in gaps or gaps between them. The edge computer system can generate a 3D welding profile using a software application's scanning operation. This 3D welding profile identifies gaps and defects in the target weld when joining the two weld components, and an adaptive welding plan can be implemented in the welding planning application to compensate for and adjust for defects collected by the 3D welding profile.

[0048] The edge computer system 5 is configured with computer-readable storage media, such as memory, storage, or optional storage disk, which can serve as both storage and scratchpad. A memory location holding data bits is a physical location having specific electrical, magnetic, optical, or organic properties corresponding to the data bits. The memory can be any type of suitable computer-readable storage media. For example, the memory can include various types of dynamic random access memory (DRAM) such as SDRAM, various types of static RAM (SRAM), and various types of non-volatile memory (PROM, EPROM, and flash memory). In certain examples, the memory is located and / or co-located on the same computer chip as the block data processor. In the depicted embodiment, the memory stores the above-mentioned instructions and applications along with one or more configurable variables in stored values.

[0049] The signal repository and database 10 is a computer-readable storage medium in the form of any suitable type of storage, including direct access storage devices such as hard drives, flash memory systems, floppy drives, and optical drives. In one example embodiment, the signal repository and database 10 can include a program product from which the memory of the edge computer system 5 can also receive programs and execute one or more embodiments of one or more processes of the present disclosure.

[0050] In another example embodiment, the program product can be directly stored in and / or otherwise accessed by the memory, disk, and / or disk (e.g., optional storage disk) of the server 15, such as the disk referenced below.

[0051] The data records can be stored in a computer-readable storage medium such as memory. An internal bus communicates to transfer programs, data, status, and other information or signals between the various components of the welding system 100. The bus can be any suitable physical or logical means of connecting computer systems and components. This includes, but is not limited to, direct hard-wired connections, fiber optics, infrared, and wireless bus technology. During operation, programs stored in the memory or the signal repository and database 10 are loaded and executed by the block data processor of the edge computer system 5.

[0052] The interface (not shown) can also include one or more network interfaces to allow communication with external mobile devices and / or manufacturing systems to allow communication and possible storage of status information that can ultimately be placed into a storage device such as the signal repository and database 10.

[0053] The display 20 is configured to display a graphical user interface of the analysis of each stage of the multi-stage welding system and data generated by the plurality of sensors of the low resolution data and the high resolution data in the corresponding stage for a user to view real-time visuals of the welding operation in the factory.

[0054] Figure 2 The plurality of stages of the welding process and the plurality of sets of sensors in each stage (which are used to generate high resolution data and low resolution data about the welding operation) are illustrated in accordance with exemplary embodiments. In Figure 2 In the exemplary diagram, a first stage 205 for positioning the welding components and the gap between the set of welding components for applying the weld is depicted. The first stage 205 includes a camera or scanning device 225 to determine the welding position and orientation gap between the substrates or welding components.

[0055] In an embodiment, the scanning device 225 is a laser line scanner that can be implemented to generate a three-dimensional (3D) profile of the weld target (i.e., the joint where the two pieces or welding component pair to be welded are mated, and put together and joined, bonded, etc.) based on the geometry of the welding components (via an algorithm executed by the edge computer system 5). In some instances, the mating of the two parts is not perfect due to the geometry of each welding part. There can be slight deviations resulting in an approximate match, bond, or abutment between the two welding components. The resulting abutment can result in an imperfect abutment where there is a material or slight spacing or gap between the two welding parts. By implementing a scanning operation by the scanning device 225 and generating a 3D weld profile that includes the determined spacing or gap between the two welding components, the edge computer system 5 via the weld planning application can determine an appropriate weld plan based on the 3D weld profile that compensates or adjusts for the imperfections by the gap spacing in the positioning and abutment of the two parts of the workpiece.

[0056] In various exemplary embodiments, the edge computer system 5 can implement a set of predetermined weld plans based on historical data that can be contained in the signal repository and database 10 or server 15. In an embodiment, based on the 3D weld profile, the edge computer system 5 can monitor, predict, or correlate expected deformations or distortions in the welding operation in real-time as the welding is being performed and can adjust the weld plan to prevent or optimize the welding operation as the welding is being performed to attempt to correct any imperfections that are believed to be likely to occur.

[0057] In an embodiment, the algorithm implemented by the edge computer system 5 for monitoring the welding process also responds to changes in the welding plan in response to changes in the welding plan. For example, depending on the changes in the welding plan, it can also be necessary to modify the monitoring algorithm accordingly to adjust the different data collected for the welding operation and received as a result of the plan changes. For example, depending on the implemented welding plan, the monitoring algorithm can have to change. That is, if the welding plan changes, the same algorithm (or even if the algorithm is the same, at least the same parameters) cannot be used to monitor the welding operation.

[0058] In an embodiment, the information used in the previous welding plan can be retained by the edge computer system 5 or by the edge computer system 5 on which a part of the welding can be performed by which plan, and then the application of the edge computer system 5 can apply the appropriate classifier / algorithm to determine whether the welding can be determined as an acceptable welding. In this regard, the robot kinematics capabilities can be used for welding monitoring as well as the sensing measurements of voltage, current, wire feed, gas flow, etc. Moreover, by combining the sensing data and plan adjustments with post-weld inspection to evaluate the final welding quality in the fourth stage 220 during post-weld inspection, a welding progress-based adaptive monitoring algorithm is implemented in a welding operation with progress awareness.

[0059] In an embodiment, the welding plan can refer to various aspects of the welding, including the trajectory of the robot, the speed of the robot (which can vary over the trajectory), the relative orientation of the robot's torch along the trajectory (also referred to as torch angle), and the amount of current and voltage used. The various constituent aspects of the robot can change (potentially) throughout the trajectory, the changes that occur together with the welding operation details are captured by the changes in the welding plan.

[0060] In an embodiment, the scanning device 225 provides various 3D scanning capabilities and is implemented using a line scanner. In an embodiment, the line scanner enables a laser line to be projected from the scanner device 225 on the workpiece (i.e., the welding component). The distance of each point along the line from the scanning device 225 is measured, and some measurements include "height" information along the determined line, which is used to instruct the robot arm movement. For example, if a line in the X direction is determined and measured, the robot arm of the robot is configured to move along the y direction, and in each movement cycle, the robot arm determines the height (i.e., the Z measurement), where the Z position is determined for each X, Y position found. This movement of the robot arm enables the 3D scanning of the welding operation of the workpiece to be performed in a line-by-line operation manner by the line scanner.

[0061] In an embodiment, the sensor 230 is implemented to measure the initial clamping force, and the haptic sensor 235 senses the position of the weld joint via wire modulation and estimates the size of the gap between the weld components (e.g., the gap between the substrates), and uses the sensed information and the position of the weld joint and the estimate of the gap size as inputs to a classification algorithm executed by the edge computer system 5.

[0062] With continued reference to Figure 2 , the second stage 210 provides a plurality of sensors that enable real-time monitoring of the progress of the welding through direct and indirect sensing. The weld controller 240 provides real-time low resolution data of the welding operation, which includes data of the weld controller current, weld controller voltage, gas flow, wire feed, and electrical impedance. To monitor the welding and generate high resolution data, a set of high resolution sensors provide direct measurements of the welding operation and include a high resolution current sensor 245, a high resolution voltage monitoring sensor 250, and a high resolution flow sensor 255. A plurality of sensors are also provided to provide indirect measurements of the welding operation and include a microphone 260, a vibrometer 265, a plasma sensor 270, an ultraviolet sensor 275, a strain gauge sensor 280, a weld reaction force sensor 285, an electromagnetic spectrometer 290, a wire feed sensor 295, and an infrared camera 300.

[0063] The microphone 260 is capable of monitoring the ultrasonic and audible range frequencies of the noise of the welding operation to measure in real-time the welding sound and ultrasonic emissions, where small changes in any of the parameters can indicate a change in the quality of the weld. In an embodiment, a dual microphone array can be provided in a metal inert gas (GMAW) robotic welding process, and the arc sound signatures can be analyzed to obtain a relationship between the arc sound signatures and the deviation from the welding path.

[0064] The strain gauge sensor 280 provides measurements of the strain presented by the weld components during the welding operation and sends the measurement data to the edge computer system 5, which presents an estimate based on algorithmic analysis of the warping of the substrates in operation. From this estimate, it can be determined whether the warping has risen to a level where the components are not considered to be at a desired level of compliance. Furthermore, the strain gauge sensor 280 is attached to the substrates when measuring the strain of the substrates in the welding operation.

[0065] The weld reaction force sensor 285 provides data from which the strength of the weld segment can be estimated by an algorithm executed by the edge computer system 5 that is weighted to the position of the substrates or weld components and the clamping force that is applied to the substrates or weld components to hold them in place during the welding operation.

[0066] For welds performed with adaptive welding processes, the welding system 100 can monitor more than one set of possible threshold limits for the weld (e.g., by applying various quality determination rules) and the implemented threshold or rules also depend on the welding plan that performs the welding operation and the operating conditions at the time of performing the weld. In an embodiment, the welding monitoring rules or limits are a function of the identified weld segment, the measured gap width, and the welding plan. The welding operating conditions and plan are monitored by sensors including the microphone 260, the vibrometer 265, the plasma sensor 270, the ultraviolet sensor 275, the strain gauge sensor 280, the welding reaction force sensor 285, the electromagnetic spectrometer 290, the wire feed sensor 295, and the infrared camera 300.

[0067] In an embodiment, the infrared camera 300 sensor is a 3-D camera that measures deformation and strain of the weld components in real-time during welding. These measurements can be used to estimate the “distortion” and “residual stresses” caused by the weld.

[0068] In an embodiment, the edge computer system 5 utilizes an ML method that utilizes automatic quality inputs from 3D laser scan measurements (i.e., completed welds) to automatically assess the weld quality (e.g., weld dislocation, excessive weld expulsion) by laser scan sensors. The laser scan sensors can be mounted on the welding robot (i.e., robotic arm) close to the welding torch. The scan area directly in front of the welding electrode is pre-calibrated with a provided calibration plate (i.e., the research module and tool 335) that is placed in front of the welding electrode. Figure 3 In an embodiment, the edge computer system 5 utilizes an ML method that utilizes automatic quality inputs from 3D laser scan measurements (i.e., completed welds) to automatically assess the weld quality (e.g., weld dislocation, excessive weld expulsion) by laser scan sensors. The laser scan sensors can be mounted on the welding robot (i.e., robotic arm) close to the welding torch. The scan area directly in front of the welding electrode is pre-calibrated with a provided calibration plate (i.e., the research module and tool 335) that is placed in front of the welding electrode.

[0069] In an embodiment, the different modalities of the high-resolution sensor group of high-resolution current sensors 245, high-resolution voltage monitor sensors 250, and high-resolution flow sensors 255 for providing direct measurement results, and the different modalities of the group of sensors of the microphone 260, the vibrometer 265, the plasma sensor 270, the ultraviolet sensor 275, the strain gauge sensor 280, the welding reaction force sensor 285, the electromagnetic spectrometer 290, the wire feed sensor 295, and the infrared camera 300 for providing direct measurement results are combined for forming a collection of different sensing modalities (acoustic, electromagnetic, visual, force, and emission) from the various steps of the welding process. These signals can be correlated with the outputs from the weld quality sensors 315 in the fourth stage 220 from post-weld inspection to detect excessive expulsion based on feature analysis and sound amplitude levels (i.e., detected sound from the microphone 260). The final component geometry wavelengths are inspected in post-weld inspection by the camera 320 and compared to the data from the wire feed sensor 295 in the second stage 210 and the initial positioning data from the camera or scanning device 225 in the first stage 205.

[0070] In one embodiment, the plasma sensor 270 detects the state of the gas used in the welding operation (i.e., how much plasma is used), while the infrared sensor (i.e., infrared camera 300) is directed at the welding operation and detects the heat emitted by the steps of the welding operation.

[0071] The third level 215 of the welding system 200 includes process monitoring 305 and robot arm monitoring 310. The process monitoring 305 includes ensuring that the welding sequence is correct, ensuring proper parts by identifying and tracking the welding part number and tracking the numbers associated with the steps of the welding operation. Also, the process monitoring 305 includes monitoring the control system operation and operating conditions from the directly and indirectly sensed data of the welding operation. The robot arm monitoring 310 includes monitoring the welding plan, the actual trajectory of the robot arm 310 (i.e., ensuring proper compliance with the data from the first level 205 for generating the part position and gap position), tracking the actual speed of the robot arm 310 as it performs the welding operation, determining the joint torque applied to the welding part, and determining the actual acceleration of the robot arm 310. The robot arm monitoring 310 and the process monitoring 305 are performed by an application program of the programmed software of the edge computer system 5 that includes ML algorithms and receives input sensor data.

[0072] The fourth level 220 of post-weld inspection is directed to determining the quality associated with the aesthetic appeal or appearance and verifying the weld integrity on both sides of the weld joint and other aspects of the welding part. For example, welding discontinuities that can be seen during visual inspection, such as undersized welds, undercut, overlap, surface cracking, surface porosity, root underfill, incomplete root penetration, excessive root penetration, burn-through, and excessive reinforcement. In addition, determinations can be made based on the haptic sensing data of the distortion exhibited by the welding part and an estimate of the quality of the weld joint can be made via the edge computer system 5 without having to perform tests (i.e., non-destructive testing) that can cause defects in the weld joint or part. Further, the post-weld inspection can determine the amount of polishing or smoothing of the weld necessary.

[0073] Figure 3 An exemplary diagram of a network 380 for linking a plurality of sources in communication with a data repository 325 is illustrated, and the data repository 325 receives data from a plurality of sensors having a broad set of sensing modalities used in tandem to assess the welding quality of a welding system, in accordance with an embodiment. The network 380 includes a plurality of sensors 385, a plurality of edge computer systems 390, and a data repository 325. Figure 3In this system, a data storage repository 325 (e.g., receiving data from multiple sources to send to a factory information system 330, i.e., a remote server for factory operations) and a quality analysis 370 that can be performed on a cloud server, for example, aggregate multiple welding operation actions and use sophisticated ML algorithms for quality, planning, process monitoring, and post-inspection determination. In one embodiment, the quality analysis 370 can implement a process to combine pre-inspection, during-inspection, and post-inspection data from multiple sensors (i.e., fusing high-resolution data 345 and low-resolution data 350) as input to a classification algorithm that can identify welds using rule-based patterns of whether a weld is acceptable, or use Bayesian estimation to define the probability that a weld is acceptable or unacceptable based on welding data (i.e., pre-weld gap data 355 and post-weld weld contour data 360, for example). Furthermore, the quality analysis 370 can use data from weld wire modulation as input to a classification algorithm that senses the location of the weld joint and estimates the size of the gap between substrates to classify and characterize the weld for further inspection.

[0074] In this embodiment, the data repository 325 receives and updates data and software modules from multiple sources, including research modules and tools 335 (i.e., for determining robot kinematics, etc.); welding metadata 340 from a programmable logic controller (PLC) for programming the robot; high-resolution data 345 of current, wire feed, gas flow, etc.; low-resolution data 350 of welding controller current, voltage, and wire feed; pre-weld gap data 355 from the PLC network; and post-weld weld bead profile data 360 from the PLC network. Figure 3 As shown, multiple sensors from a wide range of sensing modes are used in series. Figure 2 The data is integrated to assess the stability of the welding process and to determine aspects of weld quality, overall weld quality, and optimize the scheduling and sequencing of welding operations in the plant by utilizing assessments at each stage of the welding process enhanced by sensors and process inputs.

[0075] Figure 4 An exemplary flowchart of welding operation 400 of welding system 100 according to an exemplary embodiment is shown. Figure 4 Including step 405 for defining weld configuration, and step 1 for achieving level 1 ( Figure 2 The pre-welding inspection is used for positioning of the welded components and gap positions, and for defining the welding schedule at step 415.

[0076] In one embodiment, for use in the first level ( Figure 2The step 410 of pre-weld inspection implemented in the first stage (400) can include a scanning operation performed by the scanning device 225, which is a laser line scanner implemented to generate a three-dimensional (3D) profile of the weld target (i.e., the joint where the two workpieces or weld components are brought together and joined, bonded, mated, etc. to be welded) based on the geometry of the weld components (via an algorithm executed by the edge computer system 5); for example, there can be an approximate match, bond, or mate between the two weld components, or there can be an inexact or imperfect bond or mate that can result in a gap or clearance between the two weld components. By implementing the scanning operation and generating the 3D weld profile that includes the determined gap or clearance between the two weld components, the edge computer system 5 via the weld planning application can determine an appropriate weld plan based on the 3D weld profile.

[0077] In an embodiment, the pre-weld inspection at step 410 includes a tactile sensor comprised of a welding wire for a dual purpose to enable determination of the position of the weld components prior to the welding operation by a touch action that impinges on the weld components and to form a consumable electrode in the weld joint. In step 410, the weld components are also clamped with a clamping force that holds each weld portion that will make up the weld joint together. The tactile sensor is also used to measure a change in shape of the at least one weld component position via tactile sensing to determine a shape deformation of the weld components resulting from the combination of forces and clamping forces caused by the welding between the set of weld components.

[0078] At step 420, the robotic welding via the robotic arm is initiated, and at the second and third stages (430, 440), the welding process and monitoring data are generated, and at step 425, an automatic inspection is performed to generate post-weld inspection data. Figure 2

[0079] In an embodiment, the monitoring operation can be performed in step 420, and the automatic inspection of step 425 can be performed based on a three-dimensional (3D) profile of the weld target of the weld component geometry and the appropriate weld plan implemented. In an embodiment, a set of predetermined weld plans based on historical data can also be implemented. In steps 420 and 425, the monitoring and correlation of the deformations expected during the welding operation as the welding is being completed or performed can be performed, and adjustments to the weld plan can also be made to prevent or optimize the welding operation as the welding is being performed to attempt to correct any defects that are deemed likely to occur.

[0080] ​In an embodiment, the algorithm implemented by the edge computer system 5 for monitoring the welding process also responds to changes in the welding plan in response to changes in the welding plan. For example, depending on the changes in the welding plan, it can also be necessary to modify the monitoring algorithm accordingly to adjust the different data generated and received about the welding operation as a result of the plan changes. For example, depending on the welding plan implemented, the monitoring algorithm can have to change. That is, if the plan changes, the same algorithm (or even if the algorithm is not the same, at least the parameters are the same) cannot be used to monitor the welding operation.

[0081] In an embodiment, the information is retained by the edge computer system 5 of which part of the weld can be performed in which plan, and then appropriate classifiers / algorithms can be applied to determine whether the weld can be considered an acceptable weld. Thus, the robot kinematic capabilities can be used for weld monitoring as well as using voltage, current, wire feed, gas flow, etc. measurements. Further, using welding schedule based adaptive monitoring algorithms in welding operations with schedule awareness by combining the adjusted welding plan data with post-weld inspection to assess the final weld quality.

[0082] In an embodiment, the welding wire of the first level of tactile sensors used in the pre-weld inspection of step 410 is also used in step 445 to generate welding and process monitoring data of the consumable electrode of the weld wire configuration that is advanced to the weldment for depositing a weld bead via a wire feed sensor that provides a measured melt rate of the consumable electrode in the welding operation that forms a weld segment of the weld between the set of weldments. In step 445, process monitoring data is produced by a plurality of sets of sensors in the second level that sense a set of components associated with the welding operation and produce high resolution measurement data from direct and indirect sensing of measurements of the set of components associated with the welding operation compared to low resolution measurement data provided by the welding controller. Also in step 445, a quality evaluation step of the weld segment formed by the welding operation is based on welding monitoring rules applied to a result of a calculation of a function using the measured weldment gap plan. In some embodiments, a microphone or array of microphones sound sensor can be implemented in step 445 for monitoring ultrasonic and audible range frequencies of noise associated with the welding operation to determine whether the weld segment is compliant. Strain gauge sensors attached to the weldment can be implemented in step 445 for measuring a set of measurements exhibited by the weldment during the welding operation to determine warping of the weldment and whether the warping of the weldment exceeds a compliant level.

[0083] In step 425, an automatic inspection is generated, and the data is sent to step 450 for post-soldering inspection determination, and then to step 430 for data fusion quality analysis. In this embodiment, the data fusion quality analysis in step 430 (i.e., Figure 3 The quality analysis (370) can realize the process of combining pre-inspection, during-inspection, and post-inspection data from multiple sensors in steps 410, 445, and 450 of the welding process for processing and fusing high-resolution and low-resolution data to classify the weld using a classification algorithm, and using data from the welding wire modulation as input to the classification algorithm, which senses the position of the weld joint and estimates the gap size between the substrates to classify and characterize the weld.

[0084] The post-weld inspection at step 450 assesses the geometry and length of the final welded component using a camera / scanner, and is performed automatically by inspecting weld segments and welded component groups according to the adjusted welding plan. Furthermore, at step 450, a post-inspection analysis can be performed for post-weld operation inspection by combining high-resolution and low-resolution data from multiple sensors and a welding controller used to determine the trajectory of the robotic device implemented during the welding operation.

[0085] In one embodiment, the data generated by the pre-weld inspection (step 410) can also be combined with the data from the welding and process monitoring step 445, and with the data from the post-weld inspection data step 450 for further processing, and is sent at step 475 to be stored in a data and signal storage library.

[0086] In one embodiment, at step 430, data from multiple sources across multiple levels—pre-weld inspection, welding and process monitoring, and post-weld inspection—are analyzed via data fusion-based quality analysis by edge computer system 5 (or at server 15) to determine the stability of the weld performed during the welding process at step 435. In this regard, as an example, a rule-based scheme is further implemented in the process flow to approve or disapprove the weld, or alternatively, a Bayesian estimation process is used to define the probability that the weld is a qualified or unqualified weld based on welding data (i.e., as an example, pre-weld gap data 355 and post-weld bead profile data 360).

[0087] In one embodiment, if the process is determined to be unstable based on the criteria of the components identified during the welding process and the threshold value of each of the set of components, at step 440, a second non-destructive evaluation (NDE) (i.e., CT scan, selective cutting / etching operation, etc.) is performed and the stability of the process is rechecked again. If the process is stable, at step 455, a process rule function is applied to determine if the weld is compliant (i.e., rule-based pattern). If the weld is deemed compliant, at step 465, the weld is accepted and if not compliant, at step 460, the weld is rejected. At step 470, the non-compliant or rejected portions of the weld or weld component are indicated and the non-compliant portions are sent for further inspection (i.e., manual inspection).

[0088] It should be understood that the Figure 4 process can include any number of additional or alternative tasks, Figure 4 the tasks shown in FIG. 10 need not be performed in the order shown and Figure 4 the processes of FIG. 10 can be incorporated into a more comprehensive process or workflow having additional functionality not Figure 4 described in detail herein. Moreover, from Figure 4 the embodiments of the processes shown in FIG. 10, one or more tasks shown can be omitted.

[0089] The foregoing detailed description has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the subject matter's embodiments or the scope of applicability of these embodiments. As used herein, the word "exemplary" means "serving as an example, instance, or illustration." Any implementation described herein as exemplary is not necessarily to be construed as preferred or advantageous over other implementations. Furthermore, there is no intention to be bound by any expressed or implied theory presented in the preceding technical field, background, brief summary or detailed description.

[0090] While at least one exemplary embodiment has been presented in the foregoing detailed description of the application, it should be appreciated that a vast number of modifications can be made to the exemplary embodiments without departing from the scope of the present disclosure. Additionally, it should be appreciated that one or more exemplary embodiments are only examples and are not intended to limit the scope, applicability, or configuration of the disclosure in any way. Rather, the foregoing detailed description will provide those skilled in the art with a convenient road map for implementing one or more exemplary embodiments.

[0091] It should be understood that various changes can be made to the function and arrangement of elements without departing from the scope of the disclosure as set forth in the appended claims and their legal equivalents.

Claims

1. A system for a welding operation, comprising: a welding system including at least a first stage for determining a position of a weldment in a welding operation by a welding wire and a second stage for monitoring the welding operation and generating high resolution measurement data of the welding operation and enabling the welding wire to be additionally used in a weld segment in the welding operation; wherein the first stage includes at least one sensor including the welding wire to enable determination of a position of at least one weldment in a set of weldments prior to the welding operation by a touch action impinging on the at least one weldment, wherein at least one component is subjected to a clamping force holding each weldment in the set of weldments together, and measuring a shape change of the at least one weldment position by a tactile sensor to determine a shape deformation of the weldment resulting from a combination of forces caused by the welding between the set of weldments and the clamping force; wherein the second stage includes the at least one sensor including the welding wire, wherein the welding wire is configured as a consumable electrode advanced to a weldment to deposit a weld bead to form a welded weld segment between the set of weldments, and the welding wire is further configured as a wire feed sensor to provide a measured melting rate of the consumable electrode in the welding operation; and wherein the second stage further includes a plurality of sensors configured to sense a set of components associated with the welding operation, generating high resolution measurement data from direct and indirect sensing of measurements of a set of components associated with the welding operation as opposed to low resolution measurement data provided by a welding controller; at least one or more of the plurality of sensors for indirectly measuring the welding operation includes a microphone, a strain gauge sensor, and a welding reaction force sensor, the microphone is configured to monitor ultrasonic and audible range frequencies of noise associated with the welding operation to determine whether the weld segment is in compliance, the strain gauge sensor is configured to measure a set of measurements exhibited by a weldment during the welding operation to determine a warpage of the weldment and whether the warpage of the weldment exceeds a compliance level of the weldment to which the strain gauge sensor is attached, and the welding reaction force sensor is configured to determine a strength of the weld segment opposite the clamping force applied to the weld portion; a third stage of the welding system includes a processing monitoring module configured to qualify the weld segment formed by the welding operation based on welding monitoring rules applied to a calculation result of a function using the measured weldment gap plan; and The fourth stage of the welding system includes a post-weld inspection module configured to automatically inspect the weld segment and the set of weld components by using at least one of a camera and a scanner to evaluate the geometry of the weld components and the length of the weld components and to classify the welding operation using a classification algorithm that evaluates the fusion of high resolution data and low resolution data and the adjusted welding plan.

2. The system of claim 1, further comprising: wherein in the second stage, the plurality of sensors providing the high resolution measurement data includes at least one or more of a set of sensors to provide direct measurements of a high resolution current sensor, a high resolution voltage monitoring sensor, and a high resolution flow sensor for the welding operation.

3. The system of claim 2, further comprising: wherein in the second stage, the plurality of sensors providing the high resolution measurement data includes at least one or more of the plurality of sensors to provide indirect measurements of a vibrometer, a plasma sensor, an ultraviolet sensor, an electromagnetic spectrometer, the wire feed sensor, and an infrared camera for the welding operation.

4. The system of claim 1, further comprising: a process monitoring module configured to combine high resolution data and low resolution data from the plurality of sensors and the welding controller to determine a trajectory of a robotic device implemented in the welding operation, and wherein the fourth stage uses the trajectory in the automatic inspection.

5. The system of claim 4, further comprising: the fourth stage of the welding system includes a quality analysis to determine whether the weld joint is stable using a classification algorithm.

6. A method for monitoring a welding operation, comprising: configuring a welding system including a first stage for determining a position of a weld component from a welding wire in a welding operation and a second stage for monitoring the welding operation for generating high resolution measured data of the welding operation and for causing the welding wire to be additionally used for a weld segment in the welding operation; configuring at least one sensor in the first stage including the welding wire determining a position of at least one weld component in the set of weld components prior to the welding operation by a touch action of the welding wire impinging on the at least one weld component, wherein at least one portion is subjected to a clamping force holding each weld component in the set of weld components together, and measuring a shape change of the at least one weld component position by a tactile sensor to determine a shape deformation of the weld component resulting from a combination of the force from the welding between the set of weld components and the clamping force; configuring the at least one sensor in the second stage including the welding wire configured as a consumable electrode advancing to the weld component to deposit a weld bead to form a weld segment of a weld between the set of weld components, and further configuring a wire feed sensor for providing a measured melting rate of the consumable electrode in the welding operation; and and In the second stage, a plurality of sensors are configured for sensing a set of components associated with the welding operation, the high resolution measurement data is generated from direct and indirect sensing of measurements of the set of components associated with the welding operation, as opposed to low resolution measurement data provided by a welding controller, At least one or more of the plurality of sensors are configured for providing indirect measurements of the welding operation, including a microphone, a strain gauge sensor, and a welding reaction force sensor; The microphone is configured for monitoring ultrasonic frequencies and audible range frequencies of noise associated with the welding operation to determine whether the weld segment is in compliance; The strain gauge sensor is configured for measuring a set of measurements exhibited by the welding component during the welding operation to determine warping of the welding component and whether the warping of the welding component exceeds a compliance level of the welding component to which the strain gauge sensor is attached; The welding reaction force sensor is configured for determining a strength of the weld segment that is opposite to the clamping force applied to the weld portion; A third stage of the welding system includes a processing monitoring module, the welding system is configured through the processing monitoring module of the third stage for identifying the weld segment formed by the welding operation based on welding monitoring rules applied to a calculation result of a function using the measured welding component gap plan; and A fourth stage of the welding system includes a post-weld inspection module, the welding system is configured through the post-weld inspection module of the fourth stage, the module uses at least one of a camera and a scanner to automatically inspect the weld segment and the set of welding components to evaluate a geometry of the welding component and a length of the welding component, and uses a classification algorithm to classify the welding operation, the classification algorithm evaluates a fusion of the high resolution data and the low resolution data and an adjusted welding plan.

Images