Welding workshop robot state real-time monitoring method based on digital twinning

Abstract

The invention discloses a welding workshop robot state real-time monitoring method based on digital twinning, which relates to the technical field of intelligent welding monitoring, and comprises the following steps: comparing twinning prediction data with aligned multi-channel data moment by moment to obtain virtual-real contrast characteristics, extracting a key process section set according to the virtual-real contrast characteristics, and carrying out real-time monitoring on the state of a welding workshop robot according to the key process section set. Calculating a fast consistency ratio mean value and a welding stability index mean value; according to the fast consistency ratio mean value and the welding stability index mean value, a stability consistency function is constructed, a welding state interval boundary is defined, the welding state of the key process section is judged, and the abnormal type of the key process section is recognized; and based on the abnormal types of the key process sections, event trigger signals are generated, real-time path adjustment is executed, and energy control, motion control and visual acquisition control in the welding process are dynamically coordinated. State recognition and abnormal parting of key process sections are achieved, and stable operation of the welding process is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of intelligent welding monitoring technology, and in particular to a method for real-time monitoring of the status of welding workshop robots based on digital twins. Background Technology

[0002] Welding robots are crucial equipment in modern manufacturing, enabling high-precision welding and efficient production. They are widely used in the manufacturing processes of automotive structural components, construction machinery, and rail transportation equipment. Conventional welding condition monitoring methods typically rely on multi-source sensor systems. These systems collect data such as welding torch posture, torch speed, welding current, welding voltage, workpiece surface temperature distribution, and weld area images, combining time synchronization and signal analysis to determine welding quality and control the process. In recent years, with the development of industrial Ethernet, machine vision, and real-time computing technologies, multi-dimensional perception and information fusion in the welding process have gradually become important directions for achieving welding process visualization and dynamic evaluation, providing fundamental support for welding stability research and process optimization.

[0003] In existing welding monitoring methods, on the one hand, the relationship between welding energy input and thermal response is mostly determined by a single data channel or a fixed threshold, which makes it difficult to reflect the dynamic coupling law of energy transfer and heat diffusion; on the other hand, there is insufficient correlation analysis between welding torch posture changes and weld formation characteristics, and there is a lack of ability to identify and locate the synchronous changes in multi-source information. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a real-time monitoring method for the status of welding workshop robots based on digital twins, which solves the problems of insufficient real-time judgment of energy and thermal response coupling in the welding process and insufficient correlation and recognition of welding torch pose changes and grayscale image features of weld area.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: This invention provides a method for real-time monitoring of the status of a welding workshop robot based on digital twins, which includes collecting multi-source data of welding operations and completing time alignment under a unified time grid to form aligned multi-channel data and a unified timestamp sequence; Based on the unified timestamp sequence, the aligned multi-channel data is recursively calculated time-by-time to obtain twin prediction data; By comparing the twin prediction data with the aligned multi-channel data time-by-time, virtual-real comparison features are obtained. Based on the virtual-real comparison features, a set of key process sections is extracted, and the mean of rapid consistency ratio and welding stability index are calculated. Based on the mean of the rapid consistency ratio and the mean of the welding stability index, a stability consistency function is constructed to define the boundary of the welding state interval, determine the welding state of key process sections, and identify the anomaly types of key process sections. Based on the anomaly types in key process sections, event trigger signals are generated, and real-time adjustment paths are executed to dynamically coordinate energy control, motion control, and vision acquisition control during the welding process.

[0007] As a preferred embodiment of the real-time status monitoring method for welding workshop robots based on digital twins described in this invention, the multi-source data of the welding operation includes welding torch posture, welding torch movement speed, welding current, welding voltage, workpiece surface temperature distribution image and weld area grayscale image.

[0008] As a preferred embodiment of the real-time monitoring method for the status of welding workshop robots based on digital twins described in this invention, the step of completing time alignment under a unified time grid includes interpolating the welding current, welding voltage and welding torch movement speed, performing time window matching on the workpiece surface temperature distribution image and the weld area grayscale image, and performing attitude interpolation correction on the welding torch pose data. Obtain aligned multi-channel data and a unified timestamp sequence under a unified time grid.

[0009] As a preferred embodiment of the real-time monitoring method for the status of welding workshop robots based on digital twins described in this invention, the step of performing time-by-time recursive calculation to obtain twin prediction data includes, at each unified timestamp, taking the aligned multi-channel data of the previous unified timestamp as input, performing forward Euler integral on the welding torch pose and welding torch movement speed to obtain twin pose prediction. A one-step retention-type prediction was performed on the welding current and welding voltage respectively to obtain twin current prediction and twin voltage prediction; An electrothermal equivalent incremental update is performed on the workpiece surface temperature distribution image to obtain twin temperature prediction.

[0010] As a preferred embodiment of the real-time monitoring method for the status of welding workshop robots based on digital twins as described in this invention, the method of obtaining virtual-real comparison features includes comparing the twin pose prediction, twin current prediction, twin voltage prediction, twin temperature prediction, and aligned multi-channel data at each unified timestamp to obtain energy closure residual, temperature synchronization degree, and grayscale edge energy of the weld area.

[0011] As a preferred embodiment of the real-time monitoring method for the status of welding workshop robots based on digital twins as described in this invention, the step of extracting the set of key process sections based on virtual-real comparison features includes: calculating the trajectory curvature of the twin pose prediction based on the geometric relationship of adjacent twin pose predictions, and performing differential calculation on the trajectory curvature of adjacent twin pose predictions to obtain the trajectory curvature change rate of the twin pose prediction. A joint time series analysis was performed on the gray-scale edge energy of the weld area and the rate of change of trajectory curvature predicted by twin pose. When the grayscale edge energy and the trajectory curvature change rate predicted by twin pose in the weld area at the same unified timestamp reach local extreme values ​​at the same time, the time point corresponding to the corresponding unified timestamp is determined to be the key process section. All records that simultaneously reach local extreme values ​​at the time points corresponding to the same timestamp constitute a set of key process sections.

[0012] As a preferred embodiment of the real-time status monitoring method for welding workshop robots based on digital twins described in this invention, the calculation of the mean value of the fast consistency ratio and the mean value of the welding stability index includes calculating the fast consistency ratio by the ratio of energy closure residual to temperature synchronization degree. Welding stability index is calculated by combining the rapid consistency ratio, gray-scale edge energy of the weld area, and trajectory curvature change rate predicted by twin pose. Within the set of key process sections, calculate the mean rapid consistency ratio and the mean welding stability index for each key process section.

[0013] As a preferred embodiment of the real-time monitoring method for the status of welding workshop robots based on digital twins described in this invention, the construction of the stability consistency function and the definition of the welding status interval boundary include, on a unified timestamp sequence, calculating the instantaneous stability change based on the difference in welding stability index corresponding to adjacent unified timestamps; The root mean square value of the instantaneous stability change is combined with the average welding stability index and the average rapid consistency ratio of each key process section to construct a stability consistency function; In the same welding task, the mean and standard deviation of the stability consistency function for all key process sections are calculated, and the stable boundary and the fluctuating boundary of the welding state are defined based on the statistical distribution characteristics of the stability consistency function.

[0014] As a preferred embodiment of the real-time status monitoring method for welding workshop robots based on digital twins described in this invention, the determination of the welding status of key process sections and the identification of the abnormality type of key process sections include determining that the welding status of key process sections is stable when the stability consistency function is not lower than the stability boundary of the welding status. When the stability consistency function lies between the stable boundary of the welding state and the fluctuating boundary of the welding state, the welding state of the key process section is determined to be slightly fluctuating. When the stability consistency function is lower than the fluctuation boundary of the welding state, the welding state of the key process section is judged to be abnormal. For key process sections with abnormal welding conditions, the relationship between the changes in energy closure residual, temperature synchronization, gray-scale edge energy of the weld area, and trajectory curvature change rate predicted by twin pose is analyzed to identify the anomaly types: welding energy mismatch anomaly, welding torch posture disturbance anomaly, visual observation anomaly, and composite anomaly.

[0015] As a preferred embodiment of the real-time status monitoring method for welding workshop robots based on digital twins described in this invention, the dynamic coordination of energy control, motion control and visual acquisition control during the welding process includes generating event trigger signals by taking welding energy mismatch anomalies, welding torch posture disturbance anomalies, visual observation anomalies and composite anomalies as trigger sources. When a welding energy mismatch anomaly is triggered, the twin voltage prediction and twin current prediction under the current unified timestamp are read, the virtual welding power estimate is calculated, and the welding power setting is obtained by correcting it through the proportional adjustment function. The welding power setting is then output to the welding power source. When a welding torch attitude disturbance anomaly is triggered, the twin pose prediction under the current unified timestamp is extracted. The attitude smoothing correction angle is calculated based on the trend of the trajectory curvature change rate of the twin pose prediction. The twin pose prediction is then corrected to obtain the corrected attitude target. The corrected attitude target is then output to the welding torch servo control unit to update the actual motion trajectory of the welding torch synchronously with the unified timestamp. When a visual observation anomaly is triggered, the grayscale image of the weld area is read and the grayscale edge energy of the weld area is calculated. If the grayscale edge energy of the weld area decreases within multiple consecutive unified timestamps, the visual acquisition link self-calibration operation is performed. When a composite anomaly is triggered, the states in which the energy closure residual, temperature synchronization, and gray-scale edge energy of the weld area deviate simultaneously from the trajectory curvature change rate predicted by the twin pose are identified. The real-time adjustment paths corresponding to welding energy mismatch anomalies, welding torch posture disturbance anomalies, and visual observation anomalies are called respectively, and synchronous coordination operations are performed under a unified timestamp. When the monitored quantities recover to the stable range, the composite anomaly is determined to be resolved.

[0016] The beneficial effects of this invention are as follows: by establishing a collaborative analysis mechanism of recursive calculation and virtual-real comparison for multi-channel data, the synchronous correlation of welding energy transfer, welding torch posture change and workpiece thermal response is realized; by constructing a stability consistency function and introducing a welding state interval boundary judgment mechanism, the state identification and anomaly classification of key process sections are realized, thereby ensuring the stable operation of the welding process and the high precision consistency of monitoring results. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a method for real-time monitoring of the status of robots in a welding workshop based on digital twins.

[0019] Figure 2 The flowchart shows how twin prediction data is calculated recursively time-by-time.

[0020] Figure 3 A flowchart for obtaining virtual-real comparison characteristics and calculating the mean of indicators.

[0021] Figure 4 This is a flowchart for determining and adjusting the welding status in real time. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0025] Reference Figures 1-4 As one embodiment of the present invention, this embodiment provides a method for real-time monitoring of the status of a welding workshop robot based on digital twins, comprising the following steps: S1. Collect multi-source data of welding operations and complete time alignment under a unified time grid to form aligned multi-channel data and a unified timestamp sequence.

[0026] Furthermore, through industrial Ethernet (e.g., OPC UA protocol) and under a unified clock (e.g., IEEE 1588 PTP protocol), multi-source data of welding operations are collected with the same time base and carrying a unified timestamp, including welding torch posture, welding torch movement speed, welding current, welding voltage, workpiece surface temperature distribution image and weld area grayscale image.

[0027] Among them, welding torch posture refers to the position and orientation of the center point of the welding torch tool in space; welding torch movement speed refers to the moving speed and rotation speed of the center point of the welding torch tool in space; welding current refers to the output current of the welding power supply; welding voltage refers to the output voltage of the welding power supply; workpiece surface temperature distribution image refers to the temperature image acquired by an infrared thermal imaging camera and corrected for emissivity, background reflection and distance compensation; weld area grayscale image refers to a single-channel grayscale image acquired by a visible light industrial camera and processed with exposure and gain lock, constant light source or anti-flicker.

[0028] Furthermore, under a unified clock and according to a unified time grid, the multi-source data of welding operations are time-aligned.

[0029] Linear interpolation or sample resampling is performed on the welding current, welding voltage, and welding torch speed to keep the welding current, welding voltage, and welding torch speed corresponding to each unified timestamp synchronized in the time dimension.

[0030] Time window matching is performed on the workpiece surface temperature distribution image and the weld area grayscale image, and synchronous matching is performed at the acquisition frame closest to the unified timestamp.

[0031] The welding torch pose data is subjected to attitude interpolation correction, and the attitude is smoothly transitioned between adjacent unified timestamps, so that the position and attitude of the welding torch tool center point remain continuous and consistent under a unified time reference.

[0032] Time base calibration is performed periodically under a unified clock to correct the synchronization offset of multi-source data for various welding operations under a unified time grid, so that the aligned multi-channel data maintains logical consistency within a unified timestamp range.

[0033] By verifying the time difference of a unified timestamp sequence, the time alignment accuracy is checked, avoiding the accumulation of time drift caused by differences in acquisition path or network latency.

[0034] The final result is a series of aligned multi-channel data (welding torch pose, welding torch speed, welding current, welding voltage, workpiece surface temperature distribution image, and weld area grayscale image) under a unified time grid, along with the corresponding unified timestamp sequence.

[0035] It should be noted that a unified clock is used to maintain time synchronization during the multi-source data acquisition process of welding operations, a common time reference is used to constrain the time consistency of multi-source data of various welding operations at the acquisition time, and a unified time grid is used to organize multi-channel data and perform time alignment under a unified clock and a common time reference.

[0036] S2. Based on the unified timestamp sequence, the aligned multi-channel data is recursively calculated time-by-time to obtain twin prediction data.

[0037] Furthermore, the aligned multi-channel data is recursively calculated time-by-time according to the unified timestamp sequence to update the welding torch pose, welding current, welding voltage and workpiece surface temperature distribution images, and obtain twin prediction data corresponding one-to-one with the unified timestamp, including twin pose prediction, twin current prediction, twin voltage prediction and twin temperature prediction.

[0038] At each unified timestamp, using the aligned multi-channel data from the previous unified timestamp as input, execute steps D1-D4.

[0039] D1. Perform forward Euler integral on the welding torch pose and welding torch velocity, and define twin pose prediction as follows: ; in, Indicates time Twin pose prediction, Indicates time The welding torch position. Indicates time The speed of the welding torch movement, Indicates the interval between adjacent unified timestamps. Represents the first in the unified timestamp sequence At that moment, Represents the first in the unified timestamp sequence That moment.

[0040] D2. Perform a one-step retention-type prediction on the welding current, defining twin current prediction as follows: ; in, Indicates time Twin current prediction, Indicates time The welding current.

[0041] D3. Perform a holding-type one-step prediction on the welding voltage, defining twin voltage prediction as follows: ; in, Indicates time Twin voltage prediction, Indicates time The welding voltage.

[0042] D4. Perform electrothermal equivalent incremental updates on the workpiece surface temperature distribution image, and define twin temperature prediction as follows: ; ; ; in, Indicates time In spatial location Twin temperature prediction, Indicates time In spatial location Twin temperature prediction, Indicates spatial location The equivalent temperature rise coefficient, Indicates the electrothermal equivalence coefficient. Indicates time Welding voltage, Indicates time Welding current, This represents the time variable in integral calculations. Indicates time In spatial location The surface temperature of the workpiece. Indicates time In spatial location The surface temperature of the workpiece. Indicated by time The right endpoint is the window length is The start time of the time interval, Indicates the length of the calibrated sliding window. Indicates the density of the material. This indicates the specific heat at constant pressure of the material. This represents the equivalent thermal conductivity thickness of the workpiece in the direction of thermal conduction. This represents the imaging coverage area of ​​the workpiece surface temperature distribution image. This indicates the area of ​​the workpiece surface region.

[0043] Repeat steps D1-D4 along the unified timestamp sequence to form twin prediction data that correspond one-to-one with the unified timestamp.

[0044] It should be noted that the length of the calibrated sliding window is... The value is determined based on the matching relationship between the energy input cycle of the welding process and the thermal diffusion response characteristics of the workpiece, with an example value range of 0.3~0.8 seconds. Specifically, during the operation of the welding robot, the welding current and welding voltage fluctuate with the power supply frequency in a short period, while the change in workpiece surface temperature is affected by the material's thermal conductivity, thickness, and thermal diffusion time constant, typically reaching thermal stability within hundreds of milliseconds. If the calibrated sliding window length is too short, the calculated temperature rise result will be affected. It is susceptible to transient fluctuations; if the calibration sliding window length is too long, it will span the arc initiation, stable welding and arc termination stages of the weld, resulting in a lag in thermal response; combined with the typical arc fluctuation period of the welding process (about 2.5 to 5 milliseconds) and the workpiece heat diffusion time (about 0.2 to 0.7 seconds), the example sets the calibration sliding window length in the range of 0.3 to 0.8 seconds, which can make the temperature rise calculation results smooth transient interference and timely reflect the real changes in welding heat input.

[0045] S3. By comparing the twin prediction data with the aligned multi-channel data time-by-time, virtual-real comparison features are obtained. Based on the virtual-real comparison features, the set of key process sections is extracted, and the mean of rapid consistency ratio and welding stability index is calculated.

[0046] Furthermore, according to the unified timestamp sequence, the twin pose prediction, twin current prediction, twin voltage prediction, twin temperature prediction, and aligned multi-channel data are compared time-by-time to calculate virtual-real comparison characteristics, including energy closure residual, temperature synchronization degree, and gray-scale edge energy of the weld area. The set of key process sections is extracted based on the change of gray-scale edge energy of the weld area and the change rate of trajectory curvature of the twin pose prediction.

[0047] The virtual welding energy input is calculated using twin voltage and twin current predictions, and compared with the heat absorption corresponding to the workpiece surface temperature distribution image. The energy closure residual is then calculated and expressed as: ; in, Indicates time The energy closure residual, with a value range of . , Indicates time Twin voltage prediction, Indicates time Twin current prediction, Indicates time In spatial location The surface temperature of the workpiece.

[0048] When the energy closes the residual When the value is close to 0 (e.g., less than 10 joules), it indicates that the virtual welding energy input calculated by the twin voltage prediction and twin current prediction is basically consistent with the heat absorption reflected by the workpiece surface temperature distribution image, and the welding energy transfer is stable.

[0049] When the energy closes the residual A slight increase within a small range (e.g., the absolute value of the difference in energy closure residuals corresponding to adjacent unified time stamps does not exceed 15% of the energy closure residual of the previous unified time stamp) indicates a slight difference between the virtual welding energy input and the heat absorption reflected in the workpiece surface temperature distribution image, typically determined by the electrothermal equivalence coefficient. Changes or thermal conductivity parameters The deviation caused the welding process to be in a quasi-steady state.

[0050] When the energy closes the residual When the difference between the energy closure residuals corresponding to adjacent unified timestamps increases continuously (for example, the difference between the energy closure residuals of the previous unified timestamp exceeds 30%), it indicates that the difference between the virtual welding energy input calculated by twin voltage prediction and twin current prediction and the heat absorption reflected by the workpiece surface temperature distribution image has widened, the welding arc stability has decreased or the thermal conductivity of the workpiece has changed, and the welding energy transfer is no longer matched.

[0051] It should be noted that the numerical values ​​in the energy closure residual judgment conditions are all exemplary selections, determined based on the dynamic coupling law between virtual welding energy input and workpiece surface heat absorption, as well as the typical fluctuation characteristics of welding energy input and workpiece thermal response: 10 joules, approximately 5% of the total virtual welding energy input per unit time, is used to distinguish the difference between virtual welding energy input and workpiece surface heat absorption, as well as the energy difference caused by normal thermal radiation loss; 15% corresponds to the normal dynamic fluctuation ratio between virtual welding energy input and workpiece surface heat absorption, used to determine that the virtual welding energy input and workpiece surface heat absorption are matched and the welding process is in a quasi-steady state; 30% represents the judgment ratio for a continuously expanding difference between virtual welding energy input and workpiece surface heat absorption, used to identify welding energy transfer mismatch and thermal response deviation caused by uneven arc energy; the above numerical values ​​can be adapted and adjusted according to the welding process type, welding arc power level, workpiece thermal conductivity characteristics, and electrothermal equivalence coefficient correction conditions to ensure the consistency between the energy closure residual judgment results and the actual welding energy transfer state.

[0052] Temperature synchronization is calculated by comparing the twin temperature prediction with the workpiece surface temperature distribution image point by point, and is expressed as: ; in, Indicates time Temperature synchronization, with a value range of: .

[0053] When temperature synchronization When the value is close to 1 (e.g., greater than 0.9), it indicates that the twin temperature prediction and the workpiece surface temperature distribution image are highly consistent in terms of spatial distribution and trend, the synchronization accuracy between the virtual temperature field and the actual temperature field is high, and the heat diffusion process in the welding area is stable.

[0054] When temperature synchronization When the value is in the middle range (e.g., 0.4 to 0.7), it indicates that there is a certain deviation between the twin temperature prediction and the workpiece surface temperature distribution image, the consistency of the temperature distribution decreases, and the thermal response is delayed.

[0055] When temperature synchronization When the value is in a lower range (e.g., 0 to 0.4), it indicates that the twin temperature prediction differs significantly from the workpiece surface temperature distribution image, and the virtual temperature field is not synchronized with the actual temperature field.

[0056] The grayscale edge energy of the weld area is calculated by analyzing the gradient intensity of the grayscale image of the weld area, and is expressed as: ; in, Indicates time The grayscale edge energy of the weld area, with a value range of [value range missing]. , Indicates time At pixel position The gradient vector of the grayscale image of the weld area. This represents the pixel coverage area of ​​the grayscale image of the weld region. This represents the pixel area of ​​the weld seam region.

[0057] The gradient vector consists of the grayscale change rate of the grayscale image of the weld area in the horizontal and vertical pixel coordinates.

[0058] When the grayscale edge energy of the weld area When the level is high (e.g., when it is more than 20% higher than the average grayscale edge energy of the weld area corresponding to adjacent unified time stamps within multiple consecutive unified time stamps), it indicates that the weld pool boundary is clear, the energy input is sufficient, and the welding morphology is stable.

[0059] When the grayscale edge energy of the weld area When the fluctuation increases (e.g., the relative change in grayscale edge energy values ​​between adjacent weld areas at the same time stamp exceeds 30%), it indicates that the weld pool morphology is changing rapidly, which may indicate abnormal arc oscillation or welding torch posture deviation.

[0060] When the grayscale edge energy of the weld area When the energy level continues to decrease (e.g., the average decrease in grayscale edge energy of weld areas corresponding to adjacent unified timestamps exceeds 25%), it indicates that the weld area boundary is blurred, the weld pool is shrinking or the energy input is insufficient, and the weld formation quality is declining.

[0061] It should be noted that the percentage values ​​in the determination criteria for grayscale edge energy changes in the weld area are all exemplary selections, determined based on the gradient intensity distribution variation law of the grayscale image of the weld area and the stability of the light intensity distribution of the weld pool: 20% corresponds to the proportion where the grayscale edge energy of the weld area remains higher than the average grayscale edge energy of the weld area at adjacent unified time stamps within multiple consecutive unified time stamps, used to determine the state of clear weld pool boundaries, matching of virtual welding energy input with workpiece surface heat absorption, and stable weld morphology; 30% corresponds to the grayscale edge energy values ​​of the weld area at adjacent unified time stamps. The relative change range between these values ​​is used to determine the state of rapid changes in weld pool morphology, frequent changes in welding torch posture, or unstable arc oscillation; 25% corresponds to the average decrease in grayscale edge energy of the weld area corresponding to adjacent unified timestamps, used to determine the state of blurred weld area boundaries, weakened virtual welding energy input, or shrinking weld pool; the above percentage values ​​can be adapted and adjusted according to the welding current fluctuation range, welding voltage stability, and grayscale dynamic response characteristics of infrared thermal imaging cameras and visible light industrial cameras to ensure that the determination results of grayscale edge energy changes in the weld area are consistent with the actual welding conditions.

[0062] On a unified timestamp sequence, the trajectory curvature of the twin pose prediction is calculated based on the geometric relationship between adjacent twin pose predictions, and is expressed as: ; in, Indicates time The trajectory curvature predicted by the twin pose. Indicates time The trajectory curvature predicted by the twin pose. The vector modulus operator.

[0063] The difference between the trajectory curvatures of adjacent twin pose predictions is calculated to obtain the rate of change of trajectory curvature in twin pose predictions, expressed as: ; in, Indicates time The rate of change of trajectory curvature predicted by twin pose, with a value range of [value range missing]. This is used to reflect the smoothness of changes in the welding torch's posture. Represents a very small positive constant to prevent the denominator from being zero, for example .

[0064] When the rate of change of trajectory curvature predicted by twin pose When there is a rapid increase or frequent fluctuation, it indicates a sudden change in the trajectory of the welding torch, which may be accompanied by weld fluctuations or abnormal heat input.

[0065] A joint time series analysis is performed on the gray-scale edge energy of the weld area and the trajectory curvature change rate predicted by the twin pose. When the gray-scale edge energy of the weld area and the trajectory curvature change rate predicted by the twin pose simultaneously reach local extreme values ​​at the same unified timestamp, the time point corresponding to the corresponding unified timestamp is determined as the key process section, and all records that simultaneously reach local extreme values ​​at the time point corresponding to the unified timestamp constitute the set of key process sections.

[0066] Simultaneous local extrema refer to the following: Calculate the time series of the gray-scale edge energy of the weld region and the rate of change of trajectory curvature predicted by the twin pose on a unified timestamp sequence. When the gray-scale edge energy of the weld region at the same unified timestamp shows a monotonic reversal in direction relative to the gray-scale edge energy of the weld region at the previous and subsequent unified timestamps, and the rate of change of trajectory curvature predicted by the twin pose also shows a monotonic reversal in direction relative to the rate of change of trajectory curvature predicted by the twin pose at the previous and subsequent unified timestamps, then the time point corresponding to the corresponding unified timestamp is determined to be the point where the gray-scale edge energy of the weld region and the rate of change of trajectory curvature predicted by the twin pose simultaneously reach local extrema.

[0067] At this point, the time point corresponding to the unified timestamp represents the synchronous response characteristics of the gray-scale edge energy change in the weld area and the welding torch posture change, which is used to determine key process sections.

[0068] It should be noted that by performing time-by-time recursive calculations on the aligned multi-channel data, twin prediction data is formed, and virtual-real comparison features are constructed using energy closure residuals, temperature synchronization, and grayscale edge energy of the weld area, thereby realizing synchronous comparison and dynamic correlation of virtual and real data in the welding process.

[0069] Furthermore, the rapid consistency ratio is defined as the ratio of energy closure residual to temperature synchronization, expressed as: ; in, Indicates time The fast consistency ratio, with a value range of 1. It is used to reflect the degree of coupling between welding energy and thermal response. This represents the welding energy normalization constant (e.g., 100 joules), used to eliminate dimensional differences.

[0070] When the fast consistency ratio When the value is close to 1, it indicates that the virtual welding energy input is consistent with the energy transfer in the actual welding process, the virtual thermal response corresponds well with the actual thermal response, and the welding energy transfer process is stable.

[0071] When the fast consistency ratio When the value decreases, it indicates that there is a deviation between the virtual welding energy input and the thermal response of the actual welding process, the correspondence between the virtual welding energy input and the changes in the temperature distribution on the workpiece surface decreases, and the stability of welding energy transfer decreases.

[0072] It should be noted that the decrease in the rapid consistency ratio refers to the fact that, within a continuous range of unified timestamps, the change in the rapid consistency ratio between adjacent unified timestamps is negative and the magnitude of the change gradually increases. In other words, the rapid consistency ratio shows a continuous decreasing trend and tends to accelerate across multiple consecutive unified timestamps. This trend reflects the continuous weakening of the correspondence between virtual welding energy input and the change in workpiece surface temperature distribution, and the continuous reduction in the dynamic matching degree between virtual welding energy input and workpiece surface heat absorption, thereby leading to a decrease in the stability of welding energy transfer.

[0073] Within each unified timestamp, the consistency between virtual and real energy and thermal response is characterized by the fast consistency ratio, the stability of weld formation characteristics is characterized by the gray-scale edge energy of the weld region, and the smoothness of the welding torch attitude is characterized by the trajectory curvature change rate predicted by twin pose.

[0074] The welding stability index is calculated by combining the rapid consistency ratio, the gray-scale edge energy of the weld area, and the trajectory curvature change rate predicted by twin pose, and is expressed as: ; in, Indicates time The welding stability index, with a value range of [value range missing]. It is used to characterize the overall operating status of the welding process. This indicates the maximum value of the grayscale edge energy in the weld area during the current welding task.

[0075] When welding stability index A value close to 1 indicates stable welding posture, uniform welding energy input, and stable forming quality.

[0076] When welding stability index A continuous decline (e.g., the welding stability index corresponding to adjacent unified timestamps is less than 0.5, and the average decline is greater than 20% of the welding stability index corresponding to the previous unified timestamp) indicates that there is an abnormality in the welding posture or welding energy during the welding process.

[0077] Within the set of key process sections, the average welding stability index is calculated for each key process section, and expressed as: ; in, Indicates the first Average welding stability index of key process sections Indicates the first The start time of each key process section Indicates the first The completion time of each key process section Indicates time Welding stability index.

[0078] Using the same method, the mean rapid consistency ratio is calculated for each key process section in the set of key process sections.

[0079] S4. Based on the mean of the rapid consistency ratio and the mean of the welding stability index, construct a stability consistency function, define the boundary of the welding state interval, determine the welding state of key process sections, and identify the abnormal types of key process sections.

[0080] Furthermore, within each unified timestamp, the instantaneous stability change is calculated based on the difference in welding stability indices corresponding to adjacent unified timestamps, and is expressed as: ; in, Indicates time The instantaneous stability change is used to evaluate the degree of dynamic fluctuation in the welding state.

[0081] When the instantaneous stability change When the value remains negative and its absolute value increases over multiple consecutive unified timestamps, it indicates a decrease in the stability of the welding process.

[0082] When the instantaneous stability change When the value changes from negative to positive, it indicates that the welding process has returned to stability.

[0083] Calculate the root mean square value of the instantaneous stability change in the set of key process sections.

[0084] By combining the root mean square value of the instantaneous stability change with the mean welding stability index and the mean rapid consistency ratio of each key process section, a stability consistency function is constructed, expressed as: ; in, Indicates the first The stability consistency function for key process sections, with a value range of [value range missing]. , Indicates the first The average rapid consistency ratio of key process sections Indicates the first The root mean square value of the instantaneous stability change of each key process section.

[0085] To prevent extreme noise from causing the stability consensus function to become negative, a suppression factor is limited in the stability consensus function. It is a non-negative number. When the inhibition factor is less than 0, the inhibition factor is limited to a value of 0.

[0086] In the same welding task, the mean and standard deviation of the stability consistency function are calculated for all key process sections. Based on the statistical distribution characteristics of the stability consistency function, the interval boundaries of the welding state are defined, including the stable boundary and the fluctuating boundary of the welding state, as follows: ; ; ; in, The stability boundary of the welding state. Indicates the fluctuation boundary of the welding state. This represents the average value of the stability consistency function across all key process sections, used to characterize the overall stability level of the current welding task. The standard deviation of the stability consistency function for all key process sections is used to reflect the fluctuation range of welding stability. Indicates the sensitivity coefficient for stability determination. This represents the average temperature synchronization calculated over the entire unified timestamp range, used to characterize the overall consistency between the virtual and actual temperature fields throughout the entire welding task. The variance represents the energy closure residual.

[0087] when At that time, the welding status of the key process section was determined to be stable.

[0088] when At that time, the welding status of the key process section was determined to be slightly fluctuating.

[0089] when At that time, the welding status of key process sections was determined to be abnormal.

[0090] Furthermore, for key process sections with abnormal welding conditions, the relationship between the changes in energy closure residual, temperature synchronization, gray-scale edge energy of the weld area, and trajectory curvature change rate predicted by twin pose is analyzed to identify the anomaly type of each key process section, including welding energy mismatch anomaly, welding torch posture disturbance anomaly, visual observation anomaly, and composite anomaly.

[0091] When the energy closure residual continues to increase, and the temperature synchronization is in a lower range or moves from the middle range to a lower range (showing a downward trend), it indicates that the deviation between the virtual welding energy input and the workpiece surface temperature response is gradually widening, and the heat input and heat diffusion are mismatched. If the grayscale edge energy of the weld area is at a high level or relatively stable at this time, and the trajectory curvature change rate predicted by the twin pose changes steadily (without rapid increase or frequent fluctuation), it is determined to be a welding energy mismatch anomaly caused by changes in thermal conductivity.

[0092] When the energy closure residual increases or fluctuates within a small range, but the temperature synchronization degree is close to 1 or remains in the middle range, and the energy fluctuation at the gray edge of the weld area increases, and the trajectory curvature change rate predicted by twin pose shows a rapid increase or frequent fluctuation, it indicates that the electrothermal transfer is stable but the welding torch posture changes drastically, and is judged as a welding torch posture disturbance anomaly.

[0093] When the energy closure residual is close to 0 or increases slightly, the temperature synchronization is close to 1 or remains in the middle range, the grayscale edge energy of the weld area continues to decrease, and the trajectory curvature change rate of the twin pose prediction changes steadily, it indicates that the grayscale image acquisition of the weld area has been attenuated or occluded, which is judged as an abnormality in visual observation.

[0094] When the energy closure residual continues to increase, the temperature synchronization is in a low range, the grayscale edge energy fluctuation in the weld area increases, and the trajectory curvature change rate predicted by twin pose shows a rapid increase or frequent fluctuation, it indicates that the welding energy input, heat diffusion and welding torch posture change are all unstable, and it is judged as a composite anomaly.

[0095] It should be noted that by calculating the mean of the rapid consistency ratio and the mean of the welding stability index in the set of key process sections, a stability consistency function is constructed and the boundary of the welding state interval is defined, so as to realize the determination of the welding state and the identification of the abnormal type in the key process sections, thereby dynamically coordinating welding energy, welding torch posture and visual acquisition, and improving the stability of the welding process and the real-time monitoring accuracy.

[0096] S5. Based on the anomaly type of key process sections, generate event trigger signals, execute real-time adjustment paths, and dynamically coordinate energy control, motion control, and vision acquisition control during the welding process.

[0097] Furthermore, within each unified timestamp, an event trigger signal is generated based on the anomaly type of the key process section for each anomaly.

[0098] The event triggering signals are triggered by welding energy mismatch anomalies, welding torch posture disturbance anomalies, visual observation anomalies, and composite anomalies, each corresponding to a different real-time adjustment path.

[0099] Furthermore, when a welding energy mismatch anomaly is triggered, the predicted twin voltage and twin current at the current moment are read, and the estimated virtual welding power is calculated, expressed as: ; in, Indicates time The estimated virtual welding power.

[0100] The welding power setting is obtained by correcting the virtual welding power estimate using a proportional adjustment function, and is expressed as: ; in, Indicates time The welding power supply is set to power.

[0101] The welding power supply is set to output power through a PLC, enabling real-time micro-compensation and adjustment of the arc energy.

[0102] Furthermore, when a welding torch attitude disturbance anomaly is triggered, the twin pose prediction for the current moment is extracted, and the attitude smoothing correction angle is calculated based on the trend of the trajectory curvature change rate, expressed as: ; in, Indicates time The attitude smoothing correction angle.

[0103] The attitude prediction is corrected by applying an attitude smoothing correction angle to the twin pose prediction, resulting in the corrected attitude target, denoted as: ; in, Indicates time The corrected attitude target.

[0104] The corrected attitude target is output to the welding torch servo control unit, and the actual motion trajectory of the welding torch is updated synchronously with a unified timestamp, so as to realize real-time fine adjustment of the welding torch attitude and smooth recovery of the trajectory.

[0105] Furthermore, when a visual observation anomaly is triggered, the grayscale image of the weld area is read and the grayscale edge energy of the weld area is calculated. Based on the change trend of the grayscale edge energy of the weld area over time, it is determined whether there is attenuation or occlusion in the grayscale image acquisition process of the weld area.

[0106] When the grayscale edge energy of the weld area decreases significantly within multiple consecutive unified time stamps, a self-calibration operation of the vision acquisition link is performed, adjusting the exposure time and gain parameters of the visible light industrial camera in sequence, and re-detecting the stability of the light source.

[0107] When the grayscale edge energy of the weld area returns to normal or increases significantly compared to the previous cycle within several acquisition cycles after the self-calibration operation, it is determined that the grayscale image acquisition process of the weld area has returned to normal, and the abnormal visual observation state is resolved.

[0108] If the grayscale edge energy of the weld area fails to recover within two consecutive acquisition cycles, the grayscale image acquisition process of the weld area is determined to be unstable. The grayscale image of the weld area is marked as data with reduced confidence. In subsequent processes, only the grayscale image of the weld area is recorded and archived, and the grayscale edge energy of the weld area is no longer updated in real time. This is to prevent abnormal grayscale edge energy of the weld area from interfering with the accuracy of judging weld morphology changes and the calculation results of welding stability index.

[0109] By jointly adjusting the exposure time, gain parameters, and the way grayscale image data of the weld area participates, the continuity and stability of grayscale image data acquisition in the weld area can be maintained under conditions of occlusion, light fluctuation, or abnormal exposure, ensuring that the welding status monitoring process is not interrupted.

[0110] Furthermore, when a composite anomaly is triggered, it is identified that the energy closure residual, temperature synchronization, gray-scale edge energy of the weld area, and trajectory curvature change rate predicted by twin pose all show significant deviations.

[0111] In the case of combined anomalies, the real-time adjustment paths corresponding to welding energy mismatch anomalies, welding torch posture disturbance anomalies, and visual observation anomalies are invoked respectively, and synchronous and coordinated operations of welding energy adjustment, welding torch posture correction, and visual self-calibration are performed under a unified timestamp.

[0112] When, after synchronous coordination, the energy closure residual decreases, the temperature synchronization degree recovers, the grayscale edge energy of the weld area tends to stabilize, and the trajectory curvature change rate predicted by twin pose decreases or returns to stability, the composite anomaly is determined to be resolved; if any parameter does not recover to the stable range, the corresponding adjustment path is kept in dynamic operation until all monitored quantities return to normal.

[0113] By multi-dimensional synchronous coordination and adjustment under a unified time reference, dynamic consistency and collaborative self-stabilization among welding power output, welding torch attitude trajectory and weld area grayscale image acquisition are achieved, ensuring that the welding process maintains a continuous, stable and controllable operating state under combined disturbance conditions.

[0114] In summary, this invention achieves synchronous correlation between welding energy transfer, welding torch posture changes, and workpiece thermal response by establishing a collaborative analysis mechanism for recursive calculation and virtual-real comparison of multi-channel data; and by constructing a stability consistency function and introducing a welding state interval boundary judgment mechanism, it achieves state identification and anomaly classification of key process sections, thereby ensuring stable operation of the welding process and high-precision consistency of monitoring results.

[0115] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A digital-twin-based real-time monitoring method for a welding shop robot state, characterized in that: The method comprises the steps of: Collecting multi-source data of welding operation, and performing time alignment under a unified time grid to form aligned multi-channel data and a unified timestamp sequence; According to the unified timestamp sequence, performing recursive calculation of the aligned multi-channel data at each time point to obtain twin prediction data; By comparing the twin prediction data with the aligned multi-channel data at each time point, virtual-real contrast features are obtained, a set of key process sections is extracted according to the virtual-real contrast features, and a fast consistency ratio average and a welding stability index average are calculated; According to the fast consistency ratio average and the welding stability index average, a stability consistency function is constructed, the welding state of the key process section is determined by defining the welding state interval boundary, and the abnormal type of the key process section is identified; Based on the abnormal type of the key process section, an event trigger signal is generated, and a real-time adjustment path is executed to dynamically coordinate the energy control, motion control and visual acquisition control in the welding process.

2. The welding plant robot state real-time monitoring method based on digital twinning of claim 1, wherein: The welding operation multi-source data includes welding gun pose, welding gun motion speed, welding current, welding voltage, workpiece surface temperature distribution image and weld area gray scale image.

3. The welding plant robot state real-time monitoring method based on digital twinning of claim 2, wherein: The time alignment under the unified time grid includes interpolating the welding current, welding voltage and welding gun motion speed, performing time window matching on the workpiece surface temperature distribution image and the weld area gray scale image, and performing pose interpolation correction on the welding gun pose data; The aligned multi-channel data and the unified timestamp sequence are obtained under the unified time grid.

4. The welding plant robot state real-time monitoring method based on digital twinning of claim 3, wherein: The recursive calculation at each time point includes, at each unified timestamp, using the aligned multi-channel data of the previous unified timestamp as input, performing forward Euler integration on the welding gun pose and the welding gun motion speed to obtain twin pose prediction; Respectively performing one-step prediction on the welding current and the welding voltage to obtain twin current prediction and twin voltage prediction; Performing electrical-thermal equivalent incremental update on the workpiece surface temperature distribution image to obtain twin temperature prediction.

5. The welding plant robot state real-time monitoring method based on digital twinning of claim 4, wherein: The virtual-real contrast features include, at each unified timestamp, comparing the twin pose prediction, the twin current prediction, the twin voltage prediction, the twin temperature prediction and the aligned multi-channel data at each time point to obtain energy closure residual, temperature synchronization degree and weld area gray scale edge energy.

6. The welding plant robot state real-time monitoring method based on digital twinning of claim 5, wherein: The set of key process sections extracted according to the virtual-real contrast features includes calculating the trajectory curvature of the twin pose prediction according to the geometric relationship of adjacent twin pose predictions, and performing differential calculation on the trajectory curvature of adjacent twin pose predictions to obtain the trajectory curvature change rate of the twin pose prediction; Time sequence joint analysis is performed on the weld area gray scale edge energy and the trajectory curvature change rate of the twin pose prediction; When the weld area gray scale edge energy and the trajectory curvature change rate of the twin pose prediction at the same unified timestamp both reach local extreme values, the time point corresponding to the corresponding unified timestamp is determined as a key process section; All records of the time points corresponding to the unified timestamps that simultaneously reach local extreme values are combined to form a set of key process sections.

7. The welding plant robot state real-time monitoring method based on digital twinning of claim 6, wherein: The computing the quick consistency ratio includes calculating the quick consistency ratio by the ratio of the energy closure residual and the temperature synchronization degree; The computing the welding stability index includes synthesizing the quick consistency ratio, the weld area gray edge energy, and the trajectory curvature change rate of the twin pose prediction; The computing the quick consistency ratio mean value and the welding stability index mean value of each key process section in the key process section set.

8. The welding plant robot state real-time monitoring method based on digital twinning of claim 7, wherein: The defining the stability consistency function includes calculating the instantaneous stability change amount according to the difference of the welding stability indexes corresponding to adjacent uniform timestamps on the uniform timestamp sequence; The stability consistency function is constructed by combining the root mean square value of the instantaneous stability change amount with the welding stability index mean value and the quick consistency ratio mean value of each key process section. The computing the average value and the standard deviation of the stability consistency function of all key process sections in the same welding task, and defining the stable boundary of the welding state and the fluctuation boundary of the welding state based on the statistical distribution characteristics of the stability consistency function.

9. The welding plant robot state real-time monitoring method based on digital twinning of claim 8, wherein: The determining the welding state of the key process section and identifying the abnormal type of the key process section include: when the stability consistency function is not lower than the stable boundary of the welding state, determining that the welding state of the key process section is stable; when the stability consistency function is between the stable boundary of the welding state and the fluctuation boundary of the welding state, determining that the welding state of the key process section is slightly fluctuating; when the stability consistency function is lower than the fluctuation boundary of the welding state, determining that the welding state of the key process section is abnormal; For the key process section with the welding state being abnormal, analyzing the change relationship of the energy closure residual, the temperature synchronization degree, the weld area gray edge energy, and the trajectory curvature change rate of the twin pose prediction to identify the welding energy mismatch type abnormality, the welding gun posture disturbance type abnormality, the visual observation abnormality, and the composite abnormality.

10. The welding plant robot state real-time monitoring method based on digital twinning of claim 9, wherein: The dynamic coordination of the energy control, the motion control, and the visual acquisition control in the welding process includes generating an event trigger signal by taking the welding energy mismatch type abnormality, the welding gun posture disturbance type abnormality, the visual observation abnormality, and the composite abnormality as a trigger source; when the welding energy mismatch type abnormality is triggered, reading the twin voltage prediction and the twin current prediction at the current uniform timestamp, calculating a virtual welding power estimated value, and correcting the welding power setting power through a proportional adjustment function to obtain the welding power setting power, and outputting the welding power setting power to the welding power source; when the welding gun posture disturbance type abnormality is triggered, extracting the twin pose prediction at the current uniform timestamp, calculating a posture smoothing correction angle according to the change trend of the trajectory curvature change rate of the twin pose prediction, correcting the twin pose prediction to obtain a corrected posture target, and outputting the corrected posture target to the welding gun servo control unit to update the actual motion trajectory of the welding gun in synchronization with the uniform timestamp; when the visual observation abnormality is triggered, reading the weld area gray image and calculating the weld area gray edge energy, and if the weld area gray edge energy decreases in consecutive uniform timestamps, performing a visual acquisition link self-calibration operation. When the composite exception is triggered, the state of simultaneous deviation of energy closure residual, temperature synchronization degree, weld area gray edge energy and twin pose prediction trajectory curvature change rate is identified, and the real-time adjustment path corresponding to the welding energy mismatch type exception, the welding gun posture disturbance type exception and the visual observation exception is called respectively, and the synchronous coordination operation is executed under the unified timestamp, and when the monitoring quantity returns to the stable interval, it is determined that the composite exception is removed.

Images