A welding workshop robot state real-time monitoring method based on digital twinning

Abstract

The application discloses a kind of based on digital twin's welding workshop robot state real-time monitoring method, it is related to intelligent welding monitoring technical field, including, by twin prediction data and after alignment multi-channel data are compared with each hour, obtain virtual real contrast feature, according to virtual real contrast feature extraction key process section set, and calculate fast consistency ratio mean and welding stability index mean;According to fast consistency ratio mean and welding stability index mean, construct stability consistency function, define welding state interval boundary, determine the welding state of key process section, and identify the abnormal type of key process section;Based on the abnormal type of key process section, generate event trigger signal, execute real-time adjustment path, carry out dynamic coordination to energy control, motion control and vision acquisition control in welding process.The application realizes key process section state identification and abnormal classification, guarantees welding process stable operation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent welding monitoring, and particularly relates to a welding workshop robot state real-time monitoring method based on digital twinning. BACKGROUND

[0002] The welding workshop robot is important equipment for realizing high-precision welding and high-efficiency production in modern manufacturing industry, and is widely applied to manufacturing processes of automobile structural parts, engineering machinery and rail transit equipment. The conventional welding state monitoring method usually relies on a multi-source sensor system, acquires data such as welding gun pose, welding gun movement speed, welding current, welding voltage, workpiece surface temperature distribution and weld area image, and realizes welding quality determination and process control in combination with time synchronization and signal analysis. In recent years, with the development of industrial Ethernet, machine vision and real-time computing technology, multi-dimensional perception and information fusion of the welding process gradually become an important direction for realizing welding process visualization and dynamic evaluation, and provide a basic support for welding stability research and process optimization.

[0003] In the existing welding monitoring method, on the one hand, the relationship between welding energy input and thermal response is determined in a single data channel or a fixed threshold determination mode, which is difficult to reflect the dynamic coupling law of energy transmission and heat diffusion; on the other hand, the correlation analysis between the welding gun pose change and the weld area forming feature is insufficient, and there is a lack of recognition and section positioning capability for the synchronous change characteristics among multi-source information. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides a welding workshop robot state real-time monitoring method based on digital twinning, which solves the problems of insufficient real-time coupling determination of welding process energy and thermal response and insufficient correlation recognition of welding gun pose change and weld area gray image feature.

[0006] To solve the above technical problems, the present application provides the following technical scheme:

[0007] The present application provides a welding workshop robot state real-time monitoring method based on digital twinning, which includes collecting multi-source data of welding operation, and completing time alignment under a unified time grid to form aligned multi-channel data and a unified timestamp sequence;

[0008] According to the unified timestamp sequence, the aligned multi-channel data is calculated recursively at each time point to obtain twin prediction data;

[0009] By comparing the twin prediction data with the aligned multi-channel data at each time point, virtual-real contrast features are obtained, a key process section set is extracted according to the virtual-real contrast features, and a fast consistency ratio mean and a welding stability index mean are calculated.

[0010] According to the fast consistency ratio mean value and the welding stability index mean value, a stability consistency function is constructed, a welding state interval boundary is defined, a welding state of the key process section is judged, and an abnormal type of the key process section is identified;

[0011] Based on the abnormal type of the key process section, an event trigger signal is generated, a real-time adjustment path is executed, and energy control, motion control and visual acquisition control in the welding process are dynamically coordinated.

[0012] As a preferred scheme of the welding shop robot state real-time monitoring method based on digital twinning, wherein: the welding operation multi-source data includes welding gun pose, welding gun motion speed, welding current, welding voltage, workpiece surface temperature distribution image and weld area gray image.

[0013] As a preferred scheme of the welding shop robot state real-time monitoring method based on digital twinning, wherein: the time alignment under the unified time grid includes: interpolating welding current, welding voltage and welding gun motion speed, performing time window matching on workpiece surface temperature distribution image and weld area gray image, and performing pose interpolation correction on welding gun pose data.

[0014] The aligned multi-channel data and the unified timestamp sequence are obtained under the unified time grid.

[0015] As a preferred scheme of the welding shop robot state real-time monitoring method based on digital twinning, wherein: the recursive calculation at each time instant to obtain the twin prediction data includes: at each unified timestamp, using the aligned multi-channel data of the previous unified timestamp as input, performing speed forward Euler integral on the welding gun pose and the welding gun motion speed to obtain the twin pose prediction.

[0016] The welding current and the welding voltage are respectively executed one-step prediction of the holding type to obtain the twin current prediction and the twin voltage prediction.

[0017] The workpiece surface temperature distribution image is executed electric heat equivalent incremental update to obtain the twin temperature prediction.

[0018] As a preferred scheme of the welding shop robot state real-time monitoring method based on digital twinning, wherein: the virtual-real contrast features include: at each unified timestamp, comparing the twin pose prediction, the twin current prediction, the twin voltage prediction, the twin temperature prediction and the aligned multi-channel data at each time instant to obtain energy closure residual, temperature synchronization degree and weld area gray edge energy.

[0019] As a preferred scheme of the welding workshop robot state real-time monitoring method based on digital twinning provided in the application, wherein: the key process section set extracted according to the virtual-real comparison features comprises: according to the geometric relationship of adjacent twin pose predictions, the trajectory curvature of the twin pose predictions is calculated, and the trajectory curvature of the adjacent twin pose predictions is differentially calculated to obtain the trajectory curvature change rate of the twin pose predictions.

[0020] The weld area gray edge energy and the trajectory curvature change rate of the twin pose predictions are jointly analyzed in time series.

[0021] When the weld area gray edge energy and the trajectory curvature change rate of the twin pose predictions reach local extreme values at the same uniform timestamp, it is determined that the time point corresponding to the uniform timestamp is a key process section.

[0022] All records of the time points corresponding to the uniform timestamps reaching local extreme values at the same time are combined to form a key process section set.

[0023] As a preferred scheme of the welding workshop robot state real-time monitoring method based on digital twinning provided in the application, wherein: the calculation of the fast consistency ratio and the welding stability index average value comprises: the fast consistency ratio is calculated by the ratio of the energy closure residual and the temperature synchronization degree.

[0024] The fast consistency ratio, the weld area gray edge energy and the trajectory curvature change rate of the twin pose predictions are combined to calculate the welding stability index.

[0025] In the key process section set, the fast consistency ratio average value and the welding stability index average value of each key process section are calculated.

[0026] As a preferred scheme of the welding workshop robot state real-time monitoring method based on digital twinning provided in the application, wherein: the construction of the stability consistency function and the definition of the welding state interval boundary comprise: on the uniform timestamp sequence, according to the difference value of the welding stability index corresponding to adjacent uniform timestamps, the instantaneous stability change amount is calculated.

[0027] The mean square root value of the instantaneous stability change amount is combined with the welding stability index average value and the fast consistency ratio average value of each key process section to construct the stability consistency function.

[0028] In the same welding task, the average value and the standard deviation of the stability consistency function of all key process sections are calculated, and the stability boundary of the welding state and the fluctuation boundary of the welding state are defined based on the statistical distribution characteristics of the stability consistency function.

[0029] As a preferred scheme of the welding shop robot state real-time monitoring method based on digital twinning provided in the application, wherein: the welding state of the key process section is determined, and the type of abnormality of the key process section is identified, including: when the stability consistency function is not lower than the stability boundary of the welding state, it is determined that the welding state of the key process section is stable;

[0030] When the stability consistency function is between the stability boundary of the welding state and the fluctuation boundary of the welding state, it is determined that the welding state of the key process section is slightly fluctuating;

[0031] When the stability consistency function is lower than the fluctuation boundary of the welding state, it is determined that the welding state of the key process section is abnormal;

[0032] For the key process section with abnormal welding state, the change relationship of the energy closure residual, the temperature synchronization degree, the weld area gray edge energy and the trajectory curvature change rate of the twin pose prediction is analyzed, and the types of welding energy mismatch type abnormality, welding gun posture disturbance type abnormality, visual observation abnormality and composite abnormality are identified.

[0033] As a preferred scheme of the welding shop robot state real-time monitoring method based on digital twinning provided in the application, wherein: the dynamic coordination of the energy control, the motion control and the visual acquisition control in the welding process includes: taking the welding energy mismatch type abnormality, the welding gun posture disturbance type abnormality, the visual observation abnormality and the composite abnormality as a trigger source to generate an event trigger signal;

[0034] When the welding energy mismatch type abnormality is triggered, the twin voltage prediction and the twin current prediction under the current unified timestamp are read, the virtual welding power estimated value is calculated, the welding power source setting power is obtained by correcting through a proportional adjustment function, and the welding power source setting power is output to the welding power source;

[0035] When the welding gun posture disturbance type abnormality is triggered, the twin pose prediction under the current unified timestamp is extracted, the posture smoothing correction angle is calculated according to the change trend of the trajectory curvature change rate of the twin pose prediction, the twin pose prediction is corrected to obtain a corrected posture target, the corrected posture target is output to the welding gun servo control unit, and the actual motion trajectory of the welding gun is updated synchronously with the unified timestamp;

[0036] When the visual observation abnormality is triggered, the weld area gray image is read and the weld area gray edge energy is calculated, and if the weld area gray edge energy decreases in a plurality of continuous unified timestamps, a visual acquisition link self-correction operation is performed;

[0037] When the composite exception is triggered, the state of the energy closure residual, the temperature synchronization degree, the weld area gray edge energy and the twin pose prediction trajectory curvature change rate deviating at the same time is identified, the real-time adjustment path corresponding to the welding energy mismatch type exception, the welding gun posture disturbance type exception and the visual observation exception is called respectively, and the synchronous coordination operation is executed under the unified timestamp, when the monitoring quantity recovers to the stable interval, the composite exception is determined to be removed.

[0038] The present application has the beneficial effects that: through the cooperative analysis mechanism of recursive calculation and virtual-real comparison of multi-channel data, the synchronous correlation of welding energy transmission, welding gun posture change and workpiece thermal response is realized; through the construction of stability consistency function and the introduction of welding state interval boundary judgment mechanism, the state recognition and exception typing of key process sections are realized, so as to guarantee the stable operation of welding process and the high precision consistency of monitoring results. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0040] Fig. 1 The flowchart of the real-time monitoring method of the welding workshop robot state based on digital twin.

[0041] Fig. 2 The flowchart of recursively calculating the twin prediction data at each moment.

[0042] Fig. 3 The flowchart of obtaining virtual-real comparison features and calculating index mean.

[0043] Fig. 4 The flowchart of welding state determination and real-time adjustment. DETAILED DESCRIPTION

[0044] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings of the specification.

[0045] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the present application, therefore the present application is not limited to the specific embodiments disclosed below.

[0046] Second, the "one embodiment" or "an embodiment" referred to herein can include, but is not limited to, any particular feature, structure, or characteristic described herein. Embodiments described herein may

[0047] Referring to Figs. 1-4 For one embodiment of the present application, the embodiment provides a welding shop robot state real-time monitoring method based on digital twinning, comprising the following steps:

[0048] S1, collect welding operation multi-source data, and complete time alignment under a unified time grid to form aligned multi-channel data and a unified timestamp sequence.

[0049] Further, the welding operation multi-source data is collected through an industrial Ethernet (such as an OPC UA protocol) and under a unified clock (such as an IEEE 1588 PTP protocol), with the same time reference and carrying a unified timestamp, including a welding gun pose, a welding gun movement speed, a welding current, a welding voltage, a workpiece surface temperature distribution image, and a weld seam area grayscale image.

[0050] The welding gun pose refers to the position and pose of the welding gun tool center point in space; the welding gun movement speed refers to the moving speed and rotating speed of the welding gun tool center point in space; the welding current refers to the welding power output current; the welding voltage refers to the welding power output voltage; the workpiece surface temperature distribution image refers to a temperature image collected by an infrared thermal imaging camera and corrected by radiation, background reflection, and distance compensation; and the weld seam area grayscale image refers to a single-channel grayscale image collected by a visible light industrial camera and processed by exposure and gain locking, constant light source, or anti-flicker.

[0051] Further, the welding operation multi-source data is time-aligned under a unified clock according to a unified time grid.

[0052] Linear interpolation or sample resampling is performed on the welding current, the welding voltage, and the welding gun movement speed, so that the welding current, the welding voltage, and the welding gun movement speed corresponding to each unified timestamp remain synchronized in the time dimension.

[0053] Time window matching is performed on the workpiece surface temperature distribution image and the weld seam area grayscale image, and synchronous matching is performed at the closest collection frame to the unified timestamp.

[0054] Pose interpolation correction is performed on the welding gun pose data, and pose smooth transition is performed between adjacent unified timestamps, so that the position and pose of the welding gun tool center point remain continuous and consistent under the unified time reference.

[0055] The time reference correction is periodically performed under the unified clock, the synchronization offset of the multi-source data of various welding operations under the unified time grid is corrected, and the aligned multi-channel data under the unified time stamp range is kept logically consistent.

[0056] The time alignment accuracy is checked through the time difference inspection of the unified time stamp sequence, and the time drift accumulation caused by the difference in collection path or network delay is avoided.

[0057] Finally, the aligned multi-channel data (welding gun pose, welding gun movement speed, welding current, welding voltage, workpiece surface temperature distribution image, and welding seam area grayscale image) under the unified time grid and the corresponding unified time stamp sequence are formed.

[0058] It should be noted that the unified clock is used to maintain the time synchronization of the multi-source data collection process of the welding operation, the same time reference is used to constrain the time consistency of various welding operation multi-source data at the collection time, and the unified time grid is used to organize the multi-channel data under the unified clock and the same time reference and perform time alignment.

[0059] S2, according to the unified time stamp sequence, the aligned multi-channel data is calculated recursively at each time, and the twin prediction data is obtained.

[0060] Further, according to the unified time stamp sequence, the aligned multi-channel data is calculated recursively at each time, and the twin prediction data is obtained.

[0061] At each unified time stamp, the aligned multi-channel data of the previous unified time stamp is inputted, and steps D1-D4 are performed.

[0062] D1, the speed forward Euler integral is performed on the welding gun pose and the welding gun movement speed, and the twin pose prediction is defined, which is represented as:

[0063] ;

[0064] Wherein, represents the twin pose prediction at time , represents the welding gun pose at time , represents the welding gun movement speed at time , represents the interval between adjacent unified time stamps, represents the first time in the unified time stamp sequence, represents the first time in the unified time stamp sequence, represents the first time in the unified time stamp sequence, at time

[0065] D2, performing a hold-type one-step prediction on the welding current, defining a twin current prediction, denoted as

[0066] ;

[0067] wherein denotes the twin current prediction at time , denotes the welding current at time .

[0068] D3, performing a hold-type one-step prediction on the welding voltage, defining a twin voltage prediction, denoted as

[0069] ;

[0070] wherein denotes the twin voltage prediction at time , denotes the welding voltage at time .

[0071] D4, performing an electro-thermal equivalent incremental update on the workpiece surface temperature distribution image, defining a twin temperature prediction, denoted as

[0072] ;

[0073] ;

[0074] ;

[0075] wherein denotes the twin temperature prediction at time at spatial position , denotes the twin temperature prediction at time at spatial position , denotes the equivalent temperature rise coefficient for spatial position , denotes the electro-thermal equivalent coefficient, denotes the welding voltage at time , denotes the welding current at time , denotes the time variable in the integral calculation, denotes the workpiece surface temperature at time at spatial position , denotes the workpiece surface temperature at time at spatial position The surface temperature of the workpiece. Indicated by time The right endpoint is the window length is The start time of the time interval, Indicates the length of the calibrated sliding window. Indicates the density of the material. This indicates the specific heat at constant pressure of the material. This represents the equivalent thermal conductivity thickness of the workpiece in the direction of thermal conduction. This represents the imaging coverage area of ​​the workpiece surface temperature distribution image. This indicates the area of ​​the workpiece surface region.

[0076] Repeat steps D1-D4 along the unified timestamp sequence to form twin prediction data that correspond one-to-one with the unified timestamp.

[0077] It should be noted that the length of the calibrated sliding window is... The value is determined based on the matching relationship between the energy input cycle of the welding process and the thermal diffusion response characteristics of the workpiece, with an example value range of 0.3~0.8 seconds. Specifically, during the operation of the welding robot, the welding current and welding voltage fluctuate with the power supply frequency in a short period, while the change in workpiece surface temperature is affected by the material's thermal conductivity, thickness, and thermal diffusion time constant, typically reaching thermal stability within hundreds of milliseconds. If the calibrated sliding window length is too short, the calculated temperature rise result will be affected. It is susceptible to transient fluctuations; if the calibration sliding window length is too long, it will span the arc initiation, stable welding and arc termination stages of the weld, resulting in a lag in thermal response; combined with the typical arc fluctuation period of the welding process (about 2.5 to 5 milliseconds) and the workpiece heat diffusion time (about 0.2 to 0.7 seconds), the example sets the calibration sliding window length in the range of 0.3 to 0.8 seconds, which can make the temperature rise calculation results smooth transient interference and timely reflect the real changes in welding heat input.

[0078] S3. By comparing the twin prediction data with the aligned multi-channel data time-by-time, virtual-real comparison features are obtained. Based on the virtual-real comparison features, the set of key process sections is extracted, and the mean of rapid consistency ratio and welding stability index is calculated.

[0079] Furthermore, according to the unified timestamp sequence, the twin pose prediction, twin current prediction, twin voltage prediction, twin temperature prediction, and aligned multi-channel data are compared time-by-time to calculate virtual-real comparison characteristics, including energy closure residual, temperature synchronization degree, and gray-scale edge energy of the weld area. The set of key process sections is extracted based on the change of gray-scale edge energy of the weld area and the change rate of trajectory curvature of the twin pose prediction.

[0080] The virtual welding energy input is calculated by the twin voltage prediction and the twin current prediction, and compared with the heat absorption amount corresponding to the workpiece surface temperature distribution image, to calculate the energy closure residual, which is represented as:

[0081] ;

[0082] wherein, represents the energy closure residual at time , and the value range is , represents the twin voltage prediction at time , represents the twin current prediction at time , represents the workpiece surface temperature at time at spatial position .

[0083] When the energy closure residual is close to 0 (for example, less than 10 joules), it indicates that the virtual welding energy input calculated by the twin voltage prediction and the twin current prediction is basically consistent with the heat absorption amount reflected by the workpiece surface temperature distribution image, and the welding energy transmission is stable.

[0084] When the energy closure residual rises in a small range (for example, the absolute value of the difference between the energy closure residuals corresponding to adjacent uniform timestamps is not more than 15% of the energy closure residual at the last uniform timestamp), it indicates that there is a slight difference between the virtual welding energy input and the heat absorption amount reflected by the workpiece surface temperature distribution image, which is usually caused by the change of the electrothermal equivalent coefficient or the deviation of the heat conduction parameter , and the welding process is in a quasi-stable state.

[0085] When the energy closure residual continuously increases (for example, the difference between the energy closure residuals corresponding to adjacent uniform timestamps exceeds 30% of the energy closure residual at the last uniform timestamp), it indicates that the difference between the virtual welding energy input calculated by the twin voltage prediction and the twin current prediction and the heat absorption amount reflected by the workpiece surface temperature distribution image is enlarged, the stability of the welding arc decreases or the heat conduction characteristics of the workpiece change, and the welding energy transmission no longer matches.

[0086] It should be noted that the numerical values in the energy closure residual determination condition are exemplary selected, and are determined according to the dynamic coupling law between the virtual welding energy input and the workpiece surface heat absorption amount, and the typical fluctuation characteristics of the welding energy input and the workpiece thermal response: wherein 10 joules is about 5% of the total amount of virtual welding energy input per unit time, used to distinguish the difference between the virtual welding energy input and the workpiece surface heat absorption amount and the energy difference caused by normal heat radiation loss; 15% corresponds to the conventional dynamic fluctuation proportion between the virtual welding energy input and the workpiece surface heat absorption amount, used to determine that the virtual welding energy input matches the workpiece surface heat absorption amount and the welding process is in a quasi-stable state; 30% represents the determination proportion of the continuous expansion of the difference between the virtual welding energy input and the workpiece surface heat absorption amount, used to identify the thermal response deviation caused by welding energy transmission mismatch and uneven arc energy; the above numerical values can be adapted and adjusted according to the welding process type, welding arc power level, workpiece thermal conductivity characteristics and electric heat equivalent coefficient correction conditions, to ensure the consistency of the energy closure residual determination result and the actual welding energy transmission state.

[0087] By comparing the temperature prediction and the workpiece surface temperature distribution image point by point, the temperature synchronization degree is calculated, which is represented as:

[0088]

[0089] represents the temperature synchronization degree at time , and the value range is .

[0090] When the temperature synchronization degree approaches 1 (for example, greater than 0.9), it indicates that the twin temperature prediction and the workpiece surface temperature distribution image are highly consistent in spatial distribution and change trend, the synchronization accuracy of the virtual temperature field and the actual temperature field is high, and the thermal diffusion process of the welding area is stable.

[0091] When the temperature synchronization degree is in the middle interval (for example, 0.4 to 0.7), it indicates that there is a certain deviation between the twin temperature prediction and the workpiece surface temperature distribution image, the consistency of the temperature distribution decreases, and the thermal response exists lag.

[0092] When the temperature synchronization degree is in the lower interval (for example, 0 to 0.4), it indicates that the twin temperature prediction and the workpiece surface temperature distribution image have large differences, and the virtual temperature field and the actual temperature field are out of synchronization.

[0093] By analyzing the gradient intensity of the weld area gray scale image, the weld area gray scale edge energy is calculated, which is represented as:

[0094] ​​​

[0095] wherein, represents the weld zone gray scale edge energy at time , with a value range of , represents the gradient vector of the weld zone gray scale image at pixel position at time , represents the pixel coverage domain of the weld zone gray scale image, represents the pixel domain area of the weld zone.

[0096] The gradient vector is composed of the gray scale change rate of the weld zone gray scale image in the horizontal pixel coordinate and the vertical pixel coordinate.

[0097] When the weld zone gray scale edge energy is at a high level (for example, it remains higher than the average value of the weld zone gray scale edge energy corresponding to the adjacent uniform timestamp by more than 20% within a plurality of consecutive uniform timestamps), it indicates that the weld pool boundary is clear, the energy input is sufficient, and the welding appearance is stable.

[0098] When the weld zone gray scale edge energy fluctuates increases (for example, the relative change amplitude between the weld zone gray scale edge energy values of adjacent uniform timestamps exceeds 30%), it indicates that the weld pool shape changes rapidly, and there may be arc swing abnormalities or welding gun posture deviation.

[0099] When the weld zone gray scale edge energy continuously decreases (for example, the average decrease amplitude of the weld zone gray scale edge energy corresponding to adjacent uniform timestamps exceeds 25%), it indicates that the weld zone boundary is blurred, the weld pool shrinks or the energy input is insufficient, and the weld forming quality decreases.

[0100] It should be noted that the percentage values in the weld seam area gray edge energy change determination condition are all exemplary selected values, which are determined according to the gradient intensity distribution change rule of the weld seam area gray image and the weld pool light intensity distribution stability: wherein, 20% corresponds to the proportion of the weld seam area gray edge energy being higher than the average value of the weld seam area gray edge energy of the adjacent uniform time stamp in the continuous multiple uniform time stamps, which is used to determine the state that the weld pool boundary is clear, the virtual welding energy input matches the workpiece surface heat absorption, and the welding appearance is stable; 30% corresponds to the relative change amplitude between the weld seam area gray edge energy values of the adjacent uniform time stamps, which is used to determine the state that the weld pool shape changes rapidly, the welding gun posture changes frequently or the arc swings unstably; 25% corresponds to the average decrease amplitude of the weld seam area gray edge energy of the adjacent uniform time stamps, which is used to determine the state that the weld seam area boundary is blurred, the virtual welding energy input is weakened or the weld pool is contracted; the above percentage values can be adapted and adjusted according to the welding current fluctuation range, the welding voltage stability, the gray dynamic response characteristics of the infrared thermal imaging camera and the visible light industrial camera, so as to ensure that the determination result of the weld seam area gray edge energy change is consistent with the actual welding working condition.

[0101] On the uniform time stamp sequence, according to the geometric relationship of the adjacent twin pose predictions, the trajectory curvature of the twin pose predictions is calculated, which is represented as:

[0102] ;

[0103] wherein, represents the trajectory curvature of the twin pose prediction at time , represents the trajectory curvature of the twin pose prediction at time , represents the vector module length operator.

[0104] The trajectory curvatures of the adjacent twin pose predictions are differentially calculated to obtain the trajectory curvature change rate of the twin pose predictions, which is represented as:

[0105] ;

[0106] wherein, represents the trajectory curvature change rate of the twin pose prediction at time , the value range is , which is used to reflect the smoothness of the welding gun posture change, represents a very small positive constant to prevent the denominator from being zero, for example, .

[0107] When the trajectory curvature change rate of the twin pose prediction is When there is a rapid rise or frequent fluctuation, it indicates that the trajectory of the welding torch changes rapidly, which may be accompanied by weld fluctuation or abnormal heat input.

[0108] The gray edge energy of the weld area and the trajectory curvature rate of the twin pose prediction are analyzed in time series. When the gray edge energy of the weld area and the trajectory curvature rate of the twin pose prediction reach local extrema at the same unified timestamp, the time point corresponding to the unified timestamp is determined as a key process section, and all records of the time points corresponding to the unified timestamp that reach local extrema at the same time are formed into a key process section set.

[0109] The local extrema reached at the same time means that the time series of the gray edge energy of the weld area and the trajectory curvature rate of the twin pose prediction are calculated on the unified timestamp sequence. When the gray edge energy of the weld area reverses the monotonic change direction with respect to the gray edge energy of the weld area of the previous and next unified timestamps, and the trajectory curvature rate of the twin pose prediction also reverses the monotonic change direction with respect to the trajectory curvature rate of the twin pose prediction of the previous and next unified timestamps, the time point corresponding to the unified timestamp is determined as the time point at which the gray edge energy of the weld area and the trajectory curvature rate of the twin pose prediction reach local extrema at the same time.

[0110] At this time, the time point corresponding to the unified timestamp represents the synchronous response characteristics of the change of the gray edge energy of the weld area and the change of the pose of the welding torch, which is used to determine the key process section.

[0111] It should be noted that the twin prediction data is formed by performing recursive calculation of the aligned multi-channel data at each time, and the virtual-real comparison features are constructed by the energy closure residual, the temperature synchronization degree, and the gray edge energy of the weld area, so as to realize the synchronous comparison and dynamic correlation of the virtual-real data of the welding process.

[0112] Furthermore, the fast consistency ratio is defined by the ratio of the energy closure residual and the temperature synchronization degree, which is represented as:

[0113] ;

[0114] Wherein, represents the fast consistency ratio at time , the value range is , which is used to reflect the coupling degree between the welding energy and the thermal response, represents the welding energy normalization constant (for example, 100 joules), which is used to eliminate the dimensional difference.

[0115] When the fast consistency ratio When the value is close to 1, it indicates that the virtual welding energy input is consistent with the energy transfer in the actual welding process, the virtual thermal response corresponds well with the actual thermal response, and the welding energy transfer process is stable.

[0116] When the fast consistency ratio When the value decreases, it indicates that there is a deviation between the virtual welding energy input and the thermal response of the actual welding process, the correspondence between the virtual welding energy input and the changes in the temperature distribution on the workpiece surface decreases, and the stability of welding energy transfer decreases.

[0117] It should be noted that the decrease in the rapid consistency ratio refers to the fact that, within a continuous range of unified timestamps, the change in the rapid consistency ratio between adjacent unified timestamps is negative and the magnitude of the change gradually increases. In other words, the rapid consistency ratio shows a continuous decreasing trend and tends to accelerate across multiple consecutive unified timestamps. This trend reflects the continuous weakening of the correspondence between virtual welding energy input and the change in workpiece surface temperature distribution, and the continuous reduction in the dynamic matching degree between virtual welding energy input and workpiece surface heat absorption, thereby leading to a decrease in the stability of welding energy transfer.

[0118] Within each unified timestamp, the consistency between virtual and real energy and thermal response is characterized by the fast consistency ratio, the stability of weld formation characteristics is characterized by the gray-scale edge energy of the weld region, and the smoothness of the welding torch attitude is characterized by the trajectory curvature change rate predicted by twin pose.

[0119] The welding stability index is calculated by combining the rapid consistency ratio, the gray-scale edge energy of the weld area, and the trajectory curvature change rate predicted by twin pose, and is expressed as:

[0120] ;

[0121] in, Indicates time The welding stability index, with a value range of [value range missing]. It is used to characterize the overall operating status of the welding process. This represents the maximum value of the grayscale edge energy in the weld area during the current welding task.

[0122] When welding stability index A value close to 1 indicates stable welding posture, uniform welding energy input, and stable forming quality.

[0123] When welding stability index A continuous decline (e.g., the welding stability index corresponding to adjacent unified timestamps is less than 0.5, and the average decline is greater than 20% of the welding stability index corresponding to the previous unified timestamp) indicates that there is an abnormality in the welding posture or welding energy during the welding process.

[0124] In the set of key process segments, the welding stability index average value of each key process segment is calculated and represented as:

[0125] ;

[0126] wherein, represents the welding stability index average value of the i-th key process segment, represents the start time of the i-th key process segment, represents the end time of the i-th key process segment, represents the welding stability index at time t.

[0127] Using the same method, the fast consistency ratio average value of each key process segment in the set of key process segments is calculated.

[0128] S4, according to the fast consistency ratio average value and the welding stability index average value, a stability consistency function is constructed, the welding state interval boundary is defined, the welding state of the key process segment is determined, and the abnormal type of the key process segment is identified.

[0129] Further, within each unified timestamp, the instantaneous stability change amount is calculated according to the difference value of the welding stability index corresponding to adjacent unified timestamps, and is represented as:

[0130] ;

[0131] wherein, represents the instantaneous stability change amount at time t, which is used to evaluate the dynamic fluctuation degree of the welding state. When the instantaneous stability change amount

[0132] remains negative and the absolute value increases in a plurality of consecutive unified timestamps, it indicates that the welding process stability is declining. When the instantaneous stability change amount

[0133] changes from negative to positive, it indicates that the welding process is restored to stability.

[0134] In the set of key process segments, the root mean square value of the instantaneous stability change amount is calculated.

[0135] The root mean square value of the instantaneous stability change amount is combined with the welding stability index average value and the fast consistency ratio average value of each key process segment to construct a stability consistency function, which is represented as:

[0136] ;

[0137] wherein,​​​​​ the stability consistency function of the i-th key process segment, taking a value in the range of the rapid consistency ratio mean value of the i-th key process segment, the root mean square value of the instantaneous stability change of the i-th key process segment.

[0138] To prevent extreme noise from causing the stability consistency function to be negative, the suppression factor in the stability consistency function is limited to and the suppression factor is limited to 0 when it is less than 0.

[0139] In the same welding task, the average value and the standard deviation of the stability consistency function of all key process segments are calculated, and the interval boundaries of the welding state are defined based on the statistical distribution characteristics of the stability consistency function, including the stability boundary of the welding state and the fluctuation boundary of the welding state, represented as:

[0140] ;

[0141] ;

[0142] ;

[0143] wherein, represents the stability boundary of the welding state, represents the fluctuation boundary of the welding state, represents the average value of the stability consistency function of all key process segments, used to represent the overall stability level of the current welding task, represents the standard deviation of the stability consistency function of all key process segments, used to reflect the fluctuation amplitude of the welding stability, represents the stability determination sensitivity coefficient, represents the temperature synchronization degree average value calculated within the range of all unified timestamps, used to represent the overall consistency of the virtual temperature field and the actual temperature field during the entire welding task, represents the variance of the energy closure residual.

[0144] When , it is determined that the welding state of the key process segment is stable.

[0145] When , it is determined that the welding state of the key process segment is slightly fluctuating.

[0146] When , it is determined that the welding state of the key process segment is abnormal. ​​​​

[0147] Further, for the key process segments with abnormal welding states, the change relationship of the energy closure residual, the temperature synchronization degree, the weld area gray edge energy and the trajectory curvature change rate of the twin pose prediction is analyzed, the abnormal type of each abnormal key process segment is identified, including welding energy mismatch type abnormality, welding gun pose disturbance type abnormality, visual observation abnormality and composite abnormality.

[0148] When the energy closure residual continuously increases, and the temperature synchronization degree is in a lower interval or moves from an intermediate interval to a lower interval (a downward trend occurs), it indicates that the deviation between the virtual welding energy input and the workpiece surface temperature response gradually expands, and the heat input and heat diffusion are mismatched; if the weld area gray edge energy is at a high level or relatively stable at this time, and the trajectory curvature change rate of the twin pose prediction changes smoothly (without rapid rise or frequent fluctuations), it is determined as a welding energy mismatch type abnormality caused by a change in thermal conductivity.

[0149] When the energy closure residual rises within a small range or fluctuates slightly, but the temperature synchronization degree is close to 1 or remains in the intermediate interval, and the weld area gray edge energy fluctuates and increases, and the trajectory curvature change rate of the twin pose prediction rapidly rises or frequently fluctuates, it indicates that the electric heat transfer is stable but the welding gun pose changes dramatically, and it is determined as a welding gun pose disturbance type abnormality.

[0150] When the energy closure residual is close to 0 or rises within a small range, and the temperature synchronization degree is close to 1 or remains in the intermediate interval, and the weld area gray edge energy continuously decreases, and the trajectory curvature change rate of the twin pose prediction changes smoothly, it indicates that the weld area gray image acquisition appears to be attenuated or blocked, and it is determined as a visual observation abnormality.

[0151] When the energy closure residual continuously increases, the temperature synchronization degree is in a lower interval, the weld area gray edge energy fluctuates and increases, and the trajectory curvature change rate of the twin pose prediction rapidly rises or frequently fluctuates, it indicates that the welding energy input, heat diffusion and welding gun pose change are jointly unstable, and it is determined as a composite abnormality.

[0152] It should be noted that by calculating the fast consistency ratio mean and the welding stability index mean in the key process segment set, a stability consistency function is constructed and the welding state interval boundary is defined, the welding state of the key process segment is determined and the abnormal type is identified, so as to dynamically coordinate the welding energy, the welding gun pose and the visual acquisition, and improve the stability and real-time monitoring accuracy of the welding process.

[0153] S5, based on the abnormal type of the key process segment, an event trigger signal is generated, and a real-time adjustment path is executed to dynamically coordinate the energy control, motion control and visual acquisition control in the welding process.

[0154] Further, in each unified timestamp, according to the abnormal type of each abnormal key process section, an event trigger signal is generated.

[0155] The event trigger signal takes the welding energy mismatch type abnormality, the welding gun posture disturbance type abnormality, the visual observation abnormality and the composite abnormality as the trigger source, and corresponds to different real-time adjustment paths respectively.

[0156] Further, when the welding energy mismatch type abnormality is triggered, the twin voltage prediction and the twin current prediction at the current time are read, a virtual welding power estimation value is calculated, and is represented as:

[0157] ;

[0158] wherein, represents the virtual welding power estimation value at time .

[0159] The virtual welding power estimation value is corrected by a proportional adjustment function to obtain a welding power source set power, and is represented as:

[0160] ;

[0161] wherein, represents the welding power source set power at time .

[0162] The welding power source set power is output to the welding power source through the PLC to realize the micro compensation adjustment of the real-time arc energy.

[0163] Further, when the welding gun posture disturbance type abnormality is triggered, the twin pose prediction at the current time is extracted, and according to the change trend of the trajectory curvature change rate, a posture smoothing correction angle is calculated, and is represented as:

[0164] ;

[0165] wherein, represents the posture smoothing correction angle at time .

[0166] The twin pose prediction is corrected in posture by the posture smoothing correction angle to obtain a corrected posture target, and is represented as:

[0167] ;

[0168] wherein, represents the corrected posture target at time .

[0169] The corrected pose target is output to a welding gun servo control unit, and the welding gun actual motion trajectory is synchronously updated with a unified timestamp, so that real-time fine adjustment of the welding gun pose and smooth recovery of the trajectory are realized.

[0170] Further, when the visual observation anomaly is triggered, the weld area gray image is read and the weld area gray edge energy is calculated, and according to the change trend of the weld area gray edge energy over time, it is judged whether the weld area gray image acquisition process exists attenuation or shielding.

[0171] When the weld area gray edge energy significantly decreases in a plurality of consecutive unified timestamps, a visual acquisition link self-checking operation is performed, the exposure time and gain parameters of the visible light industrial camera are adjusted in turn, and the light source stability detection is re-performed.

[0172] When the weld area gray edge energy recovers to a normal level or significantly increases compared with the previous period within a certain number of acquisition periods after the self-checking operation, it is determined that the weld area gray image acquisition process is normal, and the visual observation abnormal state is released.

[0173] When the weld area gray edge energy does not recover in two consecutive acquisition periods, it is determined that the weld area gray image acquisition process is still unstable, the weld area gray image is marked as low-confidence data, and only the weld area gray image is recorded and archived in the subsequent process, and the weld area gray edge energy is no longer updated in real time, so as to prevent the weld area gray edge energy from being abnormal, and to interfere with the judgment accuracy of the weld appearance change and the calculation result of the welding stability index.

[0174] Through the joint adjustment of the exposure time, gain parameters and weld area gray image data participation mode, the continuity and data stability of the weld area gray image data acquisition are maintained under the conditions of weld area gray image acquisition being shielded, light fluctuation or exposure abnormality, and the welding state monitoring process is ensured not to be interrupted.

[0175] Further, when the composite anomaly is triggered, the state in which the energy closure residual, the temperature synchronization degree, the weld area gray edge energy and the trajectory curvature change rate of the twin pose prediction simultaneously deviate significantly is identified.

[0176] In the case of composite anomaly, the real-time adjustment paths corresponding to the welding energy mismatch type anomaly, the welding gun pose disturbance type anomaly and the visual observation anomaly are called respectively, and the synchronous coordination operation of welding energy adjustment, welding gun pose correction and visual self-checking is performed under the unified timestamp.

[0177] When the energy closure residual decreases, the temperature synchronization degree recovers, the gray edge energy of the weld area tends to be stable, and the trajectory curvature change rate of the twin pose prediction decreases or recovers to be stable after being synchronously coordinated, it is determined that the composite abnormality is removed; if any parameter does not recover to the stable interval, the corresponding adjustment path remains in a dynamic running state until all monitoring quantities recover to be normal.

[0178] Through multi-dimensional synchronous coordination adjustment under a unified time reference, dynamic consistency and cooperative self-stabilization among the welding power output, the welding gun pose trajectory and the weld area gray image acquisition are realized, so that the welding process still maintains a continuous, stable and controllable running state under composite disturbance conditions.

[0179] In summary, the present application realizes the synchronous correlation of welding energy transmission, welding gun pose change and workpiece thermal response by establishing a recursive calculation and virtual-real comparison cooperative analysis mechanism for multi-channel data; and realizes the state recognition and abnormal classification of key process sections by constructing a stability consistency function and introducing a welding state interval boundary determination mechanism, so as to guarantee the stable operation of the welding process and the high-precision consistency of the monitoring results.

[0180] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application, and all should be covered in the scope of the claims of the present application.

Claims

1. A digital-twin-based real-time monitoring method for a welding shop robot state, characterized in that: The method comprises the steps of: Collecting multi-source data of welding operation, and performing time alignment under a unified time grid to form aligned multi-channel data and a unified timestamp sequence; According to the unified timestamp sequence, performing recursive calculation of the aligned multi-channel data at each time point to obtain twin prediction data; By comparing the twin prediction data with the aligned multi-channel data at each time point, virtual-real contrast features are obtained, a set of key process sections is extracted according to the virtual-real contrast features, and a fast consistency ratio average and a welding stability index average are calculated; According to the fast consistency ratio average and the welding stability index average, a stability consistency function is constructed, the welding state of the key process section is determined by defining the welding state interval boundary, and the abnormal type of the key process section is identified; Based on the abnormal type of the key process section, an event trigger signal is generated, and a real-time adjustment path is executed to dynamically coordinate the energy control, motion control and visual acquisition control in the welding process.

2. The welding plant robot state real-time monitoring method based on digital twinning of claim 1, wherein: The welding operation multi-source data includes welding gun pose, welding gun motion speed, welding current, welding voltage, workpiece surface temperature distribution image and weld area gray scale image.

3. The welding plant robot state real-time monitoring method based on digital twinning of claim 2, wherein: The time alignment under the unified time grid includes interpolating the welding current, welding voltage and welding gun motion speed, performing time window matching on the workpiece surface temperature distribution image and the weld area gray scale image, and performing pose interpolation correction on the welding gun pose data; The aligned multi-channel data and the unified timestamp sequence are obtained under the unified time grid.

4. The welding plant robot state real-time monitoring method based on digital twinning of claim 3, wherein: The recursive calculation at each time point includes, at each unified timestamp, using the aligned multi-channel data of the previous unified timestamp as input, performing forward Euler integration on the welding gun pose and the welding gun motion speed to obtain twin pose prediction; Respectively performing one-step prediction on the welding current and the welding voltage to obtain twin current prediction and twin voltage prediction; Performing electrical-thermal equivalent incremental update on the workpiece surface temperature distribution image to obtain twin temperature prediction.

5. The welding plant robot state real-time monitoring method based on digital twinning of claim 4, wherein: The virtual-real contrast features include, at each unified timestamp, comparing the twin pose prediction, the twin current prediction, the twin voltage prediction, the twin temperature prediction and the aligned multi-channel data at each time point to obtain energy closure residual, temperature synchronization degree and weld area gray scale edge energy.

6. The welding plant robot state real-time monitoring method based on digital twinning of claim 5, wherein: The set of key process sections extracted according to the virtual-real contrast features includes calculating the trajectory curvature of the twin pose prediction according to the geometric relationship of adjacent twin pose predictions, and performing differential calculation on the trajectory curvature of adjacent twin pose predictions to obtain the trajectory curvature change rate of the twin pose prediction; Time sequence joint analysis is performed on the weld area gray scale edge energy and the trajectory curvature change rate of the twin pose prediction; When the weld area gray scale edge energy and the trajectory curvature change rate of the twin pose prediction at the same unified timestamp both reach local extreme values, the time point corresponding to the corresponding unified timestamp is determined as a key process section; All records of the time points corresponding to the unified timestamps that simultaneously reach local extreme values are combined to form a set of key process sections.

7. The welding plant robot state real-time monitoring method based on digital twinning of claim 6, wherein: The computing the quick consistency ratio includes calculating the quick consistency ratio by the ratio of the energy closure residual and the temperature synchronization degree; The computing the welding stability index includes synthesizing the quick consistency ratio, the weld area gray edge energy, and the trajectory curvature change rate of the twin pose prediction; The computing the quick consistency ratio mean value and the welding stability index mean value of each key process section in the key process section set.

8. The welding plant robot state real-time monitoring method based on digital twinning of claim 7, wherein: The defining the stability consistency function includes calculating the instantaneous stability change amount according to the difference of the welding stability indexes corresponding to adjacent uniform timestamps on the uniform timestamp sequence; The stability consistency function is constructed by combining the mean square root value of the instantaneous stability change amount with the welding stability index mean value and the quick consistency ratio mean value of each key process section. The computing the average value and the standard deviation of the stability consistency function of all key process sections in the same welding task, and defining the stable boundary of the welding state and the fluctuation boundary of the welding state based on the statistical distribution characteristics of the stability consistency function.

9. The welding plant robot state real-time monitoring method based on digital twinning of claim 8, wherein: The determining the welding state of the key process section and identifying the abnormal type of the key process section include: when the stability consistency function is not lower than the stable boundary of the welding state, determining that the welding state of the key process section is stable; when the stability consistency function is between the stable boundary of the welding state and the fluctuation boundary of the welding state, determining that the welding state of the key process section is slightly fluctuating; when the stability consistency function is lower than the fluctuation boundary of the welding state, determining that the welding state of the key process section is abnormal; For the key process section with the welding state being abnormal, analyzing the change relationship of the energy closure residual, the temperature synchronization degree, the weld area gray edge energy, and the trajectory curvature change rate of the twin pose prediction to identify the welding energy mismatch type abnormality, the welding gun posture disturbance type abnormality, the visual observation abnormality, and the composite abnormality.

10. The welding plant robot state real-time monitoring method based on digital twinning of claim 9, wherein: The dynamic coordination of the energy control, the motion control, and the visual acquisition control in the welding process includes generating an event trigger signal by taking the welding energy mismatch type abnormality, the welding gun posture disturbance type abnormality, the visual observation abnormality, and the composite abnormality as a trigger source; when the welding energy mismatch type abnormality is triggered, reading the twin voltage prediction and the twin current prediction at the current uniform timestamp, calculating a virtual welding power estimated value, and correcting the welding power setting power through a proportional adjustment function to obtain the welding power setting power, and outputting the welding power setting power to the welding power source; when the welding gun posture disturbance type abnormality is triggered, extracting the twin pose prediction at the current uniform timestamp, calculating a posture smoothing correction angle according to the change trend of the trajectory curvature change rate of the twin pose prediction, correcting the twin pose prediction to obtain a corrected posture target, and outputting the corrected posture target to the welding gun servo control unit to update the actual motion trajectory of the welding gun in synchronization with the uniform timestamp; when the visual observation abnormality is triggered, reading the weld area gray image and calculating the weld area gray edge energy, and if the weld area gray edge energy decreases in consecutive uniform timestamps, performing a visual acquisition link self-calibration operation. When the composite exception is triggered, the state of simultaneous deviation of energy closure residual, temperature synchronization degree, weld area gray edge energy and twin pose prediction trajectory curvature change rate is identified, and the real-time adjustment path corresponding to the welding energy mismatch type exception, the welding gun posture disturbance type exception and the visual observation exception is called respectively, and the synchronous coordination operation is executed under the unified timestamp, and when the monitoring quantity returns to the stable interval, it is determined that the composite exception is removed.

Images