Welding abnormality evaluation and defect positioning method and system based on multi-modal fusion

Abstract

The application belongs to the technical field of intelligent manufacturing and nondestructive testing, and discloses a welding abnormality evaluation and defect positioning method and system based on multi-modal fusion, which comprises the following steps: S1, acquiring multi-modal physical signals in a welding process; S2, constructing a structured multi-modal time sequence feature tensor; S3, online identifying an abnormal type and a risk confidence in the current welding process; S4, activating a double-flow thermodynamic coupling multi-dimensional mapping model; S5, determining whether there is a substantial welding defect, and obtaining an absolute three-dimensional space coordinate of the defect in a workpiece global coordinate system; and S6, uploading the database to generate a spot repair mark, so as to realize accurate positioning and full life cycle tracing of the welding defect. The system comprises various modules for executing the above method. The application has the beneficial effect of constructing deep physical coupling of thermodynamic size and kinematic lag time, which makes the larger the defect size, the more accurate the lag compensation distance of the system adaptive adjustment.

Description

Technical Field

[0001] This invention relates to the fields of intelligent manufacturing and non-destructive testing technology, specifically to a method and system for welding anomaly assessment and defect localization based on multimodal fusion. Background Technology

[0002] In the manufacturing process of high-performance transportation vehicles such as new energy vehicles, rail transit, and aerospace, key load-bearing components often feature a variety of materials, large thickness ranges, and complex joint forms. Constrained by incoming material tolerances and fluctuating heat dissipation conditions, automated welding is highly prone to producing hidden quality defects such as incomplete fusion and porosity. For a long time, the industry has primarily relied on post-weld non-destructive testing (such as full-line X-ray inspection). This "weld first, inspect later" model is severely outdated, and for large components, blindly searching for hidden defects is extremely costly. Online monitoring technologies developed in recent years suffer from the following bottlenecks: first, limited sensing methods and high false negative rates; second, black-box evaluation algorithms that cannot combine real physical boundaries to output the quantitative indicators of weld depth and weld width urgently needed by the engineering field; and third, a lack of time-space conversion capabilities—even if the system reports an error, it can only record the time of the anomaly, unable to convert it into precise coordinates in the workpiece's three-dimensional physical space, making post-weld repair still difficult. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a method and system for welding anomaly assessment and defect localization based on multimodal fusion.

[0004] The present invention achieves the above-mentioned technical objectives through the following technical means.

[0005] The welding anomaly assessment and defect localization method based on multimodal fusion includes the following steps:

[0006] Step S1: Acquire multimodal physical signals during the robotic welding process of the complex component to be welded;

[0007] Step S2: Perform microsecond-level time synchronization processing on the multimodal physical signal based on hardware clock, and perform multi-source feature extraction and cross-attention fusion through a dual-stream thermodynamic decoupling architecture to construct a structured multimodal temporal feature tensor;

[0008] Step S3: Input the multimodal temporal feature tensor into the pre-constructed temporal state monitoring model, extract the long-range temporal dependencies of the multimodal data, and identify the types of transient physical anomalies and their transient risk confidence in the current welding process online;

[0009] Step S4: When the transient risk confidence and its continuous integral risk meet the adaptive dynamic threshold triggering condition, activate the dual-flow thermodynamic coupling multidimensional mapping model to quantify the predicted weld penetration and predicted weld width under the current physical state, and the model is embedded with an asymmetric physical energy conservation loss function.

[0010] Step S5: Calculate the depth deviation and width deviation based on the predicted depth and width. When the depth deviation or width deviation exceeds the standard tolerance, it is determined that there is a substantial welding defect. Then, through the kinematic homogeneous transformation matrix and the time-space lag compensation of the solidification of the molten pool, the absolute three-dimensional spatial coordinates of the defect in the global coordinate system of the workpiece are obtained.

[0011] Step S6: The identified anomaly type, deviation size, and absolute three-dimensional spatial coordinates and timestamp are synchronously uploaded to the digital twin quality database to generate a fixed-point rework mark, thereby realizing the accurate location and full life cycle traceability of welding defects.

[0012] Furthermore, the multimodal physical signals include electrical signals acquired by a high-frequency data acquisition card, visual images of the molten pool acquired by an industrial camera, acoustic emission signals acquired by an acoustic sensor, and temperature field signals acquired by an infrared thermal imager.

[0013] Furthermore, the process of multi-source feature extraction and cross-attention fusion in the described dual-flow thermodynamic decoupling architecture is as follows:

[0014] First, based on the multimodal physical signal obtained in step S1, the transient variation coefficient of the electrical signal and the surge peak of acoustic emission energy are obtained by sliding window statistics of one-dimensional high-frequency signal and temporal abrupt change detection, respectively; the visual solidification edge shrinkage rate and infrared temperature gradient are obtained by spatial geometry and pixel gradient calculation of two-dimensional low-frequency image, respectively.

[0015] Secondly, the transient variation coefficients of electrical signals with sampling rates higher than 10 kHz and the surge peaks of acoustic emission energy are used to construct the transient excitation current tensor. The visual solidification edge shrinkage rate with a sampling rate below 1 kHz, combined with the infrared temperature gradient and prior plate thickness information, is used to construct the boundary dissipative flow tensor. ;

[0016] Finally, the coupling weight matrix is ​​calculated using the thermodynamic cross-attention mechanism. Its formula is:

[0017]

[0018] in, and For the network's learnable weight matrix, For feature dimensions.

[0019] Furthermore, the temporal state monitoring model is a temporal convolutional network, which is composed of multiple stacked residual blocks; the residual blocks adopt a one-dimensional dilated causal convolutional layer.

[0020] Furthermore, the adaptive dynamic threshold triggering condition is: comparing the transient risk confidence level P(t) with the passing line Th(t); if the transient risk confidence level P(t) is less than the passing line Th(t), and when the continuous integral risk... Greater than the preset residence time constant of the molten pool At that time, a substantial anomaly risk is confirmed, and the dual-flow thermodynamic coupling multidimensional mapping model is activated.

[0021] Furthermore, the formula for the passing grade Th(t) is:

[0022]

[0023] In the formula, Th(t) is the dynamic security threshold at the current moment. Based on the basic security threshold, This is the threshold float during the arc initiation phase. Let be the time decay function. t is the attenuation coefficient, and t is the time variable.

[0024] Furthermore, the architecture of the dual-flow thermodynamic coupling multidimensional mapping model is as follows:

[0025] The model input receives the coupling weight matrix output by the thermodynamic cross-attention mechanism. The model contains at least three hidden layers; the output layer is a fully connected regression layer; and it is trained using an asymmetric physical energy conservation loss function, with the mixed loss function including the asymmetric physical energy conservation loss function. The announcement is as follows:

[0026]

[0027] In the formula, For data-driven loss, As a form of asymmetric risk punishment, As a physical penalty for the conservation of energy, , , All are coefficients;

[0028] Among them, the physical penalty of energy conservation The formula is expressed as follows:

[0029]

[0030] In the formula, Predicting melting depth To predict melt width, For thermal efficiency, U and I represent transient arc voltage and current, respectively, and v represents welding speed. For the volumetric heat capacity of the material, The melting temperature difference of the material;

[0031] Asymmetric risk penalty for:

[0032]

[0033] in, As an exponential penalty factor, It is an exponential penalty coefficient. For coefficients, This represents the actual melting depth.

[0034] Furthermore, the formula for calculating the absolute three-dimensional spatial coordinates is as follows:

[0035]

[0036] In the formula, The absolute three-dimensional spatial coordinates of the defect in the workpiece's global coordinate system; Let be the homogeneous transformation matrix of the workpiece relative to the robot base; Here is the homogeneous transformation matrix of the robot's forward kinematics based on the DH parameters. These are the joint angles of the robot's six axes; This is the static offset vector of the arc center under calibration; The instantaneous spatial velocity vector at the tool's center point at the moment the anomaly occurs; This represents the hysteresis time of the molten pool liquid-solid phase transformation.

[0037] Furthermore, the formula for the hysteresis time of the molten pool liquid-solid phase transformation is expressed as follows:

[0038]

[0039] In the formula, k is a proportionality coefficient affected by the current thermal properties of the material. This is the infrared edge gradient correction term. Predicting melting depth To predict melt width.

[0040] A welding anomaly assessment and defect localization system based on multimodal fusion, used to execute the welding anomaly assessment and defect localization method based on multimodal fusion, includes:

[0041] A multimodal physical field synchronous sensing module is used to execute step S1;

[0042] The spatiotemporal alignment and dual-stream feature fusion module is used to execute step S2;

[0043] A cascaded intelligent quality assessment module is used to execute steps S3 and S4, and has a built-in time-series state monitoring model and a dual-flow thermodynamic coupling multidimensional mapping model.

[0044] The kinematic space mapping and hysteresis compensation positioning module is used to execute step S5, read the transient pose and velocity and perform homogeneous coordinate transformation;

[0045] The digital twin traceability and documentation module is used to execute step S6 and generate a fixed-point rework quality map containing three-dimensional coordinates.

[0046] Compared with the prior art, the present invention has the following significant advantages:

[0047] 1. Reconstructing the feature fusion paradigm to break through the black-box limitations of perception algorithms. Abandoning the traditional model's simple stitching of multi-sensor data, an innovative dual-flow thermodynamic coupling architecture is proposed. Through a cross-attention mechanism, low-frequency heat dissipation boundaries dynamically reshape high-frequency transient excitation features, replicating the physical law that heat dissipation conditions determine the effectiveness of thermal input at the algorithm architecture level.

[0048] 2. Reshaping the underlying logic of quantization mapping to give the evaluation extremely high industrial reliability. A multi-dimensional mapping evaluation model with embedded physical energy conservation penalty and asymmetric risk penalty was constructed. This not only eliminates the network output from violating the first law of heat transfer, but also addresses the pain point in engineering practice where the risk of false alarms is much greater than the risk of false alarms by applying a conservative exponential penalty, making the quantization results extremely robust and secure.

[0049] 3. A multi-dimensional deviation adaptive-driven solidification hysteresis compensation positioning mechanism is designed, leading to a three-dimensional non-inspection rework mode. This invention breaks the technical bias of separating attribute assessment and location tracking in traditional monitoring systems. It is understood that the physical size of a defect directly changes the local heat capacity of the liquid metal, thus affecting the solidification hysteresis time. Therefore, the melt depth and melt width output by quantitative assessment are used as dynamic pre-inputs for spatial kinematic positioning, constructing a deep physical coupling between thermodynamic size and kinematic hysteresis time. This makes the hysteresis compensation distance more accurate as the defect size increases. Attached Figure Description

[0050] Figure 1 This is a flowchart of the welding anomaly assessment and defect location method based on multimodal fusion described in this invention.

[0051] Figure 2 This is a logic diagram of the welding anomaly assessment and defect location system based on multimodal fusion described in this invention. Detailed Implementation

[0052] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the scope of protection of the present invention is not limited thereto.

[0053] Example 1

[0054] A method for welding anomaly assessment and defect localization based on multimodal fusion, such as Figure 1 As shown, it includes the following steps:

[0055] Step S1: Acquire multimodal physical signals during the robotic welding process of the complex component to be welded. The multimodal physical signals include electrical signals acquired by a high-frequency data acquisition card, visual images of the molten pool acquired by an industrial camera, acoustic emission signals acquired by an acoustic sensor, and temperature field signals acquired by an infrared thermal imager.

[0056] Step S2: Perform microsecond-level time synchronization processing on the multimodal physical signal based on hardware clock, and perform multi-source feature extraction and cross-attention fusion through a dual-stream thermodynamic decoupling architecture to construct a structured multimodal temporal feature tensor;

[0057] Step S3: Input the multimodal temporal feature tensor into the pre-constructed temporal state monitoring model, extract the long-range temporal dependencies of the multimodal data, and identify the types of transient physical anomalies (such as incomplete penetration, incomplete fusion, burn-through, and porosity) and their transient risk confidence in the current welding process online.

[0058] Step S4: When the transient risk confidence and its continuous integral risk meet the adaptive dynamic threshold triggering condition, activate the dual-flow thermodynamic coupling multidimensional mapping model to quantify the predicted weld penetration and predicted weld width under the current physical state, and the model is embedded with an asymmetric physical energy conservation loss function.

[0059] Step S5: Calculate the depth deviation and width deviation based on the predicted depth and width. When the depth deviation or width deviation exceeds the standard tolerance, it is determined that there is a substantial welding defect. Then, through the kinematic homogeneous transformation matrix and the time-space lag compensation of the solidification of the molten pool, the absolute three-dimensional spatial coordinates of the defect in the global coordinate system of the workpiece are calculated.

[0060] Step S6: The identified defect type, deviation size, and absolute three-dimensional spatial coordinates and timestamp are synchronously uploaded to the digital twin quality database to generate a fixed-point rework mark, thereby realizing the accurate positioning and full life cycle traceability of welding defects.

[0061] The specific process of feature extraction and fusion in step S2 is as follows:

[0062] First, based on the multimodal physical signal obtained in step S1, the transient variation coefficient of the electrical signal and the surge peak of acoustic emission energy are obtained by sliding window statistics of one-dimensional high-frequency signal and temporal abrupt change detection, respectively; the visual solidification edge shrinkage rate and infrared temperature gradient are obtained by spatial geometry and pixel gradient calculation of two-dimensional low-frequency image (light / heat), respectively.

[0063] Secondly, the transient variation coefficients of electrical signals with sampling rates higher than 10 kHz and the surge peaks of acoustic emission energy are used to construct the transient excitation current tensor. The visual solidification edge shrinkage rate with a sampling rate below 1 kHz, combined with the infrared temperature gradient and prior plate thickness information, is used to construct the boundary dissipative flow tensor. ;

[0064] Finally, the coupling weight matrix is ​​calculated using the thermodynamic cross-attention mechanism. (i.e., multimodal temporal feature tensor), its formula is:

[0065]

[0066] in, and For the network's learnable weight matrix, The feature dimension is used to characterize the nonlinear evolution of effective heat input under different heat dissipation conditions by dynamically reshaping the value of transient excitation with boundary dissipation conditions as the key.

[0067] The dual-flow tensor constructed in this embodiment, during the feature engineering stage, strictly decouples high-frequency and low-frequency signals into transient energy excitation and boundary heat dissipation based on the time scale difference in heat transfer. Simultaneously, a cross-attention mechanism sets the boundary dissipation flow tensor as the dominant weight, achieving asymmetric physical modulation of the transient excitation flow tensor. This architecture incorporates the nonlinear metallurgical principle that heat dissipation conditions determine the actual heat input efficiency in metal welding, solving the industrial problem of evaluating failure under extreme heat dissipation conditions using conventional methods.

[0068] The temporal state monitoring model in step S3 is a temporal convolutional network (TCN), which consists of multiple stacked residual blocks. The residual blocks use one-dimensional dilated causal convolutional layers, and the dilation coefficient of the network layers increases exponentially to expand the temporal receptive field of the model while maintaining causal constraints, capture the historical thermal history state before the melt pool instability, and output the transient risk confidence P(t).

[0069] The specific process in step S4 is as follows:

[0070] The passing grade Th(t) is derived from the welding time. Th(t) exhibits a nonlinear decay evolution during the heat accumulation stage, expressed by the following formula:

[0071]

[0072] In the formula, Th(t) is the dynamic security threshold at the current moment. It is the benchmark for judging the transient risk confidence of the TCN network output. Only when the actual risk exceeds this dynamically changing line will it be considered whether to trigger an alarm. Based on the fundamental (steady-state) safety threshold, when the welding process has lasted long enough and the heat accumulation reaches a relatively balanced state (steady state), the subsequent exponential decay term will approach 0, at which point Th(t) equals... This represents the baseline for judgment after dynamic tightening during the stable welding period. This represents the threshold float (initial tolerance) during the arc initiation phase. At the moment the welding arc is just initiated (t=0), the exponential term... At this point, the total threshold is Because the electric arc and molten pool are extremely unstable at the moment of arc initiation (physical field fluctuations are normal), this value represents an extra tolerance to prevent false alarms. The time decay function is a nonlinear multiplier that decreases over time and is used to simulate the physical process of welding transitioning from the unstable arc initiation stage to a stable steady state. This is the decay coefficient (or time constant), whose magnitude typically depends on the material's thermal diffusivity, volumetric heat capacity, and current heat dissipation boundary conditions. The faster the heat dissipation, the shorter the time to reach steady state. The larger the value, the better. t is a time variable, usually referring to the welding time from the start of arc ignition (i.e., heat accumulation time).

[0073] The transient risk confidence level P(t) obtained through TCN is compared with the passing score Th(t); if the transient risk confidence level P(t) is less than the passing score Th(t), i.e., the score is failing, and the continuous integral risk... Greater than the preset residence time constant of the molten pool Only then is a substantial anomaly risk confirmed, and the dual-flow thermodynamic coupling multidimensional mapping model activated.

[0074] The dual-flow thermodynamic coupled multidimensional mapping model employs a multilayer perceptron (MLP) as its main mapping architecture. Its input receives the coupling weight matrix output by the thermodynamic cross-attention mechanism. It contains at least three hidden layers to decouple complex nonlinear thermodynamic mappings; its output is a fully connected regression layer that directly outputs a two-dimensional continuous variable vector. , , representing the predicted melt depth and predicted melt width, respectively; and a hybrid loss function is used to train the model.

[0075] The specific structure of the two-flow thermodynamic coupling multidimensional mapping model is as follows:

[0076] First, the feature tensor fused by the cross-attention mechanism is flattened into a one-dimensional vector. .

[0077] Secondly, the one-dimensional vector is mapped layer by layer recursively, as shown in the following formula:

[0078] nth hidden layer:

[0079] In the formula, Let n be the weight matrix of the nth layer. For bias vectors, For nonlinear activation functions with anti-gradient vanishing properties, this embodiment preferably uses the Leaky ReLU function to preserve the weak negative heat dissipation characteristics.

[0080] Finally, the output layer passes the features extracted from the hidden layer depth through a fully connected matrix. Mapping to physical size space. Since the actual physical dimensions (melt depth, melt width) cannot be negative, the output layer is forced to use the standard ReLU (Rectified Linear Unit) operator for hard truncation, as shown in the following formula:

[0081]

[0082] In the formula, ReLU is the activation function of the rectified linear unit; , For corresponding melting depth The exclusive bias value, For corresponding melt width Specific bias value; It is a fully connected matrix.

[0083] The hybrid loss function of the two-flow thermodynamic coupled multidimensional mapping model adopts an asymmetric physical energy conservation loss function. The announcement is as follows:

[0084]

[0085] In the formula, For data-driven loss, As a form of asymmetric risk punishment, As a physical penalty for the conservation of energy, , , All are coefficients.

[0086] Among them, the physical penalty of energy conservation The theoretical limit for constraining the actual molten metal volume to not exceed the effective arc heat input is expressed by the following formula:

[0087]

[0088] In the formula, For thermal efficiency, U and I represent transient arc voltage and current, respectively, and v represents welding speed. For the volumetric heat capacity of the material, The material melting temperature difference is used as a penalty term to prevent the model from outputting size predictions that violate classical heat transfer laws.

[0089] At the same time, asymmetric risk penalties are introduced. For predicting melting depth Greater than the actual melting depth (Unreporting) incurs a huge exponential penalty coefficient. For the predicted transient melting depth Less than the actual melting depth Apply a smaller coefficient in cases of false alarms. The forced model tends towards conservative assessments to ensure absolute industrial safety. Asymmetric risk penalty. for:

[0090]

[0091] in, As an exponential penalty factor, it can exponentially increase the sensitivity of the loss function to errors such as underreporting.

[0092] The specific process of step S5 is as follows:

[0093] First, the weld depth deviation and weld width deviation are calculated based on the predicted weld depth and predicted weld width. When the weld depth deviation or weld width deviation exceeds the standard tolerance (in this embodiment, the standard tolerance range for weld width is (-0.5mm, 0.5mm) and the standard tolerance range for weld depth is (-1.0mm, 0.5mm)), a substantial welding defect is determined to exist.

[0094] Secondly, homogeneous transformation matrix operations are performed: the transient joint angles of the six axes of the welding robot are read at high speed, and the pose transformation matrix of TCP (tool center point) relative to Base (robot base global coordinate system) is obtained by multiplying the transformation matrix based on the DH parameter method. Based on matrix It can accurately know the absolute position and angle of the welding torch in space at any given moment.

[0095] Finally, time-space lag compensation for molten pool solidification is performed: because the molten pool is still liquid when the anomaly occurs, macroscopic defects solidify at the tail end. This embodiment introduces dual compensation of spatial geometry and phase transition time to calculate the absolute three-dimensional spatial coordinates of the defect in the workpiece's global coordinate system, as shown in the following formula:

[0096]

[0097] In the formula, The absolute three-dimensional spatial coordinates of the defect in the workpiece's global coordinate system; Let be the homogeneous transformation matrix of the workpiece relative to the robot base. This is the homogeneous transformation matrix of the robot's forward kinematics based on DH parameters. It is obtained by reading the joint angles of the robot's six axes at this moment. arrive ), calculate the accurate orientation and position of the tool center point (TCP, i.e., welding torch) relative to the robot base. The static offset vector of the arc center under calibration is used to correct for the slight mechanical installation tolerance between the mechanical control point (TCP) of the welding torch and the actual center point of arc combustion. The instantaneous spatial velocity vector at the tool center point (TCP) at the moment the anomaly occurs; This represents the hysteresis time of the molten pool liquid-solid phase transformation.

[0098] In this embodiment, in order to achieve deep coupling between thermodynamic quantification state and spatial position, Instead of fixed lookup table values, the current transient predicted melt depth is output by a dual-flow thermodynamic coupled multidimensional mapping model. and predicted melt width This is jointly determined. The physical basis for this is that the volume of the transient molten pool determines the latent heat released during solidification and the cooling time. Its adaptive coupling calculation logic is as follows:

[0099]

[0100] In the formula, k is a proportionality coefficient affected by the current thermal properties of the material. This is the infrared edge gradient correction term. By introducing this coupling formula, the forming size quantized in step S4 is directly converted into a hysteresis time fine-tuning quantity that affects spatial positioning accuracy. Therefore, the dot product term in the dual compensation formula of spatial geometry and phase transition time successfully offsets the spatial position drift caused by welding speed and non-standard heat accumulation size.

[0101] A welding anomaly online assessment and defect location system based on electro-optical-acoustic-thermal multimodal information fusion, such as Figure 2 As shown, it includes:

[0102] A multimodal physical field synchronous sensing module is used to execute step S1;

[0103] The spatiotemporal alignment and dual-stream feature fusion module is used to execute step S2;

[0104] A cascaded intelligent quality assessment module is used to execute steps S3 and S4, and has a built-in time-series state monitoring model and a dual-flow thermodynamic coupling multidimensional mapping model.

[0105] The kinematic space mapping and hysteresis compensation positioning module is used to execute step S5, read the transient pose and velocity and perform homogeneous coordinate transformation;

[0106] The digital twin traceability and documentation module is used to execute step S6 and generate a fixed-point rework quality map containing three-dimensional coordinates.

[0107] Example 2

[0108] For automated welding of variable cross-section joints in high-strength steel battery trays for new energy vehicles, hidden defects are easily generated due to fluctuations in local plate thickness and assembly gaps. The specific process for the system to perform anomaly assessment and precise defect location is as follows:

[0109] S1: The robot initiates arc welding according to the nominal trajectory, synchronously acquiring multimodal physical signals. During the welding process, the multimodal physical field synchronous sensing module synchronously acquires the physical signals of the component to be welded in real time around the clock, including: arc voltage and current signals acquired by a high-frequency data acquisition card, visual images of the molten pool acquired by a high-speed industrial camera, high-frequency acoustic emission signals acquired by an acoustic sensor, and temperature field signals acquired by an infrared thermal imager.

[0110] S2: Microsecond-level spatiotemporal synchronization and fusion of dual-stream thermodynamic features. First, the acquired multimodal signals undergo microsecond-level time synchronization processing based on a hardware clock to ensure strict alignment of data on a unified time axis. Then, dual-stream feature extraction is performed: the transient variation coefficient of the electrical signal and the acoustic emission energy surge peak are used to construct a transient excitation flow tensor; the solidification edge contraction rate and infrared temperature gradient in the visual image are combined with plate thickness information to construct a boundary dissipation flow tensor. Through a thermodynamic cross-attention mechanism, the values ​​of the transient excitation are dynamically reshaped using boundary dissipation conditions (such as faster heat dissipation due to variable cross-section) as keys, thus constructing a structured multimodal temporal feature tensor.

[0111] S3: Intelligent identification of transient risks based on Temporal Convolutional Network (TCN). When the welding torch travels to the variable cross-section transition zone, the pre-built TCN monitoring model extracts long-range temporal dependencies and identifies online the nonlinear shift in the current molten pool thermal history. The model determines that there is a transient physical anomaly risk of incomplete penetration in the current physical state and outputs the corresponding risk confidence level (89%).

[0112] S4: Since the confidence level (89%) and duration of the warning risk meet the preset adaptive dynamic threshold triggering conditions, the quantitative evaluation based on the multidimensional mapping model is cascaded and activated, which in turn calls the dual-flow thermodynamic coupled multidimensional mapping model. This model embeds an asymmetric physical energy conservation loss function, and combined with prior knowledge of transient heat transfer, it is calculated that under the current physical state of extremely rapid local heat dissipation, the actual weld penetration depth is only 1.5mm.

[0113] S5: Three-dimensional spatial homogeneous transformation and solidification hysteresis compensation positioning. When quantitative assessment confirms the existence of substantial incomplete penetration defects (actual weld penetration depth is only 1.5mm, with an absolute deviation of -1.0mm from the target process standard (design penetration depth 2.5mm)), based on the current transient six-axis joint pose data (θ1...θ6) and instantaneous linear velocity of the welding robot, To eliminate the spatial difference between the arc position and the actual solidification position at the tail of the molten pool, a homogeneous transformation matrix is ​​used, and a dot product is introduced to introduce the lag time of the molten pool liquid-solid phase transformation, thus obtaining the absolute three-dimensional spatial coordinates of the defect in the workpiece's global coordinate system. Specifically, based on the relatively small actual melt depth (1.5mm) output in S4, due to the reduced local melting volume leading to accelerated cooling, the lag time constant is adaptively shortened using a coupling formula (dynamically adjusted to a lag of 0.18 seconds, instead of the standard 0.25 seconds). After this physical property-driven lag double compensation calculation, the absolute three-dimensional spatial coordinates of the defect in the workpiece's global coordinate system are accurately obtained (X: 1250.5mm, Y: 430.2mm, Z: 85.0mm).

[0114] S6: The digital twin traceability and full lifecycle documentation system packages the identified defect type (incomplete penetration), quantified forming deviation dimension (-1.0mm), and calculated absolute three-dimensional spatial coordinates (X:1250.5, Y:430.2, Z:85.0) along with a timestamp, and synchronously uploads them to the workshop-level digital twin quality database. A red fixed-point repair marker is automatically generated at the corresponding coordinates in the 3D CAD / BIM model of the welded component. Subsequent repair stations only need to read this twin map to directly access the defect for precise grinding and repair welding without full inspection, achieving full lifecycle traceability of welding defects.

[0115] The embodiments described above are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments. Any obvious improvements, substitutions or modifications that can be made by those skilled in the art without departing from the essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for welding anomaly assessment and defect localization based on multimodal fusion, characterized in that, Includes the following steps: Step S1: Acquire multimodal physical signals during the robotic welding process of the complex component to be welded; Step S2: Perform microsecond-level time synchronization processing on the multimodal physical signal based on hardware clock, and perform multi-source feature extraction and cross-attention fusion through a dual-stream thermodynamic decoupling architecture to construct a structured multimodal temporal feature tensor; Step S3: Input the multimodal temporal feature tensor into the pre-constructed temporal state monitoring model, extract the long-range temporal dependencies of the multimodal data, and identify the types of transient physical anomalies and their transient risk confidence in the current welding process online; Step S4: When the transient risk confidence and its continuous integral risk meet the adaptive dynamic threshold triggering condition, activate the dual-flow thermodynamic coupling multidimensional mapping model to quantify the predicted weld penetration and predicted weld width under the current physical state, and the model is embedded with an asymmetric physical energy conservation loss function. Step S5: Calculate the depth deviation and width deviation based on the predicted depth and width. When the depth deviation or width deviation exceeds the standard tolerance, it is determined that there is a substantial welding defect. Then, through the kinematic homogeneous transformation matrix and the time-space lag compensation of the solidification of the molten pool, the absolute three-dimensional spatial coordinates of the defect in the global coordinate system of the workpiece are obtained. Step S6: The identified anomaly type, deviation size, and absolute three-dimensional spatial coordinates and timestamp are synchronously uploaded to the digital twin quality database to generate a fixed-point rework mark, thereby realizing the accurate location and full life cycle traceability of welding defects.

2. The welding anomaly assessment and defect location method based on multimodal fusion according to claim 1, characterized in that, The multimodal physical signals include electrical signals acquired by a high-frequency data acquisition card, visual images of the molten pool acquired by an industrial camera, acoustic emission signals acquired by an acoustic sensor, and temperature field signals acquired by an infrared thermal imager.

3. The welding anomaly assessment and defect location method based on multimodal fusion according to claim 1, characterized in that, The process of multi-source feature extraction and cross-attention fusion in the dual-flow thermodynamic decoupling architecture is as follows: First, based on the multimodal physical signal obtained in step S1, the transient variation coefficient of the electrical signal and the surge peak of acoustic emission energy are obtained by sliding window statistics of one-dimensional high-frequency signal and temporal abrupt change detection, respectively; the visual solidification edge shrinkage rate and infrared temperature gradient are obtained by spatial geometry and pixel gradient calculation of two-dimensional low-frequency image, respectively. Secondly, the transient variation coefficients of electrical signals with sampling rates higher than 10 kHz and the surge peaks of acoustic emission energy are used to construct the transient excitation current tensor. The visual solidification edge shrinkage rate with a sampling rate below 1 kHz, combined with the infrared temperature gradient and prior plate thickness information, is used to construct the boundary dissipative flow tensor. ; Finally, the coupling weight matrix is ​​calculated using the thermodynamic cross-attention mechanism. Its formula is: ； in, and For the network's learnable weight matrix, For feature dimensions.

4. The welding anomaly assessment and defect location method based on multimodal fusion according to claim 1, characterized in that, The temporal state monitoring model is a temporal convolutional network, which is composed of multiple stacked residual blocks; the residual blocks adopt a one-dimensional dilated causal convolutional layer.

5. The welding anomaly assessment and defect location method based on multimodal fusion according to claim 1, characterized in that, The adaptive dynamic threshold triggering condition is: a comparison between the transient risk confidence level P(t) and the passing score Th(t); If the transient risk confidence level P(t) is less than the passing line Th(t), and when the continuous integral risk Greater than the preset residence time constant of the molten pool At that time, a substantial anomaly risk is confirmed, and the dual-flow thermodynamic coupling multidimensional mapping model is activated.

6. The welding anomaly assessment and defect location method based on multimodal fusion according to claim 5, characterized in that, The formula for the passing grade Th(t) is: ； In the formula, Th(t) is the dynamic security threshold at the current moment. Based on the basic security threshold, This is the threshold float during the arc initiation phase. Let be the time decay function. t is the attenuation coefficient, and t is the time variable.

7. The welding anomaly assessment and defect location method based on multimodal fusion according to claim 1, characterized in that, The architecture of the dual-flow thermodynamic coupling multidimensional mapping model is as follows: The model input receives the coupling weight matrix output by the thermodynamic cross-attention mechanism. The model contains at least three hidden layers; the output layer is a fully connected regression layer; and it is trained using an asymmetric physical energy conservation loss function, with the mixed loss function including the asymmetric physical energy conservation loss function. The announcement is as follows: ； In the formula, For data-driven loss, As a form of asymmetric risk punishment, As a physical penalty for the conservation of energy, , , All are coefficients; Among them, the physical penalty of energy conservation The formula is expressed as follows: ； In the formula, Predicting melting depth To predict melt width, For thermal efficiency, U and I represent transient arc voltage and current, respectively, and v represents welding speed. For the volumetric heat capacity of the material, The melting temperature difference of the material; Asymmetric risk penalty for: ； in, As an exponential penalty factor, It is an exponential penalty coefficient. For coefficients, This represents the actual melting depth.

8. The welding anomaly assessment and defect location method based on multimodal fusion according to claim 1, characterized in that, The formula for calculating the absolute three-dimensional spatial coordinates is: ； In the formula, The absolute three-dimensional spatial coordinates of the defect in the workpiece's global coordinate system; Let be the homogeneous transformation matrix of the workpiece relative to the robot base; Here is the homogeneous transformation matrix of the robot's forward kinematics based on the DH parameters. These are the joint angles of the robot's six axes; This is the static offset vector of the arc center under calibration; The instantaneous spatial velocity vector at the tool's center point at the moment the anomaly occurs; This represents the hysteresis time of the molten pool liquid-solid phase transformation.

9. The welding anomaly assessment and defect location method based on multimodal fusion according to claim 8, characterized in that, The formula for the hysteresis time of the molten pool liquid-solid phase transformation is as follows: ； In the formula, k is a proportionality coefficient affected by the current thermal properties of the material. This is the infrared edge gradient correction term. Predicting melting depth To predict melt width.

10. A welding anomaly assessment and defect location system based on multimodal fusion, used to execute the method of any one of claims 1-9, characterized in that, include: A multimodal physical field synchronous sensing module is used to execute step S1; The spatiotemporal alignment and dual-stream feature fusion module is used to execute step S2; A cascaded intelligent quality assessment module is used to execute steps S3 and S4, and has a built-in time-series state monitoring model and a dual-flow thermodynamic coupling multidimensional mapping model. The kinematic space mapping and hysteresis compensation positioning module is used to execute step S5, read the transient pose and velocity and perform homogeneous coordinate transformation; The digital twin traceability and documentation module is used to execute step S6 and generate a fixed-point rework quality map containing three-dimensional coordinates.

Images