A steel structure adaptive intelligent welding method and system based on multi-modal perception

Abstract

The application relates to the technical field of artificial intelligence, and discloses a steel structure self-adaptive intelligent welding method and system based on multi-modal perception, which comprises a multi-modal perception and data fusion module, a dynamic welding quality image construction module, a self-adaptive strategy matching and decision module, a strategy precise execution and feedback module and a system self-learning optimization module; a dynamic quality evaluation engine embedded with a deep learning model is used to identify, quantify and predict the type, position and severity of welding defects in real time, generate a real-time dynamic welding quality image, and call a self-adaptive strategy matching engine according to the features of the image, match the optimal welding parameter adjustment strategy from a pre-constructed welding process parameter intelligent adjustment algorithm library, and then perform real-time dynamic adjustment on the welding robot, so that closed-loop self-adaptive control of the welding process is realized, and the real-time guarantee capability of the welding quality is improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to an adaptive intelligent welding method and system for steel structures based on multimodal perception. Background Technology

[0002] Steel structures are structures made of steel materials and are one of the main types of building structures. The structure is mainly composed of steel beams, steel columns, steel trusses and other components made of steel sections and steel plates, and uses rust removal and rust prevention processes such as silanization, pure manganese phosphating, water washing and drying, and galvanizing. The components or parts are usually connected by welds, bolts or rivets. Because of its light weight and simple construction, it is widely used in large factories, stadiums, super high-rise buildings, bridges and other fields.

[0003] Currently, due to the involvement of multi-source heterogeneous working conditions and complex environmental interference in the steel structure welding process, existing welding systems struggle to achieve deep fusion and real-time accurate evaluation of multi-modal sensing data when performing adaptive intelligent welding based on multi-modal perception. If welding quality fluctuations or deviations occur, it may lead to a decrease in welding accuracy, frequent defects, and an inability to guarantee the stability of the welding process.

[0004] Therefore, an adaptive intelligent welding method and system for steel structures based on multimodal perception is proposed to solve the above problems. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an adaptive intelligent welding method and system for steel structures based on multimodal perception, which solves the problems of decreased welding accuracy, frequent defects, and inability to guarantee the stability of the welding process mentioned in the background technology.

[0006] To achieve the above objectives, the present invention provides the following technical solution: an adaptive intelligent welding method and system for steel structures based on multimodal perception, comprising the following steps: S1. By deploying a multimodal sensor array at the welding station, the bevel geometry of the workpiece to be welded, the spectral information of the base material, the thermal radiation image of the molten pool during the welding process, the arc acoustic signal and electrical parameter data are collected simultaneously to generate the original multimodal sensing data set. S2. Perform temporal domain alignment and spatial coordinate registration on the original multimodal sensing data set to generate a standardized sensing data stream with spatiotemporal consistency; S3. Based on the standardized perception data stream, a dynamic quality assessment engine with an embedded deep learning model is used to identify, quantify and predict the type, location and severity of welding defects in real time, and generate a real-time dynamic welding quality profile. S4. Based on the characteristics of the real-time dynamic welding quality profile, call the adaptive strategy matching engine to match the optimal welding parameter adjustment strategy from the pre-built intelligent adjustment algorithm library of welding process parameters and generate a target strategy identifier. S5. Based on the target strategy identifier, call the corresponding strategy execution module to dynamically adjust the walking speed, wire feeding speed, welding current and oscillation trajectory of the welding robot in real time, and generate strategy execution log and welding quality calibration data. S6. Based on the strategy execution log, welding quality calibration data, and historical welding data, iteratively optimize and self-learn the intelligent adjustment algorithm library for welding process parameters and the dynamic quality assessment engine.

[0007] Preferably, generating the original multimodal sensing data set in S1 includes the following steps: S11. By deploying a laser profile scanner and a hyperspectral camera at the welding start end, data on the width, depth, angle and blunt edge size of the groove to be welded, as well as the alloy element content and surface oxide layer thickness of the base material are collected to generate a subset of workpiece state perception data. S12. By deploying a high-speed infrared thermal imager and a miniature microphone array on the welding torch, the temperature field distribution of the molten pool, the solidification front morphology, and the acoustic signature data of the electric arc combustion are collected in real time to generate a subset of process state perception data. S13. By integrating Hall sensors and current transformers into the welding power source and wire feeding mechanism, the voltage, current waveforms and wire feeding resistance fluctuation data of the welding circuit are collected in real time to generate a subset of equipment status perception data. S14. The workpiece state perception data subset, process state perception data subset, and equipment state perception data subset are aggregated to form the original multimodal perception data set.

[0008] Preferably, the temporal alignment and spatial coordinate registration in S2 include the following steps: S21. Receive the original multimodal sensing data set, and standardize and synchronize the timestamps of each subset of data based on a unified global clock source to eliminate the time delay caused by the difference in sampling frequency of different sensors; S22. Based on the pre-established three-dimensional welding workpiece digital twin model and robot kinematics model, assign spatial position labels in the world coordinate system to all perceived data. S23. Based on the preset welding process data ontology model, perform semantic parsing and relation mapping on the data with unified spatiotemporal labels, and associate data of different modalities and formats with specific weld numbers, weld layers, welding directions and robot joint postures. S24. The data that has completed spatiotemporal alignment and semantic association is integrated into a standardized perceptual data stream with coherent contextual information according to the advancement sequence and physical logic of the welding path.

[0009] Preferably, the generation of a real-time dynamic welding quality profile in step S3 includes the following steps: S31. Obtain the standardized perception data stream and input it into the dynamic quality assessment engine. The engine has a built-in multimodal feature extractor based on the fusion of Transformer and graph neural network to extract deep spatiotemporal coupling features from multi-source heterogeneous data streams. S32. The extracted spatiotemporal coupling features are matched and reasoned with the preset welding defect knowledge graph, which includes defect cause chains and evolution rules constructed from historical nondestructive testing reports, welding process evaluation data and industry expert experience. S33. Quantitatively assess the probability of occurrence and hazard level of the matched potential defect elements, and predict their evolution trend and expansion range in subsequent welds through a time-series prediction model. S34. Integrate and encapsulate all identified potential defect elements, quantitative evaluation results, evolution trend prediction values, and their associated welding process parameters and spatial location information into a structured real-time dynamic welding quality profile.

[0010] Preferably, generating the target policy identifier in step S4 includes the following steps: S41. Construct a smart algorithm library for adjusting welding process parameters, which stores preset parameter adjustment strategy models in the form of a vector database, including penetration compensation adjustment model, sidewall fusion enhancement model, residual height control optimization model and spatter suppression adjustment model. S42. Extract the key feature vector from the real-time dynamic welding quality profile. The key feature vector includes the defect type code, the degree of defect deviation tolerance, the offset of the molten pool oscillation frequency, and the heat dissipation condition index of the current weld bead. S43. Perform a hybrid similarity calculation based on Euclidean distance and cosine similarity between the key feature vector and the standard applicable feature vector of each strategy model in the intelligent adjustment algorithm library for welding process parameters; S44. Based on the hybrid similarity calculation results, the optimal parameter adjustment strategy model with the highest comprehensive score is matched through the adaptive strategy matching engine, and its unique encoding is output as the target strategy identifier.

[0011] Preferably, the execution steps of the adaptive strategy matching engine in S44 are as follows: S441. Initialization: Set the maximum number of iterations for the matching search, and randomly initialize the position and velocity of a group of particles in a high-dimensional decision space composed of the standard applicable feature vectors of all policy models. S442, Exploration phase: Drive the particle swarm to fly in the decision space and calculate the fitness of the feature vector of the policy model corresponding to the new position of each particle and the feature vector of the real-time quality profile. If the fitness of the new position is better than the individual's historical best position, then update the individual's extreme value; If the fitness of the new position is better than the group's historical best position, then update the global extremum; S443. Utilization phase: guide the particle swarm to converge toward the currently discovered global optimal solution region, perform local search in the neighborhood of the optimal solution, calculate new candidate positions and evaluate their fitness. If the fitness of a candidate position is better than that of the current global optimum, then update the global optimum. S444. Repeatedly execute the exploration and utilization steps until the maximum number of iterations is reached, and output the strategy model code corresponding to the global optimal position, i.e., the target strategy identifier.

[0012] Preferably, the generation of strategy execution logs and welding quality calibration data in step S5 includes the following steps: S51. Based on the target strategy identifier, call the predefined strategy execution logic script corresponding to the identifier from the process parameter adjustment algorithm library; S52. The predefined strategy execution logic script accesses the standardized perception data stream and the real-time dynamic welding quality profile, and combines the real-time pose and kinematic constraints of the current welding robot to generate a detailed instruction set containing specific adjustment parameters, adjustment gradients, effective timing and execution priority. S53. The detailed instruction set is sent to the underlying execution units of the welding robot controller, welding power supply and wire feeding mechanism via the industrial Ethernet protocol; S54. Synchronously monitor the execution process of the command, collect the actual response values ​​of the adjusted welding current, voltage, wire feeding speed and robot motion trajectory in real time, record the execution status, execution results and system feedback, and form a strategy execution log. S55. Within the preset weld length and time period after the strategy is executed, continue to collect subsequent multimodal sensing data, evaluate the reduction effect of welding defects and the improvement of weld formation quality, compare with the expected target, and generate welding quality calibration data.

[0013] Preferably, the iterative optimization and self-learning update in S6 includes the following steps: S61. Perform correlation analysis between the strategy execution log and the welding quality calibration data to calculate the effectiveness score of this strategy execution in defect suppression and the efficiency score of parameter adjustment. S62. Store the complete data package of this response, including the real-time dynamic welding quality profile, the selected target strategy identifier, the execution result and effectiveness score, as a new success or failure case in the historical case library. S63. Based on the historical case library, the welding defect knowledge graph in the dynamic quality assessment engine is updated incrementally on a regular basis, and the parameters of the multimodal feature extractor are fine-tuned and optimized using transfer learning technology. S64. Based on the historical case library, the standard applicable feature vectors and matching weights of each strategy model in the intelligent adjustment algorithm library for welding process parameters are periodically adjusted and updated using the Q-learning algorithm in reinforcement learning.

[0014] Preferably, the system includes: The multimodal perception and data fusion module gathers the raw signals from each stage of the welding station through the multi-source sensor acquisition unit, performs standardization processing using the data time domain alignment unit and the spatial coordinate registration unit, and outputs a standardized perception data stream with spatiotemporal consistency. The dynamic welding quality profile construction module receives the standardized perception data stream, identifies potential defects through a multimodal feature extraction unit and a defect knowledge graph reasoning unit, calculates the defect level and evolution trend through a defect quantification and prediction unit, and outputs a structured real-time dynamic welding quality profile. The adaptive strategy matching and decision-making module receives the real-time dynamic welding quality profile, analyzes key features through the feature vectorization unit, searches and makes decisions in a preset process parameter adjustment algorithm library using the adaptive strategy matching engine, and outputs the target strategy identifier.

[0015] Preferably, the system further includes: The strategy precision execution and feedback module receives the target strategy identifier, generates detailed control instructions through the strategy instruction compilation unit, controls the welding robot and welding power supply through the actuator drive interface unit, and outputs strategy execution logs and welding quality calibration data through the execution monitoring and effect evaluation unit. The system self-learning optimization module receives the strategy execution log and welding quality calibration data, builds a historical case library through the case management and analysis unit, updates the system core model and strategy using the model incremental training unit and algorithm library tuning unit, and outputs system optimization instructions.

[0016] Compared with existing technologies, this invention provides an adaptive intelligent welding method and system for steel structures based on multimodal perception, which has the following beneficial effects: 1. In this invention, data is synchronously collected by a multimodal sensor array deployed at the welding station to generate an original multimodal sensing data set. The data set is then aligned in the time domain and registered in the spatial coordinates to generate a standardized sensing data stream that is consistent in time and space. This achieves deep fusion and unified representation of multi-source heterogeneous sensing data, providing a data foundation for subsequent welding quality assessment and parameter adjustment.

[0017] 2. In this invention, a dynamic quality assessment engine embedded with a deep learning model is used to identify, quantify, and predict the type, location, and severity of welding defects in real time, generating a real-time dynamic welding quality profile. Based on the features of this profile, an adaptive strategy matching engine is invoked to match the optimal welding parameter adjustment strategy from a pre-built intelligent adjustment algorithm library for welding process parameters. This allows for real-time dynamic adjustment of the welding robot, achieving closed-loop adaptive control of the welding process and improving the real-time assurance capability of welding quality.

[0018] 3. In this invention, by iteratively optimizing and self-learning the intelligent adjustment algorithm library for welding process parameters and the dynamic quality assessment engine, and by storing the complete data package of each response in the historical case library, and by using transfer learning and reinforcement learning techniques to fine-tune and update the feature extractor and strategy model, the system can continuously accumulate experience and evolve, thereby achieving a long-term improvement in the autonomous optimization capability of welding processes. Attached Figure Description

[0019] Figure 1 This is a flowchart of an adaptive intelligent welding method for steel structures based on multimodal perception, according to the present invention. Figure 2 This is a structural diagram of an adaptive intelligent welding system for steel structures based on multimodal perception, according to the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] For specific implementation examples, please refer to: Figures 1-2 An adaptive intelligent welding method and system for steel structures based on multimodal perception includes the following steps: S1. By deploying a multimodal sensor array at the welding station, the bevel geometry of the workpiece to be welded, the spectral information of the base material, the thermal radiation image of the molten pool during the welding process, the arc acoustic signal and electrical parameter data are collected simultaneously to generate the original multimodal sensing data set. S2. Perform temporal domain alignment and spatial coordinate registration on the original multimodal sensing data set to generate a standardized sensing data stream with spatiotemporal consistency; S3. Based on standardized perception data stream, through a dynamic quality assessment engine embedded with a deep learning model, the type, location and severity of welding defects are identified, quantified and predicted in real time, generating a real-time dynamic welding quality profile. S4. Based on the characteristics of the real-time dynamic welding quality profile, the adaptive strategy matching engine is invoked to match the optimal welding parameter adjustment strategy from the pre-built intelligent adjustment algorithm library of welding process parameters and generate the target strategy identifier. S5. Based on the target strategy identifier, call the corresponding strategy execution module to dynamically adjust the walking speed, wire feeding speed, welding current and oscillation trajectory of the welding robot in real time, and generate strategy execution log and welding quality calibration data. S6. Based on strategy execution logs, welding quality calibration data, and historical welding data, the intelligent adjustment algorithm library for welding process parameters and the dynamic quality assessment engine are iteratively optimized and updated through self-learning.

[0022] The process of generating the raw multimodal sensing data set in S1 includes the following steps: S11. By deploying a laser profile scanner and a hyperspectral camera at the welding start end, data on the width, depth, angle and blunt edge size of the groove to be welded, as well as the alloy element content and surface oxide layer thickness of the base material are collected to generate a subset of workpiece state perception data. S12. By deploying a high-speed infrared thermal imager and a miniature microphone array on the welding torch, the temperature field distribution of the molten pool, the solidification front morphology, and the acoustic signature data of the electric arc combustion are collected in real time to generate a subset of process state perception data. S13. By integrating Hall sensors and current transformers into the welding power source and wire feeding mechanism, the voltage, current waveforms and wire feeding resistance fluctuation data of the welding circuit are collected in real time to generate a subset of equipment status perception data. S14. The workpiece status sensing data subset, process status sensing data subset, and equipment status sensing data subset are aggregated to form the original multimodal sensing data set.

[0023] Temporal alignment and spatial coordinate registration in S2 include the following steps: S21. Receive the raw multimodal sensing data set, and standardize and synchronize the timestamps of each subset of data based on a unified global clock source to eliminate the time delay caused by the difference in sampling frequency of different sensors; First, a high-precision timestamp injection mechanism is introduced, using GNSS timing to hard synchronize the clocks of all acquisition nodes, ensuring consistent time starting points for data from all sensors. Second, in the data preprocessing stage, linear interpolation is used to resample and align different modal data such as images, point clouds, and electrical signals. Finally, to eliminate dimensional differences and unify feature scales, multimodal data is normalized; the specific normalization formula is as follows: ; in Data samples were collected from the original sensor. This is the mean of the data in this batch. This represents the standard deviation of the data in this batch. This refers to normalized data that has undergone standardization processing. S22. Based on the pre-established three-dimensional welding workpiece digital twin model and robot kinematics model, assign spatial position labels in the world coordinate system to all perceived data. This step aims to construct a virtual twin that maps to the actual physical welding system, providing a geometric and kinematic foundation for subsequent perception data overlay and motion planning. First, a 3D laser scanner is used to perform reverse engineering scanning of the welded workpiece to obtain its high-precision actual geometric topology. Combined with weld trajectory data from the design drawings, a 3D model with material properties and assembly gaps is reconstructed in virtual space. Second, for the welding industrial robot, its forward and inverse kinematics solution model is established. A homogeneous transformation matrix is ​​used to describe the relationship between the robot's joint angles and the end effector's pose in Cartesian space. S23. Based on the preset welding process data ontology model, perform semantic parsing and relation mapping on the data with unified spatiotemporal labels, and associate data of different modalities and formats with specific weld numbers, weld layers, welding directions and robot joint postures. The Mask R-CNN instance segmentation network from deep learning is used to process images acquired by the visual sensor, and a point cloud clustering algorithm is used to identify geometric defects such as bevel gaps and misalignments. Subsequently, this semantic information is bound and mapped to specific trajectory segments in the robot's kinematic model; the specific spatial relationship projection formula is as follows: ; in This refers to the coordinate vector of feature points acquired by the sensing device in its own local coordinate system. To transform the sensor coordinate system to a rigid body transformation system in a universal coordinate system, These are the absolute coordinates of the feature points in the global world coordinate system; S24. The data that has completed spatiotemporal alignment and semantic association is integrated into a standardized perceptual data stream with coherent contextual information according to the advancement sequence and physical logic of the welding path.

[0024] Generating a real-time dynamic welding quality profile in S3 includes the following steps: S31. Acquire standardized perception data streams and input them into the dynamic quality assessment engine. The engine has a built-in multimodal feature extractor based on the fusion of Transformer and graph neural network to extract deep spatiotemporal coupling features from multi-source heterogeneous data streams. This step designs a cross-modal feature interaction architecture. First, a graph neural network (GNN) is used to construct the topology of multimodal data, aggregating electrical parameters and acoustic signals as node features and spatiotemporal dependencies as edge features. Then, the aggregated graph feature sequence is input into a Transformer encoder, which uses its self-attention mechanism to capture long-range temporal dependencies. Through this fusion mechanism, deep coupling features that cannot be represented by a single modality can be extracted, achieving an increase in the dimension of data perception. S32. The extracted spatiotemporal coupling features are matched and reasoned with the preset welding defect knowledge graph. The welding defect knowledge graph contains defect cause chains and evolution rules constructed from historical non-destructive testing reports, welding process evaluation data and industry expert experience. S33. Quantitatively assess the probability of occurrence and hazard level of the matched potential defect elements, and predict their evolution trend and expansion range in subsequent welds through a time-series prediction model. Based on the extracted deep features, a defect prediction model is constructed. First, a fully connected layer maps high-dimensional features to a low-dimensional probability space, outputting the probability of occurrence of various welding defects. Second, a hazard level assessment matrix is ​​introduced, comprehensively considering the defect location and weld stress, to calculate a comprehensive hazard index. Finally, a long short-term memory network is used to fit the time series of defect features to predict their dimensional changes in future weld passes. The specific hazard level calculation formula is as follows: ; in For comprehensive hazard scoring, For the first Weighting coefficients for class defects For the first Severity rating of the defect type; The radius prediction formula is as follows: ; in For the current moment Predicted high-temperature diffusion radius at the tail of the molten pool. For the predicted time step, The flow velocity of the molten metal in the pool is predicted based on LSTM. The angle between the flow direction and the transverse expansion direction of the weld; S34. Integrate and encapsulate all identified potential defect elements, quantitative evaluation results, evolution trend prediction values, and their associated welding process parameters and spatial location information into a structured real-time dynamic welding quality profile.

[0025] Generating the target policy identifier in S4 includes the following steps: S41. Construct a smart algorithm library for adjusting welding process parameters, which stores preset parameter adjustment strategy models in the form of a vector database, including penetration compensation adjustment model, sidewall fusion enhancement model, residual height control optimization model and spatter suppression adjustment model. S42. Extract key feature vectors from the real-time dynamic welding quality profile. Key feature vectors include defect type encoding, degree of defect deviation tolerance, molten pool oscillation frequency offset, and heat dissipation condition index of the current weld bead. S43. Perform hybrid similarity calculation based on Euclidean distance and cosine similarity on the key feature vectors and the standard applicable feature vectors of each strategy model in the intelligent adjustment algorithm library for welding process parameters. The formula for calculating mixed similarity is as follows: ; ; in This is the feature vector of the quantized welding quality profile collected at the current moment. For the first in the strategy library Standard feature vectors of historical cases The Euclidean distance between the two is... , Let be the weight coefficients of Euclidean distance similarity and cosine similarity, respectively, and satisfy . ; S44. Based on the hybrid similarity calculation results, the optimal parameter adjustment strategy model with the highest comprehensive score is matched through the adaptive strategy matching engine, and its unique encoding is output as the target strategy identifier.

[0026] The execution steps of the adaptive strategy matching engine in S44 are as follows: S441. Initialization: Set the maximum number of iterations for the matching search, and randomly initialize the position and velocity of a group of particles in a high-dimensional decision space composed of the standard applicable feature vectors of all policy models. S442, Exploration phase: Drive the particle swarm to fly in the decision space and calculate the fitness of the feature vector of the policy model corresponding to the new position of each particle and the feature vector of the real-time quality profile. The fitness calculation formula is as follows: ; in For the particle at position fitness value at that location This is the standard applicable feature vector of the policy model corresponding to the new position of the particle. This is the feature vector of the current real-time welding quality profile. This is an adjustment factor for the priority of strategy matching. The estimated time and resource consumption required to switch to and implement this strategy. The adjustment factor for the switching cost penalty coefficient; If the fitness of the new position is better than the individual's historical best position, then update the individual's extreme value; If the fitness of the new position is better than the group's historical best position, then update the global extremum; S443. Utilization phase: guide the particle swarm to converge toward the currently discovered global optimal solution region, perform local search in the neighborhood of the optimal solution, calculate new candidate positions and evaluate their fitness. The formula for updating candidate positions is as follows: ; ; in , For the particle's velocity vector before and after the update, , Let these be the particle's position vectors before and after the update. For inertial weights, , The acceleration factors represent the weights of individual cognition and social cognition, respectively. , In order to be in Uniformly distributed random numbers within an interval. This is the optimal position found through the particle's own history search. The optimal position found throughout the entire particle swarm history. If the fitness of a candidate position is better than that of the current global optimum, then update the global optimum. S444. Repeatedly execute the exploration and exploitation steps until the maximum number of iterations is reached, and output the strategy model code corresponding to the global optimal position, i.e., the target strategy identifier.

[0027] The process of generating strategy execution logs and weld quality calibration data in S5 includes the following steps: S51. Based on the target strategy identifier, call the predefined strategy execution logic script corresponding to the identifier from the process parameter adjustment algorithm library; S52. The predefined strategy execution logic script accesses the standardized perception data stream and real-time dynamic welding quality profile, and combines the real-time pose and kinematic constraints of the current welding robot to generate a detailed instruction set containing specific adjustment parameters, adjustment gradients, effective timing and execution priority. S53. The detailed instruction set is sent to the underlying execution units of the welding robot controller, welding power supply and wire feeding mechanism via the industrial Ethernet protocol. S54. Synchronously monitor the execution process of the command, collect the actual response values ​​of the adjusted welding current, voltage, wire feeding speed and robot motion trajectory in real time, record the execution status, execution results and system feedback, and form a strategy execution log. S55. Within the preset weld length and time period after the strategy is executed, continue to collect subsequent multimodal sensing data, evaluate the reduction effect of welding defects and the improvement of weld formation quality, compare with the expected target, and generate welding quality calibration data. After the strategy has been implemented for a period of time, the multimodal sensing device is reactivated for a new round of data acquisition. To quantify the evaluation effect, the system introduces the intersection-over-union ratio (IoU) and mean absolute error (MAE) indices. First, the adjusted actual weld contour is compared with the ideal standard contour to calculate the geometric overlap. Second, the deviation between the actual measured surface roughness and weld height and the theoretical value is defined as the improvement amount. The specific calculation and judgment logic for the improvement amount is as follows: ; in The average improvement error index for weld formation quality. To assess the number of quality dimensions, For the first Actual measurement values ​​in each dimension For the first Preset target values ​​in each dimension; when If the result is below the preset acceptable threshold, the strategy is deemed effective and the defect is reduced.

[0028] Iterative optimization and self-learning updates in S6 include the following steps: S61. Perform correlation analysis between the strategy execution log and the welding quality calibration data to calculate the effectiveness score of this strategy execution in defect suppression and the efficiency score of parameter adjustment. The formulas for calculating effectiveness and efficiency scores are as follows: ; ; in To score the effectiveness of defect suppression, Efficiency rating for parameter adjustment This represents the area of ​​high-risk defect regions before strategy execution. This represents the area of ​​residual defects after the strategy is executed. This refers to the total changes in the key process parameters involved in this adjustment. This is the conversion factor between time and parameter amplitude. The time cost required to achieve this inhibition rate; S62. Store the complete data package of this response, including real-time dynamic welding quality profile, selected target strategy identifier, execution results and effectiveness score, as a new success or failure case in the historical case library; S63. Based on the historical case library, the welding defect knowledge graph in the dynamic quality assessment engine is updated incrementally on a regular basis, and the parameters of the multimodal feature extractor are fine-tuned and optimized using transfer learning technology. To prevent catastrophic forgetting and accelerate model convergence, the system employs a fine-tuning strategy from transfer learning. First, the parameters of the underlying general feature extraction layer in the multimodal feature extractor, i.e., the basic convolutional blocks of ResNet, are locked, keeping their pre-trained weights unchanged. Then, only the top-level high-level semantic classification layer and fully connected layer are trained using newly generated historical case library data through mini-batch gradient descent. The loss function uses L2-regularized cross-entropy loss, as shown in the following formula: ; in This represents the total loss function value of the model. For the first The true defect label of each sample The model predicts the first The probability that a sample belongs to any type of defect. This represents the number of samples in the current training batch. The regularization coefficient is . This is the trainable parameter vector of the model; S64. Based on the historical case library, the standard applicable feature vectors and matching weights of each strategy model in the intelligent adjustment algorithm library for welding process parameters are regularly adjusted and updated using the Q-learning algorithm in reinforcement learning. The formula for updating the Q value is as follows: ; in In order to be in Always in a state of readiness And perform the action The expected cumulative return after that, For learning rate, For immediate rewards from environmental feedback after an action is performed, It is a discount factor with a value range of [0,1]. For the next state The maximum expected return among all possible actions.

[0029] The system includes: The multimodal perception and data fusion module gathers the raw signals from each stage of the welding station through the multi-source sensor acquisition unit, performs standardization processing using the data time domain alignment unit and the spatial coordinate registration unit, and outputs a standardized perception data stream with spatiotemporal consistency. The dynamic welding quality profile construction module receives standardized perception data streams, identifies potential defects through a multimodal feature extraction unit and a defect knowledge graph reasoning unit, calculates the defect level and evolution trend through a defect quantification and prediction unit, and outputs a structured real-time dynamic welding quality profile. The adaptive strategy matching and decision-making module receives a real-time dynamic welding quality profile, analyzes key features through a feature vectorization unit, uses an adaptive strategy matching engine to search and make decisions in a pre-set process parameter adjustment algorithm library, and outputs a target strategy identifier.

[0030] The system also includes: The strategy precision execution and feedback module receives the target strategy identifier, generates detailed control instructions through the strategy instruction compilation unit, controls the welding robot and welding power supply through the actuator drive interface unit, and outputs strategy execution logs and welding quality calibration data through the execution monitoring and effect evaluation unit. The system's self-learning optimization module receives strategy execution logs and welding quality calibration data, builds a historical case library through the case management and analysis unit, updates the system's core model and strategy using the model incremental training unit and algorithm library tuning unit, and outputs system optimization instructions.

[0031] The operation steps of an adaptive intelligent welding method and system for steel structures based on multimodal perception are as follows: Step 1: Multimodal sensing data acquisition and fusion By deploying a multimodal sensor array at the welding station, the system simultaneously acquires the bevel geometry of the workpiece to be welded, the spectral information of the base material, thermal radiation images of the molten pool during welding, arc acoustic signals, and electrical parameter data, generating a raw multimodal sensing data set. This data set is then temporally aligned and spatially registered to eliminate time delays caused by differences in sampling frequencies between different sensors. All sensing data is assigned a spatial location label in the world coordinate system, ultimately integrating into a standardized sensing data stream with coherent contextual information.

[0032] Step 2: Construction of Dynamic Welding Quality Profile Standardized perceptual data streams are input into a dynamic quality assessment engine, which incorporates a multimodal feature extractor based on the fusion of Transformer and graph neural networks to extract deep-level spatiotemporal coupling features from multi-source heterogeneous data streams. These extracted spatiotemporal coupling features are matched and reasoned against a pre-defined welding defect knowledge graph. The probability and severity of occurrence of the matched potential defect elements are quantitatively assessed, and their evolution trend and expansion range in subsequent weld passes are predicted using a time-series prediction model. Finally, the data is integrated and encapsulated into a structured, real-time dynamic welding quality profile.

[0033] Step 3: Adaptive Strategy Matching and Decision Making Key feature vectors are extracted from the real-time dynamic welding quality profile, including defect type encoding, defect deviation tolerance, molten pool oscillation frequency offset, and the current weld heat dissipation condition index. These key feature vectors are then compared with the standard applicable feature vectors of various strategy models in a pre-built intelligent welding process parameter adjustment algorithm library. A hybrid similarity calculation based on Euclidean distance and cosine similarity is performed. An adaptive strategy matching engine, using a particle swarm optimization algorithm, searches in a high-dimensional decision space to find the optimal parameter adjustment strategy model with the highest comprehensive score. This model is then uniquely encoded and output as the target strategy identifier.

[0034] Step 4: Precise Implementation and Feedback of the Strategy Based on the target strategy identifier, the corresponding predefined strategy execution logic script is called from the process parameter adjustment algorithm library. Combined with the real-time pose and kinematic constraints of the welding robot, a detailed instruction set is generated, including specific adjustment parameters, adjustment gradients, activation timing, and execution priority. This instruction set is then distributed to the underlying execution units of the welding robot controller, welding power supply, and wire feeding mechanism via the industrial Ethernet protocol, enabling real-time dynamic adjustments to the welding robot's walking speed, wire feeding speed, welding current, and oscillation trajectory. The instruction execution process is monitored synchronously, recording execution status and system feedback to form a strategy execution log. Furthermore, multimodal sensing data continues to be collected after strategy execution to evaluate the reduction effect of welding defects and the improvement of weld formation quality, generating welding quality calibration data.

[0035] Step 5: System Self-Learning Optimization The strategy execution logs are correlated with welding quality calibration data to calculate the effectiveness score of defect suppression and the efficiency score of parameter adjustment for this strategy execution. The complete data package of this response is stored as a new success or failure case in the historical case library. Based on the historical case library, the welding defect knowledge graph in the dynamic quality assessment engine is incrementally updated regularly, and the parameters of the multimodal feature extractor are fine-tuned and optimized using transfer learning technology. At the same time, the standard applicable feature vectors and matching weights of each strategy model in the intelligent adjustment algorithm library for welding process parameters are fine-tuned and updated using the Q-learning algorithm in reinforcement learning, so as to achieve continuous iterative optimization of the system's core model and strategy.

[0036] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0037] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An adaptive intelligent welding method for steel structures based on multimodal perception, characterized in that: Includes the following steps: S1. By deploying a multimodal sensor array at the welding station, the bevel geometry of the workpiece to be welded, the spectral information of the base material, the thermal radiation image of the molten pool during the welding process, the arc acoustic signal and electrical parameter data are collected simultaneously to generate the original multimodal sensing data set. S2. Perform temporal domain alignment and spatial coordinate registration on the original multimodal sensing data set to generate a standardized sensing data stream with spatiotemporal consistency; S3. Based on the standardized perception data stream, a dynamic quality assessment engine with an embedded deep learning model is used to identify, quantify and predict the type, location and severity of welding defects in real time, and generate a real-time dynamic welding quality profile. S4. Based on the characteristics of the real-time dynamic welding quality profile, call the adaptive strategy matching engine to match the optimal welding parameter adjustment strategy from the pre-built intelligent adjustment algorithm library of welding process parameters and generate a target strategy identifier. S5. Based on the target strategy identifier, call the corresponding strategy execution module to dynamically adjust the walking speed, wire feeding speed, welding current and oscillation trajectory of the welding robot in real time, and generate strategy execution log and welding quality calibration data. S6. Based on the strategy execution log, welding quality calibration data, and historical welding data, iteratively optimize and self-learn the intelligent adjustment algorithm library for welding process parameters and the dynamic quality assessment engine.

2. The adaptive intelligent welding method for steel structures based on multimodal perception according to claim 1, characterized in that: The generation of the original multimodal sensing data set in S1 includes the following steps: S11. By deploying a laser profile scanner and a hyperspectral camera at the welding start end, data on the width, depth, angle and blunt edge size of the groove to be welded, as well as the alloy element content and surface oxide layer thickness of the base material are collected to generate a subset of workpiece state perception data. S12. By deploying a high-speed infrared thermal imager and a miniature microphone array on the welding torch, the temperature field distribution of the molten pool, the solidification front morphology, and the acoustic signature data of the electric arc combustion are collected in real time to generate a subset of process state perception data. S13. By integrating Hall sensors and current transformers into the welding power source and wire feeding mechanism, the voltage, current waveforms and wire feeding resistance fluctuation data of the welding circuit are collected in real time to generate a subset of equipment status perception data. S14. The workpiece state perception data subset, process state perception data subset, and equipment state perception data subset are aggregated to form the original multimodal perception data set.

3. The adaptive intelligent welding method for steel structures based on multimodal perception according to claim 2, characterized in that: The temporal alignment and spatial coordinate registration in S2 include the following steps: S21. Receive the original multimodal sensing data set, and standardize and synchronize the timestamps of each subset of data based on a unified global clock source to eliminate the time delay caused by the difference in sampling frequency of different sensors; S22. Based on the pre-established three-dimensional welding workpiece digital twin model and robot kinematics model, assign spatial position labels in the world coordinate system to all perceived data. S23. Based on the preset welding process data ontology model, perform semantic parsing and relation mapping on the data with unified spatiotemporal labels, and associate data of different modalities and formats with specific weld numbers, weld layers, welding directions and robot joint postures. S24. The data that has completed spatiotemporal alignment and semantic association is integrated into a standardized perceptual data stream with coherent contextual information according to the advancement sequence and physical logic of the welding path.

4. The adaptive intelligent welding method for steel structures based on multimodal perception according to claim 3, characterized in that: The process of generating a real-time dynamic welding quality profile in S3 includes the following steps: S31. Obtain the standardized perception data stream and input it into the dynamic quality assessment engine. The engine has a built-in multimodal feature extractor based on the fusion of Transformer and graph neural network to extract deep spatiotemporal coupling features from multi-source heterogeneous data streams. S32. The extracted spatiotemporal coupling features are matched and reasoned with the preset welding defect knowledge graph, which includes defect cause chains and evolution rules constructed from historical nondestructive testing reports, welding process evaluation data and industry expert experience. S33. Quantitatively assess the probability of occurrence and hazard level of the matched potential defect elements, and predict their evolution trend and expansion range in subsequent welds through a time-series prediction model. S34. Integrate and encapsulate all identified potential defect elements, quantitative evaluation results, evolution trend prediction values, and their associated welding process parameters and spatial location information into a structured real-time dynamic welding quality profile.

5. The adaptive intelligent welding method for steel structures based on multimodal perception according to claim 4, characterized in that: The generation of the target policy identifier in step S4 includes the following steps: S41. Construct a smart algorithm library for adjusting welding process parameters, which stores preset parameter adjustment strategy models in the form of a vector database, including penetration compensation adjustment model, sidewall fusion enhancement model, residual height control optimization model and spatter suppression adjustment model. S42. Extract the key feature vector from the real-time dynamic welding quality profile. The key feature vector includes the defect type code, the degree of defect deviation tolerance, the offset of the molten pool oscillation frequency, and the heat dissipation condition index of the current weld bead. S43. Perform a hybrid similarity calculation based on Euclidean distance and cosine similarity between the key feature vector and the standard applicable feature vector of each strategy model in the intelligent adjustment algorithm library for welding process parameters; S44. Based on the hybrid similarity calculation results, the optimal parameter adjustment strategy model with the highest comprehensive score is matched through the adaptive strategy matching engine, and its unique encoding is output as the target strategy identifier.

6. The adaptive intelligent welding method for steel structures based on multimodal perception according to claim 5, characterized in that: The execution steps of the adaptive strategy matching engine in S44 are as follows: S441. Initialization: Set the maximum number of iterations for the matching search, and randomly initialize the position and velocity of a group of particles in a high-dimensional decision space composed of the standard applicable feature vectors of all policy models. S442, Exploration phase: Drive the particle swarm to fly in the decision space and calculate the fitness of the feature vector of the policy model corresponding to the new position of each particle and the feature vector of the real-time quality profile. If the fitness of the new position is better than the individual's historical best position, then update the individual's extreme value; If the fitness of the new position is better than the group's historical best position, then update the global extremum; S443. Utilization phase: guide the particle swarm to converge toward the currently discovered global optimal solution region, perform local search in the neighborhood of the optimal solution, calculate new candidate positions and evaluate their fitness. If the fitness of a candidate position is better than that of the current global optimum, then update the global optimum. S444. Repeatedly execute the exploration and utilization steps until the maximum number of iterations is reached, and output the strategy model code corresponding to the global optimal position, i.e., the target strategy identifier.

7. The adaptive intelligent welding method for steel structures based on multimodal perception according to claim 6, characterized in that: The generation of strategy execution logs and welding quality calibration data in S5 includes the following steps: S51. Based on the target strategy identifier, call the predefined strategy execution logic script corresponding to the identifier from the process parameter adjustment algorithm library; S52. The predefined strategy execution logic script accesses the standardized perception data stream and the real-time dynamic welding quality profile, and combines the real-time pose and kinematic constraints of the current welding robot to generate a detailed instruction set containing specific adjustment parameters, adjustment gradients, effective timing and execution priority. S53. The detailed instruction set is sent to the underlying execution units of the welding robot controller, welding power supply and wire feeding mechanism via the industrial Ethernet protocol; S54. Synchronously monitor the execution process of the command, collect the actual response values ​​of the adjusted welding current, voltage, wire feeding speed and robot motion trajectory in real time, record the execution status, execution results and system feedback, and form a strategy execution log. S55. Within the preset weld length and time period after the strategy is executed, continue to collect subsequent multimodal sensing data, evaluate the reduction effect of welding defects and the improvement of weld formation quality, compare with the expected target, and generate welding quality calibration data.

8. The adaptive intelligent welding method for steel structures based on multimodal perception according to claim 7, characterized in that: The iterative optimization and self-learning update in S6 include the following steps: S61. Perform correlation analysis between the strategy execution log and the welding quality calibration data, and calculate the effectiveness score of this strategy execution for defect suppression and the efficiency score of parameter adjustment. S62. Store the complete data package of this response, including the real-time dynamic welding quality profile, the selected target strategy identifier, the execution result and effectiveness score, as a new success or failure case in the historical case library. S63. Based on the historical case library, the welding defect knowledge graph in the dynamic quality assessment engine is updated incrementally on a regular basis, and the parameters of the multimodal feature extractor are fine-tuned and optimized using transfer learning technology. S64. Based on the historical case library, the standard applicable feature vectors and matching weights of each strategy model in the intelligent adjustment algorithm library for welding process parameters are periodically adjusted and updated using the Q-learning algorithm in reinforcement learning.

9. A multimodal sensing-based adaptive intelligent welding system for steel structures, used to implement the multimodal sensing-based adaptive intelligent welding method for steel structures according to any one of claims 1-8, characterized in that: The system includes: The multimodal perception and data fusion module gathers the raw signals from each stage of the welding station through the multi-source sensor acquisition unit, performs standardization processing using the data time domain alignment unit and the spatial coordinate registration unit, and outputs a standardized perception data stream with spatiotemporal consistency. The dynamic welding quality profile construction module receives the standardized perception data stream, identifies potential defects through a multimodal feature extraction unit and a defect knowledge graph reasoning unit, calculates the defect level and evolution trend through a defect quantification and prediction unit, and outputs a structured real-time dynamic welding quality profile. The adaptive strategy matching and decision-making module receives the real-time dynamic welding quality profile, analyzes key features through the feature vectorization unit, searches and makes decisions in a preset process parameter adjustment algorithm library using the adaptive strategy matching engine, and outputs the target strategy identifier.

10. The adaptive intelligent welding system for steel structures based on multimodal perception according to claim 9, characterized in that: The system also includes: The strategy precision execution and feedback module receives the target strategy identifier, generates detailed control instructions through the strategy instruction compilation unit, controls the welding robot and welding power supply through the actuator drive interface unit, and outputs strategy execution logs and welding quality calibration data through the execution monitoring and effect evaluation unit. The system self-learning optimization module receives the strategy execution log and welding quality calibration data, builds a historical case library through the case management and analysis unit, updates the system core model and strategy using the model incremental training unit and algorithm library tuning unit, and outputs system optimization instructions.

Images