Welding process control model construction method, welding process control method and device

Abstract

This invention relates to the field of industrial artificial intelligence technology, and discloses a method for constructing a welding process control model, a welding process control method, and an apparatus. The method for constructing the welding process control model includes: constructing a triplet training dataset containing multimodal inputs, physical reasoning thought chain text, and strategic intent; based on the training dataset, supervising fine-tuning a pre-trained visual-language-action large model to obtain a fine-tuned large model; constructing positive and negative sample reasoning path pairs, and using a direct preference optimization algorithm to perform preference alignment training on the fine-tuned large model to obtain the welding process control model. This invention performs supervised fine-tuning and preference alignment training on the visual-language-action large model, enabling the model to possess a deep physical cognition ability of the dynamic behavior of the weld pool. It can generate thought chain text containing phenomenon observation, physical attribution, and strategic intent based on multimodal perception information, thereby realizing the understanding of the physical state of the welding process and causal reasoning of the causes of defects.

Description

Technical Field

[0001] This invention relates to the field of industrial artificial intelligence technology, specifically to a method for constructing a welding process control model, a welding process control method, and an apparatus. Background Technology

[0002] Laser welding is characterized by strong nonlinearity and transient multiphysics coupling, and its quality is highly dependent on the complex interaction between the dynamic behavior of the molten pool and process parameters. However, existing automated laser welding systems mainly rely on preset parameters and low-dimensional feedback control, and their technical approaches generally suffer from the following limitations: First, open-loop control based on a fixed process window lacks dynamic reasoning capabilities; second, single-modal sensing feedback can only perceive surface phenomena such as molten pool brightness and width, lacking semantic-level understanding of the physical process; third, although data-driven black-box models can directly regress quality, they lack explicit physical cognition, making the decision results difficult to interpret; fourth, although imitation learning schemes can reproduce operating skills under specific working conditions, they are essentially shallow trajectory fitting and are difficult to cope with environmental changes.

[0003] The aforementioned deficiencies prevent existing systems from performing causal attribution analysis on welding defects, lack dynamic adjustment and cognitive closed loops from human-like experts, make it difficult to integrate multimodal process knowledge, and lack a continuous evolution mechanism to optimize welding strategies by using post-weld quality information. Summary of the Invention

[0004] This invention provides a method for constructing a welding process control model, a welding process control method, and an apparatus to address the problem that existing laser welding systems lack a deep understanding of the welding physical process and the ability to reason causally, thus failing to achieve closed-loop control similar to that of human experts.

[0005] In a first aspect, the present invention provides a method for constructing a welding process control model, the method comprising: Construct a triplet training dataset containing multimodal inputs, physical reasoning thought chain texts, and policy intentions; Based on the triplet training dataset, the pre-trained large vision-language-action model is fine-tuned under supervision to obtain the fine-tuned large model. Positive and negative sample inference path pairs are constructed, and the direct preference optimization algorithm is used to train the fine-tuned large model for preference alignment to obtain the welding process control model.

[0006] This invention provides a method for constructing a welding process control model. By constructing a triplet dataset containing physical reasoning thought chains and strategic intentions, the method performs supervised fine-tuning and preference alignment training on a large vision-language-action model. This enables the model to possess a deep physical cognition ability of the dynamic behavior of the weld pool. Based on multimodal perception information, it can generate thought chain text containing phenomenon observation, physical attribution, and strategic intentions, thereby realizing the understanding of the physical state of the welding process and causal reasoning of the causes of defects. This solves the technical problem of existing methods that can only perceive surface phenomena and lack physical cognition and attribution capabilities, and realizes human-like expert closed-loop control of the welding process.

[0007] In one alternative implementation, a pre-trained large vision-language-action model is supervised fine-tuned based on a triplet training dataset to obtain a fine-tuned large model, including: Use the pre-trained vision-language-action large model as the base model, and load the pre-trained weights of the base model; Based on pre-trained weights, the physical reasoning thought chain text is input into the base model, and the full parameters are fine-tuned using the next word prediction method. The model is forced to establish conditional probability dependencies from visual perception to logical reasoning to policy output through a label masking strategy, resulting in a fine-tuned large model.

[0008] In the above technical solution, by loading pre-trained weights and using text data containing physical reasoning thought chains and strategic intent as training data, and taking word prediction as the training objective, the base model is fine-tuned with full parameter supervision. A label masking strategy is employed to force the model to establish conditional probability dependencies from multimodal perceptual input to physical logic reasoning and then to policy output, thereby effectively injecting physical knowledge from the welding field into the large model. This enables the trained large model to judge the physical state and attribute causality to the dynamic process of the weld pool, understand the physical stage of the weld pool, and infer the root cause of potential defects, solving the technical problem that existing methods can only perceive surface phenomena of the weld pool and lack deep physical understanding.

[0009] In one optional implementation, positive and negative sample inference path pairs are constructed, and the fine-tuned large model is trained with a direct preference optimization algorithm to achieve preference alignment, thereby obtaining a welding process control model, including: Construct positive sample reasoning paths that conform to physical common sense and welding strategies, and negative sample pairs that violate physical common sense, to obtain positive and negative sample reasoning path pairs; Based on positive and negative sample inference path pairs, the direct preference optimization algorithm is used to train the model for preference alignment, thus obtaining the welding process control model.

[0010] In the above technical solution, positive sample reasoning paths that conform to physical facts and negative sample pairs that violate common sense are constructed. A direct preference optimization algorithm is then used to train the fine-tuned large model with preference alignment, causing the model's reasoning path to converge to true physical causal logic. This method effectively constrains the model's reasoning process, ensuring that its generated thought chains strictly follow the physical laws of welding, avoiding reasoning illusions that violate fundamental physical principles such as energy conservation or heat conduction. This significantly enhances the model's accuracy in judging the physical state of the welding process and the reliability of causal attribution, solving the technical problem of existing black-box models lacking physical constraints in their reasoning process and having unreliable decision-making results.

[0011] Secondly, the present invention provides a welding process control method, the method comprising: Multimodal perception information during the welding process is acquired and mapped to the semantic embedding space to obtain aligned multimodal features; Multimodal features are input into the welding process control model constructed using the welding process control model construction method of the first aspect or any of its corresponding embodiments to perform physical state reasoning, and physical reasoning thought chain text and implicit policy vector are generated. Based on implicit policy vectors and real-time perception information, a continuous motion trajectory is generated and sent to the welding actuator to execute the continuous motion trajectory.

[0012] This invention provides a welding process control method that, through semantic alignment and fusion of multimodal perception information, enables the system to comprehensively acquire visual, state, and process context information of the welding process. Then, it utilizes a welding process control model with physical cognitive capabilities to perform reasoning, generating thought chain text containing physical state judgments and causal attributions. This gives the system a deep physical cognition and defect cause tracing capability similar to that of human experts. Simultaneously, by encoding the reasoning results into implicit policy vectors and combining them with real-time perception information to generate continuous action trajectories, closed-loop control from physical cognition to precise execution is achieved. This method solves the technical problems of existing welding control systems that can only perceive surface phenomena of the molten pool, lack a deep understanding of the physical process and causal reasoning capabilities, and are difficult to achieve adaptive dynamic adjustment.

[0013] In one alternative implementation, the multimodal perception information includes: a visual image of the molten pool, robot body perception information, and process context information; Multimodal awareness information is mapped to the semantic embedding space to obtain aligned multimodal features, including: The visual image of the molten pool is input into a visual encoder to extract visual feature vectors; The robot's perception information and welding equipment parameters are input into the state encoder to extract the state feature vector. The process context information is input into the text encoder to extract semantic feature vectors; Visual feature vectors, state feature vectors, and semantic feature vectors are projected into a unified semantic embedding space, and cross-modal feature alignment and fusion are performed to obtain aligned multimodal features.

[0014] In the above technical solution, modality-specific feature encoding is performed on the visual image of the molten pool, the robot's body state and welding equipment parameters, and process context information. The extracted visual, state, and semantic feature vectors are then projected onto a unified semantic embedding space for cross-modal alignment and fusion, achieving a unified representation of heterogeneous multi-source information. This method effectively solves the heterogeneity problem of different modal data in terms of time scale, physical meaning, and expression form, enabling the system to integrate complementary information from visual dynamics, equipment state, and process knowledge to form a complete and consistent multi-dimensional description of the welding process.

[0015] In one optional implementation, multimodal features are input into a welding process control model constructed using the welding process control model construction method of the first aspect or any corresponding implementation thereof, and physical state reasoning is performed to generate physical reasoning thought chain text and implicit policy vectors, including: Based on multimodal features, the next-word prediction method is used to generate physical reasoning thought chain text, and based on the physical reasoning thought chain text, the corresponding strategy intent text is generated; the physical reasoning thought chain text includes observation and description of the current welding phenomenon, judgment of physical state, and causal attribution analysis of potential defect risks. Implicit policy vectors are generated synchronously with the physical reasoning thought chain text; the implicit policy vectors are implicit representations of the physical control intentions expressed in the policy intention text, and are used to directly guide the generation of downstream actions.

[0016] In the aforementioned technical solution, multimodal features are input into a welding process control model with physical cognitive capabilities. A next-term prediction method is used to generate a physical reasoning thought chain text that includes phenomenon observation, physical state judgment, and defect causal attribution. Simultaneously, an implicit policy vector encoding the corresponding physical control intent is output. This method enables the system to perform deep physical state understanding and causal reasoning of the welding process, much like a human expert. It transforms the invisible dynamic behavior of the molten pool into an interpretable semantic description, while encoding the reasoning results into machine-executable implicit representations. This achieves a complete mapping from physical cognition to control intent, solving the technical problem of existing methods that can only perceive surface phenomena of the molten pool and lack the ability to understand the physical process and perform causal attribution.

[0017] In one optional implementation, a continuous motion trajectory is generated based on implicit policy vectors and real-time sensing information, and then sent to the welding actuator to execute a smooth motion trajectory, including: The implicit policy vector is fused with real-time perception information and then input into the conditional flow matching network. Continuous motion trajectories are generated by solving differential equations. The continuous motion trajectory is parsed into motion commands that the welding actuator can recognize, and the motion commands are sent to the welding robot and welding equipment in real time to drive the welding actuator to complete a smooth and continuous welding operation.

[0018] In the above technical solution, implicit policy vectors are fused with real-time perception information and input into a conditional flow matching network. Continuous motion trajectories are generated by solving ordinary differential equations, and these trajectories are parsed into action commands recognizable by the welding robot and welding equipment for real-time distribution. This method utilizes the generative modeling capabilities of flow matching to achieve a smooth mapping from high-level physical control intentions to high-frequency continuous actions, effectively avoiding the motion jitter and discontinuity problems caused by traditional discrete command control. Simultaneously, by fusing real-time perception information, the generated motion trajectories can respond to the dynamic changes in the welding process, ensuring the execution layer's adaptive adjustment to changes in the physical environment. This solves the technical problems of existing methods exhibiting stiff movements, lack of continuity, and insufficient real-time response capabilities in complex welding scenarios.

[0019] In one optional embodiment, the welding process control method further includes: The system compares real-time perceived information during the welding process with the expected state generated by the welding process control model based on historical reasoning. When deviations exist, the implicit policy vector is dynamically updated to drive the smooth correction of subsequent action trajectories.

[0020] In the above technical solution, by comparing the perceived information during the welding process with the expected state generated by the model based on historical reasoning in real time, the implicit policy vector is dynamically updated when deviations occur to drive the smooth correction of subsequent action trajectories, thus establishing a complete cognitive closed loop of "perception-reasoning-expectation-correction". This method enables the system to have self-reflective capabilities similar to human experts, enabling it to monitor whether the welding process deviates from the expected physical state in real time, and to proactively adjust the policy based on cause and effect when anomalies occur. At the same time, the smooth correction mechanism avoids secondary disturbances to welding quality caused by sudden changes in actions, solving the technical problem that existing control systems lack dynamic adaptive capabilities and cannot correct deviations in real time when facing fluctuations in operating conditions or environmental disturbances.

[0021] Thirdly, the present invention provides a welding process control model construction device, the device comprising: The triplet training dataset construction module is used to build a triplet training dataset that includes multimodal inputs, physical reasoning thought chain texts, and policy intentions. The supervised fine-tuning module is used to perform supervised fine-tuning on the pre-trained large vision-language-action model based on the triplet training dataset, so as to obtain the fine-tuned large model. The preference alignment training module is used to construct positive and negative sample inference path pairs. The direct preference optimization algorithm is used to perform preference alignment training on the fine-tuned large model to obtain the welding process control model.

[0022] Fourthly, the present invention provides a welding process control device, the device comprising: The multimodal perception and semantic alignment module is used to acquire multimodal perception information during the welding process and map the multimodal perception information to the semantic embedding space to obtain aligned multimodal features; The cognitive reasoning module is used to input multimodal features into the welding process control model constructed using the welding process control model construction method of the first aspect or any of its corresponding embodiments to perform physical state reasoning, and generate physical reasoning thought chain text and implicit policy vector. The motion generation module is used to generate continuous motion trajectories based on implicit policy vectors and real-time perception information, and then send them to the welding actuator to execute the continuous motion trajectories.

[0023] Fifthly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the welding process control model construction method of the first aspect or any corresponding embodiment thereof, or to perform the welding process control method of the second aspect or any corresponding embodiment thereof.

[0024] In a sixth aspect, the present invention provides a computer-readable storage medium storing computer instructions, the computer instructions being configured to cause a computer to execute the welding process control model construction method of the first aspect or any corresponding embodiment thereof, or to execute the welding process control method of the second aspect or any corresponding embodiment thereof.

[0025] In a seventh aspect, the present invention provides a computer program product, including computer instructions, which are used to cause a computer to execute the welding process control model construction method of the first aspect or any corresponding embodiment thereof, or to execute the welding process control method of the second aspect or any corresponding embodiment thereof. Attached Figure Description

[0026] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram of an application scenario according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating the welding process control model construction method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the first type of welding process control method according to an embodiment of the present invention; Figure 4 This is a schematic diagram of a second process control method for welding according to an embodiment of the present invention; Figure 5 This is a structural block diagram of a welding process control model construction device according to an embodiment of the present invention; Figure 6 This is a structural block diagram of a welding process control device according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0030] As an optional application scenario of this invention, such as Figure 1 As shown, this invention provides a welding process control system. Inspired by the concept of "brain planning and cerebellum fine-tuning" in biological neuroscience, the system constructs a hierarchical, cerebellum-cerebellum collaborative architecture. By injecting welding physics knowledge into the "brain side" for logical reasoning and common-sense constraints on high-level process states, and injecting multimodal ontological perception and introspective feedback mechanisms into the "cerebellum side," precise alignment between process strategies and underlying mechanical actions is achieved. Through multimodal feature alignment and a physical thought chain reasoning mechanism, the system forms an integrated intelligent closed loop of "perception-understanding-reasoning-decision-introspection."

[0031] The system includes: a laser welding multimodal perception and semantic alignment module, a process reasoning module with physical motion cognition (analogous to the brain), and a flow-matching-based action generation module (analogous to the cerebellum, used for high-frequency execution and trajectory generation). Among them: The laser welding multimodal perception and semantic alignment module, as the system's perception front-end, is responsible for unified modeling and spatiotemporal alignment of heterogeneous multi-source information during the welding process, providing comprehensive data support for the entire system. The input data specifically includes: External visual perception information: Microscopic video streams of the molten pool acquired by coaxial or off-axis high-speed cameras, capturing physical appearances including molten pool brightness distribution, keyhole morphology, and liquid metal flow characteristics; Robot proprioceptive information: Simulating biological proprioception, collecting joint angles of the welding robot, real-time pose (position and attitude quaternions) of the end effector (Tool Center Point, TCP), instantaneous velocity, acceleration, and real-time feedback status of the laser (actual output power, light emission signal, alarm code). Process context information: includes a text description of the current welding task (e.g., “3mm stainless steel weld”), a table of preset process parameters, and historical operation records.

[0032] The system maps the aforementioned visual features and ontological perception states to a unified semantic embedding space through multimodal encoders (such as ViT (Vision Transformer) and MLP (Multilayer Perceptron) projection layers), enabling the model to simultaneously understand the relationship between "external environment changes" and "its own action state," providing aligned feature representations for subsequent cognitive reasoning.

[0033] The process reasoning module (analogous to the brain), possessing physical motion cognition, serves as the core cognitive center of the system. It is built upon a Transformer-based VLA (Vision-Language-Action) large-scale model. Unlike traditional rule controllers or black-box neural networks, this module, through post-training with domain knowledge injection and multimodal semantic alignment technology, enables the general-purpose large-scale model to understand the deep physical mechanisms of laser welding. Instead of directly outputting motor pulses, this module generates structured text sequences containing physical state judgments, causal analysis, and high-level strategic intentions, providing interpretable "cognitive guidance" to the downstream execution layer.

[0034] The flow-matching-based action generation module serves as the real-time motion control center of the system. It sheds the burden of complex logical reasoning, focusing instead on high-frequency, precise action generation. Unlike traditional deterministic regression networks, this module employs a generative policy network based on flow matching. It uses the abstract policy output by the brain as a condition, mapping Gaussian noise into continuous action trajectories that conform to physical laws, achieving real-time response.

[0035] In the process reasoning module with physical motion cognition, a method for constructing a welding process control model is provided. This method is a large-scale VLA (Vision-Language-Action) model construction method based on the Transformer architecture. Figure 2 This is a flowchart of a welding process control model construction method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps: Step S201: Construct a triplet training dataset containing multimodal inputs, physical reasoning thought chain texts, and policy intentions.

[0036] Specifically, a triplet dataset consisting of <multimodal input, physical reasoning thought chain text, and policy intent> was constructed, where: Multimodal input: includes an embedded vector of aligned molten pool image feature sequence and robot body state (joint current, TCP (Tool Center Point, such as the reference coordinate point of the robot end effector (such as the welding torch) velocity, etc.).

[0037] Mind chain annotation: Unlike simple tag classification, we manually or semi-automatically synthesize physical reasoning thought chain text describing physical processes. For example: The current molten pool width fluctuation variance exceeds the threshold (observation), and the laser reflectivity exhibits high-frequency oscillations (phenomenon). Combined with the current 3000W high-power output stage (context), it is inferred that the gas pressure inside the keyhole is unstable, and there is a risk of collapse (physical attribution).

[0038] Strategic intent: High-level recommendations based on the above reasoning, such as "energy density should be reduced to stabilize the keyhole."

[0039] For example, triplet training data ,in: : Current multimodal input (melt pool image + body state).

[0040] : Expert thought chain text. This is the training objective for the large model (slow flow), used to explain physical states.

[0041] High-frequency motion trajectory segments. This is the training objective for the small model (fast flow).

[0042] Step S202: Based on the triplet training dataset, supervised fine-tuning is performed on the pre-trained vision-language-action large model to obtain the fine-tuned large model.

[0043] This step belongs to the first stage of supervised fine-tuning of the two-stage training strategy. In an optional implementation, step S202 includes: Step S2021: Use the pre-trained vision-language-action large model as the base model and load the pre-trained weights of the base model.

[0044] Specifically, the vision-language-action big model is a multimodal artificial intelligence model that can simultaneously understand visual input (such as molten pool images), process language instructions (such as process descriptions), and directly generate control actions (such as robot joint instructions).

[0045] Pre-trained weights refer to the parameter file obtained after a large-scale neural network model has undergone the first stage of self-supervised learning on a large-scale general dataset (such as image-text pairs or video data).

[0046] Step S2022: Based on the pre-trained weights, the physical reasoning thought chain text is input into the base model, and the full parameters are fine-tuned using the next word prediction method. The model is forced to establish the conditional probability dependency from visual perception to logical reasoning to policy output through the label masking strategy, and the fine-tuned large model is obtained.

[0047] Specifically, domain knowledge injection involves fully fine-tuning the pre-trained VLA base model (such as Qwen3-VL and other VLMs) using the aforementioned dataset. The training objective is to minimize the autoregressive loss of thought chain text generation, forcing the model to "learn" to generate a reasoning process that conforms to physical logic before outputting a decision.

[0048] Full parameter fine-tuning details: Build a unified Prompt template that conforms to multimodal dialogue standards (such as ChatML): <|im_start|>user\n<Image_Tokens> \n <instruction><|im_end|>\n<|im_start|>assistant\n<CoT_Reasoning> \n<Action_Output> <|im_end|>.

[0049] When calculating the autoregressive cross-entropy loss, a strict label masking strategy is implemented: a label tensor of the same length as the input sequence is constructed, and the label at the input end is masked.<Image_Tokens> (Image token) <instruction>The label values ​​of the (instruction) and all system-specific tokens are set to ignore_index (usually -100) to mask them during gradient calculation. Only the thought chain reasoning is retained.<CoT_Reasoning> and action output<Action_Output> (and its terminator) participate in backpropagation. This aims to eliminate the interference of the input distribution on the gradient, forcing the model to establish a strong conditional probability dependency P("Action"|"Image","Instruction","CoT") from "visual perception" to "logical reasoning" and then to "precise decision-making", rather than simply memorizing the input information, where Action is the action output, Image is the image, "Instruction" is the instruction, and CoT is the thought chain.

[0050] Step S203: Construct positive and negative sample inference path pairs, and use the direct preference optimization algorithm to perform preference alignment training on the fine-tuned large model to obtain the welding process control model.

[0051] Specifically, this step belongs to the second stage of the two-stage training strategy, Physically Constrained Preference Alignment (DPO / RLHF). In order to further reduce the "illusion" of the model and ensure that the inference conforms to the physical conservation law, Direct Preference Optimization (DPO) is introduced.

[0052] In an optional implementation, step S203 includes: Step S2031: Construct positive sample reasoning paths that conform to physical common sense and welding strategies, and negative sample pairs that violate physical common sense, to obtain positive and negative sample reasoning path pairs.

[0053] Specifically, paired inference data are constructed, and inferences that conform to physical common sense and welding strategies are marked as "positive samples," while inferences that violate physical common sense (such as "increased speed leads to increased heat input") are marked as "negative samples." By optimizing the strategy model, its inference path is made to strictly converge to the true physical causal logic.

[0054] Step S2032: Based on the positive and negative sample inference path pairs, the direct preference optimization algorithm is used to train the model for preference alignment to obtain the welding process control model.

[0055] Specifically, by constructing positive sample reasoning paths that conform to physical facts and negative sample pairs that violate physical common sense, the direct preference optimization algorithm is used to train the fine-tuned large model with preference alignment, so that the model's reasoning path strictly converges to the real physical causal logic, significantly enhancing the model's accurate understanding of the welding physical process and its causal attribution ability, and effectively suppressing reasoning illusions that violate basic principles such as energy conservation or heat conduction, a welding process control model with physical common sense constraints and reliable decision-making ability is finally obtained.

[0056] The welding process control model constructed by the method provided in this embodiment is a fast-slow dual-flow collaborative control architecture based on the VLA model, that is, a hierarchical collaborative architecture of "brain" (cognitive reasoning layer) and "cerebellum" (action execution layer). The "brain," based on a large vision-language-action model, is responsible for low-frequency welding physical state reasoning and macro-strategy planning; the "cerebellum" is responsible for receiving the brain's strategy and combining it with real-time feedback to generate high-frequency actions. This architecture effectively solves the problems of large model reasoning latency and the need for strong alignment between industrial semantic intent and physical actions.

[0057] Establish a unified semantic embedding space for vision, robot state, and process text. Introduce semantic-action consistency constraints during training to maximize the mutual information between the thought chain text and policy vectors, ensuring that the cognitive intent generated by the large model (such as "suppress spatter") can be accurately and losslessly translated into underlying physical control signals. Address the contradiction between the millisecond-level response requirements of welding.

[0058] According to an embodiment of the present invention, a welding process control method embodiment is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0059] This embodiment provides a welding process control method, which can be used in the aforementioned welding process control system. Figure 3 This is a flowchart of a welding process control method according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps: Step S301: Obtain multimodal perception information during the welding process and map the multimodal perception information to the semantic embedding space to obtain aligned multimodal features.

[0060] Specifically, the laser welding multimodal perception and semantic alignment module (multi-sensory input) in the welding process control system is responsible for unified modeling and spatiotemporal alignment of heterogeneous multi-source information during the welding process, providing comprehensive data support for the "big brain" (both the sensory and non-sensory aspects of the system). The input data specifically includes: External visual perception information: Microscopic video streams of the molten pool acquired by coaxial or off-axis high-speed cameras, capturing physical appearances including molten pool brightness distribution, keyhole morphology, and liquid metal flow characteristics; Robot body perception information: Simulate biological proprioception to collect the joint angles of the welding robot, the real-time pose (position and attitude quaternions) of the end effector (TCP), instantaneous velocity, acceleration, and the real-time feedback status of the laser (actual output power, light emission signal, alarm code). Process context information: includes a text description of the current welding task (e.g., "3mm stainless steel welding"), a table of preset process parameters, and historical operation records.

[0061] The system maps the aforementioned visual features and ontological perception states to a unified semantic embedding space through multimodal encoders (such as ViT and MLP projection layers), enabling the model to simultaneously understand the relationship between "external environment changes" and "its own action state," providing aligned feature representations for subsequent cognitive reasoning and obtaining multimodal features.

[0062] Step S302: Input the multimodal features into the welding process control model constructed using the welding process control model construction method to perform physical state reasoning, and generate physical reasoning thought chain text and implicit policy vector.

[0063] Among them, the physical reasoning thought chain text is an interpretable natural language description generated by the model, which includes observations of welding phenomena, judgments of physical states, and causal attribution analysis of defect causes.

[0064] The implicit policy vector is a high-dimensional numerical representation of the model and the thought chain text output synchronously. It encodes the physical control intention expressed by the policy intention text and is used to directly guide the downstream action generation module.

[0065] Specifically, in order to ensure that the large model's "thoughts (introspection results)" can be "precisely executed" by the cerebellum, that is, to establish a mapping relationship between cognitive intentions and physical actions, this embodiment designs a hierarchical training strategy that includes semantic alignment constraints.

[0066] This step utilizes a unified semantic embedding space encompassing vision, robot state, and process text. During training, semantic-action consistency constraints are introduced to maximize the mutual information between the thought chain text and the policy vector. This ensures that the cognitive intent generated by the large model (such as "splash suppression is required") can be accurately and losslessly transformed into underlying physical control signals, ultimately yielding the physical reasoning thought chain text and implicit policy vector.

[0067] Step S303: Based on the implicit policy vector and real-time perception information, a continuous motion trajectory is generated and sent to the welding actuator to execute the continuous motion trajectory.

[0068] Specifically, this is achieved through a flow-matching-based action generation module ("cerebellum," used for high-frequency execution and trajectory generation). This module acts as the real-time motion control center of the system, stripping away the burden of complex logical reasoning and focusing on high-frequency, precise action generation. Unlike traditional deterministic regression networks, this module employs a generative policy network based on flow matching. It uses the abstract policy output by the "brain" as a condition, mapping Gaussian noise into a continuous action trajectory that conforms to physical laws, and then sends it to the welding actuator to execute the continuous action trajectory, achieving real-time response.

[0069] The welding process control method provided in this embodiment enables the system to comprehensively acquire visual, state, and process context information of the welding process through semantic alignment and fusion of multimodal perception information. It then utilizes a welding process control model with physical cognitive capabilities to perform reasoning, generating thought chain text containing physical state judgments and causal attributions. This gives the system a deep physical cognition and defect cause tracing capability similar to that of human experts. Simultaneously, by encoding the reasoning results into implicit policy vectors and combining them with real-time perception information to generate continuous action trajectories, closed-loop control from physical cognition to precise execution is achieved. This method solves the technical problems of existing welding control systems that can only perceive surface phenomena of the molten pool, lack a deep understanding of the physical process and causal reasoning capabilities, and are difficult to achieve adaptive dynamic adjustment.

[0070] This embodiment provides a welding process control method, which can be used in the aforementioned welding process control system. Figure 4 This is a flowchart of a welding process control method according to an embodiment of the present invention, such as... Figure 4 As shown, the process includes the following steps: Step S401: Obtain multimodal perception information during the welding process and map the multimodal perception information to the semantic embedding space to obtain aligned multimodal features.

[0071] Specifically, the multimodal perception information includes: a visual image of the molten pool, robot body perception information, and process context information; step S401 above includes: Step S4011: Input the molten pool visual image into the visual encoder to extract the visual feature vector.

[0072] Specifically, visual images of the molten pool acquired during the welding process are input into a visual encoder (such as VisionTransformer). The encoder uses a multi-layer Transformer structure to perform block embedding and self-attention calculation on the images, extracting visual feature vectors that include molten pool morphology, brightness distribution, keyhole dynamics, and liquid metal flow characteristics, thereby achieving a deep characterization of the microscopic physical appearance of the molten pool.

[0073] Step S4012: Input the robot body perception information and welding equipment parameters into the state encoder and extract the state feature vector.

[0074] Specifically, the robot's body perception information (including joint angles, end pose, velocity, and acceleration) and welding equipment parameters (such as the actual output power of the laser, light emission signal, and alarm code) are input to the state encoder (such as a multilayer perceptron). A continuous state feature vector that can characterize the mechanical motion state and energy input state of the welding process is extracted through nonlinear mapping.

[0075] Step S4013: Input the process context information into the text encoder and extract the semantic feature vector.

[0076] Specifically, process context information (including welding task description, preset process parameters, historical operation records, and other natural language text) is input into a text encoder (such as a pre-trained language model), and semantic feature vectors that can represent process requirements and task semantics are extracted through word embedding and context encoding mechanisms.

[0077] Step S4014: Project the visual feature vector, state feature vector and semantic feature vector to a unified semantic embedding space, perform cross-modal feature alignment and fusion, and obtain aligned multimodal features.

[0078] Specifically, the extracted visual feature vectors, state feature vectors, and semantic feature vectors are mapped to a unified semantic embedding space through a projection layer. Contrastive learning or attention mechanisms are used to calculate the similarity and association weights between cross-modal features, achieving spatiotemporal and semantic alignment of information from different modalities. Furthermore, a comprehensive multimodal feature representation is generated through feature fusion, providing complete and consistent feature inputs for subsequent physical state reasoning.

[0079] Step S402: Input the multimodal features into the welding process control model constructed using the welding process control model construction method to perform physical state reasoning, and generate physical reasoning thought chain text and implicit policy vector.

[0080] Specifically, step S402 includes: Step S4021: Based on multimodal features, the physical reasoning thought chain text is generated using the next term prediction method, and based on the physical reasoning thought chain text, the corresponding strategy intent text is generated; the physical reasoning thought chain text includes the observation description of the current welding phenomenon, the judgment of the physical state, and the causal attribution analysis of potential defect risks.

[0081] Specifically, the aligned multimodal features are input into the trained welding process control model. The model uses the next-word prediction method to predict word by word and generate physical reasoning thought chain text. This thought chain text first describes the observation of the current welding phenomenon (such as changes in the brightness of the molten pool and fluctuations in the keyhole shape), then judges the physical state of the welding process (such as the energy accumulation stage and the overheating critical state), and finally performs causal attribution analysis of potential defect risks (such as judging that the spatter risk originates from focus drift or power abnormality), thereby achieving a deep understanding and interpretable output of the welding physical process.

[0082] Based on the completed physical reasoning thought chain text generation, the model continues to generate corresponding strategy intent text using an autoregressive approach based on the generated reasoning content. This text expresses the direction of high-level process adjustments that should be taken for the current welding state in natural language (such as "keyhole needs to be stabilized" and "heat input should be reduced"), providing operators with understandable decision-making basis and providing clear control objectives for subsequent action generation.

[0083] Step S4022: Generate implicit policy vectors synchronously with the physical reasoning thought chain text; the implicit policy vectors are implicit representations of the physical control intentions expressed in the policy intention text, and are used to directly guide the generation of downstream actions.

[0084] Specifically, during the generation of physical reasoning thought chain text and strategic intent text, the model outputs implicit policy vectors in parallel. These vectors are high-dimensional numerical codes for the physical control intentions expressed in the strategic intent text. By converting the strategic intent in natural language form into an implicit representation that the cerebellum execution model can understand, they are used to directly guide the downstream action generation module to perform high-frequency trajectory planning and control command generation.

[0085] Step S403: Based on the implicit policy vector and real-time perception information, a continuous motion trajectory is generated and sent to the welding actuator to execute the continuous motion trajectory.

[0086] Specifically, step S403 includes: Step S4031: The implicit policy vector is fused with real-time perception information and then input into the conditional flow matching network. Continuous motion trajectories are generated by solving differential equations.

[0087] To address the collapse problem of traditional policy networks in multimodal action distribution (i.e., their inability to handle multiple feasible paths), this embodiment uses a flow matching algorithm: Vector field learning: Instead of directly predicting the next action point, the model learns a velocity vector field. This vector field defines how to smoothly transform random noise into the target action space (such as joint angle increments, laser power adjustments).

[0088] Action generation based on conditional flow matching: To address the collapse problem of traditional policy networks in multimodal action distribution, this embodiment models action generation as starting from prior noise distribution. To target action distribution The continuous-time transformation employs the Conditional Flow Matching (CFM) framework, as detailed below: (1) Definition of state space and conditions: set up For the target motion vector (e.g., joint angle increment or end effector pose). This represents the time step of the flow. The generation process is controlled by the mixing condition vector. This vector is formed by fusing two parts: (1); in, This represents the policy vector originating from the brain, i.e., the implicit policy vector. This represents a state vector derived from real-time observations. It is a vector of millisecond-level real-time sensor data after simple encoding (e.g., current joint current, real-time snapshot of weld pool visual features). This is the fusion function.

[0089] (2) Optimal Transport Path: Unlike the complex denoising process of diffusion models, this embodiment uses conditional flow matching to construct a flow path from Gaussian noise. Direct target action The straight path (i.e., the optimal transmission path). At any given time... intermediate state Defined as: (2); The time derivative (i.e., the ideal velocity field) corresponding to this path is: (3); in, For the target vector field, It represents the instantaneous rate of change.

[0090] (3) Vector field learning and loss function: This embodiment trains a neural network. A policy network is used to fit the ideal velocity field described above. The network input is the current noisy state. Time t and mixed conditions The training objective is to minimize the mean squared error (MSE) between the predicted velocity and the ideal velocity, which is the flow matching loss. (4); in, , , Sampled from expert demonstration data, Let the flow loss function be... To calculate the expected value, or average value, of a random variable, The velocity vector predicted by the cerebellar network.

[0091] Step S4032: The continuous motion trajectory is parsed into motion commands that the welding actuator can recognize, and the motion commands are sent to the welding robot and welding equipment in real time to drive the welding actuator to complete a smooth and continuous welding operation.

[0092] Specifically, during the inference phase, the model extracts random noise. Starting from this point, the final action is generated by solving the Ordinary Differential Equation (ODE). Numerical integration is performed using the Euler method: (5); when From 0 points to 1 point This refers to the generated smooth motion instructions.

[0093] It should be noted that the generation process of the condition generation mechanism is strictly controlled by two sets of inputs: Brain policy vectors (from the slow-flow KV-Cache): contain safety policy intentions (such as "stabilizing the melt pool" and "suppressing splash") that have been confirmed by the brain through "introspection".

[0094] Real-time observation status (from fast flow): millisecond-level sensor data (joint current, weld pool visual features).

[0095] ODE Solving and Action Output: During the inference phase, the module solves ordinary differential equations and generates the optimal control command for the current moment in one or more steps, following the learned vector field trajectory (i.e., continuous action trajectory). ).

[0096] Step S404: Real-time comparison of real-time perceived information during the welding process with the expected state generated by the welding process control model based on historical reasoning. When there is a deviation, the implicit policy vector is dynamically updated to drive the smooth correction of subsequent action trajectories.

[0097] Specifically, this step involves a fast-slow dual-flow architecture based on cerebellar-major brain collaboration and a self-reflection and correction mechanism. To resolve the contradiction between the inference latency of the large model and the millisecond-level response of industrial welding, this system innovatively adopts a hierarchical collaborative architecture of "large model decision-making (slow flow) - small model execution (fast flow)". This architecture does not pursue the large model directly outputting high-frequency actions, but rather utilizes the strong cognitive ability of the large model for "process self-reflection" and guides the cerebellum based on flow matching to generate actions by dynamically adjusting the policy vector.

[0098] Specifically, the following steps are included: Step S4041, asynchronous collaboration, decoupling cognition and execution: The system splits the control flow into two closed loops with different frequencies, which communicate with each other through implicit policy vectors (KV-Cache): Slow Flow (Cognitive Layer - VLA Large Model): Operates at a low frequency (e.g., 1-5Hz). It not only receives current visual observations but also incorporates historical thought chain memories. Its core task is not to control the motor but to perform "introspective monitoring," that is, to determine whether the current welding status meets expectations and whether there are potential defect risks.

[0099] Fast Flow (Execution Layer - Flow Matching Cerebellum): It operates at an extremely high frequency (e.g., 100Hz+). It uses the policy vector output by the slow flow as a condition, combined with real-time sensor data, to generate smooth motion trajectories through a flow matching network.

[0100] Step S4042, Self-correction mechanism based on the difference between "expectation" and "observation": In this architecture, true "self-reflection" does not remain merely at the textual level, but is manifested as the dynamic intervention of the large model in the cerebellum's executive strategies. This process constitutes the system's metacognitive loop: 1) State assessment and thought chain generation (slow flow): The VLA large model acts as the cognitive center, receiving current visual observations in real time. And the cerebellum's ongoing action feedback, utilizing the Transformer's language decoding capabilities, asynchronously generates explanatory thought chain text in the background. For example: Keyframe detection: The brightness, width, and area of ​​the molten pool reach a peak at this moment. It looks extremely "full," almost overflowing from the grooves. The entire molten pool is bright white, and the internal churning seems to have reached its most intense level. This is a critical state. My model library tells me that in hot working, this state is the best signal for "movement."

[0101] risk assessment: 1. Excessive heat leading to burn-through: If energy continues to be input, the temperature of the liquid metal will become too high and its fluidity too strong. It may "collapse" due to gravity or surface tension imbalance, or even burn through the thin-walled workpiece directly.

[0102] 2. Poor weld formation: An excessively large molten pool will form an irregular, excessively wide or high weld after cooling, affecting quality and aesthetics.

[0103] Conclusion: It must be moved. This "full" state means the "heating" task is complete; any more would be excessive.

[0104] Delayed inference mechanism: The large model generates text such as "High-frequency oscillation of the keyhole was detected (phenomenon), and based on historical heat input, it is determined to be critical overheating (state), attributed to excessive negative defocusing at the focus (cause)." Although the generation of this text is delayed compared to the current welding moment, it contains a deep understanding of the physical essence and provides logical support for subsequent strategy adjustments.

[0105] 2) Policy Vector Reparameterization: This is a crucial step for the introspection mechanism to take effect. Once the thought chain reasoning is complete, the large model does not directly output specific motor commands, but instead re-encodes the reasoning results (such as "oscillations need to be suppressed") into a high-dimensional implicit policy vector. Alternatively, one can directly use the KV-Cache generated by the brain.

[0106] Policy correction: If the thought chain determines that the current state is normal, the policy vector remains stable; once an abnormal risk is detected, the large model will be updated immediately. .

[0107] 3) Smooth response of motion generation (fast flow): Flow matching the cerebellum's output in a large model As a condition, high-frequency inference is performed by combining real-time sensor data.

[0108] Flow field deformation: When the large model updates the policy vector, the generated flow field of the flow matching undergoes smooth deformation. This prevents the robot arm's motion trajectory from abruptly transitioning to a new control mode (such as smoothly reducing power and slowing down the feed rate), thus completing the correction of anomalies at the physical level.

[0109] Step S4043, a collaborative training strategy based on semantic-action alignment: To ensure that the large model's "thoughts (introspection results)" can be "precisely executed" by the cerebellum, that is, to establish a mapping relationship between cognitive intentions and physical actions, this embodiment designs a hierarchical training strategy that includes semantic alignment constraints.

[0110] To understand the mapping relationships between actions, this invention designs a hierarchical training strategy that includes semantic alignment constraints.

[0111] 1) Timing-based instruction data construction: This embodiment constructs training data for triples with hierarchical dependencies. ,in: : Current multimodal input (melt pool image + body state).

[0112] : Expert thought chain text. This is the training objective for the large model (slow flow), used to explain physical states.

[0113] High-frequency motion trajectory segments. This is the training objective for the small model (fast flow).

[0114] 2) Conditionally guided decoupled training objectives: Model training is divided into two stages, and joint optimization is performed using consistency loss: Phase 1: Cognitive Introspection Training of the Large Model: The VLA model is trained to... Generate a chain of thoughts And simultaneously output implicit vectors .

[0115] Phase Two: Policy-Conditioned Cerebellar Action Cloning: Stream Matching Network and vectors from the large model As a condition, fit the expert action. .

[0116] 3) Semantic-action consistency constraint: To prevent a split between "brain command" and "cerebellum execution," we introduced a semantic alignment regularization term into the loss function. The system will calculate the "thought chain text." "Semantic embedding" and "policy vector" Mutual information between them. If the text description generated by the large model is "severely unfused (power needs to be increased)," but the output policy vector... However, if the cerebellum is guided to generate a "reduced power" action, it will result in a huge penalty loss. This mechanism forces the model to unify physical cognition and motor planning in the latent space, ensuring that the introspective results of the large model can be losslessly transformed into the execution signals of the cerebellum, achieving a high degree of logical self-consistency.

[0117] In the above technical solution, by injecting the physical laws of welding into the model, the system can generate thought chain text containing causal analysis (such as attributing to focus drift or power anomalies). The system uses the difference between the "expected state" and "real-time observation" for online self-reflection, and achieves logical judgment and closed-loop correction of welding defects by dynamically adjusting the implicit strategy vector.

[0118] The welding process control method provided in this embodiment has the following beneficial effects: 1. It breaks through the traditional "black box" control and possesses interpretable physical semantic understanding capabilities: Related technologies typically rely on numerical regression based on surface features such as weld pool brightness and width, lacking an understanding of the physical processes involved in welding. This invention, by introducing a physical reasoning chain, enables the system to understand the "energy state" and "force balance" of the weld pool like a human expert, and to explain the reasons for decisions using natural language, thus achieving a leap from "perception" to "cognition."

[0119] 2. Possesses human-like causal reasoning and self-reflection / correction abilities, with stronger generalization capabilities: Traditional teaching or imitation learning schemes are essentially trajectory fitting, which is prone to failure when faced with unseen working conditions. This invention introduces an ontology introspection mechanism, where the system not only mechanically executes instructions but also compares the "expected state" with the "observed result" in real time. When anomalies occur (such as focus drift or material changes), the system can perform causal attribution based on physical common sense and dynamically adjust the strategy vector for compensation, exhibiting strong robustness and adaptability in complex and ever-changing welding environments.

[0120] This embodiment also provides a welding process control model construction device and a welding process control device, which are used to implement the above embodiments and preferred embodiments, and will not be repeated for details already described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0121] This embodiment provides a welding process control model construction device, such as... Figure 5 As shown, it includes: The triplet training dataset construction module 501 is used to construct a triplet training dataset containing multimodal inputs, physical reasoning thought chain texts, and policy intentions.

[0122] The supervised fine-tuning module 502 is used to perform supervised fine-tuning on the pre-trained large vision-language-action model based on the triplet training dataset, so as to obtain the fine-tuned large model.

[0123] The preference alignment training module 503 is used to construct positive and negative sample inference path pairs. The direct preference optimization algorithm is used to perform preference alignment training on the fine-tuned large model to obtain the welding process control model.

[0124] In some alternative implementations, the supervisory fine-tuning module 502 includes: The model initialization unit is used to use the pre-trained vision-language-action large model as the base model and load the pre-trained weights of the base model.

[0125] The supervised fine-tuning unit is used to input the physical reasoning thought chain text into the base model based on the pre-trained weights, perform full parameter fine-tuning using the next word prediction method, and force the model to establish conditional probability dependencies from visual perception to logical reasoning to policy output through the label masking strategy, so as to obtain the fine-tuned large model.

[0126] In some alternative implementations, the preference alignment training module 503 includes: The positive and negative sample reasoning path pair construction unit is used to construct positive sample reasoning paths that conform to physical facts and negative sample pairs that violate physical common sense, thus obtaining positive and negative sample reasoning path pairs.

[0127] The preference alignment training unit is used to train the model with preference alignment based on positive and negative sample inference path pairs using the direct preference optimization algorithm, thereby obtaining the welding process control model.

[0128] This embodiment also provides a welding process control device, such as... Figure 6 As shown, it includes: The multimodal perception and semantic alignment module 601 is used to acquire multimodal perception information during the welding process and map the multimodal perception information to the semantic embedding space to obtain aligned multimodal features.

[0129] The cognitive reasoning module 602 is used to input multimodal features into the welding process control model constructed using the welding process control model construction method to perform physical state reasoning, and generate physical reasoning thought chain text and implicit policy vector.

[0130] The motion generation module 603 is used to generate continuous motion trajectories based on implicit policy vectors and real-time perception information, and send them to the welding actuator to execute the continuous motion trajectory.

[0131] In some optional implementations, the multimodal perception information includes: a visual image of the molten pool, robot body perception information, and process context information; the multimodal perception and semantic alignment module 601 includes: The visual feature vector extraction unit is used to input the molten pool visual image into the visual encoder and extract the visual feature vector.

[0132] The state feature vector extraction unit is used to input robot body perception information and welding equipment parameters into the state encoder and extract state feature vectors.

[0133] The semantic feature vector extraction unit is used to input process context information into the text encoder and extract semantic feature vectors.

[0134] The alignment and fusion unit is used to project visual feature vectors, state feature vectors and semantic feature vectors onto a unified semantic embedding space to perform cross-modal feature alignment and fusion, resulting in aligned multimodal features.

[0135] In some alternative implementations, the cognitive reasoning module 602 includes: The physical reasoning thought chain text generation unit is used to generate physical reasoning thought chain text based on multimodal features and using the next term prediction method, and to generate corresponding strategy intent text based on the physical reasoning thought chain text. The physical reasoning thought chain text includes observational descriptions of the current welding phenomenon, judgments of the physical state, and causal attribution analysis of potential defect risks.

[0136] The implicit policy vector generation unit is used to generate implicit policy vectors synchronously with the physical reasoning thought chain text. The implicit policy vector is an implicit representation of the physical control intention expressed by the policy intention text, and is used to directly guide the generation of downstream actions.

[0137] In some alternative implementations, the action generation module 603 includes: The continuous motion trajectory generation unit is used to fuse implicit policy vectors with real-time perception information and input them into the conditional flow matching network, and generate continuous motion trajectories by solving differential equations.

[0138] The motion command generation and distribution unit is used to parse the continuous motion trajectory into motion commands that the welding actuator can recognize, and to distribute the motion commands to the welding robot and welding equipment in real time to drive the welding actuator to complete a smooth and continuous welding operation.

[0139] In some optional embodiments, the welding process control device further includes: The self-reflection and correction module is used to compare the real-time perceived information during the welding process with the expected state generated by the welding process control model based on historical reasoning. When there is a deviation, it dynamically updates the implicit policy vector to drive the smooth correction of subsequent action trajectories.

[0140] The welding process control model construction device provided in this embodiment of the invention can execute the welding process control model construction method provided in any embodiment of the invention, and the welding process control device provided in this embodiment of the invention can execute the welding process control method provided in any embodiment of the invention, possessing the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the above modules and units are the same as in the corresponding embodiments described above, and will not be repeated here.

[0141] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0142] The following is a detailed reference. Figure 7 This diagram illustrates a suitable structural schematic for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 701, which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) 702 or a program loaded from memory 708 into random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device. The processor 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0143] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic devices to exchange data via wireless or wired communication with other devices. Although Figure 7 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0144] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 709, or installed from a memory 708, or installed from a ROM 702. When the computer program is executed by the processor 701, it performs the functions defined in the welding process control model construction method or welding process control method of the embodiments of the present invention.

[0145] Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0146] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the welding process control model construction method or welding process control method shown in the above embodiments is implemented.

[0147] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0148] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.< / instruction> < / instruction>

Claims

1. A method for constructing a welding process control model, characterized in that, The method includes: Construct a triplet training dataset containing multimodal inputs, physical reasoning thought chain texts, and policy intentions; Based on the triplet training dataset, the pre-trained vision-language-action large model is subjected to supervised fine-tuning to obtain the fine-tuned large model. Positive and negative sample inference path pairs are constructed, and the direct preference optimization algorithm is used to train the fine-tuned large model for preference alignment to obtain the welding process control model.

2. The method according to claim 1, characterized in that, Based on the aforementioned triplet training dataset, the pre-trained large-scale vision-language-action model is subjected to supervised fine-tuning to obtain the fine-tuned large-scale model, including: The pre-trained visual-language-action large model is used as the base model, and the pre-trained weights of the base model are loaded. Based on the pre-trained weights, the physical reasoning thought chain text is input into the base model, and full parameter fine-tuning is performed using the next word prediction method. The model is forced to establish conditional probability dependencies from visual perception to logical reasoning to policy output through a label masking strategy, resulting in a fine-tuned large model.

3. The method according to claim 1, characterized in that, The process of constructing positive and negative sample inference path pairs and using the direct preference optimization algorithm to perform preference alignment training on the fine-tuned large model to obtain the welding process control model includes: Construct positive sample reasoning paths that conform to physical common sense and welding strategies, and negative sample pairs that violate physical common sense, to obtain positive and negative sample reasoning path pairs; Based on the positive and negative sample inference path pairs, the direct preference optimization algorithm is used to train the model for preference alignment, thereby obtaining the welding process control model.

4. A welding process control method, characterized in that, The method includes: Multimodal perception information during the welding process is acquired, and the multimodal perception information is mapped to the semantic embedding space to obtain aligned multimodal features; The multimodal features are input into the welding process control model constructed using the welding process control model construction method as described in any one of claims 1 to 3 to perform physical state reasoning, generating physical reasoning thought chain text and implicit policy vector; Based on implicit policy vectors and real-time perception information, a continuous motion trajectory is generated and sent to the welding actuator to execute the continuous motion trajectory.

5. The method according to claim 4, characterized in that, The multimodal perception information includes: visual images of the molten pool, robot body perception information, and process context information; The multimodal sensing information is mapped to the semantic embedding space to obtain aligned multimodal features, including: The visual image of the molten pool is input into a visual encoder to extract visual feature vectors; The robot's perception information and welding equipment parameters are input into the state encoder to extract the state feature vector. The process context information is input into a text encoder to extract semantic feature vectors; The visual feature vector, state feature vector, and semantic feature vector are projected into a unified semantic embedding space, and cross-modal feature alignment and fusion are performed to obtain aligned multimodal features.

6. The method according to claim 4 or 5, characterized in that, The multimodal features are input into the welding process control model constructed using the welding process control model construction method as described in any one of claims 1 to 3 for physical state reasoning, generating physical reasoning thought chain text and implicit policy vectors, including: Based on the multimodal features, a physical reasoning thought chain text is generated using the next term prediction method, and a corresponding strategy intent text is generated based on the physical reasoning thought chain text; the physical reasoning thought chain text includes an observational description of the current welding phenomenon, a judgment of the physical state, and a causal attribution analysis of potential defect risks. An implicit policy vector is generated synchronously with the physical reasoning thought chain text; the implicit policy vector is an implicit representation of the physical control intention expressed by the policy intention text, and is used to directly guide the generation of downstream actions.

7. The method according to claim 4, characterized in that, Based on implicit policy vectors and real-time sensing information, a continuous motion trajectory is generated and sent to the welding actuator to execute a smooth motion trajectory, including: The implicit policy vector is fused with real-time perception information and then input into the conditional flow matching network. Continuous motion trajectories are generated by solving differential equations. The continuous motion trajectory is parsed into motion commands that the welding actuator can recognize, and the motion commands are sent to the welding robot and welding equipment in real time to drive the welding actuator to complete a smooth and continuous welding operation.

8. The method according to claim 4, characterized in that, The method further includes: The system compares real-time perceived information during the welding process with the expected state generated by the welding process control model based on historical reasoning. When a deviation exists, the implicit strategy vector is dynamically updated to drive the smooth correction of subsequent action trajectories.

9. A welding process control model construction device, characterized in that, The device includes: The triplet training dataset construction module is used to build a triplet training dataset that includes multimodal inputs, physical reasoning thought chain texts, and policy intentions. The supervised fine-tuning module is used to perform supervised fine-tuning on the pre-trained visual-language-action large model based on the triplet training dataset, so as to obtain the fine-tuned large model. The preference alignment training module is used to construct positive and negative sample inference path pairs. The direct preference optimization algorithm is used to perform preference alignment training on the fine-tuned large model to obtain the welding process control model.

10. A welding process control device, characterized in that, The device includes: The multimodal perception and semantic alignment module is used to acquire multimodal perception information during the welding process and map the multimodal perception information to the semantic embedding space to obtain aligned multimodal features; The cognitive reasoning module is used to input the multimodal features into the welding process control model constructed using the welding process control model construction method as described in any one of claims 1 to 3 to perform physical state reasoning, and generate physical reasoning thought chain text and implicit policy vector; The motion generation module is used to generate continuous motion trajectories based on implicit policy vectors and real-time perception information, and then send them to the welding actuator to execute the continuous motion trajectories.

Images