Large-model-integrated weld seam recognition method and apparatus, and electronic device and storage medium
By integrating large-scale models and line laser sensors to identify weld seam information, the problem of welding robot trajectory deviation was solved, achieving efficient and precise welding path optimization, and improving welding quality and production efficiency.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- BEIJING XIAOYU INTELLISYS CO LTD
- Filing Date
- 2025-12-29
- Publication Date
- 2026-07-09
AI Technical Summary
During the welding process, welding robots may experience a decline in welding quality and reduced production efficiency due to thermal deformation, vibration, or movement of the workpiece, causing the welding trajectory to deviate from the actual weld position.
A method for identifying weld seams using a fusion large model is adopted. By acquiring images of the target workpiece and using a pre-trained large model to identify weld seam information, and combining this with a line laser sensor to identify weld seams and optimize welding positions, adaptive correction is achieved.
It improved welding quality and automation, reduced manual intervention, and increased production efficiency and welding precision.
Smart Images
Figure CN2025146868_09072026_PF_FP_ABST
Abstract
Description
Methods, devices, electronic equipment, and storage media for weld seam identification based on large-scale models Technical Field
[0001] This application relates to the fields of large model, image processing, and welding technology, and in particular to a method, apparatus, electronic device, and storage medium for weld seam recognition based on a large model. Background Technology
[0002] Welding robots are now widely used in industry. However, during the welding process, various adverse factors, such as thermal deformation, vibration, or sudden movement of the workpiece, can cause the welding robot's welding trajectory to deviate from the actual weld position. This can lead to the welding robot being unable to complete its work, a decrease in welding quality, and an impact on overall welding production efficiency. Summary of the Invention
[0003] The purpose of this application is to at least partially solve one of the technical problems in the related art.
[0004] Therefore, the first objective of this application is to propose a weld identification method that integrates a large model to achieve adaptive correction of the welding position, thereby providing a stable welding path, reducing welding defects, and improving welding quality.
[0005] The second objective of this application is to propose a weld identification device that integrates a large model.
[0006] The third objective of this application is to propose an electronic device.
[0007] The fourth objective of this application is to provide a computer-readable storage medium.
[0008] The fifth objective of this application is to provide a computer program product.
[0009] To achieve the above objectives, a first aspect of this application proposes a weld seam recognition method based on a large model, comprising: acquiring a first image of a target workpiece to be welded, inputting the first image into a pre-trained large model, having the large model perform weld seam recognition on the first image, and outputting first weld seam information of the target workpiece; determining the starting position of the weld seam in the target workpiece based on the first weld seam information, and controlling the welding torch to move to the starting position; performing weld seam recognition at the starting position based on the line laser sensor of the welding torch, obtaining second weld seam information of the target workpiece; and optimizing the welding position of the welding torch based on the first weld seam information and the second weld seam information to obtain a target welding position.
[0010] To achieve the above objectives, a second aspect of this application proposes a weld seam recognition device integrating a large model, comprising: a model recognition module for acquiring a first image of a target workpiece to be welded, inputting the first image into a pre-trained large model, wherein the large model performs weld seam recognition on the first image and outputs first weld seam information of the target workpiece; a determination module for determining the starting position of the weld seam in the target workpiece based on the first weld seam information, and controlling the welding torch to move to the starting position; a sensor recognition module for performing weld seam recognition at the starting position based on the line laser sensor of the welding torch, obtaining second weld seam information of the target workpiece; and an optimization module for optimizing the welding position of the welding torch based on the first weld seam information and the second weld seam information, thereby obtaining a target welding position.
[0011] To achieve the above objectives, a third aspect of this application provides an electronic device, comprising: a processor; and a memory communicatively connected to the processor; the memory storing computer-executable instructions; and the processor executing the computer-executable instructions stored in the memory to enable the processor to execute the weld seam identification method for fused large models described in the first aspect of the application.
[0012] To achieve the above objectives, a fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, the computer instructions being used to cause the computer to execute the weld identification method for fused large models described in the above aspect of the embodiment.
[0013] To achieve the above objectives, a fifth aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the weld seam identification method based on a fused large model as described in the above-mentioned aspect embodiment.
[0014] The weld seam recognition method, apparatus, electronic device, and storage medium fused with a large model provided in this application obtain first weld seam information of the target workpiece by acquiring a first image of the target workpiece and using a large model to recognize the weld seam in the first image. The starting position of the weld seam can then be determined from the first weld seam information, and a line laser sensor is used to identify the weld seam at the starting position to obtain second weld seam information. Furthermore, the welding position can be optimized based on the first and second weld seam information to obtain a target welding position, thereby controlling the welding torch to weld the target workpiece at the target welding position. Using a large model to recognize weld seam information ensures the accuracy of weld seam recognition, and by combining the recognition results of the large model and the line laser sensor, the welding position can be optimized, achieving adaptive correction of the welding position, thus providing a stable welding path, reducing welding defects, and improving welding quality. Using the method of the embodiments of this application, the welding torch can achieve efficient and accurate weld seam tracking in complex environments, significantly improving the level of welding automation, reducing manual intervention, and improving production efficiency and welding quality.
[0015] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description
[0016] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:
[0017] Figure 1 is a flowchart illustrating a weld identification method based on a large model provided in an embodiment of this application;
[0018] Figure 2 is a flowchart illustrating another weld identification method based on a fused large model provided in an embodiment of this application;
[0019] Figure 3 is a flowchart illustrating another weld seam identification method based on a fused large model provided in an embodiment of this application;
[0020] Figure 4 is a schematic diagram of the welding system provided in an embodiment of this application;
[0021] Figure 5 is a schematic flowchart of the process for determining the target welding position provided in an embodiment of this application;
[0022] Figure 6 is a schematic diagram of the structure of a weld seam recognition device that integrates a large model according to an embodiment of this application. Detailed Implementation
[0023] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.
[0024] The following describes a method and apparatus for weld seam identification in a fused large model according to embodiments of this application, with reference to the accompanying drawings.
[0025] Figure 1 is a flowchart of a weld seam recognition method based on a fused large model according to an embodiment of this application. As shown in Figure 1, the weld seam recognition method based on a fused large model according to an embodiment of this application includes, but is not limited to, the following steps:
[0026] S101: Acquire the first image of the target workpiece to be welded, and input the first image into the pre-trained large model. The large model performs weld seam recognition on the first image and outputs the first weld seam information of the target workpiece.
[0027] It should be noted that the execution subject of the weld seam recognition method based on the fusion of large models provided in this application embodiment is an electronic device, which can be a terminal device. Optionally, the terminal device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be personal computers (PCs), televisions, etc. This application embodiment does not impose specific limitations.
[0028] In some implementations, an image acquisition device can be used to acquire images of the target workpiece to be welded, obtaining a first image of the target workpiece. This first image is then input into a pre-trained large model, which identifies the weld seam in the first image, obtaining the first weld seam information of the target workpiece, which is then output. The large model can be a visual large model. Optionally, the first weld seam information includes the weld seam's location information, weld seam structural information, etc.
[0029] S102, based on the first weld information, determine the starting position of the weld in the target workpiece, and control the welding torch to move to the starting position.
[0030] In some implementations, the weld position information can be obtained from the first weld information, and the starting position of the weld can be determined from the weld position information by determining the welding direction during welding. For example, if the weld of the target workpiece is identified as a vertical weld, by determining that the weld is welded from top to bottom, the position of the uppermost part of the weld can be determined as the starting position.
[0031] Furthermore, once the starting position is determined, the welding torch can be controlled to weld the seam from that position. In other words, by moving the welding torch to the starting position, welding can be initiated from that position.
[0032] In some implementations, since the starting position of the weld is in the image coordinate system, while the welding torch uses coordinates in the robot coordinate system for welding, the starting position is transformed from the image coordinate system to the robot coordinate system to obtain the transformed starting position, so as to control the welding torch to move to the transformed starting position.
[0033] S103, based on the welding torch's line laser sensor, identifies the weld seam at the starting position and obtains the second weld seam information of the target workpiece.
[0034] In some implementations, before welding with the welding torch, a line laser can be used based on the welding torch's own line laser sensor to identify the weld seam and determine the second weld seam information of the target workpiece. This second weld seam information includes the weld seam's location information.
[0035] S104. Based on the first weld seam information and the second weld seam information, the welding position of the welding torch is optimized to obtain the target welding position.
[0036] In some implementations, the weld position identified by the line laser sensor is easily affected by the environment. For example, if the weld bead is uneven or there are multiple inflection points in the weld bead cross section, positioning errors may occur. Therefore, the first and second weld information identified by the large model can be used to optimize the welding position to obtain the optimized target welding position, and then control the welding torch to weld based on the target welding position.
[0037] In some implementations, an optimization function can be determined based on the first welding position in the first weld information and the second welding position in the second weld information, and the optimization function can be used to optimize the welding position to obtain the target welding position.
[0038] Optionally, in order to ensure the smoothness of the welding trajectory during the welding process, an optimization function can be constructed based on the speed and acceleration of the welding torch during welding, combined with the first welding position and the second welding position. The optimization function can then be used to optimize the welding position to obtain the target welding position.
[0039] The weld seam recognition method based on a large model provided in this application involves acquiring a first image of the target workpiece and using a large model to recognize the weld seam in the first image, thereby obtaining the first weld seam information of the target workpiece. The starting position of the weld seam can then be determined from the first weld seam information, and a line laser sensor is used to identify the weld seam at the starting position to obtain the second weld seam information. Furthermore, the welding position can be optimized based on the first and second weld seam information to obtain the target welding position, thereby controlling the welding torch to weld the target workpiece at the target welding position. Using a large model to recognize weld seam information ensures the accuracy of weld seam recognition. By combining the recognition results of the large model and the line laser sensor, the welding position can be optimized, achieving adaptive correction of the welding position, thus providing a stable welding path, reducing welding defects, and improving welding quality. Using the method of this application embodiment, the welding torch can achieve efficient and accurate weld seam tracking in complex environments, significantly improving the automation level of welding, reducing manual intervention, and increasing production efficiency and welding quality.
[0040] Figure 2 is a flowchart of a weld seam recognition method based on a fused large model according to an embodiment of this application. As shown in Figure 2, the weld seam recognition method based on a fused large model according to an embodiment of this application includes, but is not limited to, the following steps:
[0041] S201: Acquire the first image of the target workpiece to be welded, and input the first image into the pre-trained large model. The large model performs weld seam recognition on the first image and outputs the first weld seam information of the target workpiece.
[0042] In the embodiments of this application, step S201 can be implemented in any of the ways described in the embodiments of this application. This is not limited here and will not be described in detail.
[0043] S202, based on the first weld information, determine the starting position of the weld in the target workpiece, and control the welding torch to move to the starting position.
[0044] In some implementations, to ensure the integrity of weld information identification, the integrity of the weld in the first weld information can be identified before determining the starting position of the weld, so as to improve welding accuracy and prevent welding defects.
[0045] In some implementations, the integrity of a weld can be identified based on its structural information. Optionally, the three-dimensional (3D) structural information of the weld in the target workpiece can be determined based on the first weld information. Furthermore, based on the 3D structural information, it can be determined whether the first image contains a complete weld.
[0046] Optionally, the 3D structural information includes the geometric features of the weld, such as height, width, length, and shape. Feature information of the weld can be extracted from the 3D structural information, and the extracted feature information can be compared with the feature information of a standard weld to obtain a comparison result. Based on the comparison result, it can be determined whether the first image contains a complete weld.
[0047] Optionally, if the comparison result indicates that the extracted feature information is the same as the feature information of the standard weld, then the first image is determined to contain a complete weld; otherwise, the first image is determined not to contain a complete weld.
[0048] In some implementations, in response to the absence of a complete weld seam, the target workpiece is re-image acquired and its integrity identified until a second image containing the complete weld seam is obtained. Furthermore, weld seam identification can be performed on the second image based on a large model to obtain complete first weld seam information, from which the starting position of the weld seam can then be determined.
[0049] In the embodiments of this application, the method of controlling the welding torch to move to the starting position in step S202 can be implemented by any of the methods in the embodiments of this application. This is not limited here and will not be described in detail.
[0050] S203, based on the welding torch's line laser sensor, identifies the weld seam at the starting position and obtains the second weld seam information of the target workpiece.
[0051] In the embodiments of this application, step S203 can be implemented in any of the ways described in the embodiments of this application. This is not limited here and will not be described in detail.
[0052] S204, determine the first welding position based on the first weld information, and determine the second welding position based on the second weld information.
[0053] In some implementations, multiple first weld seam positions can be determined from the first weld seam information, and these positions can be converted from the image coordinate system to the robot coordinate system to obtain the first welding position. Similarly, multiple second weld seam positions can be determined from the second weld seam information, and these positions can be converted from the point cloud coordinate system to the robot coordinate system to obtain the second welding position.
[0054] S205, acquire the motion information of the welding torch during the welding process, including the speed and acceleration of the welding torch.
[0055] In some implementations, historical welding data of the welding torch can be acquired, and the speed and acceleration of the welding torch during the welding process can be obtained from the historical welding data as motion information. Optionally, multiple historical welding data can be acquired, and multiple speeds and accelerations can be extracted from the historical welding data. Then, the average speed and average acceleration are averaged and used as the speed and acceleration of the welding torch during the welding process.
[0056] S206. Based on the first welding position, the second welding position, and motion information, an optimization function is constructed and solved to obtain the target welding position.
[0057] In some implementations, in order to ensure the smoothness of the welding trajectory during the welding process, an optimization function can be constructed based on the first welding position, the second welding position, and the motion information. The optimization function can then be solved to obtain the precise and smooth target welding position.
[0058] In some implementations, an optimization function can be constructed based on the squared distance between welding positions, velocity, and acceleration, and their respective weights. That is, by determining the current welding position of the welding torch, and obtaining the first squared distance between the first welding position and the second squared distance between the second welding position and the first squared distance between the second welding position.
[0059] Optionally, the square of the Euclidean distance between the first welding position and the welding position can be calculated as the first distance squared, and the square of the Euclidean distance between the second welding position and the welding position can be calculated as the second distance squared.
[0060] Furthermore, the weights of the first squared distance, the second squared distance, and the velocity and acceleration are determined, and an optimization function is constructed based on the first squared distance, the second squared distance, the velocity and acceleration, and their respective weights.
[0061] Alternatively, the formula for the optimization function is as follows:
[0062] Among them, f dis (P(i) lidar ,P(i) target ) represents the squared second distance, f dis (P(i) vision ,P(i) target Let P(i) represent the squared first distance. lidar Let P(i) represent the second welding position. vision Let P(i) represent the first welding position. target Indicates the target welding position. Indicates speed, ω represents acceleration.ldis ω represents the weight value of the squared second distance. vdis ω represents the weight value of the squared first distance. vel The weight value representing velocity, ω acc The weight value represents the acceleration.
[0063] Optionally, the squared distance can be calculated based on the coordinates of the location. The formula for calculating the squared distance is as follows:
[0064] f dis (P ref ,P target )=(x ref -x target ) 2 +(y ref -y target ) 2 +z ref -z target ) 2 (2)
[0065] Among them, (x ref ,y ref ,z ref ) indicates position p ref The coordinates, (x target ,y target ,z target ) indicates position p target The coordinate values.
[0066] Optionally, by substituting the first welding position and the welding location into formula (2), the square of the first distance can be calculated. By substituting the second welding position and the welding location into formula (2), the square of the second distance can be calculated.
[0067] Furthermore, after obtaining the optimization function, the optimization function can be solved. By optimizing the optimization function, the welding position when the optimization function is minimized can be obtained, which is then used as the target welding position.
[0068] In some implementations, after obtaining the target welding position, the welding torch can be controlled to move to the target welding position to weld the target workpiece. During the welding process, the third weld seam information of the target workpiece is acquired in real time based on a line laser sensor. Then, based on the first weld seam information, the third weld seam information, and an optimization function, the real-time welding position of the target workpiece can be obtained, and welding can be performed on the target workpiece based on the real-time welding position.
[0069] In other words, the first welding position in the first weld information and the third welding position in the third weld information can be substituted into formula (1) to obtain the welding position when the optimization function is minimized, which can be used as the real-time welding position.
[0070] The weld seam recognition method based on a large model provided in this application involves acquiring a first image of the target workpiece and using a large model to recognize the weld seam in the first image, thereby obtaining the first weld seam information of the target workpiece. The starting position of the weld seam can then be determined from the first weld seam information, and a line laser sensor is used to identify the weld seam at the starting position to obtain the second weld seam information. Further, an optimization function can be constructed based on the first and second weld seam information, and the optimization function can be solved to obtain the target welding position, thereby controlling the welding torch to weld the target workpiece at the target welding position. Using a large model to recognize weld seam information ensures the accuracy of weld seam recognition. By combining the recognition results of the large model and the recognition results of the line laser sensor, the welding position is optimized, achieving adaptive correction of the welding position, thus providing a stable welding path, reducing welding defects, and improving welding quality. Using the method of this application embodiment, the welding torch can achieve efficient and accurate weld seam tracking in complex environments, significantly improving the automation level of welding, reducing manual intervention, and improving production efficiency and welding quality.
[0071] Figure 3 is a flowchart of a weld seam recognition method based on a fused large model according to an embodiment of this application. As shown in Figure 3, the weld seam recognition method based on a fused large model according to an embodiment of this application includes, but is not limited to, the following steps:
[0072] S301: Acquire the first image of the target workpiece to be welded, and input the first image into the pre-trained large model. The large model performs weld seam recognition on the first image and outputs the first weld seam information of the target workpiece.
[0073] In the embodiments of this application, step S301 can be implemented in any of the ways described in the embodiments of this application. This is not limited here and will not be described in detail.
[0074] S302, Based on the first weld information, determine the starting position of the weld in the target workpiece.
[0075] In the embodiments of this application, step S302 can be implemented in any of the ways described in the embodiments of this application. This is not limited here and will not be described in detail.
[0076] S303, obtain the transformation matrix between the image coordinate system and the robot coordinate system.
[0077] S304, based on the transformation matrix and the starting position, determines the welding starting position of the welding torch and controls the welding torch to move to the welding starting position.
[0078] In some implementations, since the starting position of the weld is in the image coordinate system and the welding torch is in the robot coordinate system, the starting position in the image coordinate system can be converted to the robot coordinate system according to the coordinate system transformation matrix, and then the welding torch can be controlled to move to the starting position.
[0079] Optionally, the transformation matrix between the image coordinate system and the robot coordinate system can be calculated based on the rotation matrix and translation vector between the image coordinate system and the robot coordinate system. The transformation matrix can then be used to perform coordinate system transformation on the starting position to obtain the welding starting position in the robot coordinate system where the welding torch is located. This allows the welding torch to be controlled to move to the welding starting position.
[0080] S305, a line laser sensor based on a welding torch, identifies weld seams at the starting position and obtains the second weld seam information of the target workpiece.
[0081] In the embodiments of this application, step S305 can be implemented in any of the embodiments of this application, and no limitation is made here, nor will it be described in detail.
[0082] S306, Based on the first weld seam information and the second weld seam information, the welding position of the welding torch is optimized to obtain the target welding position.
[0083] In the embodiments of this application, step S306 can be implemented in any of the embodiments of this application, and no limitation is made here, nor will it be described in detail.
[0084] The weld seam recognition method based on a large model provided in this application involves acquiring a first image of the target workpiece and using a large model to recognize the weld seam in the first image, thereby obtaining the first weld seam information of the target workpiece. The starting position of the weld seam can then be determined from the first weld seam information, and the welding torch can be controlled to move to the corresponding welding starting position. A line laser sensor is then used to recognize the weld seam, obtaining the second weld seam information. Furthermore, the welding position can be optimized based on the first and second weld seam information to obtain the target welding position, thereby controlling the welding torch to weld the target workpiece at the target welding position. Using a large model to recognize weld seam information ensures the accuracy of weld seam recognition. By combining the recognition results of the large model and the line laser sensor, the welding position can be optimized, achieving adaptive correction of the welding position. This provides a stable welding path, reduces welding defects, and improves welding quality. Using the method of this application embodiment, the welding torch can achieve efficient and accurate weld seam tracking in complex environments, significantly improving the automation level of welding, reducing manual intervention, and increasing production efficiency and welding quality.
[0085] As an example, Figure 4 shows a schematic diagram of a welding system constructed using the method provided in the embodiments of this application. Figure 4 includes a vision perception module, a line laser perception module, a control center, and a motion controller.
[0086] The visual perception module includes functions such as visual image acquisition and image preprocessing. The line laser perception module includes functions such as line laser sensor control and signal processing. The control center is used for task scheduling, model inference, and algorithm execution. The motion controller is responsible for executing motion commands and completing predetermined actions.
[0087] The control center sends image acquisition commands to the vision perception module. Upon receiving the commands, the vision perception module acquires images of the target workpiece, preprocesses the images, and then sends the preprocessed images back to the control center. The control center then uses a pre-trained large model to identify weld seams in the images, obtaining the first weld seam information.
[0088] The control center sends a weld seam identification command to the line laser sensing module. After receiving the command, the line laser sensing module uses the line laser sensor to identify the weld seam on the target workpiece, obtains the second weld seam information, and sends the second weld seam information to the control center.
[0089] After receiving the first weld seam information and the second weld seam information, the control center can use an optimization algorithm to optimize the welding position of the welding torch based on the first weld seam information and the second weld seam information to obtain the target welding position, and send a welding command carrying the target welding position to the motion controller so that the motion controller can weld the target workpiece based on the welding command.
[0090] Figure 5 illustrates the flowchart for determining the target welding position. An RGB image of the target workpiece is obtained by image acquisition, where the RGB image corresponds to the first image in this embodiment. A pre-trained large-scale visual model is used to identify the weld seam in the RGB image, obtaining the semantic and 3D structural information of the weld seam as the first weld seam information. Based on the first weld seam information, the integrity of the weld seam is determined, and the starting position of the weld seam is determined if the weld seam is complete. If the weld seam is incomplete, the RGB image is re-acquired and the weld seam integrity is re-identified until a complete weld seam is obtained.
[0091] Furthermore, the starting position is transformed to a welding starting position in the robot coordinate system, and the welding torch is controlled to move to the welding starting position. Based on the welding torch's line laser sensor, the second weld seam information of the target workpiece is acquired. Using the first weld seam information, the second weld seam information, and the welding torch's velocity and acceleration, an optimization function is determined, and the welding position that minimizes the optimization function is calculated as the target welding position. The welding torch is then controlled to weld the target workpiece according to the target welding position.
[0092] In some implementations, the welding process determines in real time whether the weld has reached its end point and terminates the welding if it does. If the weld has not reached its end point, a line laser sensor continues to identify weld information, enabling motion tracking of the welding process.
[0093] Corresponding to the weld seam recognition methods for fused large models proposed in the above embodiments, an embodiment of this application also proposes a weld seam recognition device for fused large models. Since the weld seam recognition device for fused large models proposed in this application corresponds to the weld seam recognition methods for fused large models proposed in the above embodiments, the implementation methods for the weld seam recognition methods for fused large models proposed in the above embodiments are also applicable to the weld seam recognition device for fused large models proposed in this application, and will not be described in detail in the following embodiments.
[0094] To achieve the above embodiments, this application also proposes a weld seam recognition device that integrates a large model.
[0095] Figure 6 is a schematic diagram of a weld seam recognition device that integrates a large model according to an embodiment of this application.
[0096] As shown in Figure 6, the weld seam recognition device 600 of the fused large model includes:
[0097] The model recognition module 601 is used to acquire a first image of the target workpiece to be welded, and input the first image into a pre-trained large model. The large model performs weld seam recognition on the first image and outputs the first weld seam information of the target workpiece.
[0098] The determining module 602 is used to determine the starting position of the weld in the target workpiece based on the first weld information, and control the welding torch to move to the starting position;
[0099] The sensor recognition module 603 is used to identify weld seams at the starting position based on the welding torch's line laser sensor to obtain the second weld seam information of the target workpiece.
[0100] The optimization module 604 is used to optimize the welding position of the welding torch based on the first weld seam information and the second weld seam information to obtain the target welding position.
[0101] In one possible implementation of this application, the determining module 602 is further configured to: determine the three-dimensional 3D structural information of the weld in the target workpiece based on the first weld information; determine whether the first image contains a complete weld based on the 3D structural information; and, in response to not containing a complete weld, re-acquire the image and perform integrity recognition on the target workpiece until a second image containing a complete weld is obtained.
[0102] In one possible implementation of this application embodiment, the optimization module 604 is further configured to: determine a first welding position based on the first weld information, and determine a second welding position based on the second weld information; acquire motion information of the welding torch during the welding process, the motion information including the speed and acceleration of the welding torch; construct an optimization function based on the first welding position, the second welding position and the motion information, and solve the optimization function to obtain the target welding position.
[0103] In one possible implementation of this application embodiment, the optimization module 604 is further configured to: determine the current welding position of the welding torch; obtain the first squared distance between the first welding position and the welding position, and the second squared distance between the second welding position and the welding position; determine the first squared distance, the second squared distance, and the respective weight values of velocity and acceleration; and construct an optimization function based on the first squared distance, the second squared distance, velocity and acceleration, and their respective weight values.
[0104] In one possible implementation of this application embodiment, the optimization module 604 is further configured to: optimize the optimization function to obtain the welding position when the optimization function is minimized, and use it as the target welding position.
[0105] In one possible implementation of this application embodiment, the determining module 602 is further configured to: obtain the transformation matrix between the image coordinate system and the robot coordinate system; determine the welding start position of the welding torch based on the transformation matrix and the start position, and control the welding torch to move to the welding start position.
[0106] In one possible implementation of this application embodiment, the optimization module 604 is further configured to: control the welding torch to move to the target welding position in order to weld the target workpiece.
[0107] In one possible implementation of this application embodiment, the apparatus further includes: during the welding process of the welding torch, acquiring the third weld seam information of the target workpiece in real time based on a line laser sensor; obtaining the real-time welding position of the target workpiece based on the first weld seam information, the third weld seam information, and an optimization function; and welding the target workpiece based on the real-time welding position.
[0108] In the weld seam recognition device fused with a large model provided in this application embodiment, a first image of the target workpiece is acquired, and weld seam recognition is performed on the first image using a large model to obtain the first weld seam information of the target workpiece. The starting position of the weld seam can then be determined from the first weld seam information, and a line laser sensor is used to identify the weld seam at the starting position to obtain the second weld seam information. Furthermore, the welding position can be optimized based on the first and second weld seam information to obtain the target welding position, thereby controlling the welding torch to weld the target workpiece at the target welding position. Using a large model to recognize weld seam information ensures the accuracy of weld seam recognition, and by combining the large model recognition results with the line laser sensor recognition results, the welding position is optimized, achieving adaptive correction of the welding position. This provides a stable welding path, reduces welding defects, and improves welding quality. Using the method of this application embodiment, the welding torch can achieve efficient and accurate weld seam tracking in complex environments, significantly improving the automation level of welding, reducing manual intervention, and increasing production efficiency and welding quality.
[0109] It should be noted that the foregoing explanation of the weld seam recognition method embodiment of the fused large model also applies to the weld seam recognition device of the fused large model in this embodiment, and will not be repeated here.
[0110] To implement the above embodiments, this application also proposes an electronic device, including: a processor and a memory communicatively connected to the processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the method provided in the foregoing embodiments.
[0111] To implement the above embodiments, this application also proposes a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods provided in the foregoing embodiments.
[0112] To implement the above embodiments, this application also proposes a computer program product, including a computer program that, when executed by a processor, implements the methods provided in the foregoing embodiments.
[0113] The collection, storage, use, processing, transmission, provision, and application of user personal information involved in this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0114] It should be noted that personal information collected from users should be used for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. Furthermore, such collection / sharing should only be conducted after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization that includes authorization of relevant user information before the user uses the function. In addition, any necessary steps must be taken to protect and safeguard access to such personal information data and ensure that others with access to personal information data comply with their privacy policies and procedures.
[0115] This application is intended to provide an implementation scheme for users to selectively prevent the use or access to their personal information data. Specifically, this application is intended to provide hardware and / or software to prevent or block access to such personal information data. Once personal information data is no longer needed, risks can be minimized by restricting data collection and deleting data. Furthermore, where applicable, such personal information is de-identified to protect user privacy.
[0116] In the foregoing descriptions of the embodiments, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0117] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0118] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.
[0119] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.
[0120] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0121] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.
[0122] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.
[0123] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.
Claims
1. A weld seam identification method integrating a large model, characterized in that, The method includes: A first image of the target workpiece to be welded is acquired and input into a pre-trained large model. The large model performs weld seam recognition on the first image and outputs the first weld seam information of the target workpiece. Based on the first weld information, the starting position of the weld in the target workpiece is determined, and the welding torch is controlled to move to the starting position. Based on the line laser sensor of the welding torch, weld seam identification is performed at the starting position to obtain the second weld seam information of the target workpiece; Based on the first weld seam information and the second weld seam information, the welding position of the welding torch is optimized to obtain the target welding position.
2. The method according to claim 1, characterized in that, Before determining the starting position of the weld in the target workpiece based on the first weld information, the method further includes: Based on the first weld information, the three-dimensional 3D structural information of the weld in the target workpiece is determined; Based on the 3D structural information, determine whether the first image contains a complete weld seam; In response to the absence of a complete weld seam, the target workpiece is re-image acquired and its integrity identified until a second image containing a complete weld seam is obtained.
3. The method according to claim 1, characterized in that, The step of optimizing the welding position of the welding torch based on the first weld information and the second weld information to obtain the target welding position includes: The first welding position is determined based on the first weld information, and the second welding position is determined based on the second weld information; Acquire motion information of the welding torch during the welding process, the motion information including the speed and acceleration of the welding torch; Based on the first welding position, the second welding position, and the motion information, an optimization function is constructed and solved to obtain the target welding position.
4. The method according to claim 3, characterized in that, The step of constructing an optimization function based on the first welding position, the second welding position, and the motion information includes: Determine the current welding position of the welding torch; Obtain the square of the first distance between the first welding position and the welding position, and the square of the second distance between the second welding position and the welding position; Determine the weight values for the first squared distance, the second squared distance, and the velocity and acceleration, respectively; The optimization function is constructed based on the first squared distance, the second squared distance, the velocity and the acceleration, and their respective weight values.
5. The method according to claim 4, characterized in that, Solving the optimization function to obtain the target welding position includes: The optimization function is optimized to obtain the welding position when the optimization function is minimized, which is then used as the target welding position.
6. The method according to claim 1, characterized in that, The control of moving the welding torch to the starting position includes: Obtain the transformation matrix between the image coordinate system and the robot coordinate system; Based on the transformation matrix and the starting position, the welding starting position of the welding torch is determined, and the welding torch is controlled to move to the welding starting position.
7. The method according to claim 5, characterized in that, After obtaining the target welding position, the process further includes: The welding torch is controlled to move to the target welding position to weld the target workpiece.
8. The method according to claim 7, characterized in that, The method further includes: During the welding process of the welding torch, the third weld seam information of the target workpiece is collected in real time based on the line laser sensor; Based on the first weld information, the third weld information, and the optimization function, the real-time welding position of the target workpiece is obtained, and the target workpiece is welded based on the real-time welding position.
9. A weld seam recognition device integrating a large model, characterized in that, The device includes: The model recognition module is used to acquire a first image of the target workpiece to be welded, and input the first image into a pre-trained large model. The large model performs weld seam recognition on the first image and outputs the first weld seam information of the target workpiece. The determination module is used to determine the starting position of the weld in the target workpiece based on the first weld information, and control the welding torch to move to the starting position; The sensor recognition module is used to identify the weld seam at the starting position based on the line laser sensor of the welding gun, and obtain the second weld seam information of the target workpiece; The optimization module is used to optimize the welding position of the welding torch based on the first weld information and the second weld information to obtain the target welding position.
10. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-8.
11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-8.
12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-8.