Weld seam identification method, device, electronic device and storage medium integrating large model

Abstract

The present application proposes a weld recognition method, device, electronic device and storage medium that integrates a large model. The method includes: collecting a first image of a target workpiece to be welded, and inputting the first image into a pre-trained large model, the large model performs weld recognition on the first image, and outputs the first weld information of the target workpiece; based on the first weld information, determining the starting position of the weld in the target workpiece, and controlling the welding gun to move to the starting position; based on the line laser sensor of the welding gun, performing weld recognition at the starting position to obtain the second weld information of the target workpiece; based on the first weld information and the second weld information, optimizing the welding position of the welding gun to obtain the target welding position. As a result, the present solution realizes adaptive correction of the welding position, thereby providing a stable welding path, reducing welding defects, and improving the quality of welding.

Description

Technical Field

[0001] The present application relates to the fields of large models, image processing, and welding technology, and in particular to a weld recognition method, device, electronic device, and storage medium integrated with a large model. Background Art

[0002] Currently, welding robots have been widely used in the industrial field. However, during the welding process, due to various unfavorable factors, such as thermal deformation, vibration, or sudden movement of the welding workpiece, the welding trajectory of the welding robot will deviate from the actual weld position during the welding process, resulting in the welding robot being unable to complete the work, the welding quality being reduced, and the overall welding production efficiency being affected. Summary of the invention

[0003] The purpose of this application is to solve one of the technical problems in the related art at least to some extent.

[0004] To this end, the first purpose of this application is to propose a weld identification method that integrates a large model to achieve adaptive correction of the welding position, thereby providing a stable welding path, reducing welding defects, and improving the quality of welding.

[0005] The second objective of the present application is to provide a weld identification device integrating a large model.

[0006] The third objective of the present application is to provide an electronic device.

[0007] A fourth objective of the present application is to provide a computer-readable storage medium.

[0008] A fifth object of the present application is to provide a computer program product.

[0009] To achieve the above-mentioned purpose, the first aspect of the present application proposes a weld recognition method integrating a large model, comprising: collecting a first image of a target workpiece to be welded, and inputting the first image into a pre-trained large model, wherein the large model performs weld recognition on the first image, and outputs first weld information of the target workpiece; based on the first weld information, determining the starting position of the weld in the target workpiece, and controlling the welding gun to move to the starting position; based on the line laser sensor of the welding gun, performing weld recognition at the starting position to obtain second weld information of the target workpiece; based on the first weld information and the second weld information, optimizing the welding position of the welding gun to obtain the target welding position.

[0010] To achieve the above-mentioned purpose, the second aspect of the present application proposes a weld recognition device integrated with a large model, including: a model recognition module, used to collect a first image of a target workpiece to be welded, and input the first image into a pre-trained large model, and the large model performs weld recognition on the first image, and outputs first weld information of the target workpiece; a determination module, used to determine the starting position of the weld in the target workpiece based on the first weld information, and control the welding gun to move to the starting position; a sensor recognition module, used to perform weld recognition at the starting position based on the line laser sensor of the welding gun, and obtain second weld information of the target workpiece; an optimization module, used to optimize the welding position of the welding gun based on the first weld information and the second weld information to obtain the target welding position.

[0011] To achieve the above-mentioned purpose, the third aspect embodiment of the present application proposes an electronic device, comprising: a processor; and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the processor can execute the weld identification method of the fused large model described in the first aspect embodiment above.

[0012] To achieve the above-mentioned purpose, the fourth aspect of the present application proposes a computer-readable storage medium on which a computer program is stored, and the computer instructions are used to enable the computer to execute the weld identification method of the fused large model described in the first aspect of the above-mentioned embodiment.

[0013] To achieve the above-mentioned purpose, the fifth embodiment of the present application proposes a computer program product, including a computer program, which, when executed by a processor, implements the weld identification method of the fused large model described in the first embodiment above.

[0014] The weld seam identification method, device, electronic device and storage medium provided by the present application can obtain the first weld seam information of the target workpiece by collecting the first image of the target workpiece and using the large model to perform weld seam identification on the first image. Thus, the starting position of the weld seam can be determined from the first weld seam information, and the weld seam can be identified using a line laser sensor at the starting position of the weld seam to obtain the second weld seam information. Further, the welding position can be optimized according to the first weld seam information and the second weld seam information to obtain the target welding position, so as to control the welding gun to weld the target workpiece at the target welding position. Using the large model to identify the weld seam information can ensure the accuracy of the weld seam identification, and combining the large model identification result and the line laser sensor identification result, optimize the welding position, and realize the adaptive correction of the welding position, so as to provide a stable welding path, reduce welding defects, and improve the quality of welding. Using the method of the embodiment of the present application, the welding gun can realize efficient and accurate weld seam tracking in a complex environment, greatly improve the automation level of welding, reduce manual intervention, and improve production efficiency and welding quality.

[0015] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0017] Figure 1 A schematic diagram of a flow chart of a weld identification method integrating a large model provided in an embodiment of the present application;

[0018] Figure 2 A schematic flow chart of another weld identification method integrating a large model provided in an embodiment of the present application;

[0019] Figure 3 A schematic flow chart of another weld identification method integrating a large model provided in an embodiment of the present application;

[0020] Figure 4 A schematic diagram of the structure of a welding system provided in an embodiment of the present application;

[0021] Figure 5 A schematic diagram of a process for determining a target welding position provided in an embodiment of the present application;

[0022] Figure 6 A schematic diagram of the structure of a weld identification device integrating a large model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0024] The following describes the weld recognition method and device for fusion of large models in an embodiment of the present application with reference to the accompanying drawings.

[0025] Figure 1 is a flow chart of a weld identification method integrating a large model provided in an embodiment of the present application, such as Figure 1 As shown, the weld identification method of the fusion large model in the embodiment of the present application includes but is not limited to the following steps:

[0026] S101, collecting a first image of a target workpiece to be welded, and inputting the first image into a pre-trained large model, the large model performs weld seam recognition on the first image, and outputs first weld seam information of the target workpiece.

[0027] It should be noted that the execution subject of the weld identification method for integrating a large model provided in the embodiment of the present application is an electronic device, which may be a terminal device. Optionally, the terminal device may be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device may be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc., and the non-mobile electronic device may be a personal computer (PC), a television, etc. The embodiment of the present application is not specifically limited.

[0028] In some implementations, an image acquisition device may be used to acquire an image of a target workpiece to be welded, and obtain a first image of the target workpiece. The first image is input into a pre-trained large model, and the large model recognizes the weld in the first image, and obtains and outputs the first weld information of the target workpiece. The large model may be a visual large model. Optionally, the first weld information includes the position information of the weld, the structural information of the weld, and the like.

[0029] S102, based on the first weld information, determining the starting position of the weld in the target workpiece, and controlling the welding gun to move to the starting position.

[0030] In some implementations, the position information of the weld can be obtained from the first weld information, and the starting position of the weld can be determined from the position information of the weld by determining the welding direction during welding. For example, if the weld of the target workpiece is identified as a vertical weld, by determining that the weld is welded from top to bottom, the position of the uppermost end of the weld can be determined as the starting position.

[0031] Further, when the starting position is determined, the welding gun can be controlled to weld the weld from the starting position. In other words, the welding gun is moved to the starting position so that the welding gun welds from the starting position.

[0032] In some implementations, since the starting position of the weld is in the image coordinate system, and the welding gun uses the coordinates in the robot coordinate system for welding, the starting position is converted from the image coordinate system to the robot coordinate system to obtain the converted starting position, so as to control the welding gun to move to the converted starting position.

[0033] S103, based on the line laser sensor of the welding gun, performing weld seam recognition at the starting position to obtain second weld seam information of the target workpiece.

[0034] In some implementations, before the welding gun performs welding, the weld can be identified using a line laser based on the line laser sensor of the welding gun itself to determine the second weld information of the target workpiece, wherein the second weld information includes the position information of the weld.

[0035] S104, optimizing the welding position of the welding gun based on the first weld information and the second weld information to obtain a target welding position.

[0036] In some implementations, the weld position identified by a line laser sensor is easily affected by the environment. For example, when the weld is uneven or the weld cross-section has multiple inflection points, positioning errors may occur. Therefore, the first weld information and the second weld information identified by the large model can be used to optimize the welding position to obtain an optimized target welding position, and then control the welding gun to perform welding based on the target welding position.

[0037] In some implementations, an optimization function may be determined based on the first welding position in the first weld information and the second welding position in the second weld information, and the optimization function may be used to optimize the welding position to obtain a target welding position.

[0038] Optionally, in order to ensure the smoothness of the welding trajectory during the welding process, an optimization function can be constructed based on the speed and acceleration of the welding gun, combined with the first welding position and the second welding position, and then the optimization function can be used to optimize the welding position to obtain the target welding position.

[0039] In the weld seam recognition method fused with a large model provided in the embodiment of the present application, by collecting a first image of the target workpiece and using the large model to perform weld seam recognition on the first image, the first weld seam information of the target workpiece can be obtained. Thus, the starting position of the weld seam can be determined from the first weld seam information, and the weld seam can be recognized using a line laser sensor at the starting position of the weld seam to obtain the second weld seam information. Further, the welding position can be optimized according to the first weld seam information and the second weld seam information to obtain the target welding position, so as to control the welding gun to weld the target workpiece at the target welding position. Using the large model to recognize the weld seam information can ensure the accuracy of the weld seam recognition, and combining the recognition results of the large model and the recognition results of the line laser sensor, the welding position can be optimized to achieve adaptive correction of the welding position, so as to provide a stable welding path, reduce welding defects, and improve the quality of welding. Using the method of the embodiment of the present application, the welding gun can achieve efficient and accurate weld seam tracking in a complex environment, greatly improve the automation level of welding, reduce manual intervention, and improve production efficiency and welding quality.

[0040] Figure 2 is a flow chart of a weld identification method integrating a large model provided in an embodiment of the present application, such as Figure 2 As shown, the weld identification method of the fusion large model in the embodiment of the present application includes but is not limited to the following steps:

[0041] S201, collecting a first image of a target workpiece to be welded, and inputting the first image into a pre-trained large model, allowing the large model to perform weld seam recognition on the first image, and outputting first weld seam information of the target workpiece.

[0042] In the embodiment of the present application, the implementation method of step S201 can be implemented by any method in the embodiments of the present application, which is not limited here and will not be repeated.

[0043] S202, based on the first weld information, determine the starting position of the weld in the target workpiece, and control the welding gun to move to the starting position.

[0044] In some implementations, in order to ensure the integrity of weld information identification, the weld in the first weld information may be identified for integrity before the starting position of the weld is determined to improve welding accuracy and prevent welding defects.

[0045] In some implementations, the integrity of the weld can be identified based on the structural information of the weld. Optionally, the three-dimensional (3D) structural information of the weld in the target workpiece can be determined based on the first weld information. Further, based on the 3D structural information, it can be determined whether the first image contains a complete weld.

[0046] Optionally, the 3D structural information includes geometric features of the weld, such as height, width, length and shape, etc. Feature information of the weld can be extracted from the 3D structural information, and the extracted feature information is compared with feature information of a standard weld to obtain a comparison result, and it is determined whether the first image contains a complete weld according to the comparison result.

[0047] Optionally, if the comparison result indicates that the extracted feature information is the same as the feature information of the standard weld, it is determined that the first image contains a complete weld, otherwise it is determined that the first image does not contain a complete weld.

[0048] In some implementations, in response to not including a complete weld, the target workpiece is re-imaged and integrity identified until a second image including a complete weld is obtained. Further, the second image can be identified based on the large model to obtain complete first weld information, and then the starting position of the weld can be determined from the first weld information.

[0049] In the embodiment of the present application, the method of controlling the welding gun to move to the starting position in step S202 can be implemented by any method in the various embodiments of the present application, which is not limited here and will not be repeated.

[0050] S203, based on the line laser sensor of the welding gun, perform weld recognition at the starting position to obtain second weld information of the target workpiece.

[0051] In the embodiment of the present application, the implementation method of step S203 can be implemented by any method in the embodiments of the present application, which is not limited here and will not be repeated.

[0052] S204, determining a first welding position based on the first weld information, and determining a second welding position based on the second weld information.

[0053] In some implementations, multiple first weld positions can be determined from the first weld information, and the first weld positions can be converted from the image coordinate system to the robot coordinate system to obtain the first welding positions. Multiple second weld positions can be determined from the second weld information, and the second weld positions can be converted from the point cloud coordinate system to the robot coordinate system to obtain the second welding positions.

[0054] S205, obtaining movement information of the welding gun during the welding process, where the movement information includes the speed and acceleration of the welding gun.

[0055] In some implementations, historical welding data of the welding gun may be obtained, and the speed and acceleration of the welding gun during the welding process may be obtained from the historical welding data as motion information. Alternatively, multiple historical welding data may be obtained, and multiple speeds and accelerations may be extracted from the historical welding data, and then the multiple speeds and accelerations may be averaged, and the obtained average speed and average acceleration may be used as the speed and acceleration of the welding gun during the welding process.

[0056] S206, constructing an optimization function based on the first welding position, the second welding position and the motion information, and solving the optimization function to obtain a target welding position.

[0057] In some implementations, in order to ensure the smoothness of the welding trajectory during the welding process, an optimization function can be constructed based on the first welding position, the second welding position and the motion information, and then the optimization function can be solved to obtain an accurate and smooth target welding position.

[0058] In some implementations, an optimization function may be constructed based on the square of the distance between welding positions, the speed and acceleration, and their respective weight values, that is, by determining the welding position where the welding gun is currently located, and obtaining the square of the first distance between the first welding position and the welding position, and the square of the second distance between the second welding position and the welding position.

[0059] Optionally, the square of the Euclidean distance between the first welding position and the welding position may be calculated as the first distance square, and the square of the Euclidean distance between the second welding position and the welding position may be calculated as the second distance square.

[0060] Furthermore, weight values ​​of the first distance square, the second distance square, and the speed and acceleration are determined, and an optimization function is constructed based on the first distance square, the second distance square, the speed and acceleration, and the weight values.

[0061] Optionally, the formula of the optimization function is as follows:

[0062] (1)

[0063] in, represents the second distance squared, represents the first distance square, Indicates the second welding position, Indicates the first welding position, Indicates the target welding position, Indicates speed, represents acceleration, ω ldis represents the weight value of the second distance square, ω vdisrepresents the weight value of the first distance square, ω vel represents the weight value of speed, ω acc Indicates the weight value of acceleration.

[0064] Optionally, the distance squared can be calculated based on the coordinate information of the location. The formula for calculating the distance squared is as follows:

[0065] (2)

[0066] in,( x ref , y ref , z ref ) indicates the position p ref The coordinate value of ( x target , y target , z target ) indicates the position p target The coordinate value of .

[0067] Optionally, the first welding position and the welding position are substituted into formula (2) to calculate the square of the first distance. The second welding position and the welding position are substituted into formula (2) to calculate the square of the second distance.

[0068] Furthermore, after obtaining the optimization function, the optimization function may be solved, and by optimizing the optimization function, the welding position when the optimization function is minimized is obtained as the target welding position.

[0069] In some implementations, after obtaining the target welding position, the welding gun can be controlled to move to the target welding position to weld the target workpiece, and during the welding process of the welding gun, the third weld seam information of the target workpiece is collected in real time based on the line laser sensor. Then, the real-time welding position of the target workpiece can be obtained based on the first weld seam information, the third weld seam information and the optimization function, and the target workpiece is welded based on the real-time welding position.

[0070] That is, the first welding position in the first weld information and the third welding position in the third weld information can be substituted into formula (1) to obtain the welding position when the optimization function is minimized as the real-time welding position.

[0071] In the weld seam identification method fused with a large model provided in the embodiment of the present application, by collecting a first image of the target workpiece and using the large model to perform weld seam identification on the first image, the first weld seam information of the target workpiece can be obtained. Thus, the starting position of the weld seam can be determined from the first weld seam information, and the weld seam can be identified using a line laser sensor at the starting position of the weld seam to obtain the second weld seam information. Further, an optimization function can be constructed according to the first weld seam information and the second weld seam information, and the optimization function can be solved to obtain the target welding position, so as to control the welding gun to weld the target workpiece at the target welding position. Using the large model to identify the weld seam information can ensure the accuracy of the weld seam identification, and combining the large model identification result and the line laser sensor identification result, optimize the welding position, and realize the adaptive correction of the welding position, so as to provide a stable welding path, reduce welding defects, and improve the quality of welding. Using the method of the embodiment of the present application, the welding gun can realize efficient and accurate weld seam tracking in a complex environment, greatly improve the automation level of welding, reduce manual intervention, and improve production efficiency and welding quality.

[0072] Figure 3 is a flow chart of a weld identification method integrating a large model provided in an embodiment of the present application, such as Figure 3 As shown, the weld identification method of the fusion large model in the embodiment of the present application includes but is not limited to the following steps:

[0073] S301, collecting a first image of a target workpiece to be welded, and inputting the first image into a pre-trained large model, the large model performs weld seam recognition on the first image, and outputs first weld seam information of the target workpiece.

[0074] In the embodiment of the present application, the implementation method of step S301 can be implemented by any method in the embodiments of the present application, which is not limited here and will not be repeated.

[0075] S302: Determine a starting point position of a weld in a target workpiece based on the first weld information.

[0076] In the embodiment of the present application, the implementation method of step S302 can be implemented by any method in the embodiments of the present application, which is not limited here and will not be repeated.

[0077] S303, obtaining a transformation matrix between the image coordinate system and the robot coordinate system.

[0078] S304, based on the conversion matrix and the starting point position, determine the welding starting point position of the welding gun, and control the welding gun to move to the welding starting point position.

[0079] In some implementations, since the starting position of the weld is in the image coordinate system and the welding gun is in the robot coordinate system, the starting position in the image coordinate system can be converted to the robot coordinate system according to the conversion matrix of the coordinate system, and then the welding gun can be controlled to move to the starting position.

[0080] Optionally, the transformation matrix between the image coordinate system and the robot coordinate system can be calculated based on the rotation matrix and translation vector between the image coordinate system and the robot coordinate system, and the starting point position can be transformed using the transformation matrix to obtain the welding starting point position of the robot coordinate system where the welding gun is located, and then the welding gun can be controlled to move to the welding starting point position.

[0081] S305, based on the line laser sensor of the welding gun, the weld seam is identified at the starting position to obtain the second weld seam information of the target workpiece.

[0082] In the embodiment of the present application, the implementation method of step S305 can be implemented by any method in the embodiments of the present application, which is not limited here and will not be repeated.

[0083] S306, optimizing the welding position of the welding gun based on the first weld information and the second weld information to obtain a target welding position.

[0084] In the embodiment of the present application, the implementation method of step S306 can be implemented by any method in the embodiments of the present application, which is not limited here and will not be repeated.

[0085] In the weld seam recognition method fused with a large model provided in the embodiment of the present application, by collecting the first image of the target workpiece and using the large model to perform weld seam recognition on the first image, the first weld seam information of the target workpiece can be obtained. Thus, the starting position of the weld seam can be determined from the first weld seam information, and the welding gun can be controlled to move to the welding starting position corresponding to the starting position, and the weld seam can be recognized using a line laser sensor to obtain the second weld seam information. Further, the welding position can be optimized according to the first weld seam information and the second weld seam information to obtain the target welding position, so as to control the welding gun to weld the target workpiece at the target welding position. Using the large model to recognize the weld seam information can ensure the accuracy of the weld seam recognition, and combine the recognition results of the large model and the recognition results of the line laser sensor to optimize the welding position and realize adaptive correction of the welding position, so as to provide a stable welding path, reduce welding defects, and improve the quality of welding. Using the method of the embodiment of the present application, the welding gun can realize efficient and accurate weld seam tracking in a complex environment, greatly improve the automation level of welding, reduce manual intervention, and improve production efficiency and welding quality.

[0086] For example, Figure 4The figure shows a schematic diagram of a welding system constructed using the method provided in the embodiment of the present application. Figure 4 It includes visual perception module, line laser perception module, control center and motion controller.

[0087] Among them, the visual perception module is equipped with functions such as visual image acquisition and image preprocessing. The line laser perception module is equipped with functions such as line laser sensor control and signal processing. The control center is used to complete tasks scheduling, model reasoning, algorithm operation and other functions. The motion controller is responsible for executing motion instructions and completing the given actions.

[0088] The control center sends an image acquisition instruction to the visual perception module. After receiving the instruction, the visual perception module acquires the image of the target workpiece and preprocesses the image, and then sends the preprocessed image to the control center. The control center calls the pre-trained large model and uses the large model to identify the weld in the image to obtain the first weld information.

[0089] The control center sends a weld recognition instruction to the line laser sensing module. After receiving the instruction, the line laser sensing module uses the line laser sensor to identify the weld of the target workpiece, obtains the second weld information, and sends the second weld information to the control center.

[0090] After the control center receives the first weld information and the second weld information, it can use an optimization algorithm to optimize the welding position of the welding gun according to the first weld information and the second weld information to obtain the target welding position, and send a welding instruction carrying the target welding position to the motion controller to enable the motion controller to weld the target workpiece based on the welding instruction.

[0091] Figure 5 The flowchart for determining the target welding position is shown. By performing image acquisition on the target workpiece, an RGB image of the target workpiece is obtained, wherein the RGB image corresponds to the first image in the embodiment of the present application. The RGB image is used to identify the weld seam using a pre-trained visual large model, and the semantic information and 3D structure information of the weld seam are obtained as the first weld seam information. The integrity of the weld seam is judged based on the first weld seam information, and the starting position of the weld seam is determined when the weld seam is complete. If the weld seam is incomplete, the RGB image is re-acquired and the integrity of the weld seam is identified until a complete weld seam is obtained.

[0092] Further, the starting point position is transformed into a welding starting point position in the robot coordinate system, and the welding gun is controlled to move to the welding starting point position. The line laser sensor of the welding gun collects the second weld seam information of the target workpiece, and uses the first weld seam information, the second weld seam information and the speed and acceleration of the welding gun to determine the optimization function, and calculates the welding position when the optimization function is minimized as the target welding position, so as to control the welding gun to weld the target workpiece according to the target welding position.

[0093] In some implementations, it is determined in real time during the welding process whether the weld has reached the end point of the weld, and the welding is terminated when the weld has reached the end point of the weld. If the weld has not reached the end point of the weld, the line laser sensor is used to continue to identify the weld information to achieve motion tracking of the welding process.

[0094] Corresponding to the weld recognition methods for the fused large model proposed in the above-mentioned embodiments, an embodiment of the present application further proposes a weld recognition device for the fused large model. Since the weld recognition device for the fused large model proposed in the embodiment of the present application corresponds to the weld recognition methods for the fused large model proposed in the above-mentioned embodiments, the implementation method of the weld recognition method for the fused large model is also applicable to the weld recognition device for the fused large model proposed in the embodiment of the present application, and will not be described in detail in the following embodiments.

[0095] In order to implement the above-mentioned embodiment, the present application also proposes a weld identification device integrating a large model.

[0096] Figure 6 A schematic diagram of the structure of a weld identification device integrating a large model provided in an embodiment of the present application.

[0097] like Figure 6 As shown, the weld identification device 600 of the fusion large model includes:

[0098] The model recognition module 601 is used to collect a first image of a target workpiece to be welded, and input the first image into a pre-trained large model, so that the large model performs weld seam recognition on the first image and outputs first weld seam information of the target workpiece;

[0099] A determination module 602 is used to determine the starting position of the weld in the target workpiece based on the first weld information, and control the welding gun to move to the starting position;

[0100] The sensor recognition module 603 is used to perform weld seam recognition at the starting position based on the line laser sensor of the welding gun to obtain the second weld seam information of the target workpiece;

[0101] The optimization module 604 is used to optimize the welding position of the welding gun based on the first weld information and the second weld information to obtain a target welding position.

[0102] In a possible implementation of an embodiment of the present application, the determination module 602 is further used to: determine the three-dimensional 3D structural information of the weld in the target workpiece based on the first weld information; based on the 3D structural information, determine whether the first image contains a complete weld; in response to not containing a complete weld, re-capture the image and re-identify the integrity of the target workpiece until a second image containing a complete weld is obtained.

[0103] In a possible implementation of an embodiment of the present application, the optimization module 604 is also used to: determine a first welding position based on the first weld information, and determine a second welding position based on the second weld information; obtain motion information of the welding gun during the welding process, the motion information including the speed and acceleration of the welding gun; construct an optimization function based on the first welding position, the second welding position and the motion information, and solve the optimization function to obtain a target welding position.

[0104] In a possible implementation of an embodiment of the present application, the optimization module 604 is also used to: determine the current welding position of the welding gun; obtain the first distance square between the first welding position and the welding position, and the second distance square between the second welding position and the welding position; determine the first distance square, the second distance square, and the respective weight values ​​of the speed and acceleration; construct an optimization function based on the first distance square, the second distance square, the speed and acceleration, and their respective weight values.

[0105] In a possible implementation of the embodiment of the present application, the optimization module 604 is further used to: optimize the optimization function to obtain a welding position when the optimization function is minimized as a target welding position.

[0106] In a possible implementation of an embodiment of the present application, the determination module 602 is also used to: obtain a transformation matrix between the image coordinate system and the robot coordinate system; determine the welding starting position of the welding gun based on the transformation matrix and the starting position, and control the welding gun to move to the welding starting position.

[0107] In a possible implementation of the embodiment of the present application, the optimization module 604 is further used to control the welding gun to move to a target welding position to weld the target workpiece.

[0108] In a possible implementation of an embodiment of the present application, the device also includes: during the welding process of the welding gun, based on the line laser sensor, real-time collection of the third weld information of the target workpiece; based on the first weld information, the third weld information and the optimization function, obtaining the real-time welding position of the target workpiece, and based on the real-time welding position, welding the target workpiece.

[0109] In the weld seam recognition device integrated with a large model provided in the embodiment of the present application, by collecting the first image of the target workpiece and using the large model to perform weld seam recognition on the first image, the first weld seam information of the target workpiece can be obtained. Thus, the starting position of the weld seam can be determined from the first weld seam information, and the weld seam can be recognized using a line laser sensor at the starting position of the weld seam to obtain the second weld seam information. Further, the welding position can be optimized according to the first weld seam information and the second weld seam information to obtain the target welding position, so as to control the welding gun to weld the target workpiece at the target welding position. Using the large model to recognize the weld seam information can ensure the accuracy of the weld seam recognition, and combining the recognition results of the large model and the recognition results of the line laser sensor to optimize the welding position and realize adaptive correction of the welding position, so as to provide a stable welding path, reduce welding defects, and improve the quality of welding. Using the method of the embodiment of the present application, the welding gun can realize efficient and accurate weld seam tracking in a complex environment, greatly improve the automation level of welding, reduce manual intervention, and improve production efficiency and welding quality.

[0110] It should be noted that the above explanation of the embodiment of the weld identification method of the fused large model is also applicable to the weld identification device of the fused large model of this embodiment, and will not be repeated here.

[0111] In order to implement the above embodiments, the present application also proposes an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method provided by the above embodiments.

[0112] In order to implement the above embodiments, the present application also proposes a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the methods provided by the above embodiments.

[0113] In order to implement the above embodiments, the present application also proposes a computer program product, including a computer program, which implements the methods provided by the above embodiments when executed by a processor.

[0114] The collection, storage, use, processing, transmission, provision and application of user personal information involved in this application are in compliance with relevant laws and regulations and do not violate public order and good morals.

[0115] It should be noted that personal information from users should be collected for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. In addition, such collection / sharing should be carried out after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign the agreement / authorization including authorization of relevant user information before the user uses the function. In addition, any necessary steps should be taken to protect and safeguard access to such personal information data and ensure that others who have access to personal information data comply with its privacy policy and procedures.

[0116] This application is expected to provide an implementation scheme for users to selectively block the use or access of personal information data. That is, this application is expected to provide hardware and / or software to prevent or block access to such personal information data. Once the personal information data is no longer needed, risks can be minimized by limiting data collection and deleting data. In addition, when applicable, personal identifiers are removed from such personal information to protect the privacy of users.

[0117] In the description of the aforementioned embodiments, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they contradict each other.

[0118] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0119] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0120] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0121] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0122] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0123] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0124] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A weld recognition method integrating a large model, characterized in that: The method comprises: Capturing a first image of a target workpiece to be welded, and inputting the first image into a pre-trained large model, wherein the large model performs weld seam recognition on the first image, and outputs first weld seam information of the target workpiece; Based on the first weld information, determine the starting position of the weld in the target workpiece, and control the welding gun to move to the starting position; Based on the line laser sensor of the welding gun, weld seam identification is performed at the starting point to obtain second weld seam information of the target workpiece; Based on the first weld information and the second weld information, optimizing the welding position of the welding gun to obtain a target welding position; The step of optimizing the welding position of the welding gun based on the first weld information and the second weld information to obtain a target welding position includes: determining a first welding position based on the first weld information, and determining a second welding position based on the second weld information; Acquiring motion information of the welding gun during welding, wherein the motion information includes speed and acceleration of the welding gun; Based on the first welding position, the second welding position and the motion information, construct an optimization function, and solve the optimization function to obtain the target welding position; The constructing an optimization function based on the first welding position, the second welding position and the motion information comprises: Determining the current welding position of the welding gun; Acquire a first square distance between the first welding position and the first welding position, and a second square distance between the second welding position and the first welding position; Determine the first distance squared, the second distance squared, and respective weight values ​​of the velocity and the acceleration; The optimization function is constructed based on the first distance squared, the second distance squared, the speed and the acceleration, and the respective weight values.

2. The method according to claim 1, characterized in that: Before determining the starting position of the weld in the target workpiece based on the first weld information, the method further includes: Based on the first weld information, determining three-dimensional 3D structural information of the weld in the target workpiece; Based on the 3D structural information, determining whether the first image contains a complete weld; In response to not including the complete weld seam, image acquisition and integrity recognition are performed again on the target workpiece until a second image including the complete weld seam is obtained.

3. The method according to claim 1, characterized in that The step of solving the optimization function to obtain the target welding position includes: The optimization function is optimized to obtain a welding position when the optimization function is minimized, which is used as the target welding position.

4. The method according to claim 1, characterized in that: The controlling the welding gun to move to the starting position comprises: Get the transformation matrix between the image coordinate system and the robot coordinate system; Based on the conversion matrix and the starting point position, a welding starting point position of a welding gun is determined, and the welding gun is controlled to move to the welding starting point position.

5. The method according to claim 3, characterized in that: After obtaining the target welding position, the method further includes: The welding gun is controlled to move to the target welding position to weld the target workpiece.

6. The method according to claim 5, characterized in that The method further comprises: During the welding process of the welding gun, based on the line laser sensor, the third weld seam information of the target workpiece is collected in real time; Based on the first weld information, the third weld information and the optimization function, a real-time welding position of the target workpiece is acquired, and based on the real-time welding position, the target workpiece is welded.

7. A weld identification device integrating a large model, characterized in that: The device comprises: A model recognition module is used to collect a first image of a target workpiece to be welded, and input the first image into a pre-trained large model, so that the large model performs weld seam recognition on the first image and outputs first weld seam information of the target workpiece; A determination module, configured to determine a starting position of a weld in the target workpiece based on the first weld information, and control a welding gun to move to the starting position; A sensor recognition module, used for performing weld seam recognition at the starting position based on a line laser sensor of the welding gun to obtain second weld seam information of the target workpiece; An optimization module, configured to optimize a welding position of the welding gun based on the first weld information and the second weld information to obtain a target welding position; The optimization module is further used to determine a first welding position based on the first weld information, and to determine a second welding position based on the second weld information; Acquiring motion information of the welding gun during welding, wherein the motion information includes speed and acceleration of the welding gun; Based on the first welding position, the second welding position and the motion information, construct an optimization function, and solve the optimization function to obtain the target welding position; The constructing an optimization function based on the first welding position, the second welding position and the motion information comprises: Determining the current welding position of the welding gun; Acquire a first square distance between the first welding position and the first welding position, and a second square distance between the second welding position and the first welding position; Determine the first distance squared, the second distance squared, and respective weight values ​​of the velocity and the acceleration; The optimization function is constructed based on the first distance squared, the second distance squared, the speed and the acceleration, and the respective weight values.

8. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 6 when executed by a processor.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 6 when being executed by a processor.

Images