Welding track generation method and device, equipment, storage medium and program product

Abstract

The invention relates to a welding track generation method and device, equipment, a storage medium and a program product. The method comprises the following steps: acquiring a template image and a current image shot by a to-be-welded workpiece under different conditions; converting a pixel track feature point set in the current image into a point cloud track feature point set according to feature matching points between the template image and the current image; according to the point cloud track feature point set, a first track registration matrix of the to-be-welded workpiece is generated; according to the first track registration matrix and a second track registration matrix, a welding track registration matrix of the to-be-welded workpiece is constructed, and the second track registration matrix is obtained based on iterative updating of the first track registration matrix; and generating a welding track of the to-be-welded workpiece according to the welding track registration matrix. By adopting the method, the generation accuracy of welding track generation is improved.

Description

Technical Field

[0001] The present application relates to the field of welding technology, and in particular to a welding trajectory generation method, device, computer equipment, computer-readable storage medium and computer program product. Background Art

[0002] With the continuous development of science and technology, trajectory welding technology in many industries is in continuous iteration and updating. Taking intersecting line welding technology as an example, manual teaching, visual recognition or point cloud registration are usually used to generate welding trajectories, thereby assisting in the execution of welding tasks. Among them, point cloud registration has gradually become the mainstream means of generating welding trajectories due to its advantages of high efficiency and high precision.

[0003] At present, in the process of welding trajectory generation, if the feature descriptor of the workpiece can be accurately extracted, point cloud registration can be performed by using ICP (Iterative Closest Point). However, since the shooting conditions for the welding workpiece are in dynamic change, such as changes in the workpiece position, changes in the workpiece shooting angle, or changes in the shooting time point, misleading information may be introduced in the process of point cloud registration. That is, the image obtained by shooting cannot reflect the actual matching situation between the current point cloud and the template point cloud, which makes it easy to fall into the optimal solution. Therefore, the current generation accuracy of welding trajectory is low. Summary of the invention

[0004] Based on this, it is necessary to provide a welding trajectory generation method, device, computer equipment, computer-readable storage medium and computer program product to improve the generation accuracy of welding trajectory generation in response to the above-mentioned technical problems.

[0005] In a first aspect, the present application provides a welding trajectory generation method, comprising:

[0006] Acquire a template image and a current image of the workpiece to be welded taken under different conditions;

[0007] According to the feature matching points between the template image and the current image, a pixel trajectory feature point set in the current image is converted into a point cloud trajectory feature point set;

[0008] Generating a first trajectory registration matrix of the workpiece to be welded according to the point cloud trajectory feature point set;

[0009] Constructing a welding trajectory registration matrix of the workpiece to be welded according to the first trajectory registration matrix and the second trajectory registration matrix, wherein the second trajectory registration matrix is ​​obtained by iteratively updating the first trajectory registration matrix;

[0010] The welding trajectory of the workpiece to be welded is generated according to the welding trajectory registration matrix.

[0011] In one of the embodiments, converting the pixel trajectory feature point set in the current image into a point cloud trajectory feature point set according to the feature matching points between the template image and the current image includes:

[0012] Determining a geometric transformation relationship between the template image and the current image according to feature matching points between the template image and the current image;

[0013] According to the geometric transformation relationship, the position information of the pixel trajectory feature point set in the current image is obtained by transformation;

[0014] The pixel trajectory feature point set is converted into a point cloud trajectory feature point set according to the position information and the depth information of the current image.

[0015] In one of the embodiments, before converting the pixel trajectory feature point set into a point cloud trajectory feature point set according to the position information and the depth information of the current image, the method further includes:

[0016] Selection step: selecting candidate pixel trajectory feature points from the pixel trajectory feature point set;

[0017] In the case where the candidate point cloud trajectory feature points obtained by converting the candidate pixel trajectory feature points are located in a preset space, the candidate pixel trajectory feature points are used as target pixel trajectory feature points for constructing the pixel trajectory feature point set;

[0018] Return to execute the selection step until all pixel trajectory feature points in the pixel trajectory feature point set are selected.

[0019] In one of the embodiments, converting the pixel trajectory feature point set into a point cloud trajectory feature point set according to the position information and the depth information of the current image includes:

[0020] According to the position information and the depth information of the current image, the pixel trajectory feature point set is converted into a candidate point cloud trajectory feature point set, wherein the candidate point cloud trajectory feature point set includes a plurality of candidate point cloud trajectory feature points;

[0021] According to preset geometric constraints, the point pair relationships of multiple candidate point cloud trajectory feature points are screened to obtain multiple candidate point pair relationships;

[0022] The pixel trajectory feature points corresponding to the plurality of candidate point pairs are converted to obtain the point cloud trajectory feature point set.

[0023] In one embodiment, generating a first trajectory registration matrix of the workpiece to be welded according to the point cloud trajectory feature point set includes:

[0024] Dividing step: dividing the point cloud trajectory feature point set into a plurality of first point cloud trajectory feature point pairs and a plurality of second point cloud trajectory feature point pairs;

[0025] According to the rigid body transformation matrix constructed by the plurality of first point cloud trajectory feature point pairs, counting the number of point pairs among the plurality of second point cloud trajectory feature point pairs as target point cloud trajectory feature point pairs;

[0026] Return to execute the division step until the number of divisions of the point cloud trajectory feature point set reaches a preset division threshold, and select the first trajectory registration matrix of the workpiece to be welded from multiple rigid body transformation matrices according to the number of multiple point pairs.

[0027] In one embodiment, before constructing the welding trajectory registration matrix of the workpiece to be welded according to the first trajectory registration matrix and the second trajectory registration matrix, the method further includes:

[0028] Determining a loss weight of a preset registration loss function according to the first trajectory registration matrix;

[0029] Constructing a target registration loss function according to the loss weight and the preset registration loss function;

[0030] The first trajectory registration matrix is ​​iteratively updated based on the target registration loss function to obtain the second trajectory registration matrix.

[0031] In a second aspect, the present application also provides a welding trajectory generating device, comprising:

[0032] An acquisition module, used to acquire a template image and a current image taken under different conditions of the workpiece to be welded;

[0033] A conversion module, used for converting a pixel trajectory feature point set in the current image into a point cloud trajectory feature point set according to feature matching points between the template image and the current image;

[0034] A first matrix generation module, used for generating a first trajectory registration matrix of the workpiece to be welded according to the point cloud trajectory feature point set;

[0035] A construction module, configured to construct a welding trajectory registration matrix of the workpiece to be welded according to the first trajectory registration matrix and the second trajectory registration matrix, wherein the second trajectory registration matrix is ​​obtained by iteratively updating the first trajectory registration matrix;

[0036] The trajectory generation module is used to generate the welding trajectory of the workpiece to be welded according to the welding trajectory registration matrix.

[0037] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0038] A template image and a current image taken under different conditions of a workpiece to be welded are obtained; according to feature matching points between the template image and the current image, a pixel trajectory feature point set in the current image is converted into a point cloud trajectory feature point set; according to the point cloud trajectory feature point set, a first trajectory registration matrix of the workpiece to be welded is generated; according to the first trajectory registration matrix and the second trajectory registration matrix, a welding trajectory registration matrix of the workpiece to be welded is constructed, wherein the second trajectory registration matrix is ​​obtained by iterative updating based on the first trajectory registration matrix; according to the welding trajectory registration matrix, a welding trajectory of the workpiece to be welded is generated.

[0039] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0040] A template image and a current image taken under different conditions of a workpiece to be welded are obtained; according to feature matching points between the template image and the current image, a pixel trajectory feature point set in the current image is converted into a point cloud trajectory feature point set; according to the point cloud trajectory feature point set, a first trajectory registration matrix of the workpiece to be welded is generated; according to the first trajectory registration matrix and the second trajectory registration matrix, a welding trajectory registration matrix of the workpiece to be welded is constructed, wherein the second trajectory registration matrix is ​​obtained by iterative updating based on the first trajectory registration matrix; according to the welding trajectory registration matrix, a welding trajectory of the workpiece to be welded is generated.

[0041] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0042] A template image and a current image taken under different conditions of a workpiece to be welded are obtained; according to feature matching points between the template image and the current image, a pixel trajectory feature point set in the current image is converted into a point cloud trajectory feature point set; according to the point cloud trajectory feature point set, a first trajectory registration matrix of the workpiece to be welded is generated; according to the first trajectory registration matrix and the second trajectory registration matrix, a welding trajectory registration matrix of the workpiece to be welded is constructed, wherein the second trajectory registration matrix is ​​obtained by iterative updating based on the first trajectory registration matrix; according to the welding trajectory registration matrix, a welding trajectory of the workpiece to be welded is generated.

[0043] The above-mentioned welding trajectory generation method, device, computer equipment, computer-readable storage medium and computer program product first capture the template image and the current image of the workpiece to be welded under different conditions, and then use the feature matching points between the template image and the current image to convert the pixel trajectory feature point set in the current image into a point cloud trajectory feature point set, that is, through the image feature matching relationship of the workpiece to be welded under different conditions, the point cloud trajectory feature point set of the workpiece to be welded under the current conditions is converted, that is, the actual point cloud trajectory feature point set of the workpiece to be welded under the current conditions is obtained, and then the first trajectory registration matrix of the workpiece to be welded is generated through the point cloud trajectory feature point set, and the welding trajectory registration matrix of the workpiece to be welded is constructed through the first trajectory registration matrix and the second trajectory registration matrix obtained by iterative updating based on the first trajectory registration matrix, which can accurately reflect the current point cloud and the template. The actual matching situation between the plate point clouds is finally generated through the welding trajectory registration matrix to generate the welding workpiece to be welded. Since the welding trajectory registration matrix can accurately reflect the actual matching situation between the current point cloud and the template point cloud, the accuracy of feature descriptor extraction in the point cloud registration process can be ensured, and finally the generated welding trajectory meets the welding requirements of the workpiece to be welded. Therefore, the shooting conditions for the welding workpiece are in dynamic change, such as changes in the workpiece position, changes in the workpiece shooting angle, or changes in the shooting time point. Different conditions may cause misleading information to be introduced in the process of point cloud registration, that is, relying on the image obtained by shooting, it cannot reflect the actual matching situation between the current point cloud and the template point cloud, which makes it easy to fall into the optimal solution. Technical defects, therefore, improve the generation accuracy of welding trajectory generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A schematic diagram of a process of generating a welding trajectory in one embodiment;

[0046] Figure 2 A schematic diagram of a current image of a workpiece to be welded captured by a surface structured light camera in a welding trajectory generation method in an embodiment;

[0047] Figure 3 A schematic flow chart of a welding trajectory generating method in another embodiment;

[0048] Figure 4 It is an effect display diagram of the corresponding relationship of the welding trajectory generation method in another embodiment;

[0049] Figure 5 A schematic diagram of geometric constraints of point pair relationships in a welding trajectory generation method in one embodiment;

[0050] Figure 6 A schematic diagram of the point cloud registration effect of a workpiece to be welded in a welding trajectory generation method in one embodiment;

[0051] Figure 7 is a structural block diagram of a welding trajectory generating device in one embodiment;

[0052] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] First of all, it should be understood that before welding the workpiece to be welded, generating the welding trajectory of the workpiece to be welded is an indispensable step, especially for complex welding scenarios, the welding trajectory can provide a reliable basis for the welding of the workpiece, for example, taking the intersecting line welding technology as an example, where the intersecting line welding technology is a key technology for connecting complex pipes, pipe fittings and metal structures, especially suitable for occasions where pipes cross or intersect. Thanks to the high stability and precision of robot welding, at present, the intersecting line welding technology can be realized with the cooperation of robots, and the robot intersecting line welding technology mainly includes the following three categories: 1) Manual teaching: by dividing the intersecting line trajectory on the workpiece to be welded (such as pipe fittings), and manually operating the robot to program the perspective of the marked points, and using arcs to fit the intersecting line trajectory, and finally Welding is performed by a welding robot; 2) Visual recognition: The point cloud data of the workpiece to be welded (such as a pipe) is collected through sensors, and the cylindrical surface equation is fitted and the intersection line data is inferred; 3) The contour near the intersection line is scanned by a line structured light camera, and the weld points are obtained through corner point detection, and multiple frames of data are spliced ​​to form the intersection line data. In the process of generating the welding trajectory, the point cloud registration process is involved because the workpiece to be welded is under different conditions. For example, in one feasible method, the point cloud registration process is as follows: First, the workpiece point cloud is collected as a template, and the robot teaches the intersection line weld position as the template trajectory. When the workpiece to be welded moves, the transformation matrix is ​​obtained by registering the template point cloud and the point cloud after movement, and it is applied to the template trajectory, so as to generate the welding trajectory of the workpiece to be welded after the movement.

[0055] However, the above welding technologies all have their own shortcomings. For manual teaching, the disadvantage is that it is time-consuming. Each welding point needs to be taught manually, which is not only time-consuming due to the complexity of teaching programming, but also has poor flexibility in generating welding trajectories for workpieces to be welded under different conditions. For example, once the position of the workpiece to be welded changes, a new perspective is required; and for visual recognition, the welding trajectory generation effect is easily affected by environmental factors. For example, in the case of high reflection or self-occlusion of the workpiece, the point cloud information may be true or noisy, resulting in large structural fitting errors of the workpiece to be welded, such as the accuracy of intersection line fitting. Low; as for the welding trajectory generation method based on point cloud registration, although it can avoid the above problems to a certain extent, it will also lead to poor welding trajectory generation effect due to changes in the conditions of the workpiece to be welded. For example, when the workpiece to be welded is a circular pipe, the problems of reflection and scarcity of feature points will increase the probability of matching errors, thereby affecting the rough registration effect. For another example, due to changes in the position of the workpiece, the point cloud captured by the welding workpiece will partially overlap with the template point cloud, and the non-overlapping area may introduce misleading information, causing the algorithm to mistakenly match the point cloud of the non-overlapping part as the nearest neighbor point, and then fall into a local optimal solution.

[0056] In summary, there is an urgent need for a welding trajectory generation method that can improve the generation accuracy of welding trajectory generation.

[0057] In one embodiment, Figure 1 As shown, a welding trajectory generation method is provided. This embodiment takes the method applied to a terminal as an example. The terminal includes but is not limited to a personal computer, a laptop computer, a smart phone, and a tablet computer. It can be understood that after the terminal completes the generation of the welding trajectory of the workpiece to be welded, it can control the welding robot to weld the workpiece to be welded based on the welding trajectory, wherein the terminal and the welding robot are communicatively connected, and the terminal includes an acquisition module, a conversion module, a first matrix generation module, a construction module, and a trajectory generation module. The acquisition module is used to acquire a template image and a current image obtained by photographing the workpiece to be welded under different conditions. The conversion module is used to convert a pixel trajectory feature point set in the current image into a point cloud trajectory feature point set according to feature matching points between the template image and the current image. The first matrix generation module is used to generate a first trajectory registration matrix for the workpiece to be welded according to the point cloud trajectory feature point set. The construction module is used to construct a welding trajectory registration matrix for the workpiece to be welded according to the first trajectory registration matrix and the second trajectory registration matrix, wherein the second trajectory registration matrix is ​​obtained by iteratively updating the first trajectory registration matrix. The trajectory generation module is used to generate a first trajectory registration matrix for the workpiece to be welded according to the welding trajectory registration matrix. The welding trajectory of the workpiece to be welded, through the information interaction between the acquisition module, the conversion module, the first matrix generation module, the construction module and the trajectory generation module, can convert the pixel trajectory feature point set in the current image of the workpiece to be welded into a point cloud trajectory feature point set by means of the feature matching points between the template image and the current image, and can generate the first trajectory registration matrix of the workpiece to be welded through the point cloud trajectory feature point set, and then through the first trajectory registration matrix and the second trajectory registration matrix obtained by iterative update based on the first trajectory registration matrix, the welding trajectory registration matrix of the workpiece to be welded is constructed, which can accurately reflect the actual matching situation between the current point cloud and the template point cloud, and finally generate the welding workpiece to be welded through the welding trajectory registration matrix, because the welding trajectory registration matrix can accurately reflect the actual matching situation between the current point cloud and the template point cloud, and then ensure the accuracy of the extraction of the feature descriptor in the point cloud registration process, and finally achieve the purpose of the generated welding trajectory meeting the welding requirements of the workpiece to be welded, so that the generation accuracy of the welding trajectory generation can be improved. It can be understood that this embodiment can be applied to the server, and can also be applied to the system composed of the server and the terminal. In this embodiment, the method includes the following steps:

[0058] Step 202, obtaining a template image and a current image taken when the workpiece to be welded is under different conditions.

[0059] It should be noted that the workpiece to be welded refers to a component waiting for the generation of a welding trajectory, which can be specifically a pipe, a plate, a structural part or an assembly, etc. It can be understood that the generated welding trajectory of the workpiece to be welded can be applied to both welding plane welds and welding three-dimensional welds. For example, in one feasible method, the welding trajectory is used for intersecting line welding, wherein the intersecting line refers to the intersection line generated on the surface of two solids intersecting; the template image and the current image refer to two images taken under different conditions of the workpiece to be welded. For example, the template image can refer to an image taken without moving the position of the workpiece to be welded, and the current image can be an image taken after moving the position of the workpiece to be welded. For another example, the template image It may refer to an image obtained by photographing the welding workpiece at a first viewing angle, and the current image may refer to an image obtained by photographing the welding workpiece at a second viewing angle, wherein the first viewing angle and the second viewing angle are different viewing angles. For example, in an implementable manner, the first viewing angle is a main viewing angle, and the second viewing angle is a side viewing angle. For another example, the template image may refer to an image obtained by photographing the welding workpiece based on a third viewing angle at noon, and the current image may refer to an image obtained by photographing the welding workpiece based on a third viewing angle at night. It can be understood that the template image and the current image can be acquired based on an image acquisition device, and the parameters set by the image acquisition device when acquiring the template image and the current image are the same. The image acquisition device may specifically be a surface structured light camera.

[0060] It should be noted that the template image is pre-collected before the current image is collected, and the trajectory of the workpiece to be welded under the conditions corresponding to the template image is known, that is, the template image corresponds to a template welding trajectory, and the template welding trajectory is used to provide a reference for generating the current welding trajectory of the workpiece to be welded. For example, in one feasible method, a surface structured light camera can be used to photograph the workpiece to be welded after it is moved, so as to obtain the current image of the workpiece to be welded. It should be noted that the surface structured light camera can obtain images under different structured light fringes projection, and usually uses image data that is not affected by the structured light projection, for example, referring to Figure 2 , Figure 2 It is a schematic diagram showing a current image of a workpiece to be welded captured by a surface structured light camera, wherein the current image may specifically be an image obtained after grayscale processing.

[0061] As an example, step 202 includes: after detecting that the position of the workpiece to be welded moves, obtaining a current image of the workpiece to be welded by an image acquisition device, and extracting a template image of the workpiece to be welded obtained by the image acquisition device before the position of the workpiece to be welded moves.

[0062] Step 204 : converting the pixel trajectory feature point set in the current image into a point cloud trajectory feature point set according to the feature matching points between the template image and the current image.

[0063] It should be noted that the feature matching points are used to characterize the feature points that match between the template image and the current image, which can specifically be image corner points or edge points, etc. It can be understood that by obtaining the feature matching points between the template image and the current image, the geometric correspondence of the workpieces to be welded under different conditions can be accurately reflected, that is, by using a deep network for matching in the image, the technical defect of low trajectory generation accuracy caused by point pair relationship errors can be effectively avoided, so that a more accurate point pair matching relationship can be obtained, wherein the detection of feature matching points can be implemented based on the XFeat and LightGlue feature point detection matching network. For example, in an implementable method, the template image and the current image can be input into the XFeat and LightGlue feature point detection matching network together to obtain corresponding points on the image. It can be understood that the feature point matching deep network usually performs better in two-dimensional images, the corresponding points obtained have higher confidence, and the proportion of erroneous matching points is smaller.

[0064] As an example, step 204 includes: inputting the template image and the current image into the feature point matching network together, and obtaining the feature matching points between the template image and the current image through the feature point matching network detection, and establishing a point cloud conversion relationship between the current point cloud corresponding to the current image and the workpiece template point cloud corresponding to the template image through the feature matching points, and according to the point cloud conversion relationship, converting the pixel trajectory feature point set in the current image into a point cloud trajectory feature point set.

[0065] Step 206: Generate a first trajectory registration matrix of the workpiece to be welded according to the point cloud trajectory feature point set.

[0066] It should be noted that after obtaining the point cloud trajectory feature point set of the workpiece to be welded in the three-dimensional space, since the proportion of erroneous matching points of the point cloud trajectory feature points in the three-dimensional space has been reduced, the corresponding algorithm can be used to generate the registration matrix. For example, in one feasible method, the first trajectory registration algorithm is a coarse registration algorithm. After obtaining the point cloud trajectory feature point set, coarse registration can be performed based on RANSAC (RANdom SAmple Consensus) to obtain a coarse registration matrix, thereby achieving a better trajectory fitting effect.

[0067] As an example, step 206 includes: fitting the point cloud trajectory feature point set based on a preset registration algorithm to obtain a first trajectory registration matrix of the workpiece to be welded.

[0068] Step 208: construct a welding trajectory registration matrix of the workpiece to be welded according to the first trajectory registration matrix and the second trajectory registration matrix, wherein the second trajectory registration matrix is ​​obtained by iteratively updating the first trajectory registration matrix.

[0069] It should be noted that, in order to ensure the welding trajectory accuracy of the workpiece to be welded, after obtaining the first trajectory registration matrix, the first trajectory registration matrix can be used as a reference for iterative update optimization to obtain the second trajectory registration matrix. For example, in an implementable manner, assuming that the first trajectory registration matrix is ​​a coarse registration matrix and the second trajectory registration matrix is ​​a fine registration matrix, then after obtaining the coarse registration matrix After that, the precise registration is performed, that is, the Welsh loss function is introduced into the ICP algorithm, and the Welsh loss function is introduced into the objective function to obtain the second trajectory registration matrix, where the objective function is as follows:

[0070]

[0071] Among them, R is the rotation variable of the first trajectory registration matrix, t is the translation variable of the first trajectory registration matrix, is the feature point in the workpiece template point cloud, is the feature point in the point cloud trajectory feature point set. Through the above design, the second trajectory registration matrix can be accurately registered ; Further, after obtaining the first trajectory registration matrix and the second trajectory registration matrix, the welding trajectory registration matrix of the workpiece to be welded can be constructed, wherein the welding trajectory registration matrix is ​​used to convert the welding trajectory of the workpiece to be welded, wherein the construction expression of the welding trajectory registration matrix is ​​as follows:

[0072]

[0073] Where T is the welding trajectory registration matrix, is the first trajectory registration matrix, is the registration matrix for the second track.

[0074] As an example, step 208 includes: iteratively updating and optimizing the first trajectory registration matrix to obtain a second trajectory registration matrix, and constructing a welding trajectory registration matrix of the workpiece to be welded by using the first trajectory registration matrix and the second trajectory registration matrix.

[0075] Step 210: generating a welding trajectory of the workpiece to be welded according to the welding trajectory registration matrix.

[0076] As an example, step 210 includes: obtaining a welding trajectory of the workpiece to be welded by converting a welding trajectory registration matrix.

[0077] The above-mentioned welding trajectory generation method first captures the template image and the current image of the workpiece to be welded under different conditions, and then uses the feature matching points between the template image and the current image to convert the pixel trajectory feature point set in the current image into a point cloud trajectory feature point set, that is, through the image feature matching relationship of the workpiece to be welded under different conditions, the point cloud trajectory feature point set of the workpiece to be welded under the current conditions is converted, that is, the actual point cloud trajectory feature point set of the workpiece to be welded under the current conditions is obtained, and then the first trajectory registration matrix of the workpiece to be welded is generated through the point cloud trajectory feature point set, and the welding trajectory registration matrix of the workpiece to be welded is constructed through the first trajectory registration matrix and the second trajectory registration matrix obtained by iterative updating based on the first trajectory registration matrix, which can accurately reflect the actual matching situation between the current point cloud and the template point cloud. Finally, the welding workpiece to be welded is generated through the welding trajectory registration matrix. Since the welding trajectory registration matrix can accurately reflect the actual matching situation between the current point cloud and the template point cloud, the accuracy of feature descriptor extraction in the point cloud registration process can be ensured, and finally the generated welding trajectory meets the welding requirements of the workpiece to be welded. Therefore, the shooting conditions for the welding workpiece are in dynamic change, such as changes in the workpiece position, changes in the workpiece shooting angle, or changes in the shooting time point. Under different conditions, misleading information may be introduced in the process of point cloud registration, that is, relying on the image obtained by shooting, it cannot reflect the actual matching situation between the current point cloud and the template point cloud, which makes it easy to fall into the optimal solution. Technical defects, therefore, improve the generation accuracy of welding trajectory generation.

[0078] In one embodiment, Figure 3 As shown, according to the feature matching points between the template image and the current image, the pixel trajectory feature point set in the current image is converted into a point cloud trajectory feature point set, including:

[0079] Step 302: Determine a geometric transformation relationship between the template image and the current image based on feature matching points between the template image and the current image.

[0080] It should be noted that after obtaining the feature matching points between the template image and the current image, the pixel trajectory feature points in the current image can be transformed from the plane to the space, so it is first necessary to establish a geometric transformation relationship between the template image and the current image through the feature matching points, that is, the trajectory points corresponding to the pixel trajectory feature points in the current image in the template image can be determined.

[0081] As an example, step 302 includes: constructing a geometric transformation relationship between the template image and the current image based on the first coordinate information of the first feature point of the template image and the second coordinate information of the second feature point of the current image, wherein the first feature point and the second feature point constitute a feature matching point pair between the template image and the current image.

[0082] Step 304: convert and obtain the position information of the pixel trajectory feature point set in the current image according to the geometric transformation relationship.

[0083] It should be noted that after obtaining the geometric transformation relationship, since the reference pixel feature points of the pixel trajectory feature points in the pixel trajectory feature point set in the current image in the template image and the position information of the reference pixel feature points are known, the position information of any pixel trajectory feature point in the pixel trajectory feature point set in the current image can be obtained by transformation, wherein the position information of the pixel trajectory feature point can be expressed as .

[0084] As an example, step 304 includes: a selection step: selecting any pixel trajectory feature point in the pixel trajectory feature point set in the current image as the target pixel trajectory feature point, and determining the reference pixel feature point of the target pixel trajectory feature point in the template image, converting the position information of the target pixel trajectory feature point according to the geometric transformation relationship and the position information of the reference pixel feature point, returning to execute the selection step until all the pixel trajectory feature points in the pixel trajectory feature point set are replaced by the target pixel trajectory feature point, and obtaining the position information of the pixel trajectory feature point set.

[0085] Step 306 : converting the pixel trajectory feature point set into a point cloud trajectory feature point set according to the position information and the depth information of the current image.

[0086] It should be noted that the depth information can be obtained by collecting the depth sensor. For example, in one feasible method, the depth map of the current image can be collected by the depth sensor, and the depth information can be extracted. After the depth information and the position information are obtained, the three-dimensional corresponding point pair of any pixel trajectory feature point on the current image in the point cloud can be obtained. For example, in one feasible method, it is assumed that the position information of the pixel trajectory feature point of the current image is expressed as , the depth information of the pixel trajectory feature point obtained through the depth map is d, and the internal parameter matrix of the surface structured light camera is K. Then the position information of the point cloud trajectory feature point can be solved by the following expression: , thus obtaining the three-dimensional corresponding point pairs in the point cloud:

[0087] [ x y z ] = d ⋅ K − 1 [ u v 1 ]

[0088] Among them, x is the horizontal coordinate of the point cloud trajectory feature point, y is the vertical coordinate of the point cloud trajectory feature point, z is the axis coordinate of the point cloud trajectory feature point, d is the depth information, and K is the internal parameter matrix.

[0089] As an example, step 306 includes: by inputting the position information, the depth information of the current image and the internal parameter matrix of the image acquisition device into a preset conversion expression, all pixel trajectory feature points in the pixel trajectory feature point set are converted into point cloud trajectory feature points, and by fusing multiple point cloud trajectory feature points, a point cloud trajectory feature point set is obtained.

[0090] In this embodiment, the geometric transformation relationship between the template image and the current image is first determined by the feature matching points between the template image and the current image, and then the position information of the pixel trajectory feature points in the current image is obtained by using the geometric transformation relationship. Finally, based on the imaging principle of the image acquisition device and combined with the position information and depth information, the conversion of the pixel trajectory feature point set to the point cloud trajectory feature point set is completed, so that the three-dimensional corresponding point pairs in the point cloud are obtained, thus laying a foundation for improving the generation accuracy of welding trajectory generation.

[0091] In one embodiment, before converting the pixel trajectory feature point set into the point cloud trajectory feature point set according to the position information and the depth information of the current image, the method further includes:

[0092] Selection step: selecting candidate pixel trajectory feature points from the pixel trajectory feature point set; when the candidate point cloud trajectory feature points obtained by converting the candidate pixel trajectory feature points are located in the preset space, the candidate pixel trajectory feature points are used as target pixel trajectory feature points for constructing the pixel trajectory feature point set; returning to execute the selection step until all pixel trajectory feature points in the pixel trajectory feature point set are selected.

[0093] It should be noted that in actual applications, after photographing the workpiece to be welded, the current image obtained not only contains relevant feature information of the workpiece, but also background information. It is understandable that background information does not participate in the point cloud registration process. It is redundant information and will interfere with the point cloud registration process in some scenarios. Therefore, before converting to obtain point cloud trajectory feature points, it is necessary to set relevant screening conditions to remove the interference of background matching points on the point cloud registration process. For example, in an implementable method, the workpiece is usually placed on a workbench, and its position relative to the welding robot base coordinate system satisfies specific spatial constraints. Then, the straight-through filtering method can be used to determine whether the point cloud trajectory feature points converted from the pixel trajectory feature points in the pixel trajectory feature point set are located in the specified space. If not, the corresponding relationship between the pixel trajectory feature points and the point cloud trajectory feature points is deleted to eliminate background interference. For example, refer to Figure 4 , Figure 4 The following are the effect display diagrams showing the correspondence relationship, where (a) is the correspondence relationship effect display diagram without eliminating the background interference, and (b) is the correspondence relationship effect display diagram after eliminating the background interference. It can be understood that different point cloud algorithm strategies can be adopted according to different scenarios, but the core idea is still to determine the position of the workpiece through the point cloud information and remove the influence of the background matching points.

[0094] As an example, the selection step is as follows: randomly selecting candidate pixel trajectory feature points from the pixel trajectory feature point set; detecting whether the candidate point cloud trajectory feature points obtained by converting the candidate pixel trajectory feature points are located in the preset space, if the candidate point cloud trajectory feature points are detected to be located in the preset space, the candidate pixel trajectory feature points are used as the target pixel trajectory feature points for constructing the pixel trajectory feature point set, if the candidate point cloud trajectory feature points are detected not to be located in the preset space, the candidate pixel trajectory feature points are eliminated; returning to execute the selection step until all pixel trajectory feature points in the pixel trajectory feature point set are selected, wherein the target pixel trajectory feature points for constructing the pixel trajectory feature point set can be one or more. Since the target pixel trajectory feature points for constructing the pixel trajectory feature point set are all located in the preset space, that is, the target pixel trajectory feature points reflect the workpiece features, so that the interference of the background information in the current image can be eliminated before the conversion, so the foundation is further laid for improving the generation accuracy of the welding trajectory generation.

[0095] In one embodiment, according to the position information and the depth information of the current image, the pixel trajectory feature point set is converted into a point cloud trajectory feature point set, including:

[0096] According to the position information and the depth information of the current image, the pixel trajectory feature point set is converted into a candidate point cloud trajectory feature point set, wherein the candidate point cloud trajectory feature point set includes multiple candidate point cloud trajectory feature points; according to preset geometric constraints, the point pair relationships of the multiple candidate point cloud trajectory feature points are screened to obtain multiple candidate point pair relationships; the pixel trajectory feature points corresponding to the multiple candidate point pair relationships are converted to obtain the point cloud trajectory feature point set.

[0097] It should be noted that under certain extreme lighting conditions, the performance of the feature point matching network may also be affected, which will lead to a decrease in the confidence of the corresponding points in the image and an increase in the proportion of incorrect matching points. That is, relying on feature matching points for 3D point cloud conversion will face challenges. For example, after the conversion to the 3D point cloud, the proportion of incorrect point pairs is high, which will ultimately lead to poor registration results for coarse registration based directly on RANSAC. To solve the above problems, geometric constraint relationships can be pre-set to screen point pair relationships to remove incorrect matching points. The preset geometric constraint relationships can specifically be distance constraint relationships and angle constraint relationships. For example, in one feasible method, refer to Figure 5 , Figure 5 is a schematic diagram of geometric constraints of point pair relationships, where and is the point pair correspondence, It refers to the distance between the points in the matching point pair. The nearest point, It is the distance The second closest point, and The distance points are The closest and second closest points, then and For the correct correspondence, the following formula will be established:

[0098]

[0099] in, is the L2 norm, is the distance threshold. The constraint of the above formula indicates that the distances between the two pairs of correct corresponding points in the point cloud to be registered and in the target point cloud should be consistent. If the above formula does not hold, it means that at least one pair of relationships does not hold. Then the judgment basis of the following formula is used to filter out the wrong corresponding relationships. is the distance ratio threshold. If the following formula is not satisfied, the corresponding relationship will be deleted.

[0100]

[0101] ; In addition to the above distance constraints, the angle between normals also satisfies the geometric constraints, where and Decibels per point and The normal vector of and Points and If the normal vector and For the correct correspondence, the following expression will be established:

[0102]

[0103] in, is the angle threshold. The above formula shows that after the rigid transformation, the angle between the normal vectors of the two points should remain unchanged. If the above formula is not satisfied, the two pairs of corresponding relationships are screened out. It can be understood that after the wrong relationship between the point pairs is screened out, the RANSAC-based rough registration is performed on the remaining point pairs. The rough registration effect is as follows: Figure 6 As shown, Figure 6 A schematic diagram showing the point cloud registration effect of the workpiece to be welded.

[0104] As an example, it is converted into a candidate point cloud trajectory feature point set, wherein the candidate point cloud trajectory feature point set includes multiple candidate point cloud trajectory feature points; according to preset geometric constraints, the point pair relationships of multiple candidate point cloud trajectory feature points are screened to obtain multiple candidate point pair relationships; the pixel trajectory feature points corresponding to the multiple candidate point pair relationships are converted to obtain a point cloud trajectory feature point set. This embodiment sets geometric constraints to screen the point pair relationships of multiple candidate point cloud trajectory feature points in the candidate point cloud trajectory feature point set, thereby screening multiple candidate point pair relationships that meet the preset geometric constraints, and finally relies on multiple candidate point pair relationships to convert pixel trajectory feature points, thereby obtaining a point cloud trajectory feature point set for generating a first trajectory registration matrix, which can ensure the robustness of the coarse registration, and even in the absence of point clouds, can also obtain better registration results, so as to further lay the foundation for improving the generation accuracy of welding trajectory generation.

[0105] In one embodiment, generating a first trajectory registration matrix of the workpiece to be welded according to the point cloud trajectory feature point set includes:

[0106] Division step: dividing the point cloud trajectory feature point set into multiple first point cloud trajectory feature point pairs and multiple second point cloud trajectory feature point pairs; counting the number of point pairs in the multiple second point cloud trajectory feature point pairs as target point cloud trajectory feature point pairs according to the rigid body transformation matrix constructed by the multiple first point cloud trajectory feature point pairs; returning to execute the division step until the number of divisions of the point cloud trajectory feature point set reaches a preset division threshold, and selecting the first trajectory registration matrix of the workpiece to be welded from the multiple rigid body transformation matrices according to the number of multiple point pairs.

[0107] As an example, in the division step, the point cloud trajectory feature points are randomly divided into first point cloud trajectory feature point pairs and second point cloud trajectory feature point pairs; based on multiple first point cloud trajectory feature point pairs, the rigid body transformation matrix is ​​solved, and the number of target point cloud trajectory feature point pairs whose distance errors are less than a set threshold under the action of the rigid body transformation matrix are calculated; the division step is returned to be executed until the number of divisions of the point cloud trajectory feature point set reaches a preset division threshold, and the rigid body transformation matrix corresponding to the number of point pairs with the largest value among the multiple point pairs is used as the first trajectory registration matrix of the workpiece to be welded.

[0108] In an practicable manner, the registration process for rough registration of the welding workpiece is as follows: 1) randomly selecting three first point cloud trajectory feature point pairs from the point cloud trajectory feature point set, and using the selected three first point cloud trajectory feature point pairs to perform a rigid body transformation matrix ; 2) Calculate the second point cloud trajectory feature point pair in addition to the three first point cloud trajectory feature point pairs in the point cloud trajectory feature point set, and in the obtained rigid body transformation matrix , and when the distance error is less than the set threshold, the second point cloud trajectory feature point pair is used as the sample inliers, otherwise it is used as the sample outliers, and the number of point pairs of point cloud trajectory feature points as sample inliers is counted; 3) Repeat the above steps until the upper limit of the number of iterative updates is reached, count the number of sample inliers under different rigid body transformation models, and take the one with the largest number of sample inliers as the optimal mathematical model input, and then perform point cloud registration operations, and use singular value decomposition to solve the transformation matrix T.

[0109] In this embodiment, a plurality of first point cloud trajectory feature point pairs and a plurality of second point cloud trajectory feature point pairs are obtained by dividing the point cloud trajectory feature point set, and then the rigid body transformation matrix is ​​solved by the plurality of first point cloud trajectory feature point pairs, and the number of point pairs in the plurality of second point cloud trajectory feature point pairs as target point cloud trajectory feature point pairs is counted by using the rigid body transformation matrix, and finally the rigid body transformation matrix corresponding to the number of point pairs with the largest value among the number of point pairs is used as the first trajectory registration matrix of the workpiece to be welded, so as to ensure that the registration of the first trajectory registration matrix is ​​completed with as many point cloud trajectory feature point pairs as possible, thus laying a foundation for improving the coarse registration accuracy of the workpiece to be welded.

[0110] In one embodiment, before constructing a welding trajectory registration matrix of the workpieces to be welded according to the first trajectory registration matrix and the second trajectory registration matrix, the method further includes:

[0111] According to the first track registration matrix, the loss weight of the preset registration loss function is determined; according to the loss weight and the preset registration loss function, the target registration loss function is constructed; based on the target registration loss function, the first track registration matrix is ​​iteratively updated to obtain the second track registration matrix.

[0112] It should be noted that, in the process of solving the second trajectory registration matrix, if the minimization loss function assigns a smaller weight to the data points that are farther away, that is, the loss weight is matched to the preset registration loss function, the influence of abnormal data points can be effectively reduced, thereby improving the robustness of the algorithm. For example, in an implementable manner, the loss weight of the preset registration loss function is as follows:

[0113]

[0114] Among them, R is the rotation variable of the first trajectory registration matrix, t is the translation variable of the first trajectory registration matrix, is a point in the workpiece template point cloud, is the point in the target point cloud, v is the width of the Welsch function, is the loss weight; in addition, the expression of the target registration loss function is as follows:

[0115]

[0116] R is the rotation variable of the first trajectory registration matrix, t is the translation variable of the first trajectory registration matrix, is the feature point in the workpiece template point cloud, is the feature point in the point cloud trajectory feature point set, is the loss weight. Through the above design, the second trajectory registration matrix can be accurately registered .

[0117] As an example, based on the first trajectory registration matrix, the loss weight of the preset registration loss function is constructed; the target registration loss function is jointly constructed through the loss weight and the preset registration loss function; the first trajectory registration matrix is ​​iteratively updated based on the target registration loss function to obtain the second trajectory registration matrix. This embodiment can further effectively reduce the influence of abnormal data points by assigning loss weights to the preset registration loss function, thereby improving the robustness of the algorithm. Therefore, the acquisition accuracy of the second trajectory registration matrix is ​​improved, further laying a foundation for improving the generation accuracy of welding trajectory generation.

[0118] Since the welding trajectory registration matrix can accurately reflect the actual matching situation between the current point cloud and the template point cloud, it can ensure the accuracy of feature descriptor extraction in the point cloud registration process, and finally achieve the purpose of the generated welding trajectory meeting the welding requirements of the workpiece to be welded. Therefore, it overcomes the problem that the shooting conditions for the welding workpiece are in dynamic change, such as changes in the workpiece position, changes in the workpiece shooting angle, or changes in the shooting time point. Under different conditions, misleading information may be introduced in the point cloud registration process, that is, relying on the image obtained by shooting to reflect the actual matching situation between the current point cloud and the template point cloud, which makes it easy to fall into the optimal solution. Technical defects, therefore, improve the generation accuracy of welding trajectory generation.

[0119] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0120] Based on the same inventive concept, the embodiment of the present application also provides a welding trajectory generation device for implementing the welding trajectory generation method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more welding trajectory generation device embodiments provided below can refer to the limitations of the welding trajectory generation method above, and will not be repeated here.

[0121] In an exemplary embodiment, Figure 7 As shown, a welding trajectory generation device is provided, including: an acquisition module 401, a conversion module 402, a first matrix generation module 403, a construction module 404 and a trajectory generation module 405, wherein:

[0122] An acquisition module 401 is used to acquire a template image and a current image taken under different conditions of the workpiece to be welded;

[0123] A conversion module 402, configured to convert a pixel trajectory feature point set in the current image into a point cloud trajectory feature point set according to feature matching points between the template image and the current image;

[0124] A first matrix generation module 403, used to generate a first trajectory registration matrix of the workpiece to be welded according to the point cloud trajectory feature point set;

[0125] A construction module 404 is used to construct a welding trajectory registration matrix of the workpiece to be welded according to the first trajectory registration matrix and the second trajectory registration matrix, wherein the second trajectory registration matrix is ​​obtained by iteratively updating the first trajectory registration matrix;

[0126] The trajectory generation module 405 is used to generate the welding trajectory of the workpiece to be welded according to the welding trajectory registration matrix.

[0127] In one embodiment, the conversion module 402 is further configured to:

[0128] According to the feature matching points between the template image and the current image, a geometric transformation relationship between the template image and the current image is determined; according to the geometric transformation relationship, position information of a pixel trajectory feature point set in the current image is converted; according to the position information and depth information of the current image, the pixel trajectory feature point set is converted into a point cloud trajectory feature point set.

[0129] In one embodiment, the welding trajectory generating device further includes a selection module, and the selection module is further used to:

[0130] A selection step: selecting a candidate pixel trajectory feature point from the pixel trajectory feature point set; when the candidate point cloud trajectory feature point converted from the candidate pixel trajectory feature point is located in a preset space, using the candidate pixel trajectory feature point as a target pixel trajectory feature point for constructing the pixel trajectory feature point set; returning to execute the selection step until all pixel trajectory feature points in the pixel trajectory feature point set are selected.

[0131] In one embodiment, the conversion module 402 is further configured to:

[0132] According to the position information and the depth information of the current image, the pixel trajectory feature point set is converted into a candidate point cloud trajectory feature point set, wherein the candidate point cloud trajectory feature point set includes multiple candidate point cloud trajectory feature points; according to preset geometric constraints, the point pair relationships of the multiple candidate point cloud trajectory feature points are screened to obtain multiple candidate point pair relationships; the pixel trajectory feature points corresponding to the multiple candidate point pair relationships are converted to obtain the point cloud trajectory feature point set.

[0133] In one embodiment, the first matrix generating module 403 is further used for:

[0134] The division step includes dividing the point cloud trajectory feature point set into a plurality of first point cloud trajectory feature point pairs and a plurality of second point cloud trajectory feature point pairs; counting the number of point pairs in the plurality of second point cloud trajectory feature point pairs as target point cloud trajectory feature point pairs according to the rigid body transformation matrix constructed by the plurality of first point cloud trajectory feature point pairs; returning to execute the division step until the number of divisions of the point cloud trajectory feature point set reaches a preset division threshold, and selecting the first trajectory registration matrix of the workpiece to be welded from the plurality of rigid body transformation matrices according to the number of the plurality of point pairs.

[0135] In one embodiment, the welding trajectory generating device further includes a second matrix generating module, and the second matrix generating module is further used for:

[0136] According to the first trajectory registration matrix, a loss weight of a preset registration loss function is determined; according to the loss weight and the preset registration loss function, a target registration loss function is constructed; based on the target registration loss function, the first trajectory registration matrix is ​​iteratively updated to obtain the second trajectory registration matrix.

[0137] Each module in the welding trajectory generating device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0138] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a welding trajectory generation method is implemented. Those skilled in the art can understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0139] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0140] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0141] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0142] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0143] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0144] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A welding trajectory generation method, characterized in that: The method comprises: Acquire a template image and a current image of the workpiece to be welded taken under different conditions; According to the feature matching points between the template image and the current image, a pixel trajectory feature point set in the current image is converted into a point cloud trajectory feature point set; Generating a first trajectory registration matrix of the workpiece to be welded according to the point cloud trajectory feature point set; Constructing a welding trajectory registration matrix of the workpiece to be welded according to the first trajectory registration matrix and the second trajectory registration matrix, wherein the second trajectory registration matrix is ​​obtained by iteratively updating the first trajectory registration matrix; The welding trajectory of the workpiece to be welded is generated according to the welding trajectory registration matrix.

2. The method according to claim 1, characterized in that The step of converting a pixel trajectory feature point set in the current image into a point cloud trajectory feature point set according to feature matching points between the template image and the current image comprises: Determining a geometric transformation relationship between the template image and the current image according to feature matching points between the template image and the current image; According to the geometric transformation relationship, the position information of the pixel trajectory feature point set in the current image is obtained by transformation; The pixel trajectory feature point set is converted into a point cloud trajectory feature point set according to the position information and the depth information of the current image.

3. The method according to claim 2, characterized in that Before converting the pixel trajectory feature point set into a point cloud trajectory feature point set according to the position information and the depth information of the current image, the method further includes: Selection step: selecting candidate pixel trajectory feature points from the pixel trajectory feature point set; In the case where the candidate point cloud trajectory feature points obtained by converting the candidate pixel trajectory feature points are located in a preset space, the candidate pixel trajectory feature points are used as target pixel trajectory feature points for constructing the pixel trajectory feature point set; Return to execute the selection step until all pixel trajectory feature points in the pixel trajectory feature point set are selected.

4. The method according to claim 2, characterized in that: The step of converting the pixel trajectory feature point set into a point cloud trajectory feature point set according to the position information and the depth information of the current image includes: According to the position information and the depth information of the current image, the pixel trajectory feature point set is converted into a candidate point cloud trajectory feature point set, wherein the candidate point cloud trajectory feature point set includes a plurality of candidate point cloud trajectory feature points; According to preset geometric constraints, the point pair relationships of multiple candidate point cloud trajectory feature points are screened to obtain multiple candidate point pair relationships; The pixel trajectory feature points corresponding to the plurality of candidate point pairs are converted to obtain the point cloud trajectory feature point set.

5. The method according to claim 1, characterized in that The step of generating a first trajectory registration matrix of the workpiece to be welded according to the point cloud trajectory feature point set comprises: Dividing step: dividing the point cloud trajectory feature point set into a plurality of first point cloud trajectory feature point pairs and a plurality of second point cloud trajectory feature point pairs; According to the rigid body transformation matrix constructed by the plurality of first point cloud trajectory feature point pairs, counting the number of point pairs among the plurality of second point cloud trajectory feature point pairs as target point cloud trajectory feature point pairs; Return to execute the division step until the number of divisions of the point cloud trajectory feature point set reaches a preset division threshold, and select the first trajectory registration matrix of the workpiece to be welded from multiple rigid body transformation matrices according to the number of multiple point pairs.

6. The method according to claim 1, characterized in that Before constructing the welding trajectory registration matrix of the workpiece to be welded according to the first trajectory registration matrix and the second trajectory registration matrix, the method further includes: Determining a loss weight of a preset registration loss function according to the first trajectory registration matrix; Constructing a target registration loss function according to the loss weight and the preset registration loss function; The first trajectory registration matrix is ​​iteratively updated based on the target registration loss function to obtain the second trajectory registration matrix.

7. A welding trajectory generating device, characterized in that: The device comprises: An acquisition module, used to acquire a template image and a current image taken under different conditions of the workpiece to be welded; A conversion module, configured to convert a pixel trajectory feature point set in the current image into a point cloud trajectory feature point set according to feature matching points between the template image and the current image; A first matrix generation module, used for generating a first trajectory registration matrix of the workpiece to be welded according to the point cloud trajectory feature point set; A construction module, configured to construct a welding trajectory registration matrix of the workpiece to be welded according to the first trajectory registration matrix and the second trajectory registration matrix, wherein the second trajectory registration matrix is ​​obtained by iteratively updating the first trajectory registration matrix; The trajectory generation module is used to generate the welding trajectory of the workpiece to be welded according to the welding trajectory registration matrix.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images