Model-free weld joint identification method and device

Abstract

The embodiment of the invention provides a model-free weld joint identification method and device, and relates to the technical field of computers. The method comprises the following steps: acquiring a workpiece point cloud of a to-be-identified workpiece; a stored first conversion relation between a first coordinate system where calibration point clouds are located and a second coordinate system where workpiece point clouds are located is obtained through registration calculation, the calibration point clouds are obtained by splicing a plurality of point clouds obtained by conducting point cloud collection on a calibration workpiece, and the calibration workpiece is a workpiece of the same type as the to-be-recognized workpiece; according to the first conversion relation, first welding seam labeling information corresponding to the stored calibration point cloud is converted to a target coordinate system used by the workpiece point cloud, second welding seam labeling information is obtained, and the first welding seam labeling information is used for indicating the position of a welding seam on the calibration workpiece in the first coordinate system; and performing welding seam identification on the workpiece point cloud according to the second welding seam marking information. In this way, weld joint recognition can be achieved without a workpiece model.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model-free weld identification method and device. Background Art

[0002] The current weld recognition algorithm requires the use of workpiece model information. Generally speaking, the workpiece is processed according to the IGES file of the workpiece model. When recognizing welds, it is necessary to perform surface sampling on the IGES file of the workpiece model to obtain the workpiece model point cloud, which can be called point cloud P. When actually recognizing welds, it is necessary to use a 3D camera to take pictures of the workpiece to obtain the collected workpiece point cloud, which is called point cloud Q. Use the point cloud registration algorithm to align point cloud P and point cloud Q, and obtain the rotation matrix R and translation vector T from the coordinate system where point cloud P is located (workpiece model coordinate system) to the coordinate system where point cloud Q is located (robot base coordinate system), and then use the information of R and T and the weld information corresponding to the workpiece model to identify the weld from point cloud Q.

[0003] It can be seen that the above weld recognition algorithm must use the information of the workpiece model point cloud to align the workpiece model point cloud with the collected workpiece point cloud. If there is a workpiece without a corresponding model, the above weld recognition algorithm cannot directly perform weld recognition, that is, the above weld recognition algorithm must have a workpiece model to perform weld recognition on the workpiece. Since some workpieces do not have a corresponding workpiece model when they are processed, the above algorithm will not be able to perform weld recognition at this time, and thus the subsequent welding process based on weld recognition cannot be carried out. Summary of the invention

[0004] The embodiments of the present application provide a model-free weld recognition method, device, electronic device and readable storage medium, which can realize weld recognition without a workpiece model.

[0005] The embodiments of the present application can be implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a model-free weld recognition method, the method comprising:

[0007] Obtaining a workpiece point cloud of a workpiece to be identified;

[0008] A first conversion relationship between a first coordinate system where a calibration point cloud is stored and a second coordinate system where the workpiece point cloud is stored is obtained by registering and calculating, wherein the calibration point cloud is obtained by splicing multiple point clouds obtained by performing point cloud acquisition on the calibration workpiece, and the calibration workpiece is a workpiece of the same type as the workpiece to be identified;

[0009] According to the first conversion relationship, the first weld annotation information corresponding to the saved calibration point cloud is converted to the second coordinate system to obtain second weld annotation information, wherein the first weld annotation information is used to indicate the position of the weld on the calibration workpiece in the first coordinate system;

[0010] The weld is identified on the workpiece point cloud according to the second weld annotation information.

[0011] In a second aspect, an embodiment of the present application provides a model-free weld identification device, the device comprising:

[0012] A workpiece point cloud acquisition module is used to obtain a workpiece point cloud of a workpiece to be identified;

[0013] A registration module, used for obtaining, by registration calculation, a first transformation relationship between a first coordinate system where a calibration point cloud is stored and a second coordinate system where the workpiece point cloud is stored, wherein the calibration point cloud is obtained by splicing a plurality of point clouds obtained by performing point cloud acquisition on a calibration workpiece, and the calibration workpiece is a workpiece of the same type as the workpiece to be identified;

[0014] a conversion module, configured to convert the first weld annotation information corresponding to the saved calibration point cloud into the second coordinate system according to the first conversion relationship, so as to obtain second weld annotation information, wherein the first weld annotation information is used to indicate the position of the weld on the calibration workpiece in the first coordinate system;

[0015] An identification module is used to perform weld identification on the workpiece point cloud according to the second weld annotation information.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor can execute the machine executable instructions to implement the model-free weld identification method described in the aforementioned embodiment.

[0017] In a fourth aspect, an embodiment of the present application provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the model-free weld identification method as described in the aforementioned embodiment is implemented.

[0018] The model-free weld recognition method, device, electronic device and readable storage medium provided in the embodiments of the present application first obtain the workpiece point cloud of the workpiece to be identified, then obtain the first conversion relationship between the first coordinate system where the saved calibration point cloud is located and the second coordinate system where the workpiece point cloud is located through registration, and then convert the first weld annotation information corresponding to the saved calibration point cloud to the second coordinate system based on the first conversion relationship to obtain the second weld annotation information, and finally perform weld recognition on the workpiece point cloud according to the second weld annotation information. Among them, the above-mentioned calibration point cloud is obtained by splicing multiple point clouds obtained by point cloud collection of the calibration workpiece, the calibration workpiece is a workpiece of the same type as the workpiece to be identified, and the first weld annotation information is used to indicate the position of the weld on the calibration workpiece in the first coordinate system. In this way, weld recognition can be achieved without a workpiece model. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A block diagram of an electronic device provided in an embodiment of the present application;

[0021] Figure 2 One of the flow diagrams of the model-free weld identification method provided in the embodiment of the present application;

[0022] Figure 3 A second flow chart of a model-free weld identification method provided in an embodiment of the present application;

[0023] Figure 4 for Figure 3 A schematic flow chart of the sub-steps included in step S130;

[0024] Figure 5 A third flow chart of a model-free weld identification method provided in an embodiment of the present application;

[0025] Figure 6 for Figure 2 One of the flowchart diagrams of the sub-steps included in step S180;

[0026] Figure 7 for Figure 6 A schematic flow chart of the sub-steps included in sub-step S182;

[0027] Figure 8 for Figure 7A schematic flow chart of the sub-steps included in sub-step S1822;

[0028] Fig. 9 for Figure 2 2 is a flow chart of the sub-steps included in step S180;

[0029] Fig.10 for Fig. 9 A schematic flow chart of the sub-steps included in sub-step S185;

[0030] Fig.11 One of the schematic block diagrams of the model-free weld identification device provided in the embodiment of the present application;

[0031] Fig.12 The second block diagram of the model-free weld identification device provided in the embodiment of the present application.

[0032] Icons: 100 - electronic device; 110 - memory; 120 - processor; 130 - communication unit; 200 - model-free weld recognition device; 210 - calibration point cloud acquisition module; 220 - annotation module; 230 - workpiece point cloud acquisition module; 240 - registration module; 250 - conversion module; 260 - recognition module. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0034] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0035] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0036] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0037] Please refer to Figure 1 , Figure 1 A block diagram of an electronic device 100 provided in an embodiment of the present application. The electronic device 100 may be, but is not limited to, a control unit inside a welding robot, a control device corresponding to the welding robot, etc. The electronic device 100 may include a memory 110, a processor 120, and a communication unit 130. The memory 110, the processor 120, and the communication unit 130 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0038] The memory 110 is used to store programs or data. The memory 110 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc.

[0039] The processor 120 is used to read / write data or programs stored in the memory 110 and execute corresponding functions. For example, the memory 110 stores a model-free weld identification device 200, and the model-free weld identification device 200 includes at least one software function module that can be stored in the memory 110 in the form of software or firmware. The processor 120 executes various functional applications and data processing by running software programs and modules stored in the memory 110, such as the model-free weld identification device 200 in the embodiment of the present application, that is, implementing the model-free weld identification method in the embodiment of the present application.

[0040] The communication unit 130 is used to establish a communication connection between the electronic device 100 and other communication terminals through a network, and to send and receive data through the network. For example, when the electronic device 100 is a control device for controlling a welding robot, the electronic device 100 can send the identified weld position to the welding robot so that the welding robot can perform automatic welding.

[0041] It should be understood that Figure 1 The structure shown is only a schematic diagram of the structure of the electronic device 100. The electronic device 100 may also include Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0042] Please refer to Figure 2 , Figure 2 One of the flow diagrams of the model-free weld recognition method provided in the embodiment of the present application. The method can be applied to the above-mentioned electronic device 100. The specific process of the model-free weld recognition method is described in detail below. In this embodiment, the model-free weld recognition method may include steps S150 to S180.

[0043] Step S150, obtaining a workpiece point cloud of the workpiece to be identified.

[0044] In this embodiment, the workpiece to be identified is the workpiece that currently needs weld identification. The workpiece point cloud of the workpiece to be identified can be obtained in any manner. Optionally, the workpiece point cloud can be a frame point cloud, that is, a local point cloud of the workpiece to be identified; or, the workpiece point cloud can be a point cloud obtained by splicing multiple frames of point clouds, that is, the global point cloud to be identified, which can be determined in combination with actual needs.

[0045] Since the same structure may appear at different positions on the workpiece, in order to avoid inaccurate identification of welds due to subsequent registration errors, as a possible implementation, the workpiece point cloud is a global point cloud of the workpiece to be identified.

[0046] Step S160, obtaining, by registration calculation, a first conversion relationship between a first coordinate system where the saved calibration point cloud is located and a second coordinate system where the workpiece point cloud is located.

[0047] Step S170: According to the first conversion relationship, the first weld annotation information corresponding to the saved calibration point cloud is converted to the second coordinate system to obtain second weld annotation information.

[0048] In this embodiment, the electronic device may store a calibration point cloud and the first weld annotation information corresponding to the calibration point cloud. The calibration point cloud is a global point cloud of the calibration workpiece, and the calibration point cloud is obtained by splicing multiple point clouds obtained by performing point cloud acquisition on the calibration workpiece. The calibration workpiece is a workpiece of the same type as the workpiece to be identified. The calibration workpiece is a workpiece used for weld annotation, and the workpiece to be identified is a workpiece that needs to be weld identified. The calibration workpiece and the workpiece to be identified are workpieces with the same structure. It can be understood that there may be a certain processing error between the calibration workpiece and the workpiece to be identified. The first weld annotation information is used to indicate the position of the weld on the calibration workpiece in the first coordinate system, and a first weld annotation information is used to indicate a weld. The number of first weld annotation information corresponding to the calibration point cloud can be determined in combination with actual needs. The point cloud acquisition device used for point cloud acquisition to obtain the calibration point cloud and the workpiece point cloud can be, but not limited to, a 3D camera, a radar, etc., and the point cloud acquisition device can be set at the end of the robot.

[0049] The calibration point cloud and the corresponding first weld annotation information can be obtained by other devices through certain processing in advance, or can be obtained by the electronic device collecting point clouds and taking corresponding annotation measures on the calibration workpiece in advance. The specific acquisition method can be determined in combination with actual needs. It can be understood that when the calibration point cloud is obtained through a global scan and the first weld annotation information is obtained through a weld annotation, weld identification can be performed directly based on the calibration point cloud and the first weld annotation information for other workpieces of the same structure; that is, for each type of workpiece, the calibration workpiece needs to be processed to obtain the calibration point cloud and the first weld annotation information. For the same type of workpiece, this step only needs to be performed once, and other workpieces of the same type can realize weld identification by using the global point cloud and weld annotation of the calibration workpiece.

[0050] The calibration point cloud and the workpiece point cloud may be point cloud registered to obtain a conversion relationship between a first coordinate system used by the calibration point cloud and a second coordinate system used by the workpiece point cloud as a first conversion relationship. The first conversion relationship may include a rotation matrix R and a translation vector T. It is understood that the annotation information used by the first weld annotation information is the first coordinate system used by the calibration point cloud. After obtaining the first conversion relationship, the first weld annotation information is converted according to the first conversion relationship, that is, the coordinate system used by the first weld annotation information is converted from the first coordinate system used by the calibration point cloud to the second coordinate system used by the workpiece point cloud, thereby obtaining the second weld annotation information. The coordinate system used by the second weld annotation information is the second coordinate system, that is, the coordinate system used by the position indicated by the second weld annotation information is the second coordinate system. Among them, the first coordinate system and the second coordinate system may be a robot base coordinate system or other coordinate systems, which may be determined in combination with actual needs; it is understood that the first coordinate system and the second coordinate system may be the base coordinate system of the same robot or the base coordinate system of different robots, which may be determined according to actual conditions.

[0051] Step S180: performing weld identification on the workpiece point cloud according to the second weld annotation information.

[0052] When the second weld annotation information is obtained, each second weld annotation information can be used as a reference to perform weld identification in the workpiece point cloud, thereby determining the target weld corresponding to each second weld annotation information from the workpiece point cloud.

[0053] The model-free weld recognition method provided in this embodiment can realize weld recognition without a workpiece model. For different types of workpieces, it is only necessary to perform a global scan and weld point marking on the calibration workpiece once, and then weld recognition can be performed on all subsequent new workpieces of the same type. Since a global scan and weld point marking do not require a lot of manpower and time, the model-free weld recognition method proposed in this solution is a better weld recognition solution.

[0054] Please refer to Figure 3 , Figure 3 The second flow chart of the model-free weld recognition method provided in the embodiment of the present application. In this embodiment, before step S160, the method may further include steps S110 to S130 for obtaining the calibration point cloud.

[0055] Step S110, obtaining a plurality of initial point cloud frames by performing point cloud acquisition on the calibration workpiece at different postures using a first point cloud acquisition device.

[0056] Step S120: convert the multiple initial point cloud frames into the first coordinate system according to the calibration matrix corresponding to the first coordinate system obtained in advance, so as to obtain multiple target point cloud frames.

[0057] Step S130 , stitching the multiple target point cloud frames frame by frame according to the point cloud acquisition order to obtain the calibration point cloud.

[0058] In this embodiment, the welding robot can be in the eyes-on-hand mode, that is, the first point cloud acquisition device is located at the end of the welding robot. The first point cloud acquisition device can be controlled to acquire the point cloud of the calibration workpiece in different positions, and the acquired point cloud is the multiple initial point cloud frames, and the multiple initial point cloud frames are the multiple point clouds used to obtain the calibration point cloud by splicing. Optionally, shooting at more angles can be performed to ensure the accuracy of the calibration point cloud obtained subsequently.

[0059] According to the calibration matrix corresponding to the first coordinate system obtained in advance, the multiple initial point cloud frames can be respectively converted to the first coordinate system to obtain the corresponding multiple target point cloud frames. It can be understood that one frame of initial point cloud frame corresponds to one frame of target point cloud frame. Optionally, the first coordinate system can be the robot base coordinate system of the welding robot where the first point cloud acquisition device is located. In this case, the calibration matrix corresponding to the first coordinate system can be the hand-eye calibration matrix corresponding to the welding robot. The above-mentioned multiple initial point cloud frames can be converted to the robot base coordinate system using the hand-eye calibration matrix and the TCP (tool center point) posture corresponding to each initial point cloud frame to obtain multiple frames of target point cloud frames.

[0060] In this embodiment, the acquisition areas corresponding to two adjacent frames of point clouds acquired by the first point cloud acquisition device overlap. For example, the first point cloud acquisition device performs point cloud acquisition in pose 0 and pose 1 in sequence, and the acquisition area of ​​the first point cloud acquisition device at pose 0 overlaps with the acquisition area at pose 1, which facilitates the subsequent splicing according to the point cloud acquisition sequence to obtain the calibration point cloud.

[0061] Optionally, the plurality of target point cloud frames may be spliced ​​frame by frame sequentially or frame by frame in reverse order according to the point cloud acquisition order, so as to obtain the calibration point cloud.

[0062] To ensure the splicing effect, Figure 4 The calibration point cloud is obtained by stitching in the manner shown. Figure 4 , Figure 4 for Figure 3 Schematic diagram of the flow of sub-steps included in step S130. In this embodiment, step S130 may include sub-steps S131 to S133.

[0063] Sub-step S131 , for the first point cloud and the second point cloud to be spliced ​​this time, obtain a second transformation relationship between the coordinate systems of the first point cloud and the second point cloud through registration.

[0064] Sub-step S132: using the second transformation relationship to transform the second point cloud to obtain a second point cloud to be spliced.

[0065] Sub-step S133: determining a point cloud to be added according to the second point cloud to be spliced ​​and the first point cloud, and adding the point cloud to be added to the first point cloud.

[0066] In this embodiment, during the stitching process, the first point cloud and the second point cloud to be stitched are first determined. In the first stitching, the first point cloud and the second point cloud are two frames of point clouds collected sequentially; when it is not the first stitching, the first point cloud is the stitching result of the previous stitching, and the second point cloud is a frame of point cloud to be stitched determined according to the point cloud acquisition sequence.

[0067] That is, when stitching sequentially based on the acquisition order, during the first stitching, the first point cloud and the second point cloud are the target point cloud frames corresponding to the first frame of initial point cloud frame and the second frame of initial point cloud frame that were first acquired; when it is not the first stitching, the first point cloud is the stitching result of the last stitching, and the second point cloud is the first frame of target point cloud frame b following the target point cloud frame a used in the last stitching, and the acquisition time t1 corresponding to the target point cloud frame a used in the last stitching is adjacent to the acquisition time t2 corresponding to the target point cloud frame b, and t1 is earlier than t2.

[0068] When stitching in reverse order based on the acquisition order, during the first stitching, the first point cloud and the second point cloud are target point cloud frames corresponding to the last two frames of initial point cloud frames acquired; when it is not the first stitching, the first point cloud is the stitching result of the last stitching, and the second point cloud is the first frame of target point cloud frame d before the target point cloud frame c used in the last stitching, and the acquisition time t3 corresponding to the target point cloud frame c used in the last stitching is adjacent to the acquisition time t4 corresponding to the target point cloud frame d, and t3 is later than t4.

[0069] When the first point cloud and the second point cloud are determined, the first point cloud and the second point cloud can be registered to obtain a second conversion relationship between the coordinate system of the first point cloud and the coordinate system of the second point cloud. After that, the second point cloud is converted using the second conversion relationship to obtain a second point cloud to be spliced. Next, the first point cloud and the second point cloud to be spliced ​​can be compared to determine the point cloud to be added. The point cloud to be added is a point cloud that is not included in the first point cloud and included in the second point cloud to be spliced; that is, if the position of a point p in the second point cloud to be spliced ​​is different from the position of each point in the first point cloud, then the point p is a point in the point cloud to be added. Then, the point cloud to be added can be added to the first point cloud, so that the second point cloud can be spliced ​​with the first point cloud. Repeat the above process until the last frame of the target point cloud frame is spliced, and the calibration point cloud can be obtained.

[0070] The following takes sequential stitching as an example to illustrate how to stitch frame by frame to obtain the calibration point cloud. In the following example, the first point cloud acquisition device is a 3D camera, the 3D camera is located at the end of the robot, and the first coordinate system is the robot base coordinate system.

[0071] The 3D camera collects the calibration workpiece point cloud at different camera poses, and transforms the collected local point cloud (i.e., the initial point cloud frame) into the robot base coordinate system using the hand-eye calibration matrix and the TCP pose. Then, the frame-by-frame point cloud stitching algorithm is used to stitch these local point clouds together. The stitched point clouds form a complete point cloud of the calibration workpiece (i.e., the calibration point cloud), which is called P global .

[0072] The specific process of the frame-by-frame stitching method is as follows.

[0073] First, take a local point cloud of the calibration workpiece at the camera pose 0 to obtain point cloud 0, and use the hand-eye calibration matrix to convert point cloud 0 to the robot base coordinate system to obtain point cloud 0'; then move the camera to pose 1, take a local point cloud of the calibration workpiece to obtain point cloud 1, and use the hand-eye calibration matrix to convert point cloud 1 to the robot base coordinate system to obtain point cloud 1'. Splice point cloud 1' onto point cloud 0', and use the ICP (iterative closest point) algorithm to align point cloud 1' and point cloud 0' to obtain the rotation matrix R from point cloud 1' to point cloud 0' 10 and the translation vector T 10 , and then pass point cloud 1' through R 10 and T 10Perform the conversion to obtain point cloud 1", compare point cloud 1" with point cloud 0', obtain the points in point cloud 1", which are not in point cloud 0', add these points to point cloud 0', and obtain a point cloud composed of point cloud 0 and point cloud 1, which can be represented by point cloud B. Next, shoot the local point cloud of the calibration workpiece in shooting posture 2 to obtain point cloud 2, use the hand-eye calibration matrix to convert point cloud 0 to the robot base coordinate system to obtain point cloud 2', and splice point cloud 2' onto point cloud B to obtain the updated point cloud B, and so on, until the shooting of all shooting postures of the calibration workpiece is completed. At this time, the final point cloud B is the complete point cloud of the calibration workpiece. The complete point cloud of the workpiece obtained by the frame-by-frame splicing algorithm has high accuracy.

[0074] Please refer to Figure 5 , Figure 5 The third flow chart of the model-free weld recognition method provided in the embodiment of the present application is as follows. In the present embodiment, before step S170, the method further includes step S140 for obtaining first weld annotation information.

[0075] Step S140, obtaining and saving the first weld marking information.

[0076] In this embodiment, the weld can be marked in the following two ways: manual teaching and point selection on the calibration point cloud.

[0077] Method 1: Obtain the first weld annotation information corresponding to at least one weld by moving the end of the robot to the weld on the calibration workpiece, wherein the posture of the calibration workpiece when acquiring the multiple point clouds is the same as the posture when the end of the robot is moved to the weld of the calibration workpiece, and the first coordinate system is the robot base coordinate system. Method 1 is the above-mentioned manual teaching. It can be seen that Method 1 refers to marking the weld on the calibration workpiece by manual teaching under the premise that the posture of the calibration workpiece remains unchanged; that is, the calibration workpiece remains stationary, point cloud acquisition is performed to obtain the calibration point cloud, and manual teaching is performed to obtain the corresponding first weld annotation information.

[0078] Generally speaking, there are two types of welds on a workpiece, one is a straight weld and the other is an arc weld. For a straight weld, it is necessary to mark the starting point and end point of the straight weld; in this case, the end of the welding gun can be manually moved to the starting point and end point of all the straight welds of the calibration workpiece, and the TCP position at this time can be recorded to mark all the straight welds. At this time, the first weld annotation information corresponding to the straight weld includes the first starting point and the first end point of the marked straight weld. Suppose the calibration workpiece has n straight welds, and for the i-th straight weld, its starting point position is a i , whose end point is b i .

[0079] For arc welds, three points on the arc welds need to be marked. By manually moving the end of the welding gun to the three points on the arc, the TCP position at that time is recorded, thereby marking all arc welds. At this time, the first weld annotation information corresponding to the arc weld includes the positions of the three points of the annotated arc weld. Assume that the calibration workpiece has m arc welds. For the i-th arc weld, the three marked points are respectively recorded as a i,1 、a i,2 、a i,3 .

[0080] Among them, a flag bit can also be set for the arc weld, which is used to distinguish whether the corresponding arc weld is a full circle (i.e., a complete standard circle) or a local arc on the full circle. In the case where the arc weld to be marked is a full circle, the value of the corresponding flag bit can be set to indicate that it is a full circle, and the end of the welding gun is manually moved to any three points on the full circle, and the TCP position at this time is recorded to obtain the first weld annotation information corresponding to the arc weld that is a full circle. In the case where the arc weld to be marked is a local arc on the full circle, the value of the corresponding flag bit can be set to indicate that it is not a full circle, and the end of the welding gun is manually moved to the starting point, end point and middle point of the arc, a total of three points, and the TCP position at this time is recorded to obtain the first weld annotation information corresponding to the arc weld that is not a full circle.

[0081] All marked weld points are in the robot base coordinate system, and these points are associated with the current workpiece (i.e., the calibration workpiece) posture.

[0082] Method 2: Obtain the first weld annotation information according to the weld annotation operation received for the calibration point cloud. In this method, the user can manually select points on the calibration point cloud, and the selected points are the points on the corresponding welds; that is, the user directly selects the points on the calibration point cloud P global Select the point corresponding to the weld. global The point cloud is obtained by the frame-by-frame stitching algorithm, so its accuracy is close to the model point cloud. In this way, the accuracy of the first weld annotation information can also be guaranteed by the user's point selection method. Similarly, for a straight weld, two points can be selected, namely the start point and the end point; for an arc weld, three points need to be selected. global When the first coordinate system used is the robot base coordinate system, in the calibration point cloud P global The coordinates of the point selected on the robot are also in the robot base coordinate system.

[0083] For a type of workpiece, after executing the above steps S110 to S140, it is equivalent to obtaining the workpiece model point cloud and all weld marking information of this type of workpiece. Thereafter, for the same type of workpiece, there is no need to execute the above steps S110 to S140 again. Instead, the information obtained from the above steps S110 to S140 is used to identify the welds, thereby performing intelligent welding.

[0084] Next, it is introduced how to perform weld identification based on the calibration point cloud and the first weld identification information.

[0085] The workpiece to be identified can be understood as a newly arrived workpiece of the same type as the calibration workpiece, and weld identification needs to be performed on the workpiece to be identified. In the current base coordinate system, the position and posture of the workpiece to be identified is unknown.

[0086] In order to ensure the registration effect of the workpiece point cloud of the workpiece to be identified and the calibration point cloud, the workpiece point cloud can be a global point cloud obtained by splicing at least two point clouds obtained by the second point cloud acquisition device. For the workpiece to be identified, multiple local point clouds can be collected by the second point cloud acquisition device, and then the multiple local point clouds are spliced ​​to obtain the workpiece point cloud. That is, the workpiece point cloud P new,global is the global point cloud of the workpiece to be identified. The second point cloud acquisition device and the first point cloud acquisition device can be devices of the same type, for example, both are 3D cameras located at the end of the robot. The second point cloud acquisition device and the first point cloud acquisition device can be the same point cloud acquisition device on the same welding robot, or point cloud acquisition devices of different welding robots, which is determined by actual conditions.

[0087] In order to reduce the amount of processing, the number of local point clouds collected for stitching the workpiece point cloud can be less than the number of local point clouds collected when obtaining the calibration point cloud. For example, 2 to 3 point clouds can be collected to stitch the workpiece point cloud, wherein the 2 to 3 point clouds can be taken by a 3D camera at different positions, and the 2 to 3 point clouds can describe the general overall structure of the workpiece to be identified, so as to avoid misregistration when there are repeated structures in the workpiece to be identified.

[0088] Optionally, the workpiece point cloud may be obtained by splicing a plurality of local point clouds of the workpiece point cloud in the same manner as the calibration point cloud is spliced.

[0089] Alternatively, the workpiece point cloud can be quickly spliced ​​in the following manner. The multiple local point clouds of the workpiece to be identified are converted to the second coordinate system according to the calibration matrix corresponding to the pre-acquired second coordinate system, and then superimposed (that is, directly added) to obtain the workpiece point cloud. Wherein, when the second coordinate system is the base coordinate system of the robot corresponding to the second point cloud acquisition device, in the eye-on-hand mode, the multiple local point clouds of the workpiece to be identified can be converted to the robot base coordinate system (that is, the second coordinate system) corresponding to the second point cloud acquisition device according to the TCP pose and hand-eye calibration matrix corresponding to each local point cloud.

[0090] After obtaining the workpiece point cloud P new,global In the case of new,global With the calibration point cloud P global Perform point cloud registration to obtain the calibration point cloud P global To workpiece point cloud P new,global The first conversion relationship may include the calibration point cloud P global To workpiece point cloud P new,global The rotation matrix R and translation vector T. Since P global is the global point cloud of the collected calibration workpiece, P global The shape information of the workpiece is well described. Although there is no workpiece model point cloud as a reference, P global It can be used as a good substitute for the workpiece model point cloud. global With P new,global After registration, more accurate R and T can be obtained.

[0091] When the first conversion relationship is obtained, the first weld annotation information is converted into a second coordinate system where the workpiece point cloud is located using the first conversion relationship to obtain second weld annotation information.

[0092] In the case where the first conversion relationship includes the rotation matrix R and the translation vector T, for a straight weld, the converted points of the first starting point and the first end point in the first weld annotation information corresponding to the straight weld are: a i '=R·a i +T,b i '=R·b i +T, where a i ' represents the second starting point indicated by the corresponding second weld annotation information, b i ' represents the second end point indicated by the corresponding second weld annotation information. Similarly, for the arc weld, the converted points of the three points in the first weld annotation information corresponding to the arc weld are: a' i,1 =R·ai,1 +T,a' i,2 =R·a i,2 +T,a' i,3 =R·a i,3 +T, where a' i,1 、a' i,2 、a' i,3 The three points indicated by the corresponding second weld annotation information are arc points.

[0093] Since the weld point marking is carried out on the calibration workpiece, a i 、b i These points coincide with the actual weld start and end points of the calibrated workpiece. The differences between the calibrated workpiece and the workpiece to be identified mainly exist in the following two parts: 1. Posture transformation, 2. Processing error. When the weld points of the calibrated workpiece are transferred to the second coordinate system of the workpiece to be identified through R and T, if there is no processing error between the calibrated workpiece and the workpiece to be identified, the converted points coincide with the actual weld start and end points of the workpiece to be identified. At this time, the converted points can be directly used as the result of weld identification. However, under normal circumstances, there is a certain processing error between the workpiece to be identified and the calibrated workpiece, so the converted points do not necessarily completely coincide with the actual weld start and end points on the workpiece to be identified, but are near the actual weld start and end points on the workpiece to be identified. At this time, the converted points can be used to perform local weld identification, and the start and end points corresponding to the actual weld can be found. There are many schemes for local weld identification. In this embodiment, local weld identification can be performed in the following way.

[0094] If the weld corresponding to the first weld annotation information stored includes a straight weld, Figure 6 The target straight weld is identified as shown in the figure. Figure 6 , Figure 6 for Figure 2 One of the flowchart diagrams of the sub-steps included in step S180. In this embodiment, step S180 may include sub-steps S181 to S182.

[0095] In sub-step S181, for the second weld annotation information corresponding to each linear weld, point cloud extraction is performed on the workpiece point cloud according to the second starting point and the second end point indicated by the second weld annotation information to obtain a first point cloud to be identified.

[0096] Sub-step S182, performing linear weld recognition on the first point cloud to be recognized to obtain a target linear weld.

[0097] In this embodiment, for the second weld annotation information corresponding to each straight weld, a local point cloud can be intercepted from the workpiece point cloud as the first point cloud to be identified for straight weld identification according to the second starting point and the second end point indicated by the second weld annotation information. The specific interception method for obtaining the first point cloud to be identified can be set according to actual needs.

[0098] As a possible implementation, the first point cloud to be identified is obtained based on a first cylinder, the first cylinder is obtained based on a preset radius, the second starting point and the second end point, and the second starting point and the second end point are located on the axis of the first cylinder. The first point cloud to be identified may be a local point cloud of the workpiece point cloud located within the first cylinder. Optionally, the centers of the upper and lower bottom surfaces of the first cylinder may be the second starting point and the second end point.

[0099] After obtaining a first point cloud to be identified corresponding to a second weld annotation information, linear weld identification can be performed according to the distribution of points in the first point cloud to be identified. All target linear welds in the workpiece point cloud can be identified by performing linear weld identification on the first point cloud to be identified corresponding to each second weld annotation information.

[0100] As a possible implementation, Figure 7 The method identifies the target straight weld based on a first point cloud to be identified. Figure 7 , Figure 7 for Figure 6 Schematic diagram of the flow of sub-steps included in sub-step S182. In this embodiment, sub-step S182 may include sub-steps S1821 to S1825.

[0101] Sub-step S1821: performing plane detection on the first point cloud to be identified to obtain multiple first planes.

[0102] Optionally, in this embodiment, the first point cloud to be identified can be plane-detected using Hough transform, thereby obtaining a plurality of first planes. It is understandable that plane detection can also be performed in other ways, which are not specifically limited here. After obtaining the plurality of first planes, two planes can be selected from the plurality of first planes for determining the target linear weld. Among them, the plane obtained by plane detection of the point cloud is an infinitely large plane represented by a plane equation.

[0103] Sub-step S1822, for each first plane, project the second starting point and the second end point onto the first plane respectively, and determine whether the two obtained endpoint projection points are inside the local plane corresponding to the first plane in the first point cloud to be identified.

[0104] For each of the multiple first planes obtained, the second starting point is projected onto the first plane to obtain an endpoint projection point corresponding to the second starting point, and the second endpoint is projected onto the first plane to obtain an endpoint projection point corresponding to the second endpoint. In this way, two endpoint projection points corresponding to the first plane can be obtained. It can also be determined whether the two endpoint projection points corresponding to a first plane are inside a local plane corresponding to the first plane in the first point cloud to be identified. The local plane corresponding to the first plane in the first point cloud to be identified is a plane with a size, and the local plane is formed by the point cloud points corresponding to the first plane in the first point cloud to be identified.

[0105] Optionally, you can Figure 8 The method shown in the figure determines whether an endpoint projection point is inside the corresponding local plane. Figure 8 , Figure 8 for Figure 7 Schematic diagram of the flow of sub-steps included in sub-step S1822. In this embodiment, sub-step S1822 may include sub-steps S18221 to S18224.

[0106] Sub-step S18221, for each endpoint projection point, determine a sphere with the endpoint projection point as the sphere center.

[0107] Sub-step S18222, determining whether there are point cloud points belonging to the first plane within the sphere.

[0108] If it exists, execute sub-step S18223.

[0109] Sub-step S18223, determining that the endpoint projection point is inside the local plane corresponding to the first plane.

[0110] If it does not exist, execute sub-step S18224.

[0111] Sub-step S18224, determining that the projection point of the originating endpoint is not inside the local plane corresponding to the first plane.

[0112] In this embodiment, for an endpoint projection point, the endpoint projection point is used as the center of the sphere and a value is set as the radius, thereby determining a sphere. The radius of the sphere can be relatively small, and can be determined in combination with actual needs. Then, it is determined whether there are point cloud points belonging to the first plane corresponding to the endpoint projection point in the sphere. The point cloud points belonging to the first plane are the point cloud points corresponding to the first plane in the first point cloud to be identified. If there are corresponding point cloud points of the first plane in the sphere, it is determined that the endpoint projection point is inside the local plane corresponding to the corresponding first plane. If there are no corresponding point cloud points of the first plane in the sphere, it is determined that the endpoint projection point is not inside the local plane corresponding to the corresponding first plane.

[0113] After performing the above judgment on the two endpoint projection points corresponding to a first plane, it can be determined whether the two endpoint projection points corresponding to the first plane are both inside the local plane corresponding to the first plane.

[0114] Sub-step S1823, for each first plane, calculate the sum of the distances between each point in the second starting point and the second end point and the first plane as the first total distance corresponding to the first plane.

[0115] In this embodiment, based on the second weld annotation information currently targeted, for each first plane, the distance between the second starting point in the second weld annotation information currently targeted and the first plane, and the distance between the second end point and the first plane are calculated, and the sum of the two distances is calculated as the first total distance corresponding to the first plane. The two distances used to obtain the first total distance are the vertical distances between the point and the plane.

[0116] Sub-step S1824, taking two first planes whose first total distance between the corresponding two endpoint projection points in the first plane inside the corresponding local plane is the smallest as two first target planes.

[0117] For each first plane, it can be determined whether the two endpoint projection points corresponding to the first plane are both included in the local plane corresponding to the first plane. If so, the first plane is used as the subsequent object to be selected. If not, the first plane is not used as the subsequent object to be selected. As the first plane to be selected, the two endpoint projection points corresponding to the first plane are included in the local plane corresponding to the first plane. Then, from the first planes determined as the objects to be selected, the two first planes with the smallest first total distance are selected, that is, the planes corresponding to the smallest first total distance and the second smallest first total distance are selected as the two first target planes.

[0118] Sub-step S1825, calculates the straight line intersection of the two first target planes, projects the second starting point and the second end point onto the straight line intersection respectively, and uses the obtained projection points as the two endpoints of the identified target straight line weld.

[0119] The plane equation can be obtained during plane detection. When the two first target planes corresponding to the second weld annotation information currently targeted are determined, the straight line intersection line can be calculated according to the plane equations of the two first target planes. The straight line intersection line is infinitely long. The second starting point and the second end point indicated by the second weld annotation information currently targeted can be projected onto the straight line intersection line respectively, and the projection point corresponding to the obtained second starting point and the projection point corresponding to the second end point are used as the two end points of a target straight line weld identified. In this way, a target straight line weld can be obtained based on a second weld annotation information. Weld recognition is performed based on the second weld annotation information corresponding to each straight line weld, and each target straight line weld in the workpiece point cloud can be identified.

[0120] When the weld corresponding to the first weld annotation information stored includes an arc weld, Fig. 9 The target arc weld is identified in the manner shown. Fig. 9 , Fig. 9 for Figure 2 2 is a flow chart of sub-steps included in step S180. In this embodiment, step S180 may include sub-steps S184 to S185.

[0121] In sub-step S184, for the second weld annotation information corresponding to each arc weld, according to the arc weld position indicated in the second weld annotation information, point cloud extraction is performed on the workpiece point cloud to obtain a second point cloud to be identified.

[0122] Sub-step S185, performing arc weld recognition on the second point cloud to be recognized to obtain a target cylindrical weld.

[0123] In this embodiment, for each arc weld corresponding to the second weld annotation information, a local point cloud can be intercepted from the workpiece point cloud according to the arc weld position indicated by the second weld annotation information as the second point cloud to be identified for arc weld identification. The specific interception method for obtaining the second point cloud to be identified can be set according to actual needs.

[0124] Wherein, optionally, the second weld annotation information may indicate the positions of at least three arc points, and the arc points are located on the arc weld indicated by the second weld annotation information. As a possible implementation manner, the second weld annotation information indicates the positions of three arc points.

[0125] As a possible implementation, the circumscribed circle of at least three arc points indicated by the second weld annotation information can be calculated, and then the circumscribed circle is enlarged by a certain multiple to obtain a new circle; and a plane passing through the at least three arc points is calculated, and the plane is translated upward and downward by a preset distance according to the normal of the plane, and a cylinder is formed with the previous new circle as a second cylinder. The second point cloud to be identified is extracted from the workpiece point cloud using the second cylinder. The second point cloud to be identified is a local point cloud in the workpiece point cloud located within the second cylinder.

[0126] After obtaining a second point cloud to be identified corresponding to a second weld annotation information, arc weld identification can be performed according to the distribution of points in the second point cloud to be identified. All target arc welds in the workpiece point cloud can be identified by performing arc weld identification on the second point cloud to be identified corresponding to each second weld annotation information.

[0127] As a possible implementation, Fig.10 The method shown identifies the target arc weld based on a second point cloud to be identified. Fig.10 , Fig.10 for Fig. 9 Schematic diagram of the flow of sub-steps included in sub-step S185. In this embodiment, sub-step S185 may include sub-steps S1851 to S1856.

[0128] Sub-step S1851, detecting the second point cloud to be identified to obtain at least one second plane and at least one cylindrical surface.

[0129] Optionally, in this embodiment, Hough transform can be used to perform plane detection and cylindrical surface detection on the second point cloud to be identified, so as to obtain at least one second plane and at least one cylindrical surface. It can be understood that plane detection can also be performed in other ways, which are not specifically limited here. After obtaining the at least one second plane and at least one cylindrical surface, one plane can be selected from the at least one second plane, and one cylindrical surface can be selected from the at least one cylindrical surface to determine the target arc weld.

[0130] Sub-step S1852, for each second plane, calculate the sum of the distances between each arc point indicated by the second weld annotation information and the second plane as the second total distance corresponding to the second plane.

[0131] Sub-step S1853: taking the second plane corresponding to the smallest second total distance as the second target plane.

[0132] In this embodiment, for each of the at least one second plane, the vertical distances between the at least three arc points indicated by the corresponding second weld annotation information and the second plane can be calculated, and the sum of the at least three distances obtained is used as the second total distance corresponding to the second plane. Afterwards, the second total distances corresponding to the second planes are compared to determine the smallest second total distance, and the second plane corresponding to the smallest second total distance is used as the selected second target plane.

[0133] Sub-step S1854, for each cylindrical surface, calculate the sum of the distances between each arc point indicated by the second weld annotation information and the cylindrical surface as the third total distance corresponding to the cylindrical surface.

[0134] Sub-step S1855, taking the cylindrical surface corresponding to the smallest third total distance as the target cylindrical surface.

[0135] In this embodiment, for each of the at least one cylindrical surface, the distances between the at least three arc points indicated by the corresponding second weld annotation information and the cylindrical surface can be calculated, and the sum of the at least three distances obtained is used as the third total distance corresponding to the arc surface. Afterwards, the third total distances corresponding to the cylindrical surfaces are compared to determine the smallest third total distance, and the cylindrical surface corresponding to the smallest third total distance is used as the selected target cylindrical surface.

[0136] In this way, the second target plane and the target cylindrical surface can be quickly determined.

[0137] Sub-step S1856, calculating the circular intersection line between the second target plane and the target cylindrical surface, and determining the target arc weld from the circular intersection line according to the second weld annotation information.

[0138] When performing plane detection and cylindrical surface detection, the plane equation and the cylindrical surface equation can be obtained. When the second target plane and the target cylindrical surface are determined, the circular intersection line can be calculated according to the plane equation of the second target plane and the cylindrical surface equation of the target cylindrical surface. Afterwards, at least three arc points indicated by the second weld annotation information can be projected onto the circular intersection line respectively, and the obtained projection points are used as points on the identified target arc weld. Wherein, when the arc weld corresponding to the second weld annotation information is not a full circle, the at least three arc points indicated by the second weld annotation information include a starting point, a middle point and an end point, so the target arc weld can be determined according to the projection points on the circular intersection line. The starting point of the target arc weld is the projection point of the starting point indicated by the second weld annotation information on the circular intersection line, the end point is the projection point of the end point indicated by the second weld annotation information on the circular intersection line, and there is a projection point of the middle point indicated by the second weld annotation information on the target arc weld.

[0139] It can be understood that if the welds corresponding to the saved first weld annotation information include straight line welds and arc welds, the target straight line welds and target arc welds can be identified through sub-steps S181 to S182 and sub-steps S184 to S185.

[0140] Through the above description, the model-free weld recognition method provided by the embodiment of the present application is divided into two major parts. The first part is to collect the global point cloud of the calibration workpiece and mark the weld information on the global point cloud; the second part is to perform weld recognition on the workpiece to be identified based on the obtained global point cloud and weld information. By performing a global photo shoot and a weld marking on the calibration workpiece, for all subsequent new workpieces of the same type, weld recognition can be performed directly based on the obtained calibration point cloud and the first weld marking information, and this method does not rely on the workpiece model. For different types of workpieces, it is necessary to perform a global photo shoot and weld point marking on the calibration workpiece of each type of workpiece.

[0141] In order to execute the corresponding steps in the above embodiments and various possible methods, a method for implementing a model-free weld identification device 200 is provided below. Optionally, the model-free weld identification device can adopt the above Figure 1 The device structure of the electronic device 100 is shown. Fig.11 , Fig.11 One of the block diagrams of the model-free weld recognition device provided in the embodiment of the present application. It should be noted that the basic principle and technical effect of the model-free weld recognition device 200 provided in this embodiment are the same as those of the above-mentioned embodiments. For the sake of brief description, for the parts not mentioned in this embodiment, reference can be made to the corresponding contents in the above-mentioned embodiments. In this embodiment, the model-free weld recognition device 200 may include: a workpiece point cloud acquisition module 230, a registration module 240, a conversion module 250 and a recognition module 260.

[0142] The workpiece point cloud obtaining module 230 is used to obtain the workpiece point cloud of the workpiece to be identified.

[0143] The registration module 240 is used to obtain a first conversion relationship between a first coordinate system where the saved calibration point cloud is located and a second coordinate system where the workpiece point cloud is located through registration calculation. The calibration point cloud is obtained by splicing multiple point clouds obtained by performing point cloud acquisition on the calibration workpiece, and the calibration workpiece is a workpiece of the same type as the workpiece to be identified.

[0144] The conversion module 250 is used to convert the first weld annotation information corresponding to the saved calibration point cloud into the second coordinate system according to the first conversion relationship to obtain the second weld annotation information. The first weld annotation information is used to indicate the position of the weld on the calibration workpiece in the first coordinate system.

[0145] The identification module 260 is used to perform weld identification on the workpiece point cloud according to the second weld annotation information.

[0146] Please refer to Fig.12 , Fig.12 The second block diagram of the model-free weld identification device 200 provided in the embodiment of the present application. In this embodiment, the model-free weld identification device 200 may further include a calibration point cloud acquisition module 210 .

[0147] The calibration point cloud acquisition module 210 is used to: obtain multiple initial point cloud frames obtained by performing point cloud acquisition on the calibration workpiece at different postures through a first point cloud acquisition device, wherein the multiple initial point clouds are the multiple point clouds; according to the calibration matrix corresponding to the first coordinate system obtained in advance, convert the multiple initial point cloud frames to the first coordinate system to obtain multiple target point cloud frames; according to the point cloud acquisition order, splice the multiple target point cloud frames frame by frame to obtain the calibration point cloud, wherein the acquisition areas corresponding to two frames of point clouds adjacent to each other in the acquisition order overlap.

[0148] Please refer to Fig.12 In this embodiment, the model-free weld identification device 200 may further include a labeling module 220. The labeling module 220 is used to obtain and save the first weld labeling information. In this embodiment, the labeling module 220 is specifically used to obtain the first weld labeling information corresponding to at least one weld by moving the robot end to the weld on the calibration workpiece, wherein the posture of the calibration workpiece when the multiple point clouds are collected is the same as the posture when the robot end is moved to the weld of the calibration workpiece, and the first coordinate system is the robot base coordinate system; or, the first weld labeling information is obtained according to the received weld labeling operation for the calibration point cloud.

[0149] Optionally, the above modules can be stored in the form of software or firmware. Figure 1 The memory 110 shown in the figure may be fixed in the operating system (OS) of the electronic device 100 and may be Figure 1 Meanwhile, the data and program codes required for executing the above modules may be stored in the memory 110.

[0150] An embodiment of the present application also provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the model-free weld identification method is implemented.

[0151] In summary, the embodiments of the present application provide a model-free weld recognition method, device, electronic device and readable storage medium, first obtaining a workpiece point cloud of a workpiece to be identified, then obtaining a first conversion relationship between a first coordinate system where a saved calibration point cloud is located and a second coordinate system where the workpiece point cloud is located through registration, and then based on the first conversion relationship, converting the first weld annotation information corresponding to the saved calibration point cloud to the second coordinate system to obtain the second weld annotation information, and finally performing weld recognition on the workpiece point cloud according to the second weld annotation information. Among them, the above-mentioned calibration point cloud is obtained by splicing multiple point clouds obtained by point cloud collection of a calibration workpiece, the calibration workpiece is a workpiece of the same type as the workpiece to be identified, and the first weld annotation information is used to indicate the position of the weld on the calibration workpiece in the first coordinate system. In this way, weld recognition can be achieved without a workpiece model.

[0152] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0153] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0154] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0155] The above description is only an optional embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A model-free weld identification method, characterized in that: The method comprises: Obtaining a workpiece point cloud of a workpiece to be identified; A first conversion relationship between a first coordinate system where a calibration point cloud is stored and a second coordinate system where the workpiece point cloud is stored is obtained by registering and calculating, wherein the calibration point cloud is obtained by splicing multiple point clouds obtained by performing point cloud acquisition on the calibration workpiece, and the calibration workpiece is a workpiece of the same type as the workpiece to be identified; According to the first conversion relationship, the first weld annotation information corresponding to the saved calibration point cloud is converted to the second coordinate system to obtain second weld annotation information, wherein the first weld annotation information is used to indicate the position of the weld on the calibration workpiece in the first coordinate system; The weld is identified on the workpiece point cloud according to the second weld annotation information.

2. The method according to claim 1, characterized in that Before obtaining the first conversion relationship between the first coordinate system where the calibration point cloud is stored and the second coordinate system where the workpiece point cloud is stored by the registration calculation, the method further includes: Obtaining a plurality of initial point cloud frames obtained by performing point cloud acquisition on the calibration workpiece at different postures through a first point cloud acquisition device, wherein the plurality of initial point clouds are the plurality of point clouds; According to the pre-obtained calibration matrix corresponding to the first coordinate system, the multiple initial point cloud frames are converted into the first coordinate system to obtain multiple target point cloud frames; The multiple target point cloud frames are spliced ​​frame by frame according to the point cloud acquisition sequence to obtain the calibration point cloud, wherein the acquisition areas corresponding to two adjacent point cloud frames in the acquisition sequence overlap.

3. The method according to claim 2, characterized in that The step of splicing the plurality of target point cloud frames frame by frame according to the point cloud acquisition sequence to obtain the calibration point cloud comprises: For the first point cloud and the second point cloud to be spliced ​​this time, a second transformation relationship of the coordinate systems of the first point cloud and the second point cloud is obtained by registration; wherein, in the first splicing, the first point cloud and the second point cloud are two frames of point clouds collected sequentially; when it is not the first splicing, the first point cloud is the splicing result of the previous splicing, and the second point cloud is a frame of point cloud to be spliced ​​this time determined according to the point cloud collection order; Using the second transformation relationship to transform the second point cloud, to obtain a second point cloud to be spliced; According to the second point cloud to be spliced ​​and the first point cloud, a point cloud to be added is determined, and the point cloud to be added is added to the first point cloud, wherein the point cloud to be added is a point cloud not included in the first point cloud but included in the second point cloud to be spliced.

4. The method according to claim 1, characterized in that: Before converting the first weld annotation information corresponding to the stored calibration point cloud to the second coordinate system according to the first conversion relationship to obtain the second weld annotation information, the method further includes: Obtaining and saving the first weld annotation information; Wherein, obtaining and saving the first weld annotation information includes: The first weld annotation information corresponding to at least one weld is obtained by moving the robot end to the weld on the calibration workpiece, wherein the posture of the calibration workpiece when the plurality of point clouds are acquired is the same as the posture when the robot end is moved to the weld of the calibration workpiece, and the first coordinate system is the robot base coordinate system; or The first weld annotation information is obtained according to the received weld annotation operation for the calibration point cloud.

5. The method according to any one of claims 1 to 4, characterized in that: The workpiece point cloud is a global point cloud obtained by splicing at least two point clouds; and / or, when the weld corresponding to the first weld annotation information stored includes a straight weld, the weld recognition on the workpiece point cloud according to the second weld annotation information includes: For the second weld annotation information corresponding to each straight weld, extract the workpiece point cloud according to the second starting point and the second end point indicated by the second weld annotation information to obtain a first point cloud to be identified; The first point cloud to be identified is used to identify a straight line weld to obtain a target straight line weld.

6. The method according to claim 5, characterized in that The first point cloud to be identified is obtained based on a first cylinder, the first cylinder is obtained based on a preset radius, the second starting point and the second end point, and the second starting point and the second end point are located on the axis of the first cylinder; And / or, performing linear weld recognition on the first point cloud to be recognized to obtain a target linear weld includes: Performing plane detection on the first point cloud to be identified to obtain a plurality of first planes; For each first plane, project the second starting point and the second end point onto the first plane respectively, and determine whether the two obtained end point projection points are inside the local plane corresponding to the first plane in the first point cloud to be identified; For each first plane, calculate the sum of the distances between each point in the second starting point and the second end point and the first plane as the first total distance corresponding to the first plane; The two first planes with the smallest first total distance between the projection points of the two corresponding endpoints in the first plane inside the corresponding local plane are taken as two first target planes; The straight line intersection of the two first target planes is calculated, the second starting point and the second end point are respectively projected onto the straight line intersection, and the obtained projection points are used as the two end points of the identified target straight line weld.

7. The method according to claim 6, characterized in that The determining whether the two endpoint projection points are inside a local plane corresponding to the first plane in the first point cloud to be identified includes: For each endpoint projection point, a sphere is determined with the endpoint projection point as the sphere center; Determine whether there are point cloud points belonging to the first plane in the sphere, wherein the point cloud points belonging to the first plane are point cloud points corresponding to the first plane in the first point cloud to be identified; If so, it is determined that the endpoint projection point is inside the local plane corresponding to the first plane.

8. The method according to any one of claims 1 to 4, characterized in that: The workpiece point cloud is a global point cloud obtained by splicing at least two point clouds; and / or, in a case where the weld corresponding to the first weld annotation information stored includes an arc weld, the weld recognition on the workpiece point cloud according to the second weld annotation information includes: For the second weld annotation information corresponding to each arc weld, according to the arc weld position indicated in the second weld annotation information, performing point cloud extraction on the workpiece point cloud to obtain a second point cloud to be identified; Arc weld recognition is performed on the second point cloud to be identified to obtain a target cylindrical weld.

9. The method according to claim 8, characterized in that The step of performing arc weld recognition on the second point cloud to be recognized to obtain a target cylindrical weld includes: Detecting the second point cloud to be identified to obtain at least one second plane and at least one cylindrical surface; For each second plane, the sum of the distances between each arc point indicated by the second weld annotation information and the second plane is calculated as the second total distance corresponding to the second plane, wherein the arc point is located on the arc weld indicated by the second weld annotation information; taking the second plane corresponding to the smallest second total distance as the second target plane; For each cylindrical surface, the sum of the distances between each arc point indicated by the second weld annotation information and the cylindrical surface is calculated as the third total distance corresponding to the cylindrical surface; The cylindrical surface corresponding to the smallest third total distance is taken as the target cylindrical surface; A circular intersection line between the second target plane and the target cylindrical surface is calculated, and a target arc weld is determined from the circular intersection line according to the second weld annotation information.

10. A model-free weld identification device, characterized in that: The device comprises: A workpiece point cloud acquisition module is used to obtain a workpiece point cloud of a workpiece to be identified; A registration module, used for obtaining, by registration calculation, a first transformation relationship between a first coordinate system where a calibration point cloud is stored and a second coordinate system where the workpiece point cloud is stored, wherein the calibration point cloud is obtained by splicing a plurality of point clouds obtained by performing point cloud acquisition on a calibration workpiece, and the calibration workpiece is a workpiece of the same type as the workpiece to be identified; a conversion module, configured to convert the first weld annotation information corresponding to the saved calibration point cloud into the second coordinate system according to the first conversion relationship, so as to obtain second weld annotation information, wherein the first weld annotation information is used to indicate the position of the weld on the calibration workpiece in the first coordinate system; An identification module is used to perform weld identification on the workpiece point cloud according to the second weld annotation information.

Images