Weld joint marking method and device, electronic equipment and storage medium

By collecting and processing point cloud data, weld cross-lines are determined and marked, the problems of poor migration, high usage threshold and manual labeling in the existing weld database labeling technology are solved, and efficient and accurate weld labeling and database are achieved. renew.

CN119090962APending Publication Date: 2024-12-06FAIR INNOVATION (SUZHOU) ROBOTIC SYSTEM CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411190421.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The existing weld database labeling technology has poor mobility, high usage threshold, poor expansion, and manual labeling is difficult to process high-dimensional information, resulting in low labeling efficiency, difficult to ensure accuracy and high update costs.

Method used

By collecting point cloud data, determine the registration matrix and converting it into a target point cloud collection, obtain the plane and surface structure of the workpiece model, determine the intersection line and target endpoint information, filter and label the target intersection line according to preset conditions, and improve the annotation Efficiency and precision.

Benefits of technology

It realizes the efficiency and accuracy of weld marking, reduces the time-consuming and updating cost of labeling, and improves the maintenance and accuracy of the database.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119090962A_ABST
    Figure CN119090962A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a welding seam marking method and device, electronic equipment and a storage medium, and the method comprises the steps: collecting point cloud data, determining a registration matrix of each point cloud data, and converting each point cloud data based on each registration matrix to obtain a target point cloud set, the target point cloud set comprises each target point cloud data, and the target point cloud set comprises target point cloud data; obtaining planes corresponding to a pre-stored workpiece model and a plurality of corresponding surface structures in the target point cloud set, determining intersecting lines corresponding to the planes under different surface structures, determining target end point information corresponding to the intersecting lines, determining target intersecting lines meeting a preset condition from the intersecting lines, and sending the target intersecting lines to the target point cloud set; and marking the target intersection line according to the target endpoint information corresponding to the target intersection line. According to the invention, the accuracy and efficiency of welding seam marking are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data annotation, and in particular to a weld annotation method, device, electronic equipment and storage medium. Background Art

[0002] Intelligent welding robots can complete welding work with high autonomy. They have the ability to obtain workpiece information, plan welding processes, and complete welding tasks. They can even work in conjunction with other sensors and equipment to meet diverse production requirements. In the automatic welding process, more than 80% of the information is obtained through visualization. Compared with other sensors, the information obtained by visual sensing is larger, more accurate, more resistant to interference, and more robust. For visual algorithms, building a standard database that is easy to annotate, using standard scripts to traverse the database, and verifying the algorithm can greatly reduce the time cost of algorithm verification and error detection. The verified visual algorithm is also more robust in actual use.

[0003] There are two existing weld database annotation techniques: the first is to use self-developed software to extract the structure of the model and annotate the extracted features. This annotation method is simple and fast. The second is manual information annotation.

[0004] The two existing database annotation methods have obvious disadvantages:

[0005] First, the database annotation method that relies on software: ① Poor portability. The use of the software depends on the platform. The annotation software cannot be used for self-developed platforms or incompatible systems. ② High threshold for use. The database annotation method that relies on software requires familiarity with the use of the software, which increases the workload of algorithm testers. ③ Poor scalability. Every time the database annotation entry updates the new database, the software needs to be updated and upgraded accordingly, that is, one software cannot be used for all database information annotations, and the scalability is poor.

[0006] Secondly, manual information labeling: ① It is difficult to process high-dimensional information. As the amount of database data increases, the manual labeling method makes the database establishment cycle long and costly. ② It is difficult to ensure precision and accuracy. The labeling of weld data information, such as the endpoints of straight lines and other precise contents, depends on the experience of the inspector, which may lead to misjudgment and omission, and it is difficult to ensure accuracy. ③ It is difficult to update the database. When the data test standard is updated, the database needs to update the corresponding content, which will bring a large update cost. Summary of the invention

[0007] The object of the present invention is to provide a weld marking method, device, electronic equipment and storage medium, which can improve the marking efficiency and accuracy.

[0008] In order to achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:

[0009] In a first aspect, an embodiment of the present application provides a weld marking method, the method comprising:

[0010] Collect point cloud data;

[0011] Determine a registration matrix for each of the point cloud data, and transform each of the point cloud data based on each of the registration matrices to obtain a target point cloud set, wherein the target point cloud set includes each of the target point cloud data;

[0012] Acquire a plane and a plurality of corresponding surface structures corresponding to a pre-stored workpiece model in the target point cloud set;

[0013] Determine the intersection lines corresponding to the planes under different surface structures;

[0014] Determine the target endpoint information corresponding to each of the intersection lines;

[0015] Determine a target intersection line that meets a preset condition from each of the intersection lines;

[0016] The target intersection line is marked with target endpoint information corresponding to the target intersection line.

[0017] Optionally, the step of collecting point cloud data includes:

[0018] The point cloud data of the workpiece to be inspected is collected based on different placement postures, different shooting postures, different fill lights and different downsampling methods.

[0019] Optionally, the plane includes a first plane and a second plane, the surface structure is a two-surface structure, and the step of determining the intersection lines corresponding to the planes under different surface structures includes:

[0020] Determine a product of a first normal vector and a second normal vector as a first direction vector of a first intersection line between the first plane and the second plane, wherein the first normal vector is a normal vector of the first plane, and the second normal vector is a normal vector of the second plane;

[0021] determining a first point of the first plane and a second point of the second plane;

[0022] Determine the coordinates of a perpendicular point of the first point and the first intersection line based on the coordinates of the first point, the coordinates of the second point, the first normal vector, and the second normal vector;

[0023] The first intersection line is determined based on the first direction vector and the vertical point coordinates.

[0024] Optionally, the step of determining the coordinates of a perpendicular point of the first point and the first intersection line based on the coordinates of the first point, the coordinates of the second point, the first normal vector and the second normal vector comprises:

[0025] Determine a second direction vector of the second intersection line based on the first intersection line and the second normal vector, wherein the second intersection line indicates a straight line in the first plane that is perpendicular to the first intersection line and starts from the first point;

[0026] determining the second intersection line based on the second direction vector, the coordinates of the first point, and a scaling parameter of the second direction vector;

[0027] determining a distance from the first point to the second plane;

[0028] Calculate the value of the scaling parameter based on the distance from the first point to the second plane, the second direction vector, and the second normal vector;

[0029] The vertical point coordinates are calculated based on the coordinates of the first point, the value of the scaling parameter, and the second direction vector.

[0030] Optionally, the value of the scaling parameter satisfies the following formula:

[0031]

[0032] Wherein, t is the value of the scaling parameter, D is the distance from the first point to the second plane, is the second direction vector, is the second normal vector.

[0033] Optionally, the step of determining a target intersection line that satisfies a preset condition from among the intersection lines comprises:

[0034] Acquiring the weld annotation data of the pre-stored workpiece model;

[0035] Obtaining a third direction vector in the weld annotation data;

[0036] For each of the intersection lines, determining an angle between a direction vector of the intersection line and the third direction vector;

[0037] From the angles, an intersection line corresponding to the minimum angle is obtained as a target intersection line.

[0038] Optionally, the method further comprises:

[0039] Determine the output parameters of the target intersection, wherein the output parameters include the point cloud data of the target intersection, the coordinate information corresponding to each of the point cloud data, the target endpoint information of the target intersection, the position coordinates of the end of the flange of the robot arm corresponding to the point cloud data corresponding to the target intersection, and different registration matrices corresponding to different placement postures of the workpiece to be detected corresponding to the target intersection;

[0040] The output parameters are output and displayed.

[0041] In a second aspect, an embodiment of the present application provides a weld marking method, the device comprising:

[0042] A collection module, used for collecting point cloud data;

[0043] A determination module, used to determine the registration matrix of each point cloud data, and transform each point cloud data based on each registration matrix to obtain a target point cloud set, wherein the target point cloud set contains each target point cloud data;

[0044] An acquisition module, used for acquiring a plane and corresponding multiple surface structures corresponding to a pre-stored workpiece model in the target point cloud set;

[0045] The determination module is further used to determine the intersection lines corresponding to the planes under different surface structures; determine the target endpoint information corresponding to each of the intersection lines; and determine the target intersection line that meets the preset conditions from each of the intersection lines;

[0046] The marking module is used to mark the target intersection line with the target endpoint information corresponding to the target intersection line.

[0047] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the weld marking method when executing the computer program.

[0048] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the weld marking method when executed by a processor.

[0049] This application has the following beneficial effects:

[0050] The present application collects point cloud data, determines the registration matrix of each point cloud data, and converts each point cloud data based on each registration matrix to obtain a target point cloud set, wherein the target point cloud set contains each target point cloud data, obtains the planes and corresponding multiple surface structures corresponding to the pre-stored workpiece model in the target point cloud set, determines the intersection lines corresponding to each plane under different surface structures, determines the target endpoint information corresponding to each intersection line, determines the target intersection line that meets the preset conditions from each intersection line, and marks the target intersection line with the target endpoint information corresponding to the target intersection line. The present application provides accuracy and efficiency of weld annotation. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention;

[0053] Figure 2 One of the schematic diagrams of a weld marking process provided by an embodiment of the present invention;

[0054] Figure 3 A second flow chart of a weld marking method provided by an embodiment of the present invention;

[0055] Figure 4 A schematic diagram of a two-sided structure provided by an embodiment of the present invention;

[0056] Figure 5 A third flow chart of a weld marking method provided by an embodiment of the present invention;

[0057] Figure 6 A fourth flow chart of a weld marking method provided in an embodiment of the present invention;

[0058] Figure 7 A schematic structural diagram of a weld marking device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0060] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0062] In the description of the present invention, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. appear to indicate an orientation or position relationship, they are based on the orientation or position relationship shown in the accompanying drawings, or are the orientation or position relationship in which the product of the invention is usually placed when used. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0063] In addition, the terms “first”, “second”, etc., if used, are merely used to distinguish between the descriptions and should not be understood as indicating or implying relative importance.

[0064] In the description of this application, it should also be noted that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0065] After extensive research, it was found that there are two existing weld database annotation technology methods. The first is to use self-developed software to extract features from the structure of the model and annotate the extracted features. This annotation method is simple and fast. The second is manual information annotation. Database annotation methods that rely on software: ① Poor portability. The use of software depends on the platform. For self-developed platforms or incompatible systems, the annotation software cannot be used. ② High threshold for use. Database annotation methods that rely on software require familiarity with software use, which increases the workload of algorithm testers. ③ Poor scalability. Every time the database annotation entry updates a new database, the software needs to be updated and upgraded accordingly, that is, one software cannot be used for all database information annotations, and has poor scalability. Manual information annotation: ① It is difficult to process high-dimensional information. With the increase in the amount of database data, the manual annotation method makes the database establishment cycle long and the cost high. ② It is difficult to ensure precision and accuracy. The annotation of weld data information, such as the endpoints of a straight line and other precise content, depends on the experience of the inspector, and may cause misjudgment and missed judgment, making it difficult to ensure accuracy. ③ It is difficult to update the database. When the data testing standards are updated, the database needs to update the corresponding content, which will bring a large update cost.

[0066] In view of the discovery of the above problems, the present embodiment provides a weld annotation method, device, electronic device and storage medium, which can collect point cloud data, determine the registration matrix of each point cloud data, and convert each point cloud data based on each registration matrix to obtain a target point cloud set, wherein the target point cloud set includes each target point cloud data, obtains the plane corresponding to the pre-stored workpiece model and the corresponding multiple surface structures in the target point cloud set, determines the intersection lines corresponding to each plane under different surface structures, determines the target endpoint information corresponding to each intersection line, determines the target intersection line that meets the preset conditions from each intersection line, and annotates the target intersection line with the target endpoint information corresponding to the target intersection line. In order to solve the problems existing in the existing weld annotation methods, such as time-consuming annotation, difficult maintenance, long update cycle, and difficult to ensure accuracy, the solution provided by the present embodiment is described in detail below.

[0067] This embodiment provides an electronic device that can mark welds. In a possible implementation, the electronic device can be a user terminal, for example, the electronic device can be, but is not limited to, a server, a smart phone, a personal computer (PC), a tablet computer, a personal digital assistant (PDA), a mobile Internet device (MID), and an image acquisition device.

[0068] Please refer to Figure 1 , Figure 11 is a schematic diagram of the structure of the electronic device 100 provided in the embodiment of the present application. The electronic device 100 may also include Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0069] The electronic device 100 includes a weld marking device 110 , a memory 120 and a processor 130 .

[0070] The memory 120 and the processor 130 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines. The weld annotation device 110 includes at least one software function module that can be stored in the memory 120 in the form of software or firmware or solidified in the operating system (OS) of the electronic device 100. The processor 130 is used to execute the executable modules stored in the memory 120, such as the software function modules and computer programs included in the weld annotation device 110.

[0071] The memory 120 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc. The memory 120 is used to store a program, and the processor 130 executes the program after receiving an execution instruction.

[0072] Please refer to Figure 2 , Figure 2 For application Figure 1 A flow chart of a weld marking method for an electronic device 100 is provided, and the method including each step is described in detail below.

[0073] S201: Collect point cloud data.

[0074] S202: Determine the registration matrix of each point cloud data, and transform each point cloud data based on each registration matrix to obtain a target point cloud set.

[0075] The target point cloud set includes each target point cloud data.

[0076] S203: Acquire a plane and a plurality of corresponding surface structures corresponding to a pre-stored workpiece model in the target point cloud set.

[0077] S204: Determine the intersection lines corresponding to the planes under different surface structures.

[0078] S205: Determine target endpoint information corresponding to each intersection line.

[0079] S206: Determine, from among the intersection lines, a target intersection line that meets a preset condition.

[0080] S207: Marking the target intersection line with the target endpoint information corresponding to the target intersection line.

[0081] The point cloud data can be collected in the following ways: collecting the point cloud data of the workpiece to be inspected based on different placement postures, different shooting postures, different fill lights, and different downsampling methods.

[0082] In order to enrich the types of database data, point cloud data of the workpiece to be inspected are collected in different placement postures, different shooting postures, different fill lights and different downsampling methods.

[0083] Different placement postures may include different placement postures of the workpiece to be inspected. Different placement postures, different photographing postures, different fill lights, and different downsampling methods are combined to collect point cloud data of the workpiece to be inspected.

[0084] In one example, different placement postures of the workpiece to be inspected include a first posture, a second posture, and a third posture, different photographing postures include a first photographing posture, a second photographing posture, and a third photographing posture, different fill lighting includes no fill lighting processing, normal environment fill lighting, and strong fill lighting, and different downsampling methods include direct output point cloud and downsampled point cloud.

[0085] There are rotation and translation transformations between different placement postures of the workpiece to be inspected, and there are rotation and translation transformations between different photo taking postures. The downsampling ratio of the direct output point cloud downsampling method and the downsampling ratio parameters of the downsampled point cloud are adjustable and can be adjusted accordingly according to the needs of the algorithm test.

[0086] The specific methods for collecting point cloud data can be:

[0087] The following is an explanation of the point cloud data acquisition operation: S1: Place the workpiece to be inspected on a flat workbench in a first posture. S2: Photograph the workpiece to be inspected from multiple different angles and stitch them together. S3: Align the stitched point cloud with the pre-stored workpiece model to obtain a registration matrix. S4: Move the robotic arm to the first shooting posture. S5: Determine that the fill light method is not to perform fill light processing. S6: Photograph the workpiece to be inspected, and store the direct point cloud and the downsampled point cloud. S7: Determine that the fill light method is the normal environment fill light method, photograph the workpiece to be inspected, and store the direct point cloud and the downsampled point cloud. S8: Determine that the fill light method is strong fill light, photograph the workpiece to be inspected, and store the direct point cloud and the downsampled point cloud. S9: Move the robotic arm to the second shooting posture, and execute S5-S8. S10: Move the robotic arm to the third shooting posture, and execute S5-S8. S11: Place the workpiece to be inspected on a flat workbench in the second posture, and execute S2-S10. S12: The workpiece to be inspected is placed on a flat workbench in the third posture, and S2-S10 are executed. S13: The workpiece to be inspected is replaced, and S1-S12 are executed until all the workpieces to be inspected are photographed, and each point cloud data is obtained.

[0088] There are many ways to obtain the registration matrix of each point cloud data. In one implementation, each point cloud data can be registered with a pre-stored workpiece model to obtain the registration matrix of each point cloud data. In another implementation, the point cloud data taken by the workpiece to be detected in the same placement position can be spliced, and the spliced ​​point cloud data can be registered with the pre-stored workpiece model to obtain the registration matrix of the point cloud corresponding to the workpiece to be detected in the same placement position. Each point cloud data is converted based on the corresponding registration matrix to obtain the target point cloud set.

[0089] All planes in the target point cloud set are extracted, and based on the planes and multiple plane structures corresponding to the pre-stored workpiece model, planes and plane structures corresponding to the planes and plane structures of the pre-stored workpiece model are obtained from all corresponding planes in the target point cloud set.

[0090] The planar structure can be a two-sided structure, a three-sided structure, or a two-sided structure only. A three-sided structure is a structure composed of any three sides. A two-sided structure only is a planar structure composed of only two planes.

[0091] There are many ways to determine the surface structure. The following is an example of determining the two-surface structure:

[0092] In one implementation, the general formula of the spatial plane plane is: Ax+By+Cz+D=0, where p(x, y, z) represents the coordinates of any point on the plane. is the plane normal vector, D is the plane intercept, and the plane passes through a point pp =(x p ,y p , z p ); Assume that the distance threshold from the point to the plane is ε, and the definition of the surface structure relationship is given below:

[0093] Two-sided structure: There exists p, p∈plane1 or p∈plane2, satisfying d1≤ε and d2≤ε, then the set formed by p is a spatial straight line, satisfying Ax+By+Cz+D=0, where A structure formed by two surfaces, plane 1 and plane 2, is a two-surface structure.

[0094] Determine the intersection lines of each plane under different surface structures, and determine the target endpoint information corresponding to the intersection lines, determine the target intersection lines that meet the preset conditions from each intersection line, annotate the target based on the target endpoint information corresponding to the target intersection line, and complete the weld annotation of the point cloud data.

[0095] The method of determining the target intersection line that meets the preset condition from the intersection lines is actually to filter the intersection lines to obtain the target intersection line that meets the preset condition.

[0096] All planes in the target point cloud set are extracted. When the extracted planes include the first plane and the second plane, and the first plane and the second plane form a two-plane structure, there are multiple ways to determine the intersection lines corresponding to each plane under different surface structures. In one implementation, Figure 3 As shown, the following steps are included:

[0097] S204-1: Determine the product of the first normal vector and the second normal vector as the first direction vector of the first intersection line between the first plane and the second plane.

[0098] The first normal vector is the normal vector of the first plane, and the second normal vector is the normal vector of the second plane.

[0099] S204-2: Determine a first point on the first plane and a second point on the second plane.

[0100] S204-3: Determine the coordinates of a perpendicular point between the first point and the first intersection line based on the coordinates of the first point, the coordinates of the second point, the first normal vector, and the second normal vector.

[0101] S204-4: Determine a first intersection line based on the first direction vector and the vertical point coordinates.

[0102] The step of determining the coordinates of the perpendicular point between the first point and the first intersection line based on the coordinates of the first point, the coordinates of the second point, the first normal vector and the second normal vector includes: determining the second direction vector of the second intersection line based on the first intersection line and the second normal vector, wherein the second intersection line indicates a straight line obtained by drawing a perpendicular line from the first point to the first intersection line in the first plane; determining the second intersection line based on the second direction vector, the coordinates of the first point and the scaling parameter of the second direction vector; determining the distance from the first point to the second plane; calculating the value of the scaling parameter based on the distance from the first point to the second plane, the second direction vector and the second normal vector; calculating the coordinates of the perpendicular point based on the coordinates of the first point, the value of the scaling parameter and the second direction vector.

[0103] like Figure 4 As shown, the two-surface structure is composed of a first plane, namely plane 1, and a second plane, namely plane 2, and the intersection line between plane 1 and plane 2 is l0.

[0104] Assume that space plane 1 and space plane 2 intersect at line l0.

[0105] The expression of the intersection line l0 of plane 1 and plane 2 is:

[0106] Among them, t is the scaling parameter, l, m, and n constitute the first direction vector of the intersection line l0, (x0, y0, z0) are the coordinates of the perpendicular point between the first point and the first intersection line, the first point is the coordinates of point P on plane 1, and (x, y, z) are the coordinates of the first point.

[0107] First normal vector based on the first plane and the second normal vector of the second plane Calculate the first direction vector of the first intersection because so

[0108] Starting from point P in the first plane, draw a perpendicular line to the first intersection line l0, which is the second intersection line l1, with the perpendicular point being G. Then Then the second direction vector of l1 is: The second intersection line Wherein, P is the coordinate of the first point in the first plane.

[0109] Take the second plane, that is, a point s in plane 2, as the second point, then the distance from point P to the second plane is:

[0110] Then the value of the scaling parameter satisfies the following formula:

[0111]

[0112] The coordinates of the vertical point are calculated using the following formula:

[0113]

[0114] After the equation of the first intersection line l0 is obtained, the first intersection line l0 is projected onto the first plane to obtain the endpoint coordinates of the first intersection line.

[0115] There are many ways to determine the target intersection line that meets the preset conditions from each intersection line. In one implementation, for example, Figure 5 As shown, the following steps are included:

[0116] S206-1: Acquire the weld annotation data of the pre-stored workpiece model.

[0117] S206-2: Obtain the third direction vector in the weld annotation data.

[0118] S206-3: For each intersection line, determine the angle between the direction vector of the intersection line and the third direction vector.

[0119] S206-4: From each angle, obtain the intersection line corresponding to the minimum angle as the target intersection line.

[0120] After traversing the planes detected in the point cloud set and judging the surface structure, the plane information is stored, including the normal vector corresponding to the plane, the intercept of a point on the plane, the coordinates of a point on the plane, etc., and the first intersection line information, including the starting point, end point, and first direction vector of the first intersection line, etc. 0.

[0121] The third direction vector of the weld data in the weld annotation data of the pre-stored workpiece model is converted to the robot base coordinate system through the registration transformation matrix, and the angle between the converted third direction vector and the direction vector corresponding to each intersection line is calculated, and the intersection line corresponding to the direction vector with the smallest angle is filtered out as the target intersection line. The direction vector of the target intersection line is the direction vector of the actual weld, and the target endpoint information of the target intersection line is output, that is, the starting point and the second end point of the target intersection line, which are the coordinates of the two endpoints of the actual weld.

[0122] For the generation of weld database entries, in one example, Figure 6 As shown, the following steps are included:

[0123] S301: Determine output parameters of the target intersection line.

[0124] Among them, the output parameters include the point cloud data of the target intersection line, the coordinate information corresponding to each point cloud data, the target endpoint information of the target intersection line, the robot arm flange end position coordinates corresponding to the point cloud data corresponding to the target intersection line, and different registration matrices corresponding to different placement postures of the workpiece to be inspected corresponding to the target intersection line.

[0125] S302: Outputting and displaying the output parameters.

[0126] Exemplarily, the point cloud data of the target intersection line, the coordinate information corresponding to each point cloud data, the target endpoint information of the target intersection line, the robot arm flange end position coordinates corresponding to the point cloud data corresponding to the target intersection line, and the different alignment matrices corresponding to the different placement postures of the workpiece to be inspected corresponding to the target intersection line are obtained, and the output parameters of the target intersection line are output in the corresponding format.

[0127] For example, the point cloud data corresponding to the target intersection line ends with the ".pcd image storage" format, and "-D" and "-S" are used to distinguish between direct point cloud and downsampled point cloud. The coordinate information of each point in the point cloud data of the target intersection line is stored in a file ending with the ".txt text format" format, "-P.txt" records the target endpoint information of the target intersection line, "-T.txt" records the position of the end of the robot flange when shooting the point cloud data of the target intersection line, and "-GM.txt" records the registration matrix corresponding to different placement postures of the workpiece to be detected.

[0128] The naming rule of the target intersection line can be named according to the prefix of "workpiece number-position-lighting-weld number-number of shots-downsampling method".

[0129] Generate entries corresponding to each target intersection line of weld number 5 of workpiece GJ1 to be inspected:

[0130] ① “GJ1-position 1-lighting 1-weld number 5-shooting position 1-D”.pcd

[0131] ② "GJ1-Placement Posture 1-Lighting 1-Welding Seam Number 5-Photography Posture 1-D"-T.txt

[0132] ③ "GJ1-Placement Posture 1-Lighting 1-Welding Seam Number 5-Photography Posture 1-D"-P.txt

[0133] ④ “GJ1-position 1-lighting 1-weld number 5-shooting position 1-S”.pcd

[0134] ⑤ "GJ1-position 1-lighting 1-weld number 5-shooting position 1-S"-T.txt

[0135] ⑥ "GJ1-position 1-lighting 1-weld number 5-shooting position 1-S"-P.txt

[0136] ⑦ “GJ1-position 1-lighting 1-weld number 5”-GM.txt.

[0137] Please refer to Figure 7 The present application embodiment also provides an application Figure 1The weld annotation device 110 of the electronic device 100 includes:

[0138] A collection module 111 is used to collect point cloud data;

[0139] A determination module 112 is used to determine a registration matrix of each of the point cloud data, and convert each of the point cloud data based on each of the registration matrices to obtain a target point cloud set, wherein the target point cloud set includes each of the target point cloud data;

[0140] An acquisition module 113, used for acquiring a plane and corresponding multiple surface structures corresponding to a pre-stored workpiece model in the target point cloud set;

[0141] The determination module 112 is further used to determine the intersection lines corresponding to the planes under different surface structures; determine the target endpoint information corresponding to each of the intersection lines; and determine the target intersection line that meets the preset conditions from each of the intersection lines;

[0142] The marking module 114 is used to mark the target intersection line with the target endpoint information corresponding to the target intersection line.

[0143] The present application also provides an electronic device 100, which includes a processor 130 and a memory 120. The memory 120 stores computer executable instructions, and when the computer executable instructions are executed by the processor 130, the weld marking method is implemented.

[0144] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by the processor 130, the weld marking method is implemented.

[0145] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0146] In addition, each functional module in each embodiment of the present application can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part. If the function is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk.

[0147] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0148] The above are only various implementations of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A weld marking method, characterized in that: The method comprises: Collect point cloud data; Determine a registration matrix for each of the point cloud data, and transform each of the point cloud data based on each of the registration matrices to obtain a target point cloud set, wherein the target point cloud set includes each of the target point cloud data; Acquire a plane and a plurality of corresponding surface structures corresponding to a pre-stored workpiece model in the target point cloud set; Determine the intersection lines corresponding to the planes under different surface structures; Determine the target endpoint information corresponding to each of the intersection lines; Determine a target intersection line that meets a preset condition from each of the intersection lines; The target intersection line is marked with target endpoint information corresponding to the target intersection line.

2. The method according to claim 1, characterized in that The step of collecting point cloud data includes: The point cloud data of the workpiece to be inspected is collected based on different placement postures, different shooting postures, different fill lights and different downsampling methods.

3. The method according to claim 1, characterized in that The plane includes a first plane and a second plane, the surface structure is a two-surface structure, and the step of determining the intersection lines corresponding to the planes under different surface structures includes: Determine a product of a first normal vector and a second normal vector as a first direction vector of a first intersection line between the first plane and the second plane, wherein the first normal vector is a normal vector of the first plane, and the second normal vector is a normal vector of the second plane; determining a first point of the first plane and a second point of the second plane; Determine the coordinates of a perpendicular point of the first point and the first intersection line based on the coordinates of the first point, the coordinates of the second point, the first normal vector, and the second normal vector; The first intersection line is determined based on the first direction vector and the vertical point coordinates.

4. The method according to claim 3, characterized in that The step of determining the coordinates of the perpendicular point of the first point and the first intersection line based on the coordinates of the first point, the coordinates of the second point, the first normal vector and the second normal vector comprises: Determine a second direction vector of the second intersection line based on the first intersection line and the second normal vector, wherein the second intersection line indicates a straight line in the first plane that is perpendicular to the first intersection line and starts from the first point; determining the second intersection line based on the second direction vector, the coordinates of the first point, and a scaling parameter of the second direction vector; determining a distance from the first point to the second plane; Calculate the value of the scaling parameter based on the distance from the first point to the second plane, the second direction vector, and the second normal vector; The vertical point coordinates are calculated based on the coordinates of the first point, the value of the scaling parameter, and the second direction vector.

5. The method according to claim 4, characterized in that The value of the scaling parameter satisfies the following formula: Wherein, t is the value of the scaling parameter, D is the distance from the first point to the second plane, is the second direction vector, is the second normal vector.

6. The method according to claim 1, characterized in that The step of determining a target intersection line that meets a preset condition from each of the intersection lines comprises: Acquiring the weld annotation data of the pre-stored workpiece model; Obtaining a third direction vector in the weld annotation data; For each of the intersection lines, determining an angle between a direction vector of the intersection line and the third direction vector; From the angles, an intersection line corresponding to the minimum angle is obtained as a target intersection line.

7. The method according to claim 1, characterized in that The method further comprises: Determine the output parameters of the target intersection, wherein the output parameters include the point cloud data of the target intersection, the coordinate information corresponding to each of the point cloud data, the target endpoint information of the target intersection, the position coordinates of the end of the flange of the robot arm corresponding to the point cloud data corresponding to the target intersection, and different registration matrices corresponding to different placement postures of the workpiece to be detected corresponding to the target intersection; The output parameters are output and displayed.

8. A weld marking method, characterized in that: The device comprises: A collection module, used for collecting point cloud data; A determination module, used to determine the registration matrix of each point cloud data, and transform each point cloud data based on each registration matrix to obtain a target point cloud set, wherein the target point cloud set contains each target point cloud data; An acquisition module, used for acquiring a plane and corresponding multiple surface structures corresponding to a pre-stored workpiece model in the target point cloud set; The determination module is further used to determine the intersection lines corresponding to the planes under different surface structures; determine the target endpoint information corresponding to each of the intersection lines; and determine the target intersection line that meets the preset conditions from each of the intersection lines; The marking module is used to mark the target intersection line with the target endpoint information corresponding to the target intersection line.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method according to any one of claims 1 to 7 when executing the computer program.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.