Weld information extraction method, device, equipment, storage medium and program product

Abstract

The application relates to a weld seam information extraction method, device, equipment, storage medium and program product. The method comprises the following steps: obtaining a workpiece point cloud of a target workpiece, fitting a fitting weld seam based on the workpiece point cloud, and the workpiece point cloud comprising a bottom plate plane point cloud and a rib plate upper surface point cloud; constructing a first bounding box based on the fitting weld seam, determining a weld edge application type of the fitting weld seam according to a first point cloud of the bottom plate plane point cloud in the first bounding box; constructing a second bounding box based on the fitting weld seam, determining a rib plate height according to a height difference of a second point cloud of the rib plate upper surface point cloud in the second bounding box; and determining the fitting weld seam, the weld edge application type and the rib plate height as weld seam information. The method can improve the weld seam information extraction efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent manufacturing, and in particular to a welding seam information extraction method, device, equipment, storage medium and program product. BACKGROUND

[0002] In the field of modern shipbuilding, small group assembly is a production management method and production process adopted to accelerate shipbuilding speed, improve shipbuilding quality and form mass production, mainly including assembly, welding and repair grinding of bottom plates and rib plates.

[0003] Traditional welding seam information extraction is heavily dependent on manual demonstration, and requires knowing the model, welding seam position and type of a workpiece in advance, which is manually set by an operator, thereby resulting in low efficiency of extracting welding seam information. SUMMARY

[0004] Therefore, it is necessary to provide a welding seam information extraction method, device, equipment, storage medium and program product capable of improving the efficiency of extracting welding seam information.

[0005] In a first aspect, the present application provides a welding seam information extraction method, comprising:

[0006] obtaining a workpiece point cloud of a target workpiece, fitting a fitted welding seam based on the workpiece point cloud, and the workpiece point cloud comprising a bottom plate plane point cloud and a rib plate upper surface point cloud;

[0007] constructing a first bounding box based on the fitted welding seam, and determining a welding edge application type of the fitted welding seam according to a first point cloud of the bottom plate plane point cloud in the first bounding box;

[0008] constructing a second bounding box based on the fitted welding seam, and determining a rib plate height according to a height difference of a second point cloud of the rib plate upper surface point cloud in the second bounding box.

[0009] In one embodiment, constructing the first bounding box based on the fitted welding seam, and determining the welding edge application type of the fitted welding seam according to the first point cloud of the bottom plate plane point cloud in the first bounding box comprises:

[0010] selecting at least one first search point on the fitted welding seam, and constructing a first bounding box with the first search point as the center;

[0011] determining a mass point of the first point cloud of the bottom plate plane point cloud in the first bounding box;

[0012] in a case where a distance between the mass point of the first point cloud and the search point is less than a predetermined distance threshold, determining that the welding edge application type of the fitted welding seam is double-sided welding;

[0013] In a case where the distance between the first point cloud and the search point is greater than or equal to a predetermined distance threshold, it is determined that the welding edge application type of the fitting weld is single-sided welding.

[0014] In one embodiment, a second bounding box is constructed based on the fitting weld, and the determination of the web height based on the height difference of the web upper surface point cloud within the second bounding box includes:

[0015] At least one second search point is selected on the fitting weld, and a second bounding box is constructed with the second search point as the center;

[0016] The lowest point of the second point cloud of the web upper surface point cloud within the second bounding box is determined, and the neighborhood points within a predetermined radius are searched with the lowest point as the center;

[0017] In a case where the number of points in the neighborhood points is greater than a set point number threshold, the height mean of each neighborhood point is determined;

[0018] The difference between the height mean and the height value of the second search point is determined as the web height.

[0019] In one embodiment, the fitting weld is fitted based on the workpiece point cloud, which includes:

[0020] The bottom plate plane point cloud and at least one web upper surface point cloud are extracted from the workpiece point cloud, and the web upper surface point cloud is projected onto the bottom plate plane of the target workpiece to obtain a web projection point cloud;

[0021] The welding seam endpoints are obtained by traversing the web projection point cloud, and the type of the welding seam to be extracted is determined based on the welding seam endpoints;

[0022] In a case where the type of the welding seam to be extracted is a straight-line weld, iterative straight-line extraction is performed based on the web projection point cloud to obtain a fitting line segment;

[0023] In a case where the web projection point cloud has multiple corresponding fitting line segments, the fitting line segments are merged to obtain a fitting weld.

[0024] In one embodiment, in a case where the web projection point cloud has multiple corresponding fitting line segments, the fitting line segments are merged to obtain a fitting weld, which includes:

[0025] In a case where the web projection point cloud has multiple corresponding fitting line segments, the line segment included angle between each pair of fitting line segments and the shortest distance from the endpoint of one fitting line segment to another fitting line segment in a pair of fitting line segments are determined;

[0026] In a case where the line segment included angle is less than a set angle threshold and the shortest distance is less than a predetermined distance threshold, an adjacency graph of a pair of fitting line segments is constructed;

[0027] Based on the adjacency graph, the fitting line segments to be merged are determined;

[0028] merge the inner points of the to-be-merged fitting line segments into a to-be-fitted point cloud, and determine a point cloud boundary of the to-be-fitted point cloud;

[0029] perform straight line fitting based on the point cloud boundary of the to-be-fitted point cloud to obtain a fitted weld.

[0030] In one embodiment, the method further comprises:

[0031] obtaining a relative orientation of the welding equipment and the first welding direction, sampling the fitted weld according to a preset welding distance to obtain a plurality of welding points;

[0032] In the case where the welding edge application type is single-side welding, if the to-be-welded side of the single-side welding is consistent with the relative orientation, the first welding direction is determined as the weld advancing direction;

[0033] If the to-be-welded side of the single-side welding is inconsistent with the relative orientation, the second welding direction is determined as the weld advancing direction, wherein the second welding direction is opposite to the first welding direction.

[0034] In the case where the welding edge application type is double-side welding, the first welding direction and the second welding direction are determined as the weld advancing direction.

[0035] sequencing the welding points based on the weld advancing direction to obtain a welding point sequence.

[0036] In a second aspect, the present application also provides a welding seam information extraction device, comprising:

[0037] a welding seam fitting module, configured to obtain a workpiece point cloud of a target workpiece, and fit a fitted welding seam based on the workpiece point cloud, wherein the workpiece point cloud comprises a bottom plate plane point cloud and a rib plate upper surface point cloud;

[0038] a welding edge application type determination module, configured to construct a first bounding box based on the fitted welding seam, and determine a welding edge application type of the fitted welding seam according to a first point cloud of the bottom plate plane point cloud in the first bounding box;

[0039] a rib plate height determination module, configured to construct a second bounding box based on the fitted welding seam, and determine a rib plate height according to a height difference of a second point cloud of the rib plate upper surface point cloud in the second bounding box.

[0040] In a third aspect, the present application also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0041] obtaining a workpiece point cloud of a target workpiece, and fitting a fitted welding seam based on the workpiece point cloud, wherein the workpiece point cloud comprises a bottom plate plane point cloud and a rib plate upper surface point cloud;

[0042] construct a first bounding box based on the fitted weld, determine a weld edge application type of the fitted weld according to first point clouds of the bottom plate plane point cloud within the first bounding box;

[0043] construct a second bounding box based on the fitted weld, determine a height of the rib plate according to height differences of second point clouds of the rib plate upper surface point cloud within the second bounding box.

[0044] In a fourth aspect, the present application further provides a computer readable storage medium, having a computer program stored thereon, the computer program being executed by a processor to implement the following steps:

[0045] obtain a workpiece point cloud of a target workpiece, fit a fitted weld based on the workpiece point cloud, the workpiece point cloud comprising a bottom plate plane point cloud and a rib plate upper surface point cloud;

[0046] construct a first bounding box based on the fitted weld, determine a weld edge application type of the fitted weld according to first point clouds of the bottom plate plane point cloud within the first bounding box;

[0047] construct a second bounding box based on the fitted weld, determine a height of the rib plate according to height differences of second point clouds of the rib plate upper surface point cloud within the second bounding box.

[0048] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, the computer program being executed by a processor to implement the following steps:

[0049] obtain a workpiece point cloud of a target workpiece, fit a fitted weld based on the workpiece point cloud, the workpiece point cloud comprising a bottom plate plane point cloud and a rib plate upper surface point cloud;

[0050] construct a first bounding box based on the fitted weld, determine a weld edge application type of the fitted weld according to first point clouds of the bottom plate plane point cloud within the first bounding box;

[0051] construct a second bounding box based on the fitted weld, determine a height of the rib plate according to height differences of second point clouds of the rib plate upper surface point cloud within the second bounding box.

[0052] The aforementioned weld information extraction method, apparatus, equipment, storage medium, and program products obtain a workpiece point cloud of the target workpiece, and then fit a weld seam based on the workpiece point cloud. The workpiece point cloud includes the bottom plate planar point cloud and the upper surface point cloud of the stiffener plate. A first bounding box is then constructed based on the fitted weld seam. The weld edge application type of the fitted weld seam is determined based on the first point cloud within the first bounding box of the bottom plate planar point cloud. Thus, the first bounding box is constructed based on the fitted weld seam. Based on the bottom plate planar point cloud within the first bounding box, it can be identified whether there is a bottom plate on both sides of the fitted weld seam, i.e., whether bilateral welding (weld edge application type) is required. On the other hand, a second bounding box can be constructed based on the fitted weld. The height of the stiffener is determined by the height difference between the point cloud on the upper surface of the stiffener and the second point cloud within the second bounding box. Thus, a second bounding box is constructed based on the fitted weld. The height of the stiffener can be identified based on the height difference between the midpoints of the point cloud on the stiffener plane within the second bounding box. After the fitted weld is obtained by fitting the workpiece point cloud of the target workpiece, the method of distinguishing the weld edge application type and identifying the stiffener height by constructing a bounding box based on the fitted weld reduces the influence of messy point clouds. It is also highly adaptable to the actual deformation and installation deviation of the target workpiece, greatly reducing manual intervention and improving the efficiency of weld information extraction. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is an application environment diagram of the weld information extraction method in one embodiment;

[0055] Figure 2 This is a flowchart illustrating a weld information extraction method in one embodiment;

[0056] Figure 3 This is a schematic diagram of a workpiece point cloud in one embodiment;

[0057] Figure 4 This is a schematic diagram of the point cloud projection of the stiffener in one embodiment;

[0058] Figure 5 This is a schematic diagram of a scenario where a line segment is fitted, as shown in one embodiment.

[0059] Figure 6 This is a schematic diagram of a scenario for fitting a weld in one embodiment;

[0060] Figure 7A scene diagram of a first bounding box in an embodiment;

[0061] Figure 8 A flow diagram of a weld seam information extraction method in another embodiment;

[0062] Figure 9 A flow diagram of a weld seam information extraction method in yet another embodiment;

[0063] Figure 10 A flow diagram of a weld seam information extraction method in a specific embodiment;

[0064] Figure 11 A structure block diagram of a weld seam information extraction device in an embodiment;

[0065] Figure 12 An internal structure diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0066] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0067] It should be noted that the terms "first", "second", etc. used in the present application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "include" and "have" and any variations thereof used in the present application are intended to cover non-exclusive inclusion. The term "multiple" used in the present application refers to two and more than two. The term "and / or" used in the present application refers to one of the options or any combination of multiple options.

[0068] The weld seam information extraction method provided by the embodiments of the present application can be applied to, for example, Figure 1The application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data required by the server 104 to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers. It should be noted that the welding seam information extraction method provided in the embodiments of the present application can be executed by the terminal 102, or by the server 104, or by the terminal 102 and the server together. Taking the execution of the terminal 102 and the server 104 together as an example, the terminal 102 can request the workpiece point cloud of the target workpiece from the server 104, and the server 104 sends the workpiece point cloud of the target workpiece to the terminal 102. After the terminal 102, the terminal 102 can fit the fitting welding seam based on the workpiece point cloud. The workpiece point cloud includes the bottom plate plane point cloud and the rib plate upper surface point cloud. Based on the fitting welding seam, a first bounding box is constructed. According to the first point cloud of the bottom plate plane point cloud in the first bounding box, the welding edge application type of the fitting welding seam is determined. Based on the fitting welding seam, a second bounding box is constructed. According to the height difference of the second point cloud of the rib plate upper surface point cloud in the second bounding box, the rib plate height is determined. Among them, the terminal 102 can be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things device can be a smart television, a smart robot, a projection device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The head-mounted device can be a virtual reality (Virtual Reality, VR) device, an augmented reality (Augmented Reality, AR) device, smart glasses, etc. The server 104 can be a standalone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0069] In an exemplary embodiment, as shown in Figure 2 , a welding seam information extraction method is provided. Taking the terminal 102 in Figure 1 as an example, the method includes the following steps 210 to 240. Among them:

[0070] Step 210, obtaining the workpiece point cloud of the target workpiece, fitting the fitting welding seam based on the workpiece point cloud, and the workpiece point cloud includes the bottom plate plane point cloud and the rib plate upper surface point cloud.

[0071] Among them, the target workpiece is a workpiece whose welding seam information is expected to be extracted, and the workpiece includes a bottom plate and at least one rib plate arranged on the bottom plate. The workpiece point cloud is a point cloud obtained by scanning the target workpiece by means of a scanning device (such as a laser scanner, a structured light camera). The workpiece point cloud includes the bottom plate plane point cloud and the rib plate upper surface point cloud. The bottom plate plane point cloud is the point cloud corresponding to the bottom plate plane of the target workpiece, and the rib plate upper surface point cloud is the point cloud corresponding to the upper surface of the rib plate of the target workpiece.

[0072] Exemplarily, after obtaining the workpiece point cloud of the target workpiece, the workpiece point cloud can be used to extract the bottom plate plane point cloud and at least one rib plate upper surface point cloud, and the rib plate upper surface point cloud can be projected onto the bottom plate plane of the target workpiece to obtain a rib plate projection point cloud. The rib plate projection point cloud is traversed to obtain a welding seam endpoint. Based on the welding seam endpoint, the type of the welding seam to be extracted is determined. In the case that the type of the welding seam to be extracted is a straight-line welding seam, iterative straight-line extraction is performed based on the rib plate projection point cloud to obtain a fitting line segment. In the case that the rib plate projection point cloud has only one corresponding fitting line segment, the fitting line segment is taken as a fitting welding seam. In the case that the rib plate projection point cloud has a plurality of corresponding fitting line segments, the fitting line segments are merged to obtain a fitting welding seam. In the case that the type of the welding seam to be extracted is a curve welding seam, the rib plate projection point cloud can be segmented and linearly fitted to obtain a local line segment sequence. The endpoints of each local line segment in the local line segment sequence are taken as control points, and a curve welding seam is fitted. For example, a predetermined minimum value can be set as a consensus set threshold value or a maximum number of iterations to obtain a local line segment by fitting the rib plate projection point cloud. The purpose of this is to avoid fitting the entire curve with a long straight line, but to fit the straightest part of the curve (i.e., a local line segment). After the first and most prominent local line segment (such as a tangent of a segment of the curve) is fitted, the inner points of the local line segment are removed from the rib plate projection point cloud, and the straightest segment of the remaining point cloud in the rib plate projection point cloud is fitted again. The fitting process is repeated until the number of remaining points in the rib plate projection point cloud is less than a set point number threshold value, and a series of ordered (or can be ordered in spatial order) local line segments (i.e., a local line segment sequence) are obtained. The endpoints of each local line segment in the local line segment sequence are distributed on the original curve corresponding to the welding seam. The endpoints of each local line segment in the local line segment sequence are taken as control points, and a B-spline curve or a Bezier curve is used to fit the control points to generate a smooth curve that passes through (or approximates) the control points, thereby obtaining a curve welding seam. Figure 3 Exemplarily, after obtaining the workpiece point cloud of the target workpiece, the workpiece point cloud can be used to extract the bottom plate plane point cloud and at least one rib plate upper surface point cloud, and the rib plate upper surface point cloud can be projected onto the bottom plate plane of the target workpiece to obtain a rib plate projection point cloud. The rib plate projection point cloud is traversed to obtain a welding seam endpoint. Based on the welding seam endpoint, the type of the welding seam to be extracted is determined. In the case that the type of the welding seam to be extracted is a straight-line welding seam, iterative straight-line extraction is performed based on the rib plate projection point cloud to obtain a fitting line segment. In the case that the rib plate projection point cloud has only one corresponding fitting line segment, the fitting line segment is taken as a fitting welding seam. In the case that the rib plate projection point cloud has a plurality of corresponding fitting line segments, the fitting line segments are merged to obtain a fitting welding seam. In the case that the type of the welding seam to be extracted is a curve welding seam, the rib plate projection point cloud can be segmented and linearly fitted to obtain a local line segment sequence. The endpoints of each local line segment in the local line segment sequence are taken as control points, and a curve welding seam is fitted. For example, a predetermined minimum value can be set as a consensus set threshold value or a maximum number of iterations to obtain a local line segment by fitting the rib plate projection point cloud. The purpose of this is to avoid fitting the entire curve with a long straight line, but to fit the straightest part of the curve (i.e., a local line segment). After the first and most prominent local line segment (such as a tangent of a segment of the curve) is fitted, the inner points of the local line segment are removed from the rib plate projection point cloud, and the straightest segment of the remaining point cloud in the rib plate projection point cloud is fitted again. The fitting process is repeated until the number of remaining points in the rib plate projection point cloud is less than a set point number threshold value, and a series of ordered (or can be ordered in spatial order) local line segments (i.e., a local line segment sequence) are obtained. The endpoints of each local line segment in the local line segment sequence are distributed on the original curve corresponding to the welding seam. The endpoints of each local line segment in the local line segment sequence are taken as control points, and a B-spline curve or a Bezier curve is used to fit the control points to generate a smooth curve that passes through (or approximates) the control points, thereby obtaining a curve welding seam.Figure 3 The set of mid-white points is the workpiece point cloud.

[0073] In some embodiments, fitting the fitting weld based on the workpiece point cloud comprises steps 211-214:

[0074] Step 211, extracting the bottom plate plane point cloud and at least one rib plate upper surface point cloud from the workpiece point cloud, and projecting the rib plate upper surface point cloud onto the bottom plate plane of the target workpiece to obtain the rib plate projected point cloud.

[0075] Step 212, traversing the rib plate projected point cloud to obtain weld end points, and determining the type of weld to be extracted based on the weld end points.

[0076] Step 213, in the case where the type of weld to be extracted is a straight weld, performing iterative straight line extraction based on the rib plate projected point cloud to obtain a fitting line segment.

[0077] Step 214, in the case where there are multiple corresponding fitting line segments in the rib plate projected point cloud, merging the fitting line segments to obtain the fitting weld.

[0078] The type of weld to be extracted is a type describing the shape of the weld, such as a straight weld, a curved weld, an intersecting weld (i.e., a weld where the rib plate has an intersection point), etc.

[0079] In this embodiment, the bottom plate plane can be fitted based on the workpiece point cloud, the bottom plate plane point cloud and the rib plate upper surface point cloud can be separated from the workpiece point cloud based on the bottom plate plane, and the rib plate upper surface point cloud can be projected onto the bottom plate plane. After the projection, the projected rib plate upper surface point cloud can be clustered and segmented to obtain at least one rib plate projected point cloud. For example, the bottom plate plane can be fitted based on the workpiece point cloud, and the points within the bottom plate plane can be extracted by setting a plane distance threshold to obtain the bottom plate plane point cloud. The remaining point cloud in the workpiece point cloud, excluding the bottom plate plane point cloud, is the rib plate upper surface point, which may also contain some interference points. Then, the rib plate upper surface point cloud can be projected onto the bottom plate plane, and filtering, denoising and clustering operations can be performed to segment the rib plate upper surface point cloud to obtain the rib plate projected point cloud corresponding to each rib plate on the bottom plate. Of course, to reduce the point cloud density and improve the operation efficiency, a downsampling operation, such as uniform downsampling, particle downsampling, etc., can be performed before clustering. The rib plate projected point cloud after downsampling is as follows: Figure 4As shown, the white points in the figure are the points in the rib plate projection point cloud after downsampling. For example, since the scanning of the target workpiece is performed from top to bottom, the Z axis in the world coordinate system is vertically downward, and therefore the embodiment can also find the minimum value of the Z value of the points in the bottom plate plane, and use this value as the maximum Z value to perform straight-through filtering on the remaining point cloud in the workpiece point cloud excluding the point cloud of the bottom plate plane, to filter out the interference point cloud below the bottom plate plane, to obtain the rib plate upper surface point cloud, and then project the rib plate upper surface point cloud onto the bottom plate plane, and use statistical filtering to reduce noise, uniformly downsample to reduce point cloud density, and cluster segmentation to separate the rib plate upper surface point cloud into individual rib plate projection point clouds, to obtain a rib plate on the bottom plate corresponding to each segmented rib plate projection point cloud. Furthermore, the embodiment can traverse the rib plate projection point cloud to obtain the weld end points. For example, a plurality of local windows can be constructed based on the rib plate projection point cloud, where the local window is a preset search range centered on a specified point, such as a range with a radius of r centered on a specified point P. The cosine values of the vectors formed by the specified point and other points in the local window are calculated, and if there is only a positive value or only a negative value, the specified point is determined to be a candidate end point. The candidate end points are clustered again to obtain clusters, and for each cluster, the geometric center point is taken as the weld end point to avoid multiple end points from appearing repeatedly. Then, the number of end points of the weld end points can be used to determine the type of weld to be extracted.

[0080] For example, based on the weld end points, determining the type of weld to be extracted includes: in the absence of weld end points, determining that the type of weld to be extracted is a closed curve weld; in the presence of more than two weld end points, determining that the type of weld to be extracted is an intersecting weld; in the presence of more than two weld end points, constructing a weld straight line based on the two weld end points, determining the average value of the perpendicular distance from each point in the rib plate projection point cloud to the weld straight line; in the case where the average value of the perpendicular distance is less than a predetermined perpendicular distance threshold, determining that the type of weld to be extracted is a straight line weld; and in the case where the average value of the perpendicular distance is greater than or equal to the predetermined perpendicular distance threshold, determining that the type of weld to be extracted is a curve weld.

[0081] As shown in Figure 5 Since a straight line weld may result in multiple fitting line segments in the case of a wide rib plate, the embodiment can perform iterative straight line extraction based on the rib plate projection point cloud to obtain fitting line segments in the case where the type of weld to be extracted is a straight line weld, and then in the case where there are multiple corresponding fitting line segments in the rib plate projection point cloud (as shown in Figure 5In the case that there are multiple fitting line segments (as shown by the multiple fitting line segments in the middle red dashed box), the fitting line segments are merged to obtain a fitting weld. Exemplarily, the manner of merging the fitting line segments can be to set an angle threshold, a distance threshold, in the case that the angle between the fitting line segments is less than the angle threshold or the distance between the fitting line segments is less than the distance threshold, the fitting line segments are redundantly excluded, such as merging the multiple fitting line segments that meet the case into one fitting weld, only keeping the fitting line segment with the longest length as the fitting weld, etc., to obtain the fitting weld. For example, the embodiment can determine the line segment angle between each pair of fitting line segments and the shortest distance from the end point of one fitting line segment in a pair of fitting line segments to another fitting line segment in the case that there are multiple corresponding fitting line segments in the rib plate projection point cloud. In the case that the line segment angle is less than the set angle threshold and the shortest distance is less than the predetermined distance threshold, an adjacency graph of the pair of fitting line segments is constructed, that is, it is indicated that the pair of fitting line segments is actually one weld, wherein the adjacency graph is a graph obtained after connecting the pair of fitting line segments. Since one weld can also be fitted to obtain two or more fitting line segments, based on the adjacency graph, the fitting line segments to be merged are determined, wherein the fitting line segments to be merged are the fitting line segments connected in the adjacency graph. The adjacency graph is essentially to realize the continuous merging of multiple line segments, such as fitting line segment A connecting fitting line segment B, fitting line segment B connecting fitting line segment C in the adjacency graph, then fitting line segment A, fitting line segment B and fitting line segment C are the fitting line segments to be merged.

[0082] Further, the embodiment can merge the in-points of the fitting line segments to be merged into a point cloud to be fitted, wherein the in-points of the fitting line segments are the points in the rib plate projection point cloud used to fit the fitting line segments, and the point cloud boundary of the point cloud to be fitted can be the minimum bounding box of the point cloud to be fitted. Exemplarily, the embodiment can take the center of the point cloud to be fitted as the center to construct a minimum cuboid (i.e. minimum bounding box) wrapping the point cloud to be fitted, and then the eight vertices of the minimum bounding box can be taken as the point cloud boundary, and of course some points on each face of the minimum bounding box and the set of eight vertices can also be taken as the point cloud boundary. Then the straight line fitting is performed based on the point cloud boundary of the point cloud to be fitted to obtain the fitting weld. The fitting weld is as shown by the line segments in the middle red dashed box. Figure 6

[0083] On the one hand, the embodiment realizes the screening of the multiple line segments corresponding to the same weld by constructing the adjacency graph of a pair of fitting line segments in the case that the line segment angle is less than the set angle threshold and the shortest distance is less than the predetermined distance threshold, and then determining the fitting line segments to be merged based on the adjacency graph. On the other hand, the fitting of the point cloud boundary of the point cloud to be fitted corresponding to all the fitting line segments to be merged is used as the representative, which greatly improves the calculation efficiency of the fitting weld and significantly enhances the accuracy and robustness of the fitting weld.

[0084] ​Step 220, constructing a first bounding box based on the fitted weld, and determining the welding edge application type of the fitted weld according to a first point cloud of the bottom plane point cloud in the first bounding box.

[0085] wherein the first bounding box is a bounding box with one point on the fitted weld as the center, the number of the first bounding boxes is at least one, and the first point cloud is a set of points of the bottom plane point cloud in the first bounding box.

[0086] As shown in Figure 7 three first bounding boxes are respectively constructed with two end points and one middle point on the fitted weld as the center. The embodiment can determine the first point cloud of the bottom plane point cloud in the first bounding box, and determine the distribution of the first point cloud on both sides of the fitted weld. According to the distribution of the first point cloud on both sides of the fitted weld, the welding edge application type of the fitted weld is determined. For example, the embodiment can select at least one first search point on the fitted weld, construct a first bounding box with the first search point as the center, and determine the mass point of the first point cloud of the bottom plane point cloud in the first bounding box. If the distance between the mass point of the first point cloud and the search point is less than a predetermined distance threshold, it indicates that the first point cloud is evenly distributed on both sides of the weld, i.e., the number of points of the first point cloud on both sides of the fitted weld is similar, which indicates that there is a bottom plate on both sides of the fitted weld, and it is determined that the welding edge application type of the fitted weld is double-sided welding. If the distance between the mass point of the first point cloud and the search point is greater than or equal to the predetermined distance threshold, it indicates that the first point cloud is extremely unevenly distributed on both sides of the weld, i.e., most of the points in the first point cloud are distributed on one side, and there are very few points on the other side, which indicates that there is only a bottom plate on one side of the fitted weld, and it is determined that the welding edge application type of the fitted weld is single-sided welding. In addition, the embodiment can also count the first point number and the second point number of the first point cloud on both sides of the fitted weld. If the number difference between the first point number and the second point number is less than a predetermined number threshold, it is determined that the welding edge application type of the fitted weld is double-sided welding. If the number difference between the first point number and the second point number is equal to or greater than the predetermined number threshold, it is determined that the welding edge application type of the fitted weld is single-sided welding.

[0087] In some embodiments, constructing a first bounding box based on the fitted weld, and determining the welding edge application type of the fitted weld according to a first point cloud of the bottom plane point cloud in the first bounding box includes steps 221 to 224:

[0088] Step 221, selecting at least one first search point on the fitted weld, and constructing a first bounding box with the first search point as the center.

[0089] Step 222, determining the mass point of the first point cloud of the bottom plane point cloud in the first bounding box.

[0090] Step 223, in the case that the distance between the mass point of the first point cloud and the search point is less than the predetermined distance threshold, it is determined that the welding edge application type of the fitted weld is double-sided welding.

[0091] Step 224, in the case that the distance between the mass point of the first point cloud and the search point is greater than or equal to the predetermined distance threshold, it is determined that the welding edge application type of the fitted weld is single-sided welding.

[0092] The embodiment can select at least one first search point on the fitted weld to construct a first bounding box centered on the first search point. Illustratively, the embodiment can select two end points and a center point of the fitted weld as the first search point. Then the mass point of the first point cloud of the bottom plate plane point cloud in the first bounding box is determined. Illustratively, the embodiment can obtain the arithmetic mean of the coordinates of all points in the first point cloud to obtain the mass point of the first point cloud. In the case that the distance between the mass point of the first point cloud and the search point is less than the predetermined distance threshold, it indicates that the first point cloud is evenly distributed on both sides of the fitted weld, i.e. the number of points of the first point cloud on both sides of the fitted weld is similar, which indicates that there is a bottom plate on both sides of the fitted weld, and it is determined that the welding edge application type of the fitted weld is double-sided welding. In the case that the distance between the mass point of the first point cloud and the search point is greater than or equal to the predetermined distance threshold, it indicates that the first point cloud is extremely unevenly distributed on both sides of the fitted weld, i.e. most of the points in the first point cloud are distributed on one side and there are very few points on the other side, which indicates that there is only one side of the fitted weld with a bottom plate, and it is determined that the welding edge application type of the fitted weld is single-sided welding.

[0093] The embodiment can construct a first bounding box based on the fitted weld, and identify the distribution of the first point cloud on both sides of the fitted weld according to the first point cloud of the bottom plate plane point cloud in the first bounding box, so as to accurately identify whether there is a bottom plate on only one side or on both sides of the fitted weld, and to determine whether the welding edge application type of the fitted weld is single-sided welding or double-sided welding.

[0094] Step 230, constructing a second bounding box based on the fitted weld, and determining the height of the rib plate based on the height difference of the second point cloud of the rib plate upper surface point cloud in the second bounding box.

[0095] The second bounding box is a bounding box centered on a point on the fitted weld, the number of the second bounding box is at least one, and the second point cloud is a set of points of the rib plate upper surface point cloud in the second bounding box.

[0096] The embodiment can select at least one second search point on the fitted weld seam, construct a second bounding box with the second search point as the center, and exemplarily, the embodiment can select two end points and a center point of the fitted weld seam as the second search point. Then, a second point cloud of the gusset upper surface point cloud in the second bounding box is determined. Exemplarily, the embodiment can select a lowest point of the second point cloud, find neighborhood points within a predetermined radius with the lowest point as the center, and in a case where a point quantity of the neighborhood points is greater than a set point quantity threshold, determine a height mean value of each neighborhood point, and determine a difference between the height mean value and a height value of the second search point as the gusset height. The embodiment can also select a first quantity of lowest points and a second quantity of highest points in the first point cloud, determine a first height mean value of the first quantity of lowest points and a second height mean value of the second quantity of highest points, and determine a difference between the first height mean value and the second height mean value as the gusset height.

[0097] In some embodiments, determining the gusset height based on a height difference of a second point cloud of the gusset upper surface point cloud in a second bounding box constructed based on the fitted weld seam comprises steps 231 to 234:

[0098] Step 231, selecting at least one second search point on the fitted weld seam, and constructing a second bounding box with the second search point as the center.

[0099] Step 232, determining a lowest point of the second point cloud of the gusset upper surface point cloud in the second bounding box, and finding neighborhood points within a predetermined radius with the lowest point as the center.

[0100] Step 233, in a case where a point quantity of the neighborhood points is greater than a set point quantity threshold, determining a height mean value of each neighborhood point.

[0101] Step 234, determining a difference between the height mean value and a height value of the second search point as the gusset height.

[0102] Wherein the predetermined radius is a radius value for making the second bounding box encompass the entire gusset.

[0103] In this embodiment, at least one second search point can be selected on the fitted weld, and a second bounding box can be constructed with the second search point as the center. For example, in this embodiment, the two endpoints and the midpoint of the fitted weld can be selected as the second search points. Of course, in this embodiment, at least two predetermined points can be selected at equal intervals on the fitted weld. For each predetermined point, it is first moved a predetermined distance along the Z-axis and then used as the second search point. A cuboid bounding box centered on the second search point is used as the second bounding box. Then, the second point cloud located within the second bounding box in the point cloud of the upper surface of the stiffener is searched, and the lowest point of the second point cloud in the second bounding box (i.e., the point with the smallest Z value) is determined. With the lowest point as the center, neighboring points within a predetermined radius (i.e., points in the second point cloud other than the lowest point that are within the predetermined radius) are searched. To avoid errors caused by a small number of points, in this embodiment, when the number of neighboring points is greater than a set threshold, the average height of each neighboring point is determined, and the difference between the average height and the height value of the second search point is determined as the stiffener height.

[0104] Therefore, this embodiment can determine at least one of the following as weld information: the fitted weld describing the spatial location of the weld, the weld edge application type describing the application method of the weld edge, and the stiffener height, for use in subsequent welding work, so as to efficiently provide more comprehensive weld information.

[0105] In the above-described weld information extraction method, a workpiece point cloud of the target workpiece is acquired, and a fitted weld is obtained based on the workpiece point cloud. The workpiece point cloud includes the bottom plate planar point cloud and the upper surface point cloud of the stiffener plate. Then, a first bounding box is constructed based on the fitted weld. According to the first point cloud of the bottom plate planar point cloud within the first bounding box, the weld edge application type of the fitted weld is determined. Thus, the first bounding box is constructed based on the fitted weld. Based on the bottom plate planar point cloud within the first bounding box, it can be identified whether there is a bottom plate on both sides of the fitted weld, i.e., whether bilateral welding (weld edge application type) is required. On the other hand, a second bounding box can be constructed based on the fitted weld. The height of the stiffener is determined by the height difference between the point cloud on the upper surface of the stiffener and the second point cloud within the second bounding box. Thus, a second bounding box is constructed based on the fitted weld. The height of the stiffener can be identified based on the height difference between the midpoints of the point cloud on the stiffener plane within the second bounding box. After the fitted weld is obtained by fitting the workpiece point cloud of the target workpiece, the method of distinguishing the weld edge application type and identifying the stiffener height by constructing a bounding box based on the fitted weld reduces the influence of messy point clouds. It is also highly adaptable to the actual deformation and installation deviation of the target workpiece, greatly reducing manual intervention and improving the efficiency of weld information extraction.

[0106] In some embodiments, such as Figure 8 As shown, when there are multiple corresponding fitted line segments in the point cloud on the upper surface of the stiffener, the fitted line segments are merged to obtain the fitted weld, including steps 310 to 350:

[0107] Step 310, in the case that there are multiple corresponding fitting line segments in the rib plate projection point cloud, the line segment included angle between each pair of fitting line segments and the shortest distance from the end point of one fitting line segment to another fitting line segment in a pair of fitting line segments are determined.

[0108] Step 320, in the case that the line segment included angle is less than a set angle threshold and the shortest distance is less than a predetermined distance threshold, an adjacency graph of a pair of fitting line segments is constructed.

[0109] Step 330, based on the adjacency graph, the fitting line segments to be merged are determined.

[0110] Step 340, the inliers of the fitting line segments to be merged are merged into a to-be-fitted point cloud, and the point cloud boundary of the to-be-fitted point cloud is determined.

[0111] Step 350, based on the point cloud boundary of the to-be-fitted point cloud, straight line fitting is performed to obtain a fitted weld.

[0112] In the case that there are multiple corresponding fitting line segments in the rib plate projection point cloud, the line segment included angle between each pair of fitting line segments and the shortest distance from the end point of one fitting line segment to another fitting line segment in a pair of fitting line segments are determined. In the case that the line segment included angle is less than a set angle threshold and the shortest distance is less than a predetermined distance threshold, an adjacency graph of a pair of fitting line segments is constructed, which indicates that the pair of fitting line segments is actually a weld, wherein the adjacency graph is a graph obtained after connecting the pair of fitting line segments. Since a weld can also be fitted to obtain two or more fitting line segments, based on the adjacency graph, the fitting line segments to be merged are determined, wherein the fitting line segments to be merged are the fitting line segments connected in the adjacency graph. The adjacency graph is essentially to realize the continuous merging of multiple line segments, for example, in the adjacency graph, fitting line segment A is connected to fitting line segment B, and fitting line segment B is connected to fitting line segment C, then fitting line segment A, fitting line segment B and fitting line segment C are the fitting line segments to be merged. Further, the inliers of the fitting line segments to be merged can be merged into a to-be-fitted point cloud, wherein the inliers of the fitting line segments are the points in the rib plate upper surface point cloud used to fit the fitting line segments, and the point cloud boundary of the to-be-fitted point cloud can be the minimum bounding box of the to-be-fitted point cloud. Illustratively, a minimum cuboid (i.e. minimum bounding box) wrapping the to-be-fitted point cloud can be constructed with the center of the to-be-fitted point cloud as the center, and then the 8 vertices of the minimum bounding box can be used as the point cloud boundary, or some points on each face of the minimum bounding box and the set of 8 vertices can also be used as the point cloud boundary. Then, based on the point cloud boundary of the to-be-fitted point cloud, straight line fitting is performed to obtain a fitted weld.

[0113] In one exemplary embodiment, as shown in FIG. 4, the method further includes steps 410-450. Wherein: Figure 9

[0114] ​At step 410, the relative position of the welding equipment and the first welding direction is acquired, and the fitting weld is sampled according to the preset welding distance to obtain a plurality of welding points.

[0115] At step 420, in the case of the welding edge application type being single-side welding, if the to-be-welded side of the single-side welding is consistent with the relative position, the first welding direction is determined as the weld advancing direction.

[0116] At step 430, if the to-be-welded side of the single-side welding is inconsistent with the relative position, the second welding direction is determined as the weld advancing direction, wherein the second welding direction is opposite to the first welding direction.

[0117] At step 440, in the case of the welding edge application type being double-side welding, the first welding direction and the second welding direction are determined as the weld advancing direction.

[0118] At step 450, the welding points are sorted based on the weld advancing direction to form a welding point sequence.

[0119] The welding equipment is an equipment for welding a target workpiece, such as a welding gun. The first welding direction is the advancing direction of the welding equipment, and the relative position is whether the welding equipment is on the left side or the right side of the first welding direction. The predetermined welding distance is a distance between the welding points, which can be set according to the welding requirement.

[0120] The embodiment can obtain the relative position of the welding equipment and the first welding direction, and sample the fitted weld according to the predetermined welding torch to obtain a plurality of welding points. In the case of the welding edge application type being single-sided welding, if the to-be-welded side of the single-sided welding is consistent with the relative position, the first welding direction is determined as the weld advancing direction. The to-be-welded side of the single-sided welding can be determined according to the particle position of the first point cloud of the bottom plate plane point cloud in the first bounding box, that is, the side where the particle position of the first point cloud is located is the to-be-welded side. If the to-be-welded side of the single-sided welding is inconsistent with the relative position, the second welding direction is determined as the weld advancing direction, wherein the second welding direction is opposite to the first welding direction. Exemplarily, taking the case that the relative position of the welding equipment and the first welding direction is the left side (that is, the welding equipment is on the left side of the first welding direction) as an example, when the to-be-welded side of the single-sided welding is the left side (consistent with the relative position), the first welding direction can be determined as the welding advancing direction. When the to-be-welded side of the single-sided welding is the right side (inconsistent with the relative position), since the welding equipment is arranged on the left side of the first welding direction, if the first welding direction is determined as the welding advancing direction, the welding equipment cannot realize welding of the rib plate. Therefore, the second welding direction opposite to the first welding direction needs to be taken as the welding advancing direction. In the case of the welding edge application type being double-sided welding, since the rib plate needs to be welded on both sides, the first welding direction and the second welding direction can be both determined as the weld advancing direction. The welding points are sorted based on the weld advancing direction to form a welding point sequence. In addition to the fitted weld, the welding edge application type and the rib plate height, the welding point sequence can also be determined as the weld information, so as to further reduce manual intervention, improve the efficiency of the weld information extraction and the comprehensiveness of the weld information.

[0121] The embodiment can obtain the relative position of the welding equipment and the first welding direction, and sample the fitted weld according to the predetermined welding torch to obtain a plurality of welding points. In the case of the welding edge application type being single-sided welding, if the to-be-welded side of the single-sided welding is consistent with the relative position, the first welding direction is determined as the weld advancing direction. The to-be-welded side of the single-sided welding can be determined according to the particle position of the first point cloud of the bottom plate plane point cloud in the first bounding box, that is, the side where the particle position of the first point cloud is located is the to-be-welded side. If the to-be-welded side of the single-sided welding is inconsistent with the relative position, the second welding direction is determined as the weld advancing direction, wherein the second welding direction is opposite to the first welding direction. Exemplarily, taking the case that the relative position of the welding equipment and the first welding direction is the left side (that is, the welding equipment is on the left side of the first welding direction) as an example, when the to-be-welded side of the single-sided welding is the left side (consistent with the relative position), the first welding direction can be determined as the welding advancing direction. When the to-be-welded side of the single-sided welding is the right side (inconsistent with the relative position), since the welding equipment is arranged on the left side of the first welding direction, if the first welding direction is determined as the welding advancing direction, the welding equipment cannot realize welding of the rib plate. Therefore, the second welding direction opposite to the first welding direction needs to be taken as the welding advancing direction. In the case of the welding edge application type being double-sided welding, the first welding direction and the second welding direction are determined as the weld advancing direction, and the welding points are sorted based on the weld advancing direction to form a welding point sequence. Thus, the embodiment realizes reasonable sorting of the welding points and ensures that the welding path of the welding equipment is reasonable and continuous.

[0122] In order to more clearly describe the weld information extraction method of the present application, an embodiment is taken as an example below. The embodiment is directed to a target workpiece composed of a bottom plate and a reinforcing rib (rib plate) in ship small group erection, and proposes a weld information extraction method as follows Figure 10The weld information includes weld spatial position, whether double-sided welding is required, weld point sequence, and height of the corresponding rib plate. The welding system can be composed of a welding robot, a ground rail cantilever, a line-scan camera, a weld tracking sensor, a welding machine, a programmable logic controller (PLC), and an industrial computer.

[0123] Step 1, the welding robot is hung upside down on the ground rail cantilever, the line-scan camera is installed on the ground rail cantilever, and the relative position of the line-scan camera to the robot coordinate system of the welding robot remains unchanged. The ground rail cantilever moves at a constant speed to drive the line-scan camera to scan the target workpiece to obtain point clouds, and the point clouds are converted to the robot coordinate system through the visual calibration parameters of the line-scan camera to obtain workpiece point clouds of the target workpiece.

[0124] Step 2, in order to improve the accuracy of the workpiece point clouds, the workpiece point clouds can also be filtered and denoised to filter out interference point clouds.

[0125] Step 3, the RANSAC (Random Sample Consensus) algorithm is used to fit the bottom plate plane, the plane distance threshold is set to extract the in-plane points in the workpiece point clouds, and the plane bottom plate point clouds are obtained. The remaining point clouds are the upper surface point clouds of the workpiece and interference points.

[0126] Step 4, since the Z direction in the world coordinate system is vertically downward, the minimum value of the Z direction of the in-plane points of the bottom plate is found, and this value is used as the maximum value of the Z direction. The upper surface point clouds obtained in step 2 and the interference point clouds are directly filtered, and the point clouds below the bottom plate are filtered out to obtain the rib plate upper surface point clouds. The rib plate upper surface point clouds are projected onto the bottom plate plane, statistical filtering and denoising, uniform downsampling to reduce the point cloud density, and clustering segmentation are used to separate the rib plate point clouds one by one. The centroid of each segmented point cloud is found within a radius R to downsample the point cloud, and the rib plate projection point cloud is obtained.

[0127] Step 5, the points of the downsampled rib plate projection point cloud are traversed, a local window is constructed, the cosine value of the vector formed by the point and other points in the window is calculated, and if there is only a positive value or a negative value, it is an end point. The candidate end points are clustered again, and the geometric center point of each cluster is taken as the weld end point to avoid repeated appearance of multiple end points.

[0128] Step 6, the type of weld can be determined according to the number of endpoints of the weld endpoints, if there are two weld endpoints, it is further determined whether it is a straight line weld or an open curve weld; if there are more than two weld endpoints, it is a weld where the gusset has intersection points; if there is no weld endpoint, it is a closed curve weld. For the case of two weld endpoints, a tentative straight line is constructed through the two weld endpoints, for each point in the down-sampled gusset projection point cloud, the projection point on the tentative straight line is calculated, and the perpendicular distance from the projection point to the tentative straight line is calculated, the average value of the perpendicular distance is calculated, if the average value of the perpendicular distance is less than the predetermined perpendicular distance threshold, the type of weld is determined to be a straight line weld, otherwise the type of weld is determined to be a curve weld. For a straight line weld, the RANSAC algorithm is repeatedly used to extract a straight line based on the gusset upper surface point cloud, until the number of remaining points is less than a set threshold to stop fitting, the straight line obtained each time is used as a fitting line segment, and the inliers of the fitting line segment are projected onto the fitting line segment, the weld endpoints are re-determined by the method shown in step 5, thereby realizing straight line fitting. Since there is only one weld on a gusset, if multiple fitting line segments are extracted after step 8, it means that the target workpiece is a wider gusset or there is a cover plate on the gusset, at this time the fitting line segments that meet the conditions need to be merged. The included angle of each pair of fitting line segments is calculated, if the included angle is less than a set included angle threshold, the nearest distance from the endpoint of one fitting line segment to the inlier of the other line segment is calculated, if the nearest distance is also less than a set distance threshold, the pair of fitting line segments meets the merging requirement, and an adjacency graph is constructed based on the included angle and the nearest distance. Starting from a starting node of the adjacency graph, all adjacent nodes are accessed layer by layer outwardly until all reachable nodes are accessed to traverse the connection graph, thereby screening all fitting line segments to be merged. The inliers of the fitting line segments to be merged are merged into a to-be-merged point cloud, the point cloud boundary of the to-be-merged point cloud is calculated, and the RANSAC algorithm is used to fit a straight line based on the point cloud boundary to obtain a fitted straight line, which is used as the final fitted weld of the gusset projection point cloud, thereby realizing the removal of redundant fitting line segments.

[0129] Step 7, construct a weld coordinate system, take the vector composed of the two end points of the fitted weld as the X axis of the first weld coordinate system, the default Z axis is vertical upward, and the cross product can obtain the Y axis of the first weld coordinate system. Respectively, take the midpoint and endpoint of the fitted weld as the first search point to construct a cuboid, take the X axis direction of the first weld coordinate system as the long side direction of the cuboid, the Y axis of the first weld coordinate system as the wide side direction, and the Z axis of the first weld coordinate system as the high side direction, construct a cuboid region of a certain size as the first bounding box. Search for the first point cloud of the bottom plate plane point cloud in the first bounding box, calculate the distance between the centroid of the first point cloud and the first search point, if the distance between the centroid of the first point cloud and the first search point is less than the first distance threshold, it is determined that the weld edge application type is double-sided welding, and if it is greater than the first distance threshold, it is determined that the weld edge application type is single-sided welding, thereby identifying whether the fitted weld should use double-sided welding or single-sided welding. For the identification of the welding point sequence, the welding equipment (such as a welding torch) can be defined in advance to always be on the left side of the predetermined weld advancing direction, and the fitted weld can be sampled according to the predetermined welding torch to obtain multiple welding points. If the weld edge application type is single-sided welding, the welding points need to be sorted, the unit vector composed of the welding point sequence in the predetermined weld advancing direction is defined as the X axis of the second weld coordinate system, the unit vector of the first welding point pointing to the centroid of the first point cloud is defined as the Y axis of the second weld coordinate system, and the cross product is obtained. The Z axis of the second weld coordinate system, the Z axis of the second weld coordinate system is opposite to the Z axis of the first weld coordinate system, then reverse the welding point sequence to obtain a new welding point sequence. The Z axis of the second weld coordinate system is the same as the Z axis of the first weld coordinate system, then take the welding point sequence in the predetermined weld advancing direction as the welding point sequence.

[0130] Step 8, construct a target line segment with the weld end point of the fitted weld, sample equidistantly on the target line segment to obtain a to-be-determined search point, for each to-be-determined search point, first move a predetermined search distance along the Z axis direction of the first weld coordinate system as a second search point, construct a cuboid bounding box (i.e. second bounding box) centered on the second search point, search for a second point cloud in the second point cloud in the second point cloud. Find the point with the smallest Z value (i.e. the lowest point) in the second point cloud, find the neighborhood points within a radius R centered on the lowest point, if the number of neighborhood points is greater than a set point number threshold, calculate the average of the Z values of these neighborhood points, and the difference between the average of the Z values and the Z value of the second search point is the web height.

[0131] It should be understood that although the steps in the flowcharts involved in the above embodiments are shown in sequence according to the arrows, the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the execution of the steps is not strictly limited in sequence, and the steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the above embodiments can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of the steps or stages is not necessarily sequential, but can be alternately or alternately executed with at least part of other steps or steps in other steps. It can be understood that the steps in different embodiments can be freely combined as needed, and various non-contradictory schemes formed by the combination are within the scope of protection of the present application.

[0132] Based on the same inventive concept, the embodiments of the present application also provide a weld seam information extraction device for implementing the above-mentioned weld seam information extraction method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more weld seam information extraction device embodiments provided below can refer to the limitations of the weld seam information extraction method in the above text, which will not be repeated here.

[0133] In one exemplary embodiment, as shown in Figure 11 A weld seam information extraction device 500 is provided, comprising a weld seam fitting module 510, a weld edge application type determination module 520, and a rib height determination module 530, wherein:

[0134] The weld seam fitting module 510 is configured to obtain a workpiece point cloud of a target workpiece, and fit a fitting weld seam based on the workpiece point cloud, wherein the workpiece point cloud includes a bottom plate plane point cloud and a rib upper surface point cloud.

[0135] The weld edge application type determination module 520 is configured to construct a first bounding box based on the fitting weld seam, and determine a weld edge application type of the fitting weld seam according to a first point cloud of the bottom plate plane point cloud within the first bounding box.

[0136] The rib height determination module 530 is configured to construct a second bounding box based on the fitting weld seam, and determine a rib height according to a height difference of a second point cloud of the rib upper surface point cloud within the second bounding box.

[0137] In one embodiment, the weld edge application type determination module 520 is further configured to:

[0138] At least one first search point is selected on the fitting weld, a first bounding box is constructed with the first search point as the center, a first point cloud of the bottom plate plane point cloud in the first bounding box is determined, in a case that a distance between the first point cloud of the point cloud and the search point is less than a predetermined distance threshold, it is determined that a welding edge application type of the fitting weld is double-sided welding, in a case that the distance between the first point cloud of the point cloud and the search point is greater than or equal to the predetermined distance threshold, it is determined that the welding edge application type of the fitting weld is single-sided welding.

[0139] In one embodiment, the rib plate height determination module 530 is further configured to:

[0140] At least one second search point is selected on the fitting weld, a second bounding box is constructed with the second search point as the center, a lowest point of a second point cloud of the rib plate upper surface point cloud in the second bounding box is determined, a neighborhood point within a predetermined radius is searched with the lowest point as the center, in a case that a point quantity of the neighborhood point is greater than a set point quantity threshold, a height mean value of each neighborhood point is determined, and a difference between the height mean value and a height value of the second search point is determined as the rib plate height.

[0141] In one embodiment, the weld fitting module 510 is further configured to:

[0142] The bottom plate plane point cloud and at least one rib plate upper surface point cloud are extracted from the workpiece point cloud, the rib plate upper surface point cloud is projected onto the bottom plate plane of the target workpiece to obtain a rib plate projection point cloud, the rib plate projection point cloud is traversed to obtain a weld end point, a type of the weld to be extracted is determined based on the weld end point, in a case that the type of the weld to be extracted is a straight-line weld, iterative straight-line extraction is performed based on the rib plate projection point cloud to obtain a fitting line segment, in a case that there are multiple corresponding fitting line segments in the rib plate projection point cloud, the fitting line segments are merged to obtain a fitting weld.

[0143] In one embodiment, the weld fitting module 510 is further configured to:

[0144] In a case that there are multiple corresponding fitting line segments in the rib plate projection point cloud, a line segment included angle between each pair of fitting line segments and a shortest distance from an end point of one fitting line segment to another fitting line segment in a pair of fitting line segments are determined, in a case that the line segment included angle is less than a set angle threshold and the shortest distance is less than a predetermined distance threshold, an adjacency graph of the pair of fitting line segments is constructed, based on the adjacency graph, the fitting line segments to be merged are determined, the in-points of the fitting line segments to be merged are merged into a point cloud to be fitted, a point cloud boundary of the point cloud to be fitted is determined, and straight-line fitting is performed based on the point cloud boundary of the point cloud to be fitted to obtain a fitting weld.

[0145] In one embodiment, the weld information extraction device further comprises a welding point determination module configured to:

[0146] The relative position of the welding equipment and the first welding direction is acquired, the welding seam is sampled according to the preset welding distance to obtain a plurality of welding points, in the case of a single-sided welding type of the welding edge, if the to-be-welded side of the single-sided welding is consistent with the relative position, the first welding direction is determined as the welding seam advancing direction, if the to-be-welded side of the single-sided welding is inconsistent with the relative position, the second welding direction is determined as the welding seam advancing direction, wherein the second welding direction is opposite to the first welding direction, in the case of a double-sided welding type of the welding edge, the first welding direction and the second welding direction are determined as the welding seam advancing direction, the welding points are sorted based on the welding seam advancing direction to form a welding point sequence.

[0147] Each module in the welding seam information extraction device can be realized by software, hardware, or a combination thereof, in whole or in part. Each module can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0148] In an exemplary embodiment, a computer device, which can be a terminal, has an internal structure diagram as shown in Figure 12 The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to perform wired or wireless communication with external terminals. The wireless communication can be achieved through WIFI, mobile cellular network, near field communication (NFC), or other technologies. The computer program is executed by the processor to implement a welding seam information extraction method. The display unit of the computer device is configured to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball, or touchpad arranged on the shell of the computer device. The input device can also be an external keyboard, touchpad, or mouse, etc.

[0149] Those skilled in the art can understand that Figure 12The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0150] In an exemplary embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, and the processor implementing the steps of the above embodiments when executing the computer program.

[0151] In an embodiment, a computer readable storage medium is provided, storing a computer program, and the computer program implementing the steps of the above embodiments when executed by a processor.

[0152] In an embodiment, a computer program product is provided, including a computer program, and the computer program implementing the steps of the above embodiments when executed by a processor.

[0153] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0154] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing relevant hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, the processes of the above-mentioned embodiment methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0155] The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combinations of the technical features do not exist contradictory, it should be considered as the scope of the present application.

[0156] The above embodiments only express several implementation ways of the present application, and the description is specific and detailed, but it should not be understood as a limitation to the patent scope of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for extracting weld information, characterized in that, The method includes: The workpiece point cloud of the target workpiece is obtained, and the fitted weld is obtained based on the workpiece point cloud. The workpiece point cloud includes the bottom plate plane point cloud and the upper surface point cloud of the stiffener plate. A first bounding box is constructed based on the fitted weld, and the weld edge application type of the fitted weld is determined according to the first point cloud of the base plate planar point cloud within the first bounding box. A second enclosure is constructed based on the fitted weld, and the height of the stiffener is determined according to the height difference between the point cloud on the upper surface of the stiffener and the second point cloud within the second enclosure. The construction of the second bounding box based on the fitted weld, and the determination of the stiffener height based on the height difference between the point cloud on the upper surface of the stiffener and the point cloud within the second bounding box, include: Select at least one second search point on the fitted weld, and construct a second bounding box with the second search point as the center; Determine the lowest point of the second point cloud within the second bounding box of the point cloud on the upper surface of the stiffener, and find neighboring points within a predetermined radius centered on the lowest point; If the number of neighboring points is greater than a set threshold, determine the average height of each neighboring point. The difference between the average height and the height value of the second search point is determined as the rib height.

2. The method according to claim 1, characterized in that, The process of constructing a first bounding box based on the fitted weld, and determining the weld edge application type of the fitted weld according to the first point cloud of the base plate planar point cloud within the first bounding box, includes: Select at least one first search point on the fitted weld, and construct a first bounding box with the first search point as the center; Determine the mass points of the first point cloud within the first bounding box of the base plate planar point cloud; If the distance between the particles of the first point cloud and the search point is less than a predetermined distance threshold, the weld edge application type of the fitted weld is determined to be double-sided welding; If the distance between the mass point of the first point cloud and the search point is greater than or equal to a predetermined distance threshold, the weld edge application type of the fitted weld is determined to be single-sided welding.

3. The method according to claim 1, characterized in that, The fitted weld obtained based on the workpiece point cloud includes: Extract the bottom plate plane point cloud and at least one upper surface point cloud of the rib plate from the point cloud of the workpiece, and project the upper surface point cloud of the rib plate onto the bottom plate plane of the target workpiece to obtain the rib plate projection point cloud. Traverse the projection point cloud of the stiffener plate to obtain the weld endpoints, and determine the type of weld to be extracted based on the weld endpoints; When the weld type to be extracted is a straight weld, iterative straight line extraction is performed based on the stiffener projection point cloud to obtain a fitted line segment. When there are multiple corresponding fitted line segments in the projection point cloud of the stiffener plate, the fitted line segments are merged to obtain the fitted weld.

4. The method according to claim 3, characterized in that, When multiple corresponding fitted line segments exist in the projection point cloud of the stiffener plate, the fitted line segments are merged to obtain the fitted weld, which includes: In the case that there are multiple corresponding fitted line segments in the projection point cloud of the stiffener plate, determine the angle between each pair of fitted line segments and the shortest distance from the endpoint of one fitted line segment to the other fitted line segment within a pair of fitted line segments. When the included angle of the line segments is less than a set angle threshold and the shortest distance is less than a predetermined distance threshold, an adjacency graph of the pair of fitted line segments is constructed. Based on the adjacency graph, the fitted line segments to be merged are determined; The interior points of the fitted line segments to be merged are merged into a point cloud to be fitted, and the point cloud boundary of the point cloud to be fitted is determined. Based on the point cloud boundary of the point cloud to be fitted, a straight line is fitted to obtain the fitted weld.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: The relative position of the welding equipment and the first welding direction is obtained, and the fitted weld is sampled according to the preset welding distance to obtain multiple welding points; When the welding edge is applied as a single-sided weld, if the side to be welded in the single-sided weld is consistent with the relative orientation, then the first welding direction is determined as the weld advance direction. If the side to be welded in the single-sided welding is not consistent with the relative orientation, then the second welding direction is determined as the weld advance direction, wherein the second welding direction is opposite to the first welding direction; When the welding edge is subjected to double-sided welding, the first welding direction and the second welding direction are determined as the weld advance direction; The welding points are sorted according to the direction of weld advancement to obtain a welding point sequence.

6. A weld seam information extraction device, characterized in that, The device includes: The weld fitting module is used to obtain the workpiece point cloud of the target workpiece, and to obtain the fitted weld based on the workpiece point cloud. The workpiece point cloud includes the bottom plate plane point cloud and the upper surface point cloud of the stiffener plate. The weld edge application type determination module is used to construct a first bounding box based on the fitted weld and determine the weld edge application type of the fitted weld according to the first point cloud of the base plate plane point cloud within the first bounding box. The stiffener plate height determination module is used to construct a second bounding box based on the fitted weld, and determine the stiffener plate height according to the height difference between the point cloud on the upper surface of the stiffener plate and the second point cloud within the second bounding box; The weld information determination module is used to determine the fitted weld, the weld edge application type, and the stiffener height as weld information; The stiffener height determination module is also used for: At least one second search point is selected on the fitted weld. A second bounding box is constructed with the second search point as the center. The lowest point of the second point cloud on the upper surface of the stiffener plate within the second bounding box is determined. Neighboring points within a predetermined radius are searched with the lowest point as the center. If the number of neighboring points is greater than a set threshold, the average height of each neighboring point is determined. The difference between the average height and the height of the second search point is determined as the stiffener plate height.

7. The apparatus according to claim 6, characterized in that, The weld edge application type determination module is also used for: At least one first search point is selected on the fitted weld. A first bounding box is constructed with the first search point as the center. The mass points of the first point cloud in the base plate plane point cloud within the first bounding box are determined. If the distance between the mass points of the first point cloud and the search point is less than a predetermined distance threshold, the weld edge application type of the fitted weld is determined to be double-sided welding. If the distance between the mass points of the first point cloud and the search point is greater than or equal to the predetermined distance threshold, the weld edge application type of the fitted weld is determined to be single-sided welding.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images