Welding spot positioning method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a welding spot positioning method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring point cloud data including a first straight reinforcing steel bar and a second straight reinforcing steel bar; based on the point cloud data, determining a first vertical line perpendicular to a first axis of the first straight reinforcing steel bar and a second axis of the second straight reinforcing steel bar and a second vertical line perpendicular to the first vertical line and the second axis, and determining a reference plane which takes the second vertical line as a normal and passes through the second axis; determining a first intersection point of the reference plane and the first axis, and determining a second intersection point along a first direction of the first vertical line, the distance between the second intersection point and the first intersection point being equal to a first preset distance; and points which are located on the two sides of the second intersection point and have the distance equal to a second preset distance from the second intersection point along the second axis are determined as welding points. By adopting the method, the crossed steel bar welding spots can be quickly and accurately positioned.

Description

Technical Field

[0001] This application relates to the field of welding technology, and in particular to a weld spot positioning method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] Steel structures are widely used in construction, bridges, and marine engineering due to their high strength, light weight, and good seismic performance. Welding, as the core process in steel structure manufacturing, directly affects the overall performance and production efficiency of the structure.

[0003] Currently, steel structure manufacturing still primarily employs manual and semi-automatic arc welding processes. These traditional welding methods suffer from problems such as high heat input and difficulty in deformation control, and are highly dependent on the welder's skill level. With a shortage of skilled welders and rising labor costs, the limitations of traditional welding methods are becoming increasingly apparent.

[0004] In steel structure manufacturing, cross-welding of reinforcing bars is a common and critical process. Related technologies mainly employ offline programming or manual teaching. Offline programming requires professionals to pre-set the welding path, while manual teaching relies on operators manually instructing the welders. Both methods suffer from low automation and poor adaptability, especially when dealing with cross-welding at non-standard angles, making it difficult to guarantee welding quality and efficiency, thus hindering the improvement of automation levels in steel structure manufacturing. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can quickly and accurately locate weld points of intersecting reinforcing bars in order to address the above-mentioned technical problems.

[0006] Firstly, this application provides a method for locating solder joints, including:

[0007] Based on a 3D vision sensor, point cloud data containing a first straight steel bar and a second straight steel bar are acquired. The first straight steel bar and the second straight steel bar are placed crosswise. In the shooting direction of the 3D vision sensor, the first straight steel bar occludes the second straight steel bar at the intersection.

[0008] Extract the first point cloud data of the first straight steel bar and the second point cloud data of the second straight steel bar from the point cloud data, and determine the first axis of the first straight steel bar and the second axis of the second straight steel bar based on the first point cloud data and the second point cloud data, respectively.

[0009] Determine a first perpendicular line that is perpendicular to both the first axis and the second axis, and a second perpendicular line that is perpendicular to both the first perpendicular line and the second axis; and determine a reference plane that is normal to the second perpendicular line and passes through the second axis.

[0010] Determine the first intersection point between the reference plane and the first axis, and along the first direction of the first perpendicular line, determine the second intersection point at a distance equal to a first preset distance from the first intersection point;

[0011] Points located on both sides of the second intersection point and at a distance equal to a second preset distance from the second intersection point along the second axis are identified as welding points. The first preset distance and the second preset distance are associated with a preset weld size.

[0012] In one embodiment, extracting the first point cloud data of the first straight reinforcing bar and the second point cloud data of the second straight reinforcing bar from the point cloud data includes:

[0013] Based on the cylinder fitting algorithm, the first intermediate point cloud data of the first straight steel bar and the second intermediate point cloud data of the second straight steel bar are extracted from the point cloud data;

[0014] Clustering is performed on the first intermediate point cloud data and the second intermediate point cloud data respectively, and the cluster with the largest data volume is determined to obtain the first point cloud data and the second point cloud data.

[0015] In one embodiment, prior to extracting the first point cloud data of the first straight reinforcing bar and the second point cloud data of the second straight reinforcing bar from the point cloud data, the following steps are included:

[0016] Based on the distance between the three-dimensional vision sensor and the intersection point, the target coordinate range in the shooting direction is determined, and the coordinates of the intersection point in the shooting direction are within the target coordinate range;

[0017] Based on a bandpass filter, the point cloud data of points whose coordinates in the shooting direction exceed the target coordinate range are deleted.

[0018] In one embodiment, before extracting the first point cloud data of the first straight steel bar and the second point cloud data of the second straight steel bar from the point cloud data, the method further includes:

[0019] Based on the local neighborhood density of each point in the point cloud data and a preset density threshold, the discrete point cloud data in the point cloud data is deleted.

[0020] In one embodiment, prior to acquiring point cloud data containing the first and second straight reinforcing bars based on a 3D vision sensor, the process includes:

[0021] Adjust the position of the three-dimensional vision sensor so that the distance between the three-dimensional vision sensor and the intersection point to be located is within a preset distance range. The intersection point is the intersection point of the first straight steel bar and the second straight steel bar.

[0022] Adjust the shooting direction of the three-dimensional vision sensor so that the angle between the shooting direction of the three-dimensional vision sensor and the direction of the three-dimensional vision sensor toward the intersection position is within a preset angle range.

[0023] In one embodiment, after determining that the points located on both sides of the second intersection point and at a distance equal to a second preset distance from the second intersection point along the second axis are respectively designated as solder joints, the process includes:

[0024] Determine the type of the first straight steel bar and the second straight steel bar;

[0025] Based on the model number, obtain the corresponding process parameters;

[0026] Based on the location of the weld point and the process parameters, the welding robot is controlled to perform welding.

[0027] Secondly, this application also provides a solder joint positioning device, comprising:

[0028] The acquisition module is used to acquire point cloud data containing a first straight steel bar and a second straight steel bar based on a three-dimensional vision sensor. The first straight steel bar and the second straight steel bar are placed crosswise. In the shooting direction of the three-dimensional vision sensor, the first straight steel bar occludes the second straight steel bar at the intersection.

[0029] The determining module is used to extract the first point cloud data of the first straight steel bar and the second point cloud data of the second straight steel bar from the point cloud data, and to determine the first axis of the first straight steel bar and the second axis of the second straight steel bar based on the first point cloud data and the second point cloud data, respectively.

[0030] The determining module is further configured to determine a first perpendicular line that is perpendicular to both the first axis and the second axis, and a second perpendicular line that is perpendicular to both the first perpendicular line and the second axis, and to determine a reference plane that is normal to the second perpendicular line and passes through the second axis.

[0031] The determining module is further configured to determine the first intersection point of the reference plane and the first axis, and along the first direction of the first perpendicular line, determine the second intersection point at a distance equal to the first intersection point.

[0032] The determining module is further configured to determine points located on both sides of the second intersection point and at a distance equal to a second preset distance from the second intersection point along the second axis as welding points, wherein the first preset distance and the second preset distance are associated with preset weld dimensions.

[0033] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in any of the above-mentioned embodiments.

[0034] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the preceding claims.

[0035] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the above claims.

[0036] The aforementioned weld point positioning method, device, computer equipment, computer-readable storage medium, and computer program product directly acquire point cloud data of intersecting reinforcing bars through a 3D vision sensor, realizing dynamic perception of the spatial pose of the reinforcing bars. This eliminates the reliance on manual teaching or offline programming in traditional methods, significantly improving the system's flexibility and effectively adapting to scenarios with non-perpendicular intersections or installation tilts. By extracting the reinforcing bar axis and calculating its cross product to generate a first perpendicular line, and then calculating the cross product of the upper reinforcing bar axis and the first perpendicular line to obtain a second perpendicular line, the reference plane of the lower reinforcing bar axis is finally constructed by combining the second perpendicular line and the upper reinforcing bar axis. This allows for adaptive correction of the reference plane through vector operations even when there are errors in the point cloud data, ensuring the positioning accuracy of the intersection point. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart illustrating a solder joint positioning method in one embodiment;

[0039] Figure 2 This is a schematic diagram of the first and second straight steel bars being placed intersecting in one embodiment;

[0040] Figure 3 This is a schematic diagram of a first point cloud data, a second point cloud data, a first axis, a second axis, a first vertical line, and a reference plane in one embodiment;

[0041] Figure 4 This is a schematic diagram of the first intersection point, the second intersection point, and the solder joint in one embodiment;

[0042] Figure 5 This is a flowchart illustrating the steps of extracting first point cloud data of a first straight steel bar and second point cloud data of a second straight steel bar from point cloud data in one embodiment.

[0043] Figure 6 This is a flowchart illustrating the solder joint positioning method in another embodiment;

[0044] Figure 7 This is a schematic diagram of point cloud data in one embodiment;

[0045] Figure 8 For one embodiment, delete Figure 7 The diagram shows point cloud data of points whose coordinates in the shooting direction exceed the target coordinate range, and the point cloud data obtained after discretizing the point cloud data.

[0046] Figure 9 This is a flowchart illustrating the solder joint positioning method in yet another embodiment;

[0047] Figure 10 This is a flowchart illustrating the solder joint positioning method in another embodiment;

[0048] Figure 11 This is a structural block diagram of a solder joint positioning device in one embodiment;

[0049] Figure 12 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0051] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0052] In practical applications, the placement of workpieces can be quite arbitrary. Welding via manual teaching requires pre-welding instruction, while offline programming necessitates reprogramming each time or strictly limiting workpiece tolerances and placement. In the scenario of welding intersecting rebars, due to the large number of similar repetitive weld points, traditional offline programming typically involves applying offsets to the weld point positions to achieve welding operations at multiple locations with a single program. However, this method relies on skilled technicians for programming, resulting in a high implementation threshold and low efficiency. Furthermore, the rebar crossing process is usually done manually, inevitably introducing assembly tolerances, and this method lacks sufficient flexibility to effectively address dimensional deviations caused by manual assembly. Specifically, this solution has the following limitations: it requires pre-acquiring accurate 3D model data of the intersecting rebars; it requires professionals to plan and program the robot's motion trajectory and task flow in detail, leading to high operational complexity; and when there are dimensional deviations between the actual workpiece and the 3D model, welding quality is difficult to guarantee, resulting in insufficient system adaptability.

[0053] Based on this, this application provides a solder joint positioning method. This embodiment uses the application of this method to a terminal as an example for illustration. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. In this embodiment, as... Figure 1 As shown, the method includes the following steps:

[0054] Step S101: Based on a three-dimensional vision sensor, acquire point cloud data containing a first straight steel bar and a second straight steel bar. The first straight steel bar and the second straight steel bar are placed at an intersection. In the shooting direction of the three-dimensional vision sensor, the first straight steel bar occludes the second straight steel bar at the intersection.

[0055] Please refer to Figure 2 , Figure 2This is a schematic diagram illustrating the intersecting placement of a first and second straight reinforcing bar in one embodiment. The first straight reinforcing bar L1 and the second straight reinforcing bar L2 are placed perpendicularly, forming a cross shape when viewed from above, and maintaining a certain degree of overlap at the intersection. It is understood that in practical applications, deviations may occur when the first and second straight reinforcing bars are placed perpendicularly. In one possible implementation, the intersection can be secured with wire or steel wire before welding to prevent movement during the welding process. The perpendicularly intersecting reinforcing bars are typically welded using double-sided welding, maintaining a suitable angle (e.g., 70° to 80°) between the welding rod and the welding surface to ensure sufficient fusion at the weld root.

[0056] Step S102: Extract the first point cloud data of the first straight steel bar and the second point cloud data of the second straight steel bar from the point cloud data, and determine the first axis of the first straight steel bar and the second axis of the second straight steel bar based on the first point cloud data and the second point cloud data, respectively.

[0057] For example, the point cloud data can first be fitted with cylinders and clustered. Then, the point cloud cluster of the upper steel bar (the first straight steel bar) after clustering is denoted as c1 (first point cloud data), and the point cloud cluster of the lower steel bar (the second straight steel bar) is denoted as c2 (second point cloud data). The axes of the two cylinders are denoted as r1 (first axis) and r2 (second axis) respectively.

[0058] Step S103: Determine a first perpendicular line that is perpendicular to both the first axis and the second axis, and a second perpendicular line that is perpendicular to both the first perpendicular line and the second axis, and determine a reference plane that is normal to the second perpendicular line and passes through the second axis.

[0059] For example, we can find the vector n1 (direction vector of the first perpendicular line) that is perpendicular to both r1 and r2, and then find the vector n2 (direction vector of the second perpendicular line) that is perpendicular to both r2 and n1. Using n2 as the normal and passing through point 1 on r2, we can obtain plane P1 (reference plane).

[0060] In one possible implementation, n1 can be determined using a first formula, which may include:

[0061] n1(x,y,z) = r1(x,y,z)×r2(x,y,z) (1)

[0062] The second formula can be used to determine n², and the second formula may include:

[0063] n2(x,y,z) = r2(x,y,z)×n1(x,y,z) (2)

[0064] Please refer to Figure 3 , Figure 3 This is a schematic diagram of a first point cloud data, a second point cloud data, a first axis, a second axis, a first perpendicular line, and a reference plane in one embodiment. It should be noted that the second perpendicular line is located at... Figure 3 As not shown in the diagram, when the first and second straight steel bars are placed approximately perpendicularly, r1 and n2 are not parallel.

[0065] Step S104: Determine the first intersection point between the reference plane and the first axis, and along the first direction of the first perpendicular line, determine the second intersection point whose distance from the first intersection point is equal to the first preset distance.

[0066] The first direction extends from the first straight reinforcing bar to the second straight reinforcing bar. For example, the first direction can be downward from the plane perpendicular to where the first and second straight reinforcing bars are placed. The first preset distance can be equal to the radius of the first straight reinforcing bar.

[0067] For example, we can find the intersection point pt0 (the first intersection point) of P1 and r1, and then find the point along n1 that is at a distance from pt0 equal to the radius c1 of the first straight bar, thus obtaining the intersection point pt1 (the second intersection point) of the first and second straight bars.

[0068] Step S105: Determine the points located on both sides of the second intersection point and whose distance from the second intersection point along the second axis is equal to the second preset distance as welding points. The first preset distance and the second preset distance are related to the preset weld size.

[0069] The first preset distance and the second preset distance can be associated with the size of the reinforcing bar and the preset weld size. The preset weld size can include a weld width greater than 0.8 times the diameter of the main reinforcing bar, and / or a weld width greater than 0.3 times the diameter of the main reinforcing bar.

[0070] For example, a distance d (second preset distance) can be set according to the actual welding process requirements. Along the positive and negative directions of r2, two points p_out_0 and p_out_1 with a distance of d from pt1 are calculated respectively to obtain the weld point to be welded when the first straight steel bar and the second straight steel bar are placed at an intersection.

[0071] In one possible implementation, p_out_0 can be determined using a third formula, which may include:

[0072] p_out_0(x,y,z) = pt1(x,y,z) + d*a2(x,y,z) (3)

[0073] p_out_1(x,y,z) = pt1(x,y,z) - d*a2(x,y,z) (4)

[0074] Where a2 is the unit vector of r2.

[0075] Please refer to Figure 4 , Figure 4 This is a schematic diagram of the first intersection point, the second intersection point, and the solder joint in one embodiment.

[0076] In the above-mentioned weld point positioning method, point cloud data of intersecting rebars are directly acquired through a 3D vision sensor, realizing dynamic perception of the spatial pose of the rebars. This eliminates the reliance on manual teaching or offline programming in traditional methods, significantly improving the system's flexibility and effectively adapting to scenarios with non-perpendicular intersections or installation tilts. By extracting the rebar axis and calculating its cross product to generate the first perpendicular line, and then calculating the cross product of the upper rebar axis and the first perpendicular line to obtain the second perpendicular line, the reference plane of the lower rebar axis is finally constructed by combining the second perpendicular line and the upper rebar axis. This allows the reference plane to be adaptively corrected through vector operations even if there are errors in the point cloud data, ensuring the positioning accuracy of the intersection point.

[0077] In an exemplary embodiment, a cylinder fitting algorithm can be first used to initially separate the reinforcing bar data from the original point cloud, and then a density clustering algorithm can be used to remove noise points to obtain high-precision point cloud data of the main reinforcing bar structure. Figure 5 As shown, the steps for extracting the first point cloud data of the first straight reinforcing bar and the second point cloud data of the second straight reinforcing bar from the point cloud data include:

[0078] Step A1: Based on the cylinder fitting algorithm, extract the first intermediate point cloud data of the first straight steel bar and the second intermediate point cloud data of the second straight steel bar from the point cloud data.

[0079] For example, RANSAC (RANdom SAmple Consensusransac) can be used to fit cylinders, extracting the main point clouds of two cylindrical steel bars from the point cloud, resulting in an intermediate point cloud and a second intermediate point cloud. The parameters of each cylinder can include a point P (x, y, z) on the central axis of the cylinder, the direction vector of the central axis of the cylinder (dx, dy, dz), and the cylinder radius (R), for a total of seven parameters.

[0080] Step A2: Perform clustering processing on the first intermediate point cloud data and the second intermediate point cloud data respectively, and determine the cluster with the largest data volume to obtain the first point cloud data and the second point cloud data.

[0081] For example, the point clouds falling on the two cylinders can be clustered using dbscan (Density-Based Spatial Clustering of Applications with Noise) and sorted according to the number of point clouds in each cluster. The cluster with the largest number of point clouds after clustering the point clouds of the two cylinders can be selected to obtain two point clouds containing only the main body of the steel bars.

[0082] Optionally, the main body of the point cloud can be obtained by clustering first, and the distance from the point to the cylindrical surface can be used as the error function. Then, a nonlinear optimization method (such as the Levenberg-Marquardt algorithm) can be used to minimize the error.

[0083] In an exemplary embodiment, after acquiring the point cloud data of the workpiece to be positioned (the first straight steel bar and the second straight steel bar), the point cloud data can be preprocessed to obtain high-quality overall point cloud data of the workpiece to be positioned, such as... Figure 6 As shown, the above-mentioned solder joint positioning method also includes:

[0084] Step S2011: Based on the distance between the three-dimensional vision sensor and the intersection, determine the target coordinate range in the shooting direction, and the coordinates of the intersection in the shooting direction are within the target coordinate range;

[0085] Step S2012: Based on the bandpass filter, delete the point cloud data of points whose coordinates in the shooting direction exceed the target coordinate range.

[0086] For example, after the three-dimensional vision sensor acquires the point cloud data of the intersecting steel bars, it can calculate the depth value of the intersection point in the sensor coordinate system, and set a vertical target range of 100 mm above and below this center. Through bandpass filtering, the effective point cloud data within this range is retained, and distant background and other interference objects are automatically removed.

[0087] In one possible implementation, a bandpass filter can be used to obtain point clouds with Z-direction values ​​in the range of 400 mm to 600 mm to remove the influence of cluttered backgrounds.

[0088] For further information, please continue to refer to [link / reference]. Figure 6 The above-mentioned solder joint positioning method also includes:

[0089] Step S202: Based on the local neighborhood density of each point in the point cloud data and the preset density threshold, delete the discrete point cloud data in the point cloud data.

[0090] For example, to address noise interference such as spatter and dust that may exist at the intersection of reinforcing bars, a three-dimensional spatial octree structure can be constructed to accelerate neighborhood search. For each data point, the number of neighboring points within a 5 mm radius sphere is counted as a density value. A density threshold is set according to the diameter of the reinforcing bar. When the number of neighboring points of a point is lower than the density threshold, it is determined to be a discrete noise point and is removed.

[0091] In one possible implementation, voxel downsampling can be performed on the bandpass filtered point cloud data to improve the consistency of point cloud density; then radius density filtering can be used to remove discrete points in the point cloud to reduce point cloud noise.

[0092] Please refer to Figure 7 and Figure 8 , Figure 7 This is a schematic diagram of point cloud data in one embodiment. Figure 8 For one embodiment, delete Figure 7 The diagram shows point cloud data of points whose coordinates in the shooting direction exceed the target coordinate range, and the point cloud data obtained after discretizing the point cloud data.

[0093] In one exemplary embodiment, the position and shooting direction of the 3D vision sensor can be pre-adjusted to ensure clear and stable acquisition of point cloud data at the intersection of reinforcing bars, thereby improving the accuracy and reliability of subsequent weld point positioning. Figure 9 As shown, the above-mentioned solder joint positioning method also includes:

[0094] Step S301: Adjust the position of the three-dimensional vision sensor so that the distance between the three-dimensional vision sensor and the intersection to be located is within a preset distance range. The intersection is the intersection of the first straight steel bar and the second straight steel bar.

[0095] Step S302: Adjust the shooting direction of the three-dimensional vision sensor so that the angle between the shooting direction of the three-dimensional vision sensor and the direction of the three-dimensional vision sensor toward the intersection position is within a preset angle range.

[0096] The preset distance range is related to the model of the 3D vision sensor, and / or the preset distance range is related to the fixing structure of the robotic arm connected to the 3D vision sensor. The preset angle range is used to ensure that the area to be welded (the intersection of the first and second straight steel bars) is within the field of view of the 3D vision sensor and as close as possible to the center of the field of view of the 3D vision sensor.

[0097] For example, a specified model of 3D camera can be used, fixed at a distance of 500 mm from the workpiece to be positioned, facing the welding area to be positioned. In this case, a certain deviation in distance between the camera and the workpiece is permissible (e.g., a positive or negative deviation of less than or equal to 100 mm). A certain angular deviation in the camera's orientation is permissible (e.g., less than or equal to 15°).

[0098] In one exemplary embodiment, various specifications of steel bars on site can be automatically identified, and the corresponding optimal welding parameters can be invoked to control the welding robot to perform welding. This enables mixed welding of different steel bar types without manual intervention. When a new steel bar type is added, the application scope can be expanded simply by updating the database. Figure 10 As shown, the above-mentioned solder joint positioning method also includes:

[0099] Step S401: Determine the type of the first and second straight reinforcing bars.

[0100] For example, the model of the first straight reinforcing bar and the model of the second straight reinforcing bar can be determined based on the first point cloud data and the second point cloud data, respectively. The model may include the radius.

[0101] Step S402: Based on the model number, obtain the corresponding process parameters.

[0102] For example, the process parameters may include at least one of voltage, current, welding torch moving speed, welding torch end position downward offset value and attitude offset value.

[0103] Step S403: Based on the location of the weld point and process parameters, control the welding robot to perform welding.

[0104] For example, the surface features of the reinforcing bars can be captured by an industrial camera, and the type of reinforcing bars can be determined using image recognition technology. Based on the recognition results, preset welding process parameters can be automatically called (for example, thicker reinforcing bars require a larger welding current and a slower gun movement speed, and reinforcing bars of different materials require different voltages and currents). After receiving the weld point coordinates and process parameters, the welding robot can control the movement trajectory of the welding robot and accurately move the welding gun to the position of the weld point.

[0105] In one possible implementation, the above solder joint positioning method includes the following steps:

[0106] Step S301: Adjust the position of the three-dimensional vision sensor so that the distance between the three-dimensional vision sensor and the intersection to be located is within a preset distance range. The intersection is the intersection of the first straight steel bar and the second straight steel bar.

[0107] Step S302: Adjust the shooting direction of the three-dimensional vision sensor so that the angle between the shooting direction of the three-dimensional vision sensor and the direction of the three-dimensional vision sensor toward the intersection position is within a preset angle range.

[0108] Step S101: Based on a three-dimensional vision sensor, acquire point cloud data containing a first straight steel bar and a second straight steel bar. The first straight steel bar and the second straight steel bar are placed at an intersection. In the shooting direction of the three-dimensional vision sensor, the first straight steel bar occludes the second straight steel bar at the intersection.

[0109] Step S2011: Based on the distance between the three-dimensional vision sensor and the intersection, determine the target coordinate range in the shooting direction, and the coordinates of the intersection in the shooting direction are within the target coordinate range;

[0110] Step S2012: Based on the bandpass filter, delete the point cloud data of points whose coordinates in the shooting direction exceed the target coordinate range.

[0111] Step S202: Based on the local neighborhood density of each point in the point cloud data and the preset density threshold, delete the discrete point cloud data in the point cloud data.

[0112] Step S102: Extract the first point cloud data of the first straight steel bar and the second point cloud data of the second straight steel bar from the point cloud data, and determine the first axis of the first straight steel bar and the second axis of the second straight steel bar based on the first point cloud data and the second point cloud data, respectively.

[0113] Step S103: Determine a first perpendicular line that is perpendicular to both the first axis and the second axis, and a second perpendicular line that is perpendicular to both the first perpendicular line and the second axis, and determine a reference plane that is normal to the second perpendicular line and passes through the second axis.

[0114] Step S104: Determine the first intersection point between the reference plane and the first axis, and along the first direction of the first perpendicular line, determine the second intersection point whose distance from the first intersection point is equal to the first preset distance.

[0115] Step S105: Determine the points located on both sides of the second intersection point and whose distance from the second intersection point along the second axis is equal to the second preset distance as welding points. The first preset distance and the second preset distance are related to the preset weld size.

[0116] Step S401: Determine the type of the first and second straight reinforcing bars;

[0117] Step S402: Based on the model number, obtain the corresponding process parameters;

[0118] Step S403: Based on the location of the weld point and process parameters, control the welding robot to perform welding.

[0119] Optionally, the steps described above for extracting the first point cloud data of the first straight reinforcing bar and the second point cloud data of the second straight reinforcing bar from the point cloud data include:

[0120] Step A1: Based on the cylinder fitting algorithm, extract the first intermediate point cloud data of the first straight steel bar and the second intermediate point cloud data of the second straight steel bar from the point cloud data.

[0121] Step A2: Perform clustering processing on the first intermediate point cloud data and the second intermediate point cloud data respectively, and determine the cluster with the largest data volume to obtain the first point cloud data and the second point cloud data.

[0122] In summary, the above-mentioned weld point positioning method directly acquires point cloud data of intersecting rebars through a 3D vision sensor, realizing dynamic perception of the rebar's spatial pose. This eliminates the reliance on manual teaching or offline programming in traditional methods, significantly improving the system's flexibility and effectively adapting to scenarios with non-perpendicular intersections or installation tilts. By extracting the rebar axis and calculating its cross product to generate the first perpendicular line, and then calculating the cross product of the upper rebar axis and the first perpendicular line to obtain the second perpendicular line, the reference plane of the lower rebar axis is finally constructed by combining the second perpendicular line and the upper rebar axis. This allows for adaptive correction of the reference plane through vector operations even when there are errors in the point cloud data, ensuring the positioning accuracy of the intersection point.

[0123] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0124] Based on the same inventive concept, this application also provides a solder joint positioning device for implementing the solder joint positioning method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more solder joint positioning device embodiments provided below can be found in the limitations of the solder joint positioning method described above, and will not be repeated here.

[0125] In one exemplary embodiment, such as Figure 11 As shown, a solder joint positioning device 500 is provided, including: an acquisition module 501 and a determination module 502, wherein:

[0126] The acquisition module 501 is used to acquire point cloud data containing a first straight steel bar and a second straight steel bar based on a three-dimensional vision sensor. The first straight steel bar and the second straight steel bar are placed at an intersection. In the shooting direction of the three-dimensional vision sensor, the first straight steel bar occludes the second straight steel bar at the intersection position.

[0127] The determination module 502 is used to extract the first point cloud data of the first straight steel bar and the second point cloud data of the second straight steel bar from the point cloud data, and to determine the first axis of the first straight steel bar and the second axis of the second straight steel bar based on the first point cloud data and the second point cloud data, respectively.

[0128] The aforementioned determining module 502 is further configured to determine a first perpendicular line that is simultaneously perpendicular to the first axis and the second axis, and a second perpendicular line that is simultaneously perpendicular to the first perpendicular line and the second axis, and to determine a reference plane that takes the second perpendicular line as its normal and passes through the second axis.

[0129] The aforementioned determining module 502 is further configured to determine the first intersection point of the reference plane and the first axis, and along the first direction of the first perpendicular line, determine the second intersection point whose distance from the first intersection point is equal to the first preset distance.

[0130] The aforementioned determining module 502 is further configured to determine points located on both sides of the second intersection point and whose distance from the second intersection point along the second axis is equal to the second preset distance as welding points, wherein the first preset distance and the second preset distance are associated with preset weld size.

[0131] In one embodiment, the determining module 502 includes an extraction submodule, configured to:

[0132] Based on the cylinder fitting algorithm, the first intermediate point cloud data of the first straight steel bar and the second intermediate point cloud data of the second straight steel bar are extracted from the point cloud data.

[0133] Clustering is performed on the first intermediate point cloud data and the second intermediate point cloud data respectively, and the cluster with the largest data volume is determined to obtain the first point cloud data and the second point cloud data.

[0134] In one embodiment, the solder joint positioning device 500 further includes a preprocessing module for:

[0135] Based on the distance between the 3D vision sensor and the intersection, the target coordinate range in the shooting direction is determined, and the coordinates of the intersection in the shooting direction are within the target coordinate range;

[0136] Based on a bandpass filter, point cloud data containing points whose coordinates in the shooting direction exceed the target coordinate range are removed.

[0137] In one embodiment, the preprocessing module described above is further configured to:

[0138] Based on the local neighborhood density of each point in the point cloud data and a preset density threshold, the discrete point cloud data in the point cloud data is deleted.

[0139] In one embodiment, the solder joint positioning device 500 further includes an adjustment module for:

[0140] Adjust the position of the 3D vision sensor so that the distance between the 3D vision sensor and the intersection point to be located is within a preset distance range. The intersection point is the intersection point of the first straight steel bar and the second straight steel bar.

[0141] Adjust the shooting direction of the 3D vision sensor so that the angle between the shooting direction of the 3D vision sensor and the direction of the 3D vision sensor toward the intersection position is within a preset angle range.

[0142] In one embodiment, the solder joint positioning device 500 further includes a control module for:

[0143] Determine the type of the first and second straight reinforcing bars;

[0144] Based on the model number, obtain the corresponding process parameters;

[0145] The welding robot is controlled to perform welding based on the location of the weld point and process parameters.

[0146] Each module in the aforementioned solder joint positioning device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0147] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 12As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores point cloud data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a solder joint positioning method.

[0148] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0149] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0150] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0151] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0152] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0153] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0154] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for locating solder joints, characterized in that, The method includes: Based on a 3D vision sensor, point cloud data containing a first straight steel bar and a second straight steel bar are acquired. The first straight steel bar and the second straight steel bar are placed crosswise. In the shooting direction of the 3D vision sensor, the first straight steel bar occludes the second straight steel bar at the intersection. Extract the first point cloud data of the first straight steel bar and the second point cloud data of the second straight steel bar from the point cloud data, and determine the first axis of the first straight steel bar and the second axis of the second straight steel bar based on the first point cloud data and the second point cloud data, respectively. Determine a first perpendicular line that is perpendicular to both the first axis and the second axis, and a second perpendicular line that is perpendicular to both the first perpendicular line and the second axis; and determine a reference plane that is normal to the second perpendicular line and passes through the second axis. Determine the first intersection point between the reference plane and the first axis, and along the first direction of the first perpendicular line, determine the second intersection point at a distance equal to a first preset distance from the first intersection point; Points located on both sides of the second intersection point and at a distance equal to a second preset distance from the second intersection point along the second axis are identified as welding points. The first preset distance and the second preset distance are associated with a preset weld size.

2. The method according to claim 1, characterized in that, The step of extracting the first point cloud data of the first straight steel bar and the second point cloud data of the second straight steel bar from the point cloud data includes: Based on the cylinder fitting algorithm, the first intermediate point cloud data of the first straight steel bar and the second intermediate point cloud data of the second straight steel bar are extracted from the point cloud data; Clustering is performed on the first intermediate point cloud data and the second intermediate point cloud data respectively, and the cluster with the largest data volume is determined to obtain the first point cloud data and the second point cloud data.

3. The method according to claim 1, characterized in that, Prior to extracting the first point cloud data of the first straight reinforcing bar and the second point cloud data of the second straight reinforcing bar from the point cloud data, the process includes: Based on the distance between the three-dimensional vision sensor and the intersection point, the target coordinate range in the shooting direction is determined, and the coordinates of the intersection point in the shooting direction are within the target coordinate range; Based on a bandpass filter, the point cloud data of points whose coordinates in the shooting direction exceed the target coordinate range are deleted.

4. The method according to claim 3, characterized in that, Before extracting the first point cloud data of the first straight steel bar and the second point cloud data of the second straight steel bar from the point cloud data, the method further includes: Based on the local neighborhood density of each point in the point cloud data and a preset density threshold, the discrete point cloud data in the point cloud data is deleted.

5. The method according to claim 1, characterized in that, Prior to acquiring point cloud data containing the first and second straight reinforcing bars based on a 3D vision sensor, the process includes: Adjust the position of the three-dimensional vision sensor so that the distance between the three-dimensional vision sensor and the intersection point to be located is within a preset distance range. The intersection point is the intersection point of the first straight steel bar and the second straight steel bar. Adjust the shooting direction of the three-dimensional vision sensor so that the angle between the shooting direction of the three-dimensional vision sensor and the direction of the three-dimensional vision sensor toward the intersection position is within a preset angle range.

6. The method according to claim 1, characterized in that, After determining that the points located on both sides of the second intersection point and at a distance equal to a second preset distance from the second intersection point along the second axis are respectively designated as solder joints, the process includes: Determine the type of the first straight steel bar and the second straight steel bar; Based on the model number, obtain the corresponding process parameters; Based on the location of the weld point and the process parameters, the welding robot is controlled to perform welding.

7. A solder joint positioning device, characterized in that, The device includes: The acquisition module is used to acquire point cloud data containing a first straight steel bar and a second straight steel bar based on a three-dimensional vision sensor. The first straight steel bar and the second straight steel bar are placed crosswise. In the shooting direction of the three-dimensional vision sensor, the first straight steel bar occludes the second straight steel bar at the intersection. The determining module is used to extract the first point cloud data of the first straight steel bar and the second point cloud data of the second straight steel bar from the point cloud data, and to determine the first axis of the first straight steel bar and the second axis of the second straight steel bar based on the first point cloud data and the second point cloud data, respectively. The determining module is further configured to determine a first perpendicular line that is perpendicular to both the first axis and the second axis, and a second perpendicular line that is perpendicular to both the first perpendicular line and the second axis, and to determine a reference plane that is normal to the second perpendicular line and passes through the second axis. The determining module is further configured to determine the first intersection point of the reference plane and the first axis, and along the first direction of the first perpendicular line, determine the second intersection point at a distance equal to the first intersection point. The determining module is further configured to determine points located on both sides of the second intersection point and at a distance equal to a second preset distance from the second intersection point along the second axis as welding points, wherein the first preset distance and the second preset distance are associated with preset weld dimensions.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images