An automated welding method, apparatus, equipment, and storage medium for steel box girders.

Abstract

This application discloses an automated welding method, apparatus, equipment, and storage medium for steel box girders, relating to the field of automated welding technology. The method includes: acquiring local point cloud information of the target steel box girder based on multi-view photographic poses; determining overall point cloud information of the target based on the local point cloud information and a preset point cloud stitching algorithm; determining the point clouds of each weld seam from the overall point cloud information of the target, and generating an initial weld seam point set based on the point clouds of each weld seam; transforming the point clouds of each weld seam in the initial weld seam point set to the target world coordinate system to obtain each target weld seam point, and generating a target weld seam point set based on each target weld seam point; generating a target welding path based on the target weld seam point set; determining the target welding torch pose corresponding to each target weld seam point in the target welding path, and using a target robotic arm to control the target welding torch to weld the target steel box girder based on the target welding path and the target welding torch pose. This application can improve the degree of welding automation, welding accuracy, and construction efficiency.

Description

Technical Field

[0001] This invention relates to the field of automatic welding technology, and in particular to an automated welding method, apparatus, equipment and storage medium for steel box girders. Background Technology

[0002] In steel structure bridge construction, steel box girders are widely used in highway bridges, railway bridges, and urban viaducts due to their advantages such as long spans, high load-bearing capacity, and superior torsional resistance. However, the manufacturing and welding process of steel box girders is complex, especially the welding of fillet welds, which presents numerous challenges.

[0003] On the one hand, fillet welds in steel box girders can manifest as straight welds, curved welds, or discontinuous welds. Existing fillet weld extraction methods typically set trajectories for single fillet weld cases, which is insufficient to meet the diversity and complexity of fillet welds in steel box girders. On the other hand, during the manufacturing process of steel box girders, factors such as plate cutting, assembly, and welding deformation can cause deviations between the actual weld position and the design position. Traditional welding methods rely on fixed teaching trajectories, which cannot adapt to actual assembly errors, easily leading to quality problems such as weld misalignment, incomplete fusion, and uneven weld dimensions. Thirdly, welding work on steel box girders often involves working at heights, operating in confined spaces, and in high-temperature and high-light environments, resulting in high labor intensity for welders and long-term health impacts. During manual welding, the weld quality is prone to fluctuations due to factors such as worker skill, operational stability, and fatigue levels, increasing the risk of weld defects and directly affecting the long-term safety of the bridge structure. Furthermore, traditional manual welding methods are difficult to meet the demands of large-scale production.

[0004] In conclusion, improving the level of welding automation, welding accuracy, and construction efficiency are urgent technical problems that need to be solved. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide an automated welding method, apparatus, equipment, and storage medium for steel box girders, which can improve the degree of automation, welding accuracy, and construction efficiency. The specific solution is as follows:

[0006] In a first aspect, this application provides an automated welding method for steel box girders, comprising:

[0007] The target reinforcement learning model determines the target's multi-view photography pose, and the target structured light camera acquires the target's local point cloud information corresponding to the target steel box girder based on the target's multi-view photography pose. The target's overall point cloud information is determined based on the target's local point cloud information and a preset point cloud stitching algorithm. The target reinforcement learning model is a model built based on Markov decision process.

[0008] The point cloud of each weld point of the target steel box girder is determined from the overall point cloud information of the target, and an initial weld point set is generated based on each weld point point cloud.

[0009] The point cloud of each weld point in the initial weld point set is transformed to the target world coordinate system to obtain each target weld point, and a target weld point set is generated based on each target weld point, and a corresponding target welding path is generated based on the target weld point set.

[0010] The target welding torch pose corresponding to each target weld point in the target welding path is determined, and the target robotic arm is used to control the target welding torch to weld the target steel box girder based on the target welding path and the target welding torch pose.

[0011] Optionally, before determining the target's multi-view photographic pose based on the target reinforcement learning model, the method further includes:

[0012] The target ground is captured by the target structured light camera based on a preset shooting angle to generate ground point cloud information corresponding to the target ground.

[0013] The ground point cloud information is fitted using a random sampling consensus algorithm to obtain the plane equation corresponding to the target ground.

[0014] The target structured light camera is used to capture images of the target steel box girder from the preset shooting angle to generate the first point cloud information corresponding to the target steel box girder;

[0015] Determine the distance between each point in the first point cloud information and the plane equation, and determine the target distance greater than a preset distance threshold from each of the distances;

[0016] Based on the points corresponding to the target distance, an initial point cloud information corresponding to the target steel box girder is constructed, so as to determine the overall point cloud information of the target steel box girder based on the initial point cloud information; the initial point cloud information is the local point cloud information of the target steel box girder.

[0017] Optionally, the step of acquiring local point cloud information of the target steel box girder based on the target's multi-view photographic pose using a target structured light camera, and determining the overall point cloud information of the target based on the local point cloud information and a preset point cloud stitching algorithm, includes:

[0018] Based on the target multi-view photography pose, motion commands corresponding to the target robotic arm are generated, and the target robotic arm is used to control the target structured light camera to photograph the target steel box girder based on the motion commands, so as to obtain the target local point cloud information corresponding to the target multi-view photography pose;

[0019] Construct a first pyramid corresponding to the initial point cloud information, and construct a second pyramid corresponding to the target local point cloud information;

[0020] The first pyramid and the second pyramid are registered using the iterative nearest point algorithm. Based on the registration result, each point in the target local point cloud information is stitched into the initial point cloud information to obtain the target overall point cloud information corresponding to the target steel box girder.

[0021] Optionally, determining the point cloud of each weld point of the target steel box girder from the overall point cloud information of the target includes:

[0022] The initial point cloud normal vectors corresponding to each target point cloud in the overall target point cloud information are determined by principal component analysis algorithm, and the initial point cloud normal vectors are uniformized to obtain the target point cloud normal vectors. A first point cloud normal vector dataset is constructed based on the target point cloud normal vectors.

[0023] A second point cloud normal vector dataset is obtained based on the first point cloud normal vector dataset, and the ratio between the number of the second point cloud normal vector dataset and the number of the first point cloud normal vector dataset is determined.

[0024] If the ratio is less than the first preset threshold, then the point cloud of each weld point of the target steel box girder is determined based on the second point cloud normal vector dataset;

[0025] If the ratio is greater than or equal to the first preset threshold, a target normal vector is randomly selected from the second point cloud normal vector dataset, and the second point cloud normal vector dataset is updated based on the target normal vector until the ratio between the number of the updated second point cloud normal vector dataset and the number of the first point cloud normal vector dataset is less than the first preset threshold. Based on the updated second point cloud normal vector dataset, the point clouds of each weld point of the target steel box girder are determined.

[0026] Optionally, the step of transforming the point cloud of each weld point in the initial weld point set to the target world coordinate system to obtain each target weld point includes:

[0027] The first transformation matrix from the end-effector coordinate system of the target robotic arm to the camera coordinate system corresponding to the target structured light camera is determined based on the eye-on-hand calibration method.

[0028] Determine the first translation matrix and rotation matrix from the robot arm base coordinate system to the robot arm end coordinate system of the target robot arm, and determine the second transformation matrix based on the first translation matrix and the rotation matrix;

[0029] Determine the second translation matrix from the target world coordinate system to the robot arm base coordinate system, and determine the third transformation matrix based on the second translation matrix;

[0030] Based on the first transformation matrix, the second transformation matrix, and the third transformation matrix, the point clouds of each weld point in the initial weld point set are transformed to the target world coordinate system to obtain each target weld point corresponding to each weld point point cloud.

[0031] Optionally, generating the corresponding target welding path based on the target weld point set includes:

[0032] The corners and endpoints of each target weld point are determined by using a preset corner and endpoint detector, and each target weld point is determined as a target path point by taking any one of the endpoints as the starting point and using the nearest neighbor pathfinding algorithm.

[0033] An initial welding path is generated based on the target path points, and the target welding path is determined based on the initial welding path after all the target weld points are determined as the target path points.

[0034] Accordingly, the process of generating an initial welding path based on the target path points, and determining the target welding path based on the initial welding path after all the target weld points have been determined as the target path points, further includes:

[0035] If the target weld point currently determined as the target path point is the corner point, and the corner point is the weld intersection point, then determine the number of target weld points after the corner point in the initial welding path;

[0036] Based on the number of target weld points, determine whether to re-designate the current target path point as the target weld point, so as to continue to determine the target weld points as the target path points sequentially based on the nearest neighbor pathfinding algorithm.

[0037] Optionally, determining the target welding torch pose corresponding to each target weld point in the target welding path includes:

[0038] A first target vector is determined based on the current target weld point and the next target weld point.

[0039] The first cross product result is obtained by performing a cross product on the normal vector corresponding to the current target weld point and the first target vector, and the second target vector is determined based on the first cross product result.

[0040] The first target vector and the second target vector are cross-producted to obtain a second cross-product result, and a third target vector is determined based on the second cross-product result;

[0041] The target rotation matrix is ​​determined based on the first target vector, the second target vector, and the third target vector, and the target welding torch pose corresponding to the current target weld point is determined based on the target rotation matrix.

[0042] Secondly, this application provides an automated welding device for steel box girders, comprising:

[0043] The overall target point cloud information determination module is used to determine the target multi-view photography pose based on the target reinforcement learning model, and to obtain the target local point cloud information corresponding to the target steel box girder based on the target multi-view photography pose using the target structured light camera, and to determine the overall target point cloud information based on the target local point cloud information and a preset point cloud stitching algorithm; the target reinforcement learning model is a model constructed based on Markov decision process;

[0044] The initial weld point set generation module is used to determine the point cloud of each weld point of the target steel box girder from the target overall point cloud information, and generate an initial weld point set based on each weld point point cloud;

[0045] The target welding path generation module is used to transform the point cloud of each weld point in the initial weld point set to the target world coordinate system to obtain each target weld point, generate a target weld point set based on each target weld point, and generate a corresponding target welding path based on the target weld point set.

[0046] The target steel box girder welding module is used to determine the target welding torch pose corresponding to each target weld point in the target welding path, and to use a target robotic arm to control the target welding torch to weld the target steel box girder based on the target welding path and the target welding torch pose.

[0047] Thirdly, this application provides an electronic device, comprising:

[0048] Memory, used to store computer programs;

[0049] A processor is used to execute the computer program to implement the aforementioned automated welding method for steel box girders.

[0050] Fourthly, this application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned automated welding method for steel box girders.

[0051] In this application, firstly, the target multi-view photography pose is determined based on a target reinforcement learning model, and the target structured light camera is used to acquire the target local point cloud information corresponding to the target steel box girder based on the target multi-view photography pose. Then, the target overall point cloud information is determined based on the target local point cloud information and a preset point cloud stitching algorithm. The target reinforcement learning model is a model constructed based on a Markov decision process. Next, the point clouds of each weld point of the target steel box girder are determined from the target overall point cloud information, and an initial weld point set is generated based on each weld point point cloud. Subsequently, the point clouds of each weld point in the initial weld point set are transformed to the target world coordinate system to obtain each target weld point, and a target weld point set is generated based on each target weld point. A corresponding target welding path is also generated based on the target weld point set. Finally, the target welding torch pose corresponding to each target weld point in the target welding path is determined, and a target robotic arm is used to control the target welding torch to weld the target steel box girder based on the target welding path and the target welding torch pose. As can be seen from the above, this application uses a target reinforcement learning model based on a Markov decision process to determine the target multi-view photographic pose of the target structured light camera. The target structured light camera then uses this pose to determine the target local point cloud information of the target steel box girder. Based on this local point cloud information and a pre-defined point cloud stitching algorithm, the overall point cloud information of the target steel box girder is determined. Next, the point clouds of each weld seam are determined from the overall point cloud information, and an initial weld seam point set is generated. Then, the point clouds of each weld seam point in the initial weld seam point set are transformed to the target world coordinate system to obtain each target weld seam point, generating the target weld seam point set and the corresponding target welding path. Finally, the target welding torch pose corresponding to each target weld seam point in the target welding path is determined. Using a target robotic arm, the target welding torch is controlled to weld the target steel box girder according to the target welding path and the target welding torch pose. In this way, this application can automatically identify weld seams based on the shape of the steel box girder, improving the applicability of weld seam extraction, increasing the flexibility and intelligence of the welding process, and providing stronger adaptability for automated welding. Meanwhile, this application automatically extracts weld seams and identifies welding paths without manual intervention, ensuring the accuracy of the welding trajectory and reducing human error. Furthermore, this application can handle complex components, improving the versatility and adaptability of welding. This application eliminates the need for manual teaching, reducing pre-welding preparation time and enhancing welding automation, precision, and construction efficiency. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0053] Figure 1 A flowchart of an automated welding method for steel box girders is provided in this application;

[0054] Figure 2 This application provides a schematic diagram of a specific weld extraction result; wherein, Figure 2 (a) in the figure is a schematic diagram of the extracted curved weld. Figure 2 (b) in the figure is a schematic diagram of the extracted straight weld.

[0055] Figure 3 This application provides a specific schematic diagram of a robotic arm and a world coordinate system;

[0056] Figure 4 This application provides a specific path planning flowchart;

[0057] Figure 5 This application provides a specific schematic diagram of the welding torch position;

[0058] Figure 6 This application provides a specific photographic pose diagram;

[0059] Figure 7 This application provides a flowchart of a specific automated welding method for steel box girders;

[0060] Figure 8 A schematic diagram of an automated welding device for steel box girders is provided for this application;

[0061] Figure 9 This application provides a structural diagram of an electronic device. Detailed Implementation

[0062] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] The manufacturing and welding process of steel box girders is complex, especially the welding of fillet welds, which faces numerous challenges. Firstly, fillet welds in steel box girders can be straight, curved, or discontinuous. Existing fillet weld extraction methods typically set trajectories for single fillet weld cases, making it difficult to meet the diversity and complexity of steel box girder fillet welds. Secondly, during the manufacturing process of steel box girders, factors such as plate cutting, assembly, and welding deformation can cause deviations between the actual weld position and the design position. Traditional welding methods rely on fixed teaching trajectories, which cannot adapt to actual assembly errors, easily leading to quality problems such as weld misalignment, incomplete fusion, and uneven weld dimensions. Thirdly, the welding of steel box girders often involves working at heights, in confined spaces, and in high-temperature, high-light environments, resulting in high labor intensity for welders and long-term health risks. During manual welding, weld quality is prone to fluctuations due to factors such as worker skill, operational stability, and fatigue levels, increasing the risk of weld defects and directly impacting the long-term safety of the bridge structure. Furthermore, traditional manual welding methods are insufficient for large-scale production needs. Therefore, this application provides an automated welding solution for steel box girders, which can improve the degree of automation, welding accuracy and construction efficiency.

[0064] See Figure 1 As shown in the figure, an embodiment of the present invention discloses an automated welding method for steel box girders, which may include:

[0065] Step S11: Determine the target's multi-view photography pose based on the target reinforcement learning model, and use the target structured light camera to obtain the target's local point cloud information corresponding to the target steel box girder based on the target's multi-view photography pose, and determine the target's overall point cloud information based on the target's local point cloud information and a preset point cloud stitching algorithm; the target reinforcement learning model is a model constructed based on Markov decision process.

[0066] It should be noted that for three-dimensional components in steel structure bridge projects, a single photograph is insufficient to obtain a complete point cloud of the component. Instead, multiple perspective photographs are required, and the point clouds from each perspective must be stitched together to construct a complete point cloud of the component. This allows for the extraction of weld information and the planning of welding paths. In this embodiment, to achieve multi-perspective photography and accurately construct a complete point cloud of the steel box girder, an incomplete point cloud of the steel box girder is used as the initial calculation basis. To accurately obtain the incomplete point cloud of the steel box girder, the point cloud acquired by the target structured light camera needs to be filtered. Specifically, before determining the target's multi-view photography pose based on the target reinforcement learning model, the process further includes: firstly, using the target structured light camera to photograph the target ground from a preset shooting angle to generate ground point cloud information corresponding to the target ground; then, fitting the ground point cloud information using a random sample consensus algorithm to obtain the plane equation corresponding to the target ground; subsequently, using the target structured light camera to photograph the target steel box girder from the preset shooting angle to generate first point cloud information corresponding to the target steel box girder; then, determining the distance between each point in the first point cloud information and the plane equation, and identifying a target distance greater than a preset distance threshold from each distance; finally, constructing initial point cloud information corresponding to the target steel box girder based on the points corresponding to the target distances, so as to determine the overall target point cloud information corresponding to the target steel box girder based on the initial point cloud information; the initial point cloud information is the local point cloud information of the target steel box girder. Specifically, firstly, the target structured light camera is used to take an overhead shot of the target ground to obtain ground point cloud information, and the RANSAC (Random Sample Consensus) algorithm is used to fit the ground point cloud information to obtain the plane equation of the ground, denoted as . It should be noted that the step of generating the planar equations of the ground only needs to be performed once at the same site. Afterwards, the target steel box girder is placed on the ground, and a target structured light camera is used to take an overhead image to obtain the first point cloud information. The distances from each point in the first point cloud information to the target are then calculated. The distance, with the upward direction from the ground as positive, is used to construct initial point cloud information based on points above the plane, denoted as . This refers to the incomplete point cloud information of the target steel box girder.

[0067] In this embodiment, to obtain the complete point cloud of the target steel box girder, the above-mentioned acquisition of the target local point cloud information corresponding to the target steel box girder using a target structured light camera based on the target multi-view photography pose, and determination of the target overall point cloud information based on the target local point cloud information and a preset point cloud stitching algorithm, may include: firstly, generating motion commands corresponding to the target robotic arm based on the target multi-view photography pose, and using the target robotic arm to control the target structured light camera to photograph the target steel box girder based on the motion commands, so as to obtain the target local point cloud information corresponding to the target multi-view photography pose; then constructing a first pyramid corresponding to the initial point cloud information, and constructing a second pyramid corresponding to the target local point cloud information; finally, performing point cloud registration on the first pyramid and the second pyramid based on the iterative nearest point algorithm, and stitching each point in the target local point cloud information to the initial point cloud information based on the registration result, so as to obtain the target overall point cloud information corresponding to the target steel box girder. Understandably, this embodiment can sequentially send motion commands corresponding to the target's multi-view photography poses to the robotic arm, causing the target structured light camera at the end of the robotic arm to move to a designated position to photograph the target steel box girder. Using incomplete initial point cloud information as a baseline, a point cloud stitching algorithm is used to stitch together the local point cloud information of the target captured from multiple angles and the initial point cloud information into complete overall point cloud information of the target steel box girder. Specifically, the point cloud stitching algorithm in this embodiment employs the Pyramid ICP (Iterative Closest Point) algorithm for 3D point cloud registration. The core idea of ​​the Pyramid ICP algorithm is to construct a multi-level pyramid structure and perform ICP registration layer by layer from coarse to fine, thereby improving the robustness and convergence speed of the registration. The implementation process of the Pyramid ICP algorithm is as follows:

[0068] (1) Pyramid Construction: First, pyramid structures are constructed for the initial point cloud information and the target local point cloud information respectively. Taking the initial point cloud information as an example, the process of constructing the first pyramid corresponding to the initial point cloud information includes downsampling the initial point cloud information layer by layer to generate point cloud sets with multiple resolution levels. Assuming that the pyramid contains n layers, the 0th layer is the coarsest layer, and the point cloud density can be significantly reduced by methods such as voxel lattice filtering or uniform downsampling. As the number of layers increases, the resolution of each layer gradually increases until the (n-1)th layer is the original resolution or a lightly downsampled point cloud. The pyramid structure can preserve the geometric features of the point cloud at different scales, which is convenient for subsequent layer-by-layer alignment.

[0069] (2) Layer-by-layer ICP stitching: During the stitching process, standard ICP registration is performed starting from layer 0, i.e. the lowest resolution pyramid layer. The transformation matrix is ​​solved by iteratively calculating the correspondence between each point in the target local point cloud information and the nearest point in the initial point cloud information. This minimizes the distance error between the two point clouds, completing the initial alignment. The resulting transformation matrix... As the initial estimate for the next layer of registration, this process continues, using the transformation matrix of the previous layer as the initial value in each layer to perform ICP iterations until registration is completed at the nth layer, yielding the transformation matrix. , The final required transformation matrix .

[0070] In this way, by transforming the points in the local point cloud information of all targets through a transformation matrix... When converted to the initial point cloud information, it can be merged into a complete point cloud dataset, that is, the complete overall point cloud information of the target steel box girder.

[0071] Step S12: Determine the point cloud of each weld point of the target steel box girder from the overall point cloud information of the target, and generate an initial weld point set based on each weld point point cloud.

[0072] In this embodiment, to accurately and quickly extract the welds of the target steel box girder, the above-mentioned determination of the point clouds of each weld point of the target steel box girder from the overall point cloud information of the target includes: firstly, determining the initial point cloud normal vector corresponding to each target point cloud in the overall point cloud information of the target using principal component analysis algorithm, and performing a uniformization process on each initial point cloud normal vector to obtain each target point cloud normal vector, and constructing a first point cloud normal vector dataset based on each target point cloud normal vector; then, obtaining a second point cloud normal vector dataset based on the first point cloud normal vector dataset, and determining the number of the second point cloud normal vector dataset and the number of the first point cloud normal vector dataset. The ratio between the number of points in the first point cloud normal vector dataset and the number of points in the second point cloud normal vector dataset is used to determine the point cloud of each weld point of the target steel box girder. If the ratio is less than a first preset threshold, the point cloud of each weld point of the target steel box girder is determined based on the second point cloud normal vector dataset. If the ratio is greater than or equal to the first preset threshold, a target normal vector is randomly selected from the second point cloud normal vector dataset, and the second point cloud normal vector dataset is updated based on the target normal vector until the ratio between the number of points in the updated second point cloud normal vector dataset and the number of points in the first point cloud normal vector dataset is less than the first preset threshold. The point cloud of each weld point of the target steel box girder is then determined based on the updated second point cloud normal vector dataset. Specifically, the process of determining the initial weld point set of the target steel box girder using a preset weld extraction algorithm is as follows:

[0073] Step 1: Calculate the overall point cloud information of the target object using the PCA (Principal Component Analysis) algorithm. The initial point cloud normal vectors corresponding to each target point cloud.

[0074] Step 2: A global method based on graph optimization is used to unify the initial point cloud normal vectors, obtaining the normal vectors of each target point cloud. Based on these target point cloud normal vectors, a first point cloud normal vector dataset is constructed, defined as follows: The number is .

[0075] Step 3: Copy the first point cloud normal vector dataset The second point cloud normal vector dataset is obtained. The number is .

[0076] Step 4: Determine if the condition is met: , This is the first preset threshold. If the condition is met, then it is directly based on... Determine the initial weld point set If the conditions are not met, proceed to the next step.

[0077] Step 5: Random selection A target normal vector ,calculate and Find the angle between all normal vectors in the vector and generate an angle set. .

[0078] Step 6: Select the angle set middle Overall point cloud information of the target The corresponding target point cloud is recorded in the initial weld point set. In; among them, This is the second preset threshold.

[0079] Step 7: Determine the angle set middle The corresponding number of vectors, i.e. The amount added in, and calculated in The proportion of Make a judgment: if , If the third preset threshold is used, then... from Delete it and jump to step four. Otherwise, begin step five. Select again. The target normal vector in .

[0080] In this way, the initial weld point set of the target steel box girder can be obtained through the above process. The extraction results of welds at different parts of the steel box girder are as follows: Figure 2 As shown, Figure 2The red line in (a) represents the extracted curved weld of the steel box girder. Figure 2 The red line in (b) represents the extracted straight weld seam of the steel box girder. It should be noted that the preset weld seam extraction algorithm in this embodiment only requires about 6 iterations to extract the weld seam of the steel box girder. Compared to previous methods that require traversing every point in the overall point cloud information of the target to calculate its corresponding feature value, this greatly improves computational efficiency. With the same number of points, the computational efficiency of the preset weld seam extraction algorithm in this embodiment is approximately 100 to 10,000 times that of other algorithms. Furthermore, the preset weld seam extraction algorithm in this embodiment establishes rules based on the angle between the normal vectors of point pairs. In previous algorithms, after performing separate plane or surface fitting, the intersection line of the plane or surface is calculated to obtain the weld seam position. The fitting method requires manual setting of whether the fitting target is a plane or a surface, making it impossible to make an independent judgment. However, the rules used by the preset weld seam extraction algorithm in this embodiment are independent of whether it is a plane or a surface, making it more widely applicable.

[0081] Step S13: Transform the point cloud of each weld point in the initial weld point set to the target world coordinate system to obtain each target weld point, generate a target weld point set based on each target weld point, and generate a corresponding target welding path based on the target weld point set.

[0082] In this embodiment, before processing the point cloud of each weld point in the initial weld point set into motion commands for the robotic arm, it is necessary to transform the point cloud of each weld point from the camera coordinate system to the real-world coordinate system so that the weld information can be processed into motion commands for the robotic arm. A schematic diagram of the robotic arm and the world coordinate system is shown below. Figure 3 As shown. Specifically, the above-mentioned transformation of the point cloud of each weld point in the initial weld point set to the target world coordinate system to obtain each target weld point includes: firstly, determining a first transformation matrix from the end-effector coordinate system of the target robotic arm to the camera coordinate system corresponding to the target structured light camera based on the eye-on-hand calibration method; then determining a first translation matrix and a rotation matrix from the base coordinate system of the target robotic arm to the end-effector coordinate system, and determining a second transformation matrix based on the first translation matrix and the rotation matrix; subsequently determining a second translation matrix from the target world coordinate system to the base coordinate system of the robotic arm, and determining a third transformation matrix based on the second translation matrix; finally, transforming the point cloud of each weld point in the initial weld point set to the target world coordinate system based on the first transformation matrix, the second transformation matrix, and the third transformation matrix to obtain each target weld point corresponding to each weld point point cloud. Specifically, firstly, the first transformation matrix from the end-effector coordinate system of the target robotic arm to the camera coordinate system corresponding to the target structured light camera is obtained using the eye-on-hand calibration method. Next, in order to determine the second transformation matrix from the robot arm base coordinate system to the robot arm end effector coordinate system of the target robot arm, This can be calculated using the pose of the robotic arm's end effector on the teach pendant and the pose of the robotic arm's base coordinate system. Specifically, considering the dimensionality issue of matrix multiplication, a fourth dimension is introduced into the definition of spatial point coordinates. Adding the fourth dimension 1 represents homogeneous coordinates. Then, in the robotic arm's end effector coordinate system, the coordinate column vector is... The attitude of the robotic arm's end effector will be displayed on the teach pendant. This posture represents the translation of the robot arm's base coordinate system. Rotate around its own coordinate axis Only in this way can the coordinates of the robot arm base coordinate system and the robot arm end effector coordinate system be made to coincide.

[0083] Therefore, the translation matrix for the transformation between the robot arm base coordinate system and the robot arm end effector coordinate system is... Rotation matrix around the X-axis Rotation matrix around the Y-axis Rotation matrix around the Z-axis As shown below:

[0084] ;

[0085] ;

[0086] ;

[0087] ;

[0088] It can be seen that the total rotation matrix between the robot arm base coordinate system and the robot arm end effector coordinate system is: Therefore, for a point in the coordinate system of the robotic arm's end effector... The expression in the coordinate system of the robotic arm base is as follows Finally, to transform the robot arm's base coordinate system to the target world coordinate system, this embodiment considers the robot arm's seventh axis of travel, and sets the target world coordinate system at the end of the seventh axis. When the target world coordinate system and the seventh axis of the robot arm's travel have the same coordinate axis direction, the third transformation matrix from the target world coordinate system to the robot arm's base coordinate system is... , is a translation transformation matrix, and The derivation is similar. In summary, the target transformation matrix from the target world coordinate system to the camera coordinate system can be obtained. As shown below:

[0089] ;

[0090] Therefore, any point in the camera coordinate system can be represented in the target world coordinate system as follows: In this way, the point cloud of each weld point in the initial weld point set can be transformed to the target world coordinate system, obtaining the target weld point corresponding to each weld point point cloud. This coordinate transformation facilitates unified processing of the point cloud and control of the pose of the robotic arm's end effector.

[0091] Understandably, welding steel box girders requires a specific welding sequence. Therefore, path planning for the weld points is necessary to ensure the robotic arm can perform welding smoothly and systematically. This embodiment uses a nearest neighbor algorithm for path planning. Since multiple welds intersect within the steel box girder, these intersections can serve as the start and end points of both welds, requiring multiple uses. In contrast, previous pathfinding algorithms only allowed each node to be used once. To address this, this embodiment employs a delayed revival mechanism for weld intersections, enabling them to be used multiple times.

[0092] In one specific embodiment, the above-mentioned generation of a corresponding target welding path based on the target weld point set may include: firstly, using a preset corner and endpoint detector to determine the corners and endpoints of each of the target weld points, and taking any one of the endpoints as the starting point, sequentially determining each of the target weld points as a target path point based on a nearest neighbor pathfinding algorithm; then, generating an initial welding path based on the target path points, until all the target weld points are determined as the target path points, and finally determining the target welding path based on the initial welding path. Correspondingly, the process of generating an initial welding path based on the target path points, and then determining the target welding path based on the initial welding path after all the target weld points are determined as the target path points, may further include: if the target weld point currently determined as a target path point is a corner point, and the corner point is a weld intersection point, then determining the number of target weld points after the corner point in the initial welding path; and determining, based on the number of target weld points, whether to re-designate the current target path point as the target weld point, so as to continue sequentially determining the target weld points as the target path points based on the nearest neighbor pathfinding algorithm. It should be noted that, to avoid redundant calculations caused by excessively dense weld points in subsequent path planning and to ensure trajectory generation efficiency, voxel downsampling can be performed on the target weld points first to extract sparse target weld points, which are then used as the new target weld points. The subsequent target welding path planning process is as follows:

[0093] Step 1: Feature Point Detection. The preset corner and endpoint detector is used to detect two feature points in the target weld: corners and endpoints. The principle of the preset corner and endpoint detector is as follows:

[0094] (1) Select one of the target weld points. ;

[0095] (2) Selection and The three nearest weld points ;

[0096] (3) Calculation and vector With distance ,calculate The cross product of any two vectors in the vector array is used to calculate the angle between the cross product and the third vector, resulting in three angles. ;

[0097] (4) Calculate the vector The sum of modulus and distance and and calculate average angle ;

[0098] (5) Judgment: ,and ,but If a point is a corner point, add it to the corner point set; ,and ,but If a point is an endpoint, it is placed into the endpoint set; if the above conditions are not met, it is an interior point; where threshold 4 represents the fourth preset threshold and threshold 5 represents the fifth preset threshold.

[0099] Step 2: The nearest neighbor algorithm is used for path planning.

[0100] (1) Start pathfinding: Start the nearest neighbor pathfinding process by taking one of the endpoints in the endpoint set as the starting point;

[0101] (2) Introduce a preset corner point revival mechanism. During the pathfinding process, the target weld point is determined as the target path point in sequence, and the current path point is checked to see if it is a corner point.

[0102] (3) If it is a corner point, check if there is a chance to revive;

[0103] (4) If the corner point is a weld intersection, there is a chance to revive. Use the revival opportunity and start the delayed revival mechanism. Delayed revival mechanism: When the corner point is revived, record the number of steps taken from that corner point. When the number of steps equals the sixth preset threshold, put the corner point back into the weld point where no path has been planned.

[0104] (5) If the current path point is not a corner point, or the current path point has no chance to be revived, then proceed with the path finding of the next nearest neighbor point as usual, and continue with path planning.

[0105] In this way, path planning can be performed based on the nearest neighbor algorithm in this embodiment, and combined with the preset delay revival mechanism for corner points, to ensure that the path planning can take into account the need to repeatedly traverse corner points. At the same time, this embodiment can use a Gaussian filter to smooth the target welding path.

[0106] It should be noted that the above implementation method uses the target weld points in the target world coordinate system for path planning. Alternatively, path planning can also be performed on the point cloud of each weld point in the camera coordinate system; the path planning process is described in [link to documentation]. Figure 4 As shown. Specifically, firstly, sparse point cloud data corresponding to the point cloud of each weld seam is extracted; then, a preset corner and endpoint detector is used to detect two feature points in the sparse point cloud data: corners and endpoints; then, path planning is performed on the sparse point cloud data based on the nearest neighbor pathfinding algorithm and the preset delayed revival mechanism for corners; finally, a Gaussian filter is used to smooth the welding path.

[0107] Step S14: Determine the target welding torch pose corresponding to each target weld point in the target welding path, and use the target robotic arm to control the target welding torch to weld the target steel box girder based on the target welding path and the target welding torch pose.

[0108] It is understandable that during welding, not only does the welding wire tip need to accurately reach the weld position, but the angle between the welding wire and the weld also needs to meet certain requirements to ensure welding quality. Since the welding wire and welding torch positions are directly related, this embodiment requires calculating the target welding torch pose to meet the welding requirements. The determination of the target welding torch pose corresponding to each target weld point in the target welding path can include: first, determining a first target vector based on the current target weld point and the next target weld point; then, performing a cross product between the normal vector corresponding to the current target weld point and the first target vector to obtain a first cross product result, and determining a second target vector based on the first cross product result; then, performing a cross product between the first target vector and the second target vector to obtain a second cross product result, and determining a third target vector based on the second cross product result; finally, determining a target rotation matrix based on the first target vector, the second target vector, and the third target vector, and determining the target welding torch pose corresponding to the current target weld point based on the target rotation matrix. Specifically, the position information of each target weld point in the target welding path is first obtained. Taking the current target weld point as an example, firstly, the first target vector corresponding to the current target weld point is determined, which is the welding torch travel vector. The welding torch travel vector points from the current target weld point to the next target weld point. Then, the cross product of the normal vector of the current target weld point and the welding torch travel vector is performed to obtain the first cross product result, which is the second target vector corresponding to the current target weld point, i.e., the welding torch deflection. Next, the cross product of the welding torch travel vector and the welding torch deflection is performed to obtain the second cross product result, which is the third target vector corresponding to the current target weld point, i.e., the welding torch normal vector. Finally, the target rotation matrix can be determined from the welding torch normal vector, the welding torch travel vector, and the welding torch deflection, thus obtaining the target welding torch pose corresponding to the current target weld point.

[0109] See Figure 5 As shown, For each target weld point in the target welding path, let For the current target weld point, then This is the welding torch travel vector corresponding to the current target weld point. This represents the welding torch deflection amount corresponding to the current target weld point. This is the welding torch normal vector corresponding to the current target weld point. Then, the welding torch travel vector... Welding torch deflection welding torch normal vector These vectors, acting as column vectors or row vectors, are combined to form the target rotation matrix. Based on this target rotation matrix, the current target weld point can be further determined. The corresponding target welding torch position.

[0110] This embodiment combines the obtained target welding path and target welding torch pose to form a list of motion command points for the robotic arm. Then, it selects a suitable welding process for the steel box girder and issues motion and welding commands to the target robotic arm, enabling the robotic arm to control the target welding torch to weld the target steel box girder based on these commands.

[0111] As can be seen from the above, in this embodiment, the target multi-view photography pose is first determined based on the target reinforcement learning model, and the target structured light camera is used to obtain the target local point cloud information corresponding to the target steel box girder based on the target multi-view photography pose. The target overall point cloud information is determined based on the target local point cloud information and a preset point cloud stitching algorithm. The target reinforcement learning model is a model constructed based on Markov decision process. Then, the point clouds of each weld point of the target steel box girder are determined from the target overall point cloud information, and an initial weld point set is generated based on each weld point point cloud. Subsequently, the point clouds of each weld point in the initial weld point set are transformed to the target world coordinate system to obtain each target weld point, and a target weld point set is generated based on each target weld point. A corresponding target welding path is also generated based on the target weld point set. Finally, the target welding torch pose corresponding to each target weld point in the target welding path is determined, and the target robotic arm is used to control the target welding torch to weld the target steel box girder based on the target welding path and the target welding torch pose. In this way, this embodiment can automatically identify welds based on the shape of the steel box girder, improving the applicability of weld extraction, enhancing the flexibility and intelligence of the welding process, and providing stronger adaptability for automated welding. Simultaneously, by automatically extracting welds, this embodiment can identify the welding path without manual intervention, ensuring the accuracy of the welding trajectory and reducing human error. Furthermore, this embodiment can handle complex components, improving the versatility and adaptability of welding. This embodiment requires no manual teaching, reducing pre-welding preparation time and improving the degree of welding automation, welding accuracy, and construction efficiency.

[0112] Next, in order to improve the integrity and accuracy of point cloud stitching, this embodiment will elaborate in detail on how to use a target reinforcement learning model to determine the pose for multi-view photography.

[0113] It should be noted that this embodiment uses MDP (Markov Decision Process) as the basic framework for reinforcement learning, constructing the target reinforcement learning model through four elements: State, Action, Reward, and Policy. The reinforcement learning model continuously interacts with the environment, optimizing the shooting path and angle, and gradually generating a series of high-quality viewpoint poses that effectively cover the weld area of ​​the steel box girder, thereby improving the integrity and accuracy of point cloud stitching. The specific contents of the four elements are as follows:

[0114] (1) State: includes the current three-dimensional pose of the target structured light camera, i.e., position and orientation, as well as the completeness information of the currently acquired point cloud data, including known and unknown areas, the unknown areas being the missing parts of the point cloud data;

[0115] (2) Action: Represents the movement and attitude adjustment of the target structured light camera in space, thereby reflecting the changes in the overall point cloud data;

[0116] (3) Reward function: used to measure the contribution of the current action to the point cloud completion effect, and its definition is as follows:

[0117] ;

[0118] in, The reward is for incremental information discovery, specifically the proportion of new point clouds found. ,in, Add point cloud data after capturing the image in the current pose. This represents the current number of all point clouds. This is a coverage bonus, specifically an improvement in global point cloud coverage. ,in, The area covered by the image after capturing the current pose is the area of ​​the unknown region. The area of ​​the unknown region; Point cloud redundancy penalty, ,in, For redundant point cloud data, The number of point clouds obtained during this shoot. To determine the severity of the punishment; To provide smooth movement rewards, through This can prevent drastic changes. ,in, This is the current pose for taking the photo. For the previous photo pose, For smoothing weights; For hyperparameters, the impact of different terms in the reward function is weighed.

[0119] (4) Policy: Used to learn the optimal photo pose sequence, and accurately obtain the target multi-view photo pose for multi-view stitching while ensuring sufficient point cloud coverage and stitching accuracy.

[0120] In this embodiment, the target reinforcement learning model is pre-trained, and the training process includes the following four stages:

[0121] (1) Data collection: Collect existing incomplete point cloud data and label the missing regions as the target for model learning. The labeled information is used to guide the model to identify key completion regions in the point cloud, thereby improving the pertinence and effectiveness of the photo pose selection.

[0122] (2) Environment setup: A virtual environment was built using the PyBullet simulation platform to simulate the shooting process of a real target structured light camera. The simulation environment supports the camera to move and rotate in three-dimensional space, which can realistically reflect the impact of different perspectives on the point cloud acquisition effect, and provide a controllable, safe and reproducible interactive scenario for model training.

[0123] (3) Reinforcement learning training: In the simulation environment, the target structured light camera starts from a random position and continuously adjusts its pose through continuous interaction with the environment. The model learns a set of photo-taking pose schemes that can effectively fill in the missing areas of the point cloud by optimizing the strategy based on the reward function feedback. The training objective is to maximize the point cloud coverage and the quality of the constructed complete point cloud.

[0124] (4) Model Validation: The trained target reinforcement learning model is applied to a real-world environment for testing to verify its generalization ability and effectiveness under actual conditions. By comparing the point cloud completion results, the performance of the target reinforcement learning model in terms of accuracy, efficiency, and stability is evaluated.

[0125] In this embodiment, the initial incomplete point cloud information of the target steel box girder can first be obtained using a target structured light camera. Then, a target reinforcement learning model can be used to infer the target's multi-view photographic poses from the target structured light camera, allowing the acquisition of local point cloud information corresponding to each multi-view photographic pose. This local point cloud information is then stitched into the initial point cloud information to obtain the complete overall point cloud information of the target steel box girder, facilitating the subsequent extraction of the overall weld seam. The target multi-view photographic poses inferred by the target reinforcement learning model can be found in [reference needed]. Figure 6 As shown.

[0126] As can be seen from the above, this embodiment uses Markov decision processes as the basic framework for reinforcement learning, constructs a reinforcement learning model, and builds a virtual environment on a simulation platform. In the virtual environment, the reinforcement learning model is trained, allowing it to optimize strategies based on the reward function feedback, gradually learning a set of photographic pose schemes that can effectively complete the missing areas of the point cloud, thus obtaining the target reinforcement learning model. In this way, the target reinforcement learning model can be used to infer the target's multi-view photographic poses from the structured light camera. By collecting the local point cloud information corresponding to each target's multi-view photographic pose, the complete overall point cloud information of the target steel box girder can be stitched together.

[0127] In one specific implementation, see Figure 7As shown, the specific process of the automated welding method for steel box girders can be as follows: First, incomplete point cloud information of the component is obtained based on a point cloud filtering algorithm. Then, the precise positioning and photographing pose of the component weld is calculated based on a reinforcement learning algorithm. Subsequently, the point cloud information corresponding to each photographing pose is obtained, and the complete point cloud information of the component is determined based on a point cloud stitching algorithm and the incomplete point cloud information of the component. Then, the weld of the component is extracted from the complete point cloud information of the component based on a weld extraction algorithm. Then, coordinate system transformation calculation is performed to transform the weld position to the world coordinate system. Then, welding path planning is performed for each welding point based on a pathfinding algorithm. Then, the welding torch pose corresponding to each welding point in the welding path is solved. Finally, the automated welding of the steel box girder is executed by issuing a command based on the welding torch pose and the welding path.

[0128] Accordingly, see Figure 8 As shown in the illustration, this application also provides an automated welding device for steel box girders, which may include:

[0129] The overall target point cloud information determination module 11 is used to determine the target multi-view photography pose based on the target reinforcement learning model, and to obtain the target local point cloud information corresponding to the target steel box girder based on the target multi-view photography pose using the target structured light camera, and to determine the overall target point cloud information based on the target local point cloud information and a preset point cloud stitching algorithm; the target reinforcement learning model is a model constructed based on the Markov decision process;

[0130] The initial weld point set generation module 12 is used to determine the point cloud of each weld point of the target steel box girder from the target overall point cloud information, and generate an initial weld point set based on each weld point point cloud.

[0131] The target welding path generation module 13 is used to transform the point cloud of each weld point in the initial weld point set to the target world coordinate system to obtain each target weld point, generate a target weld point set based on each target weld point, and generate a corresponding target welding path based on the target weld point set.

[0132] The target steel box girder welding module 14 is used to determine the target welding torch posture corresponding to each target weld point in the target welding path, and to control the target welding torch to weld the target steel box girder using the target robotic arm based on the target welding path and the target welding torch posture.

[0133] In some specific embodiments, the automated welding device for steel box girders may further include:

[0134] The ground point cloud information generation module is used to capture the target ground using the target structured light camera based on a preset shooting angle, so as to generate ground point cloud information corresponding to the target ground.

[0135] The plane equation determination module is used to fit the ground point cloud information based on the random sampling consensus algorithm to obtain the plane equation corresponding to the target ground.

[0136] The first point cloud information generation module is used to capture the target steel box girder using the target structured light camera based on the preset shooting angle, so as to generate the first point cloud information corresponding to the target steel box girder;

[0137] The target distance determination module is used to determine the distance between each point in the first point cloud information and the plane equation, and to determine the target distance that is greater than a preset distance threshold from each of the distances;

[0138] An initial point cloud information construction module is used to construct initial point cloud information corresponding to the target steel box girder based on the points corresponding to the target distance, so as to determine the overall point cloud information of the target steel box girder based on the initial point cloud information; the initial point cloud information is the local point cloud information of the target steel box girder.

[0139] In some specific embodiments, the target overall point cloud information determination module 11 may include:

[0140] The target local point cloud information determination unit is used to generate motion commands corresponding to the target robotic arm based on the target multi-view photography pose, and use the target robotic arm to control the target structured light camera to photograph the target steel box girder based on the motion commands, so as to obtain the target local point cloud information corresponding to the target multi-view photography pose;

[0141] A pyramid construction unit is used to construct a first pyramid corresponding to the initial point cloud information and a second pyramid corresponding to the target local point cloud information.

[0142] The target overall point cloud information determination unit is used to perform point cloud registration on the first pyramid and the second pyramid based on the iterative nearest point algorithm, and to stitch the points in the target local point cloud information to the initial point cloud information based on the registration result, so as to obtain the target overall point cloud information corresponding to the target steel box girder.

[0143] In some specific embodiments, the initial weld point set generation module 12 may include:

[0144] The first point cloud normal vector dataset construction unit is used to determine the initial point cloud normal vector corresponding to each target point cloud in the overall target point cloud information through the principal component analysis algorithm, and to perform a uniformization process on each initial point cloud normal vector to obtain each target point cloud normal vector, and to construct the first point cloud normal vector dataset based on each target point cloud normal vector.

[0145] The ratio determination unit is used to obtain a second point cloud normal vector dataset based on the first point cloud normal vector dataset, and to determine the ratio between the number of the second point cloud normal vector dataset and the number of the first point cloud normal vector dataset.

[0146] The first weld point cloud determination unit is used to determine the point clouds of each weld point of the target steel box girder based on the second point cloud normal vector dataset when the ratio is less than a first preset threshold.

[0147] The second weld point cloud determination unit is used to randomly select a target normal vector from the second point cloud normal vector dataset when the ratio is greater than or equal to the first preset threshold, and update the second point cloud normal vector dataset based on the target normal vector until the ratio between the number of the updated second point cloud normal vector dataset and the number of the first point cloud normal vector dataset is less than the first preset threshold, and determine each weld point cloud of the target steel box girder based on the updated second point cloud normal vector dataset.

[0148] In some specific embodiments, the target welding path generation module 13 may include:

[0149] The first transformation matrix determination unit is used to determine the first transformation matrix from the end-arm coordinate system of the target robotic arm to the camera coordinate system corresponding to the target structured light camera based on the eye-on-hand calibration method.

[0150] The second transformation matrix determination unit is used to determine the first translation matrix and rotation matrix from the robot arm base coordinate system to the robot arm end coordinate system of the target robot arm, and to determine the second transformation matrix based on the first translation matrix and the rotation matrix.

[0151] The third transformation matrix determination unit is used to determine the second translation matrix from the target world coordinate system to the robot arm base coordinate system, and to determine the third transformation matrix based on the second translation matrix;

[0152] The target weld point determination unit is used to transform the point cloud of each weld point in the initial weld point set to the target world coordinate system based on the first transformation matrix, the second transformation matrix and the third transformation matrix, so as to obtain each target weld point corresponding to each weld point point cloud.

[0153] In some specific embodiments, the target welding path generation module 13 may include:

[0154] The target path point determination submodule is used to determine the corners and endpoints of each of the target weld points using a preset corner and endpoint detector, and to determine each of the target weld points as target path points sequentially based on the nearest neighbor pathfinding algorithm, starting from any of the endpoints.

[0155] The target welding path determination submodule is used to generate an initial welding path based on the target path points, and to determine the target welding path based on the initial welding path after all the target weld points are determined as the target path points.

[0156] Accordingly, the target welding path determination submodule may further include:

[0157] The target weld point number determination unit is used to determine the number of target weld points after the corner point in the initial welding path if the target weld point currently determined as the target path point is the corner point and the corner point is the weld intersection point.

[0158] The condition judgment unit is used to determine whether to re-select the current target path point as the target weld point based on the number of target weld points, so as to continue to determine the target weld points as the target path points sequentially based on the nearest neighbor pathfinding algorithm.

[0159] In some specific embodiments, the target steel box girder welding module 14 may include:

[0160] The first target vector determination unit is used to determine a first target vector based on the current target weld point and the next target weld point of the current target weld point;

[0161] The second target vector determination unit is used to perform a cross product between the normal vector corresponding to the current target weld point and the first target vector to obtain a first cross product result, and determine a second target vector based on the first cross product result;

[0162] The third target vector determination unit is used to perform a cross product on the first target vector and the second target vector to obtain a second cross product result, and to determine the third target vector based on the second cross product result;

[0163] The target welding torch pose determination unit is used to determine a target rotation matrix based on the first target vector, the second target vector and the third target vector, and to determine the target welding torch pose corresponding to the current target weld point based on the target rotation matrix.

[0164] Furthermore, embodiments of this application also disclose an electronic device, Figure 9This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the automated welding method for steel box girders disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0165] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0166] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0167] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the automated steel box girder welding method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0168] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned automated welding method for steel box girders. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0169] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0170] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0171] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0172] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0173] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An automated welding method for steel box girders, characterized in that, include: The target reinforcement learning model determines the target's multi-view photography pose, and the target structured light camera acquires the target's local point cloud information corresponding to the target steel box girder based on the target's multi-view photography pose. The target's overall point cloud information is determined based on the target's local point cloud information and a preset point cloud stitching algorithm. The target reinforcement learning model is a model built based on Markov decision process. The point cloud of each weld point of the target steel box girder is determined from the overall point cloud information of the target, and an initial weld point set is generated based on each weld point point cloud. The point cloud of each weld point in the initial weld point set is transformed to the target world coordinate system to obtain each target weld point, and a target weld point set is generated based on each target weld point, and a corresponding target welding path is generated based on the target weld point set. Determine the target welding torch pose corresponding to each target weld point in the target welding path, and use the target robotic arm to control the target welding torch to weld the target steel box girder based on the target welding path and the target welding torch pose; Before determining the target's multi-view photography pose based on the target reinforcement learning model, the process also includes: The target ground is captured by the target structured light camera based on a preset shooting angle to generate ground point cloud information corresponding to the target ground. The ground point cloud information is fitted using a random sampling consensus algorithm to obtain the plane equation corresponding to the target ground. The target structured light camera is used to capture images of the target steel box girder from the preset shooting angle to generate the first point cloud information corresponding to the target steel box girder; Determine the distance between each point in the first point cloud information and the plane equation, and determine the target distance greater than a preset distance threshold from each of the distances; Based on the points corresponding to the target distance, an initial point cloud information corresponding to the target steel box girder is constructed, so as to determine the overall point cloud information of the target steel box girder based on the initial point cloud information; the initial point cloud information is the local point cloud information of the target steel box girder. The process of acquiring local point cloud information of the target steel box girder using a target structured light camera based on the target's multi-view photographic pose, and determining the overall point cloud information of the target based on the local point cloud information and a preset point cloud stitching algorithm, includes: Based on the target multi-view photography pose, motion commands corresponding to the target robotic arm are generated, and the target robotic arm is used to control the target structured light camera to photograph the target steel box girder based on the motion commands, so as to obtain the target local point cloud information corresponding to the target multi-view photography pose; Construct a first pyramid corresponding to the initial point cloud information, and construct a second pyramid corresponding to the target local point cloud information; The first pyramid and the second pyramid are registered using the iterative nearest point algorithm, and the points in the local point cloud information of the target are stitched into the initial point cloud information based on the registration result to obtain the overall point cloud information of the target corresponding to the target steel box girder. The step of determining the point cloud of each weld point of the target steel box girder from the overall point cloud information of the target includes: The initial point cloud normal vectors corresponding to each target point cloud in the overall target point cloud information are determined by principal component analysis algorithm, and the initial point cloud normal vectors are uniformized to obtain the target point cloud normal vectors. A first point cloud normal vector dataset is constructed based on the target point cloud normal vectors. A second point cloud normal vector dataset is obtained based on the first point cloud normal vector dataset, and the ratio between the number of the second point cloud normal vector dataset and the number of the first point cloud normal vector dataset is determined. If the ratio is less than the first preset threshold, then the point cloud of each weld point of the target steel box girder is determined based on the second point cloud normal vector dataset; If the ratio is greater than or equal to the first preset threshold, a target normal vector is randomly selected from the second point cloud normal vector dataset, and the second point cloud normal vector dataset is updated based on the target normal vector until the ratio between the number of the updated second point cloud normal vector dataset and the number of the first point cloud normal vector dataset is less than the first preset threshold. Based on the updated second point cloud normal vector dataset, the point clouds of each weld point of the target steel box girder are determined.

2. The automated welding method for steel box girders according to claim 1, characterized in that, The step of transforming the point cloud of each weld point in the initial weld point set to the target world coordinate system to obtain each target weld point includes: The first transformation matrix from the end-effector coordinate system of the target robotic arm to the camera coordinate system corresponding to the target structured light camera is determined based on the eye-on-hand calibration method. Determine the first translation matrix and rotation matrix from the robot arm base coordinate system to the robot arm end coordinate system of the target robot arm, and determine the second transformation matrix based on the first translation matrix and the rotation matrix; Determine the second translation matrix from the target world coordinate system to the robot arm base coordinate system, and determine the third transformation matrix based on the second translation matrix; Based on the first transformation matrix, the second transformation matrix, and the third transformation matrix, the point clouds of each weld point in the initial weld point set are transformed to the target world coordinate system to obtain each target weld point corresponding to each weld point point cloud.

3. The automated welding method for steel box girders according to claim 1, characterized in that, The step of generating the corresponding target welding path based on the target weld point set includes: The corners and endpoints of each target weld point are determined by using a preset corner and endpoint detector, and each target weld point is determined as a target path point by taking any one of the endpoints as the starting point and using the nearest neighbor pathfinding algorithm. An initial welding path is generated based on the target path points, and the target welding path is determined based on the initial welding path after all the target weld points are determined as the target path points. Accordingly, the process of generating an initial welding path based on the target path points, and determining the target welding path based on the initial welding path after all the target weld points have been determined as the target path points, further includes: If the target weld point currently determined as the target path point is the corner point, and the corner point is the weld intersection point, then determine the number of target weld points after the corner point in the initial welding path; Based on the number of target weld points, determine whether to re-designate the current target path point as the target weld point, so as to continue to determine the target weld points as the target path points sequentially based on the nearest neighbor pathfinding algorithm.

4. The automated welding method for steel box girders according to any one of claims 1 to 3, characterized in that, Determining the target welding torch pose corresponding to each target weld point in the target welding path includes: A first target vector is determined based on the current target weld point and the next target weld point. The first cross product result is obtained by performing a cross product on the normal vector corresponding to the current target weld point and the first target vector, and the second target vector is determined based on the first cross product result. The first target vector and the second target vector are cross-producted to obtain a second cross-product result, and a third target vector is determined based on the second cross-product result; The target rotation matrix is ​​determined based on the first target vector, the second target vector, and the third target vector, and the target welding torch pose corresponding to the current target weld point is determined based on the target rotation matrix.

5. An electronic device, characterized in that, The electronic device includes a processor and a memory; wherein the memory is used to store a computer program, which is loaded and executed by the processor to implement the automated welding method for steel box girders as described in any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that, Used to store a computer program, which, when executed by a processor, implements the automated welding method for steel box girders as described in any one of claims 1 to 4.

Images