A method and system for profile extraction and model alignment based on line laser scanning

Abstract

The present application relates to the technical field of intelligent manufacturing, and provides a contour extraction and model alignment method and system based on line laser scanning. The method comprises the following steps: determining a registration area of a steel structure to be processed based on a steel structure model; preprocessing initial point cloud data of the steel structure to be processed to generate target point cloud data; determining target structure features of the registration area based on the target point cloud data corresponding to the registration area; determining reference structure features corresponding to the registration area based on the steel structure model; aligning the reference structure features with the target structure features to determine target registration parameters corresponding to the registration area; registering original welding paths corresponding to the registration area based on the target registration parameters to generate sub-welding paths corresponding to the registration area; determining target welding paths of the steel structure to be processed based on the sub-welding paths; and controlling a robot to perform welding on the steel structure to be processed based on the target welding paths.

Description

Technical Field

[0001] This invention relates to the field of intelligent manufacturing technology, and in particular to a method and system for contour extraction and model alignment based on line laser scanning. Background Technology

[0002] In the engineering and construction industry, steel structure systems offer advantages such as high mechanization, short construction cycles, and stable quality. Steel structure production lines process items characterized by single-piece production, small batches, and high flexibility. Currently, the overall level of automation and intelligence in steel structure production lines is relatively low, making it difficult to meet the flexible production requirements of different products. Furthermore, the labor input is high, and efficiency needs improvement. Welding is one of the core processes in steel structure production; therefore, improving the automation and intelligence level of welding has significant value in enhancing the efficiency of steel structure production.

[0003] Therefore, it is necessary to provide a contour extraction and model alignment method and system based on line laser scanning to improve welding accuracy and production efficiency. Summary of the Invention

[0004] The invention includes a contour extraction and model alignment method based on line laser scanning. The method includes: determining the registration region of the steel structure to be processed based on a steel structure model; preprocessing the initial point cloud data of the steel structure to generate target point cloud data; determining the target structural features of the registration region based on the target point cloud data corresponding to the registration region; determining the reference structural features corresponding to the registration region based on the steel structure model; aligning the reference structural features with the target structural features to determine the target registration parameters corresponding to the registration region; registering the original welding path corresponding to the registration region based on the target registration parameters to generate a sub-welding path corresponding to the registration region; determining the target welding path of the steel structure to be processed based on the sub-welding path; and controlling a robot to perform welding on the steel structure to be processed based on the target welding path.

[0005] The invention includes a contour extraction and model alignment system based on line laser scanning. The system comprises: an acquisition module configured to acquire initial point cloud data of a steel structure to be processed; a preprocessing module configured to preprocess the initial point cloud data to generate target point cloud data; and a control module configured to: determine the registration region of the steel structure to be processed based on a steel structure model; determine the target structural features of the registration region based on the target point cloud data corresponding to the registration region; determine the reference structural features corresponding to the registration region based on the steel structure model; align the reference structural features with the target structural features to determine the target registration parameters corresponding to the registration region; register the original welding path corresponding to the registration region based on the target registration parameters to generate a sub-welding path corresponding to the registration region; determine the target welding path of the steel structure to be processed based on the sub-welding path; and control a robot to perform welding on the steel structure to be processed based on the target welding path.

[0006] The invention includes a contour extraction and model alignment device based on line laser scanning. The device includes at least one processor and at least one memory; wherein the at least one memory is used to store computer instructions; and the at least one processor is used to execute at least a portion of the computer instructions to implement the aforementioned method.

[0007] The invention includes a computer-readable storage medium. This storage medium stores computer instructions, and when a computer reads the computer instructions from the storage medium, the computer executes the aforementioned contour extraction and model alignment method based on line laser scanning.

[0008] The beneficial effects of the above invention include, but are not limited to: (1) By accurately determining the registration area, extracting and aligning the target structural features and the reference structural features, reliable target registration parameters can be obtained efficiently. Then, the original welding path can be registered based on the target registration parameters to generate a target welding path that is suitable for the steel structure to be processed, which can improve the welding efficiency and accuracy of the robot; (2) It can automatically adapt to steel structures of different specifications, effectively compensate for workpiece errors, and has higher stability. It is less affected by rust, oil stains or changes in light on the workpiece surface, which can significantly improve the reliability and efficiency of automated welding; (3) By introducing overlapping areas for contour feature point registration and determining the comprehensive registration error based on the registration error, the quantitative evaluation of the pose continuity between adjacent registration areas is realized. Attached Figure Description

[0009] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein:

[0010] Figure 1 This is an exemplary schematic diagram of a contour extraction and model alignment system based on line laser scanning, as shown in some embodiments of this specification.

[0011] Figure 2 This is an exemplary flowchart of a contour extraction and model alignment method based on line laser scanning, as shown in some embodiments of this specification.

[0012] Figure 3 This is an exemplary flowchart illustrating the updating of target registration parameters according to some embodiments of this specification;

[0013] Figure 4 This is a schematic diagram illustrating the determination of predicted structural features according to some embodiments of this specification. Detailed Implementation

[0014] To more clearly illustrate the technical solutions of the embodiments in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some examples or embodiments of this specification. For those skilled in the art, these drawings can be applied to other similar scenarios without creative effort. Unless obvious from the context or otherwise specified, the same reference numerals in the drawings represent the same structures or operations.

[0015] It should be understood that the terms “system,” “device,” “unit,” and / or “module” used herein are one way to distinguish different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0016] As indicated in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0017] Figure 1 This is an exemplary schematic diagram of a contour extraction and model alignment system based on line laser scanning, according to some embodiments of this specification. In some embodiments, the contour extraction and model alignment system 100 based on line laser scanning may include an acquisition module 110, a preprocessing module 120, and a control module 130.

[0018] The acquisition module 110 is configured to acquire the initial point cloud data of the steel structure to be processed.

[0019] The preprocessing module 120 is configured to preprocess the initial point cloud data to generate the target point cloud data.

[0020] The control module 130 is configured to: determine the registration region of the steel structure to be processed based on the steel structure model; determine the target structural features of the registration region based on the target point cloud data corresponding to the registration region; determine the reference structural features corresponding to the registration region based on the steel structure model; align the reference structural features with the target structural features to determine the target registration parameters corresponding to the registration region; register the original welding path corresponding to the registration region based on the target registration parameters to generate the sub-welding path corresponding to the registration region; determine the target welding path of the steel structure to be processed based on the sub-welding path; and control the robot to perform welding on the steel structure to be processed based on the target welding path.

[0021] In some embodiments, the line laser scanning-based contour extraction and model alignment system 100 may include a processor and a storage device. The processor may be used to process data from at least one component of the line laser scanning-based contour extraction and model alignment system 100 or from an external data source.

[0022] In some embodiments, the processor may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), a dedicated instruction set processor (ASIP), a graphics processing unit (GPU), a physical processing unit (PPU), or any combination thereof. The preprocessing module 120 and the control module 130 may be integrated into the processor.

[0023] The storage device can be used to store data, instructions, and / or any other information. In some embodiments, the storage device can store data and / or information acquired from at least one component of the line laser scanning-based contour extraction and model alignment system 100 or an external data source. In some embodiments, the storage device may include a mass storage device, a removable memory, or any combination thereof. The storage device may be integrated into a processor.

[0024] Figure 2 This is an exemplary flowchart of a contour extraction and model alignment method based on line laser scanning, as shown in some embodiments of this specification. In some embodiments, process 200 may be executed by a processor. Figure 2 As shown, process 200 may include the following steps:

[0025] Step 210: Based on the steel structure model, determine the registration area of ​​the steel structure to be processed.

[0026] Steel structures to be treated refer to physical steel structures that require welding. For example, steel structures to be treated include sub-areas (such as webs, flanges, or connection nodes) within complete steel beams, columns, or other steel structures that require localized treatment.

[0027] The region to be registered refers to the basic unit used for registration in the target point cloud data representing the steel structure to be processed. For example, when the structure of the steel structure to be processed is simple, the region to be registered can be the entire steel structure (i.e., the entire target point cloud data). Alternatively, when the structure of the steel structure to be processed is complex, the steel structure can be divided into multiple sub-regions (i.e., the target point cloud data can be divided into multiple sub-regions), and each sub-region is considered a region to be registered. For an explanation of the target point cloud data, see step 220 and its related description.

[0028] A steel structure model refers to the standard structural model corresponding to the steel structure to be processed. The standard structural model can be a pre-generated theoretical 3D model, which can be used as a benchmark for comparison or registration. For example, the steel structure model can be at least one of BIM (Building Information Model) model, CAD (Computer-Aided Design) model, etc.

[0029] In some embodiments, technicians can pre-generate a steel structure model using modeling software such as CAD, and determine the set of coordinate points corresponding to the original welding path on the steel structure model based on the model coordinate system. The origin of the model coordinate system can be the geometric center of the steel structure model, or a specific reference point (such as a specific corner point on the bottom surface of the steel structure model).

[0030] The original welding path refers to the welding path generated during the generation of the steel structure model according to the welding requirements. The original welding path on the steel structure model is represented by a set of coordinate points.

[0031] In some embodiments, the steel structure model pre-stores partitioning parameters for dividing the steel structure to be processed. The processor can call the partitioning parameters in the steel structure model to partition the target point cloud data representing the steel structure to be processed, obtaining multiple regions to be registered.

[0032] In some embodiments, the processor can determine the structural complexity of the steel structure to be processed based on the steel structure model; in response to the structural complexity satisfying the first partitioning condition, the steel structure to be processed is divided into multiple registration regions.

[0033] Structural complexity refers to a comprehensive index that measures the complexity of the geometry, topology, and component connections of the steel structure to be processed.

[0034] The higher the structural complexity, the more complex the steel structure to be processed is in terms of geometric details, connection nodes or overall configuration, which may lead to greater difficulty in subsequent processing (such as registration and welding path planning).

[0035] In some embodiments, the processor can collect multiple indicators from the steel structure model that can measure structural complexity, normalize these indicators, and perform a comprehensive calculation to obtain the structural complexity. For example, the processor can map each of the multiple indicators to the interval [0, 1], determine the corresponding value for each indicator through a preset correspondence, and then add the corresponding values ​​for each indicator to obtain the structural complexity of the steel structure to be processed. The preset correspondence can be preset by technical personnel. Alternatively, the processor can obtain the structural complexity of the steel structure to be processed by weighted summation based on a pre-set weight for each indicator.

[0036] In some embodiments, the metrics include at least one of contour smoothness, number of abrupt changes, average curvature, number of intersecting nodes, and number of component connections. The processor can determine the metrics for measuring structural complexity in a variety of ways.

[0037] Contour smoothness refers to an index characterizing the continuity of the contour curve change on the surface of the steel structure to be processed. The processor can determine the contour smoothness by calculating the ratio of the total length of the original welding path to the Euclidean distance between the first and last points of the original welding path.

[0038] Mean curvature is an indicator that quantifies the average degree of bending of the steel structure to be processed. The processor can extract curves representing the welding trajectory based on the geometric data of the steel structure model and the original welding path, and calculate the rate of change of the tangent vector at each discrete point on the curve. The mean of the rate of change of the tangent vector at all discrete points is taken as the mean curvature.

[0039] The number of mutation points refers to the total number of points in the original welding path where the path direction changes significantly. The processor can calculate the angle between adjacent path segments in the original welding path and count the number of points with an angle value not less than a preset angle threshold to obtain the number of mutation points. The preset angle threshold can be preset by technicians.

[0040] The number of intersecting nodes refers to the number of nodes in the steel structure to be processed that are connected to multiple edges. The processor can perform skeleton extraction on the steel structure model, generate a structural skeleton diagram of the steel structure model, and traverse the nodes in the structural skeleton diagram to count the number of nodes with more than 2 connected edges, thus obtaining the number of intersecting nodes.

[0041] The component connection count refers to the number of pairs of components (usually two components) that are physically connected in the steel structure being processed. The processor can obtain the component connection count by parsing the assembly topology of the steel structure model and counting the number of physically connected component pairs.

[0042] The first zoning condition refers to the criteria used to determine whether the steel structure to be treated needs to be divided into zones based on its structural complexity. For example, the first zoning condition may include a structural complexity not less than a first threshold. The first threshold can be preset based on historical experience.

[0043] In some embodiments, in response to the structural complexity satisfying the first partitioning condition, the processor divides the steel structure to be processed into multiple registration regions by the above-described method of determining multiple registration regions.

[0044] In some embodiments of this specification, the need to divide the steel structure to be processed into multiple registration regions is dynamically determined based on the structural complexity. For highly complex steel structures to be processed, this method can avoid the computational burden and error accumulation of global registration, allowing subsequent registration tasks to focus on smaller, easier-to-process local areas, thereby improving the registration success rate and speed.

[0045] In some embodiments, the processor can also determine the welding complexity of the steel structure to be processed based on the original welding path corresponding to the steel structure to be processed; in response to the structural complexity satisfying the first partitioning condition or the welding complexity satisfying the second partitioning condition, the steel structure to be processed is divided into multiple registration regions.

[0046] Welding complexity refers to an indicator that measures the complexity of the welding process of the steel structure to be treated.

[0047] In some embodiments, welding complexity can be determined in a variety of ways. For example, the processor can comprehensively evaluate multiple welding features (such as length, curvature, number of welds, etc.) of the original welding path and perform a weighted calculation based on the preset weight of each welding feature to obtain the welding complexity.

[0048] The second zoning condition refers to the criteria used to determine whether the welding complexity meets the requirements for dividing the steel structure to be treated into zones. The second zoning condition may include a welding complexity not less than a second threshold. The determination of the second threshold can be similar to the determination of the first threshold.

[0049] In some embodiments, in response to the structural complexity satisfying the first partitioning condition or the welding complexity satisfying the second partitioning condition, the processor can divide the steel structure to be processed into multiple registration regions by the above-described method of determining multiple registration regions.

[0050] In some embodiments, the processor may also perform a weighted summation of structural complexity and welding complexity. If the weighted summation value is not less than a third threshold, the processor divides the steel structure to be processed into multiple sub-regions to be processed. The method for determining the third threshold can be similar to the method for determining the first threshold.

[0051] In some embodiments of this specification, the steel structure to be processed is reasonably divided into multiple registration regions based on the welding complexity or structural complexity of the steel structure to be processed. This improves the sensitivity of identifying steel structures with complex structures and / or difficult welding processes, and avoids the risk of missed identification.

[0052] Step 220: Preprocess the initial point cloud data of the steel structure to be processed to generate target point cloud data.

[0053] Initial point cloud data refers to the set of point cloud data used to represent the geometry of the surface of the steel structure to be processed. In some embodiments, the initial point cloud data includes three-dimensional coordinate information of multiple spatial points representing the geometry of the surface of the steel structure to be processed. The initial point cloud data can be acquired by a 3D scanning device (such as a 3D scanner) and sent to the processor by the 3D scanning device.

[0054] In some embodiments, the processor can also control a robot equipped with a laser vision sensor to move continuously along a preset fixed path and scan the steel structure to be processed using a line scanning method to obtain initial point cloud data.

[0055] In some embodiments, when the surface of the steel structure to be processed has uneven reflectivity due to oxide scale, rust, and metallic luster, the robot can use high dynamic range multi-exposure fusion technology to obtain initial point cloud data.

[0056] Target point cloud data refers to the set of point cloud data obtained after preprocessing the initial point cloud data. Preprocessing includes invalid point removal, noise reduction, downsampling, etc.

[0057] In some embodiments, the scanning speed of the registration region in the steel structure to be processed is negatively correlated with the structural complexity of the registration region.

[0058] Scanning speed refers to the robot's movement speed and scanning frequency when collecting initial point cloud data.

[0059] Understandably, excessively fast scanning speeds by the robot can lead to insufficient resolution of the initial point cloud data along the robot's movement direction, making it easy to miss subtle features. Conversely, excessively slow scanning speeds may result in collecting too much data, causing processing timeouts. Therefore, reducing the scanning speed in areas with complex structures to be registered ensures the acquisition of higher-density initial point cloud data, while increasing the scanning speed in areas with simpler structures improves scanning efficiency.

[0060] In some embodiments of this specification, the scanning speed of the region to be registered is dynamically adjusted according to the structural complexity of the region. This can reduce the scanning speed in structurally complex regions to ensure data accuracy, and increase the scanning speed in structurally simple regions to improve efficiency.

[0061] Step 230: Based on the target point cloud data corresponding to the region to be registered, determine the target structural features of the region to be registered.

[0062] Target structural features refer to the set of features extracted from target point cloud data that characterize the geometric shape of the steel structure to be processed. In some embodiments, target structural features may include, but are not limited to, point features (e.g., corner points, weld endpoints), line features (e.g., edge lines), surface features, or contour features. Contour features may include the cross-sectional contour lines of the surface of the steel structure to be processed or edge boundary lines fitted by the target point cloud data.

[0063] In some embodiments, the processor can determine the target structural features in various ways. For example, the processor can determine the target structural features using an edge extraction method based on curvature and normal vectors. Specifically, the processor calculates the normal vector and surface curvature of each point in the target point cloud data, designating points with surface curvature less than a curvature threshold or whose normal vector is less than an angle threshold as planar points, and designating points with surface curvature not less than a curvature threshold or whose normal vector is not less than an angle threshold as corner points or contour points. The curvature threshold and angle threshold can be preset by a technician.

[0064] For example, the processor can also use a contour extraction method based on slice projection to cut slices along a predetermined axis (such as the direction of weld extension), perform two-dimensional contour fitting on the points in the slices (such as fitting them into straight line segments), and extract the endpoints of all fitted line segments to form contour features.

[0065] Step 240: Based on the steel structure model, determine the reference structural features corresponding to the region to be registered.

[0066] Reference structural features refer to the set of features that characterize the geometry of a steel structure model. In some embodiments, the content of the reference structural features is similar to that of the target structural features. For example, both the target structural features and the reference structural features consist of discrete coordinate points in three-dimensional space, both characterize the geometry of the region to be registered, and both possess geometric topological consistency.

[0067] In some embodiments, the processor can determine the reference structural features by acquiring geometric boundary line information (such as identifying the geometric edges and vertices of the steel structure model) and uniformly discretizing the geometric boundary lines in the geometric boundary line information. In some embodiments, the sampling interval for the processor to uniformly discretize the geometric boundary lines is the same as the average point spacing of the target point cloud data.

[0068] Step 250: Align the reference structural features with the target structural features to determine the target registration parameters corresponding to the region to be registered.

[0069] Target registration parameters refer to mathematical quantities used to perform spatial coordinate transformations between reference and target structural features. In some embodiments, target registration parameters may include rotation matrices and translation vectors.

[0070] Aligning reference and target structural features involves transforming the reference structural features in spatial coordinates to make them coincide in space. In some embodiments, the processor can determine the target registration parameters using algorithms such as ICP (Iterative Closest Point) or singular value decomposition. For example, the processor acquires the target and reference structural features, sets an initial pose for the reference structural features (including the identity matrix and translation zero vector), and determines the target registration parameters through iterative updates.

[0071] For example, the iterative update includes: for each point in the reference structural features, the processor searches for the point closest to that point in the target structural features, establishes a point-to-point correspondence, and based on multiple point-to-point correspondences, constructs a first error function and solves for the registration parameters (i.e., rotation matrix, translation vector, etc.) that minimize the output of the first error function, until the convergence condition is met. The final output rotation matrix and translation vector are the target registration parameters. The convergence condition includes the error change between two adjacent iterations being less than a threshold, the number of iterations reaching an upper limit, etc. The first error function can be the sum of squared Euclidean distances between multiple point pairs.

[0072] In some embodiments, multiple overlapping regions exist among multiple adjacent regions to be registered along the original welding path corresponding to the steel structure to be processed in multiple regions to be registered. The processor can, for each overlapping region,: select multiple contour feature points from the model region of the steel structure model corresponding to the overlapping region; register the multiple contour feature points based on the target registration parameters to generate a first feature point set and a second feature point set corresponding to the overlapping region; determine the registration error between the first feature point set and the second feature point set; determine the comprehensive registration error based on the registration error corresponding to the multiple overlapping regions; and, in response to the comprehensive registration error satisfying the iterative update condition, iteratively update the target registration parameters until the objective function satisfies the convergence condition, generating a second registration parameter corresponding to the multiple regions to be registered.

[0073] The overlapping region refers to the area shared by two adjacent registration regions on the original welding path corresponding to the steel structure to be processed. The model region refers to the part in the steel structure model corresponding to the overlapping region. The contour feature point refers to the point on the contour within the model region.

[0074] In some embodiments, the processor can acquire contour feature points in various ways. For example, the processor can select feature points as contour feature points based on geometric characteristics, such as the edges, corners, or locations with significant curvature changes within the model region.

[0075] The first feature point set refers to the set of points obtained by registering multiple contour feature points according to the target registration parameters of the first region to be registered in the adjacent regions to be registered. The second feature point set refers to the set of points obtained by registering multiple contour feature points according to the target registration parameters of the second region to be registered in the adjacent regions to be registered.

[0076] In some embodiments, for each contour feature point within the overlapping region, the processor registers the contour feature point according to the target registration parameters, which includes: performing coordinate transformation (such as rotation and translation) on the coordinate information of the contour feature point according to the rotation matrix and translation vector in the target registration parameters, so as to obtain the registered contour feature point as a point in the first feature point set or the second feature point set.

[0077] Registration error refers to the error between corresponding point pairs in the first feature point set and the second feature point set.

[0078] In some embodiments, for each overlapping region, the processor can calculate the Euclidean distance between each pair of corresponding points within the overlapping region based on the first feature point set and the second feature point set, and calculate the sum of squares of the Euclidean distances of all corresponding point pairs to obtain the registration error corresponding to the overlapping region.

[0079] The overall registration error refers to the index that characterizes the pose continuity between multiple overlapping regions.

[0080] In some embodiments, the processor can perform a weighted summation of the registration errors corresponding to multiple overlapping regions to obtain a comprehensive registration error. The weights of the overlapping regions can be preset; for example, the larger the area of ​​the overlapping region, the higher its weight.

[0081] Iterative update conditions can be used to determine whether iterative updates to the target registration parameters are needed based on the overall registration error. These conditions may include ensuring the overall registration error is not less than an error threshold. This error threshold is preset based on historical experience.

[0082] In some embodiments, the processor iteratively updates the target registration parameters by constructing a target function, updating the target registration parameters using an iterative update algorithm, and substituting the updated target registration parameters into the target function. If the target function does not meet the convergence condition, the target registration function is updated again. If the target function meets the convergence condition, the updated target registration parameters are used as the second registration parameters corresponding to the multiple regions to be registered (i.e., the new target registration parameters determined according to this method).

[0083] The objective function characterizes the weighted total error between local matching accuracy and pose continuity among multiple overlapping regions. In some embodiments, the objective function can be a sum of a second error function constructed based on a first feature point set and a second feature point set, and a comprehensive registration error with preset weights. The second error function is similar to the first error function. Iterative update algorithms include, but are not limited to, gradient descent, Gauss-Newton algorithm, trust region algorithm, etc.

[0084] In some embodiments of this specification, contour feature point registration is performed by introducing overlapping regions, and a comprehensive registration error is determined based on the registration error, thereby achieving a quantitative evaluation of the pose continuity between adjacent regions to be registered. In response to the comprehensive registration error satisfying the iteration condition, an objective function is used to simultaneously optimize both local matching accuracy and pose continuity between overlapping regions, thus iteratively updating the target registration parameters. This method effectively avoids the "ghosting" problem caused by inconsistent registration parameters at overlapping areas of different registration regions, significantly improving the smoothness and accuracy of the overall welding path.

[0085] Step 260: Based on the target registration parameters, register the original welding path corresponding to the region to be registered, and generate the sub-welding path corresponding to the region to be registered.

[0086] The sub-welding path corresponding to the area to be registered refers to the trajectory information used to guide the welding of the area to be registered.

[0087] In some embodiments, the processor can use the rotation matrix and translation vector in the target registration parameters corresponding to the region to be registered to perform coordinate transformation on the original welding path corresponding to the region to be registered in the steel structure model to obtain a sub-welding path.

[0088] Step 270: Based on the sub-welding path, determine the target welding path for the steel structure to be processed.

[0089] The target welding path refers to the trajectory information used to guide the welding of the steel structure to be treated.

[0090] In some embodiments, the processor can perform a union operation on the sub-welding paths of all regions to be registered (i.e., connect all sub-welding paths and treat the overlapping parts between sub-welding paths as the same path) to obtain the target welding path.

[0091] Step 280: Control the robot to perform welding on the steel structure to be treated based on the target welding path.

[0092] In some embodiments, the processor can convert the target welding path into robot-recognizable instruction codes. For example, the processor converts discrete points and attitude information on the target welding path into instruction codes such as G-code or RAPID instructions, and sends the instruction codes to the robot. The robot then welds the steel structure to be processed along the target welding path according to the instruction codes.

[0093] In some embodiments of this specification, by accurately determining the region to be registered, extracting and aligning the target structural features with the reference structural features, reliable target registration parameters can be efficiently obtained. Then, based on these target registration parameters, the original welding path is registered to generate a target welding path adapted to the steel structure to be processed, thereby improving the robot's welding efficiency and accuracy. This method can automatically adapt to steel structures of different specifications, effectively compensate for workpiece errors, and exhibits higher stability. It is less affected by rust, oil, or changes in lighting on the workpiece surface, significantly improving the reliability and efficiency of automated welding.

[0094] Figure 3 This is an exemplary flowchart illustrating the updating of target registration parameters according to some embodiments of this specification.

[0095] Step 310: Based on the steel structure model, determine the topology diagram of the steel structure to be processed. For details regarding the steel structure model and the steel structure to be processed, please refer to... Figure 2 And its related descriptions.

[0096] A topology diagram is a graphical representation of the connection relationships of a steel structure to be processed. It consists of nodes and edges, describing the spatial skeleton and connection logic of the steel structure. Nodes can include weld joints, structural corner points, etc. Edges can include connecting structures (such as beams, plates, etc.) or weld paths, etc.

[0097] In some embodiments, the processor can parse the model data of the steel structure model, identify multiple connection structures, corner points, weld paths and weld joints in the steel structure model, and construct a topology diagram based on the identification results.

[0098] Step 320: Based on the target welding path, welding parameters, topology diagram, and target registration parameters, the predicted structural features of the region to be registered after welding are determined using a feature prediction model. For an explanation of the target welding path and target registration parameters, please refer to [link to relevant documentation]. Figure 2 And its related descriptions.

[0099] Welding parameters refer to the parameters used to control robotic welding. In some embodiments, welding parameters include welding current, welding voltage, and the speed at which the welding torch moves. Welding parameters can be preset by a technician.

[0100] Predicted structural features refer to the set of features of the predicted geometric shape of the region to be registered after welding.

[0101] Figure 4 This is a schematic diagram illustrating the determination of predicted structural features according to some embodiments of this specification.

[0102] like Figure 4 As shown, the input to the feature prediction model 450 may include a topology diagram 410, a target welding path 420, welding parameters 430, and target registration parameters 440 for the region to be registered. The output of the feature prediction model 450 may include predicted structural features 460 corresponding to the region to be registered.

[0103] In some embodiments, the feature prediction model is a machine learning model. For example, the feature prediction model may include any one or a combination of Convolutional Neural Networks (CNN) models, Deep Neural Networks (DNN) models, or other custom model structures. The actual structural features of the historical steel structure to be processed are extracted from each historical registration region after the actual welding of the historical steel structure. Training samples and labels can be obtained based on historical data.

[0104] In some embodiments, the processor can train an initial feature prediction model using multiple sets of labeled training samples to obtain the feature prediction model. The training samples may include historical welding paths, historical welding parameters, historical topology diagrams, and historical registration parameters corresponding to historical registration regions. The historical topology diagram can be determined based on the steel structure model corresponding to the historical steel structure to be processed. Labels may include actual structural features extracted from the historical registration region after the historical steel structure to be processed has undergone actual welding. The training samples and labels can be obtained based on historical data.

[0105] In some embodiments, the processor can input multiple labeled training samples into an initial feature prediction model, construct a loss function using the labels and the results of the initial feature prediction model, and iteratively update the parameters of the initial feature prediction model based on the loss function using gradient descent or other methods. When preset conditions are met, model training is complete, and a trained feature prediction model is obtained. These preset conditions may include loss function convergence, the number of iterations reaching a threshold, etc.

[0106] In some embodiments, such as Figure 4 As shown, the input to the feature prediction model 450 also includes the current environment parameter 470.

[0107] Current environmental parameters refer to parameters related to the environment in which the steel structure to be treated is located. For example, current environmental parameters may include ambient temperature, wind speed, etc.

[0108] In some embodiments, current environmental parameters can be obtained through corresponding sensors. For example, the processor can monitor and acquire current environmental parameters in real time based on temperature sensors and wind speed sensors deployed in the environment of the steel structure to be treated.

[0109] In some embodiments, before the current environmental parameters are input into the feature prediction model, the processor can perform preprocessing such as normalization, standardization, dimensionality reduction, or feature combination on the current environmental parameters to optimize data quality. If the input to the feature prediction model includes the current environmental parameters, the training samples also include historical environmental parameters. Historical environmental parameters can be obtained from historical data.

[0110] In some embodiments of this specification, by incorporating real-time environmental data (such as temperature and wind speed) into the model, the feature prediction model can more accurately capture the impact of the environment on the predicted structural features, thus significantly improving prediction accuracy.

[0111] Step 330: Based on the predicted structural features and the reference structural features, determine the welding error corresponding to the region to be registered.

[0112] Welding error refers to the deviation in weld position that may occur when welding according to the target welding path.

[0113] In some embodiments, the processor can construct a reference structural feature as a continuous trajectory curve and calculate the minimum normal distance between each discrete point in the predicted structural feature and the trajectory curve, and use the statistical value of the minimum normal distance (such as the mean or maximum value) as the welding error.

[0114] Step 340: Determine the overall welding error based on the welding error.

[0115] Comprehensive welding error refers to an index that characterizes the overall welding error of the steel structure to be treated.

[0116] In some embodiments, the processor may sum or perform a weighted summation of the welding errors of multiple regions to be registered to obtain a comprehensive welding error. The weight of each region to be registered is preset based on experience.

[0117] Step 350: In response to the comprehensive welding error meeting the error condition, the target registration parameters are updated to generate the first registration parameters corresponding to the region to be registered.

[0118] Error conditions refer to the conditions used to determine whether the target registration parameters need to be updated based on the overall welding error. In some embodiments, error conditions may include the overall welding error being not less than a welding error threshold. The welding error threshold may be preset by a technician.

[0119] In some embodiments, the error condition is related to the structural complexity of the steel structure to be processed. See [link to documentation on structural complexity]. Figure 2 And its related descriptions.

[0120] In some embodiments, the welding error threshold can be negatively correlated with the structural complexity of the steel structure to be processed. The processor can determine the welding error threshold by querying the correspondence between structural complexity and the welding error threshold based on the structural complexity. The correspondence between structural complexity and the welding error threshold can be set by technicians based on experience, industry standards, or data analysis; the higher the structural complexity, the lower the welding error threshold.

[0121] In some embodiments of this specification, by adapting the error conditions to the structural complexity of the steel structure to be processed, it is possible to more intelligently and accurately determine whether the target registration parameters need to be updated, thereby improving the welding accuracy of complex steel structures or the welding efficiency of simple steel structures.

[0122] The first registration parameter refers to the updated target registration parameter.

[0123] In some embodiments, the processor determines the first registration parameter corresponding to the region to be registered by vector matching based on the reference structural features, target registration parameters, original welding path and welding parameters corresponding to the region to be registered.

[0124] For example, the processor can construct a matching vector based on the reference structural features corresponding to the region to be registered, the target registration parameters, the original welding path, and the welding parameters. It then matches a reference vector from a vector library that meets the matching conditions, and determines the registration parameters corresponding to the reference vector as the target registration parameters for the region to be registered, i.e., the first registration parameters. Matching conditions include a vector similarity greater than a similarity threshold, and a negative correlation between vector similarity and vector distance (such as Euclidean distance). The similarity threshold is pre-set based on historical experience.

[0125] In some embodiments, the vector library can be constructed based on historical data, including multiple reference vectors and registration parameters corresponding to each reference vector. The processor can construct reference vectors based on the reference structural features corresponding to the historical region to be registered, the target registration parameters, the original welding path, and the welding parameters.

[0126] In some embodiments, in response to the overall welding error satisfying an error condition, the processor can update the second registration parameters corresponding to the region to be registered, generating the first registration parameters. For a description of the second registration parameters, please refer to [link to documentation]. Figure 2 And its related descriptions.

[0127] In some embodiments, the processor may use an iterative update algorithm to update the second registration parameters to obtain the first registration parameters. For a description of the iterative update algorithm, see [link to documentation]. Figure 2 And its related descriptions.

[0128] In some embodiments of this specification, by updating the second registration parameters to generate the first registration parameters, the accuracy of the final target registration parameters can be improved, thereby improving the accuracy of the target welding path and the welding quality.

[0129] In some embodiments, in response to the comprehensive welding error satisfying the error condition, the processor can also perform alignment processing between the reference structural features and the predicted structural features to determine the reference registration parameters corresponding to the region to be registered; and determine the first registration parameters based on the reference registration parameters and the target registration parameters.

[0130] Reference registration parameters are mathematical quantities used to perform spatial coordinate transformations between reference structural features and predicted structural features.

[0131] In some embodiments, the reference registration parameters are similar to the target registration parameters. The process by which the processor aligns the reference structural features with the predicted structural features to determine the reference registration parameters is similar to the process by which the processor aligns the reference structural features with the target structural features to determine the target registration parameters. For details on the process of aligning the reference structural features with the target structural features to determine the target registration parameters, please refer to [link to relevant documentation]. Figure 2 And its related descriptions.

[0132] In some embodiments, the processor can determine the translation vector in the first registration parameter by weighted summation. For example, the processor performs a weighted summation of the translation vector in the target registration parameter and the translation vector in the reference registration parameter to obtain the translation vector in the first registration parameter. The weights of the reference registration parameter and the target registration parameter can be preset based on historical experience.

[0133] In some embodiments, the processor can convert the rotation matrices in the target registration parameters and the rotation matrices in the reference registration parameters into a weighted summable format using methods such as quaternion interpolation or Lie algebra, perform a weighted summation on the rotation matrices in the target registration parameters and the rotation matrices in the reference registration parameters, and finally convert the weighted summation value into matrix form to obtain the rotation matrix in the first registration parameters.

[0134] In some embodiments, in the weighted summation of the reference registration parameters and the target registration parameters, the weight of the reference registration parameters may be positively correlated with the welding error.

[0135] In some embodiments, the processor can determine the weights of the reference registration parameters by querying the correspondence between the weights of the reference registration parameters and the welding errors, based on the welding errors. The correspondence between the weights of the reference registration parameters and the welding errors can be set by technicians based on experience, industry standards, or data analysis. The higher the welding error, the higher the weight of the reference registration parameter.

[0136] In some embodiments of this specification, the weight of the reference registration parameters is correlated with the welding error, enabling a strategy to dynamically adjust and determine the final registration parameters based on potential welding errors in the area to be registered. When the welding error is large, the reference registration parameters receive a higher weight, making the first registration parameters more inclined towards the reference registration parameters, thereby effectively correcting potential welding errors in the area to be registered and ensuring welding quality.

[0137] In some embodiments of this specification, the first registration parameter is determined based on the reference registration parameter and the target registration parameter. This can effectively overcome the limitations of a single data source and take into account possible welding errors in advance, thereby improving the accuracy and robustness of the finally determined registration parameter and thus improving the quality and efficiency of automated welding.

[0138] In some embodiments of this specification, a topological structure diagram is constructed using a steel structure model, and a feature prediction model is used to accurately predict the structural features after welding. Based on the predicted structural features and reference structural features, welding errors are determined, which can accurately quantify potential welding errors, compensate for systematic deviations that are difficult to detect by pure geometric algorithms, and help improve the welding accuracy of the steel structure to be processed.

[0139] Some embodiments of this specification also provide a contour extraction and model alignment apparatus based on line laser scanning. The apparatus includes at least one processor and at least one memory. The at least one memory is used to store computer instructions. The at least one processor is used to execute at least a portion of the computer instructions to implement the methods described in the above embodiments.

[0140] Some embodiments of this specification also provide a computer-readable storage medium. The storage medium stores computer instructions. When a computer reads the computer instructions from the storage medium, the computer executes the methods described in the above embodiments.

[0141] The basic concepts have been described above. Obviously, for those skilled in the art, the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are suggested in this specification and therefore remain within the spirit and scope of the exemplary embodiments described herein.

[0142] It should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments described herein. Other variations may also fall within the scope of this specification. Therefore, alternative configurations of the embodiments described herein are intended to be consistent with the teachings of this specification, rather than as examples or limitations. Accordingly, the embodiments described herein are not limited to those explicitly introduced and described herein.

Claims

1. A contour extraction and model alignment method based on line laser scanning, characterized in that, The method includes: Based on the steel structure model, the registration area of ​​the steel structure to be processed is determined; The initial point cloud data of the steel structure to be processed is preprocessed to generate target point cloud data; Based on the target point cloud data corresponding to the region to be registered, the target structural features of the region to be registered are determined. Based on the steel structure model, the reference structural features corresponding to the region to be registered are determined; Align the reference structural features with the target structural features to determine the target registration parameters corresponding to the region to be registered; Based on the target registration parameters, the original welding path corresponding to the region to be registered is registered to generate a sub-welding path corresponding to the region to be registered. Based on the sub-welding path, the target welding path of the steel structure to be processed is determined; The robot is controlled to perform welding on the steel structure to be processed based on the target welding path; The method further includes: Based on the steel structure model, the topological structure diagram of the steel structure to be processed is determined; Based on the target welding path, welding parameters, topology diagram, and target registration parameters, the predicted structural features of the region to be registered after welding are determined by a feature prediction model, wherein the feature prediction model is a machine learning model. Based on the predicted structural features and the reference structural features, the welding error corresponding to the region to be registered is determined; Based on the aforementioned welding errors, the overall welding error is determined; In response to the comprehensive welding error satisfying the error condition, the target registration parameters are updated to generate the first registration parameters corresponding to the region to be registered.

2. The method according to claim 1, characterized in that, The steel structure model is the standard structural model corresponding to the steel structure to be processed. The determination of the registration region of the steel structure to be processed based on the steel structure model includes: Based on the steel structure model, the structural complexity of the steel structure to be processed is determined; In response to the structural complexity satisfying the first partitioning condition, the steel structure to be processed is divided into multiple registration regions.

3. The method according to claim 1, characterized in that, The step of updating the target registration parameters and generating the first registration parameters corresponding to the region to be registered in response to the comprehensive welding error satisfying the error condition includes: Align the reference structural features with the predicted structural features to determine the reference registration parameters corresponding to the region to be registered; The first registration parameter is determined based on the reference registration parameter and the target registration parameter.

4. A contour extraction and model alignment system based on line laser scanning, characterized in that, The system includes: The acquisition module is configured to collect the initial point cloud data of the steel structure to be processed. The preprocessing module is configured to preprocess the initial point cloud data to generate target point cloud data; The control module is configured as follows: Based on the steel structure model, the registration area of ​​the steel structure to be processed is determined; Based on the target point cloud data corresponding to the region to be registered, the target structural features of the region to be registered are determined. Based on the steel structure model, the reference structural features corresponding to the region to be registered are determined; Align the reference structural features with the target structural features to determine the target registration parameters corresponding to the region to be registered; Based on the target registration parameters, the original welding path corresponding to the region to be registered is registered to generate a sub-welding path corresponding to the region to be registered. Based on the sub-welding path, the target welding path of the steel structure to be processed is determined; The robot is controlled to perform welding on the steel structure to be processed based on the target welding path; The control module is further configured to: Based on the steel structure model, the topological structure diagram of the steel structure to be processed is determined; Based on the target welding path, welding parameters, topology diagram, and target registration parameters, the predicted structural features of the region to be registered after welding are determined by a feature prediction model, wherein the feature prediction model is a machine learning model. Based on the predicted structural features and the reference structural features, the welding error corresponding to the region to be registered is determined; Based on the aforementioned welding errors, the overall welding error is determined; In response to the comprehensive welding error satisfying the error condition, the target registration parameters are updated to generate the first registration parameters corresponding to the region to be registered.

5. The system according to claim 4, characterized in that, The steel structure model is a standard structural model corresponding to the steel structure to be processed, and the control module is further configured as follows: Based on the steel structure model, the structural complexity of the steel structure to be processed is determined; In response to the structural complexity satisfying the first partitioning condition, the steel structure to be processed is divided into multiple registration regions.

6. The system according to claim 4, characterized in that, The control module is further configured to: Align the reference structural features with the predicted structural features to determine the reference registration parameters corresponding to the region to be registered; The first registration parameter is determined based on the reference registration parameter and the target registration parameter.

7. A contour extraction and model alignment device based on line laser scanning, characterized in that, The device includes at least one processor and at least one memory; wherein... The at least one memory is used to store computer instructions; The at least one processor is configured to execute at least a portion of the computer instructions to implement the method as described in any one of claims 1-3.

8. A computer-readable storage medium, characterized in that, The storage medium stores computer instructions. When the computer reads the computer instructions in the storage medium, the computer executes the method as described in any one of claims 1-3.

Images