Ship sub-assembly weld seam identification method and system, device, and storage medium

Abstract

A ship sub-assembly weld seam identification method, comprising: creating a point cloud weld seam identification training dataset by means of point cloud discretization, virtual sampling, weld-zone semi-automatic interactive annotation, and style transfer-based augmentation of ship sub-assembly CAD data; training a point cloud weld seam identification model, and exporting a weld seam identification model data file; using a 3D point cloud camera array as a sampling camera to collect workpiece 3D point cloud data of a sub-assembly workpiece to be welded at the production site; by means of the weld seam identification model data file, obtaining weld seam annotation position data and positioning a weld seam; and issuing a welding process plan and instruction to control a welding robotic arm and a control device therefor to complete sub-assembly workpiece welding. Also provided are a ship sub-assembly weld seam identification system, a computer device, and a computer-readable storage medium. The method overcomes the shortcomings in conventional 3D weld seam identification methods such as point cloud incompleteness and identification failures caused by steel types and the imaging angles of 3D cameras, thereby greatly improving the adaptability of automatic weld seam identification methods to complex workpieces.

Description

Ship small assembly weld seam identification method, system, device and storage medium TECHNICAL FIELD

[0001] The present application relates to the technical field of ship automatic welding, in particular to a ship small assembly weld seam identification method, system, device and storage medium. BACKGROUND

[0002] Welding is a key supporting technology and process link of modern shipbuilding, and small assembly is the smallest unit of ship body composition. The welding workload of small assembly accounts for 30%-40% of the cost of shipbuilding. The traditional shipbuilding process adopts manual welding, which has safety hazards and low efficiency. With the progress of automation and robot technology, domestic and foreign shipyards have gradually researched and adopted automatic robot welding instead of manual welding.

[0003] With the upgrading of ship configuration and design, the small assembly component composition of the ship has expanded from a simple panel form to a multi-grid and multi-rib plate configuration composed of irregular sections such as T-shaped steel and ball flat steel. The traditional weld seam identification method based on online tracking or 3D cameras has problems such as data acquisition noise caused by rib plates or pre-welding points on new small assembly configurations, incomplete data acquisition caused by limited photographing angles, and is only suitable for small assembly workpiece identification in regular shape and camera field of view without obstruction. On the other hand, although the offline teaching programming automatic welding method based on the design model has comprehensive data and accurate position, it needs manual editing and adjustment of the weld seam information, which indirectly increases the workload. Therefore, the existing online tracking and offline teaching automatic welding methods cannot meet the requirements of automatic welding of complex configuration small assembly components. SUMMARY

[0004] The purpose of the present application is to provide a ship small assembly weld seam identification method, system, device and storage medium to solve the technical problems of missing and occlusion of original data acquisition in weld seam identification of complex configuration ship small assembly workpieces such as T-shaped steel, ball flat steel, multi-rib plate and multi-grid, and template matching failure.

[0005] To achieve the above-mentioned purpose, the technical solutions of the present application are as follows:

[0006] The present application provides a ship small assembly weld seam identification method, comprising the following steps:

[0007] Discretizing, virtually sampling, semi-automatically interactive labeling of weld seam area, and style migration and expansion of ship small assembly CAD data to establish a point cloud weld seam identification training data set;

[0008] Data preprocessing and training model construction are performed using the labeled point cloud data to obtain a point cloud weld seam identification model;

[0009] Iterative training and parameter adjustment are performed on the point cloud weld identification model, and whether the average intersection-over-union ratio and the average accuracy meet the preset target is verified. If the preset target is met, the structure and parameters of the point cloud weld identification model are solidified, and the identification model data file is exported;

[0010] A 3D point cloud data acquisition and transmission is performed on the ship small assembly workpiece to be welded in the production site using a 3D point cloud camera array;

[0011] The acquired 3D point cloud data of the ship small assembly workpiece is preprocessed, and weld mark position data is obtained and the weld is positioned in combination with the identification model data file.

[0012] According to the weld mark position data, a welding mechanical arm end trajectory is planned, a welding process planning is issued, and a welding mechanical arm and its control equipment are controlled to complete the welding of the small assembly workpiece.

[0013] Further, the ship small assembly CAD data is established by virtual sampling to establish a point cloud weld identification training data set, which includes:

[0014] A virtual 3D sampling camera is established according to the focal length, horizontal resolution, vertical resolution, optical center position parameters and perspective projection equation of the virtual 3D sampling camera;

[0015] After the virtual 3D sampling camera is established, the center of the three-dimensional bounding box of the ship small assembly CAD model is used as the origin, and the Fibonacci spherical sampling algorithm is used to generate uniformly distributed sampling position points on the half spherical surface;

[0016] The virtual 3D sampling camera is made to traverse the sampling position points, and at each sampling position point, the optical axis direction of the virtual 3D sampling camera is aligned with the line connecting the sampling position point and the workpiece center point, and a two-dimensional virtual image of the workpiece three-dimensional model is collected;

[0017] For the collected two-dimensional virtual sampling weld image, the point cloud weld automatic labeling function uses the Canny edge detection operator to identify the edge pixel points of the sampling image, filter the background edges, perform Hough line fitting, and obtain the 2D pixel position of the candidate weld intersection line in the two-dimensional virtual sampling weld image;

[0018] According to the perspective projection transformation relationship, the three-dimensional position points of the workpiece model corresponding to the candidate weld intersection line 2D pixel points extracted in the multi-view image are calculated as candidate marked weld points, and the non-weld intersection points are filtered according to the constraint of the angle between the two planes of the marked weld, to obtain the candidate marked weld points;

[0019] After coloring the candidate marking weld points, region clustering is performed using a clustering algorithm to obtain marking weld rectangular bounding box coordinates and regions, and a weld rectangular bounding box representing the weld region is generated, which is overlapped with the sub-assembly workpiece CAD model in the interface viewport, and the user interacts to select and identify the incorrect weld region, and finally the correct marking weld region bounding box is retained;

[0020] The three-dimensional sub-assembly workpiece CAD model is discretized into point cloud data, and the point cloud data in the marking weld region bounding box is marked as a target feature point, and the marking point cloud data is exported and saved.

[0021] Further, after the virtual 3D sampling camera is established, the center of the sub-assembly CAD model three-dimensional bounding box is used as the origin, and a Fibonacci spherical sampling algorithm is used to generate uniformly distributed sampling position points on a hemisphere.

[0022] The calculation formula of the Fibonacci spherical sampling algorithm is set as follows:

[0023] Where, θ is the azimuth angle of the sampling point, i is the index of the sampling point, and φ is the golden ratio.

[0024] Further, the candidate marking weld points corresponding to the workpiece model three-dimensional position points extracted from the multi-view image according to the perspective projection transformation relationship are calculated.

[0025] The calculation formula of the perspective projection transformation relationship is set as follows:

[0026] Where, (X, Y, Z) is the 3D point cloud position point, (x, y) is the 2D pixel position, and f is the focal length of the virtual 3D sampling camera.

[0027] Further, the marking point cloud data collected by the virtual 3D sampling camera is preprocessed, and the weld marking position data is obtained and positioned in combination with the recognition model data file.

[0028] The collected marking point cloud data is segmented by the weld body bounding box and filtered by the abnormal value to realize preprocessing;

[0029] The trained point cloud weld recognition model is used to identify the preprocessed marking point cloud data and the recognition model data file, obtain the weld marking position data, and position and match the recognition point three-dimensional position coordinates with the welding robot coordinate system according to the welding robot position relationship.

[0030] Further, the marking point cloud data is preprocessed and the model is trained to obtain the point cloud weld recognition model.

[0031] noise filtering on the marked point cloud data;

[0032] performing point cloud three-dimensional Hilbert encoding serialization on the filtered point cloud data, mapping three-dimensional coordinates of each point in the point cloud to an index of a Hilbert curve to generate one-dimensional serialized point cloud data;

[0033] performing feature tessellation on the serialized point cloud data;

[0034] dividing the feature tessellated point cloud data according to an average grid division method, and performing pooling on point cloud training data of the divided grid;

[0035] performing multiple data shuffling and rearrangement on the pooled point cloud training data;

[0036] adding conditional positional encoding to the rearranged point cloud training data, generating a positional encoding vector according to the point cloud data and its eight-direction adjacent point cloud class label information, and increasing the class association between the training point cloud and the neighborhood;

[0037] constructing a multi-layer convolutional network on the point cloud training data after adding the conditional positional encoding, each layer of the convolutional network including convolution operation, pooling operation and nonlinear activation function, performing convolution processing on point cloud features layer by layer to obtain a high-dimensional feature vector of the point cloud features, inputting the high-dimensional feature vector into a classifier or a regressor, and performing regression calculation on the weld point cloud recognition model parameters through back propagation, and ending the training and outputting the point cloud weld recognition model when the loss function value reaches a preset convergence condition.

[0038] Further, the noise filtering on the marked point cloud data comprises:

[0039] setting a filter radius r and a minimum number of neighbor points according to the size of the group assembly workpiece, removing point cloud data with a number of neighbor points less than a threshold n within the filter radius r, to obtain filtered point cloud data;

[0040] setting the filter formula as P' = {p e P | count (q e P, d (p, q) < r) >= n}; wherein P is the original point cloud data, P' is the filtered point cloud data, and d(p, q) is the distance between 3D points p and q.

[0041] Further, the point cloud three-dimensional Hilbert encoding serialization on the filtered point cloud data, mapping three-dimensional coordinates of each point in the point cloud to an index of a Hilbert curve to generate one-dimensional serialized point cloud data comprises:

[0042] setting the Hilbert serialization formula as follows: H(p) = hilbert_index(p.x, p.y, p.z);

[0043] Wherein, H(p) is the Hilbert index of point p, and hilbert_index(p.x, p.y, p.z) is a function for calculating the Hilbert index.

[0044] Further, the feature embedding of the serialized point cloud data comprises:

[0045] The coordinates and category color labels (r, g, b) of each point in the serialized point cloud data are embedded to generate a high-dimensional feature vector.

[0046] The linear transformation and the nonlinear activation function formula are set as: Wherein, f(x') is the output of the activation function, and x' is the input.

[0047] The embedded features are processed using the linear transformation and the nonlinear activation function, and the feature representation data is obtained.

[0048] Further, the segmented point cloud data after feature embedding is divided by the average grid division method, comprising:

[0049] The average grid division formula is set as:

[0050] Wherein, G(p) is the grid coordinate of point p, Δx, Δy and Δz are the size of the grid, p.x is the horizontal coordinate of point p, p.y is the vertical coordinate of point p, and p.z is the vertical coordinate of point p, and are the minimum values of the point cloud data in each direction.

[0051] Further, the regression calculation of the weld point cloud recognition model parameters by back propagation comprises:

[0052] The back propagation formula is set as:

[0053] Wherein, Δw is the update of the weight, η is the learning rate, L is the loss function, and w is the weight.

[0054] Further, the iterative training and parameter adjustment of the point cloud weld recognition model, verification of whether the average intersection over union and average accuracy meet the preset target, comprise:

[0055] The average intersection over union calculation formula is set as:

[0056] Wherein TP k True positive, i.e. the number of pixels correctly predicted for this category; FP k False positive, i.e. the number of pixels incorrectly predicted as this category; FN kThe mIoU is the mean value of the IoU obtained for all k classes. k

[0057] The application also provides a ship sub-assembly weld joint identification system, which comprises a weld joint identification model training device, a weld joint identification and welding robot control device, a 3D point cloud camera array, and a welding robot and its control device.

[0058] The weld joint identification model training device is used to discretize ship sub-assembly CAD data through point cloud, semi-automatically interactively label weld joint areas, and expand a point cloud weld joint identification training data set through style migration. The marked point cloud data are used for data preprocessing and training model construction to obtain a point cloud weld joint identification model. The point cloud weld joint identification model is iteratively trained and parameter-adjusted to verify whether the average intersection-over-union ratio and the average accuracy rate meet preset targets. If the preset targets are met, the structure and parameters of the point cloud weld joint identification model are solidified, and the identification model data file is exported.

[0059] The 3D point cloud camera array is used to collect and transmit 3D point cloud data of a sub-assembly workpiece to be welded in a production site.

[0060] The weld joint identification and welding robot control device is used to preprocess the collected 3D point cloud data of the sub-assembly workpiece, obtain weld joint marking position data in combination with the identification model data file, and position the weld joint. The weld joint marking position data are used to plan an end trajectory of a welding robot, and welding process planning and instructions are issued.

[0061] The welding robot and its control device are used to complete welding of the sub-assembly workpiece according to the issued welding process planning and instructions.

[0062] Further, the weld joint identification model training device comprises a CAD assembly model-point cloud conversion module, a weld joint semi-automatic interactive labeling module, a training set expansion module based on style migration, and a point cloud weld joint identification model training module.

[0063] The CAD assembly model-point cloud conversion module is used to import a three-dimensional CAD design model of a ship sub-assembly sample, and discretize the surface patch model into point cloud format data for training.

[0064] The weld joint semi-automatic interactive labeling module is used to review and edit the sampling and identification of weld joint feature labeling results.

[0065] The training set expansion module based on style migration uses a style migration network to perform multi-style data fusion generation on the texture, illumination, and roughness of the sampling three-dimensional model, and maps the weld joint feature data labeled by the weld joint semi-automatic interactive labeling module to the point cloud data of the sub-assembly with the same configuration but different materials and textures.​

[0066] The point cloud weld identification model training module constructs a sub-assembly weld point cloud identification network model based on a PTV3 neural network, and is used for training weld identification model parameters on the sub-assembly point cloud data after the extension mark, and obtaining a point cloud weld identification model file.

[0067] Further, the weld identification and welding robot control device comprises a workpiece 3D point cloud data acquisition module, a point cloud data preprocessing module, a weld identification and positioning module, and a welding process planning and instruction issuing module.

[0068] The 3D point cloud data acquisition module is used for controlling a 3D point cloud camera array to acquire and transmit point cloud data.

[0069] The point cloud data preprocessing module is used for performing weld part bounding box segmentation and abnormal value filtering on the acquired point cloud.

[0070] The weld identification and positioning module uses the trained weld identification model to identify the preprocessed point cloud data, obtains weld mark position data, and positions and matches the identification point three-dimensional position coordinates with the welding robot coordinate system according to the welding robot position relationship.

[0071] The welding process planning and instruction issuing module is used for automatically planning a robot end trajectory according to the identified weld position data, and performing virtual simulation verification until a collision-free robot end process execution scheme compilation instruction is generated, and the scheme compilation instruction is issued to the welding robot and its control device through a control bus.

[0072] In another aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the ship sub-assembly weld identification method described above when executing the computer program.

[0073] In yet another aspect, a computer readable storage medium is provided, which stores a computer program, wherein the computer program is executed by a processor to implement the steps of the ship sub-assembly weld identification method described above.

[0074] Compared with the prior art, the beneficial effects of the present application are: by discretizing the ship sub-assembly CAD data through point cloud, virtual sampling, semi-automatic interactive marking of the weld area, and style migration expansion to establish a point cloud weld recognition training data set, using a point cloud weld recognition model to export a recognition model data file, using a 3D camera array to collect 3D point cloud data of a ship sub-assembly workpiece to be welded in a production environment, combining the recognition model data file to obtain weld marking position data and positioning the weld, and then issuing a welding process plan and instructions to control a welding robot and its control equipment to complete the welding of the sub-assembly workpiece, an automatic and intelligent weld recognition and positioning scheme can be provided for the welding of complex ship sub-assembly parts, the weld recognition model only needs to complete automatic feature sampling, marking and training according to the welding CAD model, the problem of point cloud missing and recognition failure caused by the type of steel and the shooting angle of the 3D camera in the traditional 3D weld recognition method is overcome, and the adaptability of the automatic weld recognition method to complex workpieces is greatly improved. BRIEF DESCRIPTION OF DRAWINGS

[0075] The drawings in the specification are only used for illustrative purposes and should not be understood as limiting the patent; it is understandable to those skilled in the art that some known structures and their descriptions in the drawings can be omitted.

[0076] Fig. 1 is a structural schematic diagram of a ship sub-assembly weld recognition system of the present application;

[0077] Fig. 2 is a flowchart of a ship sub-assembly weld recognition method of the present application;

[0078] Fig. 3 is a flowchart of a method for generating a weld recognition model training data of a sub-assembly workpiece CAD model of the present application;

[0079] Fig. 4 is a flowchart of a training method of a weld recognition model of the present application;

[0080] Fig. 5 is a structural schematic diagram of a computer device of the present application.

[0081] The symbols in the drawings are as follows: 100, weld recognition model training equipment; 101, CAD assembly model-point cloud conversion module; 102, weld semi-automatic interactive marking module; 103, training set expansion module based on style migration; 104, point cloud weld recognition model training module; 200, point cloud weld recognition model file; 300, 3D point cloud camera array; 301, 3D point cloud camera; 400, weld recognition and welding robot control equipment; 401, workpiece 3D point cloud data acquisition module; 402, point cloud data preprocessing module; 403, weld recognition and positioning module; 404, welding process planning and instruction issuing module; 500, welding robot and its control equipment. DETAILED DESCRIPTION

[0082] The technical solutions of the present application will be described clearly and completely below in conjunction with the drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0083] Embodiment 1

[0084] As shown in FIG. 1, a ship small assembly weld seam identification system is provided in Embodiment 1 of the present application, which comprises a weld seam identification model training device 100, a weld seam identification and welding robot control device 400, a 3D point cloud camera array 300, and a welding robot and its control device 500.

[0085] As shown in FIG. 1, the weld seam identification model training device 100 is loaded with a weld seam identification model training application. The application contains a CAD assembly model-point cloud conversion module 101, which is used to import a three-dimensional CAD design model of a ship small assembly sample and discretize the facet model into point cloud format data for training.

[0086] As shown in FIG. 1, the weld seam identification model training application contains a weld seam semi-automatic interactive labeling module 102, which uses a virtual camera to sample and identify weld seam feature labeling results for auditing and editing.

[0087] As shown in FIG. 1, the weld seam identification model training application contains a training set expansion module 103 based on style transfer, which uses a style transfer network to generate multi-style data fusion of the texture, illumination, and roughness of the sampled three-dimensional model, and maps the weld seam feature data labeled by the weld seam semi-automatic interactive labeling module 102 to more small assembly point cloud data of the same configuration, different materials, and textures.

[0088] As shown in FIG. 1, the point cloud weld seam identification model training module 400 constructs a small assembly weld seam point cloud identification network model based on a PTV3 neural network, which is used to train the weld seam identification model parameters on the expanded labeled small assembly point cloud data, thereby obtaining a point cloud weld seam identification model file 200.

[0089] As shown in FIG. 1, the weld seam identification and welding robot control device 400 is loaded with an automatic welding identification and control program.

[0090] The automatic welding identification and control program contains a 3D point cloud data acquisition module 401, which is used to control the 3D point cloud camera array 300 to acquire and transmit point cloud data.

[0091] The automatic welding identification and control program contains a point cloud data preprocessing module 402, which is used to perform weld part bounding box segmentation and outlier filtering on the acquired point cloud.

[0092] The automatic welding recognition and control program comprises a welding seam recognition and positioning module 403, which uses a trained welding seam recognition model to recognize the preprocessed point cloud data, obtains welding seam marker position data, and positions and matches the recognized point three-dimensional position coordinates with the robot coordinate system according to the robot position relationship.

[0093] The automatic welding recognition and control program comprises a welding process planning and instruction issuing module 404, which is used for automatically planning the mechanical arm end trajectory according to the recognized welding seam position data, and performing virtual simulation verification until a collision-free mechanical arm end process execution scheme compilation instruction is generated, and the scheme compilation instruction is issued to the welding mechanical arm and its control equipment through a control bus, to complete the small group assembly welding.

[0094] Embodiment 2

[0095] As shown in FIG. 2, the embodiment 2 of the present application also provides a ship small group assembly welding seam recognition method, which specifically comprises the following steps:

[0096] S1: Import the ship small group assembly CAD model to the welding seam recognition model training application program. The program function module uses the CAD assembly model-point cloud conversion function to convert the face sheet type CAD data into marked point cloud data for welding seam recognition model training after point cloud discretization, semi-automatic interactive annotation of the welding seam area, and welding seam recognition training set style migration and expansion processing. This step comprises the following sub-steps:

[0097] S11: The user imports the ship small group assembly three-dimensional CAD model file through the function button on the interactive interface of the welding seam recognition model training application program, adjusts the model file position, and makes the model located in the center of the interface viewport, so as to facilitate operation and observation.

[0098] S12: The user executes the point cloud welding seam automatic annotation function through the function button on the interactive interface of the welding seam recognition model training application program to obtain the point cloud welding seam recognition model training data set. After the point cloud welding seam automatic annotation function is started, the system establishes a virtual sampling camera according to the virtual sampling camera focal length, horizontal resolution, vertical resolution, optical center position parameters and perspective projection equation.

[0099] S13: After the virtual sampling camera is established, the point cloud welding seam automatic annotation function uses the Fibonacci spherical sampling algorithm to generate virtual camera sampling position points with uniform distribution on the half spherical surface, taking the center of the three-dimensional bounding box of the ship small group assembly CAD model as the origin and r as the radius. The calculation formula of the Fibonacci spherical sampling algorithm is as follows:

[0100] Wherein, θ is the azimuth angle of the sampling point, i is the index of the sampling point, and φ is the golden ratio.

[0101] The virtual camera traverses the virtual camera sampling position points, aligns the virtual camera optical axis direction with the sampling position point-workpiece center point line at each sampling position point, and collects a two-dimensional virtual image of the workpiece three-dimensional model.

[0102] S14: For the collected two-dimensional virtual sampling weld image, the point cloud weld automatic labeling function uses the Canny edge detection operator to identify the edge pixel points of the sampling image, filter the background edges, and perform Hough line fitting to obtain the 2D pixel point position I={(x,y) | 0≤xW, 0≤yH} of the candidate weld intersection line in the two-dimensional virtual sampling weld image. (x,y) is the 2D pixel point position, W is the width of the two-dimensional virtual sampling weld image, and H is the length of the two-dimensional virtual sampling weld image.

[0103] S15: According to the perspective projection transformation relationship, the 3D position points of the workpiece model corresponding to the candidate weld intersection line 2D pixel points extracted in the multi-view image are calculated as candidate marked weld points. The calculation formula of the perspective projection transformation relationship is as follows:

[0104] Where (X,Y,Z) is the 3D point cloud position point, (x,y) is the 2D pixel point position, and f is the focal length. According to the constraint of the angle between the two sides of the marked weld, non-weld intersection points are filtered to obtain candidate marked weld points. It can be understood that X is the horizontal coordinate of the 3D point cloud position point (X,Y,Z), Y is the vertical coordinate of the 3D point cloud position point (X,Y,Z), Z is the vertical coordinate of the 3D point cloud position point (X,Y,Z) perpendicular to the XY plane, x is the horizontal coordinate of the 2D pixel point (x,y), and y is the vertical coordinate of the 2D pixel point (x,y).

[0105] S16: After coloring the candidate marked weld points, Kmeans is used for regional clustering to obtain the marked weld rectangular bounding box coordinates and regions, and a weld rectangular bounding box representing the weld region is generated, which is coincided with the small group assembly workpiece CAD model in the viewport. The user interacts to select and identify the wrong weld region, and finally retains the correct marked weld region bounding box.

[0106] S17: Discretize the three-dimensional small group assembly workpiece CAD model into point cloud data, and mark the point cloud data in the marked weld region bounding box as target feature points. Export and save the marked point cloud data.

[0107] S2: The point cloud weld recognition model training function of the weld recognition model training application program is used to pre-process the marked point cloud data obtained in S1 and construct a model to obtain a weld point cloud recognition training model. This step includes the following sub-steps:

[0108] S21: The program function module filters the noise of the marked data. The user sets the filtering radius r and the minimum number of neighbor points according to the workpiece size of the group, removes the point cloud data whose number of neighbor points within the specified radius r is less than the threshold value n, and obtains the filtered point cloud data. The filtering formula is P' = {p e P | count (q e P, d (p, q) < r) >= n}. Wherein, P is the original point cloud data, P' is the filtered point cloud data, and d(p, q) is the distance between 3D points p and q.

[0109] S22: The program function module serializes the point cloud three-dimensional Hilbert encoding of the optimized training point cloud data. The three-dimensional coordinates of each point in the point cloud are mapped to the index of the Hilbert curve to generate one-dimensional serialized point cloud data. The Hilbert serialization formula is as follows: H(p) = hilbert_index(p.x, p.y, p.z) Wherein, H(p) is the Hilbert index of point p, and hilbert_index(p.x, p.y, p.z) is a function for calculating the Hilbert index.

[0110] S23: The program function module performs feature embedding on the serialized point cloud data. The coordinates and category color labels (r, g, b) of each point are embedded to generate high-dimensional feature vectors. The embedded features are processed using linear transformation and nonlinear activation functions, and the features represent the data. The activation function formula is: Wherein, f(x') is the output of the activation function, and x' is the input.

[0111] S24: The program function module divides the feature-embedded point cloud data according to the average grid division method. The average grid division formula is as follows:

[0112] Wherein, G(p) is the grid coordinates of point p, Δx, Δy and Δz are the size of the grid, p.x is the horizontal coordinate of point p, p.y is the vertical coordinate of point p, and p.z is the vertical coordinate of point p, And are the minimum values of the point cloud data in each direction.

[0113] S25: The program function module performs multiple data reordering on the pooled training point cloud data to enhance the robustness of the model.

[0114] S26: The program function module adds conditional position encoding to the training point cloud data. The position encoding vector is generated according to the point cloud data and its eight-direction adjacent point cloud category label information, which increases the category association between the training point cloud and the neighborhood, and further improves the model prediction accuracy.

[0115] S27: The program function module constructs a multi-layer convolutional network for the point cloud training data after adding the conditional position coding, each layer containing convolution operation, pooling operation and nonlinear activation function, and convolves the point cloud features layer by layer to obtain a high-dimensional feature vector of the point cloud features, inputs the high-dimensional feature vector into a classifier or regressor, and performs regression calculation on the weld point cloud recognition model parameters through back propagation, and ends the training and outputs the recognition model when the loss function value reaches the preset convergence condition. The back propagation formula is: where Δw is the update of the weight, η is the learning rate, L is the loss function, and w is the weight.

[0116] S3: Iterative training and parameter adjustment are performed on the point cloud recognition training model constructed in S2, and whether the Mean Intersection Over Union (mIoU) and Mean Average Precision (mAP) indicators meet the preset target is verified, and if so, the model structure and parameters are solidified, and exported as a point cloud weld recognition model file 200. The calculation formula of mIoU is: where TP k is the number of pixels of the True Positives, i.e. the pixels of this category are correctly predicted; FP k is the number of pixels of the False Positives, i.e. the pixels of this category are incorrectly predicted; FN k is the number of pixels of the False Negatives, i.e. the pixels of this category are incorrectly predicted as other categories, and mIoU is the mean of the IoU k of all k categories.

[0117] S4: Import the point cloud weld recognition model file 200 obtained in S3 into the automatic welding recognition and control program loaded on the weld recognition and welding robot control device 400 in the production environment, and then obtain the to-be-recognized point cloud data through the workpiece 3D point cloud data acquisition function and the point cloud data preprocessing function, automatically recognize the weld position of the small assembly workpiece through the point cloud weld recognition model obtained in S3, and control the welding robot and its control device to complete the welding of the small assembly workpiece through the welding process planning and instruction issuing function.

[0118] It can be understood that after the welding of the small assembly workpiece is completed, it also includes:

[0119] S5: Weld point cloud recognition in the production environment.

[0120] The application changes the traditional way of recognizing and positioning a weld by projecting point cloud data and template matching, combines automatic 3D virtual sampling, transfer learning, Hilbert sparse coding and Transformer convolution recognition technology, provides an automatic and intelligent weld recognition and positioning scheme for complex ship small assembly part welding, so that the weld recognition model only needs to complete automatic feature sampling, marking and training according to a welding part CAD model, overcomes the point cloud loss and recognition failure caused by the steel type and 3D camera shooting angle in the traditional 3D weld recognition method, and greatly improves the adaptability of the automatic weld recognition method to complex workpieces.

[0121] Embodiment 3

[0122] The application provides a ship small assembly weld recognition method, which comprises the following steps:

[0123] The ship small assembly CAD data is discretized into point cloud data, virtually sampled, semi-automatically marked in the weld area, and style migration and expansion are performed to establish a point cloud weld recognition training data set;

[0124] The marked point cloud data is preprocessed and a model is trained to obtain a point cloud weld recognition model;

[0125] The point cloud weld recognition model is iteratively trained and parameter-adjusted, and whether the average intersection-over-union ratio and the average accuracy meet the preset target is verified, if the preset target is met, the structure and parameters of the point cloud weld recognition model are fixed, and the recognition model data file is exported;

[0126] A 3D camera array is used to collect 3D point cloud data of a ship small assembly workpiece to be welded in a production site;

[0127] The 3D point cloud data of the ship small assembly workpiece collected by the 3D camera array in the production site is preprocessed, the weld marking position data is obtained in combination with the recognition model data file, and the weld is positioned;

[0128] A welding mechanical arm trajectory is planned according to the weld marking position data, a welding process plan and instructions are issued to control the welding mechanical arm and its control equipment to complete the welding of the small assembly workpiece.

[0129] Further, the virtual sampling step in the step of constructing a point cloud weld recognition training data set using ship small assembly CAD data comprises:

[0130] A virtual 3D sampling camera is established according to the focal length, horizontal resolution, vertical resolution, optical center position parameters and perspective projection equation of the virtual 3D sampling camera;

[0131] After the virtual 3D sampling camera is established, a sampling position point is generated by using a Fibonacci spherical surface sampling algorithm with the center of the three-dimensional bounding box of the small assembly CAD model of the ship as the origin;

[0132] The virtual 3D sampling camera is made to traverse the sampling position points, the direction of the optical axis of the virtual 3D sampling camera is aligned with the line connecting the sampling position points and the center point of the workpiece at each sampling position point, and a two-dimensional virtual image of the three-dimensional model of the workpiece is collected;

[0133] For the collected two-dimensional virtual sampling weld image, the point cloud weld automatic labeling function uses a Canny edge detection operator to identify the edge pixel points of the sampling image, filter the background edges, perform Hough line fitting, and obtain the 2D pixel point positions of the candidate weld intersection lines in the two-dimensional virtual sampling weld image;

[0134] According to the perspective projection transformation relationship, the three-dimensional position points of the workpiece model corresponding to the candidate weld intersection line 2D pixel points extracted in the multi-view image are calculated as candidate marked weld points, and the candidate marked weld points are filtered according to the constraint of the angle between the two planes on both sides of the marked weld to obtain the candidate marked weld points;

[0135] After the candidate marked weld points are colored, a clustering algorithm is used for regional clustering to obtain the coordinates and regions of the marked weld rectangular bounding boxes, and a weld rectangular bounding box representing the weld region is generated, which is overlapped with the small assembly workpiece CAD model in the interface viewport, and the user interacts to select and identify the incorrect weld region, and finally the correct marked weld region bounding box is retained;

[0136] Discretize the three-dimensional small assembly workpiece CAD model into point cloud data, and mark the point cloud data in the marked weld region bounding box as target feature points.

[0137] Further, after the virtual 3D sampling camera is established, a sampling position point is generated by using a Fibonacci spherical surface sampling algorithm with the center of the three-dimensional bounding box of the small assembly CAD model of the ship as the origin, including:

[0138] The calculation formula of the Fibonacci spherical surface sampling algorithm is set as follows:

[0139] Where, θ is the azimuth angle of the sampling point, i is the index of the sampling point, and φ is the golden ratio.

[0140] Further, the candidate marked weld points corresponding to the candidate weld intersection line 2D pixel points extracted in the multi-view image are calculated according to the perspective projection transformation relationship, including:

[0141] The calculation formula of the perspective projection transformation relationship is set as follows:

[0142] wherein (X, Y, Z) is a 3D point cloud position point, (x, y) is a 2D pixel position, and f is a focal length of the virtual 3D sampling camera.

[0143] Further, the pre-processing of the mark point cloud data collected by the virtual 3D sampling camera, obtaining the weld mark position data in combination with the recognition model data file and positioning the weld seam include:

[0144] The mark point cloud data collected is segmented by a weld part body bounding box, and abnormal values are filtered to achieve pre-processing;

[0145] The mark point cloud data after pre-processing and the recognition model data file are recognized by using the trained point cloud weld seam recognition model, the weld mark position data is obtained, and the three-dimensional position coordinates of the recognition points are positioned and matched with the welding robot coordinate system according to the position relationship of the welding robot.

[0146] Further, the point cloud weld seam recognition model is obtained by using the mark point cloud data for data pre-processing and training model construction, and the point cloud weld seam recognition model includes:

[0147] The mark point cloud data is filtered for noise;

[0148] The point cloud three-dimensional Hilbert encoding serialization is performed on the filtered point cloud data, the three-dimensional coordinates of each point in the point cloud are mapped to the index of the Hilbert curve, and one-dimensional serialized point cloud data is generated;

[0149] The feature inlaying is performed on the serialized point cloud data;

[0150] The point cloud data after feature inlaying is segmented by the average grid division method, and the point cloud training data of the segmented grid is pooled;

[0151] The point cloud training data after pooling is rearranged multiple times in a disorderly manner;

[0152] The point cloud training data after rearrangement is added with conditional position encoding, a position encoding vector is generated according to the point cloud data and the eight-direction adjacent point cloud class label information, and the class association between the training point cloud and the neighborhood is increased;

[0153] The point cloud training data after adding the conditional position encoding is constructed into a multi-layer convolutional network, each layer of the convolutional network includes convolution operation, pooling operation and nonlinear activation function, the point cloud features are processed by convolution layer by layer to obtain a high-dimensional feature vector of the point cloud features, the high-dimensional feature vector is input into a classifier or a regressor, the parameters of the weld seam point cloud recognition model are calculated by regression through back propagation, and when the loss function value reaches the preset convergence condition, the training is ended and the point cloud weld seam recognition model is output.

[0154] Further, the noise filtering of the mark point cloud data comprises:

[0155] According to the group workpiece size setting filtering radius r and the minimum number of neighbor points, the point cloud data with a number of neighbor points less than a threshold value n within the filtering radius r is removed, to obtain filtered point cloud data;

[0156] The filtering formula is set as P'={p∈P∣count(q∈P,d(p,q)<r)≥n}; wherein, P is the original point cloud data, P' is the filtered point cloud data, and d(p,q) is the distance between 3D points p and q.

[0157] Further, the point cloud three-dimensional Hilbert encoding serialization of the filtered point cloud data, mapping the three-dimensional coordinates of each point in the point cloud to the index of the Hilbert curve, to generate one-dimensional serialized point cloud data comprises:

[0158] The Hilbert serialization formula is set as follows: H(p)=hilbert_index(p.x,p.y,p.z);

[0159] Wherein, H(p) is the Hilbert index of point p, and hilbert_index(p.x,p.y,p.z) is a function for calculating the Hilbert index.

[0160] Further, the feature embedding of the serialized point cloud data comprises:

[0161] The coordinates and category color labels (r,g,b) of each point in the serialized point cloud data are embedded to generate a high-dimensional feature vector;

[0162] The linear transformation and nonlinear activation function formula are set as: Wherein, f(x') is the output of the activation function, and x' is the input;

[0163] The embedded features are processed using the linear transformation and nonlinear activation function, and the feature representation data.

[0164] Further, the segmentation of the feature-embedded point cloud data according to the average grid division method comprises:

[0165] The average grid division formula is set as follows:

[0166] Wherein, G(p) is the grid coordinate of point p, Δx, Δy and Δz are the size of the grid, p.x is the horizontal coordinate of point p, p.y is the vertical coordinate of point p, and p.z is the vertical coordinate of point p, and is the minimum value of the point cloud data in each direction.

[0167] Further, the regression calculation of the weld point cloud identification model parameters by back propagation includes:

[0168] The back propagation formula is set as:

[0169] Where, Δw is the update of the weight, η is the learning rate, L is the loss function, and w is the weight.

[0170] Further, the iteration training and parameter adjustment of the point cloud weld identification model, and the verification of whether the average intersection over union and the average accuracy meet the preset target include:

[0171] The calculation formula of the average intersection over union is set as:

[0172] Where, TP k is the true positive, i.e., the number of pixels of the category that are correctly predicted; FP k is the false positive, i.e., the number of pixels that are incorrectly predicted as the category; FN k is the false negative, i.e., the number of pixels of the category that are incorrectly predicted as other categories, and mIoU is the mean of the IoU k of all k categories.

[0173] Embodiment 4

[0174] As shown in FIG. 1, the application embodiment 4 provides a ship small assembly weld identification system, which includes a weld identification model training device 100, a weld identification and welding robot control device 400, a 3D point cloud camera array, and a welding robot and a control device 500 thereof;

[0175] The weld identification model training device 100 is used to discretize the ship small assembly CAD data through point cloud, semi-automatically interactively label the weld area, and expand the style to establish a point cloud weld identification training data set. The marked point cloud data is used for data preprocessing and training model construction to obtain a point cloud weld identification model. The point cloud weld identification model is iteratively trained and parameter adjusted to verify whether the average intersection over union and the average accuracy meet the preset target. If the preset target is met, the structure and parameters of the point cloud weld identification model are solidified, and the identification model data file is exported.

[0176] The 3D point cloud camera array 300 is used to collect and transmit the 3D point cloud data of the ship small assembly workpiece to be welded in the production site.

[0177] The weld seam recognition and welding robot control device 400 is used for pre-processing the collected ship sub-assembly workpiece 3D point cloud data, obtaining weld seam mark position data in combination with the recognition model data file, and positioning the weld seam; the weld seam mark position data is used for planning the welding robot end trajectory, and the welding process planning and instruction are issued;

[0178] The welding robot and the control device 500 thereof are used for completing the sub-assembly workpiece welding according to the issued welding process planning and instruction.

[0179] Further, the weld seam recognition model training device 100 comprises a CAD assembly model-point cloud conversion module 101, a weld seam semi-automatic interactive labeling module 102, a training set expansion module based on style migration 103, and a point cloud weld seam recognition model training module 104.

[0180] The CAD assembly model-point cloud conversion module 101 is used for importing a ship sub-assembly sample three-dimensional CAD design model, and discretizing the face sheet model into point cloud format data for training;

[0181] The weld seam semi-automatic interactive labeling module 102 is used for auditing and editing the sampling recognition weld seam feature labeling results;

[0182] The training set expansion module based on style migration 103 uses a style migration network to perform multi-style data fusion generation on the texture, illumination and roughness of the sampling three-dimensional model, and maps the weld seam feature data labeled by the weld seam semi-automatic interactive labeling module 102 to the sub-assembly point cloud data with the same configuration but different materials and textures;

[0183] The point cloud weld seam recognition model training module 104 constructs a sub-assembly weld seam point cloud recognition network model based on a PTV3 neural network, is used for training weld seam recognition model parameters on the extended and labeled sub-assembly point cloud data, and obtains a point cloud weld seam recognition model file 200.

[0184] Further, the weld seam recognition and welding robot control device 400 comprises a workpiece 3D point cloud data acquisition module 401, a point cloud data preprocessing module 402, a weld seam recognition and positioning module 403, and a welding process planning and instruction issuing module 404.

[0185] The workpiece 3D point cloud data acquisition module 401 is used for controlling the 3D point cloud photographing camera array 300 to acquire and transmit point cloud data;

[0186] The point cloud data preprocessing module 402 is used for performing weld part body bounding box segmentation and abnormal value filtering on the acquired point cloud;

[0187] The weld seam recognition and positioning module 403 uses the trained weld seam recognition model to recognize the pre-processed point cloud data, obtains weld seam marker position data, and positions and matches the recognized point three-dimensional position coordinates with the welding robot coordinate system according to the welding robot position relationship;

[0188] The welding process planning and instruction issuing module 404 is used for automatically planning the robot end trajectory according to the recognized weld seam position data, and performing virtual simulation verification until generating a collision-free robot end process execution scheme compilation instruction, and issuing the scheme compilation instruction to the welding robot and the control device 500 thereof through the control bus.

[0189] Embodiment 5

[0190] In embodiment 5, a computer device is provided, which can be a server, and the internal structure diagram thereof can be as shown in FIG. 5. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store ship small assembly weld seam recognition data. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement a ship small assembly weld seam recognition method.

[0191] In one embodiment, a computer device is provided, including a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the ship small assembly weld seam recognition method described above when executing the computer program.

[0192] For specific limitations of the steps implemented by the processor when executing the computer program, refer to the limitations of the ship small assembly weld seam recognition method described above, which will not be repeated here.

[0193] Embodiment 6

[0194] In embodiment 6, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by the processor to implement the steps of the ship small assembly weld seam recognition method described above.

[0195] For specific limitations of the steps implemented by the processor when executing the computer program, refer to the limitations of the ship small assembly weld seam recognition method described above, which will not be repeated here.

[0196] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, storage, databases, or other media in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0197] The technical features of the above embodiments can be combined in any way. In order to make the description simple, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the present application.

[0198] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method of ship panel line weld identification, characterized by, The method comprises the steps of: Discretizing, virtually sampling, semi-automatically marking, and style migration and expansion of ship sub-assembly CAD data through point cloud to establish a point cloud weld recognition training data set; Data preprocessing and model training are performed using the point cloud weld recognition training data set to obtain a point cloud weld recognition model; Iterative training and parameter adjustment are performed on the point cloud weld recognition model to verify whether the average intersection-over-union ratio and the average accuracy meet the preset target, and if so, the structure and parameters of the point cloud weld recognition model are fixed, and the recognition model data file is exported; A 3D point cloud camera array is used to collect and transmit 3D point cloud data of a ship sub-assembly workpiece to be welded in a production site; The collected 3D point cloud data of the ship sub-assembly workpiece are preprocessed, and weld mark position data are obtained and the welds are located in combination with the recognition model data file; A weld mechanical arm end trajectory is planned according to the weld mark position data, and a weld process plan and instructions are issued to control the weld mechanical arm and its control equipment to complete the welding of the sub-assembly workpiece.

2. The method of group weld seam identification of claim 1, wherein The method of establishing a point cloud weld recognition training data set through virtual sampling of ship sub-assembly CAD data comprises: A virtual 3D sampling camera is established according to the focal length, horizontal resolution, vertical resolution, optical center position parameters, and perspective projection equation of the virtual 3D sampling camera; After the virtual 3D sampling camera is established, the center of the three-dimensional bounding box of the ship sub-assembly CAD model is taken as the origin, and a semi-spherical surface is uniformly distributed using the Fibonacci spherical sampling algorithm to generate sampling position points; The virtual 3D sampling camera is made to traverse the sampling position points, and at each sampling position point, the virtual 3D sampling camera optical axis direction is aligned with the connecting line between the sampling position point and the workpiece center point, and a two-dimensional virtual image of the workpiece three-dimensional model is collected; For the collected two-dimensional virtual sampling weld image, the point cloud weld automatic marking function uses the Canny edge detection operator to identify the edge pixel points of the sampling image, filter the background edges, perform Hough line fitting, and obtain the 2D pixel point positions of the candidate weld intersection lines in the two-dimensional virtual sampling weld image; According to the perspective projection transformation relationship, the three-dimensional position points of the candidate weld intersection lines 2D pixel points extracted in the multi-view image are calculated as candidate marked weld points, and the non-weld intersection points are filtered according to the plane angle constraint of the two sides of the marked weld, to obtain the candidate marked weld points; After coloring the candidate marked weld points, a clustering algorithm is used for regional clustering to obtain the marked weld rectangular bounding box coordinates and regions, and a weld rectangular bounding box representing the weld region is generated, which is overlapped with the sub-assembly workpiece CAD model in the interface viewport, and the user interacts to select the incorrectly recognized weld region, and finally the correct marked weld region bounding box is retained; The three-dimensional sub-assembly workpiece CAD model is discretized into point cloud data, the point cloud data in the marked weld region bounding box are marked as target feature points, and the marked point cloud data are exported and saved.

3. The method of claim 2, wherein: After the virtual 3D sampling camera is established, the center of the three-dimensional bounding box of the ship sub-assembly CAD model is taken as the origin, and a semi-spherical surface is uniformly distributed using the Fibonacci spherical sampling algorithm to generate sampling position points; The calculation formula of the Fibonacci sphere sampling algorithm is set as follows: Wherein, θ is the azimuth angle of the sampling point, i is the index of the sampling point, and φ is the golden ratio.

4. The method of group weld seam identification of claim 2, wherein, The 3D position point of the candidate weld intersection line 2D pixel point extracted from the multi-view image is calculated according to the perspective projection transformation relationship, and the 3D position point is a candidate marked weld point. The calculation formula for setting the perspective projection transformation relationship is as follows: Wherein, (X, Y, Z) is a 3D point cloud position point, (x, y) is a 2D image pixel position, and f is the focal length of the virtual 3D sampling camera.

5. The method of group weld seam identification of claim 1, wherein, The marked point cloud data collected by the virtual 3D sampling camera is preprocessed, and the weld mark position data is obtained by combining the recognition model data file to locate the weld. The collected marked point cloud data is segmented by the weld body bounding box, and the abnormal value is filtered to realize preprocessing; The trained point cloud weld recognition model is used to identify the preprocessed marked point cloud data and the recognition model data file, obtain the weld mark position data, and position the three-dimensional position coordinates of the recognition points according to the position relationship of the welding robot coordinate system.

6. The method of group weld seam identification of claim 1, wherein, The point cloud weld recognition model is obtained by data preprocessing and training model construction using the marked point cloud data. The marked point cloud data is filtered; The filtered point cloud data is serialized by point cloud three-dimensional Hilbert encoding, the three-dimensional coordinates of each point in the point cloud are mapped to the index of the Hilbert curve, and one-dimensional serialized point cloud data is generated; The serialized point cloud data is feature inlaid; The point cloud data after feature inlay is segmented by average grid division method, and the point cloud training data of the segmented grid is pooled; The pooled point cloud training data is rearranged multiple times; The rearranged point cloud training data is added with conditional position coding, and the position coding vector is generated according to the point cloud data and its eight direction adjacent point cloud class label information, so as to increase the class association between the training point cloud and the neighborhood; The point cloud training data after adding conditional position coding is constructed into a multi-layer convolutional network, each layer of the convolutional network includes convolution operation, pooling operation and nonlinear activation function, the point cloud features are processed layer by layer, the high-dimensional feature vector of the point cloud feature is obtained, the high-dimensional feature vector is input into the classifier or regressor, and the regression calculation of the weld point cloud recognition model parameters is carried out through back propagation, and the training is ended when the loss function value reaches the preset convergence condition, and the point cloud weld recognition model is output.

7. The method of group weld seam identification of claim 6, wherein, The marked point cloud data is filtered according to the group of standing workpiece size setting filter radius r and the minimum number of neighbor points, and the point cloud data with less neighbor points within the filter radius r is removed, and the filtered point cloud data is obtained. The filter formula is set as P'={p∈P∣count(q∈P,d(p,q)<r)≥n}; wherein, P is the original point cloud data, P' is the filtered point cloud data, and d(p,q) is the distance between 3D points p and q. The filtered point cloud data is serialized by point cloud three-dimensional Hilbert encoding, the three-dimensional coordinates of each point in the point cloud are mapped to the index of the Hilbert curve, and one-dimensional serialized point cloud data is generated.

8. The method of group weld seam identification of claim 6, wherein, ​ The Hilbert serialization formula is set as follows: H(p) = hilbert_index(p.x, p.y, p.z); Wherein, H(p) is the Hilbert index of point p, and hilbert_index(p.x, p.y, p.z) is a function for calculating the Hilbert index.

9. The method of claim 6, wherein The feature embedding of the serialized point cloud data comprises: The coordinates and category color labels (r, g, b) of each point in the serialized point cloud data are embedded to generate a high-dimensional feature vector; The linear transformation and the nonlinear activation function formula are set as: Wherein, f(x') is the output of the activation function, and x' is the input; The embedded features are processed using the linear transformation and the nonlinear activation function, and the feature representation data.

10. The method of group weld seam identification of claim 6, wherein, The segmented feature-embedded point cloud data is divided according to the average grid division method, comprising: The average grid partitioning formula is set as follows: where G(p) is the grid coordinate of point p, Δx, Δy and Δz are the size of the grid, p.x is the lateral coordinate of point p, p.y is the longitudinal coordinate of point p, p.z is the vertical coordinate of point p, is the minimum value of the point cloud data in each direction.

11. The method of group weld seam identification of claim 6, wherein, The regression calculation of the weld point cloud recognition model parameters through back propagation comprises: The backpropagation formula is set as: Wherein, Δw is the update of the weight, η is the learning rate, L is the loss function, and w is the weight.

12. The method of identifying a group fillet weld as recited in claim 1, wherein, The iterative training and parameter adjustment of the point cloud weld recognition model, verification of whether the average intersection-over-union ratio and the average accuracy meet the preset target, comprise: The calculation formula of the average intersection ratio is set as: Among them TP k True cases, i.e., the number of pixels that are correctly predicted for that category; FP k False positives are the number of pixels incorrectly predicted as belonging to that category; FN k A false negative is the number of pixels of a given category that are incorrectly predicted as belonging to other categories. mIoU is the IoU obtained for all k categories. k The mean.

13. A ship block weld identification system, characterized by, The weld recognition model training device, the weld recognition and welding robot control device, the 3D point cloud camera array, and the welding robot and its control device; The weld recognition model training device is used to establish a point cloud weld recognition training data set by point cloud discretization, virtual sampling, semi-automatic interactive labeling of the weld area, and style transfer expansion of the ship small assembly CAD data; The labeled point cloud data is used for data preprocessing and training model construction to obtain a point cloud weld recognition model; The point cloud weld recognition model is iteratively trained and parameter adjusted, and whether the average intersection-over-union ratio and the average accuracy meet the preset target is verified. If the preset target is met, the structure and parameters of the point cloud weld recognition model are solidified, and the recognition model data file is exported; The 3D point cloud camera array is used to collect and transmit the 3D point cloud data of the ship small assembly workpiece to be welded in the production site; The weld recognition and welding robot control device is used to preprocess the collected 3D point cloud data of the ship small assembly workpiece, obtain weld marking position data in combination with the recognition model data file, and position the weld; the weld marking position data is used to plan the end trajectory of the welding robot, and the welding process planning and instructions are issued; The welding robot and its control device are used to complete the welding of the small assembly workpiece according to the issued welding process planning and instructions.

14. The ship panel assembly weld identification system of claim 13, wherein, The weld recognition model training device comprises a CAD assembly model-point cloud conversion module, a weld semi-automatic interactive labeling module, a training set expansion module based on style transfer, and a point cloud weld recognition model training module; The CAD assembly model-point cloud conversion module is used to import a three-dimensional CAD design model of a ship small assembly sample, and discretize the face sheet model into point cloud format data for training; The weld semi-automatic interactive labeling module is used to review and edit the sampling and recognition weld feature labeling results. The style transfer training set expansion module uses a style transfer network to generate multi-style data fusion of the sampled three-dimensional model texture, lighting, and roughness. The weld seam feature data labeled by the weld seam semi-automatic interactive labeling module is mapped to small group assembly point cloud data with the same configuration, different materials, and different textures. The point cloud weld seam recognition model training module constructs a small group assembly weld seam point cloud recognition network model based on a PTV3 neural network, which is used to train the weld seam recognition model parameters on the extended labeled small group assembly point cloud data to obtain a point cloud weld seam recognition model file.

15. The ship panel assembly weld identification system of claim 13, wherein, The weld seam recognition and welding robot control device includes a workpiece 3D point cloud data acquisition module, a point cloud data preprocessing module, a weld seam recognition and positioning module, and a welding process planning and instruction issuing module. The workpiece 3D point cloud data acquisition module is used to control a 3D point cloud camera array to acquire and transmit point cloud data. The point cloud data preprocessing module is used to perform weldment main body bounding box segmentation and outlier filtering on the acquired point cloud. The weld seam recognition and positioning module uses the trained weld seam recognition model to recognize the preprocessed point cloud data, obtains weld seam marker position data, and positions and aligns the recognition point three-dimensional position coordinates with the welding robot coordinate system according to the welding robot position relationship. The welding process planning and instruction issuing module is used to automatically plan the end-of-arm trajectory based on the recognized weld seam position data, perform virtual simulation verification until a collision-free end-of-arm process execution scheme compilation instruction is generated, and issue the scheme compilation instruction to the welding robot and its control device through a control bus.

16. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program comprises computer program code configured to cause the processor to perform the method of any one of claims 1 to 15. The processor executes the computer program to implement the steps of the ship small group assembly weld seam recognition method of any one of claims 1 to 12.

17. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the ship small group assembly weld seam recognition method of any one of claims 1 to 12.

Images