A method, system, device and storage medium for identifying a small ship weld

Abstract

The present invention discloses a method, system, device and storage medium for recognizing welds of a small group of ships. The method comprises the following steps: establishing a point cloud weld recognition training data set by extending the CAD data of a small group of ships through point cloud discretization, virtual sampling, semi-automatic interactive annotation of weld areas and style migration; training and exporting a weld recognition model data file by using a point cloud weld recognition model; collecting 3D point cloud data of a small group of ship workpieces to be welded by using a 3D camera array in a production environment; obtaining weld mark position data and locating welds by combining the weld recognition model data file; thus issuing welding process planning and instructions to control a welding robot arm and its control device to complete the welding of the small group of workpieces, overcoming the point cloud loss and recognition failure caused by the steel type and the 3D camera shooting angle of the traditional 3D weld recognition method, and greatly improving the adaptability of the automatic weld recognition method to complex workpieces.

Description

Technical Field

[0001] The present application relates to the technical field of ship automated welding, and in particular to a method, system, device and storage medium for identifying a small weld of a ship. Background Art

[0002] Welding is a key supporting technology and process link in modern shipbuilding. It is the smallest unit of the hull and the welding workload accounts for 30% to 40% of the shipbuilding cost. The manual welding method used in traditional shipbuilding technology has safety risks and low efficiency. With the advancement of automation and robotics technology, domestic and foreign shipyards have gradually studied and adopted automated robot welding to replace manual welding.

[0003] At present, with the upgrade of ship configuration and design, the composition of small-scale ship parts has expanded from simple panelization to multi-grid and multi-ribbed plate configurations welded together with irregular profiles such as T-shaped steel and bulb flat steel. The traditional weld recognition method based on online tracking or 3D camera has problems such as the presence of ribs or pre-weld points on the new small-scale configuration, which leads to data collection noise, and the limited shooting angle leads to incomplete data collection. It is only suitable for small-scale workpiece recognition in scenes with regular shapes and unobstructed camera vision. On the other hand, the offline teaching programming automatic welding method based on the design model has comprehensive data and accurate position, but it requires manual editing and adjustment of weld information, which indirectly increases the workload. Therefore, the existing automatic welding methods based on online tracking and offline teaching cannot meet the requirements of automatic welding of small-scale components with complex configurations. Summary of the invention

[0004] The purpose of the present invention is to provide a method, system, device and storage medium for identifying welds of ship sub-assemblies, so as to solve the technical problems of occlusion and missing of original data collection and template matching failure in the existing weld recognition technology for welds of ship sub-assemblies with complex configurations such as T-shaped steel, bulb flat steel, multi-rib plate, multi-grid, etc.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] The present application provides a method for identifying a small-scale weld of a ship, comprising the steps of:

[0007] The ship group CAD data is expanded through point cloud discretization, virtual sampling, semi-automatic interactive annotation of weld areas, and style transfer to establish a point cloud weld recognition training dataset;

[0008] The point cloud weld recognition training data set is used to perform data preprocessing and training model construction to obtain a point cloud weld recognition model;

[0009] Iteratively train and adjust parameters of the point cloud weld recognition model to verify whether the average intersection-over-union ratio and the average accuracy index meet the preset targets. If the preset targets are met, solidify the structure and parameters of the point cloud weld recognition model and export them as a recognition model data file;

[0010] Use 3D point cloud camera array to collect and transmit 3D point cloud data of the ship group workpiece to be welded at the production site;

[0011] Preprocessing the collected 3D point cloud data of the ship panel workpiece, combining the recognition model data file to obtain the weld mark position data and locate the weld;

[0012] The trajectory of the end of the welding robot arm is planned according to the weld mark position data, and the welding process planning and instructions are issued to control the welding robot arm and its control equipment to complete the welding of the small group of workpieces.

[0013] Furthermore, the ship group CAD data is expanded through point cloud discretization, virtual sampling, semi-automatic interactive annotation of weld areas, and style migration to establish a point cloud weld recognition training data set, including:

[0014] A virtual 3D sampling camera is established according to the focal length, horizontal resolution, vertical resolution, optical center position parameters and perspective projection equation of the virtual 3D sampling camera;

[0015] After the virtual 3D sampling camera is established, the center of the three-dimensional bounding box of the ship group CAD model is taken as the origin, and the Fibonacci spherical sampling algorithm is used to generate sampling position points evenly distributed on the hemispherical surface;

[0016] The virtual 3D sampling camera is made to traverse the sampling position points, and at each sampling position point, the optical axis direction of the virtual 3D sampling camera is aligned with the line connecting the sampling position point and the center point of the workpiece, so as to collect a two-dimensional virtual image of the three-dimensional model of the workpiece;

[0017] For the collected two-dimensional virtual sampling weldment image, the point cloud weld automatic annotation function uses the Canny edge detection operator to identify the edge pixels of the sampling image, filter the background edge, and perform Hough line fitting to obtain the 2D pixel position of the candidate weld intersection line in the two-dimensional virtual sampling weldment image;

[0018] According to the perspective projection transformation relationship, the three-dimensional position points of the workpiece model corresponding to the 2D pixel points of the candidate weld intersection extracted from the multi-view image are calculated as candidate marked weld points, and the non-weld intersection points are filtered according to the plane angle constraints on both sides of the marked weld to obtain candidate marked weld points;

[0019] After the candidate marked weld points are colored, a clustering algorithm is used to perform regional clustering, the coordinates and area of ​​the marked weld rectangular bounding box are obtained, and a weld rectangular bounding box representing the weld area is generated, which is overlapped with the small workpiece CAD model in the interface viewport. The user interactively selects the incorrectly identified weld area, and finally retains the correct marked weld area bounding box;

[0020] The 3D small-scale workpiece CAD model is discretized into point cloud data, the point cloud data within the bounding box of the marked weld area is marked as target feature points, and the marked point cloud data is exported and saved.

[0021] Furthermore, after the virtual 3D sampling camera is established, the Fibonacci spherical sampling algorithm is used to generate sampling position points evenly distributed on a hemispherical surface with the center of the three-dimensional bounding box of the ship group CAD model as the origin, including:

[0022] The calculation formula for setting the Fibonacci sphere sampling algorithm is as follows:

[0023] , ;

[0024] in, is the azimuth of the sampling point, is the index of the sampling point, It's the golden ratio.

[0025] Further, the step of calculating, according to the perspective projection transformation relationship, the three-dimensional position point of the workpiece model corresponding to the 2D pixel point of the candidate weld intersection line extracted from the multi-view image as the candidate marked weld point comprises:

[0026] The calculation formula for setting the perspective projection transformation relationship is as follows:

[0027] ;

[0028] Among them, (X, Y, Z) is the 3D point cloud position point, (x, y) is the 2D pixel point position, is the focal length of the virtual 3D sampling camera.

[0029] Furthermore, the preprocessing of the collected 3D point cloud data of the ship panel workpiece, combining the recognition model data file to obtain the weld mark position data and locate the weld includes:

[0030] The collected marked point cloud data is preprocessed by segmenting the weldment body bounding box and filtering outliers;

[0031] The preprocessed marked point cloud data and the recognition model data file are identified using the trained point cloud weld recognition model to obtain the weld mark position data, and the three-dimensional position coordinates of the recognition point are positioned and aligned with the welding robot arm coordinate system according to the position relationship of the welding robot arm.

[0032] Furthermore, the point cloud weld recognition training data set is used to perform data preprocessing and training model construction to obtain a point cloud weld recognition model, including:

[0033] Performing noise filtering on the marked point cloud data;

[0034] The filtered point cloud data is serialized using the three-dimensional Hilbert coding method, and the three-dimensional coordinates of each point in the point cloud are mapped to the index of the Hilbert curve to generate one-dimensional serialized point cloud data.

[0035] Perform feature mosaicking on the serialized point cloud data;

[0036] The point cloud data after feature mosaicking is segmented according to the average grid division method, and the point cloud training data of the segmented grid is pooled;

[0037] The pooled point cloud training data is reordered multiple times;

[0038] Add conditional position encoding to the rearranged point cloud training data, generate position encoding vectors based on the point cloud data and its eight-directional adjacent point cloud category label information, and increase the category association between the training point cloud and the neighborhood;

[0039] A multi-layer convolutional network is constructed for the point cloud training data after adding conditional position encoding. Each layer of the convolutional network contains convolution operation, pooling operation and nonlinear activation function. The point cloud features are convolved layer by layer to obtain a high-dimensional feature vector of the point cloud features. The high-dimensional feature vector is input into the classifier or regressor, and the weld point cloud recognition model parameters are regressed through back propagation. When the loss function value reaches the preset convergence condition, the training is terminated and the point cloud weld recognition model is output.

[0040] Furthermore, the performing noise filtering on the marked point cloud data includes:

[0041] According to the size of the small group of workpieces, a filtering radius r and a minimum number of neighboring points are set, and point cloud data whose number of neighboring points within the filtering radius r is less than a threshold value n is removed to obtain filtered point cloud data;

[0042] Set the filter formula to ;in, is the original point cloud data, is the filtered point cloud data, is a 3D point and The distance between.

[0043] Furthermore, performing three-dimensional Hilbert coding serialization on the filtered point cloud data, mapping the three-dimensional coordinates of each point in the point cloud to the index of the Hilbert curve, and generating one-dimensional serialized point cloud data includes:

[0044] Set the Hilbert serialization formula as follows: ;

[0045] in, Yes The Hilbert index of is the function for calculating the Hilbert index, px is the point The horizontal coordinate of point The vertical coordinate of point The vertical coordinate of .

[0046] Furthermore, the feature mosaicking of the serialized point cloud data includes:

[0047] Embed the coordinates and category color labels (r, g, b) of each point in the serialized point cloud data to generate a high-dimensional feature vector;

[0048] The linear transformation and nonlinear activation function formulas are set as follows: ,in, is the output of the activation function, is the input;

[0049] The embedded features are processed using the linear transformation and the nonlinear activation function, and the features represent the data.

[0050] Furthermore, segmenting the feature-embedded point cloud data according to the average grid division method includes:

[0051] The average grid segmentation formula is set as follows:

[0052] ;

[0053] in, Yes The grid coordinates of , and is the size of the grid, px is the point The horizontal coordinate of point The vertical coordinate of point The vertical coordinate of , and It is the minimum value of the point cloud data in all directions.

[0054] Furthermore, the regressing calculation of the weld point cloud recognition model parameters by back propagation includes:

[0055] Set the back propagation formula to: ;

[0056] Where Δw is the update of the weight, η is the learning rate, L is the loss function, and w is the weight.

[0057] Furthermore, the iterative training and parameter adjustment of the point cloud weld recognition model to verify whether the average intersection-over-union ratio and the average accuracy index meet the preset goals include:

[0058] The calculation formula for setting the average intersection-over-union ratio is: ; ;

[0059] in Represents the weld type index, 1≤ ≤j, For the The true example of the weld type recognition result is that the 3D point to be identified is correctly predicted as The number of pixels of weld-like points; For the The false positive example of the weld type recognition result is that the 3D point to be identified is incorrectly predicted as The number of pixels of weld-like points; For the The false negative example of the weld type recognition result is that the 3D point to be recognized should be Class weld point, but was incorrectly predicted as non- The number of pixels of weld-like points, Obtained for all j-type weld types The mean of .

[0060] The present application also provides a ship group stand-alone weld identification system, which includes a weld identification model training device, a weld identification and welding mechanical arm control device, a 3D point cloud camera array and a welding mechanical arm and its control device;

[0061] The weld recognition model training device is used to establish a point cloud weld recognition training data set for the ship group CAD data through point cloud discretization, semi-automatic interactive annotation of weld areas, and style migration expansion; the point cloud weld recognition training data set is used to perform data preprocessing and training model construction to obtain a point cloud weld recognition model; the point cloud weld recognition model is iteratively trained and parameter adjusted to verify whether the average intersection-over-union ratio and the average accuracy index meet the preset targets. If the preset targets are met, the structure and parameters of the point cloud weld recognition model are solidified and exported as a recognition model data file;

[0062] The 3D point cloud camera array is used to collect and transmit 3D point cloud data of the ship group workpiece to be welded at the production site;

[0063] The weld recognition and welding robot arm control device is used to pre-process the collected 3D point cloud data of the ship group workpiece, obtain the weld mark position data and locate the weld in combination with the recognition model data file; plan the end trajectory of the welding robot arm according to the weld mark position data, and issue welding process planning and instructions;

[0064] The welding robot arm and its control device are used to complete the welding of small groups of workpieces according to the issued welding process plan and instructions.

[0065] Furthermore, the weld recognition model training device includes a CAD assembly model-point cloud conversion module, a weld semi-automatic interactive annotation module, a style transfer-based training set expansion module, and a point cloud weld recognition model training module;

[0066] The CAD assembly model-point cloud conversion module is used to import the three-dimensional CAD design model of the ship assembly sample and discretize the patch model into point cloud format data for training;

[0067] The weld semi-automatic interactive annotation module is used to review and edit the sampling and identification weld feature annotation results;

[0068] The style transfer training set expansion module uses a style transfer network to generate multi-style data fusion for the texture, illumination, and roughness of the sampled 3D model, and maps the weld feature data annotated by the weld semi-automatic interactive annotation module to small group point cloud data with the same configuration but different materials and textures;

[0069] The point cloud weld recognition model training module constructs a small group of independent weld point cloud recognition network models based on a deep neural network, and is used to train weld recognition model parameters on the small group of independent point cloud data after expansion and marking to obtain a point cloud weld recognition model file.

[0070] Furthermore, the weld seam identification and welding robot arm control device includes a workpiece 3D point cloud data acquisition module, a point cloud data preprocessing module, a weld seam identification and positioning module, and a welding process planning and instruction issuing module;

[0071] The 3D point cloud data acquisition module is used to control the 3D point cloud camera array to acquire and transmit point cloud data;

[0072] The point cloud data preprocessing module is used to perform weldment main body bounding box segmentation and outlier filtering on the collected point cloud;

[0073] The weld recognition and positioning module uses the trained weld recognition model to recognize the preprocessed point cloud data, obtains the weld mark position data, and positions and aligns the three-dimensional position coordinates of the recognition point with the welding robot arm coordinate system according to the position relationship of the welding robot arm;

[0074] The welding process planning and instruction issuing module is used to automatically plan the trajectory of the robot arm end according to the identified weld position data, and perform virtual simulation verification until a collision-free robot arm end process execution plan assembly instruction is generated, and the plan assembly instruction is issued to the welding robot arm and its control device through the control bus.

[0075] On the other hand, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the ship subassembly weld identification method described above when executing the computer program.

[0076] On the other hand, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the ship subassembly weld identification method described above are implemented.

[0077] Compared with the prior art, the beneficial effects of the present invention are as follows: a point cloud weld recognition training data set is established by extending the ship sub-assembly CAD data through point cloud discretization, virtual sampling, semi-automatic interactive annotation of weld areas, and style migration; a recognition model data file is exported using a point cloud weld recognition model; 3D point cloud data of the ship sub-assembly workpiece to be welded is collected using a 3D camera array in a production environment; weld mark position data is obtained and the weld is located in combination with the recognition model data file, thereby issuing welding process planning and instructions to control the welding robot arm and its control equipment to complete the welding of the sub-assembly workpiece, providing an automated and intelligent weld recognition and positioning solution for the welding of complex ship sub-assembly parts, so that the weld recognition model completes automatic feature sampling, marking and training only according to the weldment CAD model, overcoming the point cloud missing and recognition failure caused by the steel model and the 3D camera shooting angle of the traditional 3D weld recognition method, and greatly improving the adaptability of the automatic weld recognition method to complex workpieces. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] The drawings in the specification are only used for illustrative purposes and should not be construed as limiting the present application. For those skilled in the art, it is understandable that some well-known structures and their descriptions may be omitted in the drawings.

[0079] Figure 1 It is a structural schematic diagram of the ship group stand-alone weld identification system of the present invention;

[0080] Figure 2 It is a flow chart of the method for identifying a weld of a small ship group according to the present invention;

[0081] Figure 3 A flowchart of the steps of importing a small ship CAD model into a weld recognition model training application program of the present invention;

[0082] Figure 4 A flow chart of a training method for a weld recognition model of the present invention;

[0083] Figure 5 It is a schematic diagram of the structure of the computer device of the present invention.

[0084] The symbols in the figure are as follows: 100, weld recognition model training equipment; 101, CAD assembly model-point cloud conversion module; 102, weld semi-automatic interactive annotation module; 103, training set expansion module based on style transfer; 104, point cloud weld recognition model training module; 200, small group weld point cloud recognition network model; 300, 3D point cloud camera array; 301, 3D point cloud camera; 400, weld recognition and welding robot arm control equipment; 401, workpiece 3D point cloud data acquisition module; 402, point cloud data preprocessing module; 403, weld recognition and positioning module; 404, welding process planning and instruction issuance module; 500, welding robot arm and its control equipment. DETAILED DESCRIPTION

[0085] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0086] like Figure 1 As shown, an embodiment of the present invention provides a ship group independent weld recognition system, which includes a weld recognition model training device 100, a weld recognition and welding robot arm control device 400, a 3D point cloud camera array 300 and a welding robot arm and its control device 500.

[0087] like Figure 1As shown, the weld recognition model training device 100 is loaded with a weld recognition model training application program, which includes a CAD assembly model-point cloud conversion module 101, which is used to import a 3D CAD design model of a small assembly sample of a ship, and discretize the patch model into point cloud format data for training.

[0088] like Figure 1 As shown, the weld recognition model training application includes a weld semi-automatic interactive annotation module 102, which uses a virtual camera to sample and identify weld features and annotate the results for review and editing.

[0089] like Figure 1 As shown, the weld recognition model training application includes a training set expansion module 103 based on style transfer, which uses a style transfer network to generate multi-style data fusion for the texture, lighting, and roughness of the sampled three-dimensional model, and maps the weld feature data annotated by the weld semi-automatic interactive annotation module 102 to more small group point cloud data with the same configuration but different materials and textures.

[0090] like Figure 1 As shown, the weld recognition and welding robot arm control device 400 constructs a small group of independent weld point cloud recognition network models based on a deep neural network, which is used to train weld recognition model parameters on the small group of independent point cloud data after expansion and labeling, thereby obtaining a point cloud weld recognition model file.

[0091] The deep neural network structure used in the small-scale weld point cloud recognition network model can be any mature model or its variants such as Point Transformer, Point Net, Point Net++, etc. in the deep neural network model used for point cloud classification and recognition. The preferred deep neural network is the PTV3 neural network, i.e., Point Transformer V3.

[0092] like Figure 1 As shown, the weld recognition and welding robot arm control device 400 is loaded with an automatic welding recognition and control program.

[0093] The automatic welding recognition and control program includes a workpiece 3D point cloud data acquisition module 401, which is used to control the 3D point cloud camera array 300 to acquire and transmit point cloud data.

[0094] The automatic welding recognition and control program includes a point cloud data preprocessing module 402 for performing weldment main body bounding box segmentation and outlier filtering on the collected point cloud.

[0095] The automatic welding recognition and control program includes a weld recognition and positioning module 403, which uses a trained weld recognition model to recognize the preprocessed point cloud data, obtains weld mark position data, and locates and matches the three-dimensional position coordinates of the recognition point with the robot coordinate system according to the robot position relationship.

[0096] The automatic welding identification and control program includes a welding process planning and instruction issuing module 404, which is used to automatically plan the trajectory of the robot end according to the identified weld position data, and perform virtual simulation verification until a collision-free robot end process execution plan assembly instruction is generated, and the plan assembly instruction is issued to the welding robot arm and its control equipment through the control bus to complete the small group welding.

[0097] like Figure 2 As shown, the embodiment of the present application also provides a method for identifying a small weld of a ship, which specifically includes the following steps:

[0098] S1: Import the ship assembly CAD model into the weld recognition model training application. The program function module uses the CAD assembly model-point cloud conversion function to convert the patch CAD data into labeled point cloud data for weld recognition model training through point cloud discretization, semi-automatic interactive annotation of weld area, and style transfer expansion of weld recognition training set. This step includes the following sub-steps:

[0099] S11: The user imports the three-dimensional CAD model file of the ship group through the function buttons on the interactive interface of the weld recognition model training application program, and adjusts the position of the model file so that the model is located in the center of the interface viewport for easy operation and observation.

[0100] S12: The user executes the point cloud weld automatic labeling function through the function button on the weld recognition model training application interface to obtain the point cloud weld recognition model training data set. After the point cloud weld automatic labeling function is started, the system establishes a virtual sampling camera according to the virtual sampling camera focal length, horizontal resolution, vertical resolution, optical center position parameters and perspective projection equation.

[0101] S13: After the virtual sampling camera is established, the point cloud weld automatic annotation function uses the center of the three-dimensional bounding box of the ship group CAD model as the origin and r as the radius, and uses the Fibonacci spherical sampling algorithm to generate virtual camera sampling position points evenly distributed on the hemisphere. The calculation formula of the Fibonacci spherical sampling algorithm is as follows:

[0102] , ;

[0103] in, is the azimuth of the sampling point, is the index of the sampling point, It's the golden ratio.

[0104] The virtual camera is made to traverse the virtual camera sampling position points, and at each sampling position point, the optical axis direction of the virtual camera is aligned with the line connecting the sampling position point and the center point of the workpiece, so as to collect a two-dimensional virtual image of the three-dimensional model of the workpiece.

[0105] S14: For the collected two-dimensional virtual sampled weld image, the point cloud weld automatic annotation function uses the Canny edge detection operator to perform edge pixel recognition, background edge filtering, and Hough line fitting to obtain the 2D pixel position of the candidate weld intersection line in the two-dimensional virtual sampled weld image. . is the 2D pixel position, W is the width of the two-dimensional virtual sampling weldment image, and H is the length of the two-dimensional virtual sampling weldment image.

[0106] S15: According to the perspective projection transformation relationship, the three-dimensional position point of the workpiece model corresponding to the 2D pixel point of the candidate weld intersection extracted from the multi-view image is calculated as the candidate marked weld point. The calculation formula of the perspective projection transformation relationship is as follows:

[0107] ;

[0108] in, is the 3D point cloud location point, is the 2D pixel position, is the focal length. According to the angle constraint of the planes on both sides of the marked weld, non-weld intersection points are filtered to obtain candidate marked weld points. It can be understood that X is the position point of the 3D point cloud The horizontal coordinate of Y is the position of the 3D point cloud. The vertical coordinate of Z is the position of the 3D point cloud. The coordinate perpendicular to the XY plane, x is the 2D pixel point The horizontal coordinate of y is the 2D pixel point The vertical coordinate of .

[0109] S16: After the candidate marked weld points are colored, Kmeans is used to perform regional clustering to obtain the coordinates and area of ​​the marked weld rectangular bounding box, generate a weld rectangular bounding box representing the weld area, overlap it with the small group of workpiece CAD models in the viewport, and the user interactively selects the incorrectly identified weld area, and finally retains the correct marked weld area bounding box.

[0110] S17: discretize the three-dimensional small workpiece CAD model into point cloud data, mark the point cloud data within the bounding box of the marked weld area as target feature points, and export and save the marked point cloud data.

[0111] S2: Use the point cloud weld recognition model training function of the weld recognition model training application to perform data preprocessing and training model construction on the marked point cloud data obtained in S1 to obtain a weld point cloud recognition training model. This step includes the following sub-steps:

[0112] S21: The program function module performs noise filtering on the marked data. The user sets the filtering radius according to the size of the workpiece. and minimum number of neighbors, removing all nodes within the specified radius The number of inner neighbor points is less than the threshold The point cloud data is obtained by filtering the point cloud data. The filtering formula is .in, is the original point cloud data, is the filtered point cloud data, is a 3D point and The distance between.

[0113] S22: The program function module performs three-dimensional Hilbert coding serialization on the optimized training point cloud data. The three-dimensional coordinates of each point in the point cloud are mapped to the index of the Hilbert curve to generate one-dimensional serialized point cloud data. The Hilbert serialization formula is as follows: in, Yes The Hilbert index of is the function for calculating the Hilbert index, px is the point The horizontal coordinate of point The vertical coordinate of point The vertical coordinate of .

[0114] S23: The program function module performs feature mosaicking on the serialized point cloud data. The coordinates and category colors of each point are marked. Perform embedding processing to generate a high-dimensional feature vector. Use linear transformation and nonlinear activation function to process the embedded features, and the features represent data. The activation function formula is: in, is the output of the activation function, For input.

[0115] S24: The program function module divides the point cloud data after feature mosaicking according to the average grid division method. The average grid division formula is as follows:

[0116] ;

[0117] in, Yes The grid coordinates of , and is the size of the grid, px is the point The horizontal coordinate of point The vertical coordinate of point The vertical coordinate of , and It is the minimum value of the point cloud data in all directions.

[0118] S25: The program function module reorders the pooled training point cloud data multiple times to enhance the robustness of the model.

[0119] S26: The program function module adds conditional position coding to the training point cloud data. The position coding vector is generated according to the point cloud data and its eight-directional adjacent point cloud category labeling information, and the category association between the training point cloud and the neighborhood is increased, thereby improving the model prediction accuracy.

[0120] S27: The program function module constructs a multi-layer convolutional network for the point cloud training data after adding conditional position encoding. Each layer contains convolution operations, pooling operations and nonlinear activation functions. The point cloud features are convoluted layer by layer to obtain a high-dimensional feature vector of the point cloud features. The high-dimensional feature vector is input into the classifier or regressor. The weld point cloud recognition model parameters are regressed through back propagation. When the loss function value reaches the preset convergence condition, the training ends and the recognition model is output. The back propagation formula is: , where Δw is the update of the weight, η is the learning rate, L is the loss function, and w is the weight.

[0121] S3: Iterate the training and parameter adjustment of the point cloud recognition training model constructed in S2 to verify whether the mean intersection over union (mIoU) and mean average precision (mAP) indicators meet the preset goals. If they do, solidify the model structure and parameters and export them as a point cloud weld recognition model file. The calculation formula of mIoU is: , ,in Represents the weld type index, 1≤ j is a positive integer, For the True Positives are the true positives of the weld type recognition results, that is, the 3D point to be identified is correctly predicted as the first The number of pixels of weld-like points; For the False positives of the weld type recognition result, that is, the 3D point to be identified is incorrectly predicted as The number of pixels of weld-like points; For the False negatives of the weld type recognition result, that is, the 3D point to be recognized should be Class weld point, but was incorrectly predicted as non- The number of pixels of weld-like points, Obtained for all j-type weld types The mean of .

[0122] S4: Import the point cloud weld recognition model file obtained in S3 into the automatic welding recognition and control program loaded on the weld recognition and welding robot arm control device 400 in the production environment, and then obtain the point cloud data to be identified through the workpiece 3D point cloud data acquisition function and the point cloud data preprocessing function in turn, and automatically identify the weld position of the small group of workpieces through the point cloud weld recognition model obtained in S3, and control the welding robot arm and its control equipment through the welding process planning and instruction issuing function to complete the welding of the small group of workpieces.

[0123] It is understandable that after completing the welding of the small group workpiece, it also includes:

[0124] S5: Point cloud recognition of welds in production environment.

[0125] The present invention changes the traditional method of identifying and locating welds through point cloud data projection and template matching, and combines automatic 3D virtual sampling, transfer learning, Hilbert sparse coding, and Transformer convolutional recognition technology to provide an automated and intelligent weld identification and positioning solution for the welding of complex ship subassembly parts. The weld recognition model completes automatic feature sampling, marking, and training only based on the weldment CAD model, overcoming the point cloud missing and recognition failure caused by the steel model and the 3D camera shooting angle of the traditional 3D weld recognition method, and greatly improving the adaptability of the automatic weld recognition method to complex workpieces.

[0126] In one embodiment of the present application, a method for identifying a small weld of a ship is provided, comprising the steps of:

[0127] The ship group CAD data is expanded through point cloud discretization, virtual sampling, semi-automatic interactive annotation of weld areas, and style transfer to establish a point cloud weld recognition training dataset;

[0128] The point cloud weld recognition training data set is used to perform data preprocessing and training model construction to obtain a point cloud weld recognition model;

[0129] Iteratively train and adjust parameters of the point cloud weld recognition model to verify whether the average intersection-over-union ratio and the average accuracy index meet the preset targets. If the preset targets are met, solidify the structure and parameters of the point cloud weld recognition model and export them as a recognition model data file;

[0130] Use 3D camera array to collect 3D point cloud data for the small ship workpieces to be welded at the production site;

[0131] Preprocess the 3D point cloud data of the ship group workpiece collected by the 3D camera array at the production site, obtain the weld mark position data and locate the weld in combination with the recognition model data file;

[0132] The trajectory of the end of the welding robot arm is planned according to the weld mark position data, and the welding process planning and instructions are issued to control the welding robot arm and its control equipment to complete the welding of the small group of workpieces.

[0133] Furthermore, the virtual sampling step of using the ship group CAD data to construct a point cloud weld recognition training data set includes:

[0134] A virtual 3D sampling camera is established according to the focal length, horizontal resolution, vertical resolution, optical center position parameters and perspective projection equation of the virtual 3D sampling camera;

[0135] After the virtual 3D sampling camera is established, the center of the three-dimensional bounding box of the ship group CAD model is taken as the origin, and the Fibonacci spherical sampling algorithm is used to generate sampling position points evenly distributed on the hemispherical surface;

[0136] The virtual 3D sampling camera is made to traverse the sampling position points, and at each sampling position point, the optical axis direction of the virtual 3D sampling camera is aligned with the line connecting the sampling position point and the center point of the workpiece, so as to collect a two-dimensional virtual image of the three-dimensional model of the workpiece;

[0137] For the collected two-dimensional virtual sampling weldment image, the point cloud weld automatic annotation function uses the Canny edge detection operator to identify the edge pixels of the sampling image, filter the background edge, and perform Hough line fitting to obtain the 2D pixel position of the candidate weld intersection line in the two-dimensional virtual sampling weldment image;

[0138] According to the perspective projection transformation relationship, the three-dimensional position points of the workpiece model corresponding to the 2D pixel points of the candidate weld intersection extracted from the multi-view image are calculated as candidate marked weld points, and the non-weld intersection points are filtered according to the plane angle constraints on both sides of the marked weld to obtain candidate marked weld points;

[0139] After the candidate marked weld points are colored, a clustering algorithm is used to perform regional clustering, the coordinates and area of ​​the marked weld rectangular bounding box are obtained, and a weld rectangular bounding box representing the weld area is generated, which is overlapped with the small workpiece CAD model in the interface viewport. The user interactively selects the incorrectly identified weld area, and finally retains the correct marked weld area bounding box;

[0140] The 3D small-scale workpiece CAD model is discretized into point cloud data, the point cloud data within the bounding box of the marked weld area is marked as target feature points, and the marked point cloud data is exported and saved.

[0141] Furthermore, after the virtual 3D sampling camera is established, the Fibonacci spherical sampling algorithm is used to generate sampling position points evenly distributed on a hemispherical surface with the center of the three-dimensional bounding box of the ship group CAD model as the origin, including:

[0142] The calculation formula for setting the Fibonacci sphere sampling algorithm is as follows:

[0143] , ;

[0144] in, is the azimuth of the sampling point, is the index of the sampling point, It's the golden ratio.

[0145] Further, the step of calculating, according to the perspective projection transformation relationship, the three-dimensional position point of the workpiece model corresponding to the 2D pixel point of the candidate weld intersection line extracted from the multi-view image as the candidate marked weld point comprises:

[0146] The calculation formula for setting the perspective projection transformation relationship is as follows:

[0147] ;

[0148] in, is the 3D point cloud location point, is the 2D pixel position, is the focal length of the virtual 3D sampling camera.

[0149] Furthermore, the preprocessing of the collected 3D point cloud data of the ship panel workpiece, combining the recognition model data file to obtain the weld mark position data and locate the weld includes:

[0150] The collected marked point cloud data is preprocessed by segmenting the weldment body bounding box and filtering outliers;

[0151] The preprocessed marked point cloud data and the recognition model data file are identified using the trained point cloud weld recognition model to obtain the weld mark position data, and the three-dimensional position coordinates of the recognition point are positioned and aligned with the welding robot arm coordinate system according to the position relationship of the welding robot arm.

[0152] Furthermore, the point cloud weld recognition training data set is used to perform data preprocessing and training model construction to obtain a point cloud weld recognition model, including:

[0153] Performing noise filtering on the marked point cloud data;

[0154] The filtered point cloud data is serialized using the three-dimensional Hilbert coding method, and the three-dimensional coordinates of each point in the point cloud are mapped to the index of the Hilbert curve to generate one-dimensional serialized point cloud data.

[0155] Perform feature mosaicking on the serialized point cloud data;

[0156] The point cloud data after feature mosaicking is segmented according to the average grid division method, and the point cloud training data of the segmented grid is pooled;

[0157] The pooled point cloud training data is reordered multiple times;

[0158] Add conditional position encoding to the rearranged point cloud training data, generate position encoding vectors based on the point cloud data and its eight-directional adjacent point cloud category label information, and increase the category association between the training point cloud and the neighborhood;

[0159] A multi-layer convolutional network is constructed for the point cloud training data after adding conditional position encoding. Each layer of the convolutional network contains convolution operation, pooling operation and nonlinear activation function. The point cloud features are convolved layer by layer to obtain a high-dimensional feature vector of the point cloud features. The high-dimensional feature vector is input into the classifier or regressor, and the weld point cloud recognition model parameters are regressed through back propagation. When the loss function value reaches the preset convergence condition, the training is terminated and the point cloud weld recognition model is output.

[0160] Furthermore, the performing noise filtering on the marked point cloud data includes:

[0161] According to the size of the small group of workpieces, a filtering radius r and a minimum number of neighboring points are set, and point cloud data whose number of neighboring points within the filtering radius r is less than a threshold value n is removed to obtain filtered point cloud data;

[0162] Set the filter formula to ;in, is the original point cloud data, is the filtered point cloud data, is a 3D point and The distance between.

[0163] Furthermore, performing three-dimensional Hilbert coding serialization on the filtered point cloud data, mapping the three-dimensional coordinates of each point in the point cloud to the index of the Hilbert curve, and generating one-dimensional serialized point cloud data includes:

[0164] Set the Hilbert serialization formula as follows: ;

[0165] in, Yes The Hilbert index of is the function for calculating the Hilbert index, px is the point The horizontal coordinate of point The vertical coordinate of point The vertical coordinate of .

[0166] Furthermore, the feature mosaicking of the serialized point cloud data includes:

[0167] Embed the coordinates and category color labels (r, g, b) of each point in the serialized point cloud data to generate a high-dimensional feature vector;

[0168] The linear transformation and nonlinear activation function formulas are set as follows: ,in, is the output of the activation function, is the input;

[0169] The embedded features are processed using the linear transformation and the nonlinear activation function, and the features represent the data.

[0170] Furthermore, segmenting the feature-embedded point cloud data according to the average grid division method includes:

[0171] The average grid segmentation formula is set as follows:

[0172] ;

[0173] in, Yes The grid coordinates of , and is the size of the grid, px is the point The horizontal coordinate of point The vertical coordinate of point The vertical coordinate of , and It is the minimum value of the point cloud data in all directions.

[0174] Furthermore, the regressing calculation of the weld point cloud recognition model parameters by back propagation includes:

[0175] Set the back propagation formula to: ;

[0176] Where Δw is the update of the weight, η is the learning rate, L is the loss function, and w is the weight.

[0177] Furthermore, the iterative training and parameter adjustment of the point cloud weld recognition model to verify whether the average intersection-over-union ratio and the average accuracy index meet the preset goals include:

[0178] The calculation formula for setting the average intersection-over-union ratio is: ; ;

[0179] in Represents the weld type index, 1≤ ≤j, For the The true example of the weld type recognition result is that the 3D point to be identified is correctly predicted as The number of pixels of weld-like points; For the The false positive example of the weld type recognition result is that the 3D point to be identified is incorrectly predicted as The number of pixels of weld-like points; For the The false negative example of the weld type recognition result is that the 3D point to be recognized should be Class weld point, but was incorrectly predicted as non- The number of pixels of weld-like points, Obtained for all j-type weld types The mean of .

[0180] like Figure 1 As shown, the embodiment of the present application provides a ship group stand-alone weld recognition system, which includes a weld recognition model training device 100, a weld recognition and welding robot arm control device 400, a 3D point cloud camera array and a welding robot arm and its control device 500;

[0181] The weld recognition model training device 100 is used to establish a point cloud weld recognition training data set for the ship group CAD data through point cloud discretization, semi-automatic interactive annotation of weld areas, and style migration expansion; the point cloud weld recognition training data set is used for data preprocessing and training model construction to obtain a point cloud weld recognition model; the point cloud weld recognition model is iteratively trained and parameter adjusted to verify whether the average intersection-over-union ratio and the average accuracy index meet the preset targets. If the preset targets are met, the structure and parameters of the point cloud weld recognition model are solidified and exported as a recognition model data file;

[0182] The 3D point cloud camera array 300 is used to collect and transmit 3D point cloud data of the ship group workpiece to be welded at the production site;

[0183] The weld recognition and welding robot arm control device 400 is used to pre-process the collected 3D point cloud data of the ship group workpiece, obtain the weld mark position data and locate the weld in combination with the recognition model data file; plan the end trajectory of the welding robot arm according to the weld mark position data, and issue welding process planning and instructions;

[0184] The welding robot arm and its control device 500 are used to complete the welding of small groups of workpieces according to the issued welding process plan and instructions.

[0185] Furthermore, the weld recognition model training device 100 includes a CAD assembly model-point cloud conversion module 101, a weld semi-automatic interactive annotation module 102, a style transfer-based training set expansion module 103, and a point cloud weld recognition model training module 104;

[0186] The CAD assembly model-point cloud conversion module 101 is used to import the three-dimensional CAD design model of the ship assembly sample and discretize the patch model into point cloud format data for training;

[0187] The weld semi-automatic interactive annotation module 102 is used to review and edit the sampling and identification weld feature annotation results;

[0188] The style transfer-based training set expansion module 103 uses a style transfer network to generate multi-style data fusion of the texture, illumination, and roughness of the sampled three-dimensional model, and maps the weld feature data annotated by the weld semi-automatic interactive annotation module 102 to a small group of point cloud data with the same configuration but different materials and textures;

[0189] The point cloud weld recognition model training module 104 constructs a small group weld point cloud recognition network model 200 based on a deep neural network, which is used to train weld recognition model parameters on the small group point cloud data after expansion and labeling to obtain a point cloud weld recognition model file.

[0190] The deep neural network structure used in the small-scale weld point cloud recognition network model can be any mature model or its variants such as Point Transformer, Point Net, Point Net++, etc. in the deep neural network model used for point cloud classification and recognition. The preferred deep neural network is the PTV3 neural network, i.e., Point Transformer V3.

[0191] Furthermore, the weld seam identification and welding robot arm control device 400 includes a workpiece 3D point cloud data acquisition module 401, a point cloud data preprocessing module 402, a weld seam identification and positioning module 403, and a welding process planning and instruction issuing module 404;

[0192] The workpiece 3D point cloud data acquisition module 401 is used to control the 3D point cloud camera array 300 to acquire and transmit point cloud data;

[0193] The point cloud data preprocessing module 402 is used to perform weldment main body bounding box segmentation and outlier filtering on the collected point cloud;

[0194] The weld recognition and positioning module 403 uses the trained weld recognition model to recognize the preprocessed point cloud data, obtains the weld mark position data, and positions and aligns the three-dimensional position coordinates of the recognition point with the welding robot arm coordinate system according to the position relationship of the welding robot arm;

[0195] The welding process planning and instruction issuing module 404 is used to automatically plan the trajectory of the robot end according to the identified weld position data, and perform virtual simulation verification until a collision-free robot end process execution plan assembly instruction is generated, and the plan assembly instruction is issued to the welding robot arm and its control device 500 through the control bus.

[0196] In this embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 5 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store ship sub-unit weld identification data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a ship sub-unit weld identification method is implemented.

[0197] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the ship subassembly weld identification method described above are implemented.

[0198] For the specific limitations on the steps implemented when the processor executes the computer program, please refer to the limitations on the method for identifying welds of a small ship group mentioned above, which will not be repeated here.

[0199] In this embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the ship subassembly weld identification method described above are implemented.

[0200] For the specific limitations on the steps implemented when the computer program is executed by the processor, please refer to the limitations on the method for identifying welds of a small ship group mentioned above, which will not be repeated here.

[0201] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0202] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0203] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for identifying a small weld of a ship, characterized in that: Includes steps: The ship group CAD data is expanded through point cloud discretization, virtual sampling, semi-automatic interactive annotation of weld areas, and style transfer to establish a point cloud weld recognition training dataset; The point cloud weld recognition training data set is used to perform data preprocessing and training model construction to obtain a point cloud weld recognition model; Iteratively train and adjust parameters of the point cloud weld recognition model to verify whether the average intersection-over-union ratio and the average accuracy index meet the preset targets. If the preset targets are met, solidify the structure and parameters of the point cloud weld recognition model and export them as a recognition model data file; Use 3D point cloud camera array to collect and transmit 3D point cloud data of the ship group workpiece to be welded at the production site; Preprocessing the collected 3D point cloud data of the ship panel workpiece, combining the recognition model data file to obtain the weld mark position data and locate the weld; According to the weld mark position data, the end trajectory of the welding robot arm is planned, and the welding process planning and instructions are issued to control the welding robot arm and its control equipment to complete the welding of the small group of workpieces; The method of establishing a point cloud weld recognition training data set by extending the ship group CAD data through point cloud discretization, virtual sampling, semi-automatic interactive annotation of weld areas, and style migration includes: A virtual 3D sampling camera is established according to the focal length, horizontal resolution, vertical resolution, optical center position parameters and perspective projection equation of the virtual 3D sampling camera; After the virtual 3D sampling camera is established, the center of the three-dimensional bounding box of the ship group CAD model is taken as the origin, and the Fibonacci spherical sampling algorithm is used to generate sampling position points evenly distributed on the hemispherical surface; The virtual 3D sampling camera is made to traverse the sampling position points, and at each sampling position point, the optical axis direction of the virtual 3D sampling camera is aligned with the line connecting the sampling position point and the center point of the workpiece, so as to collect a two-dimensional virtual image of the three-dimensional model of the workpiece; For the collected two-dimensional virtual sampling weldment image, the point cloud weld automatic annotation function uses the Canny edge detection operator to identify the edge pixels of the sampling image, filter the background edge, and perform Hough line fitting to obtain the 2D pixel position of the candidate weld intersection line in the two-dimensional virtual sampling weldment image; According to the perspective projection transformation relationship, the three-dimensional position points of the workpiece model corresponding to the 2D pixel points of the candidate weld intersection extracted from the multi-view image are calculated as candidate marked weld points, and the non-weld intersection points are filtered according to the plane angle constraints on both sides of the marked weld to obtain candidate marked weld points; After the candidate marked weld points are colored, a clustering algorithm is used to perform regional clustering, the coordinates and area of ​​the marked weld rectangular bounding box are obtained, and a weld rectangular bounding box representing the weld area is generated, which is overlapped with the small workpiece CAD model in the interface viewport. The user interactively selects the incorrectly identified weld area, and finally retains the correct marked weld area bounding box; Discretize the 3D small workpiece CAD model into point cloud data, mark the point cloud data within the bounding box of the marked weld area as target feature points, and export and save the marked point cloud data; After the virtual 3D sampling camera is established, the center of the three-dimensional bounding box of the ship group CAD model is taken as the origin, and the Fibonacci spherical sampling algorithm is used to generate sampling position points evenly distributed on the hemispherical surface, including: The calculation formula for setting the Fibonacci sphere sampling algorithm is as follows: Where θ is the azimuth of the sampling point, i is the index of the sampling point, and φ is the golden ratio; The step of calculating, according to the perspective projection transformation relationship, the three-dimensional position point of the workpiece model corresponding to the 2D pixel point of the candidate weld intersection line extracted from the multi-view image as the candidate marked weld point comprises: The calculation formula for setting the perspective projection transformation relationship is as follows: Wherein, (X, Y, Z) is the 3D point cloud position point, (x, y) is the 2D image pixel point position, and f is the focal length of the virtual 3D sampling camera.

2. The method for identifying a small ship weld according to claim 1, characterized in that: The preprocessing of the collected 3D point cloud data of the ship panel workpiece, combining the recognition model data file to obtain the weld mark position data and locate the weld includes: The collected marked point cloud data is preprocessed by segmenting the weldment body bounding box and filtering outliers; The preprocessed marked point cloud data and the recognition model data file are identified using the trained point cloud weld recognition model to obtain the weld mark position data, and the three-dimensional position coordinates of the recognition point are positioned and aligned with the welding robot arm coordinate system according to the position relationship of the welding robot arm.

3. The method for identifying a small ship weld according to claim 1, characterized in that: The point cloud weld recognition training data set is used to perform data preprocessing and training model construction to obtain a point cloud weld recognition model, which includes: Performing noise filtering on the marked point cloud data; The filtered point cloud data is serialized using the three-dimensional Hilbert coding method, and the three-dimensional coordinates of each point in the point cloud are mapped to the index of the Hilbert curve to generate one-dimensional serialized point cloud data. Perform feature mosaicking on the serialized point cloud data; The point cloud data after feature mosaicking is segmented according to the average grid division method, and the point cloud training data of the segmented grid is pooled; The pooled point cloud training data is reordered multiple times; Add conditional position encoding to the rearranged point cloud training data, generate position encoding vectors based on the point cloud data and its eight-directional adjacent point cloud category label information, and increase the category association between the training point cloud and the neighborhood; A multi-layer convolutional network is constructed for the point cloud training data after adding conditional position encoding. Each layer of the convolutional network contains convolution operation, pooling operation and nonlinear activation function. The point cloud features are convolved layer by layer to obtain a high-dimensional feature vector of the point cloud features. The high-dimensional feature vector is input into the classifier or regressor, and the weld point cloud recognition model parameters are regressed through back propagation. When the loss function value reaches the preset convergence condition, the training is terminated and the point cloud weld recognition model is output.

4. The method for identifying a small ship weld according to claim 3, characterized in that: The performing noise filtering on the marked point cloud data comprises: According to the size of the small group of workpieces, a filtering radius r and a minimum number of neighboring points are set, and point cloud data whose number of neighboring points within the filtering radius r is less than a threshold value n is removed to obtain filtered point cloud data; Set the filtering formula to P′={p∈P|count(q∈P,d(p,q)<r)≥n}; where P is the original point cloud data, P′ is the filtered point cloud data, and d(p,q) is the distance between 3D points p and q.

5. The method for identifying a small ship weld according to claim 3, characterized in that: The step of performing three-dimensional Hilbert coding serialization on the filtered point cloud data, mapping the three-dimensional coordinates of each point in the point cloud to the index of the Hilbert curve, and generating one-dimensional serialized point cloud data includes: Set the Hilbert serialization formula as follows: H(p) = hilbert_index(px,py,pz); Among them, H(p) is the Hilbert index of point p, hilbert_index(px,py,pz) is the function for calculating the Hilbert index, px is the horizontal coordinate of point p, py is the longitudinal coordinate of point p, and pz is the vertical coordinate of point p.

6. The method for identifying a small ship weld according to claim 3, characterized in that: The feature mosaicking of the serialized point cloud data includes: Embed the coordinates and category color labels (r, g, b) of each point in the serialized point cloud data to generate a high-dimensional feature vector; The linear transformation and nonlinear activation function formulas are set as follows: Where f(x') is the output of the activation function and x' is the input; The embedded features are processed using the linear transformation and the nonlinear activation function, and the features represent the data.

7. The method for identifying a small ship weld according to claim 3, characterized in that: The step of segmenting the feature mosaicked point cloud data according to the average grid division method comprises: The average grid segmentation formula is set as follows: Where G(p) is the grid coordinate of point p, Δx, Δy and Δz are the sizes of the grid, px is the horizontal coordinate of point p, py is the vertical coordinate of point p, and pz is the vertical coordinate of point p. and It is the minimum value of the point cloud data in all directions.

8. The method for identifying a small-scale weld of a ship according to claim 3, characterized in that: The regressive calculation of the weld point cloud recognition model parameters by back propagation includes: Set the back propagation formula to: Where Δw is the update of the weight, η is the learning rate, L is the loss function, and w is the weight.

9. The method for identifying a small ship weld according to claim 1, characterized in that: The iterative training and parameter adjustment of the point cloud weld recognition model to verify whether the average intersection-over-union ratio and the average accuracy index meet the preset goals include: The calculation formula for setting the average intersection-over-union ratio is: Where k represents the weld type index, 1≤k≤j, TP k is the true example of the k-th weld type recognition result, that is, the number of pixels of the 3D point to be recognized that is correctly predicted to be the k-th weld point; FP k is the false positive example of the k-th weld type recognition result, that is, the number of pixels of the 3D point to be recognized that is incorrectly predicted as the k-th weld point; FN k is the false negative example of the k-th weld type recognition result, that is, the number of pixels whose 3D points to be recognized should be the k-th weld points but are mistakenly predicted as non-k-th weld points, and mIoU is the IoU obtained for all j-th weld types. k The mean of .

10. A ship subassembly weld identification system for implementing the ship subassembly weld identification method according to any one of claims 1 to 9, characterized in that: Including weld recognition model training equipment, weld recognition and welding robot arm control equipment, 3D point cloud camera array and welding robot arm and its control equipment; The weld recognition model training device is used to establish a point cloud weld recognition training data set by expanding the ship group CAD data through point cloud discretization, virtual sampling, semi-automatic interactive annotation of weld areas, and style migration; The point cloud weld recognition training data set is used to perform data preprocessing and training model construction to obtain a point cloud weld recognition model; Iteratively train and adjust parameters of the point cloud weld recognition model to verify whether the average intersection-over-union ratio and the average accuracy index meet the preset targets. If the preset targets are met, solidify the structure and parameters of the point cloud weld recognition model and export them as a recognition model data file; The 3D point cloud camera array is used to collect and transmit 3D point cloud data of the ship group workpiece to be welded at the production site; The weld recognition and welding robot arm control device is used to pre-process the collected 3D point cloud data of the ship group workpiece, obtain the weld mark position data and locate the weld in combination with the recognition model data file; plan the end trajectory of the welding robot arm according to the weld mark position data, and issue welding process planning and instructions; The welding robot arm and its control device are used to complete the welding of small groups of workpieces according to the issued welding process plan and instructions.

11. The ship group weld identification system according to claim 10, characterized in that: The weld recognition model training device includes a CAD assembly model-point cloud conversion module, a weld semi-automatic interactive annotation module, a style transfer-based training set expansion module, and a point cloud weld recognition model training module; The CAD assembly model-point cloud conversion module is used to import the three-dimensional CAD design model of the ship assembly sample and discretize the patch model into point cloud format data for training; The weld semi-automatic interactive annotation module is used to review and edit the sampling and identification weld feature annotation results; The style transfer training set expansion module uses a style transfer network to generate multi-style data fusion for the texture, illumination, and roughness of the sampled 3D model, and maps the weld feature data annotated by the weld semi-automatic interactive annotation module to small group point cloud data with the same configuration but different materials and textures; The point cloud weld recognition model training module constructs a small group of independent weld point cloud recognition network models based on a deep neural network, and is used to train weld recognition model parameters on the small group of independent point cloud data after expansion and marking to obtain a point cloud weld recognition model file.

12. The ship group weld identification system according to claim 10, characterized in that: The weld seam identification and welding robot arm control device includes a workpiece 3D point cloud data acquisition module, a point cloud data preprocessing module, a weld seam identification and positioning module, and a welding process planning and instruction issuing module; The workpiece 3D point cloud data acquisition module is used to control the 3D point cloud camera array to acquire and transmit point cloud data; The point cloud data preprocessing module is used to perform weldment main body bounding box segmentation and outlier filtering on the collected point cloud; The weld recognition and positioning module uses the trained weld recognition model to recognize the preprocessed point cloud data, obtains the weld mark position data, and positions and aligns the three-dimensional position coordinates of the recognition point with the welding robot arm coordinate system according to the position relationship of the welding robot arm; The welding process planning and instruction issuing module is used to automatically plan the trajectory of the robot arm end according to the identified weld position data, and perform virtual simulation verification until a collision-free robot arm end process execution plan assembly instruction is generated, and the plan assembly instruction is issued to the welding robot arm and its control device through the control bus.

13. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the ship subassembly weld identification method according to any one of claims 1 to 9 are implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the ship subassembly weld identification method according to any one of claims 1 to 9 are implemented.

Images