A cladding pool morphology recognition and closed-loop control method based on instance segmentation

Through example segmentation technology, the morphology and size of the cladding pool during arc welding additive manufacturing process is solved, and the problem of low monitoring efficiency in the existing technology is achieved, high-precision, low-cost detection and closed-loop control are achieved, and arc welding is used to achieve arc welding. Additive manufacturing provides reliable real-time evaluation.

CN112233130BActive Publication Date: 2024-11-01NANJING NORMAL UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011131085.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-10-21
Publication Date
2024-11-01
Estimated Expiration
2040-10-21

AI Technical Summary

Technical Problem

The prior art has low monitoring efficiency of the morphology and size of the cladding pool during arc welding additive manufacturing, making it difficult to achieve real-time and high-precision detection, affecting the quality of the finished product.

Method used

Using the cladding pool morphology recognition method based on instance segmentation, color images are collected through high-definition cameras, and instance segmentation of cladding pools is used to monitor the morphology and size of cladding pools in real time to realize closed-loop control. .

Benefits of technology

It realizes efficient and low-cost morphology detection of cladding pools, overcomes the problems of low manual monitoring efficiency and low applicability of multi-empirical parameters, provides a real-time evaluation basis for arc welding additive manufacturing, and ensures the stability of printing quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112233130B_ABST
    Figure CN112233130B_ABST
Patent Text Reader

Abstract

The present invention discloses a cladding pool morphology recognition and closed-loop control method based on instance segmentation, including the following steps: initializing arc welding additive manufacturing output power and output power variation; collecting arc welding additive manufacturing cladding pool color image; updating arc welding additive manufacturing output power; segmenting the cladding pool in the cladding pool color image, and generating the instance mask of the cladding pool and the pixel coordinates of the candidate frame at the same time, completing the instance segmentation of the cladding pool; using the pixel coordinates of the candidate frame to represent the major axis and the number of pixels of the weld width, calculating the number of pixels of the weld width, and feeding back the generated output power variation; judging whether the current cladding pool color image is the last frame. The method of the present invention has good real-time performance and high recognition accuracy. When there is a molten drop covering the edge of the cladding pool, accurate detection and recognition can be achieved, which verifies the effectiveness and robustness of the algorithm, realizes low-cost and high-efficiency detection of the cladding pool morphology, and provides a reliable basis for real-time evaluation of arc welding additive manufacturing.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of arc welding additive manufacturing and machine vision, and in particular relates to a cladding pool morphology recognition and closed-loop control method based on instance segmentation. Background Art

[0002] Arc welding additive manufacturing uses electric arc as heat source and metal welding wire as material. It converts the target's 3D CAD model into an STL model, slices the model in layers according to a certain thickness and direction, and plans the path of the generated contour. The material is filled in a layer-by-layer accumulation manner, and finally a solid workpiece is quickly generated. In the process of layer-by-layer accumulation, the cladding pool is the smallest forming unit of the target part. Changes in the external environment or process parameters will affect its shape and size, and the degree of fluctuation directly affects the quality of the target part. Therefore, it is necessary to monitor the morphology of the cladding pool online to evaluate the forming quality and provide a theoretical basis for the adjustment of subsequent process parameters.

[0003] In the process of arc welding additive manufacturing, there are many factors that affect the morphology and size of the cladding pool, such as laser power, wire feeding speed, printing speed and changes in the external environment. These changing factors can be uniformly reflected in the changes in the morphology and size of the cladding pool. Therefore, monitoring the morphology and size of the cladding pool is crucial for arc welding additive manufacturing to print high-quality engineering parts.

[0004] At present, the analysis and identification of the morphology and size of the cladding pool in the arc welding additive manufacturing process are carried out by experienced workers to judge the quality of the single-pass cladding layer printing, but this method requires workers to have rich experience and has low efficiency; some use multiple empirical parameters to perform edge detection, but the validity of the empirical parameters is difficult to verify and the applicability is low; others use the phase consistency model for edge detection, but the algorithm has large computational complexity and low efficiency, making it difficult to apply in actual engineering. Summary of the invention

[0005] Purpose of the invention: In order to overcome the shortcomings of the prior art, a method for cladding pool morphology recognition and closed-loop control based on instance segmentation is provided. The method has good real-time performance and high recognition accuracy. When there are molten droplets covering the edge of the cladding pool, accurate detection and recognition can be achieved. The effectiveness and robustness of the algorithm are verified, and low-cost and high-efficiency detection of cladding pool morphology is achieved, providing a basis for real-time evaluation of arc welding additive manufacturing.

[0006] Technical solution: To achieve the above purpose, the present invention provides a cladding pool morphology recognition and closed-loop control method based on instance segmentation, comprising the following steps:

[0007] S1: Initialize the output power and output power change of arc welding additive manufacturing;

[0008] S2: Collecting color images of arc welding additive manufacturing cladding pool;

[0009] S3: Update arc welding additive manufacturing output power;

[0010] S4: Segment the cladding pool in the cladding pool color image, and generate the instance mask of the cladding pool and the pixel coordinates of the candidate box at the same time, completing the instance segmentation of the cladding pool;

[0011] S5: using the pixel coordinates of the candidate frame to represent the size of the major axis and the number of weld width pixels, calculating the change in the number of weld width pixels, and feeding the generated output power change back to step S3;

[0012] S6: Determine whether the current cladding pool color image is the last frame, if yes, end printing, if not, return to step S2.

[0013] Furthermore, the method for collecting the color image of the arc welding additive manufacturing cladding pool in step S2 is: building a cladding pool image collection system, using a high-definition camera and a filter to collect the color image of the cladding pool during the arc welding additive manufacturing process.

[0014] Furthermore, the method for updating the arc welding additive manufacturing output power in step S3 is: updating the arc welding additive manufacturing output power P according to the output power change ΔP obtained in step S5, and the updating formula is P=P+ΔP.

[0015] Furthermore, the specific process of step S4 is as follows:

[0016] S4.1: Input the color image of the cladding pool obtained in step S2 into the backbone neural network for feature extraction and generate feature maps of different sizes, fuse the feature maps from top to bottom to generate a feature pyramid network;

[0017] S4.2: Input the feature pyramid network obtained in step S4.1 into the region proposal network, traverse all pixels in the feature map using a sliding window method, and generate candidate boxes for the areas where the cladding pool exists;

[0018] S4.3: The candidate boxes obtained in step S4.2 are unified in size through the R0IAlign layer;

[0019] S4.4: Perform a full connection operation on each candidate frame in step S4.3, regress the position of the candidate frame, determine whether the object in the candidate frame is a cladding pool, and obtain the pixel coordinates of the candidate frame;

[0020] S4.5: Generate a coarse prediction mask for each candidate box obtained in step S4.2 through a lightweight prediction head;

[0021] S4.6: Select a part of points from the feature pyramid network obtained in step S4.1 for independent prediction and input them into the multi-layer perceptron together with the rough prediction mask generated in step S4.5 to generate a mask with finer and smoother edges for the cladding pool, and complete the instance segmentation of the cladding pool image.

[0022] Furthermore, in step S4.1, the backbone neural network includes a residual network ResNet50 and a feature pyramid network;

[0023] The residual network ResNet50 has 16 residual blocks, each residual unit has three layers, and the entire residual network ResNet50 is divided into five stages, where the layer that does not change the size of the feature map is a stage. The feature extracted each time is the output of the last layer of each stage. Each residual unit can be expressed as: y i =f(h(y i-1 )+F(y i-1 ));

[0024] Among them, y i is the output of the i-th residual unit, y i-1 is the output of the i-1th residual unit, f() is the activation function ReLU, h(y i-1 ) is the identity mapping of the i-1th residual unit, F(y i-1 ) is y i-1 The residual mapping of

[0025] The feature pyramid network fuses the feature maps of the five stages output by the entire residual network ResNet50 from top to bottom. The feature maps of each stage output by the residual network ResNet50 undergo convolution and upsampling operations, and then are fused with the feature maps of other stages.

[0026] Furthermore, the step S4.2 specifically includes the following steps:

[0027] S4.2.1: For the fused feature map, a sliding window method is used to traverse all pixels in different proportions on feature maps of different sizes to generate candidate boxes ROI. The sizes of the candidate boxes are 64*64, 128*128, and 256*256, respectively. The ratios of the length and width of the candidate boxes are 2:1, 1:1, and 1:2, respectively. Each pixel will generate 9 candidate boxes of different sizes. There will be nearly 200,000 overlapping feature boxes on the entire image. Then, the candidate boxes are classified and regressed to obtain the scores and probability values ​​of the objects in the candidate boxes being foreground or background, as well as the coordinate values ​​of each candidate box.

[0028] S4.2.2: Filter and retain candidate boxes according to foreground scores, retain the top 6,000 with the highest scores, and then use non-maximum suppression (NMS algorithm) to compare the candidate box with the highest foreground score with other overlapping candidate boxes. If the overlap is greater than the set value, it is discarded, otherwise it is retained, and the best bounding box is screened out to remove redundancy.

[0029] Furthermore, the step S4.3 specifically includes the following steps:

[0030] S4.3.1: Traverse all candidate boxes and map them to corresponding positions on the feature map;

[0031] S4.3.2: Divide each candidate box into a number of units, and each unit is divided into a number of small areas. For each small area, the center pixel value is calculated using the bilinear interpolation method.

[0032] Among them, bilinear interpolation is to redistribute the pixel information of the original image, so as to lose as little pixel information as possible when the image is enlarged or reduced. The coordinate values ​​of the four vertices in each small area are A(x1,y1), B(x1,y2), C(x2,y2) and D(x2,y1), and the pixel values ​​are f(A), f(B), f(C) and f(D), respectively. Moreover, x2>x1, y2>y1, and the coordinates of the center point of this small area are G(x,y). The bilinear interpolation algorithm is as follows:

[0033] A1: Interpolate in the horizontal direction to obtain the pixel values ​​f(E) and f(F) at points E and F;

[0034]

[0035] A2: interpolate points E and F on the ordinate axis to obtain the pixel value f(G) of the pixel point G in the small area;

[0036]

[0037] S4.3.3: Perform a maximum pooling operation on the small area of ​​each unit and retain the maximum pixel value, that is, the pixel value of the center point of each small area.

[0038] Furthermore, the specific process of step S4.4 is as follows:

[0039] After the ROIAlign operation, the sizes of different candidate boxes converted to the feature map have been unified to 7*7, and the Softmax function is input after the full connection operation to map the feature map to a fixed-length feature vector, regress the position of the candidate box, classify the objects in the candidate box, and obtain the confidence probability value belonging to the cladding pool, which is used to determine whether the candidate box is a cladding pool and obtain the candidate box pixel coordinates (M, N, W, H), where M represents the horizontal distance from the upper left corner of the candidate box to the upper left corner of the input image, N represents the vertical distance from the upper left corner of the candidate box to the upper left corner of the input image, W represents the pixel length value of the candidate box, and H represents the pixel width value of the candidate box;

[0040] The Softmax function maps the output of the previous layer to a real number between 0 and 1, and the sum of the mappings is 1. Its mathematical form is as follows:

[0041]

[0042] Among them, Z j It is the ratio of the index value of the element to the sum of the index values ​​of all elements, L j is the output of the fully connected network in the previous layer, S is the total number of categories, and the value output by the fully connected network is converted into a probability value through the Softmax function to determine whether the object in the candidate box belongs to the cladding pool category;

[0043] Using the cross entropy function as the loss function of Softmax, the mathematical form is:

[0044]

[0045] Among them, i is the serial number of the element, which can be 1, 2, 3, etc., and is t i is the true value, Z j It is the ratio of the exponential value of the element to the sum of the exponential values ​​of all elements, which is also the output value of the Softmax function. For an input, the smaller the loss of the correct category prediction, the greater the loss of the wrong category prediction.

[0046] Furthermore, the specific process of step S4.5 is as follows: for all candidate boxes, a 14*14 feature map is extracted from the second layer of the feature pyramid network using bilinear interpolation, quantized by ROI-Pooling, connected to ReLU after a 2*2*256 convolution operation, the spatial size is reduced to a specified size of 7*7, and finally a 7*7 coarse prediction mask is generated for each category in the candidate box by a multi-layer perceptron;

[0047] The specific process of step S4.6 is as follows: in the feature pyramid network, starting from the layer with the smallest resolution, all pixel points are predicted first; then the segmentation results are upsampled using bilinear interpolation, and points with probability values ​​within the set range of 0.45-0.55 are selected; these points and the coarse prediction mask generated in step S4.5 are input into the multi-layer perceptron, a mask with finer and smoother edges is generated for the cladding pool, instance segmentation of the cladding pool image is performed, and the morphology and size recognition of the cladding pool are completed.

[0048] Furthermore, the step S5 comprises the following steps:

[0049] S5.1: From the candidate frame pixel coordinates (M, N, W, H) obtained in step S4.4, it can be known that the number of pixels of the long axis of the cladding pool is W, and the number of pixels of the short axis of the cladding pool, i.e., the width, is H;

[0050] S5.2: Compare the weld width pixel number H with the weld width pixel number calibration value 80 to obtain the weld width change ΔMW=H-80, and input it into the fuzzy controller, and feed back the generated output power change ΔP to step S3.

[0051] Beneficial effects: Compared with the prior art, the present invention, firstly, integrates multiple features through a feature pyramid network to characterize the target at multiple scales; secondly, based on the characteristics that the cladding pool has two inner and outer edges and a relatively smooth middle, the sampling method is optimized, more samples are taken at the edge part, and less samples are taken at the rest of the part, thereby reducing the time required for target detection and identification in the cladding pool. This method not only overcomes the problems of low efficiency and large errors in the method of manually monitoring the morphology and size of the cladding pool throughout the process and then regulating the process parameters, but also overcomes the low applicability of the method of using multiple empirical parameters for edge detection of the cladding pool. This method has good real-time performance and high recognition accuracy. When the edge of the cladding pool is covered by a molten droplet, accurate detection and identification can be achieved, which verifies the effectiveness and robustness of the algorithm, realizes low-cost and high-efficiency detection of the cladding pool morphology, and provides a reliable basis for real-time evaluation of arc welding additive manufacturing. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is the overall flow chart of the method of the present invention;

[0053] Figure 2 Annotate the cladding pool image;

[0054] Figure 3 This is the inspection effect diagram of the arc welding additive manufacturing cladding pool when there is no droplet blocking;

[0055] Figure 4 This is the detection effect diagram of the arc welding additive manufacturing cladding pool when it is blocked by molten droplets;

[0056] Figure 5It is a PR curve diagram of the detection effect of the method of the present invention. DETAILED DESCRIPTION

[0057] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0058] like Figure 1 As shown, the present invention provides a cladding pool morphology recognition and closed-loop control method based on instance segmentation, comprising the following steps:

[0059] S1: Initialize the output power and output power change of arc welding additive manufacturing;

[0060] S2: Collecting color images of arc welding additive manufacturing cladding pool;

[0061] S3: Update arc welding additive manufacturing output power;

[0062] S4: Segment the cladding pool in the cladding pool color image, and generate the instance mask of the cladding pool and the pixel coordinates of the candidate box at the same time, completing the instance segmentation of the cladding pool;

[0063] S5: using the pixel coordinates of the candidate frame to represent the size of the major axis and the number of weld width pixels, calculating the change in the number of weld width pixels, and feeding the generated output power change back to step S3;

[0064] S6: Determine whether the current cladding pool color image is the last frame. If it is the last frame, control the arc welding robot to stop printing; if it is not the last frame, return to step S2 and control the arc welding robot to continue printing.

[0065] In step S1 of this embodiment, the arc welding additive manufacturing output power P=400W is initialized, and the output power change ΔP=0.

[0066] The method for collecting the color image of the arc welding additive manufacturing cladding pool in step S2 of this embodiment is: building a cladding pool image acquisition system, using a NIKON D7100 high-definition camera and a filter to collect the color image of the cladding pool during the arc welding additive manufacturing process.

[0067] In step S3 of this embodiment, the method for updating the arc welding additive manufacturing output power is as follows: according to the output power change ΔP obtained in step S5, the arc welding additive manufacturing output power P is updated, and the updating formula is P=P+ΔP.

[0068] The specific process of step S4 in this embodiment is:

[0069] S4.1: Input the color image of the cladding pool obtained in step S2 into the backbone neural network for feature extraction and generate feature maps of different sizes, fuse the feature maps from top to bottom to generate a feature pyramid network;

[0070] S4.2: Input the feature pyramid network obtained in step S4.1 into the region proposal network, traverse all pixels in the feature map using a sliding window method, and generate candidate boxes for the areas where the cladding pool exists;

[0071] S4.3: The candidate boxes obtained in step S4.2 are unified in size through the R0IAlign layer;

[0072] S4.4: Perform a full connection operation on each candidate frame in step S4.3, regress the position of the candidate frame, determine whether the object in the candidate frame is a cladding pool, and obtain the pixel coordinates of the candidate frame;

[0073] S4.5: Generate a coarse prediction mask for each candidate box obtained in step S4.2 through a lightweight prediction head;

[0074] S4.6: Select a part of points from the feature pyramid network obtained in step S4.1 for independent prediction and input them into the multi-layer perceptron together with the rough prediction mask generated in step S4.5 to generate a mask with finer and smoother edges for the cladding pool, and complete the instance segmentation of the cladding pool image.

[0075] In step S4.1, the backbone neural network includes a residual network ResNet50 and a feature pyramid network;

[0076] The residual network ResNet50 has 16 residual blocks, each residual unit has three layers, and the entire residual network ResNet50 is divided into five stages, where the layer that does not change the size of the feature map is a stage. The feature extracted each time is the output of the last layer of each stage. Each residual unit can be expressed as: y i =f(h(y i-1 )+F(y i-1 ));

[0077] Among them, y i is the output of the i-th residual unit, y i-1 is the output of the i-1th residual unit, f() is the activation function ReLU, h(y i-1 ) is the identity mapping of the i-1th residual unit, F(y i-1 ) is y i-1 The residual mapping of

[0078] The feature pyramid network fuses the feature maps of the five stages output by the entire residual network ResNet50 from top to bottom. The feature maps of each stage output by the residual network ResNet50 undergo convolution and upsampling operations, and then are fused with the feature maps of other stages.

[0079] Step S4.2 specifically includes the following steps:

[0080] S4.2.1: For the fused feature map, a sliding window method is used to traverse all pixels in different proportions on feature maps of different sizes to generate candidate boxes ROI. The sizes of the candidate boxes are 64*64, 128*128, and 256*256, respectively. The ratios of the length and width of the candidate boxes are 2:1, 1:1, and 1:2, respectively. Each pixel will generate 9 candidate boxes of different sizes. There will be nearly 200,000 overlapping feature boxes on the entire image. Then, the candidate boxes are classified and regressed to obtain the scores and probability values ​​of the objects in the candidate boxes being foreground or background, as well as the coordinate values ​​of each candidate box.

[0081] S4.2.2: Filter and retain candidate boxes according to foreground scores, retain the top 6,000 with the highest scores, and then use non-maximum suppression (NMS algorithm) to compare the candidate box with the highest foreground score with other overlapping candidate boxes. If the overlap is greater than 0.5, discard it, otherwise retain it, and filter out the best bounding box to remove redundancy.

[0082] Step S4.3 specifically includes the following steps:

[0083] S4.3.1: Traverse all candidate boxes and map them to corresponding positions on the feature map;

[0084] S4.3.2: Divide each candidate box into 2*2 units, and each unit is divided into 4 small areas. For each small area, the center pixel value is calculated using the bilinear interpolation method.

[0085] Among them, bilinear interpolation is to redistribute the pixel information of the original image, so as to lose as little pixel information as possible when the image is enlarged or reduced. The coordinate values ​​of the four vertices in each small area are A(x1,y1), B(x1,y2), C(x2,y2) and D(x2,y1), and the pixel values ​​are f(A), f(B), f(C) and f(D), respectively. Moreover, x2>x1, y2>y1, and the coordinates of the center point of this small area are G(x,y). The bilinear interpolation algorithm is as follows:

[0086] A1: Interpolate in the horizontal direction to obtain the pixel values ​​f(E) and f(F) at points E and F;

[0087]

[0088] A2: interpolate points E and F on the ordinate axis to obtain the pixel value f(G) of the pixel point G in the small area;

[0089]

[0090] S4.3.3: Perform a maximum pooling operation on the small area of ​​each unit and retain the maximum pixel value, that is, the pixel value of the center point of each small area.

[0091] The specific process of step S4.4 is:

[0092] After the ROIAlign operation, the sizes of different candidate boxes converted to the feature map have been unified to 7*7, and the Softmax function is input after the full connection operation to map the feature map to a fixed-length feature vector, regress the position of the candidate box, classify the objects in the candidate box, and obtain the confidence probability value belonging to the cladding pool, which is used to determine whether the candidate box is a cladding pool and obtain the candidate box pixel coordinates (M, N, W, H), where M represents the horizontal distance from the upper left corner of the candidate box to the upper left corner of the input image, N represents the vertical distance from the upper left corner of the candidate box to the upper left corner of the input image, W represents the pixel length value of the candidate box, and H represents the pixel width value of the candidate box;

[0093] The Softmax function maps the output of the previous layer to a real number between 0 and 1, and the sum of the mappings is 1. Its mathematical form is as follows:

[0094]

[0095] Among them, Z j It is the ratio of the index value of the element to the sum of the index values ​​of all elements, L j is the output of the fully connected network in the previous layer, S is the total number of categories, and the value output by the fully connected network is converted into a probability value through the Softmax function to determine whether the object in the candidate box belongs to the cladding pool category;

[0096] Using the cross entropy function as the loss function of Softmax, the mathematical form is:

[0097]

[0098] Among them, i is the serial number of the element, which can be 1, 2, 3, etc., and is t i is the true value, Z j It is the ratio of the exponential value of the element to the sum of the exponential values ​​of all elements, which is also the output value of the Softmax function. For an input, the smaller the loss of the correct category prediction, the greater the loss of the wrong category prediction.

[0099] The specific process of step S4.5 is as follows: for all candidate boxes, a 14*14 feature map is extracted from the second layer of the feature pyramid network using bilinear interpolation, quantized through ROI-Pooling, connected to ReLU after a 2*2*256 convolution operation, the spatial size is reduced to the specified size of 7*7, and finally a 7*7 coarse prediction mask is generated for each category in the candidate box through a multi-layer perceptron;

[0100] The multilayer perceptron is also called an artificial neural network. It has an input layer, a hidden layer, and an output layer. The layers are fully connected. Its output is f(x)=K(b2+W2(s(b1+W1x))). In the formula, K is the Softmax function, W1 and b1 are the weights and bias parameters between the input layer and the hidden layer, W2 and b2 are the weights and bias parameters between the hidden layer and the output layer, and s is the sigmoid function. Its function form is sigmoid(α)=1 / (1+e -α ), where α = b1 + W1x, all weights and bias parameters are randomly initialized using gradient descent and iterated 1000 times.

[0101] The specific process of step S4.6 is as follows: in the feature pyramid network, starting from the layer with the smallest resolution, all pixels are predicted first; then the segmentation results are upsampled using bilinear interpolation, and points with probability values ​​within the set range of 0.45-0.55 are selected; these points and the coarse prediction mask generated in step S4.5 are input into the multi-layer perceptron to generate a mask with finer and smoother edges for the cladding pool, and instance segmentation of the cladding pool image is performed to complete the morphology and size recognition of the cladding pool.

[0102] Step S5 includes the following steps:

[0103] S5.1: From the candidate frame pixel coordinates (M, N, W, H) obtained in step S4.4, it can be known that the number of pixels of the long axis of the cladding pool is W, and the number of pixels of the short axis of the cladding pool, i.e., the width, is H;

[0104] S5.2: Compare the weld width pixel number H with the weld width pixel number calibration value 80 to obtain the weld width change ΔMW=H-80, and input it into the fuzzy controller, and feed back the generated output power change ΔP to step S3.

[0105] In order to verify the effect of the method of the present invention, the above method was put into practical application, such as Figure 2 The figure shows the effect of manually labeling the original image of the cladding pool using the LabelME tool in this experiment and saving it as a json format file. Figure 3 and Figure 4 The effect diagram, Figure 3This is the effect diagram of the cladding pool detection when there is no molten droplet blocking. Figure 4 This is the effect diagram of cladding pool detection when there is molten droplet obstruction. It can be seen from the figure that the cladding pool can be well detected with and without molten droplet obstruction, thus solving the problem of difficulty in completing high-quality detection when there is molten droplet obstruction, and the number of pixels of the long axis and short axis of the current cladding pool can be detected.

[0106] In order to more intuitively show the effect of the method of the present invention, the performance of the method of the present invention, Faster R-CNN, and Mask R-CNN are compared in this embodiment. Figure 5 It can be seen that the performance of the proposed method is better than Faster R-CNN and Mask R-CNN.

Claims

1. A cladding pool morphology recognition and closed-loop control method based on instance segmentation, characterized in that: The steps include: S1: Initialize the output power and output power change of arc welding additive manufacturing; S2: Collecting color images of arc welding additive manufacturing cladding pool; S3: Update arc welding additive manufacturing output power; S4: Segment the cladding pool in the cladding pool color image, and generate the instance mask of the cladding pool and the pixel coordinates of the candidate box at the same time, completing the instance segmentation of the cladding pool; S5: using the pixel coordinates of the candidate frame to represent the size of the major axis and the number of weld width pixels, calculating the change in the number of weld width pixels, and feeding the generated output power change back to step S3; S6: Determine whether the current cladding pool color image is the last frame, if yes, end printing, if not, return to step S2; The specific process of step S4 is as follows: S4.1: Input the color image of the cladding pool obtained in step S2 into the backbone neural network for feature extraction and generate feature maps of different sizes, fuse the feature maps from top to bottom to generate a feature pyramid network; S4.2: Input the feature pyramid network obtained in step S4.1 into the region proposal network, traverse all pixels in the feature map using a sliding window method, and generate candidate boxes for the areas where the cladding pool exists; S4.3: The candidate boxes obtained in step S4.2 are unified in size through the R0IAlign layer; S4.4: Perform a full connection operation on each candidate frame in step S4.3, regress the position of the candidate frame, determine whether the object in the candidate frame is a cladding pool, and obtain the pixel coordinates of the candidate frame; S4.5: Generate a coarse prediction mask for each candidate box obtained in step S4.2 through a lightweight prediction head; S4.6: Select a portion of points from the feature pyramid network obtained in step S4.1 for independent prediction and input them into the multi-layer perceptron together with the rough prediction mask generated in step S4.5, generate a mask with finer and smoother edges for the cladding pool, and complete the instance segmentation of the cladding pool image; The specific process of step S4.4 is as follows: After the ROIAlign operation, the sizes of different candidate boxes converted to the feature map have been unified, and the Softmax function is input after the full connection operation to map the feature map to a fixed-length feature vector, regress the position of the candidate box, classify the objects in the candidate box, and obtain the confidence probability value belonging to the cladding pool, which is used to determine whether the candidate box is a cladding pool and obtain the candidate box pixel coordinates (M, N, W, H), where M represents the horizontal distance from the upper left corner vertex of the candidate box to the upper left corner of the input image, N represents the vertical distance from the upper left corner vertex of the candidate box to the upper left corner of the input image, W represents the pixel length value of the candidate box, and H represents the pixel width value of the candidate box; The Softmax function maps the output of the previous layer to a real number between 0 and 1, and the sum of the mappings is 1. Its mathematical form is as follows: Among them, Z j It is the ratio of the index value of an element to the sum of the index values ​​of all elements, L j is the output of the fully connected network in the previous layer, S is the total number of categories, and the value output by the fully connected network is converted into a probability value through the Softmax function to determine whether the object in the candidate box belongs to the cladding pool category; Using the cross entropy function as the loss function of Softmax, the mathematical form is: Among them, i is the serial number of the element, t i is the true value, Z j It is the ratio of the exponential value of the element to the sum of the exponential values ​​of all elements, which is also the output value of the Softmax function; The specific process of step S4.5 is as follows: for all candidate boxes, a feature map is extracted from the second layer of the feature pyramid network using bilinear interpolation, quantized by ROI-Pooling, connected to ReLU after convolution operation, the spatial size is reduced to a specified size, and finally a coarse prediction mask is generated for each category in the candidate box by a multi-layer perceptron; The specific process of step S4.6 is as follows: in the feature pyramid network, starting from the layer with the smallest resolution, all pixel points are predicted first; then the segmentation results are upsampled using bilinear interpolation, and points with probability values ​​within a set range are selected; these points and the coarse prediction mask generated in step S4.5 are input into a multi-layer perceptron to generate a mask with finer and smoother edges for the cladding pool, and instance segmentation of the cladding pool image is performed to complete the morphology and size recognition of the cladding pool.

2. The method for cladding pool morphology recognition and closed-loop control based on instance segmentation according to claim 1 is characterized in that: The method for collecting the color image of the arc welding additive manufacturing cladding pool in step S2 is: building a cladding pool image collection system, using a high-definition camera and a filter to collect the color image of the cladding pool during the arc welding additive manufacturing process.

3. The method for cladding pool morphology recognition and closed-loop control based on instance segmentation according to claim 1 is characterized in that: The method for updating the arc welding additive manufacturing output power in step S3 is: updating the arc welding additive manufacturing output power P according to the output power change ΔP obtained in step S5, and the updating formula is P=P+ΔP.

4. The method for cladding pool morphology recognition and closed-loop control based on instance segmentation according to claim 1 is characterized in that: In step S4.1, the backbone neural network includes a residual network ResNet50 and a feature pyramid network; The residual network ResNet50 has 16 residual blocks, each residual unit has three layers, and the entire residual network ResNet50 is divided into five stages, where the layer that does not change the size of the feature map is a stage. The feature extracted each time is the output of the last layer of each stage. Each residual unit can be expressed as: y i =f(h(y i-1 )+F(y i-1 )); Among them, y i is the output of the i-th residual unit, y i-1 is the output of the i-1th residual unit, f() is the activation function ReLU, h(y i-1 ) is the identity mapping of the i-1th residual unit, F(y i-1 ) is y i-1 The residual mapping of The feature pyramid network fuses the feature maps of the five stages output by the entire residual network ResNet50 from top to bottom. The feature maps of each stage output by the residual network ResNet50 undergo convolution and upsampling operations, and then are fused with the feature maps of other stages.

5. The method for cladding pool morphology recognition and closed-loop control based on instance segmentation according to claim 1 is characterized in that: The step S4.2 specifically includes the following steps: S4.2.1: For the fused feature map, a sliding window is used to traverse all pixels in different proportions on feature maps of different sizes to generate candidate boxes ROIs; then the candidate boxes are classified and regressed to obtain the score, probability value and coordinate value of the object in the candidate box being the foreground or background; S4.2.2: Candidate boxes are filtered and retained according to the foreground score, and then the candidate box with the highest foreground score is compared with other overlapping candidate boxes through non-maximum suppression. If the overlap is greater than the set value, it is discarded, otherwise it is retained, and the best bounding box is screened out to remove redundancy.

6. The method for cladding pool morphology recognition and closed-loop control based on instance segmentation according to claim 1 is characterized in that: The step S4.3 specifically includes the following steps: S4.3.1: Traverse all candidate boxes and map them to corresponding positions on the feature map; S4.3.2: Divide each candidate box into a number of units, and each unit is divided into a number of small areas. For each small area, the center pixel value is calculated using the bilinear interpolation method. Among them, bilinear interpolation is to redistribute the pixel information of the original image. The coordinate values ​​of the four vertices in each small area are A(x1,y1), B(x1,y2), C(x2,y2) and D(x2,y1), and the pixel values ​​are f(A), f(B), f(C) and f(D), respectively. Moreover, x2>x1, y2>y1, and the coordinates of the center point of this small area are G(x,y). The bilinear interpolation algorithm is as follows: A1: Interpolate in the horizontal direction to obtain the pixel values ​​f(E) and f(F) at points E and F; A2: interpolate points E and F on the ordinate axis to obtain the pixel value f(G) of the pixel point G in the small area; S4.3.3: Perform a maximum pooling operation on the small area of ​​each unit and retain the maximum pixel value, that is, the pixel value of the center point of each small area.

7. The method for cladding pool morphology recognition and closed-loop control based on instance segmentation according to claim 1 is characterized in that: The step S5 comprises the following steps: S5.1: From the pixel coordinates (M, N, W, H) of the candidate frame obtained in step S4.4, it can be known that the number of pixels of the long axis of the cladding pool is W, and the number of pixels of the short axis of the cladding pool, i.e., the width, is H; S5.2: Compare the weld width pixel number H with the weld width pixel number calibration value to obtain the weld width change, and input it into the fuzzy controller, and feed back the generated output power change ΔP to step S3.