A deep learning-based precision barrel plating process discrimination method and system
By combining deep learning methods with image information, the problem of identifying surface anomalies in precision barrel plating processes has been solved, achieving efficient and accurate detection of plating anomalies and improving the accuracy of part quality inspection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUIZHOU ZHENGQIANG TECH CO LTD
- Filing Date
- 2025-09-10
- Publication Date
- 2026-04-24
AI Technical Summary
In existing precision barrel plating processes, there is a risk of scratches, wear, or deformation on the surface of parts, making it difficult to accurately determine whether scratches, blackening, or yellowing of areas where electroplating has accumulated are present.
Using deep learning methods, combining images captured by barrel plating, images captured by barrel plating laser, and images of uneven surfaces, features are extracted and image information is fused through a 3D convolutional network and a target detection network to identify the types and areas of plating anomalies on the surface of the parts.
It achieves high accuracy in judging precision barrel plating processes, enabling more accurate identification of plating abnormalities on the surface of parts, and improving the efficiency and accuracy of part quality inspection.
Smart Images
Figure CN121169860B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of precision barrel plating process discrimination, and more specifically, to a method and system for discrimination of precision barrel plating processes based on deep learning. Background Technology
[0002] Precision barrel plating, also known as tilting barrel plating or oscillating barrel plating, is an automated production process that involves placing a large number of small parts into a specialized barrel (barrel plating machine) and electroplating them while they are in a rolling state. Although precision barrel plating is designed with a gentler rolling or oscillating method to reduce collisions, friction and collisions between parts are still unavoidable. For precision parts that require extremely high surface finish (such as high-end electronic connectors), there is still a risk of scratches, wear, or deformation.
[0003] Therefore, it is necessary to determine whether the parts have scratches, blackening or yellowing of the electroplated areas. Summary of the Invention
[0004] The purpose of this invention is to provide a method and system for judging precision barrel plating processes based on deep learning, so as to solve the above-mentioned problems existing in the prior art.
[0005] In a first aspect, embodiments of the present invention provide a method for discriminating precision barrel plating processes based on deep learning, comprising: acquiring barrel plating images of a part at multiple angles and corresponding barrel plating laser images; wherein the barrel plating laser images are images of the surface of a part after the barrel plating process scanned by a laser scanner; and the barrel plating images are images of the surface of a part captured by a camera device.
[0006] Based on the barrel plating laser image, a first barrel plating state value and an unevenness position image are obtained; the unevenness position image represents the unevenness shape of the surface of the part after the barrel plating process; the first barrel plating state value represents the value combining the state of the plating thickness and the distribution of the corresponding concave and convex areas.
[0007] Through deep learning, based on barrel plating images, barrel plating laser images, and images of uneven surfaces, a second barrel plating category and a second barrel plating region are obtained; the second barrel plating category represents the category of plating anomalies on the surface of the part; the second barrel plating region represents the region corresponding to the second barrel plating category.
[0008] Based on the first barrel plating state value, the second barrel plating category, and the second barrel plating area, a discrimination level is obtained; the discrimination level represents the level of discrimination performed by the precision barrel plating process.
[0009] Optionally, the method for acquiring the barrel plating laser image includes:
[0010] Multiple coating thickness values are acquired and a coating thickness value is set; the coating thickness value represents the thickness of the coating scanned by the laser scanner at one position on the surface of the part; the set coating thickness value represents the thickness of the coating set on the surface of the part by the barrel plating process.
[0011] Obtain a blank 3D image; the length of the blank 3D image is equal to the length of the barrel plating image; the width of the blank 3D image is equal to the width of the barrel plating image; the height of the blank 3D image is twice the set plating thickness value;
[0012] Construct a three-dimensional coordinate system; the lower left corner of the blank three-dimensional image is the origin; the length of the blank three-dimensional image corresponds to the horizontal coordinate of the three-dimensional matrix; the width of the blank three-dimensional image corresponds to the vertical coordinate of the three-dimensional matrix; the height of the blank three-dimensional image corresponds to the vertical coordinate of the three-dimensional matrix.
[0013] In a blank 3D image, the coating thickness value is matched with the value of the vertical axis to obtain a barrel-coated laser image.
[0014] Optionally, obtaining the first barrel plating state value and the concave / convex position image based on the barrel plating laser image includes:
[0015] Multiple values in the barrel-coating laser image are clustered to obtain a clustered set and a non-clustered set; the clustered set represents the locations where the coating thickness does not change abruptly; the non-clustered set represents the locations where the coating thickness changes abruptly.
[0016] The standard deviation of multiple values in the cluster set is calculated to obtain a first state value; the first state value represents the overall difference in coating thickness in the barrel-plating laser image.
[0017] Based on the clustered and non-clustered sets, the uniformity of the concavity and convexity of the barrel-coating laser image is determined, and the concavity and convexity position image is obtained;
[0018] Based on the concave-convex position image, a second state value is obtained; the second state value represents the distribution state of the concave and convex regions.
[0019] Multiply the second state value by the first state value to obtain the first barrel plating state value.
[0020] Optionally, the step of determining the uniformity of the unevenness of the barrel-coating laser image based on the clustered and non-clustered sets to obtain the unevenness position image includes:
[0021] By using mean-shift clustering, the positions corresponding to the positive values in the cluster set are clustered according to the x-coordinate and y-coordinate, resulting in n convex cluster sets; n represents the number of convex regions of the coating.
[0022] By using mean-shift clustering, the positions corresponding to the negative values in the cluster set are clustered by the x and y coordinates to obtain m concave cluster sets; m represents the number of concave regions of the coating.
[0023] Construct a blank two-dimensional image; the length of the blank two-dimensional image is equal to the length corresponding to the horizontal coordinate of the barrel-plated laser image; the width of the blank two-dimensional image is equal to the width corresponding to the vertical coordinate of the barrel-plated laser image;
[0024] In a blank two-dimensional image, the positions formed by the x and y coordinates corresponding to the values of convex clusters are set to 125, the positions formed by the x and y coordinates corresponding to the values of concave clusters are set to 255, and the positions formed by the x and y coordinates corresponding to the values of non-clustered sets are set to 0, thus obtaining a concave-convex position image.
[0025] Optionally, obtaining the second state value based on the concave / convex position image includes:
[0026] The concave and convex position images are segmented to obtain concave position images and convex position images; the concave position image means that the position of the cluster center with a concave and convex position image of 255 is set to 1, and the other positions are set to 0; the convex position image means that the position of the cluster center with a concave and convex position image of 125 is set to 1, and the other positions are set to 0.
[0027] Calculate the distribution of 1 values in the concave position image to obtain the first correlation value; the distribution state of the concave region of the first correlation value;
[0028] Calculate the distribution of values of 1 in the convex position image to obtain the second correlation value; the distribution state of the region where the first correlation value is convex;
[0029] Multiply the reciprocal of the first correlation value by the reciprocal of the second correlation value by 1 / (m+n) to obtain the second state value.
[0030] Optionally, the step of obtaining a second barrel plating category and a second barrel plating region through deep learning, based on barrel plating images, barrel plating laser images, and images of uneven surfaces, includes:
[0031] The barrel plating image is input into a first three-dimensional convolutional network to extract features and obtain a first training feature map;
[0032] The image of the barrel-plating laser is input into the second target detection network to extract features and obtain the second training feature map;
[0033] Based on the barrel plating image, the barrel plating laser image, and the concave / convex position image, a third training feature map is obtained;
[0034] The first training feature map, the second training feature map, and the third training feature map are fused to obtain the second barrel plating category and the corresponding second barrel plating region; the second barrel plating category represents the type of surface error; the second barrel plating region represents the region corresponding to the barrel plating category of surface error.
[0035] Optionally, the step of obtaining a third training feature map based on the barrel plating image, the barrel plating laser image, and the concave / convex position image includes:
[0036] In the barrel plating captured images and barrel plating laser images, the positions corresponding to the non-clustered sets in the concave and convex position images are extracted to obtain partial captured images and partial laser images;
[0037] The captured images and the laser images are superimposed to obtain a superimposed image;
[0038] The superimposed image is input into the third target detection network to obtain the third training feature map.
[0039] Optionally, the training method for the first three-dimensional convolutional network includes:
[0040] Multiple training barrel plating laser images and multiple annotation data are obtained; the annotation data includes the annotation barrel plating category and the corresponding annotation barrel plating area.
[0041] The training image of the roller-coating laser is input into the first three-dimensional convolutional network to extract features and obtain the first training feature map.
[0042] The first training feature map is input into a fully connected neural network for classification to obtain the first training category and the first training region.
[0043] The loss is calculated for the first training category and the labeled rolling coating category, and the loss is calculated for the first training region and the labeled rolling coating region. The first 3D convolutional network is then trained by backpropagation.
[0044] Optionally, the determination of the discrimination level based on the first barrel plating state value, the second barrel plating category, and the second barrel plating area includes:
[0045] The area of the second barrel plating area is calculated to obtain the second area value;
[0046] Obtain a standard table; the standard table contains multiple first barrel plating state intervals, multiple barrel plating categories, and multiple barrel plating areas, as well as standard discrimination levels corresponding to the first barrel plating state intervals, barrel plating categories, and barrel plating areas;
[0047] In the standard table, find the standard discrimination level corresponding to the first barrel plating state value, the second barrel plating category, and the second barrel plating area to obtain the discrimination level.
[0048] Secondly, embodiments of the present invention provide a discrimination system for precision barrel plating processes based on deep learning, comprising:
[0049] The acquisition module is used to acquire barrel plating images of the part from multiple angles and corresponding barrel plating laser images; the barrel plating laser images are images of the part surface after the barrel plating process scanned by a laser scanner; the barrel plating images are images of the part surface captured by a camera device.
[0050] The bump module is used to obtain a first barrel plating state value and a bump position image based on the barrel plating laser image; the bump position image represents the bump shape of the surface of the part after the barrel plating process; the first barrel plating state value represents the value combining the state of the plating thickness and the distribution of the corresponding concave and convex areas.
[0051] The deep learning module is used to obtain a second barrel plating category and a second barrel plating region based on barrel plating images, barrel plating laser images, and images of uneven surfaces using deep learning. The second barrel plating category represents the category of plating anomalies on the surface of the part. The second barrel plating region represents the region corresponding to the second barrel plating category.
[0052] The discrimination module is used to obtain a discrimination level based on a first barrel plating state value, a second barrel plating category, and a second barrel plating area; the discrimination level represents the discrimination level of the precision barrel plating process.
[0053] Compared with the prior art, the embodiments of the present invention achieve the following beneficial effects:
[0054] This invention also provides a method and system for identifying precision barrel plating processes based on deep learning.
[0055] In this invention, the thickness and color of the coating on the surface of the part are determined using both barrel plating laser images and barrel plating photographs. Clustering is used to identify locations of abrupt changes and variations in the unevenness of the coating in the barrel plating laser images. If the areas of concavity and convexity are small and uniform, it indicates a normal phenomenon during barrel plating, as a perfectly smooth coating is difficult to achieve. If the areas are large and unevenly distributed, it indicates the presence of protrusions or concavities outside the acceptable range, leading to inaccurate part matching. By recording the locations of protrusions, concavities, and abrupt changes in the unevenness images, a convolutional neural network can extract color and shape features to identify the categories and areas of coating anomalies. The presence of scratches and discoloration on the part surface is assessed using both arithmetic and deep learning methods to evaluate the precision barrel plating process. This results in a more accurate acquisition of the user's desired part technical effect. Attached Figure Description
[0056] Figure 1This is a flowchart of a discrimination method for a precision barrel plating process based on deep learning, provided by an embodiment of the present invention. Detailed Implementation
[0057] The present invention will now be described in detail with reference to the accompanying drawings.
[0058] Example 1
[0059] like Figure 1 As shown, this embodiment of the invention provides a method for discriminating precision barrel plating processes based on deep learning, the method comprising:
[0060] S101: Acquire barrel plating images of the part from multiple angles and corresponding barrel plating laser images; the barrel plating laser images are images of the surface of the part after the barrel plating process scanned by a laser scanner; the barrel plating images are images of the surface of the part captured by a camera device.
[0061] The barrel plating image and the corresponding barrel plating laser image are used to extract the same area on the surface of the part. The part refers to the object undergoing barrel plating. The part surface refers to the surface of the part after the barrel plating process.
[0062] The images captured during the barrel plating process were taken using a high-definition industrial camera.
[0063] The barrel plating image is a two-dimensional image, and the barrel plating laser image is a three-dimensional image.
[0064] Among them, the barrel plating images taken from multiple angles include all angles of the part.
[0065] S102: Based on the barrel plating laser image, a first barrel plating state value and a concave-convex position image are obtained; the concave-convex position image represents the concave-convex shape of the surface of the part after the barrel plating process; the first barrel plating state value represents the value combining the state of the plating thickness and the distribution of the corresponding concave and convex areas.
[0066] S103: Through deep learning, based on barrel plating images, barrel plating laser images, and images of uneven positions, a second barrel plating category and a second barrel plating region are obtained; the second barrel plating category represents the category of plating abnormality on the surface of the part; the second barrel plating region represents the region corresponding to the second barrel plating category;
[0067] S104: Based on the first barrel plating state value, the second barrel plating category, and the second barrel plating area, a discrimination level is obtained; the discrimination level represents the level of discrimination performed by the precision barrel plating process.
[0068] Optionally, the method for acquiring the barrel plating laser image includes:
[0069] Multiple coating thickness values are obtained and a coating thickness value is set; the coating thickness value represents the thickness of the coating scanned by the laser scanner at one position on the surface of the part; the set coating thickness value represents the thickness of the coating set on the surface of the part by the barrel plating process.
[0070] In this context, one position on the surface of the part represents the position of one pixel in the barrel plating image.
[0071] Specifically, the thickness of the coating can be calculated by subtracting the thickness of the uncoated part's surface from the thickness of the coated surface at one location. The corresponding locations are then matched using the `matchtemplate` function in OpenCV.
[0072] Obtain a blank 3D image; the length of the blank 3D image is equal to the length of the barrel plating image; the width of the blank 3D image is equal to the width of the barrel plating image; the height of the blank 3D image is twice the set plating thickness value.
[0073] In this embodiment, conventional electroplating zinc is used for the coating, so the coating thickness is set to 10 μm. This means the height of the blank 3D image is 10 * 2 = 20, where a height of 1 represents a thickness of 1 μm. The reason the height of the blank 3D image is twice the set coating thickness is that, using a barrel plating process, a coating thickness of 10 μm results in a blank 3D image height of 10. If the coating is completely damaged, the blank 3D image height will be 0. Furthermore, if clumped areas appear, the coating thickness is greater than 10 μm, and the blank 3D image height will be greater than 10.
[0074] Construct a three-dimensional coordinate system; the lower left corner of the blank three-dimensional image is the origin; the length of the blank three-dimensional image corresponds to the horizontal coordinate of the three-dimensional matrix; the width of the blank three-dimensional image corresponds to the vertical coordinate of the three-dimensional matrix; and the height of the blank three-dimensional image corresponds to the vertical coordinate of the three-dimensional matrix.
[0075] The vertical coordinate of the roller-coating laser image is 10, indicating that the coating thickness value is set.
[0076] In a blank 3D image, the coating thickness value is matched with the value of the vertical axis to obtain a barrel-coated laser image.
[0077] In this embodiment, if the coating thickness value is greater than 10, the position matching the coating thickness value on the vertical axis is found in the blank 3D image, and all values greater than 10 in the vertical axis are set to 1. If the coating thickness value is less than 10, the position matching the coating thickness value on the vertical axis is found in the blank 3D image, and all values less than 10 in the vertical axis are set to 1.
[0078] Optionally, obtaining the first barrel plating state and unevenness position image based on the barrel plating laser image includes:
[0079] Multiple values in the barrel-coating laser image are clustered to obtain a clustered set and a non-clustered set; the clustered set represents the location where the coating thickness does not change abruptly; the non-clustered set represents the location where the coating thickness changes abruptly.
[0080] In this embodiment, the k-means clustering method is used, and the number of clusters is set to 1.
[0081] Specifically, clustering multiple values in the barrel-plating laser image represents clustering the values on the vertical coordinate corresponding to the barrel-plating laser image in three-dimensional coordinates.
[0082] The non-clustered set represents the location where a barrel plating process error was detected by clustering.
[0083] The standard deviation of multiple values in the cluster set is calculated to obtain a first state value; the first state value represents the overall difference in coating thickness in the barrel-plating laser image.
[0084] Based on the clustered and non-clustered sets, the uniformity of the concavity and convexity of the barrel-plating laser image is determined, and the concavity and convexity position image is obtained.
[0085] Based on the concave and convex position image, a second state value is obtained; the second state value represents the distribution state of the concave and convex regions.
[0086] Multiply the second state value by the first state value to obtain the first barrel plating state value.
[0087] The first barrel plating state value represents a value that combines the plating thickness state with the distribution state of the corresponding recessed and protruding areas.
[0088] Optionally, the step of determining the uniformity of the unevenness of the barrel-coating laser image based on the clustered and non-clustered sets to obtain the unevenness position image includes:
[0089] Mean drift clustering is used to cluster the positions corresponding to positive values in the cluster set by their x and y coordinates, resulting in n convex cluster sets; n represents the number of convex regions.
[0090] Where n is a positive integer.
[0091] In this embodiment, the barrel plating image and the corresponding barrel plating laser image are 64*64 pixels in size, with the unit being millimeters.
[0092] In this embodiment, Mean Shift clustering is used for clustering, and the bandwidth of Mean Shift is 4 mm. Mean Shift is a nonparametric clustering algorithm based on density estimation. Its core idea is to find local maxima (patterns) of the data distribution density function and gather the data into these high-density points (modalities) to form clusters. It does not require pre-specifying the number of clusters.
[0093] By using mean-shift clustering, the positions corresponding to the negative values in the cluster set are clustered by the x-coordinate and y-coordinate to obtain m concave cluster sets; m represents the number of concave regions of the coating.
[0094] Where m is a positive integer.
[0095] Construct a blank two-dimensional image; the length of the blank two-dimensional image is equal to the length corresponding to the horizontal axis of the barrel-plated laser image; the width of the blank two-dimensional image is equal to the width corresponding to the vertical axis of the barrel-plated laser image.
[0096] The initial values in the blank two-dimensional matrix are all 0.
[0097] The coordinate axes are used to represent the orientation of the blank two-dimensional matrix and the barrel-coated laser image.
[0098] In a blank two-dimensional image, the position formed by the x-coordinate and y-coordinate corresponding to the value of the convex cluster set is set to 125, the position formed by the x-coordinate and y-coordinate corresponding to the value of the concave cluster set is set to 255, and the position formed by the x-coordinate and y-coordinate corresponding to the value of the non-cluster set is set to 0, thus obtaining a concave-convex position image; the concave-convex position image represents the morphology of the surface of the part after the barrel plating process.
[0099] Optionally, obtaining the second state value based on the concave / convex position image includes:
[0100] The concave and convex position images are segmented to obtain concave position images and convex position images; the concave position image means that the position of the cluster center with a concave and convex position image of 255 is set to 1, and the other positions are set to 0; the convex position image means that the position of the cluster center with a concave and convex position image of 125 is set to 1, and the other positions are set to 0.
[0101] The distribution of values of 1 in the concave position image is calculated to obtain the first correlation value. The distribution of the concave region of the first correlation value is then described.
[0102] In this embodiment, the Euclidean distance between each pair of values of 1 in the concave position image is calculated, and the variance of multiple Euclidean distances is used as the first correlation value.
[0103] Among them, the smaller the first correlation value, the more uniform the distribution of the concave area.
[0104] The distribution of values of 1 in the convex position image is calculated to obtain the second correlation value. The distribution of the region where the first correlation value is convex is also calculated.
[0105] In this embodiment, the Euclidean distance between each pair of values of 1 in the convex position image is calculated, and the variance of multiple Euclidean distances is used as the first correlation value.
[0106] Among them, the smaller the second correlation value, the more uniform the distribution of the protruding area.
[0107] Multiply the reciprocal of the first correlation value by the reciprocal of the second correlation value by 1 / (m+n) to obtain the second state value.
[0108] Wherein, 1 / (m+n) represents the number of uneven undulations in the coating, because the more uniform the distribution of the concave and convex areas and the greater the number of uneven undulations, the more uniform the coating is.
[0109] Optionally, the step of obtaining a second barrel plating category and a second barrel plating region through deep learning, based on barrel plating images, barrel plating laser images, and images of uneven surfaces, includes:
[0110] The image captured by the barrel plating is input into the first three-dimensional convolutional network to extract features and obtain the first training feature map.
[0111] The first three-dimensional convolutional network is a three-dimensional convolutional neural network (3D-CNN) containing 5 layers of 2*2*2 convolutional kernels.
[0112] The laser image of the barrel plating is input into the second target detection network to extract features and obtain the second training feature map.
[0113] The second object detection network is the YOLOv5 model.
[0114] The second object detection network is trained by calculating the loss by labeling the barrel plating category and the corresponding labeled barrel plating area; the second training feature map is the output of the penultimate convolutional layer of the trained second object detection network.
[0115] Based on the barrel plating image, the barrel plating laser image, and the concave / convex position image, a third training feature map is obtained.
[0116] The first training feature map, the second training feature map, and the third training feature map are all the same size.
[0117] The first training feature map, the second training feature map, and the third training feature map are fused to obtain the second barrel plating category and the corresponding second barrel plating region; the second barrel plating category represents the type of surface error; the second barrel plating region represents the region corresponding to the barrel plating category of surface error.
[0118] Specifically, the values of the same index in the first, second, and third training feature maps are averaged and fused. After fusion, the feature maps are expanded into feature vectors using the flattn function in NumPy. The vectors are then classified using a fully connected layer (FC) neural network to obtain the second barrel plating category and the corresponding second barrel plating region.
[0119] Optionally, the step of obtaining a third training feature map based on the barrel plating image, the barrel plating laser image, and the concave / convex position image includes:
[0120] In the barrel plating images and barrel plating laser images, the positions corresponding to the non-clustered sets in the concave-convex position images are extracted to obtain partial images and partial laser images.
[0121] Specifically, in the barrel plating images and barrel plating laser images, the positions corresponding to the values of 0 in the concave-convex position images are retained, while the values of other positions are set to 0.
[0122] The captured images and the laser images are superimposed to obtain a superimposed image.
[0123] In this embodiment, the number of channels in the superimposed image is equal to the number of RGB channels in the partial captured image plus the height of the partial laser image. In this embodiment, it is 3 + 20 = 23.
[0124] The superimposed image is input into the third target detection network to obtain the third training feature map.
[0125] The third object detection network is the YOLOv5 model.
[0126] The third object detection network is trained by calculating the loss by labeling the barrel plating category and the corresponding labeled barrel plating area; the third training feature map is the output of the penultimate convolutional layer of the trained third object detection network.
[0127] Optionally, the training method for the first three-dimensional convolutional network includes:
[0128] Multiple training barrel plating laser images and multiple annotation data are obtained; the annotation data includes the annotation barrel plating category and the corresponding annotation barrel plating area.
[0129] The labeled barrel plating categories include blackening, yellowing, scratches, and clumps.
[0130] The training image of the roller-coating laser is input into the first three-dimensional convolutional network to extract features and obtain the first training feature map.
[0131] The first training feature map is input into a fully connected neural network for classification to obtain the first training category and the first training region.
[0132] In this embodiment, the fully connected neural network (FCN) consists of 8 neuron layers. The feature map is expanded into feature vectors using the flattn function in NumPy.
[0133] The loss is calculated for the first training category and the labeled rolling coating category, and the loss is calculated for the first training region and the labeled rolling coating region. The first 3D convolutional network is then trained by backpropagation.
[0134] Specifically, the loss for the first training category and the labeled roller coating category is calculated using the cross-entropy loss function, and the loss for the first training region and the labeled roller coating region is calculated using the GIoU loss function.
[0135] Optionally, the determination of the discrimination level based on the first barrel plating state value, the second barrel plating category, and the second barrel plating area includes:
[0136] The area of the second barrel plating area is calculated to obtain the second area value.
[0137] The area is calculated using a method that counts pixels.
[0138] Obtain a standard table; the standard table contains multiple first barrel plating state intervals, multiple barrel plating categories, and multiple barrel plating areas, as well as standard discrimination levels corresponding to the first barrel plating state intervals, barrel plating categories, and barrel plating areas.
[0139] The standard table is designed based on parts that meet the user's needs.
[0140] In the standard table, find the standard discrimination level corresponding to the first barrel plating state value, the second barrel plating category, and the second barrel plating area to obtain the discrimination level.
[0141] Multiple images of barrel plating taken from various angles and their corresponding barrel plating laser images are used to obtain multiple discrimination levels. If a part has one discrimination level that does not meet the requirements, the part is marked, and the area on the part surface corresponding to that angle is also marked.
[0142] Example 2
[0143] Based on the above-mentioned deep learning-based method for discriminating precision barrel plating processes, this embodiment of the invention also provides a deep learning-based system for discriminating precision barrel plating processes, the system comprising an acquisition module, a concave / convex module, a deep learning module, and a discrimination module.
[0144] The acquisition module is used to acquire barrel plating images of the part from multiple angles and corresponding barrel plating laser images; the barrel plating laser images are images of the surface of the part after the barrel plating process scanned by a laser scanner; the barrel plating images are images of the surface of the part captured by a camera device.
[0145] The convexity / concaveness module is used to obtain a first barrel plating state value and a convexity / concaveness position image based on the barrel plating laser image; the convexity / concaveness position image represents the convexity / concaveness shape of the surface of the part after the barrel plating process; the first barrel plating state value represents a value that combines the state of the plating thickness and the distribution of the corresponding concave and convex areas.
[0146] The deep learning module is used to obtain a second barrel plating category and a second barrel plating region based on barrel plating images, barrel plating laser images, and images of uneven positions through deep learning; the second barrel plating category represents the category of plating abnormalities on the surface of the part; the second barrel plating region represents the region corresponding to the second barrel plating category.
[0147] The discrimination module is used to obtain a discrimination level based on a first barrel plating state value, a second barrel plating category, and a second barrel plating area; the discrimination level represents the discrimination level of the precision barrel plating process.
[0148] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages is for the purpose of disclosing the best mode of implementation of the invention.
[0149] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.
[0150] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the apparatus according to embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.
Claims
1. A method for discriminating precision barrel plating processes based on deep learning, characterized in that, include: Acquire barrel plating images of the parts from multiple angles and the corresponding barrel plating laser images; The barrel plating laser image is an image of the surface of the part after the barrel plating process, scanned by a laser scanner; the barrel plating photograph is an image of the surface of the part taken by a camera device. Based on the barrel plating laser image, a first barrel plating state value and an unevenness position image are obtained; the unevenness position image represents the unevenness shape of the surface of the part after the barrel plating process. The first barrel plating state value represents the value that combines the state of the plating thickness with the distribution of the corresponding recessed and protruding areas; Through deep learning, based on barrel plating images, barrel plating laser images, and images of concave and convex positions, a second barrel plating category and a second barrel plating region are obtained. The second barrel plating category indicates the category of plating abnormalities on the surface of the part; The second barrel plating area represents the area corresponding to the second barrel plating category; The discrimination level is obtained based on the first barrel plating state value, the second barrel plating category, and the second barrel plating area; The discrimination level indicates the discrimination level performed by the precision barrel plating process; The step of obtaining the first barrel plating state value and the concave / convex position image based on the barrel plating laser image includes: Multiple values in the barrel-coating laser image are clustered to obtain a clustered set and a non-clustered set; the clustered set represents the locations where the coating thickness does not change abruptly; the non-clustered set represents the locations where the coating thickness changes abruptly. The standard deviation of multiple values in the cluster set is calculated to obtain a first state value; the first state value represents the overall difference in coating thickness in the barrel-plating laser image. Based on the clustered and non-clustered sets, the uniformity of the concavity and convexity of the barrel-coating laser image is determined, and the concavity and convexity position image is obtained; Based on the concave-convex position image, a second state value is obtained; the second state value represents the distribution state of the concave and convex regions. Multiply the second state value by the first state value to obtain the first barrel plating state value; The determination of the discrimination level based on the first barrel plating state value, the second barrel plating category, and the second barrel plating area includes: The area of the second barrel plating area is calculated to obtain the second area value; Obtain a standard table; the standard table contains multiple first barrel plating state intervals, multiple barrel plating categories, and multiple barrel plating areas, as well as standard discrimination levels corresponding to the first barrel plating state intervals, barrel plating categories, and barrel plating areas; In the standard table, find the standard discrimination level corresponding to the first barrel plating state value, the second barrel plating category, and the second barrel plating area to obtain the discrimination level.
2. The method for discriminating precision barrel plating processes based on deep learning according to claim 1, characterized in that, The method for acquiring the barrel plating laser image includes: Multiple coating thickness values are acquired and a coating thickness value is set; the coating thickness value represents the thickness of the coating scanned by the laser scanner at one position on the surface of the part; the set coating thickness value represents the thickness of the coating set on the surface of the part by the barrel plating process. Obtain a blank 3D image; the length of the blank 3D image is equal to the length of the barrel plating image; the width of the blank 3D image is equal to the width of the barrel plating image; the height of the blank 3D image is twice the set plating thickness value; Construct a three-dimensional coordinate system; the lower left corner of the blank three-dimensional image is the origin; the length of the blank three-dimensional image corresponds to the horizontal coordinate of the three-dimensional matrix; the width of the blank three-dimensional image corresponds to the vertical coordinate of the three-dimensional matrix; the height of the blank three-dimensional image corresponds to the vertical coordinate of the three-dimensional matrix. In a blank 3D image, the coating thickness value is matched with the value of the vertical axis to obtain a barrel-coated laser image.
3. The discrimination method for precision barrel plating process based on deep learning according to claim 1, characterized in that, The step of determining the uniformity of the unevenness of the barrel-coating laser image based on the clustered and non-clustered sets to obtain the unevenness position image includes: By using mean-shift clustering, the positions corresponding to the positive values in the cluster set are clustered according to the x-coordinate and y-coordinate, resulting in n convex cluster sets; n represents the number of convex regions of the coating. By using mean-shift clustering, the positions corresponding to the negative values in the cluster set are clustered by the x and y coordinates to obtain m concave cluster sets; m represents the number of concave regions of the coating. Construct a blank two-dimensional image; the length of the blank two-dimensional image is equal to the length corresponding to the horizontal coordinate of the barrel-plated laser image; the width of the blank two-dimensional image is equal to the width corresponding to the vertical coordinate of the barrel-plated laser image; In a blank two-dimensional image, the positions formed by the x and y coordinates corresponding to the values of convex clusters are set to 125, the positions formed by the x and y coordinates corresponding to the values of concave clusters are set to 255, and the positions formed by the x and y coordinates corresponding to the values of non-clustered sets are set to 0, thus obtaining a concave-convex position image.
4. The discrimination method for precision barrel plating process based on deep learning according to claim 1, characterized in that, The process of obtaining the second state value based on the concave / convex position image includes: The concave and convex position images are segmented to obtain concave position images and convex position images; the concave position image means that the position of the cluster center with a concave and convex position image of 255 is set to 1, and the other positions are set to 0; the convex position image means that the position of the cluster center with a concave and convex position image of 125 is set to 1, and the other positions are set to 0. Calculate the distribution of 1 values in the concave position image to obtain the first correlation value; the distribution state of the concave region of the first correlation value; Calculate the distribution of values of 1 in the convex position image to obtain the second correlation value; the distribution state of the region where the first correlation value is convex; Multiply the reciprocal of the first correlation value by the reciprocal of the second correlation value by 1 / (m+n) to obtain the second state value.
5. The method for discriminating precision barrel plating processes based on deep learning according to claim 1, characterized in that, The process of obtaining a second barrel plating category and a second barrel plating region through deep learning, based on barrel plating images, barrel plating laser images, and images of concave and convex positions, includes: The barrel plating image is input into a first three-dimensional convolutional network to extract features and obtain a first training feature map; The image of the barrel-plating laser is input into the second target detection network to extract features and obtain the second training feature map; Based on the barrel plating image, the barrel plating laser image, and the concave / convex position image, a third training feature map is obtained; The first training feature map, the second training feature map, and the third training feature map are fused to obtain the second barrel plating category and the corresponding second barrel plating area.
6. The discrimination method for precision barrel plating process based on deep learning according to claim 1, characterized in that, The third training feature map is obtained based on the barrel plating image, the barrel plating laser image, and the concave / convex position image, including: In the barrel plating captured images and barrel plating laser images, the positions corresponding to the non-clustered sets in the concave and convex position images are extracted to obtain partial captured images and partial laser images; The captured images and the laser images are superimposed to obtain a superimposed image; The superimposed image is input into the third target detection network to obtain the third training feature map.
7. The method for discriminating precision barrel plating processes based on deep learning according to claim 5, characterized in that, The training method for the first three-dimensional convolutional network includes: Multiple training barrel plating laser images and multiple annotation data are obtained; the annotation data includes the annotation barrel plating category and the corresponding annotation barrel plating area. The training image of the roller-coating laser is input into the first three-dimensional convolutional network to extract features and obtain the first training feature map. The first training feature map is input into a fully connected neural network for classification to obtain the first training category and the first training region. The loss is calculated for the first training category and the labeled rolling coating category, and the loss is calculated for the first training region and the labeled rolling coating region. The first 3D convolutional network is then trained by backpropagation.
8. A discrimination system for precision barrel plating process based on deep learning, characterized in that, include: The acquisition module is used to acquire barrel plating images of the part from multiple angles and the corresponding barrel plating laser images; The barrel plating laser image is an image of the surface of the part after the barrel plating process, scanned by a laser scanner; the barrel plating photograph is an image of the surface of the part taken by a camera device. The bump module is used to obtain a first barrel plating state value and a bump position image based on the barrel plating laser image; the bump position image represents the bump shape of the surface of the part after the barrel plating process. The first barrel plating state value represents the value that combines the state of the plating thickness with the distribution of the corresponding recessed and protruding areas; The deep learning module is used to obtain the second barrel plating category and the second barrel plating region based on the barrel plating captured image, the barrel plating laser image, and the concave and convex position image through deep learning. The second barrel plating category indicates the category of plating abnormalities on the surface of the part; The second barrel plating area represents the area corresponding to the second barrel plating category; The discrimination module is used to obtain the discrimination level based on the first barrel plating state value, the second barrel plating category, and the second barrel plating area; The discrimination level indicates the discrimination level performed by the precision barrel plating process; The step of obtaining the first barrel plating state value and the concave / convex position image based on the barrel plating laser image includes: Multiple values in the barrel-coating laser image are clustered to obtain a clustered set and a non-clustered set; the clustered set represents the locations where the coating thickness does not change abruptly; the non-clustered set represents the locations where the coating thickness changes abruptly. The standard deviation of multiple values in the cluster set is calculated to obtain a first state value; the first state value represents the overall difference in coating thickness in the barrel-plating laser image. Based on the clustered and non-clustered sets, the uniformity of the concavity and convexity of the barrel-coating laser image is determined, and the concavity and convexity position image is obtained; Based on the concave-convex position image, a second state value is obtained; the second state value represents the distribution state of the concave and convex regions. Multiply the second state value by the first state value to obtain the first barrel plating state value; The determination of the discrimination level based on the first barrel plating state value, the second barrel plating category, and the second barrel plating area includes: The area of the second barrel plating area is calculated to obtain the second area value; Obtain a standard table; the standard table contains multiple first barrel plating state intervals, multiple barrel plating categories, and multiple barrel plating areas, as well as standard discrimination levels corresponding to the first barrel plating state intervals, barrel plating categories, and barrel plating areas; In the standard table, find the standard discrimination level corresponding to the first barrel plating state value, the second barrel plating category, and the second barrel plating area to obtain the discrimination level.
Citation Information
Patent Citations
Aluminum sheet defect detection method based on YOLOv5
CN115880264A
Defect detecting device and defect detecting method
JP2012159491A