A method, device and equipment for identifying metal surface defects

Abstract

The present invention discloses a metal surface defect identification method, device and equipment, which relate to the field of defect detection technology. First, point cloud data corresponding to a preset metal surface standard profile and the metal surface profile during processing are obtained. By using three-dimensional point cloud data to represent the metal surface profile during processing, the collection of point cloud data avoids the problem of poor metal surface image clarity caused by exposure of a 2D camera. Then, template matching of the metal surface profile and the standard profile during processing is performed based on the point cloud data, which can clearly compare and obtain defect three-dimensional points. After distance clustering of the defect three-dimensional points, point cloud data corresponding to the defect area is further obtained. Compared with the image data collected by a 2D camera, the point cloud data corresponding to the defect area has richer information, thereby further improving the accuracy of metal surface defect identification when performing defect identification based on the point cloud data corresponding to each defect area.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection, and in particular to a metal surface defect recognition method, device and equipment. Background Art

[0002] In general, during the rolling process of metal materials, various defects such as scratches, cracks, rolling scars, pits, etc. will inevitably form on the metal surface due to reasons such as production technology and equipment aging. If these defects cannot be detected in time, the yield rate of the factory's products will be greatly reduced, resulting in significant economic losses. In the traditional metallurgical industry, factories usually use manual inspection to detect surface defects on rolled products. However, in actual production, the surface defects of these metals are often small in size (millimeter level), and manual inspection is difficult and time-consuming. In addition, most metal materials are moving at high speed on the processing production line. Manual inspection is difficult to perform online real-time inspection, and the detection point is delayed, making it impossible to detect defects in a timely manner. The remedy for economic cost losses is relatively limited. Therefore, it is extremely important to develop an algorithm that can identify metal surface defects in real time online.

[0003] With the development of deep learning, automatic metal surface defect recognition solutions based on machine vision have attracted widespread attention. Given the low cost of 2D cameras and the maturity of related image processing algorithms, existing technologies mostly use 2D cameras to capture real-time surface data of products moving at high speeds on the production line. Deep learning algorithms are then used to analyze this data to detect and classify surface defects.

[0004] However, since products in most production lines move at high speeds, 2D cameras need to have very short exposure times to capture images of metal surfaces without motion blur. However, when the exposure time is very low, the camera's aperture will correspondingly become smaller, resulting in a significant reduction in the amount of light entering the camera, unclear imaging, and difficulty capturing many texture details, which in turn affects the detection rate. Summary of the Invention

[0005] Based on this, it is necessary to provide a metal surface defect recognition method, device and equipment to address the technical problem of low accuracy in collecting and detecting surface data of products using 2D cameras.

[0006] The present invention adopts the following technical solutions:

[0007] The present invention provides a metal surface defect identification method, comprising:

[0008] respectively acquiring first point cloud data of the metal surface profile during the processing and second point cloud data of the preset metal surface standard profile;

[0009] For each 3D point in the first point cloud data, determining the nearest neighbor point of the 3D point in the second point cloud data based on the distance between the 3D point and each 3D point in the second point cloud data, so as to align the metal surface profile during processing with the standard metal surface profile;

[0010] Determining a defective three-dimensional point set corresponding to the defect from the first point cloud data based on the distance between each three-dimensional point in the first point cloud data and its nearest neighbor point in the second point cloud data and a preset error threshold; when the distance between a three-dimensional point in the first point cloud data and its nearest neighbor point in the second point cloud data is greater than the preset error threshold, it indicates that the metal surface profile at the three-dimensional point is different from the standard metal surface profile and a defect exists;

[0011] The defect 3D point set is clustered by the distance clustering algorithm to obtain the defect 3D point corresponding to each defect;

[0012] The three-dimensional defect points corresponding to each defect are input into the pre-trained classifier to identify the type of each defect.

[0013] Optionally, clustering the defect three-dimensional point set by a distance clustering algorithm to obtain the defect three-dimensional point corresponding to each defect specifically includes:

[0014] For each defective 3D point in the defective 3D point set, determining neighboring points within a preset search radius of the defective 3D point;

[0015] Classify the neighboring points whose Euclidean distance to the defective 3D point is less than a preset clustering distance threshold and the defective 3D point into the same type of defective 3D points;

[0016] Determine neighboring points within a preset search radius of each similar defective 3D point among similar defective 3D points of the defective 3D point, and classify the neighboring points whose Euclidean distance to the similar defective 3D point is less than a preset clustering distance threshold with the defective 3D point as similar defective 3D points;

[0017] The final similar defect 3D point of the defect 3D point is determined through multiple rounds of iterative calculation until no new similar defect 3D point can be obtained, and the defect 3D point and its similar defect 3D points are determined to be defect points corresponding to one defect.

[0018] Optionally, determining the cluster distance threshold specifically includes:

[0019] determining a contour distance between a metal surface contour during processing and a standard contour of the metal surface based on the first point cloud data and the second point cloud data;

[0020] According to the contour spacing and the collection spacing between two adjacent points of the standard metal surface contour, the clustering distance threshold is determined by the following formula:

[0021] d = 2 × max (A, B);

[0022] Among them, d is the cluster distance threshold, A is the contour spacing, and B is the collection spacing.

[0023] Optionally, the method further includes:

[0024] For each 3D point in the first point cloud data, the error distance between the metal surface profile during processing and the standard metal surface profile at the 3D point is determined by the following formula based on the normal vector of the nearest neighbor point of the 3D point in the second point cloud data and the distance between the 3D point and its nearest neighbor point:

[0025] Er_Dis(a j )=nor(b i )×dis(a j , b i ), a j ∈J,b i ∈I;

[0026] Among them, J is the first point cloud data, a j is the jth 3D point in the first point cloud data, I is the second point cloud data, b i is the i-th 3D point in the second point cloud data, and the 3D point b i is the 3D point a in the second point cloud data j The nearest neighbor point, nor(b i ) is the 3D point b in the second point cloud data i The normal vector at dis(a j , b i ) is a three-dimensional point a j Its nearest neighbor 3D point b i The Euclidean distance, Er_Dis(a j ) is a three-dimensional point a j The error distance between the metal surface profile J and the metal surface standard profile I during the machining process.

[0027] Optionally, the step of inputting the three-dimensional defect points corresponding to each defect into a pre-trained classifier to identify the type of each defect specifically includes:

[0028] The combination of the three-dimensional coordinates of the defective three-dimensional point corresponding to each defect and the error distance between the metal surface profile during processing at the defective three-dimensional point and the standard metal surface profile is used as the representation of the defective three-dimensional point;

[0029] For each defect, the corresponding three-dimensional defect point is input into a pre-trained dynamic graph convolutional network as a classifier to extract the type geometric features of the defect, and the type of the defect is identified based on the extracted type geometric features.

[0030] Optionally, training the dynamic graph convolutional network specifically includes:

[0031] Acquire historically collected metal surface contour point cloud data as initial point cloud data, and perform continuous transformation operations on the initial point cloud data to construct a point cloud sequence, where the transformation operations include at least one of rotation, translation, and scaling;

[0032] Constructing a dynamic graph convolution online network and a dynamic graph convolution target network, obtaining first sample data corresponding to the dynamic graph convolution online network and second sample data corresponding to the dynamic graph convolution target network from the point cloud sequence, and performing spatial enhancement on the first sample data and the second sample data;

[0033] Inputting the spatially enhanced first sample data into the dynamic graph convolution online network for feature mapping to obtain features of the first sample data after being transformed by the dynamic graph convolution online network, and inputting the spatially enhanced second sample data into the dynamic graph convolution target network for feature mapping to obtain features of the second sample data after being transformed by the dynamic graph convolution target network;

[0034] By adding a predictor, the mean square error between the features of the first sample data after being transformed by the dynamic graph convolution online network and the features of the second sample data after being transformed by the dynamic graph convolution target network is calculated according to the following formula:

[0035]

[0036] The spatially enhanced second sample data is fed into the dynamic graph convolution online network, and the spatially enhanced first sample data is fed into the dynamic graph convolution target network. The loss in the mean square error is symmetrized and the total loss is calculated by the following formula:

[0037]

[0038] For each training step, the parameters of the dynamic graph convolution online network are updated with the goal of minimizing the total loss. After completing the step training, the parameters of the dynamic graph convolution target network are updated according to the moving average of the dynamic graph convolution online network parameters.

[0039] in, is the mean square error between the features of the first sample data after being transformed by the dynamic graph convolution online network and the features of the second sample data after being transformed by the dynamic graph convolution target network, r(z θ ) is the feature of the first sample data after being transformed by the dynamic graph convolution online network, is the feature of the second sample data after being transformed by the dynamic graph convolution target network, is the loss in the mean square error after symmetry, For the total loss.

[0040] Optionally, the method further includes:

[0041] Performing outlier filtering on the first point cloud data by using a filtering algorithm to remove outliers in the first point cloud data;

[0042] Determining the nearest neighbor point pairs of the metal surface profile during processing and the standard profile of the metal surface based on the first point cloud data and the second point cloud data;

[0043] Calculate the absolute error distance between the nearest neighbor points and determine whether the absolute error distance is less than the preset update threshold;

[0044] If so, the average value of the non-travel direction coordinate values of the first point cloud data and the second point cloud data during the processing is calculated, and the second point cloud data corresponding to the standard profile of the metal surface is updated;

[0045] If not, the second point cloud data corresponding to the standard profile of the metal surface is not updated using the first point cloud data.

[0046] Optionally, the method further includes:

[0047] When the nearest neighbor point of the standard contour of the metal surface cannot be found in the first point cloud data corresponding to the contour of the metal surface during processing, determining whether the amount of missing three-dimensional points in the first point cloud data corresponding to the contour of the metal surface during processing is greater than a preset missing threshold;

[0048] If yes, the second point cloud data corresponding to the standard profile of the metal surface is not updated using the first point cloud data;

[0049] If not, three-dimensional point interpolation is performed on the first point cloud data corresponding to the metal surface contour during the processing, and the nearest neighbor point of the metal surface standard contour is re-determined from the first point cloud data corresponding to the metal surface contour during the processing.

[0050] The present invention provides a metal surface defect recognition device, wherein each module in the metal surface defect recognition device is used to execute the above-mentioned metal surface defect recognition method.

[0051] The present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the metal surface defect identification method described above is implemented.

[0052] The present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned metal surface defect identification method when executing the program.

[0053] The above technical solution adopted by the present invention can achieve the following beneficial effects:

[0054] First, the point cloud data corresponding to the preset metal surface standard contour and the metal surface contour during processing are obtained. Then, template matching is performed on the two contours based on the point cloud data of the two to determine the closest 3D point pairs in the two contours. Then, the defect 3D point set in the metal surface contour during processing is determined based on the distance between the 3D point pairs and a distance clustering analysis is performed on it to obtain the defect 3D points corresponding to each defect. Finally, the type of each defect is identified based on the classifier.

[0055] The present invention uses three-dimensional point cloud data to represent the metal surface contour during the processing process. The collection of point cloud data avoids the problem of poor metal surface image clarity caused by exposure of 2D cameras. At the same time, template matching of the metal surface contour and the standard contour during the processing is performed based on the point cloud data, and the defective three-dimensional points can be clearly compared. After distance clustering of the defective three-dimensional points, point cloud data corresponding to the defective area is further obtained. Compared with the image data collected by the 2D camera, the point cloud data corresponding to the defective area has richer information, thereby improving the accuracy of metal surface defect recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0057] Figure 1 A schematic flow chart of a metal surface defect identification method provided by the present invention;

[0058] Figure 2 A schematic diagram of a point cloud profile image of a moving hot-rolled rail captured by a 3D line laser profiler provided by the present invention;

[0059] Figure 3 A schematic diagram of comparison before and after filtering the first point cloud data using a filtering algorithm provided by the present invention;

[0060] Figure 4 A schematic diagram of comparison before and after interpolation of first point cloud data provided by the present invention;

[0061] Figure 5 A schematic diagram of a point cloud self-supervised representation learning network structure provided by the present invention;

[0062] Figure 6 A schematic diagram of a comprehensive process for identifying metal surface defects provided by the present invention;

[0063] Figure 7This is a schematic diagram of a metal surface defect identification device provided by the present invention. DETAILED DESCRIPTION

[0064] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present invention and corresponding drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0065] Currently, because products in most production lines move at high speeds, 2D cameras need to have very short exposure times to capture images of metal surfaces without motion blur. However, when the exposure time is very low, the camera's aperture becomes correspondingly smaller, significantly reducing the amount of light entering the camera, resulting in unclear images and difficulty capturing many texture details, which in turn affects the detection rate. Furthermore, 2D cameras cannot obtain depth information on the metal surface. For metal surface defects, the depth difference between the defect and the surface is a key feature and is of great reference value for defect classification and grading. The lack of this information leads to poor defect classification accuracy and difficulty in grading defects.

[0066] The present invention discloses a method for identifying metal surface defects. Taking into account the shortcomings of the above-mentioned metal surface defect detection based on 2D vision, the method of the present invention does not use a 2D camera to collect metal surface data, but uses a 3D line laser profiler to collect metal surface data. The 3D line laser profiler has the advantages of being less affected by light, the point cloud imaging effect is less affected by metal movement, and can accurately capture the depth information of the metal surface. Accordingly, the present invention also proposes a metal surface defect recognition algorithm that is compatible with the 3D data acquisition solution. Specifically, due to the high real-time requirements of industrial detection and the large amount of metal surface point cloud data, considering the running speed problem, the present invention does not use 3D object detection or segmentation methods to detect and classify defects, but first uses the template matching algorithm to detect defect points on the point cloud contour data, and then uses the distance clustering algorithm to cluster defects, and then sends the clustered defect data to the deep learning model Dynamic Graph Convolutional Neural Network (Dynamic Graph CNN, DGCNN) for specific defect point recognition and classification.

[0067] The technical solutions provided by various embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0068] Figure 1 The following is a flow chart of a metal surface defect identification method according to the present invention, which specifically includes the following steps:

[0069] S101: Acquire first point cloud data of a metal surface profile during processing and second point cloud data of a preset metal surface standard profile.

[0070] Generally, during the rolling process of metal materials, the metal material can be processed at high speed along the production line in the direction of travel and through various steps. During this process, the surface information of the metal material can be collected by a pre-installed 3D laser profiler to obtain point cloud data corresponding to the metal material surface.

[0071] Therefore, the server of the business platform can obtain the first point cloud data J of the metal surface contour during the processing process collected by the 3D laser profiler, and obtain the second point cloud data I of the preset metal surface standard contour, so as to subsequently use the metal surface standard contour as a template to perform template matching on the metal surface contour during the processing process to determine the data information corresponding to the metal surface defects.

[0072] Among them, the preset metal surface standard profile can be set according to the detection user experience. Of course, it can also be converted into point cloud data as the metal surface standard profile through a Computer-Aided Design (CAD) model. The preset metal surface standard profile can be pre-stored in the storage device of the server and obtained from the storage device when executing the metal surface defect recognition method.

[0073] Figure 2 This is a schematic diagram of a point cloud profile image of a moving hot-rolled rail captured by a 3D line laser profiler in the present invention.

[0074] Furthermore, during the rolling process, metal materials are often subject to small deformations such as thermal expansion due to heat and other process factors. Taking thermal expansion as an example, thermal expansion of metal materials during processing is usually normal. However, since the point cloud data corresponding to the CAD model is an absolute standard contour, there is a difference between the metal surface contour after thermal expansion during the actual processing process. This may lead to the misidentification of defect points when the standard metal surface contour is used as a template for template matching of the metal surface contour during processing.

[0075] Therefore, in one or more embodiments of the present invention, after acquiring the first point cloud data and the second point cloud data, the server may also update the standard profile of the metal surface in real time based on the metal surface profile during processing according to the first point cloud data and the second point cloud data.

[0076] Specifically, the server may first filter the first point cloud data using a filtering algorithm to remove outliers. For metal surface contours during processing, the shooting environment and changes in the metal surface may cause 3D point displacement, missing points, or the presence of noise at certain locations in the point cloud data. Therefore, the server may first filter the first point cloud data to remove abnormal outliers. Figure 3 2 is a schematic diagram showing a comparison before and after filtering the first point cloud data using a filtering algorithm in the present invention. Figure 3 The left side of the figure shows the metal surface contour during machining, corresponding to the first point cloud data before filtering, and the right side shows the metal surface contour during machining, corresponding to the first point cloud data after filtering. It can be seen that the outliers within the rectangular box are removed after filtering.

[0077] Then, the server can determine the nearest neighbor point pair of the metal surface profile and the metal surface standard profile during the processing based on the first point cloud data and the second point cloud data. Assume that the first point cloud data is J = {a j}, a j =(x j ,y j , z j ), the second point cloud data is I={b i}, b i =(x i ,y i , z i The server may first determine the point closest to the second point cloud data I from the first point cloud data J, and then determine the point closest to the first point cloud data J from the second point cloud data I. These two points are the nearest neighbor point pair of the metal surface profile and the standard metal surface profile during the processing.

[0078] Finally, the server can calculate the absolute error distance between the nearest neighbor point pairs and determine whether the absolute error distance is less than a preset update threshold. Here, the absolute error distance between the nearest neighbor point pairs represents the degree of difference between the metal surface profile during processing and the standard metal surface profile. The absolute error distance between the nearest neighbor point pairs is positively correlated with the degree of difference between the metal surface profile during processing and the standard metal surface profile. The preset update threshold can be set according to business needs and is not limited in this invention.

[0079] If so, the average value of the non-movement direction coordinate values of the first point cloud data and the second point cloud data during the processing is calculated, and the second point cloud data corresponding to the standard profile of the metal surface is updated. When the absolute error distance is less than the preset update threshold, it means that the difference between the metal surface profile and the standard profile of the metal surface during the processing is small. At this time, the standard profile of the metal surface can be dynamically updated based on the metal surface profile during the processing. Assuming that the metal material moves along the y-axis during the processing, the x-axis coordinate value and the z-axis coordinate value of each three-dimensional point of the second point cloud data corresponding to the standard profile of the metal surface should fluctuate within a certain range. At this time, the server can calculate the average value of the x-axis coordinate value and the z-axis coordinate value between each corresponding three-dimensional point pair of the metal surface profile and the metal surface standard profile during the processing, and update the x-axis coordinate value and the z-axis coordinate value of the three-dimensional point in the point cloud data of the standard profile of the metal surface according to the average value.

[0080] If not, the second point cloud data corresponding to the standard metal surface profile will not be updated using the first point cloud data. When the absolute error distance is greater than or equal to the preset update threshold, it indicates that the difference between the metal surface profile during processing and the standard metal surface profile is significant. In this case, the dynamic update of the standard metal surface profile based on the metal surface profile during processing can be abandoned.

[0081] By dynamically updating the standard profile of the metal surface based on the metal surface profile during the processing corresponding to the first point cloud data, the interference of the normal impact of the processing procedure on the metal surface on the identification of metal surface defects is reduced, and the impact of defect identification caused by the jitter of the production equipment can be at least partially eliminated, thereby improving the accuracy of the defects of the metal surface profile during the subsequent identification process.

[0082] The server mentioned in the present invention can be a server set up on a business platform, or a device such as a desktop computer, a notebook computer, etc. that can execute the solution of the present invention. For the sake of convenience, the following description will only be made with the server as the execution subject.

[0083] S102: For each 3D point in the first point cloud data, determine the nearest neighbor point of the 3D point in the second point cloud data based on the distance between the 3D point and each 3D point in the second point cloud data, so as to match and align the metal surface contour during processing with the standard metal surface contour.

[0084] S103: Determine a defective three-dimensional point set corresponding to the defect from the first point cloud data based on the distance between each three-dimensional point in the first point cloud data and its nearest neighbor point in the second point cloud data and a preset error threshold; when the distance between a three-dimensional point in the first point cloud data and its nearest neighbor point in the second point cloud data is greater than the preset error threshold, it indicates that the metal surface profile at the three-dimensional point is different from the standard profile of the metal surface and there is a defect.

[0085] S104: Clustering the defect three-dimensional point set using a distance clustering algorithm to obtain a three-dimensional defect point corresponding to each defect.

[0086] S105: Input the three-dimensional defect points corresponding to each defect into a pre-trained classifier based on a neural network model to identify the type of each defect.

[0087] After acquiring the first point cloud data of the metal surface contour during processing and the second point cloud data of the preset metal surface standard contour, the server can perform template matching on the metal surface contour during processing based on the metal surface standard contour.

[0088] First, the server may determine, for each 3D point in the first point cloud data, the nearest neighbor point of the 3D point in the second point cloud data according to the distance between the 3D point and each 3D point in the second point cloud data.

[0089] The server can then determine the defective 3D point set corresponding to the defect from the first point cloud data based on the distance between each 3D point in the first point cloud data and its nearest neighbor and a preset error threshold. That is, the metal surface contour and the standard metal surface contour during processing are matched. Here, the distance between a 3D point in the first point cloud data and its nearest neighbor represents the degree of match between the two contours at that point. The distance between a 3D point in the first point cloud data and its nearest neighbor is negatively correlated with the degree of match between the two contours at that 3D point. That is, the higher the degree of match between the two contours at a certain 3D point, the more the two contours overlap at that 3D point, and the smaller the distance between the 3D point and its nearest neighbor should be, and vice versa. The error threshold here can be set specifically as needed, and the present invention does not impose any restrictions on this.

[0090] For example, the server can set the error threshold to α, then the defect 3D point set is:

[0091] error_point={a j , dis>α}

[0092] Among them, error_point is the defect three-dimensional point set, a j is a 3D point in the first point cloud data, and dis is the distance between the 3D point in the first point cloud data and its nearest neighbor in the second point cloud data.

[0093] The server can then cluster the defect 3D point set using a distance clustering algorithm to obtain the 3D defect point corresponding to each defect. The distance clustering algorithm here can be a Euclidean distance clustering algorithm or a DBCAN distance clustering algorithm, etc. The specific distance clustering algorithm used is not limited by the present invention.

[0094] Taking the Euclidean distance clustering algorithm as an example, in one or more embodiments of the present invention, the server may first determine the neighboring points within a preset search radius for each defective 3D point in the defective 3D point set. The neighboring points whose Euclidean distance from the defective 3D point is less than a preset clustering distance threshold are then grouped together with the defective 3D point as similar defective 3D points. Next, the neighboring points within the preset search radius of each similar defective 3D point are determined, and the neighboring points whose Euclidean distance from the similar defective 3D point is less than a preset clustering distance threshold are grouped together with the defective 3D point as similar defective 3D points. Thus, the final similar defective 3D point for the defective 3D point is determined through multiple rounds of iterative calculations until no new similar defective 3D points can be obtained, and the defective 3D point and its similar defective 3D points are determined to be the defective 3D points corresponding to one defect. By using the Euclidean distance clustering algorithm, the defective 3D point corresponding to each defect can be quickly and easily determined.

[0095] Furthermore, in one or more embodiments of the present invention, the server may determine the above clustering distance threshold through the following steps.

[0096] The server may first determine the contour distance between the metal surface contour during processing and the standard metal surface contour based on the first point cloud data and the second point cloud data. Then, the server may determine the clustering distance threshold based on the contour distance and the acquisition distance between two adjacent points of the standard metal surface contour using the following formula:

[0097] d = 2 × max(A, B)

[0098] Where d is the clustering distance threshold, A is the contour spacing, and B is the acquisition spacing. By setting an appropriate clustering distance threshold, the clustering accuracy of the 3D defect points for each defect is further improved.

[0099] Finally, the server can input the 3D points corresponding to each defect into a pre-trained neural network-based classifier, which can then identify the type of each defect. The classifier can be a mature classifier algorithm model, such as a Support Vector Machine (SVM) model, a Dynamic Graph Convolutional Network (LDGCNN) model, or a Linked Dynamic Graph Convolutional Network (LDGCNN) model. The specific classification model used can be determined based on needs and is not limited by the present invention.

[0100] Taking the DGCNN model as an example, in one or more embodiments of the present invention, the server can input the three-dimensional defect points corresponding to each defect into a pre-trained DGCNN model, extract the type geometric features of the defect, and identify the type of the defect based on the extracted type geometric features.

[0101] The dynamic graph convolutional network is a convolutional neural network that can directly extract features from point clouds and perform classification tasks. Its core module is the 3D graph convolution layer. i And the neighbor of this point {a j : (i, j)∈ε}, then after the mth 3D graph convolution layer:

[0102]

[0103] Among them, θ m and φ m is the learnable parameter of the mth 3D graph convolutional layer, updated during network training. A dynamic convolutional network is constructed by stacking 3D graph convolutional layers. In a dynamic graph convolutional network, the neighbors of fixed nodes in each layer dynamically change based on the distance metric, dynamically generating the graph structure of each layer. This allows the network to maintain point cloud arrangement invariance and effectively capture the local geometric features of the point cloud, improving the model's ability to express and describe point cloud structure and enhance defect recognition.

[0104] based on Figure 1 The metal surface defect recognition method shown in the figure first obtains a preset metal surface standard profile and a metal surface profile during processing collected by a 3D laser profiler, then performs template matching on the two profiles to determine the closest three-dimensional point pairs in the two profiles, and then determines the defect three-dimensional point set in the metal surface profile during processing based on the distance between the three-dimensional point pairs and performs distance clustering analysis on it, thereby obtaining the defect three-dimensional points corresponding to each defect, and finally identifies the type of each defect based on a classifier.

[0105] The present invention uses a 3D laser profiler to collect the metal surface contour during the processing process, thereby avoiding the problem of poor metal surface image clarity caused by exposure of a 2D camera. At the same time, based on the point cloud data, template matching of the metal surface contour and the standard contour during the processing process is performed, so that the three-dimensional defect points can be clearly compared and obtained. After distance clustering of the three-dimensional defect points, point cloud data corresponding to the defect area is further obtained. Compared with the image data collected by a 2D camera, the point cloud data corresponding to the defect area has richer information, thereby improving the accuracy of metal surface defect recognition.

[0106] When applying the metal surface defect identification method provided by the present invention, it is not necessary to Figure 1The steps are executed in the order shown. The specific execution order of the steps can be determined according to needs, and the present invention does not limit this.

[0107] Furthermore, in one or more embodiments of the present invention, when the nearest neighbor of the standard metal surface profile cannot be found in the first point cloud data corresponding to the metal surface profile during processing, the server may determine that there are missing three-dimensional points in the metal surface profile during processing. In this case, the server may further analyze the missing points, namely, determine whether the amount of missing three-dimensional points in the first point cloud data corresponding to the metal surface profile during processing is greater than a preset missing threshold. The preset missing threshold can be set specifically based on business needs and is not limited by the present invention.

[0108] If so, the second point cloud data corresponding to the standard metal surface profile is not updated using the first point cloud data. When the number of missing 3D points in the first point cloud data exceeds a preset missing threshold, the server may determine that the first point cloud data has a serious lack of 3D points. In this case, the server may abandon updating the second point cloud data corresponding to the standard metal surface profile using the first point cloud data.

[0109] If not, three-dimensional point interpolation is performed on the first point cloud data corresponding to the metal surface contour during processing, and the nearest neighbor points of the standard metal surface contour are re-determined from the first point cloud data corresponding to the metal surface contour during processing. If the number of missing three-dimensional points in the first point cloud data is less than or equal to a preset missing threshold, the server may determine that some areas of the first point cloud data may have sparse three-dimensional points. In this case, the server may use an interpolation algorithm to interpolate the metal surface contour during processing.

[0110] Assume that for two points M(x1, z1) and N(x2, z2) in the first point cloud data corresponding to the metal surface contour during a certain frame processing process, where there are three-dimensional point missing areas, the server can first calculate the coordinate difference between the two points using the following formula:

[0111] nor=(x1-x2)+(z1-z2)

[0112] Then, the server can calculate the distance difference between points A and B using the following formula:

[0113]

[0114] Of course, the server can further calculate the unit coordinate difference using the following formula:

[0115] nor_d=nor / dis

[0116] Afterwards, the server may determine that if dis is greater than a preset interpolation threshold (e.g., 0.2 mm), no interpolation processing is performed. Otherwise, the server may perform interpolation according to the following formula:

[0117] Insertion point coordinates = M + nor_d * 0.05

[0118] That is, the server can insert a 3D point every 0.05nor_d. Figure 4 is a schematic diagram of comparison before and after interpolation of the first point cloud data in the present invention, Figure 4 The left side of the middle diagram shows the first point cloud data before interpolation. It can be seen that there are missing 3D points in the three rectangular boxes. Figure 4 The middle right side is an illustration of the interpolation of the first point cloud data. It can be seen that the 3D points in the three rectangular box areas corresponding to the left are supplemented, and there is no longer any missing 3D points.

[0119] In addition, since metal materials are three-dimensional materials, they may have abnormal defects such as dents, scratches, and protrusions. The point cloud data collected by the 3D laser profiler is three-dimensional data. When performing template matching based on the standard profile of the metal surface and the profile of the metal surface during processing, it may be difficult to determine the specific defect only by the distance between the corresponding three-dimensional point pairs.

[0120] Therefore, in one or more embodiments of the present invention, before performing the distance cluster analysis in step S103, the server may further determine, for each 3D point in the first point cloud data, the error distance between the metal surface profile during processing and the standard metal surface profile at the 3D point based on the normal vector of the nearest neighbor of the 3D point in the second point cloud data and the distance between the 3D point and its nearest neighbor, using the following formula:

[0121] Er_Dis(a j )=nor(b i )×dis(a j , b i )

[0122] a j ∈J,b i ∈I

[0123] Among them, J is the first point cloud data, a j is the jth 3D point in the first point cloud data, I is the second point cloud data, b i is the i-th 3D point in the second point cloud data, and the 3D point b i is the 3D point a in the second point cloud data j The nearest neighbor point, nor(b i ) is the 3D point b in the second point cloud data i The normal vector at dis(a j , b i ) is a three-dimensional point a j Its nearest neighbor 3D point b iThe Euclidean distance, Er_Dis(a j ) is a three-dimensional point a j The error distance between the metal surface profile J and the standard metal surface profile I during machining is calculated. The error distance between the 3D point pairs of the two profiles can be used to further determine the error direction of the 3D defect points in the metal surface profile during machining, thereby facilitating the determination of whether the defect corresponds to a convex or concave form.

[0124] Furthermore, in one or more embodiments of the present invention, the server may represent each defect by combining the 3D coordinates of the defect point corresponding to each defect and the error distance between the metal surface contour during machining and the standard metal surface contour at that point. Then, for each defect, the corresponding 3D defect point is input into a pre-trained dynamic graph convolutional network to identify the defect type. Incorporating the error distance between the metal surface contour during machining and the standard metal surface contour further improves the accuracy of metal surface defect identification.

[0125] In addition, due to the small amount of point cloud defect data and unclear defect features, directly training the classifier model may have shortcomings such as poor recognition accuracy and generalization.

[0126] Therefore, in one or more embodiments of the present invention, the server may perform distance clustering analysis on the first point cloud data of the metal surface profile during multiple frames of processing, thereby obtaining a relatively dense point cloud data representation of each defect. For example, distance clustering analysis may be performed on the first point cloud data of the metal surface profile during every 1,000 frames of processing.

[0127] Furthermore, in one or more embodiments of the present invention, the server may utilize the method of Self-supervised Representation Learning for Point Clouds (STRL) to improve the representation learning and generalization capabilities of the model, using a dynamic graph convolutional network as an example.

[0128] Specifically, the server may first obtain historically collected metal surface contour point cloud data as initial point cloud data, and perform continuous transformation operations on the initial point cloud data to construct a point cloud sequence, where the transformation operations include at least one of rotation, translation, and scaling.

[0129] Then, the server can construct a dynamic graph convolution online network and a dynamic graph convolution target network, obtain first sample data corresponding to the dynamic graph convolution online network from the point cloud sequence, and obtain second sample data corresponding to the dynamic graph convolution target network, and perform spatial enhancement on the first sample data and the second sample data.

[0130] Secondly, the server can input the spatially enhanced first sample data into the dynamic graph convolution online network for feature mapping to obtain the features of the first sample data after transformation by the dynamic graph convolution online network, and input the spatially enhanced second sample data into the dynamic graph convolution target network for feature mapping to obtain the features of the second sample data after transformation by the dynamic graph convolution target network.

[0131] Next, the server can calculate the mean square error between the features of the first sample data after being transformed by the dynamic graph convolution online network and the features of the second sample data after being transformed by the dynamic graph convolution target network according to the following formula by using the additional predictor:

[0132]

[0133] Afterwards, the server can feed the spatially enhanced second sample data into the dynamic graph convolution online network, and feed the spatially enhanced first sample data into the dynamic graph convolution target network, symmetrize the loss in the mean square error, and calculate the total loss using the following formula:

[0134]

[0135] In each training step, the parameters of the dynamic graph convolution online network are updated with the goal of minimizing the total loss. After completing the step training, the parameters of the dynamic graph convolution target network are updated according to the moving average of the dynamic graph convolution online network parameters.

[0136] in, is the mean square error between the features of the first sample data after being transformed by the dynamic graph convolution online network and the features of the second sample data after being transformed by the dynamic graph convolution target network, r(z θ ) is the feature of the first sample data after being transformed by the dynamic graph convolution online network, is the feature of the second sample data after being transformed by the dynamic graph convolution target network, is the loss in the mean square error after symmetry, For the total loss.

[0137] For example, the server can form a point cloud continuous change sequence based on a point cloud p0 by continuously rotating, translating, and scaling the point cloud. Where, the superscript u represents the first sample data of the corresponding dynamic graph convolution online network, the superscript v represents the second sample data of the corresponding dynamic graph convolution target network, {P} is a set of 3D point cloud sequences, Γ and E are the temporal sampler and spatial enhancement function. Then, this series of point clouds is spatially enhanced by random cropping, random jittering, downsampling, etc. to generate a series of online point cloud datasets (transform data): a u, a v ∈E,x u =a u (p u ), x t =a v (p v ), where x u As the input of the dynamic graph convolution online network, a u is the spatial enhancement function corresponding to the dynamic graph convolution online network, p u is the point cloud dataset corresponding to the dynamic graph convolution online network, x t As the input of the dynamic graph convolution target network, a v is the spatial enhancement function corresponding to the dynamic graph convolution target network, p v This is the point cloud dataset corresponding to the dynamic graph convolution target network. The DGCNN (excluding the MLP layer) dynamic graph convolution method is used as the encoder. g θ , followed by a multi-layer perceptron (MLP) with activation and batch normalization for spatial projection f θ , Preferably, since most defect data are small, the cropping jitter ratio can be set to 0.2. When the number of original point cloud data does not meet the downsampling setting parameters, the server can use repeated random sampling to meet the same number of data points. The predictor r is implemented by a linear layer with activation and batch normalization, and the dynamic graph convolution target network parameters are obtained by the dynamic graph convolution online network parameters. To update, where τ∈[0,1] is the decay rate of the moving average, which is related to the number of model training rounds. The higher the training round, the smaller the decay rate. is the target network parameter of the dynamic graph convolution, and θ is the online network parameter of the dynamic graph convolution. Table 1 is a comparative table showing whether self-supervised representation learning is adopted in the present invention.

[0138] Table 1 Comparison results

[0139] method Acc Directly train DGCNN 89.72％ Self-supervised representation learning + DGCNN 94.03％

[0140] Furthermore, in one or more embodiments of the present invention, the distance clustering algorithm may also employ DBCAN distance clustering. The neural network-based classifier for defect type identification may also employ machine learning algorithms such as SVM or deep learning-related models such as LDGCNN. Self-supervised representation learning methods may also employ deep-clustering and other approaches. This is not a limitation of the present invention, and specific methods may be determined based on actual needs.

[0141] Figure 5 This is a schematic diagram of a point cloud self-supervised representation learning network structure in the present invention. Figure 5 For each part, please refer to the corresponding description above.

[0142] Figure 6 This is a schematic diagram of a comprehensive process for identifying metal surface defects in the present invention. Figure 6 It can be seen that the server can dynamically update the standard contour based on the real-time acquisition of each frame of 3D contour data, dynamically generate the second point cloud data of the standard contour of the metal surface, and then perform template matching on the metal surface contour and the standard contour of the metal surface during the processing to determine the defective three-dimensional points in the first point cloud data. Then, cluster analysis is performed based on the first point cloud data of the metal surface contour during every thousand frames of processing to obtain the defective three-dimensional points corresponding to each defect. Finally, the point cloud data of each defect is classified through the classification model and the classification results are output.

[0143] The present invention does not use the traditional 2D method to detect steel defects, but uses a 3D defect detection method to avoid the impact of motion blur caused by high-resolution images on defect detection. Furthermore, instead of using traditional 3D object detection or segmentation methods for defect detection, a template matching algorithm is used to eliminate the real-time problem caused by the model operation rate. Instead of using the traditional CAD model to convert into point cloud data as a template, a dynamic template generation method is used to reduce the impact of thermal expansion of hot rails and irregular effects of point cloud data caused by other factors. The directional distance algorithm can also be used to calculate the positive and negative difference between the defect and the template, allowing the classification model to obtain more information. Furthermore, due to the difficulty in labeling defect data sets and the small amount of data, self-supervised representation learning can also be used to improve model learning and generalization capabilities.

[0144] The above is a metal surface defect recognition method provided by one or more embodiments of the present invention. Based on the same idea, the present invention also provides a corresponding metal surface defect recognition device, such as Figure 7 shown.

[0145] Figure 7 A schematic diagram of a metal surface defect identification device provided by the present invention, comprising:

[0146] An acquisition module 201 is used to respectively acquire first point cloud data of a metal surface profile during processing and second point cloud data of a preset standard metal surface profile;

[0147] a determination module 202 for determining, for each 3D point in the first point cloud data, a nearest neighbor point of the 3D point in the second point cloud data based on the distance between the 3D point and each 3D point in the second point cloud data, so as to align the metal surface profile during processing with the standard metal surface profile;

[0148] The defect determination module 203 is configured to determine a defective three-dimensional point set corresponding to a defect from the first point cloud data based on the distance between each three-dimensional point in the first point cloud data and its nearest neighbor point in the second point cloud data and a preset error threshold; when the distance between a three-dimensional point in the first point cloud data and its nearest neighbor point in the second point cloud data is greater than the preset error threshold, it is determined that the metal surface profile at that three-dimensional point is different from the standard metal surface profile and a defect exists;

[0149] The clustering module 204 is used to cluster the defect three-dimensional point set using a distance clustering algorithm to obtain a three-dimensional defect point corresponding to each defect.

[0150] Identification module 205 is used to input the three-dimensional defect points corresponding to each defect into the pre-trained classifier to identify the type of each defect

[0151] Optionally, the clustering module 204 determines, for each defective 3D point in the defective 3D point set, neighboring points within a preset search radius of the defective 3D point, and classifies the neighboring points whose Euclidean distance to the defective 3D point is less than a preset clustering distance threshold as similar defective 3D points together with the defective 3D point, determines, among similar defective 3D points of the defective 3D point, neighboring points within a preset search radius of each similar defective 3D point, and classifies the neighboring points whose Euclidean distance to the similar defective 3D point is less than a preset clustering distance threshold as similar defective 3D points together with the defective 3D point, and determines the final similar defective 3D point of the defective 3D point through multiple rounds of iterative calculation until no new similar defective 3D point can be obtained, and determines that the defective 3D point and its similar defective 3D points are defective 3D points corresponding to one defect.

[0152] Optionally, the clustering module 203 determines the contour spacing between the metal surface contour and the standard contour of the metal surface during the processing process based on the first point cloud data and the second point cloud data, and determines the clustering distance threshold according to the contour spacing and the acquisition spacing between two adjacent points of the standard contour of the metal surface by the following formula: d = 2×max(A, B), where d is the clustering distance threshold, A is the contour spacing, and B is the acquisition spacing.

[0153] Optionally, the clustering module 203 determines, for each 3D point in the first point cloud data, the error distance between the metal surface profile during processing and the standard metal surface profile at the 3D point according to the normal vector of the nearest neighbor point of the 3D point in the second point cloud data and the distance between the 3D point and its nearest neighbor point by the following formula: Er_Dis(a j )=nor(b i )×dis(a j , b i ), a j ∈J,b i ∈I, where J is the first point cloud data, aj is the jth 3D point in the first point cloud data, I is the second point cloud data, b i is the i-th 3D point in the second point cloud data, and the 3D point b i is the 3D point a in the second point cloud data j The nearest neighbor point, nor(b i ) is the 3D point b in the second point cloud data i The normal vector at dis(a j , b i ) is a three-dimensional point a j Its nearest neighbor 3D point b i The Euclidean distance, Er_Dis(a j ) is a three-dimensional point a j The error distance between the metal surface profile J and the metal surface standard profile I during the machining process.

[0154] Optionally, the identification module 205 uses the combination of the three-dimensional coordinates of the defective three-dimensional point corresponding to each defect and the error distance between the metal surface contour and the standard contour of the metal surface during the processing at the defective three-dimensional point as the representation of the defective three-dimensional point. For each defect, the defective three-dimensional point corresponding to the defect is input into a pre-trained dynamic graph convolutional network to extract the type geometric features of the defect, and the type of the defect is identified based on the extracted type geometric features.

[0155] Optionally, the device further includes a training module for obtaining historically collected metal surface contour point cloud data as initial point cloud data, performing continuous transformation operations on the initial point cloud data to construct a point cloud sequence, where the transformation operations include at least one of rotation, translation, and scaling, constructing a dynamic graph convolution online network and a dynamic graph convolution target network, obtaining first sample data corresponding to the dynamic graph convolution online network from the point cloud sequence, and obtaining second sample data corresponding to the dynamic graph convolution target network, and spatially enhancing the first sample data and the second sample data, inputting the spatially enhanced first sample data into the dynamic graph convolution online network for feature mapping to obtain features of the first sample data after transformation by the dynamic graph convolution online network, and inputting the spatially enhanced second sample data into the dynamic graph convolution target network for feature mapping to obtain features of the second sample data after transformation by the dynamic graph convolution target network, and calculating, by an additional predictor, the mean square error between the features of the first sample data after transformation by the dynamic graph convolution online network and the features of the second sample data after transformation by the dynamic graph convolution target network according to the following formula: The spatially enhanced second sample data is fed into the dynamic graph convolution online network, and the spatially enhanced first sample data is fed into the dynamic graph convolution target network. The loss in the mean square error is symmetrized and the total loss is calculated by the following formula: For each training step, the parameters of the dynamic graph convolution online network are updated with the goal of minimizing the total loss. After completing the step training, the parameters of the dynamic graph convolution target network are updated according to the moving average of the dynamic graph convolution online network parameters, where: is the mean square error between the features of the first sample data after being transformed by the dynamic graph convolution online network and the features of the second sample data after being transformed by the dynamic graph convolution target network, r(z θ ) is the feature of the first sample data after being transformed by the dynamic graph convolution online network, is the feature of the second sample data after being transformed by the dynamic graph convolution target network, is the loss in the mean square error after symmetry, For the total loss.

[0156] Optionally, the acquisition module 201 filters outliers on the first point cloud data through a filtering algorithm to remove outliers in the first point cloud data, determines the nearest neighbor point pairs of the metal surface contour and the metal surface standard contour during the processing process based on the first point cloud data and the second point cloud data, calculates the absolute error distance between the nearest neighbor point pairs, and determines whether the absolute error distance is less than a preset update threshold. If so, calculates the average value of the non-travel direction coordinate values of the first point cloud data and the second point cloud data during the processing process, and updates the second point cloud data corresponding to the metal surface standard contour. If not, the second point cloud data corresponding to the metal surface standard contour is not updated through the first point cloud data.

[0157] Optionally, when the nearest neighbor point of the standard contour of the metal surface cannot be found in the first point cloud data corresponding to the metal surface contour during the processing, the acquisition module 201 determines whether the amount of missing three-dimensional points in the first point cloud data corresponding to the metal surface contour during the processing is greater than a preset missing threshold. If so, the second point cloud data corresponding to the standard contour of the metal surface is not updated through the first point cloud data; if not, three-dimensional point interpolation is performed on the first point cloud data corresponding to the metal surface contour during the processing, and the nearest neighbor point of the standard contour of the metal surface is re-determined from the first point cloud data corresponding to the metal surface contour during the processing.

[0158] The specific definitions of the metal surface defect identification device can be found in the definitions of the metal surface defect identification method above and will not be repeated here. The various modules in the above-mentioned metal surface defect identification device can be implemented in whole or in part through software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor of the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each of the above modules.

[0159] The present invention also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 A metal surface defect identification method is provided.

[0160] The present invention also provides a computer device. At the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory. Of course, it may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 A metal surface defect identification method is provided.

[0161] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0162] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present invention.

Claims

1. A method for identifying metal surface defects, characterized in that: include: respectively acquiring first point cloud data of the metal surface profile during the processing and second point cloud data of the preset metal surface standard profile; The first point cloud data is collected by a 3D laser profiler; For each 3D point in the first point cloud data, determining the nearest neighbor point of the 3D point in the second point cloud data based on the distance between the 3D point and each 3D point in the second point cloud data, so as to align the metal surface profile during processing with the standard metal surface profile; Determining a defective three-dimensional point set corresponding to the defect from the first point cloud data based on the distance between each three-dimensional point in the first point cloud data and its nearest neighbor point in the second point cloud data and a preset error threshold; when the distance between a three-dimensional point in the first point cloud data and its nearest neighbor point in the second point cloud data is greater than the preset error threshold, it indicates that the metal surface profile at the three-dimensional point is different from the standard metal surface profile and a defect exists; For each defective 3D point in the defective 3D point set, determining neighboring points within a preset search radius of the defective 3D point; Classify the neighboring points whose Euclidean distance to the defective 3D point is less than a preset clustering distance threshold and the defective 3D point into the same type of defective 3D points; Determine similar defect 3D points of the defect 3D point, and within a preset search radius of each similar defect 3D point, classify the neighboring points whose Euclidean distance to the similar defect 3D point is less than a preset clustering distance threshold as similar defect 3D points together with the defect 3D point; determine the final similar defect 3D point of the defect 3D point through multiple rounds of iterative calculation until no new similar defect 3D point can be obtained, and determine that the defect 3D point and its similar defect 3D points are defect 3D points corresponding to one defect; The three-dimensional defect points corresponding to each defect are input into the pre-trained classifier to identify the type of each defect.

2. The metal surface defect identification method according to claim 1, characterized in that: Determining the cluster distance threshold specifically includes: determining a contour distance between a metal surface contour during processing and a standard contour of the metal surface based on the first point cloud data and the second point cloud data; According to the contour spacing and the collection spacing between two adjacent points of the standard metal surface contour, the clustering distance threshold is determined by the following formula: d = 2 × max (A, B); Among them, d is the cluster distance threshold, A is the contour spacing, and B is the collection spacing.

3. The metal surface defect identification method according to claim 1, characterized in that: The method further comprises: For each 3D point in the first point cloud data, the error distance between the metal surface profile during processing and the standard metal surface profile at the 3D point is determined by the following formula based on the normal vector of the nearest neighbor point of the 3D point in the second point cloud data and the distance between the 3D point and its nearest neighbor point: Er_Dis(a j )=nor(b i )×dis(a j ,b i ),a j ∈J,b i ∈I; Among them, J is the first point cloud data, a j is the jth 3D point in the first point cloud data, I is the second point cloud data, b i is the i-th 3D point in the second point cloud data, and the 3D point b i is the 3D point a in the second point cloud data j The nearest neighbor point, nor(b i ) is the 3D point b in the second point cloud data i The normal vector at dis(a j ,b i ) is a three-dimensional point a j Its nearest neighbor 3D point b i The Euclidean distance, Er_Dis(a j ) is a three-dimensional point a j The error distance between the metal surface profile J and the metal surface standard profile I during the machining process.

4. The metal surface defect identification method according to claim 3, characterized in that: The three-dimensional defect points corresponding to each defect are input into a pre-trained classifier to identify the type of each defect, specifically including: The combination of the three-dimensional coordinates of the defective three-dimensional point corresponding to each defect and the error distance between the metal surface profile during processing at the defective three-dimensional point and the standard metal surface profile is used as the representation of the defective three-dimensional point; For each defect, the corresponding three-dimensional defect point is input into a pre-trained dynamic graph convolutional network as a classifier to extract the type geometric features of the defect, and the type of the defect is identified based on the extracted type geometric features.

5. The metal surface defect identification method according to claim 4, characterized in that: Training the dynamic graph convolutional network specifically includes: Acquire historically collected metal surface contour point cloud data as initial point cloud data, and perform continuous transformation operations on the initial point cloud data to construct a point cloud sequence, where the transformation operations include at least one of rotation, translation, and scaling; Constructing a dynamic graph convolution online network and a dynamic graph convolution target network, obtaining first sample data corresponding to the dynamic graph convolution online network and second sample data corresponding to the dynamic graph convolution target network from the point cloud sequence, and performing spatial enhancement on the first sample data and the second sample data; Inputting the spatially enhanced first sample data into the dynamic graph convolution online network for feature mapping to obtain features of the first sample data after being transformed by the dynamic graph convolution online network, and inputting the spatially enhanced second sample data into the dynamic graph convolution target network for feature mapping to obtain features of the second sample data after being transformed by the dynamic graph convolution target network; By adding a predictor, the mean square error between the features of the first sample data after being transformed by the dynamic graph convolution online network and the features of the second sample data after being transformed by the dynamic graph convolution target network is calculated according to the following formula: The spatially enhanced second sample data is fed into the dynamic graph convolution online network, and the spatially enhanced first sample data is fed into the dynamic graph convolution target network. The loss in the mean square error is symmetrized and the total loss is calculated by the following formula: For each training step, the parameters of the dynamic graph convolution online network are updated with the goal of minimizing the total loss. After the step training is completed, the parameters of the dynamic graph convolution target network are updated according to the moving average of the dynamic graph convolution online network parameters. in, is the mean square error between the features of the first sample data after being transformed by the dynamic graph convolution online network and the features of the second sample data after being transformed by the dynamic graph convolution target network, r(z θ ) is the feature of the first sample data after being transformed by the dynamic graph convolution online network, is the feature of the second sample data after being transformed by the dynamic graph convolution target network, is the loss in the mean square error after symmetry, For the total loss.

6. The metal surface defect identification method according to claim 1, characterized in that: The method further comprises: Performing outlier filtering on the first point cloud data by using a filtering algorithm to remove outliers in the first point cloud data; Determining the nearest neighbor point pairs of the metal surface profile during processing and the standard profile of the metal surface based on the first point cloud data and the second point cloud data; Calculate the absolute error distance between the nearest neighbor points and determine whether the absolute error distance is less than the preset update threshold; If so, the average value of the non-travel direction coordinate values of the first point cloud data and the second point cloud data during the processing is calculated, and the second point cloud data corresponding to the standard profile of the metal surface is updated; If not, the second point cloud data corresponding to the standard profile of the metal surface is not updated using the first point cloud data.

7. The metal surface defect identification method according to claim 6, characterized in that: The method further comprises: When the nearest neighbor point of the standard contour of the metal surface cannot be found in the first point cloud data corresponding to the contour of the metal surface during processing, determining whether the amount of missing three-dimensional points in the first point cloud data corresponding to the contour of the metal surface during processing is greater than a preset missing threshold; If yes, the second point cloud data corresponding to the standard profile of the metal surface is not updated using the first point cloud data; If not, three-dimensional point interpolation is performed on the first point cloud data corresponding to the metal surface contour during the processing, and the nearest neighbor point of the metal surface standard contour is re-determined from the first point cloud data corresponding to the metal surface contour during the processing.

8. A metal surface defect recognition device, characterized in that: include: An acquisition module, configured to respectively acquire first point cloud data of a metal surface profile during processing and second point cloud data of a preset metal surface standard profile; The first point cloud data is collected by a 3D laser profiler; a determination module for determining, for each 3D point in the first point cloud data, a nearest neighbor point of the 3D point in the second point cloud data based on the distance between the 3D point and each 3D point in the second point cloud data, so as to align the metal surface profile during processing with the standard metal surface profile; a defect determination module for determining a defective three-dimensional point set corresponding to a defect from the first point cloud data based on the distance between each three-dimensional point in the first point cloud data and its nearest neighbor point in the second point cloud data and a preset error threshold; when the distance between a three-dimensional point in the first point cloud data and its nearest neighbor point in the second point cloud data is greater than the preset error threshold, it is indicated that the metal surface profile at the three-dimensional point is different from the standard metal surface profile and a defect exists; A clustering module is used to determine neighboring points within a preset search radius of each defective 3D point in the defective 3D point set; Classify the neighboring points whose Euclidean distance to the defective 3D point is less than a preset clustering distance threshold and the defective 3D point into the same type of defective 3D points; Determine similar defect 3D points of the defect 3D point, and within a preset search radius of each similar defect 3D point, classify the neighboring points whose Euclidean distance to the similar defect 3D point is less than a preset clustering distance threshold as similar defect 3D points together with the defect 3D point; determine the final similar defect 3D point of the defect 3D point through multiple rounds of iterative calculation until no new similar defect 3D point can be obtained, and determine that the defect 3D point and its similar defect 3D points are defect 3D points corresponding to one defect; The recognition module is used to input the three-dimensional defect points corresponding to each defect into a pre-trained classifier to identify the type of each defect.

9. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

Images