Workpiece defect detection method, terminal device and storage medium

Abstract

The application relates to a workpiece defect detection method, a terminal device and a storage medium, the method is improved on the basis of a traditional DGCNN network, the original EdgeConv is improved in the improved DGCNN network, and point pair features (PPF) and a center point normal vector are added on the basis of original features of the traditional EdgeConv; secondly, different weights are given to the outputs of each layer MLP, and the weights gradually increase from the output of the lowest layer MLP to the output of the highest layer MLP; a DCP point cloud registration network model based on the improved DGCNN network matches point pairs of a workpiece to be detected and a workpiece without defects, and whether a point is a defect point is judged based on the Euclidean distance between the point pairs. The application improves the registration effect and improves the accuracy of defect detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of defect detection, and in particular to a workpiece defect detection method, a terminal device and a storage medium. BACKGROUND

[0002] In recent years, new energy electric vehicles have developed very rapidly, and pure electric vehicles have gradually become a development trend. As the heart of new energy vehicles, the quality detection of lithium batteries is very important. From the perspective of lithium battery product packaging routes, lithium batteries can be divided into soft package, square and cylindrical batteries. Lithium battery appearance defect detection based on machine vision is usually to replace human eyes with machines for detection. At present, most visual products are mainly based on 2D images, or use 2D plus depth data, and through 2D image processing, it can be detected whether the lithium battery can meet the safety standard before leaving the factory. However, the traditional 2D image has its limitations, and it is difficult to restore the specific appearance of the real battery to be tested, such as assembly error or damage deformation.

[0003] The 3D point cloud image can provide more rich and intuitive image information than the 2D image. The defect detection scheme based on registration technology is a common method. However, the traditional registration algorithm is sensitive to the initial pose of the point cloud. When the pose of the object to be tested is large, the traditional algorithm will often fall into a local optimal situation in the iterative calculation process, resulting in poor registration effect. In the prior art, a coarse registration algorithm is usually used to provide an initial pose, and a fine registration is used to correct to obtain a good registration effect. However, since a lot of time is spent in the coarse registration to calculate the PFH or FPFH and other feature descriptors of the point cloud, the real-time requirement of the defect detection based on registration cannot be met. SUMMARY

[0004] To solve the above problems, the present application provides a workpiece defect detection method, a terminal device and a storage medium.

[0005] The specific scheme is as follows:

[0006] A workpiece defect detection method, comprising the following steps:

[0007] S1: preparing a registration data set of workpieces for training a registration network, the registration data set comprising a source point cloud image, a target point cloud image and a rigid transformation matrix between the two;

[0008] S2: extracting features of each point cloud image by improving the DGCNN network;

[0009] The improved DGCNN network is first improved from the original EdgeConv, and point pair features PPF and a center point normal vector are added to the original features of the traditional EdgeConv; secondly, different weights are given to the outputs of each layer of MLP, and the weights gradually increase from the lowest layer of MLP output to the highest layer of MLP output;

[0010] S3: Constructing a DCP point cloud registration network model based on the improved DGCNN network, training the model through a registration data set, and obtaining an optimal registration model after training;

[0011] S4: Obtaining the matching relationship of point pairs in the point clouds of the to-be-tested workpiece and the corresponding defect-free workpiece through the optimal registration model, calculating the Euclidean distance between each matching point pair, and setting the point pairs whose Euclidean distance is greater than an error threshold as defect points.

[0012] Further, the method for making the registration data set is:

[0013] S101: Collecting point cloud images of defect-free workpieces and point cloud images of defective workpieces;

[0014] S102: Randomly rotating and translating the point cloud images to obtain a source point cloud image, a target point cloud image and a rigid transformation matrix therebetween, and combining the source point cloud image, the target point cloud image and the rigid transformation matrix therebetween to form a registration data set.

[0015] Further, the step S101 further includes preprocessing the collected point cloud images, and the preprocessing includes: removing outliers through radius filtering and then down-sampling through uniform sampling.

[0016] Further, the symmetry function part of the improved DGCNN network adopts a new pooling method of a max-pooling layer plus an average-pooling layer.

[0017] A workpiece defect detection terminal device, comprising a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described above when executing the computer program.

[0018] A computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the method described above.

[0019] The technical scheme is adopted, the traditional DGCNN network is improved, the registration effect is improved, and the accuracy of defect detection is improved. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1A flow chart of an embodiment of the present application is shown.

[0021] Figure 2 A modified EgdeConv schematic diagram in this embodiment is shown.

[0022] Figure 3 A PPF point pair feature diagram in this embodiment is shown.

[0023] Figure 4 A SL-DGCNN network structure diagram in this embodiment is shown.

[0024] Figure 5 A modified DCP point cloud registration network structure diagram in this embodiment is shown.

[0025] Figure 6 A transformer network structure diagram in this embodiment is shown. DETAILED DESCRIPTION

[0026] To further illustrate the embodiments, the present application provides accompanying drawings. These accompanying drawings are part of the disclosure of the present application, mainly used to illustrate the embodiments, and can be used to explain the operating principle of the embodiments in conjunction with the related description of the specification. Those of ordinary skill in the art should understand other possible implementations and advantages of the present application in conjunction with these contents.

[0027] The present application will be further described in conjunction with the accompanying drawings and specific embodiments.

[0028] Embodiment one:

[0029] The embodiment of the present application provides a workpiece defect detection method, as shown in the figure, the method comprises the following steps: Figure 1

[0030] S1: preparing a registration dataset of workpieces for training a registration network, the registration dataset comprising a source point cloud image, a target point cloud image and a rigid transformation matrix therebetween.

[0031] The preparation method of the registration dataset is:

[0032] S101: collecting point cloud images of defect-free workpieces and point cloud images of defective workpieces;

[0033] S102: randomly rotating and translating the point cloud images to obtain a source point cloud image, a target point cloud image and a rigid transformation matrix therebetween, and combining the source point cloud image, the target point cloud image and the rigid transformation matrix therebetween to form a registration dataset.

[0034] ​Further, since the matching point pairs between the point cloud image of the workpiece collected in the production process and the point cloud image of the standard workpiece are not one-to-one, Gaussian noise is added to the point cloud image, so that the model obtained by subsequent training can better cope with the actual situation.

[0035] In this embodiment, the workpiece is selected as a lithium battery, specifically a square aluminum shell lithium battery.

[0036] Further, the collected point cloud image also needs to be preprocessed. In this embodiment, the pre-processing process includes: removing outliers by radius filtering and then down-sampling by uniform sampling. In this embodiment, 1024 points are finally obtained by down-sampling.

[0037] In this embodiment, the point cloud image is collected by a 3D laser scanner. Further, since the 3D laser scanner under a single view can only obtain part of the point cloud image under that view, it is necessary to design the placement of multiple view 3D laser scanners in combination with certain mechanical structures to obtain a complete point cloud. Multiple view point cloud images can be converted to the same spatial coordinate system and spliced into one image.

[0038] S2: Extract features of each point cloud image by improving the DGCNN network.

[0039] In the DGCNN network, local feature extraction is first performed by EdgeConv. As shown in Figure 2 , EdgeConv constructs the topological relationship between points in the point cloud to obtain a local feature edge graph by KNN (nearest neighbor algorithm). A key point Xi can find K neighboring points to form K edges (Xi, Xi1),... (Xi, Xik). These K edges form the edge graph of the key point. In the traditional EdgeConv, the local features of the center point can be extracted, which only contains the coordinate information of the center point Pi and the directed edge information △Pi,j. In this embodiment, the EdgeConv is expanded to include point pair features PPF (PPF(Pi, Pn)) and center point normal vector (normal(Pi)) based on the original features, in order to enrich the local information and expand the feature dimension. Specifically, PPF is a four-tuple point pair feature as shown in Figure 3 , which includes the distance between point pairs, point normal vectors, and the angle between point pair vectors, and can be represented as [d, ∠<d,n1>, ∠<d,n2>, ∠<n1,n2>]. If there are N points, N 2 features will be constructed. In addition, in combination with the normal vector of each center point, the feature is expanded from the original 6 channels to 13 channels, so that the network obtains more rich local features.

[0040] DGCNN (Dynamic Graph Convolutional Neural Network) employs multi-scale feature fusion to integrate low-dimensional and high-dimensional feature information, thereby strengthening the connection between the output information of each layer. For example... Figure 4 As shown, the improved DGCNN (SL-DGCNN) proposed in this embodiment is an improvement on the original DGCNN. First, SL-DGCNN uses multi-scale feature fusion to integrate low-dimensional and high-dimensional feature information. Specifically, it concatenates the outputs of convolutional layers that are far apart together and uses them as the output of the next layer, thereby strengthening the connection between the output information of each layer.

[0041] In traditional DGCNN, the outputs of each convolutional layer are directly concatenated. This embodiment considers that the features obtained from the initial EdgeConv layer are closest to the actual point cloud data. However, the features obtained after multiple MLP layers become increasingly abstract. Therefore, this embodiment assigns different weights to the outputs of each MLP layer in SL-DGCNN, with the weights gradually increasing from the lowest to the highest layer (higher weights for nearest-neighbor MLPs and lower weights for more distant MLPs). This allows the network to obtain feature descriptions that more closely resemble the actual point cloud data. Simultaneously, a novel pooling method combining max pooling and average pooling layers is used in the symmetric function part to effectively compensate for information loss.

[0042] S3: Construct a DCP point cloud registration network model based on an improved DGCNN network, train the model using a registration dataset, and obtain the optimal registration model after training.

[0043] like Figure 5 As shown, the DCP point cloud registration network is divided into a feature extraction module, an attention-based Pointer module, and an SVD decomposition layer for calculating the rigid body transformation matrix.

[0044] In the network, the source point cloud X and the target point cloud Y are first processed by a shared SL-DGCNN network to obtain features FX and FY for each point in X and Y. Since the feature extraction is performed independently, there is no connection between FX and FY. To establish a relationship between them and achieve better matching results, the network employs a method such as... Figure 6 The Transformer is used to predict soft correspondences between point clouds. Therefore, if FX and FY are taken as input, we can obtain ΦX = FX + f(FX, FY) and ΦY = FY + f(FY, FX), where f is the mapping function (Ri) learned by the Transformer. N*P *R N*P ->R N*P), which is equivalent to the Transformer learning a conditional residual variable from the features. Specifically, the self-attention and MLP of the encoder in this module encode FX into an embedding space, and the decoder encodes FY into the embedding space as the encoder does. Then the co-attention in the decoder processes the two point clouds mapped into the embedding space to output ΦX and ΦY. For ΦX and ΦY with interaction information obtained by the transformer, the pointer uses them to find a probability-based point-to-point correspondence. Specifically, the pointer calculates the similarity between the feature of each point in X i T represents the feature of X i , respectively, and the feature of each point in ΦY, and the similarity is represented by the dot product. Then, the similarity is converted into probability by m(X i ,Y) = softmax(ΦYΦX i T ). After obtaining m(X i ,Y), the SVD decomposition layer calculates how to adjust X to align with Y. For each point X i , an average point Y^ i is calculated by weighted summation. T m(X i ,y) matches it. Similarly, the target point of each point in the point cloud X can be calculated by this method, and finally the rotation and translation matrix R i -> Y^ i and T xy of X xy -> Y^ xy are obtained. xy The input of the DCP point cloud registration network model is X and Y point clouds, and the output is R xy and T xy (rigid transformation matrix). The loss function LOSS is constructed using R xy and T and the true value (in the registration dataset) as follows:

[0046]

[0047] R

[0001] is an orthogonal matrix, so in the ideal state, the following equation should be established:

[0048]

[0049] After iterative training to minimize the loss function, the optimal registration model is obtained.

[0050] S4: Obtain the matching relationship of the point pairs in the point clouds of the to-be-tested workpiece and the corresponding defect-free workpiece through the optimal registration model, calculate the Euclidean distance between each matching point pair, and set the point pairs whose Euclidean distance is greater than the error threshold, so as to set the points belonging to the to-be-tested workpiece in the point pairs as defect points.

[0051] It should be noted that when the defect detection of the to-be-tested workpiece is performed, the collected point cloud image needs to be preprocessed first, and then the features of the point cloud images of the to-be-tested workpiece and the corresponding defect-free workpiece are obtained through the method of step S2, and finally the two features are input into the optimal registration model.

[0052] The size of the error threshold can be set by the person skilled in the art according to the requirements, and is not limited here.

[0053] All defect points in the point cloud image of the to-be-tested workpiece are marked, and the corresponding regions of all defect points in the point cloud image of the to-be-tested workpiece are obtained, so as to realize the defect detection of the to-be-tested workpiece.

[0054] S6, calculate the Euclidean distance between the matching point pairs in the point cloud images of the to-be-tested workpiece and the defect-free workpiece, and detect and locate the positions of defects in the to-be-tested workpiece.

[0055] In the step S5, whether the calculated Euclidean distance between the corresponding matching point pairs is greater than the set threshold value is used as the standard for determining the abnormal points, and finally all abnormal points are marked, the positions of defects are extracted, and the defect detection of the square aluminum shell battery is completed.

[0056] The embodiment of the application improves the registration effect by improving the traditional DGCNN network, and improves the accuracy of defect detection.

[0057] Embodiment two:

[0058] The application also provides a workpiece defect detection terminal device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps in the above method embodiments of the embodiment one of the application when executing the computer program.

[0059] Further, as an executable solution, the workpiece defect detection terminal device can be a desktop computer, a notebook computer, a palm computer, a cloud server and the like. The workpiece defect detection terminal device can include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the above-mentioned composition structure of the workpiece defect detection terminal device is only an example of the workpiece defect detection terminal device, and does not constitute a limitation on the workpiece defect detection terminal device, and can include more or fewer components than the above, or combine certain components, or different components, for example, the workpiece defect detection terminal device can also include an input / output device, a network access device, a bus and the like, and the embodiments of the present application do not limit this.

[0060] Further, as an executable solution, the processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The processor is the control center of the workpiece defect detection terminal device, and connects various parts of the workpiece defect detection terminal device through various interfaces and lines.

[0061] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the workpiece defect detection terminal device by running or executing the computer program and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required by a function; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory devices.

[0062] The present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the above-mentioned method of the embodiments of the present application.

[0063] The modules / units integrated in the workpiece defect detection terminal equipment, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by instructing related hardware through a computer program. The computer program can be stored in a computer readable storage medium. When the computer program is executed by a processor, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution medium, etc.

[0064] Although the present application is specifically shown and described in conjunction with the preferred embodiments, those skilled in the art should understand that various changes in form and details can be made to the present application without departing from the spirit and scope of the present application as defined in the appended claims.

Claims

1. A method of detecting defects in a workpiece, characterized by, The method comprises the following steps: S1: preparing a registration dataset of a workpiece for training a registration network, the registration dataset comprising a source point cloud image, a target point cloud image, and a rigid transformation matrix between the two; S2: extracting features of each point cloud image by improving a DGCNN network; The improved DGCNN network first improves the original EdgeConv, and adds point pair features PPF and a central point normal vector to the original features of the traditional EdgeConv; secondly, different weights are assigned to the outputs of each layer of MLP, and the weights gradually increase from the output of the lowest layer of MLP to the output of the highest layer of MLP; the point pair features PPF include the distance between point pairs, the point normal vector, and the included angle between the point pair vector in the point cloud image; S3: constructing a DCP point cloud registration network model based on the improved DGCNN network, training the model through the registration dataset, and obtaining an optimal registration model after training; The DCP point cloud registration network model comprises a feature extraction module, a Pointer module based on attention, and an SVD decomposition layer for calculating a rigid transformation matrix; S4: obtaining the matching relationship of point pairs in the point clouds of the workpiece to be tested and the corresponding defect-free workpiece through the optimal registration model, calculating the Euclidean distance between each matching point pair, and setting the point belonging to the workpiece to be tested in the point pair as a defect point if the Euclidean distance is greater than an error threshold.

2. The method of claim 1, wherein: The method for preparing the registration dataset comprises: S101: collecting point cloud images of a defect-free workpiece and a defective workpiece; S102: randomly rotating and translating the point cloud images to obtain a source point cloud image, a target point cloud image, and a rigid transformation matrix therebetween, and combining the source point cloud image, the target point cloud image, and the rigid transformation matrix therebetween to form a registration dataset.

3. The method of claim 2, wherein: The step S101 further comprises preprocessing the collected point cloud images, which comprises removing outliers through radius filtering and then down-sampling through uniform sampling.

4. The method of claim 1, wherein: The improved DGCNN network adopts a new pooling method of a max-pooling layer plus an average-pooling layer for the symmetry function part.

5. A workpiece defect detection terminal device, characterized by: The computer program is executed by the processor to implement the steps of the method according to any one of claims 1-4.

6. A computer readable storage medium storing a computer program, characterized in that: The computer program is executed by the processor to implement the steps of the method according to any one of claims 1-4.

Images