Surface defect detection model based on tactile feedback and construction method and detection method thereof

Abstract

The invention belongs to the technical field of product surface defect detection, and particularly discloses a surface defect detection model based on tactile feedback and a construction method and a detection method thereof, and the method comprises the steps: training a convolutional neural network model through a product surface tactile signal and a defect category label thereof; the convolutional neural network model comprises a feature extractor and a classifier, during training, the input of the feature extractor is a product surface tactile signal and a defect category label thereof, and the feature extractor adopts a class-guided contrast learning strategy to extract discriminative features capable of distinguishing signal categories from the tactile signal and compress the discriminative features into representation vectors for output; the classifier inputs representation vectors and corresponding defect category labels, and outputs detected defect categories; and taking the trained convolutional neural network model as a surface defect detection model. According to the invention, the fine defects on the surface of the product can be automatically detected, the fine defects of 10 microns can be detected, and the detection result is more accurate.

Description

Technical Field

[0001] The present invention belongs to the technical field of product surface defect detection, and more specifically, relates to a surface defect detection model based on tactile feedback, a construction method thereof, and a detection method. Background Art

[0002] High-quality surfaces are crucial for ensuring product performance and longevity. Many cutting-edge manufacturing industries, such as aircraft manufacturing, nuclear fuel container manufacturing, and artificial organ production, have stringent requirements for product surface quality. To ensure product durability and safety, surface defects in these industries are typically limited to 0.01mm. Therefore, to improve finished product quality and eliminate safety hazards, surface quality inspection must be included in the manufacturing process.

[0003] Manual inspection, including visual inspection, contact testing, and optical inspection, was a common surface quality inspection method in early industrial fields. These methods rely on manpower and are inefficient. As modern industry's demand for efficiency, precision, and automation continues to increase, the drawbacks of traditional methods are becoming increasingly apparent. In recent years, the development of computer vision has provided an automated, efficient, and low-cost solution for product surface defect detection, attracting widespread attention from academia and industry. It is currently being applied in many manufacturing fields, such as steel stamping, ceramic sintering, car body welding, and printed circuit boards.

[0004] However, when it comes to detecting subtle defects, these methods, which rely on visual images, have the following limitations: ① Due to the physical properties of optical sensors, they cannot operate in narrow or enclosed low-light conditions or in irradiated environments; ② Imaging results are easily affected by the curvature, texture, and reflection of the material surface, and the imaging results of tiny defects may vary under different lighting angles. Therefore, there is an urgent need to develop a highly adaptable, efficient, and high-precision surface defect detection method. Summary of the Invention

[0005] In response to the above-mentioned defects or improvement needs of the prior art, the present invention provides a surface defect detection model based on tactile feedback, a construction method thereof, and a detection method thereof, the purpose of which is to achieve high-precision detection of subtle defects on the product surface.

[0006] To achieve the above objectives, according to a first aspect of the present invention, a method for constructing a surface defect detection model based on tactile feedback is proposed, comprising the following steps:

[0007] The convolutional neural network model is trained using the product surface tactile signals and their defect category labels. The convolutional neural network model includes a feature extractor and a classifier. During training:

[0008] The feature extractor takes the tactile signal from the product surface and its defect category label as input. It uses a class-guided contrastive learning strategy to extract discriminative features that can distinguish signal categories from the tactile signal and compress them into a representation vector output. The classifier takes the representation vector and the corresponding defect category label as input and outputs the detected defect category.

[0009] The trained convolutional neural network model is used as the surface defect detection model.

[0010] As a further preference, the feature extractor and classifier adopt an independent training strategy, that is, the feature extractor is first trained based on the tactile signal of the product surface and its defect category label; then the representation vector is obtained through the trained feature extractor, and the classifier is trained based on the representation vector and the corresponding defect category label.

[0011] As a further preference, the loss function used in feature extractor training is as follows:

[0012]

[0013] in, represents the feature extractor loss function, K represents the number of categories, n k represents the number of samples in each category, represents the set of positive samples of the same type as sample i, Represent the representation vectors corresponding to samples i, p, and a respectively, τ represents the hyperparameter that controls the similarity between representation vectors, and W represents the learnable weight matrix. represents the regularization penalty term based on the Gram matrix.

[0014] As a further preferred embodiment, when training the feature extractor, the SGD optimizer is used to optimize the parameters of each layer of the feature extractor generation by generation.

[0015] As a further preference, a cross entropy loss function is used during classifier training, and an Adam optimizer is used to optimize the parameters of each layer of the classifier generation by generation.

[0016] As a further preference, the feature extractor uses ResNet18 as the skeleton, and adopts depth-wise convolution and point-wise convolution as convolution layers, and uses GeLU activation function after each convolution layer to achieve nonlinear transformation.

[0017] As a further preference, the classifier adopts an FFN architecture, which includes an input layer, a hidden layer and an output layer, and the hidden layer uses Dropout to randomly inactivate neurons.

[0018] As a further preferred embodiment, the method for obtaining the product surface tactile signal and its defect category label includes:

[0019] Plan the robot arm's trajectory based on the product's 3D modeling;

[0020] The robotic arm moves so that the tactile sensor fixed at the end of the arm touches all the surface areas of the product to be tested and collects tactile signals;

[0021] Manually label the categories of tactile signals to form defect category labels.

[0022] According to a second aspect of the present invention, a surface defect detection model is provided, which is constructed using the above-mentioned surface defect detection model construction method based on tactile feedback.

[0023] According to a third aspect of the present invention, a surface defect detection method is provided, comprising the following steps: inputting a tactile signal of the surface of a product to be tested into the above-mentioned surface defect detection model, a feature extractor outputting a representation vector based on the tactile signal, and a classifier outputting a detected defect category based on the representation vector.

[0024] In general, the above technical solutions conceived by the present invention have the following technical advantages compared with the existing technology:

[0025] 1. The present invention constructs a defect detection model based on tactile signals, enabling it to capture morphological features at the 10-micron level, improving the ability to perceive subtle surface defects, and realizing high-precision automated detection of subtle surface defects on products.

[0026] 2. This invention uses tactile signals to precisely capture surface defects in transparent, highly reflective, and complex-textured materials, overcoming the limitations of visual methods for such materials. Furthermore, tactile signal acquisition does not require a light source. Therefore, compared to computer vision-based methods, this method has a wider range of applications.

[0027] 3. The present invention adopts a class-guided contrastive learning strategy, uses label prior knowledge to optimize the contrastive learning training process, and expands the microscopic sample-level contrast constraints to the macroscopic category-level contrast constraints, thereby achieving a compressed distribution of similar samples and a differential representation of heterogeneous samples, significantly improving the model detection precision and accuracy.

[0028] 4. The independent training of the feature extractor and classifier adopted in the present invention provides a simpler fine-tuning method to ensure that the model training achieves optimal performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a flow chart of a method for detecting subtle surface defects of products based on machine touch according to an embodiment of the present invention;

[0030] Figure 2 Schematic diagram of a tactile test bench constructed for an embodiment of the present invention;

[0031] Figure 3 Schematic diagram of tactile signals and sizes of different types of subtle surface defects collected by an embodiment of the present invention;

[0032] Figure 4 A structural diagram of a surface defect detection model constructed for an embodiment of the present invention;

[0033] Figure 5 The figure shows the training principle and flow chart of the defect detection model according to the embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0035] The embodiment of the present invention provides a method for constructing a surface defect detection model based on tactile feedback, such as Figure 1 As shown, the following steps are included:

[0036] S1. Obtain tactile signals from the product surface and label their defect categories to build a tactile signal dataset.

[0037] Specifically, the tactile signal of the product surface is obtained by the tactile sensor. In this embodiment, a tactile test bench is constructed, which includes a mechanical arm, a clamping device, a tactile sensor and an engineering host. Figure 2 The engineering host is used for data analysis and controlling the movement of the robotic arm, the clamping device is used to fix the tactile sensor, and the tactile sensor is used to obtain tactile signals.

[0038] Based on the 3D modeling of the workpiece to be tested, the robot arm's trajectory is planned so that the tactile sensor can contact all the test areas of the workpiece to be tested, and the tactile signal is collected as the medium signal for surface fine defect detection. Specifically, the python SDK is used to write the path instructions and import them into the engineering host. The engineering host sends instructions to control the robot arm to clamp the tactile sensor and move it so that the measuring end of the tactile sensor contacts the product to obtain the tactile signal with a resolution of 64×64 and stored in the .png file format. The category of the tactile signal is manually labeled to indicate what category of surface defect the tactile signal represents. The tactile signals of some defect categories are as follows: Figure 3 As shown. On this basis, a tactile signal dataset is constructed

[0039] It should be noted that this is only a preferred embodiment, and other robots or direct manual acquisition of tactile signals may also be used; that is, the specific method and structure are not limited, as long as the surface tactile signal can be acquired through the tactile sensor.

[0040] S2. Construct a convolutional neural network model for tactile signal analysis.

[0041] The PyTorch library is used to build a convolutional neural network model TouchNet for tactile signal analysis. The model includes a feature extractor and a classifier, such as Figure 4 As shown. The feature extractor extracts the original tactile signal Extract the discriminative features that can distinguish signal categories and compress them into representation vectors Classifier for decoding information and maps it into classification probabilities

[0042] Furthermore, the feature extractor uses ResNet18 as the backbone and embeds depth-wise convolution and point-wise convolution to compress the model parameters. The dimension is 512. After each convolution layer, the GeLU activation function is used to achieve nonlinear transformation.

[0043] Furthermore, the classifier uses the FFN architecture, which consists of a three-layer neural network consisting of an input layer, a hidden layer, and an output layer. The number of neurons in the hidden layer is 64, and Dropout is used to randomly inactivate neurons to avoid overfitting.

[0044] S3. Through tactile signal dataset Train the TouchNet model, such as Figure 5 As shown in Figure 3, the training process optimizes the parameters of the model generation by generation, so that it can effectively detect and classify different types of defects and obtain a surface defect detection model.

[0045] Specifically, the feature extractor and classifier adopt independent training strategies, including:

[0046] (1) The input of the feature extractor is the tactile signal and its category labels The output is a representation vector The feature extractor is trained through a class-guided contrastive learning strategy, which utilizes prior knowledge of class labels to compress the distribution space of similar signals and stretch the spatial distance of heterogeneous signals, thereby guiding the hypersphere clustering process during training at the macro-category level.

[0047] For a K-classification task, assuming each class has n k samples, then for sample x i , positive sample set Negative sample set That is, sample x i The remaining samples in the same class are considered as positive sample pairs, and the samples in other classes are considered as negative sample pairs. The loss function of class-guided contrastive learning is:

[0048]

[0049] Among them, W is the learnable weight matrix, K is the total number of categories, k is a sample set of a certain category, and n k is the number of samples in the k-sample set, n s is the total number of samples of other classes except class k, i, j, k are different samples; apply quadratic regularization to the weight vector Penalize similarity and promote the orthogonalization of W, optimize the training process and avoid feature attenuation. The loss function of the feature extractor is for:

[0050]

[0051] in Represents the positive sample set of the same type as sample i, and the temperature τ is the control d rep The training process uses the SGD optimizer to optimize the parameters of each layer of the feature extractor generation by generation.

[0052] (2) The input of the classifier is a representation vector and defect category labels The output is the defect class detected After the feature extractor is trained, the classifier is trained by cross entropy loss. For a K-classification task, assuming that each class has n k samples, then the loss function of the classifier is for:

[0053]

[0054] in, is the true category of the defect, The training process uses the Adam optimizer to optimize the parameters of each layer of the classifier generation by generation.

[0055] Furthermore, the surface defect detection model constructed based on the above method realizes micron-level surface defect detection, including the following steps:

[0056] S4. Obtain tactile signals from the surface of the product to be tested, and input them into a surface defect detection model to obtain corresponding defect categories.

[0057] Specifically, Docker is used in advance to deploy the trained surface defect detection model to the engineering host of the tactile test bench for analyzing the tactile signals of subsequent products to be tested. The constructed tactile test bench is then used to collect the tactile signals of the products to be tested and input them into the surface defect detection model in the engineering host. The feature extractor extracts the representation vector of the tactile signal, and then the classifier classifies the feature vector to obtain the detection result of the sample to be tested. Specifically, during the detection process, the engineering host sends a control instruction to the robotic arm to move the robotic arm to the target position and bring the measuring end of the tactile sensor into contact with the position to be tested. The signal collected by the tactile sensor is transmitted to the engineering host in real time and input into the pre-deployed defect detection model for processing. The defect detection model performs online analysis on the received tactile signal and outputs the defect detection result of the target position in real time.

[0058] The following are specific embodiments:

[0059] To verify the practical application of the method, a tactile test bench was constructed to collect tactile signals of common surface defects, such as normal, bumps, pits, and scratches. A tactile signal dataset was constructed and used to verify the algorithm. This example used a JAKA Zu5 robotic arm, a Gelsight mini tactile sensor, and an industrial host computer equipped with a Core i5 14600kf CPU and an Nvidia RTX 4070Ti Super GPU, respectively.

[0060] VGG16, ResNet18, ResNet50, DEGAN, and MoCo were selected as comparison models for TouchNet. The models' complexity, computational efficiency, and accuracy were comprehensively evaluated by calculating multiple metrics, including the number of model parameters, MACs, sample computation time, and accuracy. Each set of experiments underwent five independent training runs. The trained models were deployed on a tactile testbed and their performance was tested using new, untrained DUTs. The average results from these five runs served as the final evaluation metrics, as shown in Table 1.

[0061] Table 1 Comparative experimental results

[0062]

[0063] It can be clearly seen from Table 1 that the TouchNet model proposed in the present invention has achieved better overall results. It is worth noting that the MoCo model based on the classic contrastive learning framework performed poorly, which is mainly attributed to the natural defects of its unsupervised training method. In contrast, the TouchNet based on class-guided contrastive learning proposed in the present invention significantly improved the accuracy and greatly reduced the number of parameters while slightly increasing the computing time. This experimental result verifies that the introduction of prior labels can effectively guide high-dimensional feature clustering and enhance the model's ability to mine discriminative features. On the other hand, the depth-wise convolution and point-wise convolution in the TouchNet feature extractor are hardware-friendly operations that can significantly compress the number of parameters, but they also inevitably increase the amount of calculation and computing time slightly. In addition, although the DEGAN model based on the generative framework also achieved high accuracy, its complex structure resulted in a much higher number of parameters and computational cost than other models, so it is easily limited by computing resources in practical applications.

[0064] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for constructing a surface defect detection model based on tactile feedback, characterized in that: The steps include: The convolutional neural network model is trained using the product surface tactile signals and their defect category labels. The convolutional neural network model includes a feature extractor and a classifier. During training: The feature extractor takes the tactile signal from the product surface and its defect category label as input. It uses a class-guided contrastive learning strategy to extract discriminative features that can distinguish signal categories from the tactile signal and compress them into a representation vector output. The classifier takes the representation vector and the corresponding defect category label as input and outputs the detected defect category. The trained convolutional neural network model is used as the surface defect detection model.

2. The method for constructing a surface defect detection model based on tactile feedback according to claim 1, wherein: The feature extractor and classifier adopt an independent training strategy, that is, the feature extractor is first trained based on the tactile signal of the product surface and its defect category label; then the representation vector is obtained through the trained feature extractor, and the classifier is trained based on the representation vector and the corresponding defect category label.

3. The method for constructing a surface defect detection model based on tactile feedback according to claim 2, wherein: The loss function used in feature extractor training is as follows: in, represents the feature extractor loss function, K represents the number of categories, n k represents the number of samples in each category, represents the set of positive samples of the same type as sample i, Represent the representation vectors corresponding to samples i, p, and a respectively, τ represents the hyperparameter that controls the similarity between representation vectors, and W represents the learnable weight matrix. represents the regularization penalty term based on the Gram matrix.

4. The method for constructing a surface defect detection model based on tactile feedback according to claim 3, wherein: When training the feature extractor, the SGD optimizer is used to optimize the parameters of each layer of the feature extractor generation by generation.

5. The method for constructing a surface defect detection model based on tactile feedback according to claim 2, wherein: The cross entropy loss function is used in classifier training, and the Adam optimizer is used to optimize the parameters of each layer of the classifier generation by generation.

6. The method for constructing a surface defect detection model based on tactile feedback according to claim 1, wherein: The feature extractor uses ResNet18 as the skeleton, and adopts depth-wise convolution and point-wise convolution as convolution layers. After each convolution layer, GeLU activation function is used to realize nonlinear transformation.

7. The method for constructing a surface defect detection model based on tactile feedback according to claim 1, wherein: The classifier adopts the FFN architecture, which includes an input layer, a hidden layer and an output layer, and the hidden layer uses Dropout to randomly inactivate neurons.

8. The method for constructing a surface defect detection model based on tactile feedback according to any one of claims 1 to 7, wherein: The method for obtaining the product surface tactile signal and its defect category label includes: Plan the robot arm's trajectory based on the product's 3D modeling; The robotic arm moves so that the tactile sensor fixed at the end of the arm touches all the surface areas of the product to be tested and collects tactile signals; Manually label the categories of tactile signals to form defect category labels.

9. A surface defect detection model, characterized in that: The surface defect detection model is constructed using the tactile feedback-based surface defect detection model construction method according to any one of claims 1 to 8.

10. A surface defect detection method, characterized in that: The method comprises the following steps: inputting a tactile signal of the surface of the product to be tested into the surface defect detection model as claimed in claim 9, a feature extractor outputting a representation vector according to the tactile signal, and a classifier outputting a detected defect category according to the representation vector.

Images