A workpiece surface defect detection method, device, equipment and storage medium

Abstract

The application provides a workpiece surface defect detection method, device and equipment and a storage medium, and belongs to the technical field of mechanical equipment component defect detection. The method comprises the following steps: acquiring multiple workpiece surface images with defects; constructing a deep residual neural network; inputting the multiple workpiece surface images into the deep residual neural network; and outputting a defect recognition result. The workpiece surface image is inputted into the deep residual neural network, and the defect recognition result is outputted. Specifically, the method comprises the following steps: extracting defect features in the workpiece surface image through a feature extraction module; capturing different levels of features in the image at different scales through a pyramid module, and fusing the captured different levels of features; and identifying the defects on the workpiece surface according to the fusion result through a classifier. The method uses two ordinary residual modules to extract defect features, and uses an inverse residual module in a deep layer of the network to further extract features, so as to prevent information loss. Three different residual modules are used to extract features, and the extracted features are fused in multiple scales through three different dilated convolution rates, and then are sent to a global average pooling and a full connection layer, so as to realize defect detection and classification.

Description

Technical Field

[0001] This invention belongs to the field of mechanical equipment component defect detection technology, and relates to a workpiece surface defect detection method, device, equipment and storage medium, specifically to a defect detection method based on residual neural network of feature fusion at different levels. Background Technology

[0002] Gears are one of the most important basic components of mechanical equipment. Defects in gear components have a significant impact on their function and lifespan, potentially leading to performance degradation and affecting the safe operation of the equipment. Because image data of gears is easy to acquire and can clearly and intuitively reflect the state of gear defects, image-based defect detection technology is widely used in the field of gear defect detection. Traditional gear defect detection methods rely on manual visual inspection, which suffers from low efficiency and a high risk of errors. In today's big data era, gear manufacturers need to effectively and promptly analyze the large quantities of gears produced. In this context, manual visual inspection methods are insufficient for efficiently handling defect detection.

[0003] With the continuous development of computer technology, artificial intelligence has gradually become a hot topic in today's technological development. Deep learning is an extremely important and novel research and development direction in artificial intelligence, and it has been widely used in many fields such as image recognition, intelligent healthcare, and data mining. Deep learning-based detection technology has also been widely applied in the field of gear defect detection. This technology uses networks such as convolutional neural networks and residual neural networks to automatically extract defect features from image samples. It performs better in extracting complex nonlinear features of defect data, especially with large datasets, and can clearly reflect the mapping relationship between large datasets and gear conditions. Currently, shallow convolutional neural networks have poor detection accuracy when faced with large amounts of complex defect image data. Deep residual neural networks, due to their excellent defect detection performance, have become one of the commonly used methods in the field of defect detection, effectively avoiding the network degradation problem caused by deepening the network.

[0004] Typical residual neural networks have dozens or even hundreds of feature extraction layers, which are complex in structure and have a large number of parameters, limiting their potential industrial application in gear defect detection. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides a method for detecting surface defects in workpieces. It constructs a pyramid module with convolutional layers of varying expansion rates to simplify the model structure and reduce the number of parameters. A feature extractor is constructed using a residual module to capture features at different levels in image data at different scales. These captured features are then fused and fed into a global average pooling and fully connected layer. A classifier is then used to identify defects on the workpiece surface, achieving defect detection and classification.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for detecting surface defects in a workpiece includes the following steps:

[0008] Acquire multiple images of the workpiece surface with defects;

[0009] A deep residual neural network is constructed, comprising a feature extraction module, a pyramid module, and a classifier. The feature extraction module consists of two residual modules and one inverse residual module. The pyramid module consists of three dilated convolutional layers stacked sequentially with different dilation rates. The classifier consists of global average pooling and fully connected layers.

[0010] Multiple images of the workpiece surface are input into the deep residual neural network, and the defect recognition result is output.

[0011] The workpiece surface image is input into a deep residual neural network, and the defect identification result is output. The specific steps include:

[0012] Defect features are extracted from the workpiece surface image using a feature extraction module.

[0013] The pyramid module captures features at different levels in the image at different scales and then fuses the captured features at different levels.

[0014] The classifier identifies defects on the workpiece surface based on the fusion results.

[0015] Preferably, before inputting the workpiece surface image into the deep residual neural network, multiple images are adjusted to the same size.

[0016] Preferably, the pyramid module consists of three dilated convolutional layers with dilation rates of d = m + 2, d = m + 1, and d = m, respectively, where m is the dilation rate at the bottom of the pyramid structure, and in this invention, m is 1. The specific steps for capturing features at different levels in the image at different scales through the pyramid module and fusing the captured features at different levels are as follows:

[0017] The shallow features are first processed by a 3×3 dilated convolutional layer with an inflation rate of d = m + 2, as shown in the following expression:

[0018] y d1 =f d＝m+2 (y o (1)

[0019] In the formula, yo represents shallow features, and f d＝m+2 (·) represents a convolution operation with a 3×3 convolution kernel and an expansion rate of m+2, y d1 This represents the output of the dilated convolutional layer;

[0020] Then, a 3×3 dilated convolutional layer with an inflation rate of d = m+1 is used to process the mid-layer features, performing a linear mapping on the input features. The expression is as follows:

[0021] y d2 =f d＝m+1 (y1) (2)

[0022] In the formula, y1 is the middle layer feature, f d＝m+1 (·) represents a convolution operation with a 3×3 convolution kernel and an expansion rate of m+1, y d2 This represents the output of the dilated convolutional layer;

[0023] Finally, a 3×3 dilated convolutional layer with an inflation rate of d = m is used to process the deep features, performing a linear mapping on the input features. The expression is as follows:

[0024] y d3 =f d＝m (y2) (3)

[0025] In the formula, y2 represents the deep feature, and f d＝m (·) represents a convolution operation with a 3×3 convolution kernel and an expansion rate of m, y d3 This represents the output of the dilated convolutional layer and fuses the features processed by three different dilated convolutional layers, as shown in the following expression:

[0026] y s =y d1 +y d2 +y d3 (4)

[0027] Where y s These are the features after fusion.

[0028] Preferably, when the convolution kernel size is the same, the pyramid module obtains different receptive fields by setting different dilation rates, thereby obtaining multi-scale information. Specifically, let k be the original convolution kernel size, d be the dilation rate, and k* be the equivalent kernel size.

[0029] k * = k + (k-1) × (d-1) (5)

[0030] R l+1 * =R l +(k * -1)×S l (6)

[0031] Among them, R l+1 * It is the current sensory field, R l S represents the receptive field of the previous layer. lIt represents the product of the step sizes of all preceding layers.

[0032] Preferably, the feature extraction module first uses a 7×7 convolutional layer as the input convolutional layer. The input convolutional layer is used to perform initial convolution operations on the input image to extract low-level feature information, and then passes the low-level feature information to subsequent network layers for further feature extraction and learning. The extracted low-level feature information is further processed by two residual modules. The two residual modules are respectively composed of a 1×1 convolutional layer, a 3×3 convolutional layer, and a 1×1 convolutional layer stacked sequentially. The input of the residual module first passes through a batch normalization layer and the ReLU activation function, and then enters the convolutional layer. At the bottom layer of the feature extraction module, an inverse bottleneck structure is used to extract high-level defect features. The inverse bottleneck structure consists of a 3×3 convolutional layer and two 1×1 convolutional layers.

[0033] Preferably, the step of identifying defects on the workpiece surface by a classifier based on the fusion result specifically includes: the input data is processed through multiple residual blocks to learn feature representations; global average pooling is used to average the feature map on each channel, converting it into a fixed-length vector; the global average pooling operation averages the features at each position in the feature map, enabling the network to have a global perception of the entire image; the fully connected layer receives the feature vector obtained by the global average pooling operation as input and maps it to different categories to achieve defect classification.

[0034] Preferably, the method further includes training a deep residual neural network. The method for constructing training samples and test samples is as follows: the defects are manually calibrated on the surface image of the defective workpiece to obtain the workpiece defect image. The workpiece defect image is divided into training samples and test samples, and the defect images in the training samples are adjusted to the same size as image samples.

[0035] Preferably, the training process for the deep residual neural network includes:

[0036] The input image sample data is preprocessed by an input convolutional layer with a 7×7 kernel, a batch normalization layer, and a ReLU activation function. The input convolutional layer performs an initial convolution operation on the input image samples. After the convolution operation, the output data is input into the batch normalization layer. The batch normalization layer standardizes each feature channel so that its mean is close to 0 and its variance is close to 1. After the batch normalization layer, the ReLU activation function is applied. The ReLU activation function sets all negative values ​​to zero and keeps all positive values ​​unchanged. The preprocessing process is shown in formula (7).

[0037] y i =max{H(f(x′)),0} (7)

[0038] Where x′ represents the input data, f(·) represents the 7×7 convolution operation, H(·) represents the batch normalization operation, and the ReLU activation function is ReLU(·)=max{x,0}, y i This indicates the output characteristics after preprocessing;

[0039] The feature extraction module extracts the feature information of the input grayscale image, including: firstly, the first residual module of the feature extraction module extracts shallow features, which is composed of three convolutions of 1×1, 3×3, and 1×1; the calculation process is shown in formula (8):

[0040] y o =f1(f3(f1(max{H(y i ),0}))) (8)

[0041] In the formula y i The input features are represented by H(·), which represents the batch normalization operation. The ReLU activation function is ReLU(·) = max{x,0}. f1(·) represents a 1×1 convolutional layer operation, and f3(·) represents a 3×3 convolutional layer operation. o This represents the output of the first residual module;

[0042] Similarly, the operation of the second residual module is the same as that of the first module. The output of the first residual module is used as the input of the second residual module to extract the mid-layer feature information. The operation process is as follows:

[0043] y1=f1(f3(f1(max{H(y o ),0}))) (9)

[0044] In the formula y o y1 represents the input feature, and y1 represents the output feature of the second residual module;

[0045] In the feature extraction module, the inverse residual module is used to extract deep features. It uses a different structural shape than the residual module. The structure is composed of batch normalization, ReLU activation function, 3×3 convolutional layer, 1×1 convolutional layer and 1×1 convolutional layer stacked in sequence. The calculation process is as follows:

[0046] y2=f1(f1(f3(max{H(y1),0}))) (10)

[0047] In the formula, y1 represents the input feature and y2 represents the output feature of the inverse residual module;

[0048] The constructed residual neural network uses lateral connections and consists of three different convolutional layers with dilation rates of d = m + 2, d = m + 1, and d = m to form the pyramid module.

[0049] The fused features are further processed using average pooling, and its expression is as follows:

[0050] S = Avg(y s (11)

[0051] In the formula, Avg(·) represents the averaging operation, S represents the output of global average pooling, and the fully connected layer classifies the defect types and outputs the detection results, as shown in the following expression.

[0052] y l =F(∑w l S) (12)

[0053] In the formula, S represents the input feature, and w l y represents the connection weights of neurons in the intermediate layer, F(·) represents the convolution operation of the fully connected layer, and y represents the connection weights of neurons in the intermediate layer. l To output the results, backpropagation is performed based on the obtained cross-entropy loss value after the training phase, and the training parameters are continuously adjusted. The process of calculating the cross-entropy loss value is as follows:

[0054]

[0055] In the formula, C represents the number of state categories, with corresponding labels L = [0, 1, ..., C-1]; β is the true label value corresponding to the random n states; γ is the predicted value, which is the result y. l The predicted label values ​​corresponding to the above n random states are used to continuously backpropagate based on the obtained loss values ​​to update the parameters.

[0056] The process continues until the set final iteration number is reached, at which point the minimum loss value is obtained, and the final output workpiece surface defect detection result is obtained.

[0057] The present invention also provides a workpiece surface defect detection device, characterized in that it comprises:

[0058] The image acquisition module is used to acquire multiple images of the surface of a workpiece with defects;

[0059] The model building module is used to build a deep residual neural network, which includes a feature extraction module, a pyramid module, and a classifier. The feature extraction module consists of two residual modules and one inverse residual module. The pyramid module consists of three dilated convolutional layers with different dilation rates stacked sequentially. The classifier consists of global average pooling and fully connected layers.

[0060] The defect identification module is used to input multiple images of the workpiece surface into the deep residual neural network and output defect identification results.

[0061] The feature extraction module is used to extract defect features from the workpiece surface image;

[0062] The pyramid module is used to capture features at different levels in an image at different scales and to fuse the captured features at different levels.

[0063] The classifier is used to identify defects on the workpiece surface based on the fusion results.

[0064] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement any of the steps in the workpiece surface defect detection method.

[0065] The present invention also provides a computer-readable storage medium storing a computer program, characterized in that, when the computer program is loaded by a processor, it is capable of executing any of the steps in the workpiece surface defect detection method.

[0066] The workpiece surface defect detection method provided by this invention has the following beneficial effects:

[0067] This method employs two ordinary residual modules to extract defect features, and a deeper inverse residual module to further extract features, preventing information loss. Three dilated convolutions with different dilation rates are used to fuse the features extracted by the three different residual modules at multiple scales before feeding them into a global average pooling and fully connected layer, achieving defect detection and classification. Attached Figure Description

[0068] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0069] Figure 1 This is a flowchart of the workpiece surface defect detection method according to Embodiment 1 of the present invention;

[0070] Figure 2 Here are examples of gear defect types and accompanying images;

[0071] Figure 3 This is a diagram of the residual neural network structure based on feature fusion at different levels according to the present invention;

[0072] Figure 4 This is a diagram of the residual module and the inverse residual module;

[0073] Figure 5This is a schematic diagram of dilated convolution with dilation rates d = m, d = m + 1, and d = m + 2 (taking m = 1 as an example);

[0074] Figure 6 This is the confusion matrix of the proposed method on gear data;

[0075] Figure 7 This is a graph showing the classification accuracy versus loss curve of the proposed method. Detailed Implementation

[0076] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0077] Example 1

[0078] This invention proposes a residual neural network defect detection method based on the fusion of features at different levels. It fuses features at different depth levels using dilated convolutional layers with different dilation rates, thereby achieving the fusion of shallow, medium, and deep features at different feature layers, enriching defect information and further improving defect detection performance.

[0079] This method manually calibrates defects on acquired gear images and adjusts the calibrated images to the same size. It employs two standard residual modules to extract defect features, and a deeper inverse residual module further extracts features to prevent information loss. Three dilated convolutions with different dilation rates are used to fuse the features extracted by the three residual modules at multiple scales before feeding them into a global average pooling and fully connected layer, achieving defect detection and classification. During model training, an Adam gradient descent algorithm is used to accelerate training and improve the model's generalization ability.

[0080] Specifically, the present invention provides a method for detecting surface defects in a workpiece. In this embodiment, the object of detection is a gear. Figure 1 The flowchart of the intelligent detection method based on residual neural networks with feature fusion at different levels is shown. This detection method mainly includes the following steps:

[0081] Step 1: Take a picture of the gear and manually mark the defects on the gear image to obtain a gear defect image.

[0082] Specifically, gear components have four defect states: tooth surface cracks, chipping, missing teeth, and powder shedding, as shown in the attached diagram. Figure 2As shown, four different states are labeled with 0-3. In this embodiment, there are a total of 400 samples of these four types of gear defect images. These 400 samples are divided into training samples and test samples in a ratio of 7:3.

[0083] Step 2: Adjust the gear defect images in the training samples to the same size and use them as image samples.

[0084] This embodiment uses Python software to compress or stretch each image data in each state, converting it into a uniform 64×64 image sample, with 100 images in each state.

[0085] Step 3: Establish a deep residual neural network based on the fusion of features at different levels, and train the network using the image samples from Step 2.

[0086] First, the deep residual neural network in this embodiment includes a feature extraction module, a pyramid module, and a classifier. The feature extraction module consists of two residual modules and one inverse residual module. The pyramid module consists of three dilated convolutional layers stacked sequentially with different dilation rates. The classifier consists of global average pooling and fully connected layers. The detailed structure of the entire network model is as follows: Figure 3 As shown. Training the network includes the following steps:

[0087] (1) First, the input data is preprocessed using a 7×7 convolutional kernel input convolutional layer, a batch normalization layer, and a ReLU activation function. The input convolutional layer performs an initial convolution operation on the input image samples to extract low-level feature information and passes it to subsequent network layers for further feature extraction and learning. After the convolution operation, the output data can be input into the batch normalization layer. The batch normalization layer standardizes each feature channel so that its mean is close to 0 and its variance is close to 1. Standardization can accelerate the training process of the network and improve the convergence speed of the network. After the batch normalization layer, the ReLU activation function is applied. The ReLU activation function sets all negative values ​​to zero and keeps all positive values ​​unchanged. This introduces non-linearity and increases the expressive power of the network. The preprocessing process is shown in formula (3). The formulas below are labeled with different subscripts to distinguish the results of each process.

[0088] y i =max{H(f(x′)),0} (3)

[0089] Where x′ represents the input data, f(·) represents the 7×7 convolution operation, H(·) represents the batch normalization operation, and the ReLU activation function is ReLU(·)=max{x,0}, y i This indicates the output characteristics after preprocessing.

[0090] (2) Then, the feature extraction module extracts the feature information of the input image. First, the first residual module of the feature extraction module extracts shallow features. This residual module is composed of three convolutions: 1×1, 3×3, and 1×1. The 1×1 convolutional layer can reduce or increase the dimension, so that the 3×3 convolution can be operated on the input with a relatively low dimension, thereby shortening the computation time. Batch normalization is used before the three convolutional layers to reduce the effect of overfitting. The ReLU activation function is used to further enhance the nonlinear expression capability. Its main calculation process is shown in formula (4):

[0091] y o =f1(f3(f1(max{H(y i ),0}))) (4)

[0092] In the formula y i The input features are represented by H(·), which represents the batch normalization operation. The ReLU activation function is ReLU(·) = max{x,0}. f1(·) represents a 1×1 convolutional layer operation, and f3(·) represents a 3×3 convolutional layer operation. o This represents the output of the first residual module.

[0093] Similarly, the operation of the second residual module is the same as that of the first module, except that the output of the first residual module is used as the input of the second residual module to extract the mid-layer feature information. The operation process is as follows:

[0094] y1=f1(f3(f1(max{H(y o ),0}))) (5)

[0095] In the formula y o y1 represents the input feature, and y1 represents the output feature of the second residual module.

[0096] In the feature extraction module, an inverse residual module is used to extract deep features. To avoid information loss, a different structural shape is used compared to the residual module. The structure consists of batch normalization, ReLU activation function, 3×3 convolutional layer, 1×1 convolutional layer, and 1×1 convolutional layer stacked sequentially. The calculation process is shown below:

[0097] y2=f1(f1(f3(max{H(y1),0}))) (6)

[0098] In the formula, y1 represents the input feature and y2 represents the output feature of the inverse residual module.

[0099] (3) Furthermore, lateral connections are used in the constructed residual neural network. A pyramid is formed using three convolutional layers with dilation rates d = m + 2, d = m + 1, and d = m to obtain multi-scale defect features for extracting features at different levels. Shallow, mid, and deep features are then fused to reuse features at different levels and enrich the expression of defect features. The specific process is as follows: First, shallow features are processed by a 3×3 dilated convolutional layer with a dilation rate of d = m + 2. The expression is as follows:

[0100] y d1 =f d＝m+2 (y o (7)

[0101] In the formula, yo represents shallow features, and f d＝m+2 (·) represents a convolution operation with a 3×3 convolution kernel and an expansion rate of m+2, y d1 This represents the output of the dilated convolutional layer.

[0102] Then, a 3×3 dilated convolutional layer with an dilation rate of d = m+1 is used to process the mid-layer features. A linear mapping is applied to the input features, expressed as:

[0103] y d2 =f d＝m+1 (y1) (8)

[0104] In the formula, y1 is the middle layer feature, f d＝m+1 (·) represents a convolution operation with a 3×3 convolution kernel and an expansion rate of m+1, y d2 This represents the output of the dilated convolutional layer.

[0105] Deep features are processed using 3×3 dilated convolutional layers with an dilation rate of d = m. A linear mapping is applied to the input features, expressed as:

[0106] y d3 =f d＝m (y2) (9)

[0107] In the formula, y2 represents the deep feature, and f d＝m (·) represents a convolution operation with a 3×3 convolution kernel and an expansion rate of m, y d3 This represents the output of the dilated convolutional layer. The features processed by the three different dilated convolutional layers are then fused, as shown in the following expression:

[0108] y s =y d1 +y d2 +y d3 (10)

[0109] Where y sThese are the fused features. Then, to reduce computational complexity, average pooling is used to further process the fused features, its mathematical expression is:

[0110] S = Avg(y s (11)

[0111] In the formula, Avg(·) represents the averaging operation, and S represents the output of global average pooling. The fully connected layer then performs defect type classification and outputs the detection results. Its mathematical expression is shown below.

[0112] y l =F(∑w l S) (12)

[0113] In the formula, S represents the input feature, and w l y represents the connection weights of neurons in the intermediate layer, F(·) represents the convolution operation of the fully connected layer, and y represents the connection weights of neurons in the intermediate layer. l This is the output result. After the training phase, backpropagation is performed based on the obtained cross-entropy loss value to continuously adjust the training parameters. The process of calculating the cross-entropy loss value is shown below:

[0114]

[0115] In the formula, C represents the number of state categories, with corresponding labels L = [0, 1, ..., C-1]. β is the true label value corresponding to the random n states; γ is the predicted value, which is the result y. l The predicted label values ​​are the n random states mentioned above. Here, the size of n is determined by the minimum batch value n, which is formed by the combination of labels corresponding to C different states. The model performs calculations based on the batch value each time.

[0116] Unlike regular convolution, dilated convolution introduces a hyperparameter called the "dilation rate," which defines the distance between convolutional kernels during data processing. When the kernel size is the same, different dilation rates can be set to obtain different receptive fields, thus capturing multi-scale information. Let k be the original convolution kernel size, d be the dilation rate, and k* be the equivalent kernel size.

[0117] k * = k + (k-1) × (d-1) (1)

[0118] R l+1 * =R l +(k * -1)×S l (2)

[0119] Among them, R l+1 * It is the current sensory field, R lS represents the receptive field of the previous layer. l This represents the product of the strides of all preceding layers. In summary, with the same kernel size, as the dilation rate increases, the receptive field becomes larger, which means it can contain more global features and higher semantic features.

[0120] Finally, backpropagation is continuously performed based on the obtained loss value to update the parameters. In other words, this embodiment calculates the loss value using the cross-entropy loss function based on the model output, performs backpropagation in a timely manner based on the loss value, and uses the Adam optimizer to continuously update the network model parameters, iterating throughout the training process. When the predetermined final iteration count of 100 is reached, the minimum loss value is obtained, the final output result is obtained, the network training ends, and a trained deep residual neural network is obtained.

[0121] In this embodiment, the output layer sets sample labels as the network output based on the defect types recorded during image calibration. In the parameter settings, the minimum batch size is 70, and the learning rate is set to 0.0001. The network is trained 100 times, using backpropagation and stochastic gradient descent during training. The optimizer chosen is Adam (see reference: Adam: A Method for Stochastic Optimization).

[0122] (4) Input the test sample into the deep residual neural network and test the network.

[0123] Step 4: Input the images of the different defect types to be detected into the trained deep residual neural network (e.g., Figure 3 In the network model structure shown, features are extracted sequentially through the top-down residual module and inverse residual module. Features are then fused using dilated convolutional layers with dilation rates of d=m+2, d=m+1, and d=m in the lateral connections. Finally, a classifier identifies defects on the workpiece surface based on the fusion results.

[0124] The classification performance of the proposed model is represented by a confusion matrix, such as... Figure 6 As shown, the horizontal axis of the confusion matrix represents the predicted class value of the sample, and the vertical axis represents the actual state of the sample. It can be seen that the accuracy is approximately 97%.

[0125] In such Figure 7 As shown in the accuracy and loss curves, the accuracy and loss curves tend to stabilize during the continuous iteration process. The defect identification accuracy gradually approaches 1, and the loss value gradually approaches 0, proving the effectiveness of the invention.

[0126] Based on the same inventive concept, the present invention also provides a workpiece surface defect detection device, including an image acquisition module, a model construction module, and a defect recognition module.

[0127] Specifically, the image acquisition module is used to acquire multiple images of the surface of the workpiece with defects; the model building module is used to construct a deep residual neural network, which includes a feature extraction module, a pyramid module, and a classifier. The feature extraction module consists of two residual modules and one inverse residual module, the pyramid module consists of three dilated convolutional layers stacked sequentially with different dilation rates, and the classifier consists of global average pooling and fully connected layers; the defect recognition module is used to input multiple workpiece surface images into the deep residual neural network and output defect recognition results.

[0128] Specifically, the feature extraction module is used to extract defect features from the workpiece surface image; the through-pyramid module is used to capture features at different levels in the image at different scales and fuse the captured features at different levels; the classifier is used to identify defects on the workpiece surface based on the fusion result.

[0129] Each module in the aforementioned workpiece surface defect detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0130] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps in the embodiments of the workpiece surface defect detection method. Specific implementation methods can be found in the method embodiments, and will not be repeated here.

[0131] Furthermore, the present invention also provides a non-transitory computer-readable storage medium containing instructions, on which a computer program is stored. For example, a memory containing instructions that can be executed by a processor of a computer device to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device. When the computer program is executed by the processor, it can implement the steps in the embodiments of the workpiece surface defect detection method. Specific implementation methods can be found in the method embodiments, which will not be repeated here.

[0132] This invention addresses image data of gear components and employs a defect detection method based on a residual neural network using feature fusion at different levels. After defect calibration, the calibrated gear defect images are adjusted to a uniform size, highlighting the feature differences between different defects. Furthermore, the residual neural network structure based on feature fusion at different levels uses two residual modules to extract shallow and mid-level features, and an inverse residual module to extract deep features to prevent information loss. Low-level features have higher resolution and contain more positional and detailed information, but due to fewer convolutions, they have lower semantics and more noise. High-level features have stronger semantic information, but low resolution and poor perception of details. Therefore, three dilated convolutional layers with dilation rates d = m + 2, d = m + 1, and d = m are used to fuse feature maps at different levels, enabling the reuse of shallow and deep features, enriching the expression of defect information, and further improving the defect detection effect. This also lays the groundwork for subsequent intelligent defect detection methods for different practical problems.

[0133] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0134] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0135] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0136] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0137] It should be noted that the specific embodiments described above enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although the present invention has been described in detail in this specification and embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention; and all technical solutions and improvements that do not depart from the spirit and scope of the present invention are covered within the protection scope of the patent of the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for detecting surface defects in a workpiece, characterized in that, Includes the following steps: Acquire multiple images of the workpiece surface with defects; A deep residual neural network is constructed, comprising a feature extraction module, a pyramid module, and a classifier. The feature extraction module consists of two residual modules and one inverse residual module. The pyramid module consists of three dilated convolutional layers stacked sequentially with different dilation rates. The classifier consists of global average pooling and fully connected layers. Multiple images of the workpiece surface are input into the deep residual neural network, and the workpiece surface defect recognition result is output. The workpiece surface image is input into a deep residual neural network, and the workpiece surface defect identification result is output. The specific steps include: Defect features are extracted from the workpiece surface image using a feature extraction module. The pyramid module captures features at different levels in the image at different scales and then fuses the captured features at different levels. The classifier identifies defects on the workpiece surface based on the fusion results; The pyramid module consists of three dilated convolutional layers with dilation rates of d=m+2, d=m+1, and d=m, where m is the dilation rate at the base of the pyramid structure. The pyramid module captures features at different levels in the image at different scales and then fuses these captured features. The specific steps are as follows: The shallow features are first processed by a 3×3 dilated convolutional layer with an inflation rate of d=m+2, as shown in the following expression: (1) In the formula It is a shallow feature. This represents a convolution operation with a 3×3 convolution kernel and an expansion rate of m+2. This represents the output of a dilated convolutional layer; Then, a 3×3 dilated convolutional layer with an inflation rate of d=m+1 is used to process the mid-layer features, performing a linear mapping on the input features. The expression is as follows: (2) In the formula It is a mid-level feature. This represents a convolution operation with a 3×3 convolution kernel having an expansion rate of m+1. This represents the output of a dilated convolutional layer; Finally, a 3×3 dilated convolutional layer with an inflation rate of d=m is used to process the deep features, performing a linear mapping on the input features. The expression is as follows: (3) In the formula It is a deep feature. This represents a convolution operation with a 3×3 convolution kernel and an expansion rate of m. This represents the output of the dilated convolutional layer and fuses the features processed by three different dilated convolutional layers, as shown in the following expression: (4) in These are the features after fusion.

2. The workpiece surface defect detection method according to claim 1, characterized in that, When the convolution kernel size is the same, the pyramid module obtains different receptive fields by setting different dilation rates, thereby acquiring multi-scale information. Specifically, let k be the original convolution kernel size, d be the dilation rate, and k* be the equivalent kernel size: (5) (6) in, This is the sensory field of the current layer. Indicates the receptive field of the previous layer. It represents the product of the step sizes of all preceding layers.

3. The workpiece surface defect detection method according to claim 2, characterized in that, The feature extraction module first uses a 7×7 convolutional layer as the input convolutional layer. This input convolutional layer performs an initial convolution operation on the input image to extract low-level feature information, which is then passed to subsequent network layers for further feature extraction and learning. The extracted low-level feature information is further processed by two residual modules, which are composed of a 1×1 convolutional layer, a 3×3 convolutional layer, and a 1×1 convolutional layer stacked sequentially. The input of the residual module first passes through a batch normalization layer and the ReLU activation function, and then enters the convolutional layer. At the bottom layer of the feature extraction module, an inverse bottleneck structure is used to extract high-level defect features. This inverse bottleneck structure consists of a 3×3 convolutional layer and two 1×1 convolutional layers.

4. The workpiece surface defect detection method according to claim 3, characterized in that, The process of identifying defects on the workpiece surface using a classifier based on the fusion results specifically includes: input data being processed through multiple residual blocks to learn feature representations; global average pooling averaging the feature maps on each channel to convert them into a fixed-length vector; and global average pooling averaging the features at each location in the feature maps to enable the network to perceive the entire image globally. The fully connected layer receives the feature vector obtained from the global average pooling operation as input and maps it to different categories to achieve defect classification.

5. The method for detecting surface defects in a workpiece according to claim 1, characterized in that, It also includes training a deep residual neural network. The method for constructing training and test samples is as follows: defects are manually calibrated on the surface image of a workpiece with defects to obtain a workpiece defect image. The workpiece defect image is divided into training samples and test samples, and the defect images in the training samples are adjusted to the same size as image samples.

6. The method for detecting surface defects in a workpiece according to claim 5, characterized in that, The training process for the deep residual neural network includes: The input image sample data is preprocessed by an input convolutional layer with a 7×7 kernel, a batch normalization layer, and a ReLU activation function. The input convolutional layer performs an initial convolution operation on the input image samples. After the convolution operation, the output data is input into the batch normalization layer. The batch normalization layer standardizes each feature channel so that its mean is close to 0 and its variance is close to 1. After the batch normalization layer, the ReLU activation function is applied. The ReLU activation function sets all negative values ​​to zero and keeps all positive values ​​unchanged. The preprocessing process is shown in formula (7). (7) Where x′ represents the input data, This represents a 7×7 convolution operation. This represents the batch normalization operation; the ReLU activation function formula is: , This indicates the output characteristics after preprocessing; The feature extraction module extracts the feature information of the input grayscale image, including: firstly, the first residual module of the feature extraction module extracts shallow features, which is composed of three convolutions of 1×1, 3×3, and 1×1; the calculation process is shown in formula (8): (8) In the formula Indicates input features, This represents the batch normalization operation; the ReLU activation function formula is: , Represents a 1×1 convolutional layer operation. This represents a 3×3 convolutional layer operation. This represents the output of the first residual module; Similarly, the operation of the second residual module is the same as that of the first module. The output of the first residual module is used as the input of the second residual module to extract the mid-layer feature information. The operation process is as follows: (9) In the formula Indicates input features, This represents the output characteristics of the second residual module; In the feature extraction module, the inverse residual module is used to extract deep features. It uses a different structural shape than the residual module. The structure is composed of batch normalization, ReLU activation function, 3×3 convolutional layer, 1×1 convolutional layer and 1×1 convolutional layer stacked in sequence. The calculation process is as follows: (10) In the formula Indicates input features, This indicates the output characteristics of the inverse residual module; The constructed residual neural network uses lateral connections and consists of three different convolutional layers with dilation rates of d=m+2, d=m+1, and d=m to form the pyramid module. The fused features are further processed using average pooling, and its expression is as follows: (11) In the formula, This indicates an averaging operation. This represents the output of global average pooling; and the fully connected layer classifies the defect type and outputs the detection result, as shown in the following expression. (12) In the formula, S represents the input feature. The connection weights of the intermediate layer neurons. This represents the convolution operation in a fully connected layer. To output the results, backpropagation is performed based on the obtained cross-entropy loss value after the training phase, and the training parameters are continuously adjusted. The process of calculating the cross-entropy loss value is as follows: (13) In the formula, C represents the number of state categories, and its corresponding label L=[0,1,…,C-1]; These are the actual label values ​​corresponding to n random states; The predicted value is the result. The predicted label values ​​corresponding to the above n random states are used to continuously backpropagate based on the obtained loss values ​​to update the parameters. The process continues until the set final iteration number is reached, at which point the minimum loss value is obtained, and the workpiece surface defect detection result is output.

7. A workpiece surface defect detection device, characterized in that, include: The image acquisition module is used to acquire multiple images of the surface of a workpiece with defects; The model building module is used to build a deep residual neural network, which includes a feature extraction module, a pyramid module, and a classifier. The feature extraction module consists of two residual modules and one inverse residual module. The pyramid module consists of three dilated convolutional layers with different dilation rates stacked sequentially. The classifier consists of global average pooling and fully connected layers. The defect identification module is used to input multiple images of the workpiece surface into the deep residual neural network and output defect identification results. The defect identification module includes: The feature extraction module is used to extract defect features from the workpiece surface image; The pyramid module is used to capture features at different levels in an image at different scales and to fuse the captured features at different levels. A classifier is used to identify defects on the workpiece surface based on the fusion results; The pyramid module consists of three dilated convolutional layers with dilation rates of d=m+2, d=m+1, and d=m, where m is the dilation rate at the base of the pyramid structure. The pyramid module captures features at different levels in the image at different scales and then fuses these captured features. The specific steps are as follows: The shallow features are first processed by a 3×3 dilated convolutional layer with an inflation rate of d=m+2, as shown in the following expression: (1) In the formula It is a shallow feature. This represents a convolution operation with a 3×3 convolution kernel and an expansion rate of m+2. This represents the output of a dilated convolutional layer; Then, a 3×3 dilated convolutional layer with an inflation rate of d=m+1 is used to process the mid-layer features, performing a linear mapping on the input features. The expression is as follows: (2) In the formula It is a mid-level feature. This represents a convolution operation with a 3×3 convolution kernel having an expansion rate of m+1. This represents the output of a dilated convolutional layer; Finally, a 3×3 dilated convolutional layer with an inflation rate of d=m is used to process the deep features, performing a linear mapping on the input features. The expression is as follows: (3) In the formula It is a deep feature. This represents a convolution operation with a 3×3 convolution kernel and an expansion rate of m. This represents the output of the dilated convolutional layer and fuses the features processed by three different dilated convolutional layers, as shown in the following expression: (4) in These are the features after fusion.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is loaded by the processor, it is able to perform the steps of the method according to any one of claims 1 to 6.

Images