Workpiece surface defect detection method and system

Abstract

The invention discloses a workpiece surface defect detection method and system, and relates to the field of image processing, and the method comprises the following steps: data preparation: collecting workpiece surface image samples, carrying out sample data processing, and classifying defect images into a micro defect data set and a slender defect data set; model construction: constructing a workpiece surface defect detection network model and a workpiece surface segmentation network model; model training: training a workpiece surface defect detection network model by using the micro defect data set, and training the workpiece surface defect detection network model by using the slender defect data set; and defect detection: respectively inputting a workpiece surface image to be detected into the workpiece surface segmentation network model and the workpiece surface defect detection network model for defect detection. According to the invention, the detection precision is improved, the number of parameters is reduced, and the production efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method and system for detecting defects on the surface of workpieces. Background Art

[0002] In recent years, with the rapid development of technology, deep learning has been increasingly widely used in the industrial field. For example, a mask defect detection method based on deep learning is used to solve the dilemma faced by traditional computer vision detection algorithms when dealing with various non-linear defects and virtual point defects on masks; an automatic road crack recognition algorithm based on deep learning provides an efficient way for road maintenance management. A multi-task model is used for automatic detection of steel plate defects. This model realizes pixel-based defect segmentation and severity estimation in a dual-branch network, improving the detection accuracy of steel plate defects and exceeding the original technical level in this field; a hybrid attention network adopts an adaptive spatial feature fusion (ASFF) module to improve the accuracy by extracting multi-scale information of defects and introduces the CIOU algorithm to optimize the training loss of the baseline model; aiming at the limitations of traditional detection algorithms in detecting defects on the surface of workpieces, a metal workpiece surface defect detection algorithm based on improved RCNN is proposed, which combines the advantages of RCNN and Fast RCNN, combines Fast RCNN with the region proposal network model (RPN), and trains the fully connected layer through a shared convolutional layer to construct a complete metal workpiece defect detection system; aiming at the problems of easy missed detection and false detection in machine vision detection when there are cracks, holes and other defects on the surface of powder metallurgy workpieces, an improved YOLOv5s target detection algorithm is proposed, which improves the network average detection accuracy (mAP) from 58.5% to 63.6%, and the detection speed reaches 65 ms; combining image generation technology and semi-supervised learning strategy, a metal workpiece surface defect segmentation method that only requires very few defect samples is proposed, which can achieve high segmentation accuracy under the condition of only using 5 defect sample images.

[0003] However, under the existing technology, although deep learning has made certain developments in workpiece defect detection, there are still some deficiencies:

[0004] Weak ability to detect tiny defects: The size of tiny defects is small and the features are not obvious. Existing deep learning models are difficult to effectively capture and identify them, resulting in low detection accuracy. Difficult to segment slender defects: The shapes of slender defects are diverse, the background is complex and the edges are blurred, which poses a great challenge for deep learning models to segment defects, and it is difficult to accurately segment the defect area. Limited classification accuracy of the model: The proportion of the defect area in the overall image is often small, which makes the model vulnerable to a large number of normal areas during classification, resulting in difficulty in improving the classification accuracy. Poor adaptability to edge devices: Due to the limited computing power of edge devices, when deep learning models run on edge devices, problems such as slow running speed and insufficient memory often occur, resulting in poor performance in actual industrial applications. Summary of the Invention

[0005] In view of the above deficiencies in the prior art, the present invention provides a workpiece surface defect detection method and system, which solve the problems of weak detection ability for tiny defects, difficult segmentation of slender defects, limited classification accuracy of the model, and poor adaptability to edge devices in the prior art.

[0006] In order to achieve the above invention purpose, on the one hand, the technical solution adopted by the present invention is: a workpiece surface defect detection method, including:

[0007] S1. Data preparation: Collect workpiece surface image samples, perform sample data processing, and classify the defect images into a tiny defect data set and a slender defect data set;

[0008] S2. Model construction: Construct a workpiece surface defect detection network model and a workpiece surface segmentation network model;

[0009] S3. Model training: Use the tiny defect data set to train the workpiece surface defect detection network model, and use the slender defect data set to train the workpiece surface defect detection network model;

[0010] S4. Defect detection: Input the workpiece surface image to be detected into the workpiece surface segmentation network model and the workpiece surface defect detection network model respectively for defect detection.

[0011] Further: S1 includes:

[0012] S11. Collect normal workpiece surface images and defective workpiece surface images, and perform data cleaning, data screening, geometric correction, defect annotation, and data augmentation to obtain preprocessed pattern samples;

[0013] S12. According to the defect annotation of the preprocessed pattern samples, classify the defective workpiece surface images into a tiny defect data set and a slender defect data set.

[0014] Further: In S2, the workpiece surface defect detection network model includes a defect backbone feature extraction network, a convolution module, and a first feature fusion module connected in sequence;

[0015] The backbone feature extraction network is used to extract the shallow feature map and deep feature map of the tiny defect image data;

[0016] The convolution module includes a parallel convolution module and a serial convolution module. The parallel convolution module is used to perform parallel convolution on the shallow feature map; the serial convolution module is used to perform serial convolution on the deep feature map;

[0017] The first feature fusion module is used to perform feature fusion on the parallel convolution result and the serial convolution result to generate image data with a rectangular detection frame and class label of the defect area.

[0018] Furthermore, the workpiece surface segmentation network model includes a segmentation backbone feature extraction network, a feature enhancement module, a feature fusion module, and a sampling module connected in sequence;

[0019] The segmentation backbone feature extraction network is used to extract the shallow feature map and the deep feature map of the slender defect image data;

[0020] The feature enhancement module is used to perform feature enhancement on the shallow feature map and the deep feature map respectively;

[0021] The second feature fusion module is used to perform feature fusion on the enhanced shallow feature map and the deep feature map;

[0022] The sampling module is used to perform upsampling or downsampling on the feature fusion to obtain the classification result of the workpiece surface segmentation network model.

[0023] Furthermore, the feature enhancement module uses a hybrid attention mechanism to perform feature enhancement on the input features. The processing method of the hybrid attention mechanism for the input features includes:

[0024] S21. Perform max pooling and average pooling on the input features respectively to obtain local features and global features;

[0025] S22. Input the local features and the global features into a shared fully connected layer to obtain weighted local features and weighted global features;

[0026] S23. Multiply the weighted local features and the weighted global features, and generate channel attention through the Relu activation function;

[0027] S24. Multiply the input features by the channel attention to obtain the enhanced channel attention;

[0028] S25. Input the enhanced channel attention into a convolutional layer and generate spatial attention through the Relu activation function;

[0029] S26. Multiply the enhanced channel attention by the spatial attention to obtain the output features of the hybrid attention mechanism.

[0030] Furthermore, both the workpiece surface defect detection network model and the workpiece surface segmentation network model are obtained through transfer learning using a pre-trained network model.

[0031] Furthermore, during the defect detection process in S4, the image after the defect is detected and / or segmented is subjected to fixed-point data optimization and systolic array data stream convolution operation optimization.

[0032] On the other hand, the present solution also provides a workpiece surface defect detection system for performing the workpiece surface defect detection method, including:

[0033] Data Processing and Preparation Module: Used to collect images of the workpiece surface, preprocess the collected images, and construct a dataset;

[0034] Micro Defect Detection Network Module: Based on the workpiece surface defect detection network model, used to extract multi-scale features of micro defects, generate defect detection boxes and class labels;

[0035] Slender Defect Segmentation Network Module: Based on the workpiece surface segmentation network model, used to segment the features of slender defects;

[0036] Network Training and Optimization Module: Used to train the micro defect detection network module and the slender defect segmentation network module, and optimize the model parameters according to the loss function;

[0037] Edge Hardware Acceleration Module: A hardware accelerator based on FPGA, used to perform fixed-point data optimization and systolic array data stream convolution operation optimization on the image after defect detection and / or segmentation;

[0038] Evaluation Module: Used to calculate the loss function in the defect detection process.

[0039] Furthermore: The expression of the loss function L calculated by the evaluation module is:

[0040] L = α·L Dice +(1 - α)·L Focal

[0041]

[0042] Among them, α is the weight factor, L Dice is the Dice loss, L Focal is the Focal loss, p i represents the pixel value in the predicted probability matrix, g i represents the pixel value in the true label matrix, ε is a non-zero constant, p t represents the probability of predicting as the positive class, α t represents the weight for balancing positive and negative samples, and γ is the focusing parameter.

[0043] The beneficial effects of the present invention are:

[0044] 1. When constructing the workpiece surface defect detection network model and the workpiece surface segmentation network model, transfer learning is used to lightweight the network architecture, which helps to perform edge computing;

[0045] 2. In the workpiece surface defect detection network model, a parallel convolution module and a serial convolution module are combined, which significantly reduces the amount of calculation and the number of model parameters while ensuring the accuracy;

[0046] 3. Introduce the channel and spatial dual attention mechanism into the workpiece surface segmentation network model to accurately locate the morphological features of slender defects;

[0047] 4. Based on the edge - side hardware acceleration module, design a systolic array acceleration circuit. Through parallel pipelining processing and data reuse, reduce the number of multiply - add operations in convolution operations and achieve real - time inference at the edge - side. Description of the Drawings

[0048] Figure 1 is the flowchart of the workpiece surface defect detection method;

[0049] Figure 2 is the technical roadmap for constructing the workpiece surface defect dataset;

[0050] Figure 3 is the technical roadmap for the workpiece surface defect detection network model;

[0051] Figure 4 is the structure diagram of the convolution module of the workpiece surface micro - defect detection network;

[0052] Figure 5 is the schematic diagram of the first feature fusion module;

[0053] Figure 6 is the technical roadmap for the workpiece surface segmentation network model;

[0054] Figure 7 is the example diagram of the segmentation backbone feature extraction network;

[0055] Figure 8 is the schematic diagram of the hybrid attention mechanism of the feature enhancement module;

[0056] Figure 9 is the example diagram of the U - Net model before improvement and the improved U - Net model;

[0057] Figure 10 is the basic logic schematic diagram of the edge - side hardware acceleration module;

[0058] Figure 11 is the schematic diagram of two data formats;

[0059] Figure 12 is the systolic array diagram at T = 2;

[0060] Figure 13 is the systolic array diagram at T = 3;

[0061] Figure 14 is the systolic array diagram at T = 7. Detailed Implementation Manner

[0062] The specific embodiments of the present invention will be described below to facilitate those skilled in the art to understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention defined and determined by the appended claims, these changes are obvious, and all inventions created using the concept of the present invention are within the scope of protection.

[0063] As Figure 1 shown, this embodiment provides a method for detecting workpiece surface defects, including:

[0064] S1. Data preparation: Collect workpiece surface image samples, perform sample data processing, and classify the defect images into a micro-defect data set and an elongated defect data set;

[0065] S2. Model construction: Construct a workpiece surface defect detection network model and a workpiece surface segmentation network model;

[0066] S3. Model training: Use the micro-defect data set to train the workpiece surface defect detection network model, and use the elongated defect data set to train the workpiece surface defect detection network model;

[0067] S4. Defect detection: Input the workpiece surface image to be detected into the workpiece surface segmentation network model and the workpiece surface defect detection network model respectively for defect detection.

[0068] S1 includes:

[0069] S11. Collect normal workpiece surface images and defective workpiece surface images, and perform data cleaning, data screening, geometric correction, defect annotation, and data augmentation to obtain preprocessed pattern samples;

[0070] S12. According to the defect annotation of the preprocessed pattern samples, classify the defective workpiece surface images into a micro-defect data set and an elongated defect data set.

[0071] As Figure 2 shown, when collecting normal workpiece surface images and defective workpiece surface images, image collection can be performed from the surfaces of workpieces including but not limited to high-voltage cables, integrated circuit QFN lead frames, printed circuit boards, diode glass shells, machine tool spindles, magnetic disk workpieces, and electronic commutators.

[0072] As Figure 3 shown, in S2, the workpiece surface defect detection network model includes a defect backbone feature extraction network, a convolution module, and a first feature fusion module connected in sequence;

[0073] The backbone feature extraction network is used to extract the shallow feature map and deep feature map of the micro-defect image data;

[0074] As an example, the backbone feature extraction network can adopt the MobileNetV3 network to realize the lightweight of the model and greatly reduce the number of parameters.

[0075] As Figure 4 shown, the convolution module includes a parallel convolution module and a serial convolution module. The parallel convolution module is used for parallel convolution of the shallow feature map; the serial convolution module is used for serial convolution of the deep feature map;

[0076] The first feature fusion module is used for feature fusion of the parallel convolution result and the serial convolution result to generate image data with a rectangular detection frame and class label of the defective area.

[0077] As Figure 5 shown, dilated convolution is used for feature fusion of the parallel convolution result and the serial convolution result, which can greatly reduce the amount of calculation and the number of model parameters while ensuring the accuracy.

[0078] As Figure 6 shown, the workpiece surface segmentation network model includes a segmentation backbone feature extraction network, a feature enhancement module, a feature fusion module, and a sampling module connected in sequence;

[0079] The segmentation backbone feature extraction network is used to extract the shallow feature map and the deep feature map of the slender defect image data;

[0080] The feature enhancement module is used to perform feature enhancement on the shallow feature map and the deep feature map respectively;

[0081] The second feature fusion module is used for feature fusion of the enhanced shallow feature map and the deep feature map;

[0082] The sampling module is used to perform upsampling or downsampling on the feature fusion to obtain the classification result of the workpiece surface segmentation network model.

[0083] As Figure 7 shown, as an example, the segmentation backbone feature extraction network of the workpiece surface segmentation network model adopts the ShuffleNetV2 backbone network, and the basic building unit of the ShuffleNetV2 backbone network can be as Figure 7 (a) shown, with channel split; or it can be as Figure 7 (b) shown, without channel split.

[0084] As Figure 8 shown, the feature enhancement module uses the hybrid attention mechanism to perform feature enhancement on the input feature. The processing method of the hybrid attention mechanism for the input feature includes:

[0085] S21. Perform max pooling and average pooling on the input feature respectively to obtain the local feature and the global feature;

[0086] S22. Input the local features and the full-image features into a shared fully-connected layer to obtain weighted local features and weighted full-image features;

[0087] S23. Multiply the weighted local features and the weighted full-image features, and generate channel attention through the Relu activation function;

[0088] S24. Multiply the input features by the channel attention to obtain enhanced channel attention;

[0089] S25. Input the enhanced channel attention into a convolutional layer and generate spatial attention through the Relu activation function;

[0090] S26. Multiply the enhanced channel attention by the spatial attention to obtain the output features of the hybrid attention mechanism.

[0091] The workpiece surface defect detection network model and the workpiece surface segmentation network model are both obtained through transfer learning using a pre-trained network model.

[0092] As Figure 9 shown, as an example, a specific structure of a pre-trained network model is provided. As Figure 9 (a) shown, the U-Net network is provided as the pre-trained network model. The U-Net was originally often used for biomedical image segmentation, and it performs excellently in image feature extraction and segmentation tasks, with strong adaptability and scalability, and is very suitable for steel defect detection.

[0093] In particular, as Figure 9 (b) shown, this embodiment provides an improvement scheme for the U-Net network, making the improved U-Net network more suitable for transfer learning and feature extraction. The improvements include:

[0094] Adjustment of the number of output categories: Since in practice, workpiece defect detection belongs to binary classification (i.e., defective and non-defective), the number of output categories at the end is changed from the original number applicable to biomedical multi-class segmentation to 2;

[0095] Optimization of the number of feature maps: Considering the characteristics of workpiece defect detection, to improve the overall recognition performance of the network, the number of feature maps in some layers is changed and the most suitable number of feature maps for workpiece defect data is found. For example, reducing the number of feature maps in some intermediate layers can not only reduce the computational complexity but also highlight the key features, reducing the amount of computation for subsequent data feature extraction.

[0096] In particular, during the defect detection process in S4, the images after the defects are detected and / or segmented are subjected to fixed-point data optimization and systolic array data flow convolution operation optimization.

[0097] Specifically, as Figure 10 shown, an edge-side hardware acceleration module is used for fixed-point data optimization and systolic array data stream convolution operation optimization.

[0098] Fixed-point data optimization:

[0099] In this embodiment, a zero-padding operation is adopted. The zero-padding operation is mainly to ensure that before the convolution operation, the tensors of the matrices are consistent, and at the same time control the convolution output size and retain the edge information. The association with floating-point numbers is reflected in the processing of data formats in hardware implementation.

[0100] Floating-point and fixed-point formats are the two main representations of real numbers in computing. As Figure 11 shown, the structures of these two formats for the same 32-bit length data are shown. The IEEE-754 single-precision binary floating-point format consists of 1 sign bit, 8 exponent bits, and 23 mantissa bits, totaling 32 bits. The data is represented by its significance and scaled by the exponent as a power of 2.

[0101] The term "floating-point number" means that the decimal point in a number can move relative to the significant digits. Therefore, the floating-point format can represent various numbers. The fixed-point format consists of integer bits (including the sign bit) and fractional bits. Fixed-point data is essentially binary data shifted by a given static factor. Different from floating-point data, the positions of the bits in the fixed-point format are fixed without additional displacement. Although the precision of the data is limited by the number of bits, the computational efficiency of fixed-point operations is higher. For hardware implementation, using fixed-point data types can reduce the area, power consumption, and latency of arithmetic processing units.

[0102] The zero-padding operation ensures the stability of the feature map size after convolution (such as padding 1 layer of zeros with a 3×3 convolution kernel to make the output the same size as the input) by filling zero values at the edges of the input feature map, and at the same time avoids the loss of edge pixel information. In the floating-point operation scenario, the alignment of the sign bit and the exponent bit is maintained to ensure that the filled zero values are correctly mapped to numerical zeros in the floating-point representation, avoiding feature extraction deviation caused by floating-point precision loss. This design not only meets the requirements of the network structure for size but also adapts to the hardware optimization needs of floating-point operations.

[0103] Systolic array data stream convolution operation optimization:

[0104] There are a large number of convolution operations in neural networks. Using a CPU for traditional matrix operations requires a large number of multiplications and additions. For example, calculating an n*n matrix requires n 3 multiplications and n 2(n - 1) additions are required, and data exchange with the memory is needed after each calculation, which reduces the operation efficiency. Therefore, in this design, a systolic matrix is designed by FPGA for convolution operation. A systolic array is an array structure in which data "flows" rhythmically among the processing units of the array in a pre-determined "pipelined" manner. During the data flow, all processing units simultaneously and parallelly process the data flowing through them, so it can achieve a very high parallel processing speed.

[0105] For example, when calculating a 3*3 matrix, the systolic array structure diagram is as Figure 12 、 Figure 13 and Figure 14 shown. Among them, the two matrices are distributed in a trapezoidal shape and enter the array in a "pipelined" manner for multiplication and addition operations. Multiple calculation units perform parallel operations, and the results are retained in the calculation unit after each calculation, which improves the data reuse rate and does not require data exchange with the memory. Using a systolic array to calculate a 3*3 matrix only requires 9 multiplications and 9 additions. By analogy, an n*n matrix requires n 2 multiplications and n 2 additions, reducing the number of multiplication and addition operations and improving the efficiency of convolution operation.

[0106] In this embodiment, a workpiece surface defect detection system is also provided for implementing the workpiece surface defect detection method, including:

[0107] Data processing and preparation module: used to collect the workpiece surface image, preprocess the collected image, and construct a data set;

[0108] Micro defect detection network module: based on the workpiece surface defect detection network model, used to extract multi-scale features of micro defects and generate defect detection boxes and class labels;

[0109] Slender defect segmentation network module: based on the workpiece surface segmentation network model, used to segment the features of slender defects;

[0110] Network training and optimization module: used to train the micro defect detection network module and the slender defect segmentation network module, and optimize the model parameters according to the loss function;

[0111] Edge-side hardware acceleration module: based on the FPGA hardware accelerator, used to perform fixed-point data optimization and systolic array data stream convolution operation optimization on the image after defect detection and / or segmentation;

[0112] Evaluation module: used to calculate the loss function during the defect detection process.

[0113] Specifically, the expression of the loss function L calculated by the evaluation module is:

[0114] L = α·L Dice +(1 - α)·L Focal

[0115]

[0116] where α is the weight factor, L Dice is the Dice loss, and L Focal is the Focal loss, p i represents the pixel value in the predicted probability matrix, and g i represents the pixel value in the ground truth label matrix, ε is a non - zero constant, and p t represents the probability of predicting the positive class, and α t represents the weight for balancing positive and negative samples, and γ is the focusing parameter.

[0117] This application combines a high - precision detection architecture design, a real - time optimization scheme, a low - resource adaptation structure, a generalization and robustness enhancement strategy, a defect classification detection mechanism, and a hardware acceleration integration process, making the present invention have the characteristics of high precision, high efficiency, and low resource consumption, and being applicable to real - time defect detection in industrial production lines.

[0118] Specifically, for the high - precision detection architecture design, the structural improvement: By means of lightweight networks (MobileNetV3 / ShuffleNetV2) and a feature fusion module, the number of parameters is reduced by 70%. Advantages: While significantly reducing the number of parameters, it can still maintain the mean average precision (mAP) of micro - defect detection not less than 85%, and the intersection over union (IoU) of slender - defect segmentation not less than 78%, achieving high - precision detection.

[0119] For the real - time optimization scheme, the structural improvement: Adopt an FPGA - based acceleration scheme. Advantages: The inference speed is increased to 30 frames per second (Zynq platform), fully meeting the stringent real - time requirements of industrial scenarios.

[0120] For the low - resource adaptation structure, the structural improvement: Apply fixed - point quantization technology and adopt a systolic array design. Advantages: The fixed - point quantization technology successfully compresses the model memory occupancy to less than 2MB, which can be adapted to low - computing - power edge devices such as Raspberry Pi and industrial industrial computers; the systolic array design reduces the number of convolution operations by 50% and the power consumption by 40%, greatly improving the resource utilization efficiency.

[0121] Generalization and robustness enhancement strategy, construction improvement: Implement a multi-source data augmentation strategy and adopt a hybrid loss function (Dice + Focal). Advantages: The multi-source data augmentation strategy enables the model to control the detection accuracy fluctuation within 5% when facing cross-workpiece types (metal / non-metal) and complex backgrounds; the hybrid loss function significantly reduces the missed detection rate from the original 15% to 3%, effectively ensuring the reliability and stability of the detection results.

[0122] Defect classification and detection mechanism, construction improvement: After obtaining the image, classify the defects in the image and adopt different defect segmentation and detection network structures for different categories. Advantages: Facilitates separate operations for different defects and improves the operation efficiency.

[0123] Hardware acceleration integration process, construction improvement: After the defects are detected or segmented, they are accelerated and integrated through the FPGA hardware acceleration circuit. Advantages: Accelerates the output result and efficiently ends the entire detection process.

[0124] The above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A workpiece surface defect detection method, characterized in that, Including: S1. Data Preparation: Collect workpiece surface image samples, process the sample data, and classify the defect images into a micro-defect data set and an elongated defect data set; S2. Model Construction: Construct a workpiece surface defect detection network model and a workpiece surface segmentation network model; S3. Model Training: Use the micro-defect data set to train the workpiece surface defect detection network model, and use the elongated defect data set to train the workpiece surface defect detection network model; S4. Defect Detection: Input the workpiece surface images to be detected into the workpiece surface segmentation network model and the workpiece surface defect detection network model respectively for defect detection.

2. The workpiece surface defect detection method according to claim 1, wherein S1 includes: S11. Collect normal workpiece surface images and defective workpiece surface images, and perform data cleaning, data screening, geometric correction, defect annotation, and data augmentation to obtain preprocessed pattern samples; S12. According to the defect annotation of the preprocessed pattern samples, classify the defective workpiece surface images into a micro-defect data set and an elongated defect data set.

3. The workpiece surface defect detection method according to claim 1, wherein In S2, the workpiece surface defect detection network model includes a defect backbone feature extraction network, a convolution module, and a first feature fusion module connected in sequence; The backbone feature extraction network is used to extract the shallow feature map and the deep feature map of the micro-defect image data; The convolution module includes a parallel convolution module and a serial convolution module. The parallel convolution module is used to perform parallel convolution on the shallow feature map; the serial convolution module is used to perform serial convolution on the deep feature map; The first feature fusion module is used to perform feature fusion on the parallel convolution result and the serial convolution result, and generate image data with a rectangular detection box and class label of the defect area.

4. The workpiece surface defect detection method according to claim 1, characterized in that, The workpiece surface segmentation network model includes a segmentation backbone feature extraction network, a feature enhancement module, a feature fusion module, and a sampling module connected in sequence; The segmentation backbone feature extraction network is used to extract the shallow feature map and the deep feature map of the elongated defect image data; The feature enhancement module is used to perform feature enhancement on the shallow feature map and the deep feature map respectively; The second feature fusion module is used to perform feature fusion on the enhanced shallow feature map and the deep feature map; The sampling module is used to perform upsampling or downsampling on the feature fusion to obtain the classification result of the workpiece surface segmentation network model.

5. The workpiece surface defect detection method according to claim 4, wherein The feature enhancement module uses a hybrid attention mechanism to perform feature enhancement on the input features. The processing method of the hybrid attention mechanism for the input features includes: S21. Perform max pooling and average pooling on the input features respectively to obtain local features and global features; S22. Input the local features and global features into a shared fully connected layer to obtain weighted local features and weighted global features; S23. Multiply the weighted local features and the weighted global features, and generate channel attention through the Relu activation function; S24. Multiply the input features by the channel attention to obtain the enhanced channel attention; S25. Input the enhanced channel attention into a convolutional layer and generate spatial attention through the Relu activation function; S26. Multiply the enhanced channel attention by the spatial attention to obtain the output features of the hybrid attention mechanism.

6. The workpiece surface defect detection method according to claim 1, characterized in that Both the workpiece surface defect detection network model and the workpiece surface segmentation network model are obtained through transfer learning from a pre-trained network model.

7. The workpiece surface defect detection method according to claim 1, wherein During the defect detection process in S4, the image after the defect is detected and / or segmented is optimized for fixed-point data and optimized for systolic array data stream convolution operation.

8. The workpiece surface defect detection system is characterized in that, For implementing the method according to any one of claims 1-7, comprising: Data processing and preparation module: for collecting workpiece surface images, preprocessing the collected images, and constructing a data set; Micro defect detection network module: based on the workpiece surface defect detection network model, for extracting multi-scale features of micro defects, generating defect detection boxes and class labels; Elongated defect segmentation network module: based on the workpiece surface segmentation network model, for segmenting features of elongated defects; Network training and optimization module: for training the micro defect detection network module and the elongated defect segmentation network module, and optimizing the model parameters according to the loss function; Edge-side hardware acceleration module: a hardware accelerator based on FPGA, for optimizing fixed-point data and optimizing systolic array data stream convolution operation on the image after the defect is detected and / or segmented; Evaluation module: for calculating the loss function of the defect detection process.

9. The workpiece surface defect detection system according to claim 8, wherein, The expression of the loss function L calculated by the evaluation module is: L = α·L Dice +(1 - α)·L Focal Among them, α is the weight factor, L Dice is the Dice loss, L Focal is the Focal loss, p i represents the pixel value in the predicted probability matrix, g i represents the pixel value in the true label matrix, ε is a non-zero constant, p t represents the probability of predicting the positive class, α t represents the weight for balancing positive and negative samples, and γ is the focusing parameter.

Images