Defect detection method and device, electronic equipment and computer readable storage medium

Abstract

The application provides a defect detection method and device, electronic equipment and a computer readable storage medium. The method comprises: acquiring a to-be-tested image of a target workpiece; inputting the to-be-tested image into a defect detection model to output corresponding defect prediction information, wherein the defect prediction information comprises predicted classification information and predicted position information of a defect; wherein the defect detection model comprises a feature extraction network, a feature fusion network and a classification prediction network; the feature extraction network is used for feature extraction on the to-be-tested image to obtain an intermediate feature map corresponding to the to-be-tested image; the feature fusion network is used for feature fusion on the intermediate feature map to obtain a fused feature map; and the classification prediction network is used for defect prediction on the fused feature map to obtain corresponding defect prediction information. The defect detection model is used for automatic detection, and manual defect detection is not required, so that the detection efficiency is higher.

Description

Technical Field

[0001] This application relates to the field of computer vision inspection technology, and in particular to defect detection methods, devices, electronic equipment and computer-readable storage media. Background Technology

[0002] Workpiece quality control is a crucial link in industrial production, directly affecting the quality of the entire system-level product. Therefore, automating and intelligentizing workpiece quality inspection is imperative. However, most workpiece manufacturers still rely on manual methods to inspect and control product quality. This is a time-consuming and labor-intensive process, requiring significant investment of manpower to ensure product quality. Manual inspection methods obviously have many shortcomings, such as high time and cost, worker fatigue and boredom from repetitive inspections, subjective results, and low efficiency.

[0003] Therefore, there is an urgent need to provide defect detection methods, devices, electronic equipment, and computer-readable storage media to improve existing technologies. Summary of the Invention

[0004] The purpose of this application is to provide a defect detection method, apparatus, electronic device, and computer-readable storage medium that eliminates the need for manual defect detection, improves detection efficiency, and enhances the detection accuracy and robustness of the defect detection model.

[0005] The objective of this application is achieved through the following technical solution:

[0006] In a first aspect, this application provides a defect detection method, the method comprising:

[0007] Acquire the image of the target workpiece to be measured;

[0008] The image to be tested is input into the defect detection model to output corresponding defect prediction information, which includes the predicted classification information and predicted location information of the defect.

[0009] The defect detection model includes a feature extraction network, a feature fusion network, and a classification prediction network.

[0010] The feature extraction network is used to extract features from the image under test to obtain an intermediate feature map corresponding to the image under test.

[0011] The feature fusion network is used to fuse features in the intermediate feature map to obtain a fused feature map.

[0012] The classification prediction network is used to predict defects in the fused feature map to obtain corresponding defect prediction information.

[0013] The beneficial effects of this technical solution are as follows: Defect detection is performed on the target workpiece using a defect detection model. The detection process is as follows: First, the image to be tested is input into the feature extraction network for feature extraction to obtain multiple intermediate feature maps corresponding to the image to be tested. Then, the feature fusion network is used to fuse these multiple intermediate feature maps to obtain the corresponding fused feature map. Finally, the classification prediction network is used to predict defects in the fused feature map to obtain the corresponding defect prediction information.

[0014] The defect detection method of this application utilizes a defect detection model to achieve automatic detection, eliminating the need for manual defect detection and resulting in high detection efficiency. Furthermore, compared to defect detection methods that utilize machine learning algorithms (which only include feature extraction and classification prediction), the defect detection model, in addition to feature extraction and classification prediction networks, also includes a feature fusion network. The feature fusion network can fuse multiple intermediate feature maps output by the feature extraction network, enabling the classification prediction network to predict defects based on the fused feature map, thereby improving the detection accuracy and robustness of the defect detection model.

[0015] In some optional embodiments, the feature extraction network includes N feature extraction modules stacked in layers, the input information of the first feature extraction module is the image to be tested, the output result of the previous feature extraction module is the input information of the next feature extraction module, and N is an integer greater than 1;

[0016] Each feature extraction module includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a pooling layer, a normalization layer, and an activation layer;

[0017] The process of extracting features from the image to be tested to obtain the intermediate feature map corresponding to the image to be tested includes:

[0018] For each feature extraction module, the corresponding first convolutional layer, second convolutional layer, third convolutional layer, pooling layer, normalization layer and activation layer are used to perform the first convolutional processing, the second convolutional processing, the third convolutional processing, pooling processing, normalization processing and activation processing on the input information to output the corresponding initial feature map, and the initial feature map is used as the output result of the feature extraction module;

[0019] The N initial feature maps output by the N feature extraction modules are used as the intermediate feature maps corresponding to the image under test.

[0020] The beneficial effects of this technical solution are as follows: The feature extraction network includes N feature extraction modules (the first feature extraction module to the Nth feature extraction module). Each feature extraction module includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a pooling layer, a normalization layer, and an activation layer. The feature extraction process of the feature extraction network is as follows: First, the image to be tested is used as the input information of the first feature extraction module. It passes through the first convolutional layer, the second convolutional layer, the third convolutional layer, the pooling layer, the normalization layer, and the activation layer of the first feature extraction module in sequence. The output is the first initial feature map corresponding to the first feature extraction module. Then, the first initial feature map is used as the input information of the second feature extraction module. After the same information processing (convolution, pooling, normalization, and activation), the output is the second initial feature map. And so on, the output of the previous feature extraction module is used as the input information of the next feature extraction module, so that the corresponding feature map information is continuously passed down between each feature extraction module, which further improves the detection accuracy and robustness of the defect detection model.

[0021] In some optional embodiments, N is an integer greater than 2;

[0022] Each feature fusion module includes an upsampling layer and a fusion layer;

[0023] The process of fusing features from the intermediate feature maps to obtain the fused feature map includes:

[0024] For the first feature fusion module, the corresponding upsampling layer is used to upsample the Nth initial feature map output by the Nth feature extraction module to obtain the first upsampled feature map. The corresponding fusion layer is used to fuse the first upsampled feature map with the N-1 initial feature map output by the (N-1)th feature extraction module to output the corresponding superimposed feature map. The superimposed feature map of the first feature fusion module is used as the output result of the first feature fusion module.

[0025] For the k-th feature fusion module, the output result of the (k-1)-th feature fusion module is upsampled using the corresponding upsampling layer to obtain the k-th upsampled feature map. The k-th upsampled feature map is then fused with the Nk-th initial feature map output by the Nk-th feature extraction module using the corresponding fusion layer to output the corresponding superimposed feature map. The superimposed feature map of the k-th feature fusion module is then used as the output result of the k-th feature fusion module, where k is an integer greater than 1 and not greater than N-2.

[0026] The superimposed feature map output by the (N-2)th feature fusion module is used as the fused feature map.

[0027] The beneficial effects of this technical solution are as follows: The feature fusion network includes N-2 feature fusion modules (the 1st feature fusion module to the N-2nd feature fusion module). Each feature fusion module includes an upsampling layer and a fusion layer. The feature fusion process of the feature fusion network is as follows: First, the Nth initial feature map output by the Nth feature extraction module and the N-1th initial feature map output by the N-1th feature extraction module are used as the input information of the 1st feature fusion module. The upsampling layer of the 1st feature fusion module upsamples the Nth initial feature map to obtain the 1st upsampled feature map. Then, the fusion layer of the 1st feature fusion module fuses the 1st upsampled feature map with the N-1th initial feature map to output the superimposed feature map of the 1st feature fusion module (i.e., the output result of the 1st feature fusion module). Then, the 1st feature fusion module... The superimposed feature map of the first feature fusion module and the Nk-th initial feature map output by the Nk-th feature extraction module are used as input information for the second feature fusion module. The upsampling layer of the second feature fusion module is used to upsample the superimposed feature map of the first feature fusion module to obtain the k-th upsampled feature map. Then, the fusion layer of the second feature fusion module is used to fuse the k-th upsampled feature map and the Nk-th initial feature map to output the superimposed feature map of the second feature fusion module. This process is repeated. The input information of each feature fusion module (k-th feature fusion module) includes the output result of the previous feature fusion module (k-1-th feature fusion module) and the corresponding (Nk-th) initial feature map, so that the corresponding feature map information is continuously passed down between each feature fusion module, further improving the detection accuracy and robustness of the defect detection model.

[0028] In some optional embodiments, the training process of the defect detection model includes:

[0029] Obtain a training set, which includes multiple training data. Each training data includes a sample workpiece image and the defect annotation information corresponding to the sample workpiece image. The defect annotation information includes the defect annotation classification information and annotation location information.

[0030] For each training data set, each sample workpiece image is input into the defect detection model to obtain the corresponding defect prediction information;

[0031] Using a preset loss function, the defect prediction information corresponding to each sample workpiece image, and the defect annotation information corresponding to each sample workpiece image, the loss corresponding to each sample workpiece image is calculated.

[0032] The parameters of the defect detection model are updated based on the loss corresponding to each sample workpiece image.

[0033] The beneficial effects of this technical solution are as follows: a defect detection model can be trained using a training set, which includes multiple training data sets. Each training data set includes a sample workpiece image and its corresponding defect annotation information. The training process of the defect detection model is as follows: the sample workpiece image is input into the defect detection model to obtain the defect prediction information corresponding to the sample workpiece image. The defect prediction information and defect annotation information of the sample workpiece image are analyzed and calculated using a preset loss function to obtain the loss corresponding to each sample workpiece image. The parameters of the defect detection model are updated according to the loss. The defect detection model trained in this way can obtain the defect prediction information corresponding to the target workpiece based on the image to be tested of the target workpiece, and the calculation results are highly accurate and reliable.

[0034] In some optional embodiments, after obtaining the training set, the training process of the defect detection model further includes:

[0035] For each training data, data augmentation is performed on each sample workpiece image to obtain a new sample workpiece image. The new sample workpiece image and its corresponding defect annotation information are then used as new training data and placed into the training set.

[0036] The data augmentation operations include at least one of the following: rotation, scaling, flipping, translation, and mirroring.

[0037] The beneficial effect of this technical solution is that the performance of the model is related not only to the network structure itself, but also to the amount of training data. The more training data, the better the performance of the model. The amount of training data often determines the upper limit of the model's performance.

[0038] Data augmentation allows for the generation of more training data based on existing data, making the augmented training data as close as possible to the true data distribution, thereby improving the detection accuracy of the defect detection model. Furthermore, data augmentation enables the defect detection model to learn more robust features, effectively improving its generalization ability.

[0039] In some optional embodiments, the method further includes:

[0040] Obtain the application industry of the target workpiece;

[0041] Based on the application industry, obtain the repair conditions corresponding to the target workpiece;

[0042] When the target workpiece does not meet the repair conditions, the target workpiece shall be scrapped or recycled.

[0043] When the target workpiece meets the repair conditions, a repair strategy for the target workpiece is obtained based on the defect prediction information of the target workpiece, so that the repair equipment can perform repair processing on the target workpiece according to the repair strategy.

[0044] The beneficial effects of this technical solution are as follows: different application industries have different performance requirements for the same workpiece. For a gasket with a defect category of scratch, the gasket may meet the repair conditions corresponding to the home furnishing industry, but it may not meet the repair conditions corresponding to the aerospace industry. The corresponding repair conditions can be determined according to the application industry of the target workpiece. When the target workpiece does not meet the repair conditions, it is scrapped or recycled. When the target workpiece meets the repair conditions, the repair strategy determined by the defect prediction information enables the repair equipment to repair the target workpiece according to the repair strategy.

[0045] Based on the application industry of the target workpiece, the corresponding repair conditions can be set specifically for the target workpiece. Workpieces that do not meet the repair conditions can be screened out, avoiding the waste of manpower and resources to repair workpieces that do not meet the repair conditions (high degree of defect, even if repaired, the requirements will not be met), and reducing repair costs.

[0046] In some alternative embodiments, the target workpiece is any one of the following: piston rod, gasket, bearing, and oil seal, and the predicted classification information is used to indicate at least one of the following defect categories: raw material defects, scratches, dents, and poor polishing, and the predicted location information is used to indicate the boundary box of the defect.

[0047] The beneficial effects of this technical solution are as follows: the target workpiece can be a piston rod, gasket, bearing, oil seal, etc. The predicted classification information is used to indicate the defect category, such as raw material defects, scratches, bumps, poor polishing, etc. The predicted location information is used to indicate the boundary box of the defect. In other words, the defect category of the target workpiece can be known through the predicted classification information, and the location and size of the defect area can be known through the position and size of the boundary box in the predicted location information.

[0048] Secondly, this application provides a defect detection device, the device comprising:

[0049] The image acquisition module is used to acquire the image of the target workpiece to be tested.

[0050] The defect prediction module is used to input the image to be tested into the defect detection model and output corresponding defect prediction information, which includes the predicted classification information and predicted location information of the defect.

[0051] The defect detection model includes a feature extraction network, a feature fusion network, and a classification prediction network.

[0052] The feature extraction network is used to extract features from the image under test to obtain an intermediate feature map corresponding to the image under test.

[0053] The feature fusion network is used to fuse features in the intermediate feature map to obtain a fused feature map.

[0054] The classification prediction network is used to predict defects in the fused feature map to obtain corresponding defect prediction information.

[0055] In some optional embodiments, the feature extraction network includes N feature extraction modules stacked in layers, the input information of the first feature extraction module is the image to be tested, the output result of the previous feature extraction module is the input information of the next feature extraction module, and N is an integer greater than 1;

[0056] Each feature extraction module includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a pooling layer, a normalization layer, and an activation layer;

[0057] The process of extracting features from the image to be tested to obtain the intermediate feature map corresponding to the image to be tested includes:

[0058] For each feature extraction module, the corresponding first convolutional layer, second convolutional layer, third convolutional layer, pooling layer, normalization layer and activation layer are used to perform the first convolutional processing, the second convolutional processing, the third convolutional processing, pooling processing, normalization processing and activation processing on the input information to output the corresponding initial feature map, and the initial feature map is used as the output result of the feature extraction module;

[0059] The N initial feature maps output by the N feature extraction modules are used as the intermediate feature maps corresponding to the image under test.

[0060] In some optional embodiments, N is an integer greater than 2;

[0061] Each feature fusion module includes an upsampling layer and a fusion layer;

[0062] The process of fusing features from the intermediate feature maps to obtain the fused feature map includes:

[0063] For the first feature fusion module, the corresponding upsampling layer is used to upsample the Nth initial feature map output by the Nth feature extraction module to obtain the first upsampled feature map. The corresponding fusion layer is used to fuse the first upsampled feature map with the N-1 initial feature map output by the (N-1)th feature extraction module to output the corresponding superimposed feature map. The superimposed feature map of the first feature fusion module is used as the output result of the first feature fusion module.

[0064] For the k-th feature fusion module, the output result of the (k-1)-th feature fusion module is upsampled using the corresponding upsampling layer to obtain the k-th upsampled feature map. The k-th upsampled feature map is then fused with the Nk-th initial feature map output by the Nk-th feature extraction module using the corresponding fusion layer to output the corresponding superimposed feature map. The superimposed feature map of the k-th feature fusion module is then used as the output result of the k-th feature fusion module, where k is an integer greater than 1 and not greater than N-2.

[0065] The superimposed feature map output by the (N-2)th feature fusion module is used as the fused feature map.

[0066] In some optional embodiments, the training process of the defect detection model includes:

[0067] Obtain a training set, which includes multiple training data. Each training data includes a sample workpiece image and the defect annotation information corresponding to the sample workpiece image. The defect annotation information includes the defect annotation classification information and annotation location information.

[0068] For each training data set, each sample workpiece image is input into the defect detection model to obtain the corresponding defect prediction information;

[0069] Using a preset loss function, the defect prediction information corresponding to each sample workpiece image, and the defect annotation information corresponding to each sample workpiece image, the loss corresponding to each sample workpiece image is calculated.

[0070] The parameters of the defect detection model are updated based on the loss corresponding to each sample workpiece image.

[0071] In some optional embodiments, after obtaining the training set, the training process of the defect detection model further includes:

[0072] For each training data, data augmentation is performed on each sample workpiece image to obtain a new sample workpiece image. The new sample workpiece image and its corresponding defect annotation information are then used as new training data and placed into the training set.

[0073] The data augmentation operations include at least one of the following: rotation, scaling, flipping, translation, and mirroring.

[0074] In some alternative embodiments, the apparatus further includes:

[0075] The industry acquisition module is used to acquire the application industry of the target workpiece.

[0076] The repair conditions module is used to obtain the repair conditions corresponding to the target workpiece based on the application industry.

[0077] The scrapping and recycling module is used to scrap or recycle the target workpiece when the target workpiece does not meet the repair conditions.

[0078] The repair processing module is used to obtain a repair strategy for the target workpiece based on the defect prediction information of the target workpiece when the target workpiece meets the repair conditions, so that the repair equipment can perform repair processing on the target workpiece according to the repair strategy.

[0079] In some alternative embodiments, the target workpiece is any one of the following: piston rod, gasket, bearing, and oil seal, and the predicted classification information is used to indicate at least one of the following defect categories: raw material defects, scratches, dents, and poor polishing, and the predicted location information is used to indicate the boundary box of the defect.

[0080] Thirdly, this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above-mentioned defect detection methods.

[0081] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above-described defect detection methods. Attached Figure Description

[0082] The present application will be further described below with reference to the accompanying drawings and embodiments.

[0083] Figure 1 This is a flowchart illustrating a defect detection method provided in an embodiment of this application.

[0084] Figure 2 This is a structural block diagram of a feature extraction module provided in an embodiment of this application.

[0085] Figure 3 This is a structural block diagram of a feature extraction network provided in an embodiment of this application.

[0086] Figure 4 This is a structural block diagram of a feature extraction network and a feature fusion network provided in an embodiment of this application.

[0087] Figure 5 This is a structural block diagram of a defect detection device provided in an embodiment of this application.

[0088] Figure 6 This is a structural block diagram of an electronic device provided in an embodiment of this application.

[0089] Figure 7This is a schematic diagram of the structure of a program product provided in an embodiment of this application. Detailed Implementation

[0090] The present application will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0091] See Figure 1 , Figure 1 This is a flowchart illustrating a defect detection method provided in an embodiment of this application.

[0092] The method includes:

[0093] Step S101: Obtain the image of the target workpiece to be tested;

[0094] Step S102: Input the image to be tested into the defect detection model to output the corresponding defect prediction information, which includes the defect prediction classification information and the prediction location information.

[0095] The defect detection model includes a feature extraction network, a feature fusion network, and a classification prediction network.

[0096] The feature extraction network is used to extract features from the image under test to obtain an intermediate feature map corresponding to the image under test.

[0097] The feature fusion network is used to fuse features in the intermediate feature map to obtain a fused feature map.

[0098] The classification prediction network is used to predict defects in the fused feature map to obtain corresponding defect prediction information.

[0099] Therefore, the defect detection model is used to detect defects in the target workpiece. The detection process is as follows: First, the image to be tested is input into the feature extraction network to extract features and obtain multiple intermediate feature maps corresponding to the image to be tested. Then, the feature fusion network is used to fuse these multiple intermediate feature maps to obtain the corresponding fused feature map. Finally, the classification prediction network is used to predict defects in the fused feature map to obtain the corresponding defect prediction information.

[0100] The defect detection method in this application utilizes a defect detection model to achieve automatic detection, eliminating the need for manual defect detection and resulting in high detection efficiency. Furthermore, compared to defect detection methods that utilize machine learning algorithms (which only include feature extraction and classification prediction), the defect detection model, in addition to feature extraction and classification prediction networks, also includes a feature fusion network. The feature fusion network can fuse multiple intermediate feature maps output by the feature extraction network, enabling the classification prediction network to predict defects from the fused feature map, thereby improving the detection accuracy and robustness of the defect detection model.

[0101] In some alternative embodiments, the target workpiece is any one of the following: piston rod, gasket, bearing, and oil seal, and the predicted classification information is used to indicate at least one of the following defect categories: raw material defects, scratches, dents, and poor polishing, and the predicted location information is used to indicate the boundary box of the defect.

[0102] Therefore, the target workpiece can be a piston rod, gasket, bearing, oil seal, etc. The predicted classification information is used to indicate the defect category, such as raw material defects (defects in the raw material itself), scratches, bumps, poor polishing, etc. The predicted location information is used to indicate the boundary box of the defect. In other words, the defect category of the target workpiece can be known through the predicted classification information, and the location and size of the defect area can be known through the position and size of the boundary box in the predicted location information.

[0103] In some optional embodiments, the feature extraction network includes N feature extraction modules stacked in layers, the input information of the first feature extraction module is the image to be tested, the output result of the previous feature extraction module is the input information of the next feature extraction module, and N is an integer greater than 1;

[0104] Each feature extraction module includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a pooling layer, a normalization layer, and an activation layer;

[0105] The process of extracting features from the image to be tested to obtain the intermediate feature map corresponding to the image to be tested includes:

[0106] For each feature extraction module, the corresponding first convolutional layer, second convolutional layer, third convolutional layer, pooling layer, normalization layer and activation layer are used to perform the first convolutional processing, the second convolutional processing, the third convolutional processing, pooling processing, normalization processing and activation processing on the input information to output the corresponding initial feature map, and the initial feature map is used as the output result of the feature extraction module;

[0107] The N initial feature maps output by the N feature extraction modules are used as the intermediate feature maps corresponding to the image under test.

[0108] Therefore, the feature extraction network includes N feature extraction modules (the first feature extraction module to the Nth feature extraction module). Each feature extraction module includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a pooling layer, a normalization layer, and an activation layer. The feature extraction process of the feature extraction network is as follows: First, the image to be tested is used as the input information of the first feature extraction module. It passes through the first convolutional layer, the second convolutional layer, the third convolutional layer, the pooling layer, the normalization layer, and the activation layer of the first feature extraction module in sequence. The output is the first initial feature map corresponding to the first feature extraction module. Then, the first initial feature map is used as the input information of the second feature extraction module. After the same information processing (convolution, pooling, normalization, and activation), the output is the second initial feature map. And so on, the output of the previous feature extraction module is used as the input information of the next feature extraction module, so that the corresponding feature map information is continuously passed between each feature extraction module, which further improves the detection accuracy and robustness of the defect detection model.

[0109] Feature extraction networks are also known as the backbone networks of defect detection models.

[0110] See Figure 2 , Figure 2 This is a structural block diagram of a feature extraction module provided in an embodiment of this application.

[0111] Each feature extraction module includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a pooling layer, a normalization layer, and an activation layer.

[0112] In this embodiment, the kernel size of the first, second, and third convolutional layers is not limited. For example, the kernel size of the first convolutional layer is 3*3, the kernel size of the second convolutional layer is 3*3, and the kernel size of the third convolutional layer is 1*1.

[0113] The role of convolutional layers (also known as convolutional neural networks, CNNs) is to extract image features from the image to be tested. These image features are represented by each pixel in the image in a combination or independently, such as the texture features and color features of the image.

[0114] The role of the normalization layer (Batch Normalization, BN) is to perform column normalization on the output of the convolutional layer, scaling it to an appropriate range, removing interference from the units of measurement, and preventing features with excessively small values ​​from being submerged. In addition, it can solve the problem of gradient vanishing during backpropagation and avoid model overfitting.

[0115] The function of pooling layers is to select features extracted from convolutional layers. Common pooling operations include max pooling and average pooling. Pooling layers are calculated by sliding a matrix window, similar to convolutional layers, except that instead of performing cross-correlation operations, they find the maximum or average value in the matrix corresponding to the matrix window.

[0116] The characteristics of pooling layers are: 1. Selecting image information that is not affected by position; 2. Reducing the dimensionality of features to improve the receptive field of subsequent features, that is, making a pixel after pooling correspond to a region in the previous image; 3. Because pooling layers do not perform backpropagation and reduce the number of variables in the feature map, pooling layers can reduce the amount of computation.

[0117] In a specific application, the pooling operation corresponding to the pooling layer is the max pooling operation. Each time the feature map passes through the pooling layer, the size of the feature map is reduced to 1 / 2 of its original size (the width and height of the feature map are reduced to 1 / 2 of their original size).

[0118] See Figure 3 , Figure 3 This is a structural block diagram of a feature extraction network provided in an embodiment of this application.

[0119] N=5. The feature extraction network includes 5 feature extraction modules (the first to the fifth feature extraction modules). After the image to be tested is processed by the first feature extraction module, the size of the first initial feature map is 1 / 2 of the size of the image to be tested. After the first initial feature map is processed by the second feature extraction module, the size of the second initial feature map is 1 / 4 of the size of the image to be tested. After the second initial feature map is processed by the third feature extraction module, the size of the third initial feature map is 1 / 8 of the size of the image to be tested. After the third initial feature map is processed by the fourth feature extraction module, the size of the fourth initial feature map is 1 / 16 of the size of the image to be tested. After the fourth initial feature map is processed by the fifth feature extraction module, the size of the fifth initial feature map is 1 / 32 of the size of the image to be tested. These 5 initial feature maps (the first to the fifth initial feature maps) are used as the intermediate feature maps corresponding to the image to be tested.

[0120] In some implementations, the activation layer can use the ReLU function. Firstly, compared to the sigmoid or tanh functions, the derivative of the ReLU function is easier to calculate. Backpropagation is essentially a process of continuously updating parameters. Because its derivative is not complex, the model training is simple and efficient. Secondly, the ReLU function itself is a non-linear function, turning all negative values ​​to 0 while leaving positive values ​​unchanged. This operation is called one-sided suppression, which, when applied to neural networks, can take the form of grid fitting of non-linear mappings. Thirdly, when values ​​are too large or too small, the derivatives of the sigmoid and tanh functions approach 0, while the ReLU function, as a non-saturating activation function, does not exhibit this phenomenon. Fourthly, since the computational result for values ​​less than 0 is 0, and only the computational result for values ​​greater than 0 has a value, overfitting can be reduced.

[0121] In some optional embodiments, N is an integer greater than 2;

[0122] Each feature fusion module includes an upsampling layer and a fusion layer;

[0123] The process of fusing features from the intermediate feature maps to obtain the fused feature map includes:

[0124] For the first feature fusion module, the corresponding upsampling layer is used to upsample the Nth initial feature map output by the Nth feature extraction module to obtain the first upsampled feature map. The corresponding fusion layer is used to fuse the first upsampled feature map with the N-1 initial feature map output by the (N-1)th feature extraction module to output the corresponding superimposed feature map. The superimposed feature map of the first feature fusion module is used as the output result of the first feature fusion module.

[0125] For the k-th feature fusion module, the output result of the (k-1)-th feature fusion module is upsampled using the corresponding upsampling layer to obtain the k-th upsampled feature map. The k-th upsampled feature map is then fused with the Nk-th initial feature map output by the Nk-th feature extraction module using the corresponding fusion layer to output the corresponding superimposed feature map. The superimposed feature map of the k-th feature fusion module is then used as the output result of the k-th feature fusion module, where k is an integer greater than 1 and not greater than N-2.

[0126] The superimposed feature map output by the (N-2)th feature fusion module is used as the fused feature map.

[0127] Therefore, the feature fusion network includes N-2 feature fusion modules (the 1st feature fusion module to the (N-2nd feature fusion module). Each feature fusion module includes an upsampling layer and a fusion layer. The feature fusion process of the feature fusion network is as follows: First, the Nth initial feature map output by the Nth feature extraction module and the N-1th initial feature map output by the (N-1th feature extraction module) are used as the input information of the 1st feature fusion module. The upsampling layer of the 1st feature fusion module upsamples the Nth initial feature map to obtain the 1st upsampled feature map. Then, the fusion layer of the 1st feature fusion module fuses the 1st upsampled feature map with the N-1th initial feature map to output the superimposed feature map of the 1st feature fusion module (i.e., the output result of the 1st feature fusion module). Then, the superimposed feature map of the 1st feature fusion module is... The feature map and the Nk-th initial feature map output by the Nk-th feature extraction module are used as input information for the second feature fusion module. The upsampling layer of the second feature fusion module is used to upsample the superimposed feature map of the first feature fusion module to obtain the k-th upsampled feature map. Then, the fusion layer of the second feature fusion module is used to fuse the k-th upsampled feature map and the Nk-th initial feature map to output the superimposed feature map of the second feature fusion module. This process is repeated. The input information of each feature fusion module (k-th feature fusion module) includes the output result of the previous feature fusion module (k-1-th feature fusion module) and the corresponding (Nk-th) initial feature map, so that the corresponding feature map information is continuously passed down between each feature fusion module, which further improves the detection accuracy and robustness of the defect detection model.

[0128] Feature fusion networks are also known as the neck network of defect detection models.

[0129] Since the size of the output feature map often becomes smaller after features are extracted from the input image through a feature extraction network, it is usually necessary to restore the image to its original size for further calculations. This operation of mapping the image from a small resolution to a large resolution is called upsampling. Upsampling can be implemented in the following ways:

[0130] 1. Interpolation, including bilinear interpolation, nearest neighbor interpolation, trilinear interpolation, etc.;

[0131] 2. Transposed convolution, also known as deconvolution, can make the output feature map larger than the input feature map by padding the input feature map with zeros and then performing standard convolution calculations.

[0132] Feature fusion refers to combining features extracted from an image into a single feature that is more discriminative than the input features (generating a new fused feature using multiple existing feature sets). On the one hand, it can obtain the most differentiated information from the multiple original feature sets involved in the fusion; on the other hand, it can eliminate redundant information caused by the correlation between different feature sets.

[0133] In a specific application, the size of the feature map is doubled each time it passes through an upsampling layer (the width and height of the feature map are each doubled).

[0134] See Figure 4 , Figure 4 This is a structural block diagram of a feature extraction network and a feature fusion network provided in an embodiment of this application.

[0135] N=5, the feature fusion network includes 3 feature fusion modules (the first feature fusion module to the third feature fusion module).

[0136] The data processing procedure of the first feature fusion module is as follows: the fifth initial feature map (size is 1 / 32 of the size of the image to be tested) is upsampled to obtain the first upsampled feature map (size is 1 / 16 of the size of the image to be tested). The first upsampled feature map and the fourth initial feature map are fused to obtain the corresponding first superimposed feature map (denoted as FeatureMap1).

[0137] The data processing procedure of the second feature fusion module is as follows: the first superimposed feature map is upsampled to obtain the second upsampled feature map (the size of which is 1 / 8 of the size of the image to be tested), and the second upsampled feature map is fused with the third initial feature map to obtain the corresponding second superimposed feature map (denoted as FeatureMap2).

[0138] The data processing procedure of the third feature fusion module is as follows: the second superimposed feature map is upsampled to obtain the third upsampled feature map (the size of which is 1 / 4 of the size of the image to be tested), and the third upsampled feature map is fused with the second initial feature map to obtain the corresponding third superimposed feature map (denoted as FeatureMap3).

[0139] The third superimposed feature map output by the third feature fusion module is used as the fusion feature map corresponding to the image under test.

[0140] In some implementations, the classification prediction network is also referred to as the head network of the defect detection model.

[0141] A classification detection network can include a pooling layer, two fully connected layers, and an activation layer. The activation layer of the classification detection network can use the sigmoid function, as shown in the formula:

[0142] The sigmoid function can be used to predict the type of defect.

[0143] The sigmoid function, also known as the logistic function, is used for the output of hidden layer neurons. Its value range is (0, 1), and it maps a real number to the interval (0, 1). It can be used for binary classification. It performs well when the features differ significantly or are not particularly large in difference.

[0144] In some optional embodiments, the training process of the defect detection model includes:

[0145] Obtain a training set, which includes multiple training data. Each training data includes a sample workpiece image and the defect annotation information corresponding to the sample workpiece image. The defect annotation information includes the defect annotation classification information and annotation location information.

[0146] For each training data set, each sample workpiece image is input into the defect detection model to obtain the corresponding defect prediction information;

[0147] Using a preset loss function, the defect prediction information corresponding to each sample workpiece image, and the defect annotation information corresponding to each sample workpiece image, the loss corresponding to each sample workpiece image is calculated.

[0148] The parameters of the defect detection model are updated based on the loss corresponding to each sample workpiece image.

[0149] Therefore, a defect detection model can be trained using a training set. The training set includes multiple training data sets, each containing a sample workpiece image and its corresponding defect annotation information. The training process of the defect detection model is as follows: input the sample workpiece image into the defect detection model to obtain the defect prediction information corresponding to the sample workpiece image; analyze and calculate the defect prediction information and defect annotation information of the sample workpiece image using a preset loss function to obtain the loss corresponding to each sample workpiece image; update the parameters of the defect detection model based on the loss. The defect detection model trained in this way can obtain the defect prediction information corresponding to the target workpiece based on the image to be tested of the target workpiece, and the calculation results are highly accurate and reliable.

[0150] In some implementations, the preset loss function may be the CIOU_Loss function.

[0151] A good regression localization loss function should consider the following three parameters: overlap area, center point distance, and aspect ratio. Regression localization losses mainly include the DIoU Loss function and the CIoU Loss function. The DIoU Loss function already considers the first two parameters (overlap area and center point distance), while the CIoU Loss function incorporates the aspect ratio parameter into the loss function.

[0152] In a specific application, defect prediction information is used to indicate the prediction box (the bounding box in the defect prediction information), and defect annotation information is used to indicate the annotation box (the bounding box in the defect annotation information).

[0153] The CIOU_Loss function can be used to calculate the degree of overlap between the predicted bounding box and the labeled bounding box, and the degree of overlap is used as the loss (defect localization) corresponding to the sample workpiece image.

[0154] In some optional embodiments, after obtaining the training set, the training process of the defect detection model further includes:

[0155] For each training data, data augmentation is performed on each sample workpiece image to obtain a new sample workpiece image. The new sample workpiece image and its corresponding defect annotation information are then used as new training data and placed into the training set.

[0156] The data augmentation operations include at least one of the following: rotation, scaling, flipping, translation, and mirroring.

[0157] Therefore, the performance of a model depends not only on the network structure itself, but also on the amount of training data. The more training data there is, the better the model's performance. The amount of training data often determines the upper limit of the model's performance.

[0158] Data augmentation allows for the generation of more training data based on existing data, making the augmented training data as close as possible to the true data distribution, thereby improving the detection accuracy of the defect detection model. Furthermore, data augmentation enables the defect detection model to learn more robust features, effectively improving its generalization ability.

[0159] In a specific application, scaling can be adaptive scaling. Adaptive scaling scales the image according to its aspect ratio, padding the shorter side with zeros, thus maintaining the aspect ratio and preventing deformation of defective parts due to scaling.

[0160] In some optional embodiments, the method further includes:

[0161] Obtain the application industry of the target workpiece;

[0162] Based on the application industry, obtain the repair conditions corresponding to the target workpiece;

[0163] When the target workpiece does not meet the repair conditions, the target workpiece shall be scrapped or recycled.

[0164] When the target workpiece meets the repair conditions, a repair strategy for the target workpiece is obtained based on the defect prediction information of the target workpiece, so that the repair equipment can perform repair processing on the target workpiece according to the repair strategy.

[0165] Therefore, different application industries have different performance requirements for the same workpiece. For a gasket with a defect category of scratch, the gasket may meet the repair conditions corresponding to the home furnishing industry, but it may not meet the repair conditions corresponding to the aerospace industry. The corresponding repair conditions can be determined according to the application industry of the target workpiece. When the target workpiece does not meet the repair conditions, it is scrapped or recycled. When the target workpiece meets the repair conditions, the repair strategy is determined based on the defect prediction information, so that the repair equipment can repair the target workpiece according to the repair strategy.

[0166] Based on the application industry of the target workpiece, the corresponding repair conditions can be set specifically for the target workpiece. Workpieces that do not meet the repair conditions can be screened out, avoiding the waste of manpower and resources to repair workpieces that do not meet the repair conditions (high degree of defect, even if repaired, the requirements will not be met), and reducing repair costs.

[0167] In some implementations, the correspondence between application industries and repair conditions can be established using a preset lookup table or a preset deep learning model.

[0168] By designing and establishing an appropriate number of neural computing nodes and a multi-layered computational hierarchy, and selecting suitable input and output layers, a pre-defined deep learning model can be obtained. Through learning and optimization of this pre-defined deep learning model, a functional relationship from input to output can be established. Although it is not possible to find a 100% accurate functional relationship between input and output, it can approximate the real-world correlation as closely as possible.

[0169] In a specific application, when the application industry is home furnishing, the repair condition can be that the defect depth of the workpiece is no more than 0.5mm; when the application industry is automotive, the repair condition can be that the defect depth of the workpiece is no more than 0.2mm.

[0170] In some implementations, the defect prediction information includes defect prediction classification information, and the correspondence between the prediction classification information and the repair strategy can be established by a preset lookup table or a preset deep learning model.

[0171] In a specific application, the target workpiece is a piston rod. When the predicted classification information indicates poor polishing, the repair strategy can be polishing; when the predicted classification information indicates scratches, the repair strategy can be spraying.

[0172] In some implementations, the application industries may include pharmaceuticals, textiles, automotive, machining, aerospace, catering, and home furnishings. Repair conditions may include at least one of the following: the defect depth of the workpiece is not greater than a preset depth threshold; the defect length of the workpiece is not greater than a preset length threshold. The defect depth can be the scratch depth or the dent depth, and the defect length can be the scratch length or the dent length.

[0173] The embodiments of this application do not limit the preset depth threshold and the preset length threshold. The preset depth threshold is, for example, 0.1mm, 0.2mm or 0.5mm, and the preset length threshold is, for example, 0.1mm, 0.2mm or 0.5mm.

[0174] Repair strategies may include at least one of the following: welding, spraying, polishing, and reaming.

[0175] Repair equipment may include at least one of the following: welding equipment, spraying machine, polishing machine, and electric drill.

[0176] In some implementations, the target workpiece is a piston rod.

[0177] With the development and maturation of artificial intelligence technology, its application in industrial quality inspection is becoming increasingly widespread, covering various industries such as pharmaceuticals, textiles, and machinery manufacturing. Piston rods (used in automotive support gas springs) are mainly used in the automotive manufacturing industry, where the quality requirements for workpieces are extremely stringent. Defects in piston rods primarily include raw material defects, scratches, dents, and poor polishing. Effectively detecting these defects is crucial to ensuring the quality of piston rod products.

[0178] Currently, the main methods for detecting surface defects in piston rods are manual visual inspection and machine learning-based detection methods. Machine learning detection methods use digital image processing technology, but because the images of surface defects on workpieces vary greatly, machine learning detection methods cannot achieve ideal results.

[0179] This application also provides a defect detection algorithm, the construction process of which is as follows:

[0180] Step a: Collect image data of defective piston rods from field equipment and construct a training dataset.

[0181] Step b: Design the input end of the defect detection algorithm and perform data augmentation on the training dataset images.

[0182] Data augmentation methods mainly include rotating and adaptively scaling image data to expand and enhance the dataset. Adaptive scaling refers to scaling the image according to its aspect ratio, padding the shorter sides with zeros, thus maintaining the image's proportions and preventing distortion of defective parts due to scaling.

[0183] Step c: Design the backbone network (also known as the feature extraction network) of the defect detection algorithm to extract features from the image output in step b and obtain the corresponding feature map.

[0184] The backbone network consists of a series of network modules (also known as feature extraction modules). These modules consist of convolutional layers (Convolutional Neural Networks, CNNs), pooling layers, batch normalization layers (BN), and activation layers (ReLU activation functions). The convolutional kernels used include 1x1 and 3x3 kernels.

[0185] Each time the feature map undergoes pooling, it is reduced by half. Five network modules (Block_1 to Block_5) are stacked together to form a 5-layer network. Each network module is called a Block, which constitutes the backbone network of the defect detection algorithm. The feature map output by each layer is half the size of the feature map output by the layer above it, and the feature map output by the bottom layer is 1 / 32 the size of the original image.

[0186] Step d: Design the neck network (also known as the feature fusion network) of the defect detection algorithm to perform feature fusion on the feature map output in step c, thereby enhancing the defect detection algorithm's ability to extract image features.

[0187] The image processing procedure for the neck network is as follows:

[0188] The feature map output by the last layer network module Block_5 in step c (which is 1 / 32 the size of the original image) is upsampled to obtain a feature map that is 1 / 16 the size of the original image. This feature map is then fused with the feature map output by the fourth layer network module Block_4 to obtain the corresponding superimposed feature map, denoted as FeatureMap p1.

[0189] FeatureMap1 is further upsampled and fused with the feature map output by Block_3, the third layer network module in step c, to output the corresponding superimposed feature map, denoted as FeatureMap2.

[0190] FeatureMap2 continues to be upsampled and fused with the feature map output by Block_2, the second layer network module in step c, to output the corresponding superimposed feature map, denoted as FeatureMap3.

[0191] Step e: Design the head network (also known as the classification prediction network) of the defect detection algorithm, predict the superimposed feature map (FeatureMap3) output in step d, and output the detection box and category information of the defect.

[0192] CIOU_Loss is used as the loss function for the detection boxes to calculate the degree of overlap between the predicted boxes and the labeled boxes.

[0193] The sigmoid function is used to predict category information, and the formula is as follows:

[0194] Step f: Train the designed defect detection algorithm and output the defect detection model.

[0195] Although similar algorithms on the market, such as Faster-RCNN and YOLO, can also achieve this function, their detection accuracy does not meet the requirements. The embodiments of this application are for piston rod surface defect detection. Based on the characteristics of the piston rod, a dedicated defect detection algorithm is designed with high detection accuracy.

[0196] By employing deep learning technology and utilizing convolutional neural networks to extract image features, train a deep model, and predict defects, the detection accuracy and robustness of the defect detection model are improved. This eliminates the need for manual visual inspection, significantly reduces the time required for piston rod inspection, and offers advantages such as saving manpower and reducing costs.

[0197] See Figure 5 , Figure 5 This is a structural block diagram of a defect detection device provided in an embodiment of this application.

[0198] The specific implementation of the defect detection device is consistent with the implementation method and the technical effect achieved as described in the embodiments of the above defect detection method, and some details will not be repeated here.

[0199] The device includes:

[0200] Image acquisition module 101 is used to acquire the image to be measured of the target workpiece;

[0201] The defect prediction module 102 is used to input the image to be tested into the defect detection model to output corresponding defect prediction information, the defect prediction information including defect prediction classification information and prediction location information;

[0202] The defect detection model includes a feature extraction network, a feature fusion network, and a classification prediction network.

[0203] The feature extraction network is used to extract features from the image under test to obtain an intermediate feature map corresponding to the image under test.

[0204] The feature fusion network is used to fuse features in the intermediate feature map to obtain a fused feature map.

[0205] The classification prediction network is used to predict defects in the fused feature map to obtain corresponding defect prediction information.

[0206] In some optional embodiments, the feature extraction network includes N feature extraction modules stacked in layers, the input information of the first feature extraction module is the image to be tested, the output result of the previous feature extraction module is the input information of the next feature extraction module, and N is an integer greater than 1;

[0207] Each feature extraction module includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a pooling layer, a normalization layer, and an activation layer;

[0208] The process of extracting features from the image to be tested to obtain the intermediate feature map corresponding to the image to be tested includes:

[0209] For each feature extraction module, the corresponding first convolutional layer, second convolutional layer, third convolutional layer, pooling layer, normalization layer and activation layer are used to perform the first convolutional processing, the second convolutional processing, the third convolutional processing, pooling processing, normalization processing and activation processing on the input information to output the corresponding initial feature map, and the initial feature map is used as the output result of the feature extraction module;

[0210] The N initial feature maps output by the N feature extraction modules are used as the intermediate feature maps corresponding to the image under test.

[0211] In some optional embodiments, N is an integer greater than 2;

[0212] Each feature fusion module includes an upsampling layer and a fusion layer;

[0213] The process of fusing features from the intermediate feature maps to obtain the fused feature map includes:

[0214] For the first feature fusion module, the corresponding upsampling layer is used to upsample the Nth initial feature map output by the Nth feature extraction module to obtain the first upsampled feature map. The corresponding fusion layer is used to fuse the first upsampled feature map with the N-1 initial feature map output by the (N-1)th feature extraction module to output the corresponding superimposed feature map. The superimposed feature map of the first feature fusion module is used as the output result of the first feature fusion module.

[0215] For the k-th feature fusion module, the output result of the (k-1)-th feature fusion module is upsampled using the corresponding upsampling layer to obtain the k-th upsampled feature map. The k-th upsampled feature map is then fused with the Nk-th initial feature map output by the Nk-th feature extraction module using the corresponding fusion layer to output the corresponding superimposed feature map. The superimposed feature map of the k-th feature fusion module is then used as the output result of the k-th feature fusion module, where k is an integer greater than 1 and not greater than N-2.

[0216] The superimposed feature map output by the (N-2)th feature fusion module is used as the fused feature map.

[0217] In some optional embodiments, the training process of the defect detection model includes:

[0218] Obtain a training set, which includes multiple training data. Each training data includes a sample workpiece image and the defect annotation information corresponding to the sample workpiece image. The defect annotation information includes the defect annotation classification information and annotation location information.

[0219] For each training data set, each sample workpiece image is input into the defect detection model to obtain the corresponding defect prediction information;

[0220] Using a preset loss function, the defect prediction information corresponding to each sample workpiece image, and the defect annotation information corresponding to each sample workpiece image, the loss corresponding to each sample workpiece image is calculated.

[0221] The parameters of the defect detection model are updated based on the loss corresponding to each sample workpiece image.

[0222] In some optional embodiments, after obtaining the training set, the training process of the defect detection model further includes:

[0223] For each training data, data augmentation is performed on each sample workpiece image to obtain a new sample workpiece image. The new sample workpiece image and its corresponding defect annotation information are then used as new training data and placed into the training set.

[0224] The data augmentation operations include at least one of the following: rotation, scaling, flipping, translation, and mirroring.

[0225] In some alternative embodiments, the apparatus further includes:

[0226] The industry acquisition module is used to acquire the application industry of the target workpiece.

[0227] The repair conditions module is used to obtain the repair conditions corresponding to the target workpiece based on the application industry.

[0228] The scrapping and recycling module is used to scrap or recycle the target workpiece when the target workpiece does not meet the repair conditions.

[0229] The repair processing module is used to obtain a repair strategy for the target workpiece based on the defect prediction information of the target workpiece when the target workpiece meets the repair conditions, so that the repair equipment can perform repair processing on the target workpiece according to the repair strategy.

[0230] In some alternative embodiments, the target workpiece is any one of the following: piston rod, gasket, bearing, and oil seal, and the predicted classification information is used to indicate at least one of the following defect categories: raw material defects, scratches, dents, and poor polishing, and the predicted location information is used to indicate the boundary box of the defect.

[0231] See Figure 6 This application also provides an electronic device 200, which includes at least one memory 210, at least one processor 220, and a bus 230 connecting different platform systems.

[0232] The memory 210 may include a readable medium in the form of volatile memory, such as random access memory (RAM) 211 and / or cache memory 212, and may further include read-only memory (ROM) 213.

[0233] The memory 210 also stores a computer program, which can be executed by the processor 220, causing the processor 220 to perform the steps of the defect detection method in the embodiments of this application. The specific implementation method is consistent with the implementation method and the technical effect achieved in the above-described defect detection method embodiments, and some contents will not be repeated.

[0234] The memory 210 may also include a utility 214 having at least one program module 215, such program module 215 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0235] Accordingly, processor 220 can execute the aforementioned computer program, and can also execute utility 214.

[0236] Bus 230 can represent one or more of several types of bus structures, including a memory bus or memory controller, peripheral bus, graphics acceleration port, processor, or a local bus using any of the various bus structures.

[0237] Electronic device 200 can also communicate with one or more external devices 240, such as keyboards, pointing devices, Bluetooth devices, etc., and with one or more devices capable of interacting with it, and / or with any device that enables it to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication can be performed via input / output interface 250. Furthermore, electronic device 200 can also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via network adapter 260. Network adapter 260 can communicate with other modules of electronic device 200 via bus 230. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 200, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0238] This application also provides a computer-readable storage medium for storing a computer program. When the computer program is executed, it implements the steps of the defect detection method in this application. The specific implementation method is consistent with the implementation method and the technical effect achieved in the above-described defect detection method embodiments, and some contents will not be repeated.

[0239] Figure 7 This embodiment illustrates a program product for implementing the aforementioned defect detection method. It can employ a portable compact disc read-only memory (CD-ROM) and include program code, and can run on a terminal device, such as a personal computer. However, the program product of this invention is not limited thereto. In this application, the readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. The program product can employ any combination of one or more readable media. A readable medium can be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0240] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium capable of sending, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, or any suitable combination thereof. Program code for performing operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code may be executed entirely on a user computing device, partially on a user device, as a standalone software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing devices can be connected to user computing devices via any type of network, including local area networks (LANs) or wide area networks (WANs), or they can be connected to external computing devices (e.g., via the Internet using an Internet service provider).

[0241] This application describes the invention from the perspectives of purpose, performance, progress, and novelty, and it meets the functional enhancement and use requirements emphasized by the Patent Law. The above description and drawings are merely preferred embodiments of this application and are not intended to limit this application. Therefore, all structures, devices, features, etc., that are similar to or identical to those of this application, i.e., all equivalent substitutions or modifications made in accordance with the scope of this patent application, shall fall within the scope of protection of this patent application.

Claims

1. A defect detection method, characterized in that, The method includes: Acquire the image of the target workpiece to be tested; the target workpiece is a piston rod; The image to be tested is input into a defect detection model to output corresponding defect prediction information, which includes defect prediction classification information and predicted location information. The training process of the defect detection model includes: acquiring a training set, which includes multiple training data, each training data including a sample workpiece image and defect annotation information corresponding to the sample workpiece image, the defect annotation information including defect annotation classification information and annotation location information; for each training data, inputting each sample workpiece image into the defect detection model to obtain corresponding defect prediction information; calculating the loss corresponding to each sample workpiece image using a preset loss function, the defect prediction information corresponding to each sample workpiece image, and the defect annotation information corresponding to each sample workpiece image; and updating the parameters of the defect detection model based on the loss corresponding to each sample workpiece image. The preset loss function is the CIOU_Loss function, which includes the DIoU Loss function and the CIoU Loss function. The DIoU Loss function incorporates parameters of overlapping area and center point distance into the CIOU_Loss function, while the CIoU Loss function incorporates parameters of aspect ratio. The defect prediction information is used to indicate the predicted bounding box, i.e., the bounding box in the defect prediction information. The defect annotation information is used to indicate the annotation box, i.e., the bounding box in the defect annotation information. The CIOU_Loss function calculates the degree of overlap between the predicted bounding box and the annotation box, and uses the degree of overlap as the loss corresponding to the sample workpiece image. Based on the loss corresponding to each sample workpiece image, the parameters of the defect detection model are updated. The defect detection model includes a feature extraction network, a feature fusion network, and a classification prediction network. The feature extraction network is used to extract features from the image under test to obtain an intermediate feature map corresponding to the image under test. The feature extraction network includes N feature extraction modules stacked in layers. The input information of the first feature extraction module is the image under test, and the output result of the previous feature extraction module is the input information of the next feature extraction module. N is an integer greater than 2. Each feature extraction module includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a pooling layer, a normalization layer, and an activation layer. The process of extracting features from the image under test to obtain an intermediate feature map corresponding to the image under test includes: for each feature extraction module, the corresponding first convolutional layer, second convolutional layer, third convolutional layer, pooling layer, normalization layer, and activation layer are used to perform a first convolutional processing, a second convolutional processing, a third convolutional processing, a pooling processing, a normalization processing, and an activation processing on the input information to output a corresponding initial feature map, and the initial feature map is used as the output result of the feature extraction module; the N initial feature maps output by the N feature extraction modules are used as the intermediate feature maps corresponding to the image under test. The feature fusion network is used to fuse the intermediate feature maps to obtain a fused feature map. Each feature fusion module includes an upsampling layer and a fusion layer. The process of fusing the intermediate feature maps to obtain the fused feature map includes: for the first feature fusion module, using the corresponding upsampling layer to upsample the Nth initial feature map output by the Nth feature extraction module to obtain a first upsampled feature map; using the corresponding fusion layer to fuse the first upsampled feature map with the N-1th initial feature map output by the (N-1)th feature extraction module to output a corresponding superimposed feature map, and combining the features of the first feature fusion module... The superimposed feature map is used as the output of the first feature fusion module. For the k-th feature fusion module, the output of the (k-1)-th feature fusion module is upsampled using the corresponding upsampling layer to obtain the k-th upsampled feature map. The k-th upsampled feature map is then fused with the Nk-th initial feature map output by the Nk-th feature extraction module using the corresponding fusion layer to output the corresponding superimposed feature map. The superimposed feature map of the k-th feature fusion module is used as the output of the k-th feature fusion module, where k is an integer greater than 1 and not greater than N-2. The superimposed feature map output by the (N-2)-th feature fusion module is used as the fused feature map. The classification prediction network is used to predict defects in the fused feature map to obtain corresponding defect prediction information.

2. The defect detection method according to claim 1, characterized in that, After obtaining the training set, the training process of the defect detection model further includes: For each training data, data augmentation is performed on each sample workpiece image to obtain a new sample workpiece image. The new sample workpiece image and its corresponding defect annotation information are then used as new training data and placed into the training set. The data augmentation operations include at least one of the following: rotation, scaling, flipping, translation, and mirroring.

3. The defect detection method according to claim 1, characterized in that, The method further includes: Obtain the application industry of the target workpiece; Based on the application industry, obtain the repair conditions corresponding to the target workpiece; When the target workpiece does not meet the repair conditions, the target workpiece shall be scrapped or recycled. When the target workpiece meets the repair conditions, a repair strategy for the target workpiece is obtained based on the defect prediction information of the target workpiece, so that the repair equipment can perform repair processing on the target workpiece according to the repair strategy.

4. The defect detection method according to any one of claims 1-3, characterized in that, The predicted classification information is used to indicate at least one of the following defect categories: raw material defects, scratches, dents, and poor polishing; the predicted location information is used to indicate the bounding box of the defect.

5. A defect detection device, characterized in that, The device includes: The image acquisition module is used to acquire the image of the target workpiece to be tested. The defect prediction module is used to input the image to be tested into the defect detection model and output corresponding defect prediction information, which includes the predicted classification information and predicted location information of the defect. The training process of the defect detection model includes: acquiring a training set, which includes multiple training data sets, each training data set including a sample workpiece image and corresponding defect annotation information, the defect annotation information including defect classification information and annotation location information; for each training data set, inputting each sample workpiece image into the defect detection model to obtain corresponding defect prediction information; calculating the loss corresponding to each sample workpiece image using a preset loss function, the defect prediction information corresponding to each sample workpiece image, and the defect annotation information corresponding to each sample workpiece image; and updating the parameters of the defect detection model based on the loss corresponding to each sample workpiece image. The preset loss function is the CIOU_Loss function, which includes the DIoU Loss function and the CIoU Loss function. The DIoU Loss function incorporates parameters of overlapping area and center point distance into the CIOU_Loss function, while the CIoU Loss function incorporates parameters of aspect ratio. The defect prediction information is used to indicate the predicted bounding box, i.e., the bounding box in the defect prediction information. The defect annotation information is used to indicate the annotation box, i.e., the bounding box in the defect annotation information. The CIOU_Loss function calculates the degree of overlap between the predicted bounding box and the annotation box, and uses the degree of overlap as the loss corresponding to the sample workpiece image. Based on the loss corresponding to each sample workpiece image, the parameters of the defect detection model are updated. The defect detection model includes a feature extraction network, a feature fusion network, and a classification prediction network. The feature extraction network is used to extract features from the image under test to obtain an intermediate feature map corresponding to the image under test. The feature extraction network includes N feature extraction modules stacked in layers. The input information of the first feature extraction module is the image under test, and the output result of the previous feature extraction module is the input information of the next feature extraction module. N is an integer greater than 2. Each feature extraction module includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a pooling layer, a normalization layer, and an activation layer. The process of extracting features from the image under test to obtain an intermediate feature map corresponding to the image under test includes: for each feature extraction module, the corresponding first convolutional layer, second convolutional layer, third convolutional layer, pooling layer, normalization layer, and activation layer are used to perform a first convolutional processing, a second convolutional processing, a third convolutional processing, a pooling processing, a normalization processing, and an activation processing on the input information to output a corresponding initial feature map, and the initial feature map is used as the output result of the feature extraction module; the N initial feature maps output by the N feature extraction modules are used as the intermediate feature maps corresponding to the image under test. The feature fusion network is used to fuse the intermediate feature maps to obtain a fused feature map. Each feature fusion module includes an upsampling layer and a fusion layer. The process of fusing the intermediate feature maps to obtain the fused feature map includes: for the first feature fusion module, using the corresponding upsampling layer to upsample the Nth initial feature map output by the Nth feature extraction module to obtain a first upsampled feature map; using the corresponding fusion layer to fuse the first upsampled feature map with the N-1th initial feature map output by the (N-1)th feature extraction module to output a corresponding superimposed feature map, and combining the features of the first feature fusion module... The superimposed feature map is used as the output of the first feature fusion module. For the k-th feature fusion module, the output of the (k-1)-th feature fusion module is upsampled using the corresponding upsampling layer to obtain the k-th upsampled feature map. The k-th upsampled feature map is then fused with the Nk-th initial feature map output by the Nk-th feature extraction module using the corresponding fusion layer to output the corresponding superimposed feature map. The superimposed feature map of the k-th feature fusion module is used as the output of the k-th feature fusion module, where k is an integer greater than 1 and not greater than N-2. The superimposed feature map output by the (N-2)-th feature fusion module is used as the fused feature map. The classification prediction network is used to predict defects in the fused feature map to obtain corresponding defect prediction information.

6. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the defect detection method according to any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the defect detection method according to any one of claims 1-4.

Images