A washing machine surface defect detection method based on multi-scale feature fusion

Abstract

The application discloses a washing machine defect detection method based on multi-scale feature fusion, comprising the following steps: after obtaining a to-be-classified image, performing image processing and cutting on the to-be-classified image to obtain an image suitable for network input; inputting the input image into a convolutional neural network and deleting an output layer to obtain a local feature map of the input image; inputting the input image into a Vision Transformer network and deleting an output layer to obtain a global feature map of the input image; performing feature fusion on the local feature map and the global feature map of the to-be-classified image through a multi-scale feature fusion network to obtain enhanced features; and using the enhanced features to train a classification network and accelerating the convergence speed of the training process through dynamic adjustment of a learning rate. The method fuses the local features and the global features of the input image through a multi-scale feature fusion module, improves the accuracy of defect image detection, and improves the network training speed through dynamic adjustment of the learning rate.

Description

Technical Field

[0001] This application relates to the field of computer vision, and specifically to a method for detecting surface defects in washing machines based on multi-scale feature fusion. Background Technology

[0002] With social development and technological advancements, washing machines have become an essential home appliance for most families, and their quality directly affects their lifespan and user experience. However, during the manufacturing process, various defects such as scratches, dents, and bumps can appear on the surface of the washing machine cabinet due to factors such as mold wear and operational errors during production. To ensure product quality and a positive user experience, defect detection of the washing machine cabinet surface has become a crucial step in the manufacturing process.

[0003] Currently, most manufacturers still use manual visual inspection or flashlight inspection to detect defects in washing machine cabinets. This method relies heavily on worker experience, resulting in uncertainties in the inspection results and compromising accuracy and efficiency. Furthermore, the repetitive nature of the work over long periods can lead to worker fatigue, further reducing efficiency. Therefore, there is an urgent need for an automated inspection method that is fast and accurate. Summary of the Invention

[0004] The purpose of this invention is to provide an accurate and reliable method for detecting surface defects in washing machines based on multi-scale feature fusion, so as to solve the problems faced by the aforementioned background technology.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for detecting surface defects in washing machines based on multi-scale feature fusion, the method comprising the following steps:

[0006] S1: An industrial camera acquires an image of the surface of the object under test;

[0007] S2: Perform image preprocessing and cropping on the image obtained in S1 to obtain an image adapted to the network input;

[0008] S3: Input the input image obtained in S2 into the convolutional neural network to obtain the local features of the image;

[0009] S4: Input the input image obtained in S2 into the Vision Transformer network to obtain the global features of the image;

[0010] S5: Input local and global features into a multi-scale feature fusion network to obtain enhanced features of the input image;

[0011] S6: Enhanced features are used to train the classification network, and the convergence speed of the training process is accelerated by dynamically adjusting the learning rate. This method improves the accuracy of defect image detection by fusing local and global features of the input image through a multi-scale feature fusion module; and improves the network training speed by dynamically adjusting the learning rate. The trained network model classifies surface images, overcoming the problem of low detection efficiency in existing manual visual inspection methods.

[0012] In step S1, an industrial camera is used to acquire surface images of the object under test, including defective and non-defective images, for use in training the network model.

[0013] In step S2, the data preprocessing includes contrast-limited adaptive histogram equalization and bilateral filtering, and cropping is the process of cropping the image to a size suitable for the network input. The specific process is as follows:

[0014] S21: Improve the contrast between defects and background by using an adaptive histogram equalization algorithm that limits contrast;

[0015] S22: Apply a bilateral filtering algorithm to the image obtained in step S21 to reduce noise without reducing contrast;

[0016] S23: The image obtained in step S22 is cropped to 224*224 to create a washing machine surface image dataset, which is convenient to input into the classification network for network training;

[0017] In step S3, the input image is input into the convolutional neural network to obtain the local features of the image. The specific process is as follows: S31: The dataset obtained in step S3 is input into the convolutional neural network for training, passing through the convolutional layer and pooling layer in sequence.

[0018] S32: Remove the output layer from the convolutional neural network to obtain the local features of the image;

[0019] In step S4, the input image is fed into the Vision Transformer network to obtain the global features of the image. The specific process is as follows:

[0020] S41: Input the dataset obtained in step S3 into the Vision Transformer network for training, passing through the Embedding Layer, Position Embedding module and Transformer Encoder module in sequence;

[0021] S42: Delete the output layer in the Vision Transformer network to obtain the global features of the image;

[0022] As an improvement of the present invention, in step S5, the multi-scale feature fusion module consists of a Mixed channel, a Channel channel, and a Spatial channel. The local features and global features of the image are combined through the three channels of the multi-scale feature fusion module to obtain the enhanced features of the image. The specific process is as follows:

[0023] S51: The image local and global features are fused through a Mixed channel, and the output is as follows: F b =Sigmoid(Conv(T⊙α+U⊙β))

[0024] In the formula F b The fused features are the output of the Mixed channel, where T is the local feature, U is the global feature, α and β are learnable hyperparameters that will gradually learn to assign precise weights as learning progresses. Conv represents the convolution operation, and Sigmoid represents the Sigmoid non-linear activation.

[0025] S52: The image local and global features are fused through a channel, and the output is as follows: F a =Sigmoid(FC(Conv(F) p )))

[0026] In the formula F a F is the fused feature map output by the Channel. p For P s With P r Feature map after performing Concat channel connection, P s P is the local feature map after the mask average pooling operation. r The feature map is generated after integrating information from the Central Fusion Module and then performing Global Average Pooling.

[0027] S53: Channels improve the model's feature extraction capabilities by focusing on the correlation information between different channels and integrating the information between channels.

[0028] S54: The image local and global features are fused through the Spatial channel, and the output is as follows: F c =Sigmoid(Conv(cos(P)) s ,U)))

[0029] In the formula F c The fusion feature of the Spatial channel output, where cos represents the cosine operation;

[0030] S55: Convert the output F obtained from the Channel channela The output F obtained from the mixed channel b Multiply them to obtain AttentionMap A;

[0031] S56: Convert the output F obtained from the Spatial channel c The output F obtained from the mixed channel b Multiply them to obtain AttentionMap B;

[0032] S57: Attention Map A and Attention Map B integrate their respective features in both channel and spatial dimensions. Finally, the fused features are added element by element to complete the multi-scale feature fusion of local and global features.

[0033] S58: The Spatial channel focuses on the spatial information of image features, performs spatial transformation on image features, extracts key features, and improves the feature extraction capability of the model.

[0034] As an improvement to the present invention, step S6 specifically includes:

[0035] S61: Input the enhanced features into the classification function Softmax to perform image classification;

[0036] S62: The learning rate is updated using a cosine decay strategy to improve training stability and accelerate convergence. The cosine decay formula is as follows:

[0037]

[0038] In the formula η min η0 is the minimum learning rate set, η0 is the initial learning rate, T is the total number of training epochs, and t is the current number of training epochs.

[0039] S63: The cosine decay learning rate adjustment strategy makes the learning rate change smoothly during training, avoids drastic oscillations in the learning rate, improves the stability of the algorithm, and speeds up the training process.

[0040] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the aforementioned method for detecting surface defects in a washing machine based on multi-scale feature fusion.

[0041] A storage medium storing a computer program, which, when executed by a processor, implements the aforementioned method for detecting surface defects in washing machines based on multi-scale feature fusion.

[0042] Compared with existing technologies, the beneficial effects of this invention are: the solution achieves high detection accuracy for surface defects in washing machines in industrial settings; by fusing local and global features of the image under test through a three-channel multi-scale feature fusion module, an enhanced image containing the location and spatial information of both local and global features is obtained, thereby enhancing the saliency of the defect target features and thus overcoming the problem of low detection accuracy caused by the indistinct surface defect features of current washing machines, thus improving detection accuracy; by dynamically adjusting the learning rate, the convergence speed of the model is accelerated, improving the detection speed.

[0043] Experimental results show that the proposed multi-scale feature fusion-based surface defect detection method for washing machines achieves a defect detection accuracy of 98.87% and a precision of 97.98%, significantly improving upon the detection accuracy of other classification networks. This method can effectively identify the presence of defects from washing machine surface images, improving the accuracy of image-based defect detection. Attached Figure Description

[0044] Figure 1 These are common defects on the surface of washing machines, where (a) is a pit, (b) is a bump, and (c) is a scratch.

[0045] Figure 2 A flowchart of the washing machine surface defect detection method based on multi-scale feature fusion provided by the present invention;

[0046] Figure 3 The images are schematic diagrams of image preprocessing, where (a) is the original image of the washing machine surface, (b) is a schematic diagram after adaptive histogram equalization with contrast limiting, and (c) is a schematic diagram after bilateral filtering.

[0047] Figure 4 This is a schematic diagram of the structure of a washing machine surface defect detection method based on multi-scale feature fusion in this invention;

[0048] Figure 5 This is a schematic diagram of the three-channel multi-scale feature fusion module in this invention;

[0049] Figure 6 This is a schematic diagram of the defect detection part of the washing machine cabinet, where (a) is a defect-free image and (b) is a defective image. Detailed Implementation

[0050] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0051] During the manufacturing process of washing machines, defects such as pits, bumps, and scratches may appear on the surface of the casing due to factors such as mold wear and operational errors during production. Figure 1As shown, current industrial site inspections still rely on manual visual inspection or flashlight irradiation to detect defects in washing machine cabinets, and automation on industrial production lines has not yet been achieved.

[0052] Example: Figure 2 As shown, this invention provides a method for detecting surface defects in washing machines based on multi-scale feature fusion, achieving automatic detection of washing machine cabinet defects in industrial settings with high accuracy. The specific steps are as follows:

[0053] S1: Use an industrial camera to acquire images of the washing machine surface to facilitate the training and validation of the subsequent network model;

[0054] S2: Perform image preprocessing on the surface image to improve the contrast between defects and background. Then, crop the preprocessed image to 224*224 to adapt to the network input. These images are then used to create a dataset, which is divided into training and test sets in a 4:1 ratio.

[0055] S3: Input the training set images into the convolutional neural network to obtain the local features of the training set images;

[0056] S4: Input the training set images into the Vision Transformer network to obtain the global features of the images;

[0057] S5: Input local and global features into a multi-scale feature fusion network to obtain enhanced features of the input image; S6: Use the enhanced features to train the classification network, and accelerate the convergence speed of the training process by dynamically adjusting the learning rate.

[0058] In step S1, an industrial camera is used to collect a large number of images of the washing machine surface, including defective and non-defective images, to facilitate subsequent network training and testing.

[0059] In step S2, the surface image is preprocessed and cropped. The specific process is as follows:

[0060] S21: Perform contrast-limited adaptive histogram equalization on the washing machine surface image from step S1, such as... Figure 3 (b) Improve the contrast between defects and the background;

[0061] S22: Perform bilateral filtering on the image obtained in step S21, such as... Figure 3 (c) Reduce image noise while minimizing changes to contrast.

[0062] S23: Crop the images obtained in step S21 to 224*224 to fit the network input, and make these images into a dataset, which is divided into training set and test set in a 4:1 ratio.

[0063] Figure 4 This is a schematic diagram of the structure of a washing machine surface defect detection method based on multi-scale feature fusion in this invention: images are trained and classified in this network;

[0064] In step S3, the image is input into a convolutional neural network to obtain local features of the image. The specific process is as follows:

[0065] S31: Input the dataset obtained in step S3 into the convolutional neural network for training, passing it through convolutional layers and pooling layers in sequence;

[0066] S32: Remove the output layer from the convolutional neural network to obtain the local features of the image;

[0067] In step S4, the image is input into the Vision Transformer network to obtain the global features of the image. The specific process is as follows:

[0068] S41: Input the dataset obtained in step S3 into the Vision Transformer network for training, passing through the Embedding Layer, Position Embedding module and Transformer Encoder module in sequence;

[0069] S42: Delete the output layer in the Vision Transformer network to obtain the global features of the image;

[0070] In step S5, the multi-scale feature fusion module consists of a Mixed channel, a Channel channel, and a Spatial channel, such as... Figure 5 As shown, the local and global features of the image are combined through the three channels of the multi-scale feature fusion module to obtain the enhanced features of the image. The specific process is as follows:

[0071] S51: The image local and global features are fused through a Mixed channel, and the output is as follows: F b =Sigmoid(Conv(T⊙α+Uβ))

[0072] In the formula F b The fused features are the output of the Mixed channel, where T is the local feature, U is the global feature, α and β are learnable hyperparameters that will gradually learn to assign precise weights as learning progresses. Conv represents the convolution operation, and Sigmoid represents the Sigmoid non-linear activation.

[0073] S52: The image local and global features are fused through a channel, and the output is as follows: F a =Sigmoid(FC(Conv(F) p)))

[0074] In the formula F a F is the fused feature map output by the Channel. p For P s With P r Feature map after performing Concat channel connection, P s P is the local feature map after the mask average pooling operation. r The feature map is generated after integrating information from the Central Fusion Module and then performing Global Average Pooling.

[0075] S53: The image local and global features are fused through the Spatial channel, and the output is as follows: F c =Sigmoid(Conv(cos(P)) s ,U)))

[0076] In the formula F c The fusion feature of the Spatial channel output, where cos represents the cosine operation;

[0077] S54: Convert the output F obtained from the Channel channel a The output F obtained from the mixed channel b Multiply them to obtain AttentionMap A;

[0078] S55: Convert the output F obtained from the Spatial channel c The output F obtained from the mixed channel b Multiply them to obtain AttentionMap B;

[0079] S56: Attention Map A and Attention Map B fuse their respective features in both channel and spatial dimensions. Finally, the fused features are added element by element to complete the multi-scale feature fusion of local and global features.

[0080] Specifically, step S6 is as follows:

[0081] S61: Input the enhanced features into the classification function Softmax to perform image classification;

[0082] S62: The learning rate is updated using a cosine decay strategy to improve training stability and accelerate convergence. The cosine decay formula is as follows:

[0083]

[0084] In the formula ηmin η0 is the minimum learning rate set, η0 is the initial learning rate, T is the total number of training epochs, and t is the current number of training epochs.

[0085] The optimal model parameters are obtained by training the network model using a training set of images of surface defects on washing machines.

[0086] After setting the relevant parameters (as shown in Table 1), import the data to start the model training process and obtain the optimal result parameters.

[0087] Table 1 Network training parameters

[0088]

[0089] After the model training was completed, defect detection tests were performed using a test set of surface defect images from washing machines. The defect detection results are as follows: Figure 6 As shown, (a) is a non-defect image and (b) is a defect image.

[0090] Furthermore, this invention uses the confusion evidence statistical classification results, and the confusion matrix of the classification results is shown in Table 2.

[0091] Table 2. Confusion Matrix of Classification Results

[0092]

[0093] In this system, TP indicates that an image belonging to the washing machine's surface defect was correctly classified; FP indicates that an image not belonging to the washing machine's surface defect was misclassified as a defect; FN indicates that a surface defect was misclassified as a non-defect; and TN indicates that an image not belonging to the washing machine's surface defect was correctly classified. Accuracy, recall, precision, and F1 score are used as evaluation metrics; higher scores indicate better network classification performance.

[0094]

[0095] The results of the experimental classification are shown in Table 3.

[0096] Table 3 Performance Comparison of Classification Network Models

[0097]

[0098] Experimental results show that the washing machine surface defect detection method based on multi-scale feature fusion proposed in this invention achieves a defect detection accuracy of 98.87% on the washing machine surface defect dataset. Compared with the mainstream classification networks AlexNet, GoogLeNet, VGG16, VGG19, ResNet18, ResNet34, ResNet50, ResNet101, and ViT, the accuracy is improved by 10.09%, 10.64%, 8.7%, 7.98%, 6.89%, 5.22%, 5.63%, and 4.89%, respectively. In summary, compared with other mainstream image classification algorithms, the washing machine surface defect detection method based on multi-scale feature fusion has higher detection accuracy, proving the superiority of the method of this invention. The technical means disclosed in this invention are not limited to the technical means disclosed in the above embodiments, but also include technical solutions composed of any combination of the above technical features. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this invention, and these improvements and modifications are also considered to be within the scope of protection of this invention.

Claims

1. A method for detecting surface defects in washing machines based on multi-scale feature fusion, characterized in that, Includes the following steps: S1: An industrial camera acquires an image of the surface of the object under test; S2: Perform image preprocessing and cropping on the image obtained in S1 to obtain an image adapted to the network input; S3: Input the input image obtained in S2 into the convolutional neural network to obtain the local features of the image; S4: Input the input image obtained in S2 into the Vision Transformer network to obtain the global features of the image; S5: Input local and global features into a multi-scale feature fusion network to obtain enhanced features of the input image; S6: Use enhanced features to train the classification network and accelerate the convergence speed of the training process by dynamically adjusting the learning rate; In step S5, the multi-scale feature fusion module consists of a Mixed channel, a Channel channel, and a Spatial channel. The local and global features of the image are combined through the three channels of the multi-scale feature fusion module to obtain the enhanced features of the image. The specific process is as follows: S51: The image's local and global features are fused using the Mixed channel, and the output is as follows: In the formula The fused features are the output of the Mixed channel, where T is the local feature, U is the global feature, α and β are learnable hyperparameters that will gradually learn to assign precise weights as learning progresses. Conv represents the convolution operation, and Sigmoid represents the Sigmoid non-linear activation. S52: The image's local and global features are fused using the Channel, and the output is as follows: In the formula This is the fused feature map output from the Channel. for and Feature map after performing Concat channel concatenation This is a local feature map after the mask average pooling operation. The feature map is generated after integrating information from the Central Fusion Module and then performing Global Average Pooling. S53: The image local and global features are fused through the Spatial channel, and the output is as follows: In the formula The fusion feature of the Spatial channel output, where cos represents the cosine operation; S54: Output obtained from the Channel Output obtained with Mixed channel Multiply them to obtain AttentionMap A; S55: Output from the Spatial channel Output obtained with Mixed channel Multiply them to obtain AttentionMap B; S56: Attention Map A and Attention Map B fuse their respective features in both channel and spatial dimensions. Finally, the fused features are added element by element to complete the multi-scale feature fusion of local and global features.

2. The method for detecting surface defects in a washing machine based on multi-scale feature fusion as described in claim 1, characterized in that, In step S1, images of the surface of the object to be tested are acquired using an industrial camera, including defective and non-defective images, for use in training the network model.

3. The method for detecting surface defects in a washing machine based on multi-scale feature fusion as described in claim 1, characterized in that, In step S2, preprocessing includes contrast-limited adaptive histogram equalization and bilateral filtering, and cropping is the process of cropping the image to a size suitable for the network input. The specific process is as follows: S21: Improve the contrast between defects and background by using an adaptive histogram equalization algorithm that limits contrast; S22: Apply a bilateral filtering algorithm to the image obtained in step S21 to reduce noise without reducing contrast; S23: The image obtained in step S22 is cropped to 224*224 to create a washing machine surface image dataset, which is convenient to input into the classification network for network training.

4. The method for detecting surface defects in a washing machine based on multi-scale feature fusion as described in claim 1, characterized in that, In step S3, the input image is input into a convolutional neural network to obtain the local features of the image. The specific process is as follows: S31: Input the dataset obtained in step S3 into the convolutional neural network for training, passing it through convolutional layers and pooling layers in sequence; S32: Delete the output layer in the convolutional neural network to obtain the local features of the image.

5. The method for detecting surface defects in a washing machine based on multi-scale feature fusion as described in claim 1, characterized in that, In step S4, the input image is input into the Vision Transformer network to obtain the global features of the image. The specific process is as follows: S41: Input the dataset obtained in step S3 into the Vision Transformer network for training, passing through the Embedding Layer, PositionEmbedding module and TransformerEncoder module in sequence; S42: Delete the output layer in the Vision Transformer network to obtain the global features of the image.

6. The method for detecting surface defects in a washing machine based on multi-scale feature fusion as described in claim 1, characterized in that, Step S6 specifically involves: S61: Input the enhanced features into the classification function Softmax to perform image classification; S62: The learning rate is updated using a cosine decay strategy to improve training stability and accelerate convergence. The cosine decay formula is as follows: In the formula Given the current learning rate, The minimum learning rate is set. is the initial learning rate, is the total number of training epochs, and is the current number of training epochs.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the washing machine surface defect detection method based on multi-scale feature fusion as described in any one of claims 1 to 6 above.

8. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the washing machine surface defect detection method based on multi-scale feature fusion as described in any one of claims 1 to 6 above.

Images