Mobile phone rear cover defect identification and detection method based on multispectral image

Abstract

The invention relates to a mobile phone rear cover defect identification and detection method based on a multispectral image, and belongs to the technical field of image processing and machine vision. Firstly, in a black room environment, a light source with multispectral distribution is used for irradiating a rear cover of a mobile phone to collect multispectral pictures required by an experiment, and the multispectral pictures comprise pictures irradiated by 16 different spectrum sections in visible light; then, the images under the spectrums are preprocessed, including the steps of denoising, illumination compensation and the like, so that the quality and the consistency of the images of the different spectrums are ensured; then, spectrum reconstruction data enhancement is carried out on the collected multispectral images, so that more color information related to defects is obtained; and finally, a cavity space pooling pyramid model and an SE attention module are added to the YOLOv7 model, the feature extraction and recognition capability of a defect area is enhanced, the preprocessed multispectral image is trained and detected by adopting an improved YOLOv7 algorithm, and the method has the advantages of high robustness and high detection precision and is suitable for automatic quality detection of the mobile phone rear cover.

Description

Technical Field

[0001] The invention relates to a method for identifying and detecting defects of a mobile phone back cover based on multispectral images, and belongs to the technical field of image processing and machine vision. Background Art

[0002] In the process of mobile phone manufacturing, the quality of the back cover of the mobile phone directly affects the appearance and user experience of the product. Traditional defect detection methods mainly rely on manual detection or automated detection of single spectral images, but these methods have limitations when dealing with complex lighting environments or tiny defects. With the development of multispectral imaging technology, images under different spectra can provide more surface and structural information, making it possible for more accurate defect detection. However, the processing of multispectral images, effective feature extraction, and high-precision defect identification are still challenges in current technology. In response to these problems, the present invention proposes a method for identifying and detecting defects in the back cover of a mobile phone based on multispectral images, which can significantly improve the accuracy of defect detection. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a method for identifying and detecting defects in the back cover of a mobile phone based on multispectral images. In complex lighting environments, there are limitations in the processing of multispectral images, effective feature extraction, and high-precision defect identification.

[0004] The technical solution of the present invention is: a method for identifying and detecting defects on the back cover of a mobile phone based on multispectral images. By combining image data under different spectra with deep learning technology, it can effectively identify and detect complex defects on the back cover of a mobile phone and improve the detection performance under complex lighting environments. The specific steps are:

[0005] Step 1: Use a multi-spectral camera to obtain image data of the back cover of the mobile phone under different spectra, which include 16 different bands of spectrum under visible light. Different spectra can capture different defect information. For example, defects are not obvious under white light but are relatively more obvious under yellow light.

[0006] Step 2: Perform preprocessing operations such as denoising, cropping, and illumination compensation on the collected images to improve image quality and consistency;

[0007] Step 3: Label the collected multispectral data of the mobile phone shell;

[0008] Step 4: Improve YOLOv7 in two parts: the void space pooling pyramid and the SE attention module;

[0009] Step 5: Use the labeled data set to train the improved YOLOv7 model;

[0010] Step 6: Use the trained model to detect the mobile phone shell image;

[0011] Step7: Output the defect detection results.

[0012] The Step 1 is specifically as follows:

[0013] The 16 different bands of light are divided into the following categories according to color: dark red, white, orange, pink, brown, red, yellow, blue, cool white, green, warm white, light blue, dark red, dark yellow, dark blue, and purple.

[0014] The Step 2 is specifically as follows:

[0015] Step 2.1: Use median filtering or bilateral filtering to denoise the image and reduce the interference of noise on defect identification.

[0016] Step 2.2: Use methods such as adaptive histogram equalization to compensate for uneven illumination in the image and ensure consistent brightness of the multispectral image.

[0017] Step 2.3: Since multispectral imaging devices may be displaced, a feature point matching algorithm is used to align images under different spectra to ensure the accuracy of subsequent image fusion and feature extraction.

[0018] The Step 3 is specifically as follows: using the LabelImg tool to label the defect image, and classifying the defects into block-shaped heterochromatic defects and point-shaped small target defects.

[0019] The Step 4 is specifically as follows:

[0020] First, a dilated space pooling pyramid module is added to the detection head of the YOLOv7 model. The dilated space pooling pyramid module allows the network to expand the receptive field of the convolution operation during the feature extraction process without reducing the resolution. Then, a SE attention mechanism module is added to the main body of the module. The importance of each channel of the feature map is obtained through automatic learning. A weight value is assigned to each channel using the importance, so that the model focuses on learning channel features with large weight values.

[0021] The Step 6 is specifically as follows:

[0022] Use the weight file trained in Step 5 to detect the defective images of mobile phone back covers in the test set.

[0023] The Step 7 is specifically as follows:

[0024] The detection result graph includes the defect type, defect location, defect size and detection confidence, as well as the numerical result output of precision, recall rate and mAP value.

[0025] The beneficial effect of the present invention is that, compared with the prior art, the present invention has the following advantages:

[0026] (1) By using multispectral images, more defect information can be detected under different spectra, improving the comprehensiveness and accuracy of detection.

[0027] (2) Through image preprocessing and fusion, the consistency of images with different spectra is ensured, and the interference of uneven illumination and noise on the detection results is reduced.

[0028] (3) The two modules of the void space pooling pyramid and attention mechanism in the deep learning model enhance the ability to recognize subtle defects, which is particularly suitable for the complex task of detecting defects on the back cover of mobile phones.

[0029] (4) This method is highly robust and can adapt to defect detection needs under different lighting conditions, and the detection results are stable and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a flow chart of the steps of the present invention;

[0031] Figure 2 It is an example diagram of a data set for the experiment of the present invention;

[0032] Figure 3 is a schematic diagram of spectrum reconstruction used in the present invention;

[0033] Figure 4 It is a structural diagram of the hollow space pooling pyramid module of the improved model of the present invention;

[0034] Figure 5 It is a structural diagram of the SE attention mechanism improved by the model of the present invention;

[0035] Figure 6 It is a structural diagram of the improved YOLOv7 model of the present invention;

[0036] Figure 7 It is a diagram of the detection result of the mobile phone back cover in the experiment of the present invention. DETAILED DESCRIPTION

[0037] The present invention will be further described below in conjunction with the accompanying drawings and specific implementation methods.

[0038] Example 1: Figure 1 As shown, a method for identifying and detecting defects in the back cover of a mobile phone based on multispectral images, the specific steps are as follows:

[0039] Step 1: Place the multispectral camera in a dark environment and use it to obtain image data of the back cover of the mobile phone under different spectra;

[0040] Step 2: Perform pre-processing operations such as denoising and illumination compensation on the obtained multi-spectral images of the mobile phone shell;

[0041] Step 3: Use the spectral reconstruction data enhancement method to enhance the mobile phone case image under white light;

[0042] Step 4: Improve the YOLOv7 model in two parts;

[0043] Step 5: Train the improved YOLOv7 model;

[0044] Step 6: Use the improved YOLOv7 model to perform defect detection on the multispectral images of the mobile phone casing;

[0045] Step7: Output the defect detection results.

[0046] The Step 1 is specifically as follows:

[0047] The multi-spectral camera is used to obtain image data of the back cover of the mobile phone under different spectra. The spectrum includes 16 different bands of spectrum under visible light, which can be divided into: dark red, white, orange, pink, brown, red, yellow, blue, cold white, green, warm white, light blue, dark red, dark yellow, dark blue, and purple according to the color of light. Different spectra can capture different defect information. For example, defects under white light are not obvious but are relatively more obvious under yellow light.

[0048] The Step 2 is specifically as follows:

[0049] Step 2.1: Multispectral image denoising operations include median filtering and bilateral filtering to reduce the interference of noise on defect identification.

[0050] Step 2.2: The illumination compensation step includes using an adaptive histogram equalization method to reduce the impact of uneven illumination on defective images.

[0051] The Step 3 is specifically as follows:

[0052] After enhancement, images of the other 31 different spectral channels corresponding to white light illumination conditions can be obtained, and a large number of high-quality multi-spectral images of simulated mobile phone back covers can be obtained.

[0053] The Step 4 is specifically as follows:

[0054] Step 4.1: The atrous spatial pyramid pooling module is introduced into the YOLOv7 network architecture. The atrous convolution at different atrous rates is used to capture information of different scales. The global information is further captured through global average pooling, which significantly enhances the performance of the model in the defect detection task in this paper.

[0055] Step 4.2: The SE attention mechanism module is introduced into the YOLOv7 network architecture. In this way, the model can automatically identify more important parts in the feature map and reduce the interference of irrelevant background information.

[0056] The Step 7 is specifically as follows:

[0057] The detection result graph includes the defect type, defect location, defect size and detection confidence, as well as the numerical result output of precision, recall rate and mAP value.

[0058] The specific implementation modes of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A method for identifying and detecting defects in a mobile phone back cover based on multispectral images, characterized in that: Step 1: Use a multispectral camera to obtain multispectral image data of the back cover of the mobile phone under different spectra, where the spectrum includes 16 different bands of light in the visible light; Step 2: Preprocess the multispectral image data by denoising, cropping, and illumination compensation; Step 3: Complete multispectral image data fusion by weighted fusion of image features under different spectra; Step 4: Add the dilated space pooling pyramid module and SE attention mechanism model to the detection head of the YOLOv7 model; Step 5: Input the multispectral image data of the mobile phone back cover into the deep learning model for training to obtain the improved YOLOv7 model; Step 6: Use the trained YOLOv7 model to detect defects in defect images; Step 7: Output the detection results including defect type and defect location.

2. The method for identifying and detecting defects of a mobile phone back cover based on multispectral images according to claim 1, characterized in that: The 16 different bands of light are divided into the following categories according to color: dark red, white, orange, pink, brown, red, yellow, blue, cool white, green, warm white, light blue, dark red, dark yellow, dark blue, and purple.

3. The method for identifying and detecting defects of a mobile phone back cover based on multispectral images according to claim 1, characterized in that: The Step 2 is specifically as follows: Step 2.1: The multispectral image denoising operation includes median filtering and bilateral filtering; Step 2.2: The illumination compensation step includes using an adaptive histogram equalization method to reduce the impact of uneven illumination on defective images.

4. The method for identifying and detecting defects of a mobile phone back cover based on multispectral images according to claim 1, characterized in that: The Step 3 is specifically as follows: the image alignment step aligns images under different spectra through a feature point matching algorithm.

5. The method for identifying and detecting defects of a mobile phone back cover based on multispectral images according to claim 1, characterized in that: The Step 4 is specifically as follows: first, a dilated space pooling pyramid module is added to the detection head of the YOLOv7 model, wherein the dilated space pooling pyramid module allows the network to expand the receptive field of the convolution operation during the feature extraction process without reducing the resolution, and then, an SE attention mechanism module is added to the main body of the module, and the importance of each channel of the feature map is obtained by automatic learning, and a weight value is assigned to each channel using the importance, so that the model focuses on learning channel features with large weight values.

6. The method for identifying and detecting defects of a mobile phone back cover based on multispectral images according to claim 1 is characterized in that The specific step of Step 7 is as follows: the detection result output visualization result diagram includes the output of defect type, location and detection confidence, as well as the output of precision, recall rate and mAP value.

Images