Dl-based notebook computer appearance defect detection method, device, medium and equipment

Abstract

The present application relates to the technical field of surface defect detection, and discloses a notebook computer appearance defect detection method, device, medium and equipment based on DL. The method comprises the following steps: obtaining a 360° surface image of a notebook computer by rotating the notebook computer using a six-axis mechanical arm, and preprocessing the surface image; constructing a defect detection deep learning model, wherein the defect detection deep learning model comprises a feature extraction module and a classification module, the feature extraction module enhances linear feature response through directional weighting, and the classification module dynamically adjusts the classification threshold of defects; training the defect detection deep learning model using the preprocessed surface image, inputting the 360° surface image of a notebook computer to be detected into the trained model to obtain a defect detection result. The device comprises an image acquisition module, an image processing module, a model construction module, a training module and a detection module, and the medium and equipment realize the method. The present application can realize omnidirectional detection of different types of defects and improve detection efficiency and accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of surface defect detection, in particular to a notebook computer appearance defect detection method and device based on DL, a medium and equipment. BACKGROUND

[0002] With the increasing precision of portable mobile terminals, the quality control requirements for notebook computers are becoming higher and higher. However, the shell of the notebook computer is usually made of aluminum alloy material, which is easy to wear; and in the process of processing and manufacturing the notebook computer, not only the surface will be damaged due to wear caused by production line transportation, but also micro defects such as paint scratches, liquid crystal screen bright lines, and dents will be produced on the surface of the notebook computer due to the complex structure, assembly process, or scratches, particles, bright spots, and other defects of the aluminum alloy panel. These defects will cause the quality of the notebook computer to be substandard, affecting the service life and user experience of the product. Therefore, quality detection needs to be performed on the appearance of the notebook computer to ensure that the product meets strict appearance quality standards.

[0003] In the prior art, there are mainly four methods for detecting defects on the appearance of a notebook computer, which are as follows:

[0004] 1. Manual detection method. The operator performs manual visual inspection. This method has high cost and poor consistency, especially when distinguishing small defects, the human eye is difficult to achieve the required precision and speed, and the sensitivity to low contrast defects and small defects is low. Moreover, affected by the operator's experience, the false detection rate fluctuates significantly, and the missed detection rate is high, especially when dealing with special materials such as matte coatings, the false detection rate increases significantly, and the efficiency of manual detection is low, which cannot meet the demand of high-paced production.

[0005] 2. Mechanical device contact detection method. The probe or sensor is used to contact the surface of the notebook computer for detection. Although this method can improve the efficiency to some extent compared with manual detection, it can only detect one side and is prone to secondary damage (such as scratches on the aluminum alloy shell), has poor flexibility, and is difficult to adapt to different sizes of products.

[0006] 3. Machine vision detection method. The image processing is used to detect possible defects of the product. This method can avoid secondary damage caused by contact, but the traditional machine vision detection method also has limitations. On the one hand, most of the existing methods rely on manual feature design, and the sensitivity to complex background and noise is low. The existing threshold segmentation and morphological feature extraction methods have limited adaptability to the scene, and cannot effectively identify different types of defects, for example, the matte aluminum alloy surface will misjudge the reflection noise as scratches due to the reflection characteristics. On the other hand, the existing classifier has insufficient detection accuracy for small defects and poor generalization ability, and is difficult to adapt to the diversified defects of different models of notebook computers.

[0007] 4. Defect detection methods based on deep learning. For example, an unsupervised notebook computer appearance defect detection method based on multi-scale normalized flow (patent publication number CN116205876A) can locate defects of different scales and types, and has good detection and defect localization effects. However, it cannot achieve all-round defect detection, and isotropic convolution has inherent limitations in metal surface defect detection, affecting the accuracy of notebook computer appearance defect detection. Summary of the Invention

[0008] Therefore, the technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a method, device, medium and equipment for detecting appearance defects of laptops based on DL, which can realize comprehensive detection of different types of defects and improve detection efficiency and accuracy.

[0009] To address the aforementioned technical problems, this invention provides a method for detecting appearance defects in laptops based on deep learning (DL), comprising:

[0010] A six-axis robotic arm is used to rotate the laptop to obtain a 360° surface image of the laptop, and the surface image is preprocessed.

[0011] A deep learning model for defect detection is constructed, which includes a feature extraction module and a classification module. The feature extraction module enhances the linear feature response through directional weighting, and the classification module dynamically adjusts the classification threshold of the defect.

[0012] A deep learning model for defect detection is trained using the preprocessed surface image. The 360° surface image of the laptop to be detected is then input into the trained deep learning model to obtain the defect detection result.

[0013] Furthermore, during the preprocessing of the surface image, an illumination-invariant transformation is performed on the surface image to obtain a surface image without reflection. The specific process is as follows:

[0014] Convert the surface image from RGB color space to HSV color space, and extract the V channel image in HSV color space, denoted as V_orig;

[0015] The V-channel image is logarithmically transformed to obtain the following result: V_log = ln(V_orig + b1), where V_log is the logarithmically transformed V-channel image and b1 is a preset coefficient.

[0016] Gaussian filtering is applied to V_log. The standard deviation in the X direction is set to σ during Gaussian filtering, and the size of the Gaussian kernel is kernel_size, where kernel_size = c1 * σ * c2 + c3, and c1, c2, and c3 are preset coefficients.

[0017] The edge enhancement coefficient is calculated as follows:

[0018] R_enhanced = R_log + lambda_val * DoG,

[0019] Where R_enhanced is the edge enhancement coefficient, R_log is the reflection component, lambda_val is the dynamic parameter, and DoG is the result of processing the V channel image using Gaussian filtering;

[0020] The metal reflection compensation index is calculated based on the edge enhancement coefficient, and the surface image with anti-reflection is calculated by inverse exponential transform:

[0021] V_new = e R_comp ,

[0022] Where R_comp is the metal reflection compensation index, and V_new is the surface image for eliminating reflections.

[0023] Furthermore, the method for calculating σ is as follows:

[0024] σ = a1+ a2 * (1 - sigma_V / a3),

[0025] Where a1, a2, and a3 are preset coefficients, and sigma_V is the texture complexity of the surface image;

[0026] The method for calculating lambda_val is as follows:

[0027] lambda_val = d1+ d2 * (sigma_V / d3),

[0028] Among them, d1, d2, and d3 are preset coefficients.

[0029] Furthermore, the method for calculating the reflection component is as follows:

[0030] R_log = V_log - V_low,

[0031] Where V_low is the result of Gaussian filtering V_log;

[0032] The calculation method for R_comp is as follows:

[0033] R_comp = R_enhanced * (f1 + f2 * M),

[0034] Where f1 and f2 are preset coefficients, and M is the RGB brightness of the surface image.

[0035] Furthermore, the feature extraction module enhances the linear feature response through direction weighting, specifically:

[0036] Separable convolution is used to extract scratch features in a specific direction. The extracted scratch features in a specific direction are as follows:

[0037] ,

[0038] in, F out Conv represents the scratch features extracted by the feature extraction module in a specific direction. 1×1 ( ) represents a 1×1 convolution operation. F in The input image is represented by ||, and the concatenation is performed by channel. Attn θ ( )express θ Convolution kernel calculation in the direction, θ An angle set for the defect type.

[0039] Furthermore, the input image is subjected to the aforementioned... θ The calculation of the convolution kernel in the direction is as follows:

[0040] ,

[0041] in, Attn θ ( F in ) indicates that the input image is processed as described. θ Convolution kernel calculation in the direction, Sigmoid ( )express Sigmoid Activation function operation, Conv 3×3 _θ ( ) indicates a 3×3 convolution operation with a convolution kernel of 3×3. θ .

[0042] Furthermore, the method for dynamically adjusting the classification threshold is as follows:

[0043] T = T_base + α * (1 - IoU_mean) + β * Entropy_map - γ * Edge_density,

[0044] Where T represents the adjusted classification threshold, T_base represents the base classification threshold, α, β, and γ are weighting coefficients, IoU_mean represents the average defect overlap of the samples detected in this batch, Entropy_map represents the texture information entropy of the current surface image, and Edge_density represents the percentage of edge pixels in the current surface image.

[0045] Furthermore, the average defect overlap of the tested samples in this batch is calculated as follows: calculate the defect bounding box of each sample in this batch, calculate the maximum cross-union ratio between every two defect bounding boxes, and take the average of all maximum cross-union ratios as the average defect overlap of the tested samples in this batch.

[0046] The present invention also provides a DL-based device for detecting appearance defects in laptops, comprising:

[0047] The image acquisition module includes a camera, a six-axis robotic arm, and an opening panel. The opening panel is equipped with a light source with adjustable brightness. The camera is located inside the opening of the opening panel. The six-axis robotic arm grips the laptop and rotates it 360°, while the camera captures 360° surface images of the laptop.

[0048] The image processing module is used to preprocess the surface image;

[0049] The model building module is used to build a deep learning model for defect detection. The deep learning model for defect detection includes a feature extraction module and a classification module. The feature extraction module enhances the linear feature response through directional weighting, and the classification module dynamically adjusts the classification threshold of the defect.

[0050] The training module is used to train a deep learning model for defect detection using preprocessed surface images;

[0051] The detection module is used to input a 360° surface image of the laptop to be inspected into the trained deep learning model for defect detection to obtain the defect detection results.

[0052] Furthermore, it also includes:

[0053] The feeding and unloading transfer methods include a feeding transfer method where a laptop computer to be inspected is placed, and an unloading transfer method with multiple areas where laptop computers with different defect types are placed respectively.

[0054] A pneumatic mechanism is connected to the feeding and transfer device and drives the feeding and transfer device to move the laptop to be inspected to the inspection position. It is also connected to the six-axis robotic arm and drives the six-axis robotic arm to grip the laptop and rotate it 360°. After obtaining the defect detection result, it drives the six-axis robotic arm to grip the laptop with different defect types and place it on the corresponding area of ​​the unloading and transfer device. Finally, it is connected to the unloading and transfer device and drives the unloading and transfer device to move the inspected laptop away from the inspection position.

[0055] The operation interface is connected to the camera and pneumatic mechanism. The operator controls the pneumatic mechanism to drive the six-axis robotic arm, and performs loading and unloading movements through the operation interface. The operator also controls the camera to capture images through the operation interface.

[0056] The display interface is used to show the defect detection results;

[0057] An alarm mechanism that issues an alarm when a defect is detected in the appearance of a laptop.

[0058] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the described DL-based method for detecting appearance defects in laptops.

[0059] The present invention also provides a DL-based laptop computer appearance defect detection device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the DL-based laptop computer appearance defect detection method.

[0060] Compared with the prior art, the above-described technical solution of the present invention has the following advantages:

[0061] This invention achieves comprehensive detection of laptop appearance defects by using a six-axis robotic arm to rotate the laptop and obtain a 360° surface image of the laptop. By using a deep learning model that enhances linear feature response and dynamically adjusts defect classification thresholds, it can detect a variety of defects, improving detection accuracy and scalability, and achieving high detection efficiency. Attached Figure Description

[0062] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein:

[0063] Figure 1 This is a flowchart of a method in a preferred embodiment of the present invention.

[0064] Figure 2 This is a flowchart of performing illumination-invariant transformation on a surface image in a preferred embodiment of the present invention.

[0065] Figure 3 This is a structural diagram of the defect detection deep learning model constructed in a preferred embodiment of the present invention.

[0066] Figure 4 This is an overall structural diagram of the device in a preferred embodiment of the present invention.

[0067] Figure 5 This is a front view of the device in a preferred embodiment of the present invention.

[0068] Figure 6 This is a side view of the device in a preferred embodiment of the present invention.

[0069] Figure 7 This is a schematic diagram of three randomly selected measurement locations on the exterior of a laptop computer during a defect detection experiment in a preferred embodiment of the present invention.

[0070] Figure 8 This is a process diagram of defect detection at a randomly selected measurement location in a preferred embodiment of the present invention.

[0071] Figure 9 This is a process diagram of defect detection at a randomly selected measurement location two in a preferred embodiment of the present invention.

[0072] Figure 10 This is a process diagram of defect detection at a randomly selected measurement location three in a preferred embodiment of the present invention.

[0073] Figure 11 This is a diagram showing the results of defect detection at three randomly selected measurement locations in a preferred embodiment of the present invention.

[0074] The following are the markings on the attached diagrams in the instruction manual: 1. Camera; 2. Six-axis robotic arm; 3. Loading and transferring; 4. Unloading and transferring; 5. Pneumatic mechanism; 6. Operating interface; 7. Display interface; 8. Alarm mechanism; 9. Housing; 10. Opening panel. Detailed Implementation

[0075] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0076] Reference Figure 1 As shown, this invention discloses a method for detecting appearance defects in laptops based on deep learning (DL), comprising the following steps:

[0077] S1: Use a six-axis robotic arm to rotate the laptop to obtain 360° surface images of the laptop. In this embodiment, 5,000 images of the aluminum alloy panel of the laptop's exterior were obtained.

[0078] S2: Preprocessed surface image.

[0079] S2-1: For matte defects such as matte scratches, use ContrastLimited Adaptive histgram equalization (CLAHE) to enhance low contrast defects.

[0080] S2-2: Perform the following steps on the surface image: Figure 2 The illumination invariance transformation shown eliminates the influence of metal surface reflection on the detection results through anti-interference optimization. The illumination invariance transformation includes:

[0081] S2-2-1: Convert the surface image from the RGB color space to the HSV color space.

[0082] S2-2-2: Extract the V channel image in the HSV color space, denoted as V_orig.

[0083] S2-2-3: The standard deviation of V_orig is used as the texture complexity of the surface image, denoted as sigma_V.

[0084] S2-2-4: Set the dynamic parameter σ, and the calculation method for σ is as follows:

[0085] σ = a1+ a2 * (1 - sigma_V / a3),

[0086] Wherein, a1, a2, and a3 are preset coefficients, and in this embodiment, a1=0.8, a2=0.4, and a3=50.

[0087] S2-2-5: Perform a logarithmic transformation on the V channel image to obtain the following logarithmic transformation result:

[0088] V_log = ln(V_orig + b1),

[0089] Where V_log is the logarithmically transformed V_orig, and b1 is a preset coefficient, in this embodiment b1=1e-3.

[0090] S2-2-6: Apply Gaussian filtering to V_log to eliminate Gaussian noise, smooth the image, and preserve edge features. The result of Gaussian filtering V_log is denoted as V_low.

[0091] The calculation method for V_low is as follows:

[0092] V_low = cv2.GaussianBlur(V_log, (kernel_size, kernel_size), σ),

[0093] Where cv2.GaussianBlur() represents a Gaussian filtering operation, (kernel_size, kernel_size) is the Gaussian kernel, kernel_size is the size of the Gaussian kernel, and the standard deviation in the X direction is set to σ when Gaussian filtering is performed.

[0094] The method for calculating the Gaussian kernel size is as follows:

[0095] kernel_size = c1 * σ * c2 + c3,

[0096] Wherein, c1, c2, and c3 are preset coefficients, and in this embodiment, c1=3, a3=2, and c3=1.

[0097] S2-2-7: Based on V_log before and after Gaussian filtering, the reflection component is calculated as follows:

[0098] R_log = V_log - V_low,

[0099] Where R_log is the reflection component.

[0100] S2-2-8: The edge enhancement coefficient is calculated as follows:

[0101] R_enhanced = R_log + lambda_val * DoG,

[0102] Where R_enhanced is the edge enhancement coefficient, lambda_val is the dynamic parameter, and DoG is the result of processing the V channel image using Gaussian filtering, i.e., edge enhancement;

[0103] The method for calculating lambda_val is as follows:

[0104] lambda_val = d1+ d2 * (sigma_V / d3),

[0105] Wherein, d1, d2, and d3 are preset coefficients, and in this embodiment, d1 = 0.15, d2 = 0.05, and d3 = 40;

[0106] The method for calculating DoG is as follows:

[0107] DoG = cv2.GaussianBlur(V_orig, (size1,size1), d4) - cv2.GaussianBlur(V_orig, (size2,size2), d5),

[0108] Where (size1,size1) and (size2,size2) are the Gaussian kernels for the two Gaussian filtering operations, and size1 and size2 are the sizes of the Gaussian kernels for the two Gaussian filtering operations. In this embodiment, size1=0 and size2=0. d4 and d5 are the standard deviations in the X direction during the two Gaussian filtering operations, used to control the intensity of blur. In this embodiment, d4=0.5 and d5=2.

[0109] S2-2-9: The metal reflection compensation index is calculated as follows:

[0110] R_comp = R_enhanced * (f1 + f2 * M),

[0111] Where R_comp is the metal reflection compensation index, f1 and f2 are preset coefficients, in this embodiment f1=1 and f2=0.2, and M is the RGB brightness of the surface image;

[0112] The method for calculating M is as follows:

[0113] M = f3*R + f4*G - f5*B,

[0114] Wherein, R, G, and B represent the brightness of the surface image in the R, G, and B channels, respectively, and f3, f4, and f5 are preset coefficients. In this embodiment, f3=0.6, f4=0.3, and f5=0.9.

[0115] S2-2-10: The surface image after eliminating reflections, calculated using the inverse exponential transform, is:

[0116] V_new = e R_comp ,

[0117] Where V_new is the surface image with anti-reflection properties.

[0118] S3: Deep Learning (DL) is a type of machine learning based on deep neural network models. This invention constructs a deep learning model for defect detection based on deep learning. The defect detection deep learning model includes a feature extraction module and a classification module. The feature extraction module enhances the linear feature response through directional weighting, and the classification module dynamically adjusts the defect classification threshold.

[0119] The defect detection deep learning model used in this embodiment is as follows: Figure 3 The YOLOv11 network model shown is equipped with depthwise separable convolution (DS-Conv).

[0120] The defect detection deep learning model constructed in this invention uses channel pruning technology to remove redundant convolutional layers in the YOLOv11 backbone network, which can compress the number of model parameters from 35M to 12M and improve the inference speed by 40%.

[0121] Laptop casing scratches are mostly linearly distributed (70% of the scratches form an angle of 15° to 75° with the edge). Therefore, this invention uses DS-Conv for direction weighting to enhance the linear feature response. Specifically, DS-Conv is used in the shallow feature extraction stage to extract scratch features in specific directions. DS-Conv can improve the sensitivity to direction and enhance the ability to extract scratch features in specific directions (such as 0°, 45°).

[0122] In this embodiment, the method for extracting scratch features in a specific direction is as follows:

[0123] ,

[0124] in, F out Conv represents the scratch features extracted by the feature extraction module in a specific direction. 1×1 ( ) represents a 1×1 convolution operation. F in The input image is represented by ||, and the concatenation is performed by channel. Attn θ ( )express θ Convolution kernel calculation in the direction, θ An angle set for specific defect types. express θ+ Convolution kernel calculation in the 90° direction.

[0125] Perform on the input image θ The calculation of the convolution kernel in the direction is as follows:

[0126] ,

[0127] in, Attn θ ( F in ) indicates that the input image is processed as described. θ Convolution kernel calculation in the direction, Sigmoid ( )express Sigmoid Activation function operation, Conv 3×3 _θ ( ) indicates a 3×3 convolution operation with a convolution kernel of 3×3. θ .

[0128] When the surface image defect type is a vertical or horizontal straight scratch or a border defect, θ Set to 0°, 0° convolution kernel is 90° convolution kernel is It is used to highlight horizontal features and enhance linear edge response.

[0129] When the defect type of the surface image is a beveled scratch or an arc-shaped indentation, θ is set to 45°, and the 45° convolution kernel is... 135° convolution kernel is It is used to capture oblique scratches and improve the accuracy of curve defect identification.

[0130] In existing deep learning models, the classification prediction threshold is fixed. However, a fixed threshold cannot adapt to the differences in manufacturing processes of different batches of laptop casings (such as variations in sandblasting particle size leading to fluctuations in texture noise). Therefore, this invention dynamically adjusts the classification threshold, ranging from 0.7 to 0.95. The method for dynamically adjusting the classification threshold is as follows:

[0131] T = T_base + α * (1 - IoU_mean) + β * Entropy_map - γ * Edge_density,

[0132] Where T represents the adjusted classification threshold, T_base represents the base classification threshold, and the value of T_base is adjusted according to the actual situation. In this embodiment, T_base=0.7; α, β, and γ are weighting coefficients, and the values ​​of α, β, and γ are adjusted according to the actual situation. In this embodiment, α=0.15, β=0.1, and γ=0.05; IoU_mean represents the average defect overlap of the samples detected in this batch, which is used to reflect the degree of defect aggregation; Entropy_map represents the texture information entropy of the current surface image, which is used to quantify the surface roughness. Entropy_map fluctuates significantly when the sandblasting process changes; Edge_density represents the proportion of edge pixels in the current surface image. A large Edge_density value indicates a complex shell structure, such as a heat dissipation hole area, and the sensitivity needs to be reduced.

[0133] The average defect overlap of the tested samples in this batch is calculated as follows: calculate the defect bounding box of each sample in this batch, calculate the maximum crossover ratio between every two defect bounding boxes, and take the average of all maximum crossover ratios as the average defect overlap of the tested samples in this batch.

[0134] S4: Train a deep learning model for defect detection using the preprocessed surface image. In this embodiment, the initial learning rate is set to 0.01, cosine annealing is used for decay, and the model is iterated 300 times.

[0135] S5: Input the 360° surface image of the laptop to be detected into the trained deep learning model for defect detection to obtain the defect detection result.

[0136] likeFigure 4 , Figure 5 , Figure 6 As shown, the present invention also discloses a laptop computer appearance defect detection device based on deep learning, including an image acquisition module, an image processing module, a model building module, a training module and a detection module.

[0137] The image acquisition module includes a camera 1, a six-axis robotic arm 2, and an open panel 10. The open panel 10 has a light source with adjustable brightness. The camera 1 is located within the open panel 10 to provide multi-angle supplementary lighting and suppress reflections from the metal surface. The six-axis robotic arm 2 grips the laptop and rotates it 360°, while the camera 1 captures 360° images of the laptop's surface. In this embodiment, the camera 1 used is a 25-megapixel high-resolution industrial camera, model MV-CH250-90GC. The six-axis robotic arm 2 has a repeatability accuracy of ±0.02mm and works in conjunction with the camera 1 to achieve a 360° rotational scan of the laptop's exterior. The acquired surface images include those used during model training and images of the laptop being inspected.

[0138] The image processing module is used to preprocess surface images.

[0139] The model building module is used to build a deep learning model for defect detection. The deep learning model for defect detection includes a feature extraction module and a classification module. The feature extraction module enhances the linear feature response through directional weighting, and the classification module dynamically adjusts the classification threshold of defects.

[0140] The training module is used to train a deep learning model for defect detection using preprocessed surface images.

[0141] The detection module is used to input a 360° surface image of the laptop to be inspected into the trained deep learning model for defect detection to obtain the defect detection results.

[0142] In this embodiment, the image processing module, model building module, training module, and detection module are mounted on a remote Intel i9-14900 + NVIDIA RTX 4090, and the image data is transmitted wirelessly.

[0143] In this embodiment, the device further includes a feeding transfer unit 3, an unloading transfer unit 4, and a pneumatic mechanism 5. The feeding transfer unit 3 holds the laptop computer to be inspected, and the unloading transfer unit 4 has multiple areas, each holding a laptop computer with a different defect type. The pneumatic mechanism 5 is connected to the feeding transfer unit 3 and drives it to move the laptop computer to the inspection position. The pneumatic mechanism 5 is also connected to the six-axis robotic arm 2 and drives it to grip the laptop computer and rotate it 360°. After obtaining the defect detection result, the pneumatic mechanism 5 drives the six-axis robotic arm 2 to grip the laptop computer with different defect types and place it into the corresponding area on the unloading transfer unit 4. Finally, the pneumatic mechanism 5 is connected to the unloading transfer unit 4 and drives it to move the inspected laptop computer away from the inspection position.

[0144] In this embodiment, the device also includes an operation interface 6, a display interface 7, and an alarm mechanism 8. The operation interface 6 is connected to the camera 1 and the pneumatic mechanism 5. The operator controls the pneumatic mechanism 5 via the operation interface 6 to drive the six-axis robotic arm 2, the loading and unloading transfer mechanism 3, and the unloading and unloading transfer mechanism 4. The operator also controls the camera 1 to acquire images via the operation interface 6. The display interface 7 is used to display the defect detection results. An alarm message is issued when a defect is detected in the appearance of the laptop computer.

[0145] In this embodiment, the device also includes a housing 9, within which the camera 1 and the six-axis robotic arm 2 are located. The loading and unloading transfer 3 and multiple unloading transfer 4 can move inside and outside the housing 9. For ease of observation, the housing 9 in this embodiment can be a transparent housing.

[0146] The overall process for detecting defects in the appearance of laptops is as follows: Step 1, such as Figure 5As shown, in the first step, the loading and transfer mechanism 3 moves the laptop to be inspected outside the outer casing 9, and the manual operator places the laptop to be inspected onto the loading and transfer mechanism 3. In the second step, the loading and transfer mechanism 3 moves the laptop to the inspection position inside the outer casing 9, and the six-axis robotic arm 2 grips the laptop. In the third step, the six-axis robotic arm 2 rotates the laptop 360°, triggering the camera 1 to capture multi-angle images (34 fixed poses in this embodiment) of the laptop's exterior, obtaining a 360° surface image. In the fourth step, the surface image is transmitted to the image processing module for preprocessing, and then transmitted to the defect detection deep learning model. In the fifth step, the defect detection deep learning model is trained. The surface image of the laptop to be inspected is used for inference detection to obtain defect detection results, which are divided into scratch defects, dent defects, color difference defects, and no defects. The sixth step is to display the defect detection results through the display interface 7. When a defect is found, the alarm mechanism 8 triggers an audible and visual alarm. The seventh step is for the six-axis robotic arm 2 to classify the laptop according to the defect detection results and pick it up and place it in different areas on the unloading transfer 4. The eighth step is for the unloading transfer 4 to remove the inspected laptop from the shell 9 and manually remove it for unloading. The ninth step is to upload the inspection results and defect locations of the product to the Manufacturing Execution System (MES) for subsequent review.

[0147] The present invention also discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a DL-based method for detecting appearance defects in laptops.

[0148] The present invention also discloses an apparatus including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a DL-based method for detecting appearance defects in laptops.

[0149] Compared with the prior art, the advantages of the present invention are as follows:

[0150] 1. This invention uses a six-axis robotic arm to rotate a laptop and obtain a 360° surface image of the laptop, which enables all-round detection of appearance defects of the laptop.

[0151] 2. By using deep learning models to detect defects, the ability to identify complex defects (such as arc-shaped indentations and bright spots) can be improved, and accurate detection of various defects can be achieved. At the same time, the model is highly scalable, able to automatically adjust detection parameters according to different manufacturers and models of laptops, adapting to diverse product needs, thereby reducing equipment replacement and debugging time and lowering costs.

[0152] 3. To address the anisotropic defect distribution of aluminum alloy shells (such as the directional preference of scratches on sandblasted surfaces), the model is improved by using directionally sensitive convolution that enhances the linear feature response, thereby increasing the detection rate of oblique micro-scratches and overcoming the inherent limitations of traditional isotropic convolution in metal surface defect detection.

[0153] 4. By dynamically adjusting the defect classification threshold of the model, the detection accuracy can be further improved.

[0154] 5. By using channel pruning technology to remove redundant convolutional layers in the YOLOv11 backbone network, the computational burden can be reduced and the detection efficiency can be improved while maintaining high-precision detection, meeting the high-speed detection requirements of production lines and improving the deployment efficiency in industrial scenarios.

[0155] 6. By performing preprocessing operations on the surface image to enhance low contrast and light invariance transformation, the influence of metal surface reflection on the detection results can be eliminated, so as to adapt to the interference of light changes, noise and other adverse external environments, and improve stability and robustness.

[0156] 7. The system of the present invention has an intuitive user interface, is easy to operate, reduces the requirements for the professional skills of operators, and can reduce labor costs.

[0157] 8. The system of this invention is highly automated, providing consistent and reliable test results. It avoids inconsistencies caused by operator fatigue or subjective judgment, common in manual inspection, ensuring that every laptop leaving the factory meets stringent quality standards. Simultaneously, automated inspection significantly reduces reliance on manual inspection, thereby lowering labor costs and optimizing enterprise resource allocation.

[0158] This invention has been applied in actual laptop production lines, achieving an average detection accuracy of 88.75%, with a detection accuracy of 98.9% for minute scratches as small as 2mm. The detection time for a single item is 53 seconds, significantly improving efficiency compared to the 180 seconds required for manual inspection. The false detection rate is 1.5%, a significant reduction compared to the 9.8% false detection rate of the traditional SVM solution. The false detection rate is 0.8%, also a significant reduction compared to the lowest false detection rate of 4.5% in existing methods.

[0159] In this embodiment, a defect detection experiment is conducted using three randomly selected measurement locations on the exterior of a laptop computer as an example. The original images of the three measurement locations are shown below. Figure 7 As shown, Figure 7 (a) shows the measurement location. Figure 7 (b) indicates the measurement location. Figure 7 (c) represents measurement position three.

[0160] The process of defect detection at measurement location one is as follows: Figure 8 As shown. Figure 8 The middle (a) image shows the result after calculating the texture complexity at the measurement location (the calculated texture complexity is 30.79). Figure 8 Image (b) shows the effect after logarithmic transformation. Figure 8 (c) shows the effect after Gaussian filtering (the standard deviation in the X direction is set to 0.95 when Gaussian filtering is used). Figure 8 Image (d) shows the result after calculating the reflection component. Figure 8 Image (e) shows the effect of edge detection using Gaussian filtering when calculating the edge enhancement coefficient. Figure 8 Image (f) shows the effect after edge enhancement. Figure 8 Image (g) shows the result of the reflection feature map calculated during reflection compensation. Figure 8 The middle (h) image shows the effect after reflection compensation. Figure 8 In the diagram (i), the result of the inverse exponential transformation is shown.

[0161] The process of defect detection at measurement location two is as follows: Figure 9 As shown. Figure 9 The middle (a) image shows the result after calculating the texture complexity at measurement position two (the calculated texture complexity is 16.49). Figure 9 Image (b) shows the effect after logarithmic transformation. Figure 9 (c) shows the effect after Gaussian filtering (the standard deviation in the X direction is set to 1.07 when Gaussian filtering is used). Figure 9 Image (d) shows the result after calculating the reflection component. Figure 9 Image (e) shows the effect of edge detection using Gaussian filtering when calculating the edge enhancement coefficient. Figure 9 Image (f) shows the effect after edge enhancement. Figure 9 Image (g) shows the result of the reflection feature map calculated during reflection compensation. Figure 9 The middle (h) image shows the effect after reflection compensation. Figure 9 In the diagram (i), the result of the inverse exponential transformation is shown.

[0162] The process of defect detection at measurement location three is as follows: Figure 10 As shown. Figure 10 The middle (a) image shows the result after calculating the texture complexity at measurement position three (the calculated texture complexity is 38.72). Figure 10 Image (b) shows the effect after logarithmic transformation. Figure 10 (c) shows the effect after Gaussian filtering (the standard deviation in the X direction is set to 0.89 when Gaussian filtering is used). Figure 10 Image (d) shows the result after calculating the reflection component. Figure 10Image (e) shows the effect of edge detection using Gaussian filtering when calculating the edge enhancement coefficient. Figure 10 Image (f) shows the effect after edge enhancement. Figure 10 Image (g) shows the result of the reflection feature map calculated during reflection compensation. Figure 10 The middle (h) image shows the effect after reflection compensation. Figure 11 In the diagram (i), the result of the inverse exponential transformation is shown.

[0163] Since the original images of measurement positions 1, 2, and 3 collected in the experiment are grayscale images, the texture complexity can be calculated directly. If the original images are RGB images, they need to be converted to HSV space and the V channel image needs to be extracted before the texture complexity can be calculated.

[0164] The results of defect detection at measurement locations 1, 2, and 3 are as follows: Figure 11 As shown. Figure 11 (a) is the original image at measurement location one. Figure 11 Image (b) shows the defect detection result obtained by inputting the image after inverse exponential transformation corresponding to measurement position one into the deep learning model for defect detection. Specifically... Figure 11 The boxed annotations in (b) indicate the detected defects; Figure 11 Image (c) is the original image at measurement location two. Figure 11 In the middle (d), the image after the inverse exponential transform of the image corresponding to measurement position two is input into the defect detection deep learning model, and the defect detection result is obtained. Figure 11 The boxed annotations in (d) indicate the detected defects; Figure 11 Image (e) is the original image of measurement location three. Figure 11 In diagram (f), the defect detection result is obtained by inputting the image after inverse exponential transformation of the three measurement positions into the deep learning model for defect detection. Specifically... Figure 11 The boxed annotations in (f) indicate the detected defects. From Figure 1 As can be seen, the present invention can enhance the defect features in the original image, thereby achieving accurate detection of defects in the appearance of laptops.

[0165] To further illustrate the advantages of the defect detection deep learning model in this invention, the original YOLOv11 model and the defect detection deep learning model in this invention were used to detect four types of defects on the appearance of a laptop computer: 0° straight scratches, 45° diagonal scratches, heat dissipation hole notches, and arc-shaped indentations. The average accuracy results for each type of defect detection are shown in Table 1.

[0166] Table 1. Comparison of Results of Different Defect Detection Methods

[0167]

[0168] As can be seen from Table 1, the deep learning model for defect detection in this invention can indeed improve the detection accuracy of various types of defects.

[0169] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0170] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0171] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0172] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes ​ The steps of the function specified in one or more boxes.

[0173] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for detecting appearance defects in laptops based on deep learning, characterized in that, include: A six-axis robotic arm is used to rotate the laptop to obtain a 360° surface image of the laptop, and the surface image is preprocessed. A deep learning model for defect detection is constructed, comprising a feature extraction module and a classification module. The feature extraction module enhances the linear feature response through directional weighting, and the classification module dynamically adjusts the defect classification threshold. The dynamic adjustment method for the classification threshold is as follows: T = T_base + α*(1-IoU_mean) + β*Entropy_map - γ*Edge_density, where T represents the adjusted classification threshold, T_base represents the base classification threshold, α, β, and γ are weighting coefficients, IoU_mean represents the average defect overlap of the samples detected in this batch, Entropy_map represents the texture information entropy of the current surface image, and Edge_density represents the proportion of edge pixels in the current surface image; A deep learning model for defect detection is trained using the preprocessed surface image. The 360° surface image of the laptop to be detected is then input into the trained deep learning model to obtain the defect detection result.

2. The method for detecting appearance defects of laptops based on deep learning according to claim 1, characterized in that: During the preprocessing of the surface image, an illumination-invariant transformation is performed on the surface image to obtain a surface image without reflection. The specific process is as follows: Convert the surface image from RGB color space to HSV color space, and extract the V channel image in HSV color space, denoted as V_orig; The V-channel image is logarithmically transformed to obtain the following result: V_log=ln(V_orig+b1), where V_log is the V-channel image after logarithmic transformation and b1 is a preset coefficient. Gaussian filtering is applied to V_log. The standard deviation in the X direction is set to σ during Gaussian filtering, and the size of the Gaussian kernel is kernel_size, where kernel_size = c1*σ*c2+c3, and c1, c2, and c3 are preset coefficients. The edge enhancement coefficient is calculated as follows: R_enhanced=R_log+lambda_val*DoG, Where R_enhanced is the edge enhancement coefficient, R_log is the reflection component, lambda_val is the dynamic parameter, and DoG is the result of processing the V channel image using Gaussian filtering; The method for calculating the reflection component is as follows: R_log = V_log - V_low Where V_low is the result of Gaussian filtering V_log; The method for calculating lambda_val is as follows: lambda_val=d1+d2*(sigma_V / d3), Wherein, d1, d2, and d3 are preset coefficients; The metal reflection compensation index is calculated based on the edge enhancement coefficient, and the surface image with anti-reflection is calculated by inverse exponential transform: V_new=e R_comp , Where R_comp is the metal reflection compensation index, and V_new is the surface image with anti-reflection function; The calculation method for R_comp is as follows: R_comp=R_enhanced*(f1+f2*M), Where f1 and f2 are preset coefficients, and M is the RGB brightness of the surface image.

3. The method for detecting appearance defects of laptops based on deep learning according to claim 2, characterized in that: The method for calculating σ is as follows: σ = a1 + a2 * (1 - sigma_V / a3), Where a1, a2, and a3 are preset coefficients, and sigma_V is the texture complexity of the surface image.

4. The method for detecting appearance defects of laptops based on deep learning according to claim 1, characterized in that: The feature extraction module enhances the linear feature response through directional weighting, specifically: Separable convolution is used to extract scratch features in a specific direction. The extracted scratch features in a specific direction are as follows: F out =Conv 1×1 (Attn θ (F in )||Attn θ+90° (F in )), Among them, F out Conv represents the scratch features extracted by the feature extraction module in a specific direction. 1×1 () represents a 1×1 convolution operation, F in Indicates the input image; || indicates concatenation by channel; Attn θ () indicates the calculation of the convolution kernel in the θ direction, where θ is the angle set for the defect type.

5. The method for detecting appearance defects of laptops based on deep learning according to claim 4, characterized in that: The convolution kernel in the θ direction is calculated on the input image, specifically as follows: Among them, Attn θ (F in ) indicates that the convolution kernel in the θ direction is calculated on the input image, Sigmoid() indicates the Sigmoid activation function operation, Conv 3×3_θ () indicates a 3×3 convolution operation with kernel θ.

6. The method for detecting appearance defects of laptops based on deep learning according to claim 1, characterized in that: The average defect overlap of the tested samples in this batch is calculated as follows: calculate the defect bounding box of each sample in this batch, calculate the maximum crossover ratio between every two defect bounding boxes, and take the average of all maximum crossover ratios as the average defect overlap of the tested samples in this batch.

7. A device for detecting appearance defects in laptops based on deep learning (DL), characterized in that, include: The image acquisition module includes a camera, a six-axis robotic arm, and an opening panel. The opening panel is equipped with a light source with adjustable brightness. The camera is located inside the opening of the opening panel. The six-axis robotic arm grips the laptop and rotates it 360°, while the camera captures 360° surface images of the laptop. The image processing module is used to preprocess the surface image; The model building module is used to construct a deep learning model for defect detection. This model includes a feature extraction module and a classification module. The feature extraction module enhances linear feature responses through directional weighting. The classification module dynamically adjusts the defect classification threshold using the following method: T = T_base + α*(1-IoU_mean) + β*Entropy_map - γ*Edge_density, where T represents the adjusted classification threshold, T_base represents the base classification threshold, α, β, and γ are weighting coefficients, IoU_mean represents the average defect overlap of the samples detected in this batch, Entropy_map represents the texture information entropy of the current surface image, and Edge_density represents the proportion of edge pixels in the current surface image; The training module is used to train a deep learning model for defect detection using preprocessed surface images; The detection module is used to input a 360° surface image of the laptop to be inspected into the trained deep learning model for defect detection to obtain the defect detection results.

8. The DL-based laptop computer appearance defect detection device according to claim 7, characterized in that, Also includes: The feeding and unloading transfer methods include a feeding transfer method where a laptop computer to be inspected is placed, and an unloading transfer method with multiple areas, each area being used to place a laptop computer with a different defect type. A pneumatic mechanism is connected to the feeding and transfer device and drives the feeding and transfer device to move the laptop to be inspected to the inspection position. It is also connected to the six-axis robotic arm and drives the six-axis robotic arm to grip the laptop and rotate it 360°. After obtaining the defect detection result, it drives the six-axis robotic arm to grip the laptop with different defect types and place it on the corresponding area of ​​the unloading and transfer device. Finally, it is connected to the unloading and transfer device and drives the unloading and transfer device to move the inspected laptop away from the inspection position. The operation interface is connected to the camera and pneumatic mechanism. The operator controls the pneumatic mechanism to drive the six-axis robotic arm, and performs loading and unloading movements through the operation interface. The operator also controls the camera to capture images through the operation interface. The display interface is used to show the defect detection results; An alarm mechanism that issues an alarm when a defect is detected in the appearance of a laptop.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the DL-based method for detecting appearance defects in laptops as described in any one of claims 1-6.

10. A laptop computer appearance defect detection device based on deep learning, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the DL-based method for detecting appearance defects in laptops as described in any one of claims 1-6.

Images