A multi-degree-of-freedom notebook computer shell surface defect detection method

Abstract

The application provides a multi-degree-of-freedom notebook shell surface defect detection method, comprising: establishing a product coordinate system, a camera coordinate system and an image coordinate system, calibrating an industrial camera and a multi-degree-of-freedom attitude adjusting mechanism, obtaining a surface region type, a collection angle sequence and a pose parameter, and collecting a multi-angle image sequence; preprocessing the multi-angle image sequence, extracting a candidate region of each collection angle, establishing a cross-view mapping relationship based on the pose parameter and the surface region type, forming a defect track and a visibility vector; constructing an angle response feature vector based on the defect track and generating a view angle weight; inputting the angle response feature vector and the view angle weight into a classification model to obtain a category probability distribution and an initial classification result; performing multi-view consistency checking based on the visibility vector, adjacent view angle feature change continuity and the category probability distribution to generate a final discrimination result and reflect the discrimination result to the product coordinate system to output a defect result set.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and surface inspection technology, and in particular to a method for detecting surface defects in multi-degree-of-freedom notebook casings. Background Technology

[0002] The surface quality of structural components in 3C consumer electronics products, such as multi-degree-of-freedom laptop casings, directly impacts the final appearance, assembly precision, and user experience. These components typically include flat areas, curved edges, and corners, and their surfaces are prone to surface defects during manufacturing, such as scratches, dents, dents, dust adhesion, water stains, and abnormal localized reflections. Current inspection methods primarily include manual visual inspection and machine vision inspection. Manual visual inspection is significantly affected by the inspector's experience and fatigue, resulting in poor inspection stability. While machine vision inspection is more efficient, it largely relies on single images viewed from a fixed perspective.

[0003] For multi-degree-of-freedom laptop casings, single-angle image detection has significant limitations: Firstly, the reflection states of curved edges and corners vary considerably under different viewing directions. The same real defect may be clearly visible at one acquisition angle, but difficult to discern at another due to reflections, shadows, or contour compression, easily leading to missed detections. Secondly, real defects such as scratches and dents often exhibit similar grayscale changes or texture features in a single image as dust, water stains, oil stains, and occasional reflections. Existing methods rely solely on local appearance information from a single image, making it difficult to effectively distinguish between real and false defects, easily resulting in false detections.

[0004] Therefore, this invention proposes a method for detecting surface defects in a multi-degree-of-freedom notebook computer casing. The information disclosed in the background section is only for enhancing understanding of the background of this disclosure and may therefore contain prior art information that is not common knowledge to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by providing a method for detecting surface defects in a multi-degree-of-freedom notebook casing, thereby solving the technical problems mentioned in the background section.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for detecting surface defects in a multi-degree-of-freedom laptop casing (hereinafter referred to as the workpiece) includes the following steps:

[0008] S1. Establish the product coordinate system, camera coordinate system and image coordinate system, calibrate the industrial camera and multi-degree-of-freedom attitude adjustment mechanism, obtain the surface area type, acquisition angle sequence and pose parameters, and acquire multi-angle image sequences according to the acquisition angle sequence.

[0009] S2. Preprocess the multi-angle image sequence, extract candidate regions from each acquisition angle, establish cross-view mapping relationship based on pose parameters and surface region type, associate the candidate regions, and obtain defect trajectory and visibility vector;

[0010] S3. Extract the mean gray level, mean background gray level, contrast, gradient magnitude, morphological features and differential features based on the defect trajectory, construct the angle response feature vector, and generate the viewpoint weight based on the local surface normal, image quality evaluation value and historical reliability.

[0011] S4. Input the angle response feature vector and viewpoint weights into the classification model to obtain the category probability distribution and initial classification results;

[0012] S5. Perform multi-view consistency test based on visibility vector, continuity of adjacent view feature changes and category probability distribution, generate final judgment result, and reflect the final judgment result back to the product coordinate system to output defect result set.

[0013] S1 specifically includes: establishing a product coordinate system, a camera coordinate system, and an image coordinate system based on the three-reference positioning relationship of the fixture; performing intrinsic parameter calibration and distortion calibration on the industrial camera to obtain the camera intrinsic parameter matrix and distortion parameter set; performing pose calibration on the multi-degree-of-freedom attitude adjustment mechanism to obtain the pose parameters corresponding to each acquisition angle; dividing the workpiece surface into planar areas, arc-edge areas, and corner areas according to the local curvature and the included angle between adjacent local normals, based on the fitting results of the workpiece CAD template or reference image, forming a surface area type table; driving the multi-degree-of-freedom attitude adjustment mechanism to move sequentially according to the acquisition angle sequence, and triggering the industrial camera to acquire data when the pose feedback meets the angle error and angular velocity stability conditions, thereby obtaining a multi-angle image sequence and a pose parameter sequence.

[0014] S2 specifically includes: performing distortion correction, region of interest cropping, brightness normalization, and specular suppression on each image in the multi-angle image sequence to obtain a preprocessed image; inputting the preprocessed image into a single-view candidate detection model to generate a defect probability map, and forming a binary mask based on the candidate detection threshold; performing connected component analysis and area, aspect ratio, and average contrast filtering on the binary mask to obtain candidate regions and their bounding boxes, centroid coordinates, region types, candidate region pixel sets, and local image patches; establishing a cross-view mapping relationship between adjacent acquisition angles based on pose parameters and surface region types, calculating the intersection-union ratio, appearance description vector distance, and normalized cross-correlation of local image patches between candidate regions to complete the cross-view association and obtain the defect trajectory and visibility vector.

[0015] S3 specifically includes: for each defect trajectory, calculating the mean gray level of the candidate region, the mean gray level of the background, the contrast, the gradient magnitude, and the morphological features at each acquisition angle, wherein the morphological features include at least area, aspect ratio, roundness, and principal orientation angle; constructing a gray level mean sequence, a contrast sequence, a gradient magnitude sequence, and a morphological feature sequence in the order of acquisition angles, and further constructing a contrast difference sequence and a gradient difference sequence to form an angle response feature vector; uniformly setting all features corresponding to invisible acquisition angles to zero, and using the visibility vector to characterize the validity of the acquisition angle; calculating and normalizing the scores of each acquisition angle based on the angle between the local surface normal of the candidate region and the camera optical axis, the image quality evaluation value, and the historical reliability, to obtain the viewpoint weight.

[0016] S4 specifically includes: constructing a single-view feature vector from the mean gray level, mean background gray level, contrast, gradient magnitude, area, aspect ratio, roundness, main orientation angle, and visibility marker corresponding to each acquisition angle; stacking the single-view feature vectors from each acquisition angle in order of acquisition angle to form a sequence input, and feeding the contrast difference sequence, gradient difference sequence, and view weight as auxiliary inputs into the classification model; the classification model outputs the category probability distribution corresponding to real scratches, real dents, dust, water stains, and other pseudo-defects, and generates the initial classification result based on the maximum category probability.

[0017] S5 specifically includes: calculating the effective observation ratio of the defect trajectory based on the visibility vector; calculating the continuity of feature changes based on the single-view feature vector of adjacent acquisition angles; generating a consistency score by combining the category probability distribution; generating the final discrimination result based on the minimum effective observation threshold, the consistency score threshold, and whether the initial classification result belongs to a real scratch or a real pit; mapping the final discrimination result back to the product coordinate system and outputting a defect result set containing the defect number, reference acquisition angle number, center coordinates in the product coordinate system, region bounding box, defect category, real defect marker, category probability distribution, consistency score, and visibility vector.

[0018] The beneficial effects of this invention are as follows:

[0019] This invention establishes a multi-angle image sequence under a sequence of acquisition angles and combines pose parameters and surface region type for detection. This allows the same multi-degree-of-freedom laptop shell surface region to be continuously observed at different acquisition angles, thereby reducing the blind spots in single-angle detection and improving the detection capability of real defects such as scratches and dents. By associating candidate regions across viewing angles to form defect trajectories, it solves the problem of difficulty in corresponding the same physical object between images from different acquisition angles. This allows multi-angle images to move beyond simple stitching, voting, or result superposition, and instead form a continuous analysis basis for the same defect object.

[0020] This invention constructs an angle response feature vector to uniformly express grayscale mean, background grayscale mean, contrast, gradient magnitude, morphological features, and differential features. This enables the characterization of the response variation of the same defect trajectory under different acquisition angles, thereby improving the ability to distinguish between real scratches, real pits, and dust, water stains, and accidental reflections. By introducing viewpoint weights, the local surface normal, image quality evaluation value, and historical reliability under different acquisition angles are comprehensively utilized. This gives greater weight to acquisition angles that contribute more to the current defect trajectory discrimination, thus improving the detection stability of complex areas such as curved edges and corners.

[0021] This invention employs a multi-view consistency check, using visibility vectors, continuity of feature changes between adjacent views, and category probability distribution as the final discrimination criteria. This reduces the false positive rate caused by dust, water stains, and accidental reflections, improving the accuracy and repeatability of the final discrimination results. The final discrimination results are then mapped back to the product coordinate system, and a defect result set is output, facilitating subsequent rework, cleaning, re-inspection, and quality grading. Therefore, it has good engineering feasibility and production application value. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of a multi-degree-of-freedom notebook casing surface defect detection method according to the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Example: This example provides a method for detecting surface defects in a multi-degree-of-freedom notebook casing (hereinafter referred to as the workpiece).

[0025] This method is deployed on a detection platform consisting of a fixture positioning mechanism, a multi-degree-of-freedom attitude adjustment mechanism, an industrial camera imaging mechanism, a controllable illumination mechanism, a pose feedback mechanism, and a computing processing mechanism.

[0026] During the inspection, the multi-degree-of-freedom laptop shell is first fixed to the fixture positioning mechanism, and then the multi-degree-of-freedom attitude adjustment mechanism drives the workpiece to move sequentially relative to the industrial camera and the controllable lighting mechanism according to a preset angle sequence, so as to obtain multi-angle images of the same physical area at each acquisition angle.

[0027] Subsequently, preprocessing and single-view suspected defect extraction are performed on each image to obtain a set of candidate regions; then, a cross-view mapping relationship is established based on the pose parameters output by the pose feedback mechanism, and the candidate regions of each view are associated to form a defect trajectory.

[0028] Then, for each defect trajectory, the mean gray level, mean background gray level, contrast, gradient magnitude, morphological features and cross-angle changes are calculated to construct an angle response feature vector; based on this, defect classification is performed by combining viewpoint weights, and the final judgment result is output through multi-view consistency test.

[0029] In this embodiment, "multi-degree-of-freedom notebook shell" refers to a shell-like workpiece whose surface simultaneously includes at least two types of areas, such as planar areas, arc-edge areas, or corner areas, and which requires attitude adjustment through two or more degrees of freedom during the acquisition process to allow the target surface to sequentially enter the effective imaging field of view.

[0030] "Candidate region" refers to the connected regions of suspected defective pixels that are retained in a single-view image after being filtered by the candidate detection model and connected components;

[0031] "Defect trajectory" refers to the candidate region association sequence corresponding to the same physical object from multiple acquisition angles;

[0032] "Angle response feature vector" refers to the set of features that describe the changes in optical response of the same defect trajectory under different acquisition angles;

[0033] "Viewpoint weight" refers to the normalized coefficient used to characterize the contribution of each acquisition angle to the current defect trajectory discrimination;

[0034] "Multi-view consistency test" refers to the process of verifying the initial classification results based on the effective observation ratio of the defect trajectory, the continuity of changes in features of adjacent viewpoints, and the classification confidence.

[0035] Specific steps: as follows Figure 1 As shown, this embodiment provides a method for detecting surface defects in a multi-degree-of-freedom laptop casing, including the following steps:

[0036] S1. Establish the product coordinate system, camera coordinate system and image coordinate system, calibrate the industrial camera and multi-degree-of-freedom attitude adjustment mechanism, obtain the surface area type, acquisition angle sequence and pose parameters, and acquire multi-angle image sequence according to the acquisition angle sequence.

[0037] Specifically, the process involves: establishing a coordinate system and performing imaging calibration, as well as acquiring multi-angle images according to a preset angle sequence.

[0038] First, the multi-degree-of-freedom laptop casing is clamped onto the fixture positioning mechanism, and a product coordinate system is established based on the three-datum positioning relationship of the fixture. Specifically, the datum point formed by the fixture positioning is taken as the origin of the product coordinate system, and the length direction of the workpiece is taken as the product coordinate system's origin. The axis, with the workpiece width direction as the product coordinate system. The axis, with the outward normal direction of the workpiece surface as the product coordinate system. Axis; with the optical center of the industrial camera as the origin of the camera coordinate system, and the direction of the optical axis as the camera coordinate system. The image coordinate system is defined by the top-left corner of the image as its origin and the horizontal direction to the right as its axis. The axis, with the vertical downward direction of the image as the coordinate system. The coordinate system defines a unified reference for subsequent pose parameters, mapping matrices, and defect result mapping, eliminating the problem of missing coordinate sources.

[0039] Perform intrinsic parameter calibration and distortion calibration on the industrial camera to obtain the camera intrinsic parameter matrix:

[0040]

[0041] In the formula, This is the camera intrinsic parameter matrix; For image Directional equivalent focal length; For image Directional equivalent focal length; The x-coordinate of the principal point; The ordinate of the principal point is used. Distortion parameters are recorded using a set of standard radial and tangential distortion parameters from an industrial camera, and are uniformly used in subsequent image preprocessing.

[0042] Let the preset acquisition angle sequence be:

[0043]

[0044] In the formula, To collect angle sequences; For the first One collection angle; This refers to the number of images acquired. To balance detection completeness and real-time performance, Choose 4 to 24; under normal production line cycle time, 6 to 12 is preferred, for example, 4 can be chosen. The interval between adjacent acquisition angles is 10°. Here, "acquisition angle" is not an abstract label, but the actual target attitude recorded value after the multi-degree-of-freedom attitude adjustment mechanism is executed.

[0045] Perform pose calibration on the multi-degree-of-freedom attitude adjustment mechanism to obtain the pose parameters corresponding to the iii-th acquisition angle:

[0046]

[0047] In the formula, For the first Pose parameters for each acquisition angle; For the first Rotation matrix corresponding to each acquisition angle; For the first The translation vector corresponding to each acquisition angle; For the first The angle value of each acquisition angle. Subsequent cross-view mapping relationships are directly called. and To build.

[0048] To ensure a clear basis for surface mapping and angle weight allocation, the workpiece surface is further divided into planar, curved edge, and corner areas during the calibration stage based on the fitting results of the workpiece CAD template or reference image. The absolute value of local curvature should not exceed [a certain value]. The region is denoted as the planar region; the absolute value of the local curvature is greater than and not greater than Regions with an angle exceeding 15° between adjacent local normals are denoted as arc-edge regions; regions with an angle exceeding 15° between adjacent local normals are denoted as corner regions. This region type record serves as input for subsequent construction of the mapping matrix, calculation of local surface normals, and assignment of viewpoint weights. The front-end output of S1 includes: camera intrinsic parameter matrix, distortion parameter set, acquisition angle sequence, pose parameter set, surface region type table, and product template coordinate and image coordinate registration table.

[0049] The multi-degree-of-freedom attitude adjustment mechanism follows the sequence of collected angles. Move sequentially. For each acquisition angle. First, the target attitude is controlled to reach its position. Then, the real-time angle error and angular velocity output by the pose feedback mechanism are read. When the angle error is no greater than 0.1° and the angular velocity is no greater than 0.5° / s within two consecutive sampling periods, the current acquisition angle is determined to have reached a stable motion state. Only after the motion is stable is established will the industrial camera trigger acquisition, thus avoiding edge blurring and candidate region offset caused by motion residue from the source.

[0050] Three frames were continuously acquired at each acquisition angle, and the frame with the largest local Laplacian variance was taken as the valid image, thus obtaining the first frame. Images from various acquisition angles:

[0051]

[0052] In the formula, For the first Valid images corresponding to each acquisition angle; For the first Pixel coordinates at each acquisition angle The grayscale or brightness value at that location. The pose parameters recorded synchronously with this valid image are still denoted as... During the acquisition process, the light source brightness for the planar area remains constant; the light source brightness for the curved edge area and corner area is switched according to a preset level table. For example, the curved edge area can use level 2 or 3 of low-angle strip light, and the corner area can use a combination of ring light and coaxial light to ensure that the brightness distribution of different areas in the image is comparable. The final output of S1 is a multi-angle image sequence. and pose parameter sequence It is available for use by S2 and S3.

[0053] S2. Preprocess the multi-angle image sequence, extract candidate regions from each acquisition angle, establish a cross-view mapping relationship based on the pose parameters and the surface region type, associate the candidate regions, and obtain the defect trajectory and visibility vector.

[0054] Specifically, this includes: image preprocessing and single-view suspected defect extraction, as well as cross-view defect tracking and matching based on motion priors.

[0055] For each image Distortion correction, region of interest (ROI) clipping, brightness normalization, and specular suppression are performed sequentially. The ROI is determined based on the product template coordinates and image coordinates registration table output by S1. First, the fixture, background, and non-detection surface areas are deleted, then the workpiece detection area is retained. Brightness normalization is performed using the following formula:

[0056]

[0057] In the formula, For the first Pixel values ​​after normalization based on the acquisition angle; For the first The average grayscale value of the background reference area at each acquisition angle; For the first The standard deviation of grayscale in the background reference area at each acquisition angle; To prevent extremely small positive numbers with a denominator of 0, it is preferable to select... The background reference area is selected as the defect-free calibration area of ​​the workpiece or a reference band extending 15 to 30 pixels outward from the edge of the workpiece, and all known candidate pixels in this area are removed. The purpose of this normalization process is to convert the overall brightness fluctuation under different acquisition angles to a unified dimension, avoiding distortion in subsequent contrast and gradient amplitude comparisons.

[0058] During highlight suppression, pixels with a grayscale value of at least 250 from the 8-bit grayscale image are first read to form an initial overexposed region. Then, mask deletion is performed on the initial overexposed region with an area of ​​at least 16 pixels to prevent specular highlights from directly entering the candidate region set. After preprocessing, a single-view candidate detection model is called to extract suspected defects from the normalized image, resulting in a defect probability map.

[0059]

[0060] In the formula, For the first Each acquisition angle pixel coordinate The probability value that the location belongs to the defect candidate region; This is a single-view candidate detection model. This single-view candidate detection model can employ a segmentation network, an object detection network, or an anomaly detection network; in this embodiment, a pixel-level segmentation network is preferred to directly output pixel probabilities.

[0061] The probability map is converted into a binary mask based on the candidate detection threshold:

[0062]

[0063] In the formula, These are binary mask values; The candidate detection threshold is preferably set to 0.4 to 0.7. Connectivity analysis is performed on the binary mask to obtain the candidate region set.

[0064] ;

[0065] In the formula, For the first A set of candidate regions for each acquisition angle; For the first The first collection angle One candidate region; For the first The number of candidate regions retained for each acquisition angle.

[0066] To ensure that the candidate region set is a workable object for cross-view matching, deletion rules are applied after connected component analysis. Connected components with an area less than 9 pixels are directly deleted; elongated connected components with a minimum bounding rectangle aspect ratio greater than 20 and an average contrast ratio less than 0.05 are identified as reflective stripes and deleted; connected components spanning two surface region types and with an area greater than 2000 pixels are split using a watershed method and then retained separately. After the above filtering, each candidate region is output with a candidate region number, bounding box, centroid coordinates, region type, candidate region pixel set, and local image patch. The initial output of S2 is the candidate region set. and its attribute records.

[0067] S2 continues to read the surface region type table, pose parameter sequence, and candidate region set output by S1, establishing a cross-view mapping relationship between adjacent acquisition angles. For candidate regions in the planar region, the mapping relationship between adjacent acquisition angles is represented by a homography matrix:

[0068]

[0069] In the formula, For the first Homogeneous coordinates of pixels in candidate regions for each acquisition angle; For the first Predicted homogeneous coordinates of pixels in candidate regions for each acquisition angle; For surface region type From the next The first collection angle to the first A mapping matrix for each acquisition angle.

[0070] The mapping matrix is ​​further represented as follows:

[0071]

[0072] In the formula, For from the first The first collection angle to the first The relative rotation matrix of each acquisition angle; For from the first The first collection angle to the first The relative translation vector of each acquisition angle; For surface region type The corresponding local surface normal vector; For surface region type The equivalent distance from the local surface to the camera's optical center; Here, T represents the inverse of the camera intrinsic parameter matrix, and T denotes the transpose of the matrix or vector. For arc-edge and corner regions, instead of treating the entire surface as a single plane, it is divided into multiple local tangent plane segments along the surface contour. A "local tangent plane segment" refers to a local region on the current surface with a length of 2mm to 5mm or an image scale of 32 to 64 pixels. The fitted plane of this local region is used to replace the real surface to establish a mapping relationship. Each local tangent plane segment records its independent local surface normal vector and equivalent distance, which are then used in the mapping process. This approach provides an executable approximate basis for the mapping relationship of the surface region, rather than simply stating "surface mapping".

[0073] In the case of the first The first collection angle towards the first After mapping is completed at each acquisition angle, the predicted region is obtained. Then calculate the prediction area. With actual candidate regions Matching similarity:

[0074]

[0075] In the formula, For the prediction area With actual candidate regions Matching similarity; The intersection-union ratio (IUU) between the predicted region and the actual candidate region; For the first The first collection angle Appearance description vectors of candidate regions; For the first The first collection angle Appearance description vectors of candidate regions; The scale parameter for the vector distance describing the appearance; The normalized cross-correlation value for a local image patch; For the first The first collection angle Local image patches corresponding to each candidate region; For the first The first collection angle Local image patches corresponding to each candidate region; , , These are the weighting coefficients, and In implementation, it is advisable to take... , , . This represents the L2 norm (or Euclidean distance) of a vector.

[0076] when At that time, determine the candidate region With candidate regions Belonging to the same physical object, The matching threshold is preferably set between 0.45 and 0.65. If the same candidate region matches multiple target candidate regions simultaneously, only the matching edge with the highest similarity is retained, and all other matching edges are deleted. If multiple candidate regions match the same target candidate region simultaneously, the Hungarian matching algorithm is used to retain the matching combination with the highest total similarity globally. This forms the defect trajectory:

[0077]

[0078] In the formula, For the first Defect trajectory; It is the first The candidate region index corresponding to the defect trajectory in the first acquisition angle; when the first... When there is no corresponding candidate region for a given acquisition angle Empty.

[0079] To characterize the first To determine whether a defect trajectory is effectively observed at each acquisition angle, a visibility vector is defined.

[0080]

[0081] In the formula, For the first The visibility vector of the defect trajectory; For the first The defect trajectory is in the first Visibility markers for each acquisition angle; when a matching candidate region exists. When no matching candidate region exists, If no new candidate region is matched for two consecutive acquisition angles, the current defect trajectory terminates; if only one acquisition angle is not matched but matches exist for both the preceding and following acquisition angles, the same defect trajectory number is retained, and only one defect is recorded at the missing acquisition angle. It does not forcibly construct false candidate regions. S2 outputs a set of defect trajectories and a set of visibility vectors for use by S3, S4, and S5.

[0082] S3. Based on the defect trajectory, extract the mean gray level, mean background gray level, contrast, gradient magnitude, morphological features and differential features, construct the angle response feature vector, and generate the viewpoint weight based on the local surface normal, image quality evaluation value and historical reliability.

[0083] Specifically, the following steps are taken: Construct a feature vector for the defect angle response and adaptively assign angle importance weights:

[0084] S3 for each defect trajectory Extract angular response features from each viewpoint. For the first viewpoint... Candidate regions from each acquisition angle First, calculate the grayscale mean:

[0085]

[0086] In the formula, For the first The defect trajectory in the first The average grayscale value of each acquisition angle; Candidate region The number of pixels. This quantity reflects the average brightness response of the area corresponding to the defect trajectory at the current acquisition angle.

[0087] Then focus on the candidate region Constructing background annular region In this embodiment, the background annular region is defined as follows: first, the candidate region boundary is expanded outward by 3 to 8 pixels to form a closed band-shaped region; then, all candidate region pixels are subtracted from this closed band-shaped region, and the remaining pixels constitute the background annular region. Based on this, the average background grayscale value is calculated:

[0088]

[0089] In the formula, For the first The defect trajectory in the first Average background grayscale value from each sampling angle; Background annular region The number of pixels. Further calculation of contrast:

[0090]

[0091] In the formula, For the first The defect trajectory in the first The contrast of each acquisition angle. The greater the contrast, the stronger the distinguishability between the candidate region and its surrounding background.

[0092] Gradient magnitude mean is adopted Operator computation:

[0093]

[0094] In the formula, For the first The defect trajectory in the first The average gradient magnitude of each acquisition angle; To normalize the image in The rate of change of grayscale in the direction; To normalize the image The rate of change of grayscale in the direction. This quantity reflects the sharpness of the defect boundary.

[0095] In terms of morphological features, the perimeter is calculated first. and area Then calculate the roundness:

[0096]

[0097] In the formula, For the first The defect trajectory in the first Roundness at each acquisition angle; The area of ​​the candidate region; Perimeter of the candidate region. Aspect ratio. The ratio of the longer side to the shorter side of the smallest bounding rectangle of the candidate region, and the principal direction angle. Image taken from the principal axis relative to the candidate region The angle between the axes. From this, we construct the morphological feature vector:

[0098]

[0099] In the formula, For the first The defect trajectory in the first The morphological feature vector of each acquisition angle.

[0100] For the For each defect trajectory, construct the gray-scale mean sequence, contrast sequence, gradient magnitude sequence, and morphological feature sequence respectively:

[0101]

[0102]

[0103]

[0104]

[0105] In the formula, For the first The gray-scale mean sequence of the defect trajectories; For the first A contrast sequence of defect trajectories; For the first The gradient magnitude sequence of the defect trajectory; For the first A sequence of morphological features of defect trajectories.

[0106] To characterize the changing trend between adjacent acquisition angles, differential features are further constructed:

[0107]

[0108]

[0109] In the formula, For the first The amount of contrast change of a defect trajectory between adjacent acquisition angles; For the first The gradient amplitude change of the defect trajectory between adjacent acquisition angles. If the... The image is not visible from any particular angle, that is... Then let the angle corresponding to that acquisition angle , , , , , , and Set all to 0, and only through the visibility vector This indicates that the acquisition angle is invalid. This processing rule ensures that the input dimensions of subsequent classification models are fixed and that missing viewpoints are not misinterpreted as low-response viewpoints.

[0110] Finally, the complete angular response feature vector is obtained:

[0111]

[0112] In the formula, For the first The angular response feature vector of the defect trajectory; For all The contrast difference sequence is composed of; For all The gradient difference sequence is formed. The first part of the output of S3 is a set of angle response feature vectors.

[0113] S3 continues to read the surface region type table and pose parameter sequence output by S1, the defect trajectory output by S2, and the angle response feature vector output by the previous segment, and generates viewpoint weights for each defect trajectory. First, for the... The defect trajectory in the first The scoring definition for each acquisition angle is:

[0114]

[0115] In the formula, For the first The defect trajectory in the first Scoring based on each collection angle; The angle between the local surface normal of the candidate region and the camera optical axis; For the first The defect trajectory in the first Image quality evaluation values ​​for each acquisition angle; For the first Historical reliability of each data collection angle; , , This is the rating coefficient, and .in, It is obtained by linear combination of local Laplacian variance, local signal-to-noise ratio and overexposure ratio after normalization, and the dimensions of the three are unified to the interval of 0 to 1. The initial value is 0.8, and it will be dynamically updated based on the statistical results after training.

[0116] Then normalize the scores into perspective weights:

[0117]

[0118] In the formula, For the first The defect trajectory in the first The viewpoint weights of each acquisition angle. (From all) Constructing the viewpoint weight vector:

[0119]

[0120] In the formula, For the first The viewpoint weight vectors for each defect trajectory. For planar areas, the viewpoint weights tend to be balanced because the local surface normal changes little between different acquisition angles; for curved and corner areas, acquisition angles that make the defect outline clearer and have greater contrast will receive higher viewpoint weights. S3 outputs a set of angle response feature vectors and a set of viewpoint weight vectors for use by S4 and S5.

[0121] S4. Input the angle response feature vector and the viewpoint weight into the classification model to obtain the category probability distribution and the initial classification result;

[0122] Specifically, the defect classification is as follows: S4 reads the angle response feature vector output by S3. and viewpoint weight vector Each defect trajectory is classified. To ensure the model input has a clear structure, a single-view feature vector is first defined:

[0123]

[0124] In the formula, For the first The defect trajectory in the first The single-view feature vectors of each acquisition angle are then stacked in order of acquisition angle to form a sequence input. and the differential feature sequence and As auxiliary input and viewpoint weight vector They are then fed into the classification model.

[0125] The preferred classification model uses a network consisting of two one-dimensional convolutional layers, one bidirectional sequence coding unit, and one fully connected layer. Its computation process is as follows:

[0126]

[0127]

[0128] In the formula, For the first High-level semantic features of defect trajectories; An angle response feature encoding network; For the first The probability distribution of the categories of the defect trajectories; This is the classification layer weight matrix; This is the bias vector for the classification layer.

[0129] The category tag set is defined as:

[0130] ;

[0131] In the formula, This is a set of category labels. From this, we obtain the initial classification result:

[0132] ;

[0133] In the formula, For the first Initial classification results of the defect trajectories; For the first The defect trajectory belongs to the category The probability value is given. To address the issue of imbalanced class samples, a weighted cross-entropy loss is used during training, with class weights normalized by the inverse of the number of training samples for each class. For normalized exponential functions, This indicates the value of the variable that causes the subsequent expression to reach its maximum value. S4 outputs the initial classification results and category probability distribution for use by S5.

[0134] S5. Perform a multi-view consistency test based on the visibility vector, the continuity of adjacent view feature changes and the category probability distribution, generate the final discrimination result, and reflect the final discrimination result back to the product coordinate system to output the defect result set.

[0135] Specifically, the process is as follows: Multi-view consistency check and final judgment: S5 reads the defect trajectory and visibility vector output by S2, the angle response feature vector output by S3, and the initial classification result and category probability distribution output by S4, and performs a multi-view consistency check on each defect trajectory. First, the consistency score is calculated:

[0136] ;

[0137] In the formula, For the first Consistency score of defect trajectories; , , The consistency score coefficient is , and ; The scale parameter represents the distance between single-view feature vectors; the first term represents the effective observation ratio, the second term represents the continuity of feature changes between adjacent acquisition angles, and the third term represents the maximum class confidence of the classification model.

[0138] To prevent the judgment of a defect as a real defect based solely on a high-confidence output from a single acquisition angle, a minimum effective observation threshold is set:

[0139]

[0140] In the formula, The minimum effective observation threshold; This is a rounding function. The formula indicates that when the total number of acquisition angles is small, at least 3 valid acquisition angles are required; when the total number of acquisition angles is large, at least 40% of the total acquisition angles are required to form a valid observation.

[0141] The final discrimination rule is defined as follows:

[0142]

[0143] In the formula, For the first The final judgment result of the defect trajectory; when When it represents a real defect; when Time indicates a pseudo-defect; The consistency score threshold is determined by the score value corresponding to the highest F1 score on the validation set. This is a logical AND operator. If the classification model outputs a true scratch or dent, but the consistency score is lower than [a certain value], then [this is the logical AND operator]. If the defect trajectory is not directly output as a real defect, it will be output as a pseudo defect or a re-inspection object. This avoids the accidental reflections or contaminants caused by "high response in a single frame and unstable sequence" being incorrectly identified as real defects.

[0144] Finally, each defect trajectory is mapped back to the product coordinate system, generating a defect result record. The defect result record includes at least: defect number, reference acquisition angle number, center coordinates in the product coordinate system, region bounding box, defect category, actual defect marker, category probability distribution, consistency score, and visibility vector. If the same defect trajectory has center coordinates at multiple acquisition angles, the coordinates corresponding to the acquisition angle with the highest viewpoint weight are taken as the primary coordinates, and the weighted average of all visible acquisition angle coordinates is used as the verification coordinates. S5 outputs a defect result set for rework, cleaning, re-inspection, and quality grading.

[0145] Model training methods:

[0146] In order to directly implement the single-view candidate detection model in S2 and the classification model in S4, this embodiment further explains the model training method.

[0147] First, collect labeled multi-angle training sample sequences. For real scratches and dents, confirm the existence of real grooves or depressions on the workpiece surface through manual re-inspection, microscopic observation, or contour measurement, and then mark them as real defects. For dust and water stains, re-collect images under the same acquisition conditions after blowing, wiping, or cleaning. If no candidate region is formed at the corresponding location twice consecutively, the original object is marked as a false defect. The training set, validation set, and test set can be divided in a ratio of 70%, 15%, and 15%, respectively, and images of the same workpiece from different acquisition angles must not be assigned to different datasets to avoid data leakage.

[0148] The above training sample organization method confirms the authenticity of defect labels through manual re-inspection, microscopic observation or contour measurement, and confirms the authenticity of defect labels through blowing, wiping or cleaning and then re-photographing.

[0149] The joint training objective function is defined as:

[0150] ;

[0151] In the formula, The joint training objective function; For candidate detection loss; For classification loss; For trajectory consistency loss; and This is a tradeoff coefficient. The role of trajectory consistency loss is to reduce the distance between the encoded features of the same physical defect at different acquisition angles, and to increase the distance between the encoded features of different objects, thereby improving the closed-loop consistency between S2 and S4.

[0152] The update of historical reliability is performed using the following formula:

[0153]

[0154] In the formula, For the first The first acquisition angle at the 1st Historical reliability of a statistical period; For the updated historical reliability; For smoothing coefficients; For the first The first acquisition angle at the 1st The effective contribution accuracy rate for each statistical period. The statistical period can be set to update once for every 100 workpieces. The update result is directly used as the input for the viewpoint weight scoring item in S3, and there is no problem of disconnection between the training module and the discrimination module.

[0155] All the above formulas are performed using dimensionless numerical calculations; the relevant formulas are based on empirical models that approximate the real situation, obtained through extensive data collection and software simulation fitting. The preset parameters and thresholds involved in the formulas can be conventionally set and adjusted by those skilled in the art according to the physical constraints of the actual application scenario.

[0156] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0157] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0158] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0159] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0160] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0161] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0162] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0163] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting surface defects in a multi-degree-of-freedom notebook computer casing, characterized in that, Includes the following steps: S1. Establish the product coordinate system, camera coordinate system and image coordinate system, calibrate the industrial camera and multi-degree-of-freedom attitude adjustment mechanism, obtain the surface area type, acquisition angle sequence and pose parameters, and acquire multi-angle image sequences according to the acquisition angle sequence. S2. Preprocess the multi-angle image sequence, extract candidate regions from each acquisition angle, establish cross-view mapping relationship based on pose parameters and surface region type, associate the candidate regions, and obtain defect trajectory and visibility vector; S3. Extract the mean gray level, mean background gray level, contrast, gradient magnitude, morphological features and differential features based on the defect trajectory, construct the angle response feature vector, and generate the viewpoint weight based on the local surface normal, image quality evaluation value and historical reliability. S4. Input the angle response feature vector and viewpoint weights into the classification model to obtain the category probability distribution and initial classification results; S5. Perform multi-view consistency test based on visibility vector, continuity of adjacent view feature changes and category probability distribution, generate final judgment result, and reflect the final judgment result back to the product coordinate system to output defect result set.

2. The method for detecting surface defects in a multi-degree-of-freedom notebook casing according to claim 1, characterized in that, S1 specifically includes: Establish product coordinate system, camera coordinate system, and image coordinate system based on the three reference positioning relationships of the fixture; Perform intrinsic parameter calibration and distortion calibration on the industrial camera to obtain the camera intrinsic parameter matrix and distortion parameter set; Perform pose calibration on the multi-degree-of-freedom attitude adjustment mechanism to obtain the pose parameters corresponding to each acquisition angle; Based on the fitting results of the workpiece CAD template or reference image, the workpiece surface is divided into planar areas, arc-edge areas, and corner areas according to the local curvature and the included angle between adjacent local normals, forming a surface area type table.

3. The method for detecting surface defects in a multi-degree-of-freedom notebook casing according to claim 2, characterized in that, It also includes: driving a multi-degree-of-freedom attitude adjustment mechanism to move sequentially according to the acquired angle sequence, triggering an industrial camera to acquire data when the pose feedback meets the conditions for angle error and angular velocity stability, and obtaining a multi-angle image sequence and a pose parameter sequence.

4. The method for detecting surface defects in a multi-degree-of-freedom notebook casing according to claim 1, characterized in that, S2 specifically includes: For each image in the multi-angle image sequence, distortion correction, region of interest cropping, brightness normalization, and specular suppression are performed sequentially to obtain a preprocessed image; The preprocessed image is input into a single-view candidate detection model to generate a defect probability map, and a binary mask is formed based on the candidate detection threshold. Perform connected component analysis and area, aspect ratio and average contrast filtering on the binary mask to obtain candidate regions and their bounding boxes, centroid coordinates, region types, candidate region pixel sets and local image patches; A cross-view mapping relationship between adjacent acquisition angles is established based on pose parameters and surface region type.

5. The method for detecting surface defects in a multi-degree-of-freedom notebook casing according to claim 4, characterized in that, It also includes: calculating the intersection-union ratio, appearance description vector distance and local image patch normalized cross-correlation between candidate regions to complete cross-view correlation and obtain defect trajectory and visibility vector.

6. The method for detecting surface defects in a multi-degree-of-freedom notebook casing according to claim 1, characterized in that, S3 specifically includes: For each defect trajectory, the mean gray level of the candidate region, the mean gray level of the background, the contrast, the gradient magnitude, and the morphological features are calculated at each acquisition angle. The morphological features include at least the area, aspect ratio, roundness, and main direction angle. The grayscale mean sequence, contrast sequence, gradient amplitude sequence, and morphological feature sequence are constructed according to the acquisition angle order, and further contrast difference sequence and gradient difference sequence are constructed to form an angle response feature vector. All features corresponding to the invisible acquisition angle are uniformly set to zero, and the validity of the acquisition angle is represented by the visibility vector.

7. The method for detecting surface defects in a multi-degree-of-freedom notebook casing according to claim 6, characterized in that, It also includes: calculating and normalizing the scores of each acquisition angle based on the angle between the local surface normal corresponding to the candidate region and the camera optical axis, the image quality evaluation value, and the historical reliability, to obtain the viewpoint weight.

8. The method for detecting surface defects in a multi-degree-of-freedom notebook casing according to claim 1, characterized in that, S4 specifically includes: The mean gray level, mean background gray level, contrast, gradient magnitude, area, aspect ratio, roundness, main direction angle, and visibility marker corresponding to each acquisition angle are used to construct a single-view feature vector. The single-view feature vectors of each acquisition angle are stacked in order of acquisition angle to form a sequence input, and the contrast difference sequence, gradient difference sequence and view weight are used as auxiliary inputs and fed into the classification model. The classification model outputs the category probability distributions corresponding to real scratches, real dents, dust, water stains and other pseudo-defects, and generates the initial classification result based on the maximum category probability.

9. The method for detecting surface defects in a multi-degree-of-freedom notebook casing according to claim 1, characterized in that, S5 specifically includes: The effective observation ratio of defect trajectories is calculated based on visibility vectors, the continuity of feature changes is calculated based on single-view feature vectors from adjacent acquisition angles, and a consistency score is generated by combining category probability distributions. The final judgment result is generated based on the minimum effective observation threshold, the consistency score threshold, and whether the initial classification result belongs to a real scratch or a real pit. The final judgment result is reflected back to the product coordinate system, and the defect result set is output.

10. The method for detecting surface defects in a multi-degree-of-freedom notebook casing according to claim 9, characterized in that, The defect results set includes defect number, reference acquisition angle number, center coordinates in the product coordinate system, region bounding box, defect category, actual defect marker, category probability distribution, consistency score, and visibility vector.

Images