An online visual inspection system for camera hanger surface defect detection

Abstract

The application discloses an online visual detection system for surface defect detection of a camera hanger, and relates to the field of visual detection, and comprises five modules of pose compensation and surface unfolding, exposure control and high dynamic fusion, image acquisition and preprocessing, defect identification and classification, and sorting and rejection. Firstly, a stripe grating is projected, three-dimensional point clouds are reconstructed, and pose compensation is carried out, so that the curved surface is unfolded into a plane. Then, high-quality images are generated through multi-exposure fusion, defects are extracted by difference with a template after pose correction and light preprocessing, and the defect types are identified through gradient features. Finally, the target position is tracked, and a pneumatic fork is driven to complete sorting. The application has the advantages that the detection problems caused by pose fluctuation of a special-shaped curved hanger and uneven reflection are effectively solved, accurate defect classification is realized through multi-scale texture identification and dynamic threshold optimization, automatic rejection is completed through a linkage production line, and the application is suitable for industrial online high-speed detection scenes.

Description

Technical Field

[0001] This invention relates to the field of visual inspection, and in particular to an online visual inspection system for detecting surface defects on camera mounts. Background Technology

[0002] With the mass production of security and vehicle-mounted camera equipment, the surface quality of the camera mount, as a core supporting component, directly affects the assembly precision, appearance, and operational stability of the equipment. Surface defects such as scratches, dents, stains, and color differences are easily generated during the mount production process. If these defects are not accurately removed, they will reduce product quality and may even lead to assembly deviations, equipment loosening, and other safety hazards.

[0003] Current mainstream visual inspection systems for hanger surfaces suffer from significant technical shortcomings and operational adaptability deficiencies. Most employ traditional two-dimensional planar imaging inspection schemes, lacking three-dimensional posture compensation and surface flattening capabilities. They are ill-suited to the irregular curved surface structures of hangers and the dynamic pose fluctuations of production lines, making them highly susceptible to imaging distortion due to workpiece tilting or offset, leading to missed or false detections of defects in curved areas. Furthermore, traditional equipment with fixed exposure parameters and a single illumination correction mode struggles to handle the varying reflectivity of hanger surfaces, resulting in overexposure in highly reflective areas and underexposure in low-reflective areas, resulting in insufficient dynamic range and poor defect feature extraction. In addition, the defect recognition algorithms in existing systems are simplistic, relying heavily on simple grayscale difference analysis without incorporating texture gradient features and multi-scale detection mechanisms. This makes them unable to accurately distinguish between similar defects such as scratches, dents, and foreign object attachments, and they lack false defect removal logic, resulting in low detection accuracy. Most equipment lacks a hardware and software collaborative feedback adjustment mechanism, making it unable to adapt to changes in the reflective characteristics of the workpiece surface. It has weak adaptability to working conditions and anti-interference capabilities. At the same time, it is difficult to achieve closed-loop detection for accurate defect location, classification, and automated rejection. The overall detection accuracy, stability, and automation level are insufficient to meet the requirements of high-precision online mass production detection. Summary of the Invention

[0004] To improve existing systems, an online visual inspection system for detecting surface defects on camera hangers is provided. This system effectively solves the detection problems caused by pose fluctuations and uneven reflections of irregularly shaped curved hangers. It achieves accurate defect classification through multi-scale texture recognition and dynamic threshold optimization, and automatically rejects defects in conjunction with the production line, making it suitable for high-speed online industrial inspection scenarios.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] An online visual inspection system for detecting surface defects in camera mounts includes:

[0007] Attitude compensation and surface unfolding module: a striped grating is projected onto the surface of the hanger, and the distorted stripes are collected by auxiliary cameras on both sides to reconstruct a three-dimensional point cloud. The spatial attitude angle is obtained based on the deviation between the point cloud normal vector and the reference normal vector. The point cloud is rotated and transformed according to the attitude angle and unfolded into a two-dimensional planar mapping along the principal curvature direction.

[0008] Exposure control and high dynamic range fusion module: Generates a multi-level exposure time series based on the grayscale statistics of the two-dimensional planar mapping map, controls the camera to acquire multiple frames of original images with different exposures according to the sequence, normalizes the grayscale values ​​of the same pixel in each frame according to the exposure time, and selects the values ​​that fall within the linear response range to synthesize a high dynamic range image.

[0009] Image acquisition and preprocessing module: Calculates the rotation offset and scaling in the image plane based on the spatial attitude angle, performs inverse affine transformation on the high dynamic range image to eliminate distortion caused by pose fluctuation, and outputs the image to be detected under the standard pose after non-uniform illumination correction.

[0010] Defect recognition and classification module: Extract candidate defect regions pixel by pixel by the image to be detected and the pre-stored template. Based on the gradient magnitude distribution and the main peak direction of the gradient direction histogram in the candidate defect regions, the defects are classified into scratches, pits and foreign object attachments. The contour coordinates and bounding box size of each defect are marked.

[0011] The sorting and rejection module: Based on the defect labeling results and the pulse tracking of the conveyor belt encoder, the module tracks the position of each rack. When the defective rack reaches the rejection station, the pneumatic fork is driven to push it into the defective product collection channel.

[0012] Preferably, the attitude compensation and surface unfolding module specifically includes:

[0013] Structured light projection unit: It has a built-in linear laser diode and cylindrical beam expander. After the laser beam is expanded, it forms a fan-shaped parallel stripe grating and is projected onto the surface of the bracket.

[0014] Stripe image acquisition unit: includes a first auxiliary camera and a second auxiliary camera respectively disposed on both sides of the projection unit and tilted symmetrically along the optical axis, used to synchronously acquire left and right distorted stripe images modulated by the surface of the bracket;

[0015] Phase calculation unit: Fourier transforms the distortion fringes on both sides to extract the fundamental frequency and inverse transforms them to obtain the wrapped phase map. After stereo matching of the left and right wrapped phase maps, triangulation is performed to calculate the three-dimensional coordinates of each sampling point and output the point cloud.

[0016] Attitude correction unit: Calculates the vertex normal vector of the point cloud and performs principal component analysis. The pitch angle is determined by the angle between the first principal component and the normal of the imaging surface, and the roll angle is determined by the angle between the second principal component and the horizontal axis of the imaging surface. The two angles are used to construct a rotation matrix to rotate and transform the point cloud so that the normal of the surface to be detected coincides with the normal of the imaging surface.

[0017] Preferably, the attitude compensation and surface unfolding module further includes:

[0018] Curvature analysis unit: Constructs a local neighborhood for each vertex in the point cloud after attitude correction and performs quadratic surface fitting to obtain the fitted surface. After calculating the Gaussian curvature and average curvature of the fitted surface, the principal curvature value and principal direction vector of each vertex are obtained.

[0019] Surface flattening unit: Taking the vertices near the geometric center of the point cloud as the flattening starting point, the vertex chain is traced line by line along the minimum curvature direction indicated by the main direction vector. The spatial distance between adjacent vertices on each vertex chain is mapped to the pixel spacing on the two-dimensional plane. The reflection intensity value of each vertex is kept as the pixel gray value, and a two-dimensional plane mapping map is generated.

[0020] The mapping compensation unit calculates the local stretching ratio of each pixel position in the mapping map caused by the flattening of the surface, and performs inverse distance weighted interpolation correction on the corresponding pixel grayscale value according to the local stretching ratio to suppress grayscale distortion during the flattening process.

[0021] Preferably, the exposure control and high dynamic range fusion module specifically includes:

[0022] Pre-exposure evaluation unit: Takes a pre-exposure image with a reference exposure time, counts the number of overexposed and underexposed pixels, and determines the exposure adjustment direction and step size when the overexposure ratio exceeds the first threshold and the underexposure ratio exceeds the second threshold.

[0023] Exposure sequence generation unit: Using the reference exposure time as the intermediate value, it generates several short exposure times and long exposure times in the decreasing and increasing directions according to the direction and step size, and arranges them in ascending order to form a multi-level exposure sequence;

[0024] Multi-frame acquisition unit: connected to the camera trigger, triggers exposure sequentially and reads each frame and its exposure time;

[0025] High Dynamic Range (HDR) Composition Unit: Divide the grayscale value of each frame by the exposure time to obtain the normalized irradiance. Select the irradiance that falls within the camera's linear response range, perform a weighted average, and write it to the corresponding position in the HDR image. The weight is inversely proportional to the frame exposure time.

[0026] Preferably, the image acquisition and preprocessing module specifically includes:

[0027] Pose parameter receiving unit: obtains pitch angle and roll angle from attitude compensation and surface unfolding module, calculates rotation angle around image center point based on pitch angle and roll angle, and calculates translation vector based on displacement of the hanger in the direction of conveyor belt;

[0028] The inverse affine transformation unit constructs a rotation matrix with the rotation angle and a translation matrix with the translation vector, and combines the two into an inverse affine transformation matrix. This inverse matrix is ​​applied to the coordinates of each pixel in the high dynamic range image to obtain the resampling coordinates. The gray value at the resampling coordinates is obtained by bilinear interpolation and filled into the corresponding position in the standard pose image.

[0029] The illumination correction unit performs Gaussian filtering on the standard pose image to extract the background illumination component. It then divides the gray value of each pixel in the standard pose image by the gray value of the corresponding background illumination component and multiplies it by the gray value of the standard illumination component, outputting the image to be detected after eliminating non-uniform illumination.

[0030] Preferably, the illumination correction unit specifically includes:

[0031] Adaptive filtering subunit: Gaussian filtering is performed on the standard pose image at three scales (large, medium, and small) to obtain the background component at the corresponding scale. The first difference between the large-scale and medium-scale background and the second difference between the medium-scale and small-scale background at each pixel are calculated. The background component at the corresponding scale is adaptively selected as the final background illumination component of the pixel based on the relative magnitude of the first and second differences.

[0032] Boundary Preservation Subunit: Performs anisotropic diffusion filtering on the final background illumination component, sets the diffusion coefficient along the image gradient direction, reduces the diffusion coefficient in areas with large gradient magnitude to preserve the edges of abrupt illumination changes, increases the diffusion coefficient in areas with small gradient magnitude to smooth the gradual illumination, and outputs the smoothed background illumination component after boundary preservation.

[0033] Preferably, the defect identification and classification module specifically includes:

[0034] Template matching unit: Stores template images of defect-free hangers in standard poses, rigidly registers the image to be detected with the template, calculates subpixel translation, and then performs translation correction;

[0035] Differential extraction unit: Subtracts the grayscale of the corrected image from the template pixel by pixel to obtain the difference image, marks the pixels whose absolute difference value exceeds the noise threshold as suspicious pixels and performs eight-neighbor connectivity clustering, merges the interconnected suspicious pixels into candidate defect regions and records their bounding rectangles;

[0036] Texture analysis unit: Calculates gray-level co-occurrence matrices in four directions (horizontal, vertical, diagonal, and anti-diagonal) for each candidate region, extracts contrast and correlation parameters, and judges the degree of anisotropy by the range of contrast in the four directions.

[0037] Classification output unit: If the anisotropy exceeds the threshold and the aspect ratio exceeds the threshold, it is judged as a scratch; if the anisotropy is below the threshold and the regional mean is below the corresponding mean of the template, it is judged as a pit; if the anisotropy is centered and the correlation in all four directions is below the threshold, it is judged as a foreign object attachment. Output the label of each defect type and the coordinates of the four corners of the rectangle.

[0038] Preferably, the defect identification and classification module further includes:

[0039] Multi-scale detection unit: Construct a Gaussian pyramid for the image to be detected and perform differential extraction and texture analysis operations at the top, middle and bottom layers of the pyramid respectively to obtain candidate defect regions and anisotropy at each scale. The candidate regions with overlapping positions at each scale are merged by weighting the inverse of the scale, with the top layer having the highest weight and the bottom layer having the lowest weight. After merging, the bounding rectangle and the average anisotropy are recalculated.

[0040] The spurious defect suppression unit expands a ring-shaped neighborhood of a predetermined width around the merged candidate defect region, calculates the standard deviation of the pixel grayscale values ​​within the ring-shaped neighborhood, and determines that the region is a spurious defect and removes it from the output when the ratio of the grayscale standard deviation inside the candidate region to the grayscale standard deviation of the ring-shaped neighborhood is less than a preset threshold. Candidate regions with a ratio greater than the threshold are retained as valid defects in the output.

[0041] Preferably, a feedback coordination unit is provided between the attitude compensation and surface unfolding module and the exposure control and high dynamic range fusion module, specifically including:

[0042] The feedback coordination unit obtains the statistical value of the reflectivity dynamic range from the exposure control and high dynamic range fusion module. When the statistical value exceeds the preset range threshold, it determines that the local reflection on the surface of the bracket is too strong, causing the structured light stripes to be indistinguishable. It then sends a stripe gain adjustment signal to the attitude compensation and surface unfolding module. The module increases the laser output power of the structured light projector according to the signal and simultaneously reduces the camera acquisition gain. It then reprojects the encrypted stripe grating and acquires a new distorted stripe image for 3D point cloud reconstruction.

[0043] Simultaneously, the feedback coordination unit sends the dynamic range statistical value of reflectivity to the defect identification and classification module. Based on this statistical value, it dynamically adjusts the noise threshold during differential extraction, so that the noise threshold corresponding to the high reflectivity area is higher than the noise threshold corresponding to the low reflectivity area.

[0044] Compared with the prior art, the advantages of the present invention are:

[0045] Leveraging structured light 3D reconstruction and surface flattening technology, this system effectively solves the imaging distortion problems caused by irregular curved surfaces and production pose fluctuations in hanger systems. Through principal curvature mapping and grayscale interpolation correction, it ensures the detection accuracy of curved areas. The system incorporates a multi-level exposure high-dynamic fusion and adaptive illumination correction mechanism, adapting to complex working conditions with alternating high and low reflectivity on hanger surfaces, eliminating overexposure, underexposure, and non-uniform illumination interference, significantly improving image quality. Simultaneously, combined with multi-scale defect detection and false defect removal algorithms, it accurately distinguishes between scratches, dents, and foreign object attachments through grayscale difference and texture feature analysis, effectively reducing missed and false detection rates. With a hardware and software feedback coordination mechanism, it can dynamically adjust structured light projection parameters and detection thresholds to adapt to workpieces with different surface reflectivity characteristics. The entire system achieves full automation of image preprocessing, intelligent defect classification, and workpiece tracking and removal, offering enhanced detection stability, accuracy, and anti-interference capabilities. It perfectly adapts to the online full inspection requirements of high-speed production lines, significantly improving quality inspection efficiency and product yield control. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of an online visual inspection system for detecting surface defects on a camera mount, as proposed in this invention.

[0047] Figure 2 This is a diagram of the attitude compensation and surface unfolding module proposed in this invention;

[0048] Figure 3 This is a diagram of the defect identification and classification module proposed in this invention. Detailed Implementation

[0049] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0050] See Figure 1 As shown, an online visual inspection system for detecting surface defects in camera mounts includes:

[0051] Attitude compensation and surface unfolding module: a striped grating is projected onto the surface of the hanger, and the distorted stripes are collected by auxiliary cameras on both sides to reconstruct a three-dimensional point cloud. The spatial attitude angle is obtained based on the deviation between the point cloud normal vector and the reference normal vector. The point cloud is rotated and transformed according to the attitude angle and unfolded into a two-dimensional planar mapping along the principal curvature direction.

[0052] Exposure control and high dynamic range fusion module: Generates a multi-level exposure time series based on the grayscale statistics of the two-dimensional planar mapping map, controls the camera to acquire multiple frames of original images with different exposures according to the sequence, normalizes the grayscale values ​​of the same pixel in each frame according to the exposure time, and selects the values ​​that fall within the linear response range to synthesize a high dynamic range image.

[0053] Image acquisition and preprocessing module: Calculates the rotation offset and scaling in the image plane based on the spatial attitude angle, performs inverse affine transformation on the high dynamic range image to eliminate distortion caused by pose fluctuation, and outputs the image to be detected under the standard pose after non-uniform illumination correction.

[0054] Defect recognition and classification module: Extract candidate defect regions pixel by pixel by the image to be detected and the pre-stored template. Based on the gradient magnitude distribution and the main peak direction of the gradient direction histogram in the candidate defect regions, the defects are classified into scratches, pits and foreign object attachments. The contour coordinates and bounding box size of each defect are marked.

[0055] The sorting and rejection module: Based on the defect labeling results and the pulse tracking of the conveyor belt encoder, the module tracks the position of each rack. When the defective rack reaches the rejection station, the pneumatic fork is driven to push it into the defective product collection channel.

[0056] See Figure 2 As shown, in the attitude compensation and surface unfolding module, high-precision three-dimensional topographic data of the hanger surface is obtained through the structured light triangulation principle. Specifically, the structured light projection unit incorporates a line-shaped laser diode and a cylindrical beam expander. After beam expansion, the laser beam forms a fan-shaped parallel stripe grating and is projected onto the hanger surface. The stripe image acquisition unit includes a first auxiliary camera and a second auxiliary camera, respectively located on both sides of the projection unit and symmetrically tilted along their optical axes. Both cameras synchronously acquire left and right distorted stripe images that are deformed after being height-modulated by the hanger surface.

[0057] The phase resolution unit performs Fourier transforms on the two distorted fringe images, extracts the fundamental frequency component, and then performs an inverse Fourier transform to obtain the wrapped phase map. After stereo matching of the left and right wrapped phase maps, the three-dimensional spatial coordinates of each sampling point are calculated based on the triangulation principle. The core mapping relationship is as follows:

[0058] ;

[0059] Where Z is the depth value of the sampling point relative to the reference plane, f is the effective focal length of the auxiliary camera, b is the baseline distance between the optical centers of the first and second auxiliary cameras, and d is the parallax of the same sampling point in the left and right wrap-around phase maps. This yields the original three-dimensional point cloud characterizing the surface morphology of the mounting bracket.

[0060] The attitude correction unit first calculates the normal vectors of each vertex in the original point cloud and performs principal component analysis on all normal vectors. The direction of the first principal component is taken as the principal extension direction of the surface to be inspected by the hanger, and its angle with the normal direction of the imaging plane is used to determine the pitch angle α of the hanger's motion around the conveyor belt. The direction of the second principal component is taken as the secondary extension direction, and its angle with the horizontal axis of the imaging plane is used to determine the roll angle β of the hanger around the width direction of the conveyor belt. After obtaining α and β, a rotation matrix is ​​constructed using the Rodriguez formula, and the point cloud is rotated using the following equation:

[0061]

[0062] Where P is a matrix composed of the coordinate vectors of each vertex in the original 3D point cloud, R is a 3D rotation matrix constructed by the pitch angle α and roll angle β, and R is the point cloud matrix output after attitude correction. After this transformation, the normal of the surface to be detected on the mount coincides with the normal of the imaging plane, eliminating the detection attitude deviation introduced by the mount pose fluctuation.

[0063] The curvature analysis unit constructs local neighborhoods for each vertex in the pose-corrected point cloud, and performs quadratic surface fitting on all vertices within each local neighborhood. After obtaining the fitted surface, its Gaussian curvature and mean curvature are calculated, thereby obtaining the principal curvature values ​​and corresponding principal direction vectors of each vertex. Among these, the principal curvature... The solution can be reduced to the following matrix eigenvalue decomposition:

[0064] ;

[0065] Where H is the second-order gradient tensor matrix of the local parametric surface at that vertex, and I is the identity matrix. This involves determinant operations. Solving the characteristic equation yields two eigenvalues, which are the principal curvature values ​​at that vertex. and The corresponding eigenvectors indicate the principal curvature directions, where the principal curvature directions with smaller values ​​are the minimum bending directions when the surface is flattened.

[0066] Based on the above calculations, the surface flattening unit takes the vertices near the geometric center of the point cloud as the flattening starting point and traces the vertex chain line by line along the direction of minimum curvature. The spatial distance between adjacent vertices on each vertex chain is directly mapped to the pixel spacing on the two-dimensional plane, while keeping the reflection intensity value of each vertex as the gray value of the corresponding pixel, thereby generating a two-dimensional planar mapping map.

[0067] Finally, the mapping compensation unit calculates the local stretching ratio of each pixel position in the mapping map caused by non-uniform stretching, and performs inverse distance weighted interpolation correction on the gray value of the corresponding pixel, thereby effectively suppressing gray value distortion during the surface unfolding process and outputting a two-dimensional planar mapping map with accurate shape and gray value for subsequent modules to use.

[0068] In the exposure control and high dynamic range fusion module, a pre-exposure evaluation unit rapidly determines the reflectivity of the hanger surface. Specifically, the pre-exposure evaluation unit uses a preset reference exposure time. Control the area scan camera to capture a pre-exposure image, and count the number of overexposed pixels in the image whose grayscale values ​​exceed the camera's saturation threshold. and the number of underexposed pixels with grayscale values ​​below the dark current noise threshold. When the number of overexposed pixels is less than the total number of pixels in the image... ratio When the first threshold is exceeded, it is determined that the current mounting surface is dominated by high-reflectivity areas. In this case, the exposure time should be shortened to reduce overexposure; when the proportion of underexposed pixels... When the exposure exceeds the second threshold, the area is determined to be dominated by low-reflection regions. In this case, the exposure time should be extended to enhance details in the shadows. Based on the above determination, the pre-exposure evaluation unit outputs the exposure adjustment direction and the corresponding adjustment step size. The adjustment step size is based on or The degree of exceeding the threshold is determined by grading, and the greater the exceedance, the larger the step size.

[0069] The exposure sequence generation unit uses the reference exposure time As an intermediate value, following the aforementioned adjustment direction and step size, several short exposure times are generated by progressively decreasing the exposure time in the direction of decreasing exposure time, such as... Simultaneously, several long exposure times are generated by progressively increasing the exposure time in the direction of increasing exposure time, such as... All short exposure times, the baseline exposure time, and all long exposure times are arranged in ascending order to form a multi-level exposure time series covering low, medium, and high exposure ranges. ,in For the shortest exposure time, The longest exposure time is determined by adaptively adjusting the length of the sequence based on the dynamic range of the reflectivity of the hanger surface. The wider the dynamic range, the more exposure levels the sequence contains.

[0070] The multi-frame acquisition unit is connected to the trigger input of the area scan camera. It triggers the camera to expose sequentially according to the exposure time values ​​in the multi-level exposure time sequence described above, and reads the corresponding original grayscale image after each exposure. and its exposure time The two are associated and stored in a cache.

[0071] The high dynamic range synthesis unit performs pixel-by-pixel fusion of multiple frames of original images stored in the buffer. For each pixel position (x, y), the grayscale value of that position in each frame image is extracted. Normalize it to a uniform irradiance range using the following formula:

[0072]

[0073] in, This represents the normalized irradiance value at pixel (x, y) in the i-th frame of the image. This represents the original grayscale value of that pixel in the image frame. This represents the exposure time corresponding to that frame. After normalization, invalid data whose irradiance values ​​fall outside the camera's nonlinear response range are removed, retaining only those falling within the camera's linear response range. The effective irradiance value within the area. For multiple effective irradiance values ​​retained at the same pixel location, a weighted average is calculated using the following formula:

[0074] ;

[0075] in, Let S be the final irradiance value of the synthesized high dynamic range image at pixel (x, y), and let S be the set of frame indices corresponding to the effective irradiance value at that pixel location. Let be the weight coefficient for the i-th frame. With the exposure time of this frame Inversely proportional, that is The basis for this weighting is that short exposure frames have a high signal-to-noise ratio (SNR) when capturing bright areas, while long exposure frames have a high SNR when capturing dark areas. Using the reciprocal of the exposure time as the weight effectively suppresses the contribution of low SNR data to the synthesis result, ensuring that the synthesized high dynamic range image maintains optimal detail recording capability across all brightness ranges. Ultimately, the high dynamic range synthesis unit outputs a high dynamic range image with rich grayscale levels that is independent of a single exposure time.

[0076] In the image acquisition and preprocessing module, the pitch angle α and roll angle β are obtained from the attitude compensation and surface unfolding module through the pose parameter receiving unit. Based on α and β, the pose parameter receiving unit determines the rotation angle θ that needs to be applied to the image to be detected relative to the standard attitude around the image center point. At the same time, based on the cumulative count of conveyor belt encoder pulses in the adjacent frame acquisition interval, the displacement Δx and Δy of the hanger in the direction of conveyor belt movement are calculated and used as translation vectors in the image plane.

[0077] The inverse affine transformation unit constructs the inverse affine transformation matrix using the above parameters. Specifically, a two-dimensional rotation matrix is ​​constructed using the rotation angle θ. Construct a translation matrix T using the translation vectors (Δx, Δy), and combine the two to form an inverse transformation matrix of the following form:

[0078]

[0079] in, The inverse affine transformation unit maps the pixel coordinates in the standard pose image back to their corresponding positions in the original high dynamic range image. The inverse affine transformation unit maps each pixel coordinate in the high dynamic range image back to its corresponding position. Applying this inverse matrix, the resampled coordinates are calculated:

[0080] ;

[0081] After obtaining the resampling coordinates, the grayscale value at those coordinates is obtained through bilinear interpolation and then filled into the corresponding pixel position in the standard pose image. Through this inverse mapping, a standard pose image free from pose rotation and translation deviations can be directly generated without forward rotation and cropping of the original image, avoiding secondary blurring introduced during interpolation.

[0082] The illumination correction unit performs non-uniform illumination correction on the aforementioned standard pose image. This unit first extracts the background illumination component through an adaptive filtering subunit. The adaptive filtering subunit performs convolution filtering on the standard pose image using Gaussian kernels of large, medium, and small scales, respectively, to obtain background illumination estimation images at the corresponding scales. (x,y) (x,y) and (x, y). For each pixel location, calculate the first difference between the large-scale and mesoscale background. And the second difference between the mesoscale and small-scale backgrounds. .according to and The relative size adaptively selects the final background component: if |D1| > |D2|, it indicates that the illumination change at this location mainly originates from large-scale vignetting, then select... Conversely, if the lighting changes are mainly characterized by mid-frequency textures or small-scale shadows, then select... or The right one among them.

[0083] The boundary preservation subunit further performs anisotropic diffusion filtering on the final background illumination component. This filter sets a diffusion coefficient that varies with the gradient magnitude along the image gradient direction. In regions with large gradient magnitudes, the diffusion coefficient is reduced to preserve boundary information of abrupt illumination changes; in regions with small gradient magnitudes, the diffusion coefficient is increased to smooth gradual illumination, outputting a smoothed background illumination component after boundary preservation. .

[0084] Finally, the illumination correction unit performs pixel-level illumination compensation on the standard pose image using the following formula:

[0085] ;

[0086] in, This represents the original grayscale value at pixel (x, y) of the standard pose image after pose correction. This is the smoothed background illumination component extracted at this pixel after adaptive filtering and boundary preservation. The preset standard illumination grayscale value, This is the grayscale value of the output image to be detected after correction. Through the above division operation, non-uniform lighting such as vignetting and shadows on the surface of the bracket is effectively compensated, so that surface areas with the same reflective properties at different positions present a consistent grayscale response in the output image.

[0087] In the defect identification and classification module, a template image of a defect-free camera mount of the same model in a standard posture is loaded through the template matching unit. The image to be detected is rigidly registered with the template at the pixel level. The sub-pixel translation between the two is calculated and translation correction is performed to ensure accurate alignment.

[0088] The difference extraction unit subtracts the grayscale of the corrected image from that of the template pixel by pixel to obtain the difference image. Pixels whose absolute difference value exceeds the noise threshold are marked as suspicious pixels. Eight-neighbor connectivity clustering is performed on all suspicious pixels, and interconnected pixels are merged into candidate defect regions and their bounding rectangle boundaries are recorded.

[0089] The texture analysis unit calculates the gray-level co-occurrence matrix in four directions (horizontal, vertical, diagonal, and anti-diagonal) for each candidate region, and extracts the contrast parameter Con and correlation parameter Corr for each direction. The range of contrast in the four directions is then used as the basis for the final analysis. The degree of anisotropy is determined by the difference between the maximum and minimum contrast values ​​in each direction:

[0090]

[0091] A larger range indicates that the defect has a clear directionality, while a smaller range indicates that the grayscale changes are uniform in all directions.

[0092] The classification output unit determines the defect category based on the degree of anisotropy and other features. When the anisotropy threshold is exceeded and the region's aspect ratio exceeds the aspect ratio threshold, it is classified as a scratch; when... When the gray level is below the anisotropy threshold and the average gray level of the region is lower than the mean of the corresponding region in the template, it is classified as a pit type; when If a defect is centered and its correlation in all four directions is below the correlation threshold, it is classified as a foreign object attachment type. Output the labels for each defect type and the coordinates of the four corners of the rectangle.

[0093] This module also has a multi-scale detection unit. It constructs a Gaussian pyramid for the image to be detected, and performs differential extraction and texture analysis at the top, middle and bottom layers of the pyramid to obtain candidate defect regions and their anisotropy at each scale. The candidate regions with overlapping positions at each scale are merged by weighting the inverse of the scale, with the top layer having the highest weight and the bottom layer having the lowest weight. After merging, the bounding rectangle and the average anisotropy are recalculated.

[0094] The pseudo-defect suppression unit expands a ring-shaped neighborhood of a predetermined width around the merged candidate region and statistically analyzes the standard deviation of pixel gray levels within the ring-shaped neighborhood. and the standard deviation of gray levels within the candidate region Calculate the ratio of the two. When γ is less than the preset threshold, it indicates that the grayscale fluctuation of the candidate region is lower than that of the surrounding background, which is a false defect introduced by the illumination fluctuation and is therefore eliminated; regions with γ greater than or equal to the threshold are retained as valid defects for output.

[0095] The feedback coordination unit obtains the current dynamic range statistics of reflectivity of the hanger surface from the pre-exposure evaluation unit of the dynamic exposure control and high dynamic range fusion module. The statistical value is calculated as follows: the feedback coordination unit receives the pre-exposure image output by the pre-exposure evaluation unit, and counts the gray values ​​in the image arranged from low to high, then identifies the gray values ​​located at the upper quantile. Gray values ​​located at lower quantiles The dynamic range of reflectivity is calculated using the following formula:

[0096]

[0097] in, The gray value at the high quantile after sorting the gray values ​​in the pre-exposure image represents the reflection intensity of the brightest area on the surface of the hanger. The gray value at the low quantile point represents the reflection intensity of the darkest area; the ratio of the two is... The larger the value, the more significant the difference in reflectivity between different areas of the hanger surface, meaning that there are both strong specular high reflectivity and low reflectivity dark areas at the same time.

[0098] The feedback coordination unit will With the preset range threshold Comparison. When > When the system detects excessive localized reflection on the surface of the mounting bracket, causing the striped grating emitted by the structured light projector to become unrecognizable in highly reflective areas due to camera saturation, the wrapping phase calculation will fail in these areas. At this point, the feedback coordination unit sends a stripe gain adjustment signal to the attitude compensation and surface unfolding module. Upon receiving this signal, the module performs two adjustments: first, it increases the driving current of the line laser diode in the structured light projector to enhance the laser output power, ensuring the striped grating still has sufficient stripe contrast in highly reflective areas; second, it simultaneously reduces the acquisition gain of the auxiliary camera to prevent saturation in normally reflective areas due to increased laser power. After completing these gain adjustments, the structured light projector reprojects the encrypted striped grating, and the auxiliary camera simultaneously acquires the distorted stripe image under the new conditions and re-performs 3D point cloud reconstruction, ensuring complete and accurate surface topography data is obtained across a wide reflectivity range.

[0099] At the same time, the feedback coordination unit will display the dynamic range statistics of reflectivity. The data is sent to the differential extraction unit of the defect identification and classification module. The differential extraction unit dynamically adjusts the noise threshold during differential extraction based on this statistical value, and the adjustment method is as follows:

[0100] ;

[0101] in, As the reference noise threshold, Let be the grayscale value at pixel (x, y) in the pre-exposed image, and k be a preset adjustment coefficient. According to this formula, the noise threshold corresponding to high-reflectivity areas is dynamically increased to a value higher than the baseline to suppress spurious differential responses introduced by grayscale saturation fluctuations in high-reflectivity areas; while the noise threshold corresponding to low-reflectivity areas remains at the baseline level. Through this feedback coordination mechanism, the integrity of structured light reconstruction and the sensitivity of defect detection are both optimized over a wide reflectivity range.

[0102] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0103] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0104] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An online visual inspection system for detecting surface defects in camera mounts, characterized in that, include: Attitude compensation and surface unfolding module: a striped grating is projected onto the surface of the hanger, and the distorted stripes are collected by auxiliary cameras on both sides to reconstruct a three-dimensional point cloud. The spatial attitude angle is obtained based on the deviation between the point cloud normal vector and the reference normal vector. The point cloud is rotated and transformed according to the attitude angle and unfolded into a two-dimensional planar mapping along the principal curvature direction. Exposure control and high dynamic range fusion module: Generates a multi-level exposure time series based on the grayscale statistics of the two-dimensional planar mapping map, controls the camera to acquire multiple frames of original images with different exposures according to the sequence, normalizes the grayscale values ​​of the same pixel in each frame according to the exposure time, and selects the values ​​that fall within the linear response range to synthesize a high dynamic range image. Image acquisition and preprocessing module: Calculates the rotation offset and scaling in the image plane based on the spatial attitude angle, performs inverse affine transformation on the high dynamic range image to eliminate distortion caused by pose fluctuation, and outputs the image to be detected under the standard pose after non-uniform illumination correction. Defect recognition and classification module: Extract candidate defect regions pixel by pixel by the image to be detected and the pre-stored template. Based on the gradient magnitude distribution and the main peak direction of the gradient direction histogram in the candidate defect regions, the defects are classified into scratches, pits and foreign object attachments. The contour coordinates and bounding box size of each defect are marked. The sorting and rejection module: Based on the defect labeling results and the pulse tracking of the conveyor belt encoder, the module tracks the position of each rack. When the defective rack reaches the rejection station, the pneumatic fork is driven to push it into the defective product collection channel.

2. The online visual inspection system for detecting surface defects of a camera mount according to claim 1, characterized in that, The attitude compensation and surface unfolding module specifically includes: Structured light projection unit: It has a built-in linear laser diode and cylindrical beam expander. After the laser beam is expanded, it forms a fan-shaped parallel stripe grating and is projected onto the surface of the bracket. Stripe image acquisition unit: includes a first auxiliary camera and a second auxiliary camera respectively disposed on both sides of the projection unit and tilted symmetrically along the optical axis, used to synchronously acquire left and right distorted stripe images modulated by the surface of the bracket; Phase calculation unit: Fourier transforms the distortion fringes on both sides to extract the fundamental frequency and inverse transforms them to obtain the wrapped phase map. After stereo matching of the left and right wrapped phase maps, triangulation is performed to calculate the three-dimensional coordinates of each sampling point and output the point cloud. Attitude correction unit: Calculates the vertex normal vector of the point cloud and performs principal component analysis. The pitch angle is determined by the angle between the first principal component and the normal of the imaging surface, and the roll angle is determined by the angle between the second principal component and the horizontal axis of the imaging surface. The two angles are used to construct a rotation matrix to rotate and transform the point cloud so that the normal of the surface to be detected coincides with the normal of the imaging surface.

3. The online visual inspection system for detecting surface defects of a camera mount according to claim 1, characterized in that, The attitude compensation and surface unfolding module also includes: Curvature analysis unit: Constructs a local neighborhood for each vertex in the point cloud after attitude correction and performs quadratic surface fitting to obtain the fitted surface. After calculating the Gaussian curvature and average curvature of the fitted surface, the principal curvature value and principal direction vector of each vertex are obtained. Surface flattening unit: Taking the vertices near the geometric center of the point cloud as the flattening starting point, the vertex chain is traced line by line along the minimum curvature direction indicated by the main direction vector. The spatial distance between adjacent vertices on each vertex chain is mapped to the pixel spacing on the two-dimensional plane. The reflection intensity value of each vertex is kept as the pixel gray value, and a two-dimensional plane mapping map is generated. The mapping compensation unit calculates the local stretching ratio of each pixel position in the mapping map caused by the flattening of the surface, and performs inverse distance weighted interpolation correction on the corresponding pixel grayscale value according to the local stretching ratio to suppress grayscale distortion during the flattening process.

4. The online visual inspection system for detecting surface defects of a camera mount according to claim 1, characterized in that, The exposure control and high dynamic range fusion module specifically includes: Pre-exposure evaluation unit: Takes a pre-exposure image with a reference exposure time, counts the number of overexposed and underexposed pixels, and determines the exposure adjustment direction and step size when the overexposure ratio exceeds the first threshold and the underexposure ratio exceeds the second threshold. Exposure sequence generation unit: Using the reference exposure time as the intermediate value, it generates several short exposure times and long exposure times in the decreasing and increasing directions according to the direction and step size, and arranges them in ascending order to form a multi-level exposure sequence; Multi-frame acquisition unit: connected to the camera trigger, triggers exposure sequentially and reads each frame and its exposure time; High Dynamic Range (HDR) Composition Unit: Divide the grayscale value of each frame by the exposure time to obtain the normalized irradiance. Select the irradiance that falls within the camera's linear response range, perform a weighted average, and write it to the corresponding position in the HDR image. The weight is inversely proportional to the frame exposure time.

5. The online visual inspection system for detecting surface defects of a camera mount according to claim 1, characterized in that, The image acquisition and preprocessing module specifically includes: Pose parameter receiving unit: obtains pitch angle and roll angle from attitude compensation and surface unfolding module, calculates rotation angle around image center point based on pitch angle and roll angle, and calculates translation vector based on displacement of the hanger in the direction of conveyor belt; The inverse affine transformation unit constructs a rotation matrix with the rotation angle and a translation matrix with the translation vector, and combines the two into an inverse affine transformation matrix. This inverse matrix is ​​applied to the coordinates of each pixel in the high dynamic range image to obtain the resampling coordinates. The gray value at the resampling coordinates is obtained by bilinear interpolation and filled into the corresponding position in the standard pose image. The illumination correction unit performs Gaussian filtering on the standard pose image to extract the background illumination component. It then divides the gray value of each pixel in the standard pose image by the gray value of the corresponding background illumination component and multiplies it by the gray value of the standard illumination component, outputting the image to be detected after eliminating non-uniform illumination.

6. The online visual inspection system for detecting surface defects of a camera mount according to claim 5, characterized in that, The illumination correction unit specifically includes: Adaptive filtering subunit: Gaussian filtering is performed on the standard pose image at three scales (large, medium, and small) to obtain the background component at the corresponding scale. The first difference between the large-scale and medium-scale background and the second difference between the medium-scale and small-scale background at each pixel are calculated. The background component at the corresponding scale is adaptively selected as the final background illumination component of the pixel based on the relative magnitude of the first and second differences. Boundary Preservation Subunit: Performs anisotropic diffusion filtering on the final background illumination component, sets the diffusion coefficient along the image gradient direction, reduces the diffusion coefficient in areas with large gradient magnitude to preserve the edges of abrupt illumination changes, increases the diffusion coefficient in areas with small gradient magnitude to smooth the gradual illumination, and outputs the smoothed background illumination component after boundary preservation.

7. The online visual inspection system for detecting surface defects of a camera mount according to claim 1, characterized in that, The defect identification and classification module specifically includes: Template matching unit: Stores template images of defect-free hangers in standard poses, rigidly registers the image to be detected with the template, calculates subpixel translation, and then performs translation correction; Differential extraction unit: Subtracts the grayscale of the corrected image from the template pixel by pixel to obtain the difference image, marks the pixels whose absolute difference value exceeds the noise threshold as suspicious pixels and performs eight-neighbor connectivity clustering, merges the interconnected suspicious pixels into candidate defect regions and records their bounding rectangles; Texture analysis unit: Calculates gray-level co-occurrence matrices in four directions (horizontal, vertical, diagonal, and anti-diagonal) for each candidate region, extracts contrast and correlation parameters, and judges the degree of anisotropy by the range of contrast in the four directions. Classification output unit: If the anisotropy exceeds the threshold and the aspect ratio exceeds the threshold, it is judged as a scratch; if the anisotropy is below the threshold and the regional mean is below the corresponding mean of the template, it is judged as a pit; if the anisotropy is centered and the correlation in all four directions is below the threshold, it is judged as a foreign object attachment. Output the label of each defect type and the coordinates of the four corners of the rectangle.

8. The online visual inspection system for detecting surface defects of a camera mount according to claim 1, characterized in that, The defect identification and classification module also includes: Multi-scale detection unit: Construct a Gaussian pyramid for the image to be detected and perform differential extraction and texture analysis operations at the top, middle and bottom layers of the pyramid respectively to obtain candidate defect regions and anisotropy at each scale. The candidate regions with overlapping positions at each scale are merged by weighting the inverse of the scale, with the top layer having the highest weight and the bottom layer having the lowest weight. After merging, the bounding rectangle and the average anisotropy are recalculated. The spurious defect suppression unit expands a ring-shaped neighborhood of a predetermined width around the merged candidate defect region, calculates the standard deviation of the pixel grayscale values ​​within the ring-shaped neighborhood, and determines that the region is a spurious defect and removes it from the output when the ratio of the grayscale standard deviation inside the candidate region to the grayscale standard deviation of the ring-shaped neighborhood is less than a preset threshold. Candidate regions with a ratio greater than the threshold are retained as valid defects in the output.

9. The online visual inspection system for detecting surface defects of a camera mount according to claim 1, characterized in that, The attitude compensation and surface unfolding module and the exposure control and high dynamic range fusion module are provided with a feedback coordination unit, which specifically includes: The feedback coordination unit obtains the statistical value of the reflectivity dynamic range from the exposure control and high dynamic range fusion module. When the statistical value exceeds the preset range threshold, it determines that the local reflection on the surface of the bracket is too strong, causing the structured light stripes to be indistinguishable. It then sends a stripe gain adjustment signal to the attitude compensation and surface unfolding module. The module increases the laser output power of the structured light projector according to the signal and simultaneously reduces the camera acquisition gain. It then reprojects the encrypted stripe grating and acquires a new distorted stripe image for 3D point cloud reconstruction. Simultaneously, the feedback coordination unit sends the dynamic range statistical value of reflectivity to the defect identification and classification module. Based on this statistical value, it dynamically adjusts the noise threshold during differential extraction, so that the noise threshold corresponding to the high reflectivity area is higher than the noise threshold corresponding to the low reflectivity area.

Images