Enhancement algorithm for structured light synthesis imaging

Abstract

The invention discloses an enhancement algorithm for structured light synthesis imaging, which comprises the following steps of: 1, acquiring a structured light synthesis image which comprises a photometric stereo image or a phase deflection image; and secondly, image enhancement processing is applied to the structured light synthesis image so as to enhance the defect contrast and suppress noise, and the image enhancement processing comprises at least one of four algorithms including a nonlinear stretching algorithm, a normal local difference enhancement algorithm, a contrast and size defect suppression algorithm and a background shielding algorithm. According to the method, through a set of collaborative enhancement algorithm, the key technical bottleneck of structured light synthesis imaging in defect detection application is systematically solved, and finally, the defect detection precision, the robustness and the automation degree are comprehensively improved.

Description

Technical Field

[0001] This invention mainly relates to the technical field of industrial machine vision and optical inspection, specifically to an enhancement algorithm for structured light synthesis imaging. Background Technology

[0002] Currently existing imaging methods in the field of surface defect detection, known as 2.5D, include two methods: photometric stereo and phase deflection.

[0003] According to the shooting method, photometric stereoscopic photography is divided into static shooting with area scan camera and moving shooting with line scan camera; according to the type of lighting, it is divided into direct lighting and indirect lighting.

[0004] A typical type of indirect lighting is the Trevista photometric stereo light source from SAC Corporation, which extends the use of photometric stereo from diffuse surfaces to mirror surfaces.

[0005] On the other hand, phase deflection is also divided into static shooting with area array cameras and moving shooting with line array cameras according to the shooting method, and into traditional screens and high-frequency LED screens according to the light source type.

[0006] The high-frequency LED screen was first seen at Keyence, which successfully extended phase deflection from static shooting to motion shooting, improving efficiency and making high-resolution detection easier.

[0007] When raw synthetic images are used directly for defect detection, the following inherent limitations and challenges exist:

[0008] Low signal-to-noise ratio and insufficient defect contrast: In the original synthetic image, weak defect signals are often obscured by noise generated during the calculation process and the texture of the object itself. This results in low contrast between defects and the background, making them difficult for subsequent detection algorithms to reliably identify.

[0009] Flat areas are difficult to distinguish from defective areas: In normal vector images, even flat normal areas may have slight gradient changes, while small defects with only slight differences in height or normal are not visually distinguishable from their surroundings. Existing methods struggle to significantly enhance real defective areas while maintaining the smoothness of flat areas.

[0010] Confounding defect features interfere with subsequent processing: In the shape image obtained by phase deflection, in addition to the target defect, there are usually a large number of meaningless noise points, micro-scratches, or spurious features caused by material inhomogeneity. These non-target features are mixed with the real defects, increasing the complexity and false positive rate of subsequent identification and classification algorithms.

[0011] Boundary region calculation distortion: For workpieces with complex contours (such as mobile phone glass covers), when performing phase deflection calculations in their edge regions, background pixels will interfere with phase calculations, resulting in unstable noise and distortion in the edge shape map, which seriously affects the accurate judgment of defects in the boundary region.

[0012] Therefore, this invention systematically solves the key technical bottlenecks faced by structured light synthesis imaging in defect detection applications through a set of collaborative enhancement algorithms, and ultimately achieves a comprehensive improvement in defect detection accuracy, robustness and automation. Summary of the Invention

[0013] Therefore, the purpose of this invention is to provide an enhancement algorithm for structured light synthesis imaging to solve the technical problems mentioned in the background art.

[0014] To achieve the above objectives, the present invention provides the following technical solution:

[0015] The enhancement algorithm for structured light synthesis imaging includes the following implementation steps:

[0016] Step 1: Obtain a structured light composite image, which includes a photometric stereo image or a phase-deflection image;

[0017] Step 2: Apply image enhancement processing to the structured light synthesized image to enhance defect contrast and suppress noise, wherein the image enhancement processing includes at least one of the following algorithms:

[0018] (1) Nonlinear stretching algorithm;

[0019] (2) Algorithm for enhancing local differences in the normal direction;

[0020] (3) Contrast and size suppression defect algorithm;

[0021] (4) Background masking algorithm.

[0022] In a preferred embodiment of this technical solution, the nonlinear stretching algorithm includes:

[0023] The image pixel values ​​are nonlinearly transformed according to a mapping function, which is based on a sigmoid function, and the mapping function is as follows:

[0024]

[0025] Where X is the input pixel value, Y is the intermediate output value, and K is the stretching coefficient;

[0026] Calculate the scaling factor:

[0027]

[0028] Calculate the final output value:

[0029]

[0030] And based on the background area width set by the user Modify the mapping curve to make it so that in the interval The pixel values ​​within remain unchanged.

[0031] In a preferred embodiment of this technical solution, the normal local difference enhancement algorithm includes:

[0032] For each pixel in the normal vector image The difference between the value and the mean pixel value within the local window is calculated, and enhancement is performed based on this difference, where the enhanced pixel value... The calculation is as follows:

[0033]

[0034] in, For the window radius, The tensile coefficient is denoted as .

[0035] In a preferred embodiment of this technical solution, the normal local difference enhancement algorithm further includes an enhancement method that incorporates local variance, wherein the enhanced pixel value The calculation is as follows:

[0036]

[0037] in, It is a local mean. This is the reference variance.

[0038] In a preferred embodiment of this technical solution, the contrast and size suppression defect algorithm includes:

[0039] The shape image of the phase deflection is binarized to obtain a binary image;

[0040] Connectivity analysis was performed on the binary image to obtain multiple patches;

[0041] Screening is performed based on at least one feature of the patch, including area, horizontal span, vertical span, maximum inner circle radius, circularity, or principal axis of inertia.

[0042] Patch areas that do not meet the filtering criteria are set as background values.

[0043] In this preferred embodiment, the roundness C is defined as:

[0044]

[0045] in, The area of ​​the patch. This represents the perimeter of the plaque.

[0046] In a preferred embodiment of this technical solution, the principal axis direction θ is calculated as follows:

[0047]

[0048] in, and They are respectively along the region direction, direction and The second-order center distance in the direction.

[0049] In a preferred embodiment of this technical solution, the background masking algorithm includes:

[0050] Background pixels are segmented using a user-specified segmentation threshold to generate a background marker map, where background pixels are marked as 1 and object pixels are marked as 0.

[0051] During shape map calculations, for pixels marked as background, the calculation terms they participate in are set to zero.

[0052] In summary, the present invention has the following main beneficial effects:

[0053] Firstly, the nonlinear stretching algorithm in this invention, through a designed S-shaped mapping function, performs a nonlinear transformation on pixel values, which can significantly amplify the grayscale difference between defective areas and normal background areas. For example... Figure 2 As shown, the contrast of the white scratches and their shadows is greatly enhanced after processing. Simultaneously, by setting the background area width W, it is ensured that normal background grayscale areas remain unaffected, effectively suppressing the amplification of background noise and thus achieving an extremely high defect signal-to-noise ratio.

[0054] Secondly, the normal local difference enhancement algorithm in this invention, based on local statistical characteristics (mean and variance), can intelligently identify and enhance pixels that differ significantly from their surroundings. For flat areas, the pixel value differs little from the local mean, resulting in weak enhancement or even zero enhancement; for real defects, the pixel value differs greatly from the local mean and will be significantly enhanced. For example... Figure 3 As shown, characters and minor bumps are clearly highlighted, while flat areas remain smooth. Introducing variance adjustment further suppresses over-enhancement of noise areas, making defect enhancement more targeted.

[0055] Thirdly, the contrast and size suppression defect algorithm in this invention achieves intelligent defect filtering through binarization, connected component analysis, and multi-feature (area, span, roundness, orientation, etc.) screening. This method can accurately remove noise points that are too small in area, irregular in shape, and insignificant minor flaws, while retaining and highlighting large defects that meet the target characteristics; for example... Figure 7 As shown, the processed image has a clean background and retains only the main defect textures, which greatly simplifies the subsequent defect identification and classification process and improves detection efficiency and accuracy.

[0056] Fourth, the background masking algorithm in this invention actively excludes interfering background pixels during the calculation stage through background segmentation and marking techniques; this fundamentally solves the problem of computational noise and distortion caused by background interference at object edges. For example... Figure 8 As shown, the shape of the product edge becomes clear, stable and reliable after processing, making it possible to accurately detect defects in the edge area and expanding the application boundaries of phase deflection. Attached Figure Description

[0057] Figure 1 A schematic diagram of the mapping function curve generated by the nonlinear enhancement formula of the present invention;

[0058] Figure 2 This is a schematic diagram comparing the effects of nonlinear enhancement before and after the present invention;

[0059] Figure 3 This is a schematic diagram comparing the effects of normal enhancement before and after the present invention;

[0060] Figure 4 This is a diagram showing the patch characteristics and horizontal span of the present invention.

[0061] Figure 5 This is a diagram showing the patch characteristics and vertical span of the present invention.

[0062] Figure 6 This is a diagram showing the patch characteristics of the present invention – the maximum inner circle radius;

[0063] Figure 7 This is a schematic diagram showing the before-and-after effects of the present invention on contrast and size suppression defects;

[0064] Figure 8 This is a schematic diagram comparing the effects of background segmentation and marking techniques before and after processing according to the present invention; Detailed Implementation

[0065] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0066] The following describes the overall technical solution and corresponding appendices according to the present invention. Figure 1-8 The illustrated content described herein is used to illustrate embodiments of the present invention.

[0067] This application provides an enhancement algorithm for structured light synthesis imaging, including the following implementation steps:

[0068] Step 1: Obtain a structured light composite image, which includes a photometric stereo image or a phase-deflection image;

[0069] Step 2: Apply image enhancement processing to the structured light synthesized image to enhance defect contrast and suppress noise, wherein the image enhancement processing includes at least one of the following algorithms:

[0070] (1) Nonlinear stretching algorithm;

[0071] (2) Algorithm for enhancing local differences in the normal direction;

[0072] (3) Contrast and size suppression defect algorithm;

[0073] (4) Background masking algorithm.

[0074] In this embodiment, the nonlinear stretching algorithm includes:

[0075] The image pixel values ​​are nonlinearly transformed according to a mapping function, which is based on a sigmoid function, and the mapping function is as follows:

[0076]

[0077] Where X is the input pixel value, Y is the intermediate output value, and K is the stretching coefficient;

[0078] Calculate the scaling factor:

[0079]

[0080] Calculate the final output value:

[0081]

[0082] And based on the background area width set by the user Modify the mapping curve to make it so that in the interval The pixel values ​​within remain unchanged.

[0083] In this embodiment, the normal local difference enhancement algorithm includes:

[0084] For each pixel in the normal vector image The difference between the value and the mean pixel value within the local window is calculated, and enhancement is performed based on this difference, where the enhanced pixel value... The calculation is as follows:

[0085]

[0086] in, For the window radius, The tensile coefficient is denoted as .

[0087] The normal local difference enhancement algorithm also includes an enhancement method that incorporates local variance, wherein the enhanced pixel value The calculation is as follows:

[0088]

[0089] in, It is a local mean. This is the reference variance.

[0090] In this embodiment, the contrast and size suppression defect algorithm includes:

[0091] The shape image of the phase deflection is binarized to obtain a binary image;

[0092] Connectivity analysis was performed on the binary image to obtain multiple patches;

[0093] Screening is performed based on at least one feature of the patch, including area, horizontal span, vertical span, maximum inner circle radius, circularity, or principal axis of inertia.

[0094] Patch areas that do not meet the filtering criteria are set as background values.

[0095] The circularity C is defined as:

[0096]

[0097] in, The area of ​​the patch. This represents the perimeter of the plaque.

[0098] The principal axis of inertia direction θ is calculated as follows:

[0099]

[0100] in, and They are respectively along the region direction, direction and The second-order center distance in the direction.

[0101] In this embodiment, the background masking algorithm includes:

[0102] Background pixels are segmented using a user-specified segmentation threshold to generate a background marker map, where background pixels are marked as 1 and object pixels are marked as 0.

[0103] During shape map calculations, for pixels marked as background, the calculation terms they participate in are set to zero.

[0104] Example 1: Application of nonlinear stretching algorithm alone

[0105] This embodiment demonstrates a scenario where an image optimization is performed using a nonlinear stretching algorithm alone on existing imaging results.

[0106] 1. Image Acquisition:

[0107] Read a set of existing workpiece normal vector maps with low signal-to-noise ratio from the database.

[0108] 2. Image enhancement processing:

[0109] Enable only the nonlinear stretching algorithm. Parameter settings are as follows: (Using stronger tension) (Narrow the background protection area and carry out more aggressive enhancements).

[0110] The system iterates through each pixel of the image, strictly following the three formulas and background protection intervals defined in the nonlinear stretching algorithm. The three formulas are as follows:

[0111]

[0112]

[0113]

[0114] like Figure 2 As shown, after processing, the originally dull and unclear scratches are significantly enhanced in both bright and dark areas, becoming visible to the naked eye, while most of the background area remains unchanged.

[0115] 3. Output Results:

[0116] The output enhanced image can be used for manual review or as input for other analysis programs.

[0117] Example 2: Application of a variance-based algorithm for enhancing local differences in normal vectors

[0118] This embodiment highlights the advantages of advanced enhancement methods on workpieces with complex textures.

[0119] 1. Image Acquisition:

[0120] Obtain the normal vector map of a metal workpiece surface with a fine frosted texture.

[0121] 2. Image enhancement processing:

[0122] Apply the local difference enhancement algorithm based on normal direction. Parameters: .

[0123] The core advantage of this algorithm lies in its ability to perceive the complexity of local textures. In areas where the matte texture itself has little undulation (small local variance), the algorithm will significantly enhance small raised pits and defects; while in areas where the texture itself is complex (large local variance), the algorithm will automatically reduce the enhancement level to avoid mistakenly enhancing normal textures as defects.

[0124] This process effectively solves the problems of "difficulty in distinguishing between flat areas and defect areas" and "mixed defect features" in the background technology.

[0125] 3. Output Results:

[0126] Obtaining a clear image where defects are highlighted without excessive magnification of background texture facilitates subsequent thresholding and defect extraction.

[0127] Example 3: Detection process using multiple enhancement algorithms

[0128] This embodiment provides a complete process example for detecting defects on the surface of mobile phone glass covers, which comprehensively applies the various enhancement algorithms described above.

[0129] 1. Image Acquisition:

[0130] A line scan camera imaging system equipped with a high-frequency LED screen phase deflection light source and a Trevista photometric stereo light source is used to scan the glass cover of a mobile phone on a conveyor belt.

[0131] The system's built-in software algorithm calculates the phase deflection shape map and photometric stereo normal vector map of the workpiece. These images are the input of this invention—the "structured light synthesis image".

[0132] 2. Image enhancement processing:

[0133] First, a background masking algorithm (corresponding to claim 8) is applied to the phase deflection shape map.

[0134] The operator sets a segmentation threshold based on the image histogram. All grayscale values ​​below The pixels that are considered background are marked as 1 in the generated background marker map; the pixels of the object are marked as 0.

[0135] In all subsequent processing steps for this shape image, when a pixel is accessed, its background marker is checked first. If it is a background pixel (marked as 1), the pixel is skipped or its contribution to the calculation is set to zero. This step effectively eliminates the interference of the background on the calculation of the workpiece edge; see [link to processing results] for details. Figure 8 .

[0136] Secondly, a contrast and size suppression defect algorithm is applied to the shape image after background masking.

[0137] The Otsu adaptive thresholding method is used to binarize the image, separating the potential defective region (foreground) from the normal region (background).

[0138] An 8-connected component analysis was performed on the binary image to identify all independent patches (i.e., candidate defect regions).

[0139] Set filter criteria: patch area Pixels, circularity Horizontal span Pixels. Using the formula: as well as The area, circularity, and principal axis of inertia of each patch are calculated.

[0140] For any patch region that does not meet any of the above conditions, its pixel value will be forcibly set to 0 (background value). For example... Figure 7 As shown, this operation filters out a large amount of noise and minor flaws, retaining only the significant target defects.

[0141] Simultaneously, a local difference enhancement algorithm for the normal vector map is applied to the photometric stereo normal vector map.

[0142] The Z-component of the normal vector image is selected for processing. The local window radius is set. elongation coefficient .

[0143] For each pixel in the image Calculate its Pixel mean in the neighborhood .

[0144] First, we use the basic formula:

[0145] Enhance it.

[0146] For further optimization, the variance adaptive formula in claim 4 can be used.

[0147] Set reference variance =10.0. Calculate the variance within this neighborhood. The enhancement formula then becomes: This method provides significant enhancement in flat areas (low variance), while offering a smoother enhancement in areas with complex textures or noise (high variance). See the processing results for details. Figure 3 .

[0148] Finally, a nonlinear stretching algorithm is applied to all enhanced images to uniformly optimize the display effect.

[0149] Set the elongation coefficient Background area width .

[0150] For each pixel value in the image (Within the range of 0-255), calculate sequentially using the following formulas:

[0151]

[0152]

[0153]

[0154] examine The value, if The final output will remain the original value. Otherwise, the output is This step ensures that the background grayscale areas remain unchanged, while defect areas that deviate from the background are significantly stretched; see the mapping curve for details. Figure 1 For a comparison of the effects, please refer to [link / reference]. Figure 2 .

[0155] 3. Output Results:

[0156] After the above series of enhancement processes, the images exhibit extremely high contrast between defects and background, and noise is effectively suppressed. These images can be directly input into standard defect classification and recognition software (such as deep learning-based detection models or traditional Blob analysis tools) for final defect determination and classification, greatly improving the accuracy and robustness of the detection system.

[0157] It should be noted that:

[0158] The specific parameters mentioned in the above embodiments, such as threshold T=20, window radius w=5, and coefficient K=0.05, are illustrative and not intended to limit the invention. Those skilled in the art can adaptively adjust these parameters according to the specific characteristics of the workpiece under test (such as material, reflectivity, and defect type) and the specific conditions of the imaging system.

[0159] The various algorithms of this invention can work together as shown in Example 3 to form a complete processing pipeline, or they can be used individually or selectively in combination according to actual needs, as shown in Examples 1 and 2.

[0160] The method of this invention can be implemented through software programming and integrated into existing machine vision inspection software, or it can be embedded into instructions for dedicated image processing hardware (such as FPGA) and applied to online inspection equipment.

[0161] Although embodiments of the present invention have been shown and described, these specific embodiments are merely explanations of the invention and are not intended to limit it. The specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. After reading this specification, those skilled in the art may make modifications, substitutions, and variations to the embodiments as needed without departing from the principles and spirit of the invention, but such modifications, substitutions, and variations are protected by patent law as long as they are within the scope of the claims of the present invention.

Claims

1. An enhancement algorithm for structured light synthesis imaging, characterized in that, The implementation steps include the following: Step 1: Obtain a structured light composite image, which includes a photometric stereo image or a phase-deflection image; Step 2: Apply image enhancement processing to the structured light synthesized image to enhance defect contrast and suppress noise, wherein the image enhancement processing includes at least one of the following algorithms: (1) Nonlinear stretching algorithm; (2) Algorithm for enhancing local differences in the normal direction; (3) Contrast and size suppression defect algorithm; (4) Background masking algorithm.

2. The enhancement algorithm for structured light synthesis imaging according to claim 1, characterized in that, The nonlinear stretching algorithm includes: The image pixel values ​​are nonlinearly transformed according to a mapping function, which is based on a sigmoid function, and the mapping function is as follows: Where X is the input pixel value, Y is the intermediate output value, and K is the stretching coefficient; Calculate the scaling factor: Calculate the final output value: And based on the background area width set by the user Modify the mapping curve to make it so that in the interval The pixel values ​​within remain unchanged.

3. The enhancement algorithm for structured light synthesis imaging according to claim 1, characterized in that, The normal local difference enhancement algorithm includes: For each pixel in the normal vector image The difference between the value and the mean pixel value within the local window is calculated, and enhancement is performed based on this difference, where the enhanced pixel value... The calculation is as follows: in, For the window radius, The tensile coefficient is denoted as .

4. The enhancement algorithm for structured light synthesis imaging according to claim 3, characterized in that, The normal local difference enhancement algorithm also includes an enhancement method that incorporates local variance, wherein the enhanced pixel value The calculation is as follows: in, It is a local mean. This is the reference variance.

5. The enhancement algorithm for structured light synthesis imaging according to claim 1, characterized in that, The contrast and size suppression defect algorithm includes: The shape image of the phase deflection is binarized to obtain a binary image; Connectivity analysis was performed on the binary image to obtain multiple patches; Screening is performed based on at least one feature of the patch, including area, horizontal span, vertical span, maximum inner circle radius, circularity, or principal axis of inertia. Patch areas that do not meet the filtering criteria are set as background values.

6. The enhancement algorithm for structured light synthesis imaging according to claim 5, characterized in that, The circularity C is defined as: in, The area of ​​the patch. This represents the perimeter of the plaque.

7. The enhancement algorithm for structured light synthesis imaging according to claim 5, characterized in that, The principal axis of inertia direction θ is calculated as follows: in, and They are respectively along the region direction, direction and The second-order center distance in the direction.

8. The enhancement algorithm for structured light synthesis imaging according to claim 1, characterized in that, The background masking algorithm includes: Background pixels are segmented using a user-specified segmentation threshold to generate a background marker map, where background pixels are marked as 1 and object pixels are marked as 0. During shape map calculations, for pixels marked as background, the calculation terms they participate in are set to zero.

Images