An ultra-deep field image acquisition system and method for defect detection

Abstract

The present application relates to the technical field of image acquisition, in particular to a super-DOF image acquisition system and method for defect detection, which comprises a light source module, a lens, a prism, a camera and a displacement driving assembly, the displacement driving assembly is fixedly connected with a vertical mounting plate, the light source module, the lens, the prism and the camera are sequentially docked from bottom to top, and are fixedly installed with the surface of the vertical mounting plate as an installation reference surface, the optical axes of the light source module, the lens, the prism and the camera are collinear, the light source module is composed of LED lamps with different heights, different colors and different angles, and the camera and the light source module are connected with an external host computer. The method comprises focus stacking, image definition evaluation, image alignment and fusion optimization. The present application can overcome the limitations of fixed-focus cameras, obtain clear full-view images of devices with height differences, better meet the image acquisition requirements in defect detection, and further promote the progress of defect detection technology.

Description

Technical Field

[0001] This invention relates to the field of image acquisition technology, and specifically to a super-depth-of-field image acquisition system and method for defect detection. Background Technology

[0002] Existing defect detection methods use fixed-focus cameras to acquire images. However, fixed-focus cameras achieve optimal clarity only at specific working distances. In defect detection scenarios, when the surface of the object being inspected has defects such as dents, most of these defects exhibit height variations. This is particularly true for lead wire defect detection, where the lead wire has continuously varying arc heights, making it impossible for a fixed-focus camera to capture a clear image of the entire lead wire area. Furthermore, when the object experiences slight fluctuations during transmission, some areas may deviate from the depth of field, resulting in image blurring, which in turn reduces image quality and significantly increases the risk of missed detections or false positives.

[0003] To ensure clear imaging of the entire area under test, it is necessary to focus and take pictures separately for areas at different heights, and repeatedly adjust the camera position, which significantly reduces the detection efficiency. Therefore, how to design a super-depth-of-field image acquisition system for defect detection to solve problems such as image acquisition under multi-angle and multi-color light source environments and multi-focal length image fusion has become a technical challenge that urgently needs to be solved in this field. Summary of the Invention

[0004] To overcome the technical shortcomings of fixed-focus cameras, which result in unclear image acquisition for defect detection due to physical depth-of-field limitations, this invention provides a super-depth-of-field image acquisition system and method for defect detection.

[0005] This invention provides a super depth-of-field image acquisition system for defect detection, including a light source module, a lens, a prism, a camera, and a displacement driving assembly. A vertical mounting plate is fixedly connected to the displacement driving assembly. The light source module, lens, prism, and camera are connected sequentially from bottom to top and fixedly installed with the surface of the vertical mounting plate as the mounting reference surface. The optical axes of the light source module, lens, prism, and camera are collinear. The light source module consists of LEDs of different heights, colors, and angles. The camera and light source module are respectively connected to an external host computer.

[0006] Preferably, the light source module also includes a lampshade for mounting LED lights. The closed end of the lampshade is located at the top for docking with the lens, and the flared end of the lampshade is located at the bottom. The lampshade gradually expands outward from top to bottom into a cone shape. The inner wall of the lampshade is provided with multiple levels of annular steps from top to bottom, and the LED lights are evenly arranged circumferentially along the steps of the annular steps.

[0007] Preferably, the LED lights on the same level of the circular staircase are of the same color.

[0008] Preferably, the inner wall of the lampshade has six annular steps from top to bottom. White LEDs, red LEDs, white LEDs, green LEDs, white LEDs, and a white-blue mixed LED are respectively installed on the first to sixth annular steps. The green LED on the fourth annular step forms a 45° angle with the system's optical axis, the white LED on the fifth annular step forms a 45° angle with the system's optical axis, and the white-blue mixed LED on the sixth annular step forms a 60° angle with the system's optical axis. The optical axes of the LEDs on the remaining annular steps are parallel to the system's optical axis. In defect detection, red light is mainly used to observe defects such as cracks and dents on the surface of the inspected part. Therefore, when irradiated perpendicular to the inspected surface, the red light's optical axis is parallel to the system's optical axis, and white light at the same angle is used to supplement the brightness. Green light is primarily used to observe residues on the silicon substrate. These residues have a certain height, so a 45° tilt angle is chosen, supplemented with white light at the same angle for brightness. Blue light is used to observe minute surface defects, therefore it is positioned on the sixth-level annular step, illuminated at a 60° tilt angle, and supplemented with white light at the same angle for brightness. The white-blue hybrid LED light consists of white and blue LEDs, arranged in four layers along the circumference, from top to bottom: white LED, blue LED, white LED, and blue LED.

[0009] Preferably, the displacement drive assembly includes a horizontal displacement unit and a vertical displacement unit, with the vertical displacement unit fixedly connected to the movable end of the horizontal displacement unit, and the vertical mounting plate fixedly connected to the movable end of the vertical displacement unit.

[0010] This invention also provides a method for acquiring ultra-depth-of-field images for defect detection, which is based on the ultra-depth-of-field image acquisition system for defect detection described in this invention, and its steps are as follows:

[0011] S1. Focus Stack Acquisition: The host computer triggers a signal to control the camera to expose sequentially at different focus positions, thereby obtaining a set of images focused at different depth of field positions;

[0012] S2. Image sharpness evaluation: The sharpness response value of each input image is calculated using the Laplacian variance method;

[0013] S3. Image Alignment: Images with different focal points are aligned using the SIFT method to obtain registered images;

[0014] S4. Constructing a sharpness weight pyramid: Construct a Gaussian pyramid and a Laplacian pyramid for the registered image. Calculate and normalize the local sharpness metric based on the bottom layer of the Laplacian pyramid to generate a fusion weight pyramid.

[0015] S5. Pyramid fusion reconstruction: The coefficients of each layer of the Laplacian pyramid are weighted and fused using a fusion weighted pyramid, and the final super-depth image is obtained by reconstructing layer by layer upwards.

[0016] Preferably, in S2, the calculation steps of the Laplace variance method are as follows:

[0017] S21. Employ an eight-neighbor Laplacian convolution kernel. Its form is:

[0018] ,

[0019] S22, for dimensions of The original image Perform zero-fill at the boundaries, where Image height, The width of the image after padding for;

[0020] ,

[0021] S23, Apply the eight-neighbor Laplacian convolution kernel With the image of Hou Each pixel position Perform discrete convolution to calculate the position of each pixel. response value ;

[0022] ,

[0023] in, ;

[0024] S24, Response Value To perform statistical analysis, the global mean of the response map for all pixels is first calculated. :

[0025] ,

[0026] Then calculate its variance:

[0027] ,

[0028] As a metric for image sharpness, The larger the value, the richer the edge details of the image and the higher the clarity; conversely, the smaller the value, the blurrier the image.

[0029] Preferably, the sub-step of S3 is as follows:

[0030] S31. For the original input image Constructing a Gaussian pyramid, generating a scale-space representation using Gaussian kernels of different scales:

[0031] ,

[0032] in, , The Gaussian blur radius;

[0033] Further construct the Gaussian difference pyramid:

[0034] ,

[0035] Where k takes ;

[0036] Then, the extremum conditions in the scale space are solved:

[0037] ,

[0038] Find The local extreme points are the key points;

[0039] S32. For each key point, statistical gradient direction histogram is generated in its neighborhood, and a 128-dimensional feature vector is constructed as the descriptor of the point.

[0040] For the key points of image one Its descriptor is Then the set of key points in image one is represented as:

[0041] , This represents the total number of keypoints in Image 1.

[0042] Similarly, the set of key points in image two is represented as follows:

[0043] ; This represents the total number of keypoints in Image 1.

[0044] S33. Employ the nearest neighbor search matching strategy for each descriptor in Image 1. In the descriptor set of image two Search within.

[0045] The nearest neighbor is: ;

[0046] The next nearest neighbor is: ;

[0047] in Use the Euclidean distance function;

[0048] When the following ratio condition is met, then and Pairing, ratio condition:

[0049] ;

[0050] S34. Obtain the initial set of matching points. Then, the transformation matrix is ​​iteratively optimized using the RANSAC (Random Sample Consensus) algorithm:

[0051] S341, from Minimum subset of random sampling , the smallest subset It contains m pairs of matching points, where m depends on the degrees of freedom of the transformation model;

[0052] S342, Based on the minimum subset Estimating candidate transformation matrix ;

[0053] S343. Calculate the transformation matrix The set of interior points below ,

[0054] ,

[0055] in, Represented in homogeneous coordinates, For reprojection error, The preset error threshold is used;

[0056] S344, If the set of interior points If the cardinality of the set of interior points is greater than the cardinality of the current optimal interior point set, then update the optimal transformation matrix. = And record the optimal set of interior points. = ;

[0057] Repeat the iterative process from S341 to S344 until convergence, and output the final transformation matrix. Using the final transformation matrix Align the images with different focal points to obtain the registered image. .

[0058] Preferably, the sub-step of S4 is as follows:

[0059] S41. Construct the Gaussian pyramid;

[0060] For the registered images Construct a Gaussian pyramid, with the registered image forming the base layer. Then the coefficient of the bottom layer of the Gaussian pyramid ,

[0061] For the upper level The first image is obtained by downsampling after applying Gaussian blur to the lower layer image. Gaussian pyramid coefficient of layers :

[0062] ,

[0063] in, It is a Gaussian low-pass filter. To employ interlaced downsampling, the image size is halved layer by layer, and a Gaussian pyramid is used to capture multi-scale low-frequency information of the image;

[0064] S42. Construct the Pyramid of Laplace;

[0065] Based on the Gaussian pyramid, the Laplace pyramid is constructed to extract bandpass information at various scales. For the top layer of the Laplace pyramid... The top coefficient of the Plas pyramid ;

[0066] For the remaining layers The first image is obtained by upsampling the upper layer image. The Plas pyramid coefficient of the layer :

[0067] ;

[0068] in, For upsampling operations, it is necessary to... Interpolation magnification to Same size, the first The Plas pyramid coefficient of the layer The corresponding image's edge and detail information at different scales;

[0069] S43. Calculate local sharpness;

[0070] At the base of the Pyramid of Laplace That is, the highest resolution layer, for each pixel location Calculate the local variance within its neighborhood as a measure of the sharpness of that point:

[0071] ,

[0072] in, The local window radius, This represents the average value of the pixels within the local window.

[0073] S44. Normalize and generate a weighted pyramid;

[0074] For each pixel position The image sharpness measure at that point is normalized to obtain the underlying fusion weights:

[0075] .

[0076] Preferably, the sub-step of S5 is as follows:

[0077] S51, For each level of the pyramid Using each input image in the first Layer fusion weights Weighted fusion of the Laplace pyramid coefficients:

[0078] ,

[0079] in, The total number of input images, For the first The image in the first Layer fusion weights, For the first The image in the first The Laplace pyramid coefficient of the layer;

[0080] S52, Top-level initialization and reconstruction image, due to the top-level... No higher layers are upsampled, therefore the top layer reconstructs the image. Take the fusion Laplacian coefficient of this layer:

[0081] ,

[0082] S53, Regarding The iterative reconstruction operation is performed sequentially, the first... Layer Reconstructed Image for:

[0083] ,

[0084] S54. When the iteration is complete and the bottom layer is reached... At this point, the final fused image is obtained:

[0085] ,

[0086] The spatial resolution is consistent with the original input image. Each pixel is formed by fusing the content of the corresponding position with the highest clarity from multiple source images. Finally, F is the super depth-of-field image after the multifocal image has been processed to enhance the full depth of field.

[0087] Compared with existing technologies, the technical solution provided by this invention has the following technical advantages: The ultra-depth-of-field image acquisition system provided by this invention adopts multi-color, multi-angle, and multi-level ring light sources. Through combined illumination methods, it can highlight the height and material differences on the surface of the inspected part, thereby providing a suitable imaging environment for defects of various materials and types. This effectively solves technical challenges such as multi-angle, multi-color light source configuration, multi-focal length image acquisition, and ultra-depth-of-field image fusion. The ultra-depth-of-field image acquisition method provided by this invention uses Laplacian pyramids and weighted pyramids to fuse and reconstruct images, thereby obtaining ultra-depth-of-field images of the inspected area. This overcomes the depth-of-field limitations of fixed-focus cameras, obtaining clear, full-view images of devices with height differences, better meeting the image acquisition needs in defect detection, and further promoting the development of defect detection technology. Attached Figure Description

[0088] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0089] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0090] Figure 1 This is a schematic diagram of the structure of a super-depth-of-field image acquisition system for defect detection according to a certain embodiment of the present invention;

[0091] Figure 2 This is a cross-sectional view of the light source module described in a certain embodiment of the present invention;

[0092] Figure 3 This is a top view of the light source module described in a certain embodiment of the present invention.

[0093] In the diagram: 1. Light source module; 2. Lens; 3. Prism; 4. Camera; 5. Vertical mounting plate; 6. Lampshade; 7. Retracted end; 8. Flared end; 9. Circular step; 10. Horizontal displacement unit; 11. Vertical displacement unit. Detailed Implementation

[0094] To better understand the above-mentioned objectives, features, and advantages of the present invention, the solutions of the present invention will be further described below. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0095] In this description, it should be noted that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. It should also be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "joint" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0096] Many specific details are set forth in the following description in order to provide a full understanding of the invention, but the invention may also be practiced in other ways different from those described herein; obviously, the embodiments in the specification are only some embodiments of the invention, and not all embodiments.

[0097] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0098] In one embodiment, such as Figure 1 As shown, a super-depth-of-field image acquisition system for defect detection is disclosed, including a light source module 1, a lens 2, a prism 3, a camera 4, and a displacement driving assembly. A vertical mounting plate 5 is fixedly connected to the displacement driving assembly. The light source module 1, lens 2, prism 3, and camera 4 are sequentially connected from bottom to top and fixedly installed using the surface of the vertical mounting plate 5 as the mounting reference surface. The optical axes of the light source module 1, lens 2, prism 3, and camera 4 are collinear. The light source module 1 consists of LEDs of different heights, colors, and angles. The camera 4 and the light source module 1 are respectively connected to an external host computer. The collinearity of the optical axes of the light source module 1, lens 2, prism 3, and camera 4 ensures that the imaging light path is transmitted along the same axis, avoiding imaging distortion or light energy loss caused by optical axis offset. Through multi-dimensional light source configuration, the light source module 1 enables the system to adapt to the illumination needs of different materials and different types of defects, highlighting defect characteristics. Using the surface of the vertical mounting plate 5 as the mounting reference surface ensures that each optical component has a unified assembly reference, reducing installation errors and ensuring the accuracy of collinearity of the optical axes. Camera 4 and light source module 1 are connected to an external host computer to achieve automated and programmable control of the system. The host computer can trigger camera 4 to collect data, control the light source switch and brightness adjustment.

[0099] Based on the above embodiments, in a preferred embodiment, the light source module 1 further includes a lampshade 6 for mounting LED lights. The top of the lampshade 6 has a constricted end 7 for docking with the lens 2, and the bottom has an flared end 8. The lampshade 6 gradually expands outward from top to bottom in a conical shape. The inner wall of the lampshade 6 is provided with multiple levels of annular steps 9 from top to bottom, and the LED lights are evenly arranged circumferentially along the steps of the annular steps 9. The lampshade 6 provides a mechanical mounting carrier for the LED lights and simultaneously constrains the outgoing direction of the light, reducing stray light interference. The constricted end 7 of the lampshade 6 docks with the lens 2, allowing the illumination light path and the imaging light path to converge at the lens entrance, achieving coaxial illumination. The flared end 8 of the lampshade 6 faces the workpiece under test, increasing the illumination coverage area and ensuring uniform illumination of the area to be tested. The conical shape of the lampshade 6 allows LED lights of different heights to illuminate the workpiece at different angles, providing the physical basis for multi-angle illumination. The LED lights are evenly arranged circumferentially along the surface of the annular step 9 to ensure that the inspected part is uniformly illuminated within a 360° circumferential range, thus avoiding directional shadows from affecting the accuracy of defect detection.

[0100] Based on the above embodiments, in a preferred embodiment, the LEDs on the same level of the circular staircase 9 are of the same color. This ensures uniform and consistent lighting color at the same height level, avoids color difference interference caused by multi-color mixing on the same level, and simplifies image post-processing.

[0101] Based on the above embodiments, in a preferred embodiment, the inner wall of the lampshade 6 is provided with six annular steps from top to bottom. White LED lights, red LED lights, white LED lights, green LED lights, white LED lights, white LED lights, and white-blue mixed LED lights are respectively installed on the first to sixth annular steps from top to bottom. The green LED light of the fourth annular step is at a 45° angle to the system optical axis, the white LED light of the fifth annular step is at a 45° angle to the system optical axis, the white-blue mixed LED light of the sixth annular step is at a 60° angle to the system optical axis, and the optical axes of the LED lights on the remaining annular steps are parallel to the system optical axis. In defect detection, red light is mainly used to observe defects such as cracks and dents on the surface of the inspected part. Therefore, it is irradiated perpendicular to the inspected surface, so that the red light axis is parallel to the system's optical axis. Simultaneously, white light at the same angle is used to supplement the brightness, improving the overall image brightness and signal-to-noise ratio by adding white light to the perpendicular red light illumination, avoiding an overly dark image caused by monochromatic red light. Green light is mainly used to observe residues on silicon substrates. These residues have a certain height, so a 45° tilt angle is chosen to create shadows on the sides of the residues with a certain height. The height and volume of the residues can be determined by the shape and size of the shadows. Simultaneously, white light at the same angle is used to supplement the brightness, increasing image brightness while maintaining shadow contrast. Blue light is used to observe fine surface defects. Blue light has a shorter wavelength and stronger scattering ability. Irradiation at a large 60° tilt angle can highlight nanoscale or microscale fine surface defects, improving the detection rate of small defects. Therefore, it is set on the sixth-level annular step, irradiated at a 60° tilt angle, and supplemented with white light at the same angle to avoid image color distortion and insufficient brightness caused by monochromatic blue light illumination. The white-blue hybrid LED light consists of white and blue LEDs, arranged in four layers along the circumference from top to bottom: white LED, blue LED, white LED, and blue LED. This white-blue hybrid provides broad-spectrum illumination, balancing brightness (white light) and detail resolution (blue light). The alternating arrangement of the four layers ensures uniform mixing of the two colors in the circumferential direction, avoiding localized color shifts.

[0102] Based on the above embodiments, in a preferred embodiment, the displacement driving assembly includes a horizontal displacement unit 10 and a vertical displacement unit 11, wherein the vertical displacement unit 11 is fixedly connected to the movable end of the horizontal displacement unit 10, and the vertical mounting plate 5 is fixedly connected to the movable end of the vertical displacement unit 11. The horizontal displacement unit 10 and the vertical displacement unit 11 provide precise displacement control in two orthogonal directions, enabling the system to perform autofocus (vertical direction) and field-of-view switching / scanning (horizontal direction).

[0103] This invention also discloses a method for acquiring ultra-depth-of-field images for defect detection, which is based on the ultra-depth-of-field image acquisition system for defect detection described in this invention, and its steps are as follows:

[0104] S1. Focus Stack Acquisition: Triggered by a signal from the host computer, the camera 4 is controlled to expose sequentially at different focus positions to obtain a set of images focused at different depth of field positions;

[0105] S2. Image sharpness evaluation: The sharpness response value of each input image is calculated using the Laplacian variance method. Due to the different image acquisition methods using different focus stacks and for the sake of computational efficiency, the sharpness response value of different regions is not calculated directly on the entire image. Instead, a local window of size 128×128 pixels is used to evaluate and aggregate the statistical values ​​to calculate the sharpness response value of different regions.

[0106] The calculation steps of the Laplace variance method are as follows:

[0107] S21. Employ an eight-neighbor Laplacian convolution kernel. Its form is:

[0108] ,

[0109] This kernel is isotropic and is used to approximate the sum of the second-order partial derivatives of the image grayscale function, producing a high response to regions of abrupt changes in intensity.

[0110] S22, for dimensions of The original image Perform zero-fill at the boundaries, where Image height, The width of the image after padding for;

[0111] ,

[0112] This ensures that the convolution operation is effective for all original pixels, especially in boundary regions;

[0113] S23, Apply the eight-neighbor Laplacian convolution kernel With the image of Hou Each pixel position Perform discrete convolution to calculate the position of each pixel. response value ;

[0114] ,

[0115] in, This is equivalent to aligning the center of the convolution kernel to the current pixel and performing a neighborhood-weighted summation. This operation uses eight-neighborhood information to calculate the second-order change of image grayscale in space, and the absolute value of the response reflects the local edge intensity.

[0116] S24. To eliminate the impact of overall image brightness shift on subsequent evaluation, the response value... To perform statistical analysis, the global mean of the response map for all pixels is first calculated. :

[0117] ,

[0118] Then calculate its variance:

[0119] ,

[0120] As a metric for image sharpness, The larger the value, the richer the edge details of the image and the higher the clarity; conversely, the smaller the value, the blurrier the image.

[0121] S3. Image Alignment: Align images with different focal points using the SIFT method to obtain the registered image; S31. Process the original input image Constructing a Gaussian pyramid, generating a scale-space representation using Gaussian kernels of different scales:

[0122] ,

[0123] in, , The Gaussian blur radius;

[0124] Further construct the Gaussian difference pyramid:

[0125] ,

[0126] Where k takes ;

[0127] Then, the extremum conditions in the scale space are solved:

[0128] ,

[0129] Find The local extreme points are the key points;

[0130] S32. For each key point, statistical gradient direction histogram is generated in its neighborhood, and a 128-dimensional feature vector is constructed as the descriptor of the point.

[0131] For the key points of image one Its descriptor is Then the set of key points in image one is represented as:

[0132] , This represents the total number of keypoints in Image 1.

[0133] Similarly, the set of key points in image two is represented as follows:

[0134] ; This represents the total number of keypoints in Image 1.

[0135] S33. Employ the nearest neighbor search matching strategy for each descriptor in Image 1. In the descriptor set of image two Search within.

[0136] The nearest neighbor is: ;

[0137] The next nearest neighbor is: ;

[0138] in Use the Euclidean distance function;

[0139] When the following ratio condition is met, then and Pairing, ratio condition:

[0140] ;

[0141] S34. Obtain the initial set of matching points. Then, the transformation matrix is ​​iteratively optimized using the RANSAC (Random Sample Consensus) algorithm:

[0142] S341, from Minimum subset of random sampling , the smallest subset It contains m pairs of matching points, where m depends on the degrees of freedom of the transformation model;

[0143] S342, Based on the minimum subset Estimating candidate transformation matrix ;

[0144] S343. Calculate the transformation matrix The set of interior points below ,

[0145] ,

[0146] in, Represented in homogeneous coordinates, For reprojection error, The preset error threshold is used;

[0147] S344, If the set of interior points If the cardinality of the set of interior points is greater than the cardinality of the current optimal interior point set, then update the optimal transformation matrix. = And record the optimal set of interior points. = ;

[0148] Repeat the iterative process from S341 to S344 until convergence, and output the final transformation matrix. Using the final transformation matrix Align the images with different focal points to obtain the registered image. .

[0149] S4. Constructing a sharpness weight pyramid: Construct a Gaussian pyramid and a Laplacian pyramid for the registered image. Calculate and normalize the local sharpness metric based on the bottom layer of the Laplacian pyramid to generate a fusion weight pyramid.

[0150] S41. Construct the Gaussian pyramid;

[0151] For the registered images Construct a Gaussian pyramid, with the registered image forming the base layer. Then the coefficient of the bottom layer of the Gaussian pyramid ,

[0152] For the upper level The first image is obtained by downsampling after applying Gaussian blur to the lower layer image. Gaussian pyramid coefficient of layers :

[0153] ,

[0154] in, It is a Gaussian low-pass filter. To employ interlaced downsampling, the image size is halved layer by layer, and a Gaussian pyramid is used to capture multi-scale low-frequency information of the image;

[0155] S42. Construct the Pyramid of Laplace;

[0156] Based on the Gaussian pyramid, the Laplace pyramid is constructed to extract bandpass information at various scales. For the top layer of the Laplace pyramid... The top coefficient of the Plas pyramid ;

[0157] For the remaining layers The first image is obtained by upsampling the upper layer image. The Plas pyramid coefficient of the layer :

[0158] ;

[0159] in, For upsampling operations, it is necessary to... Interpolation magnification to Same size, the first The Plas pyramid coefficient of the layer The corresponding image's edge and detail information at different scales;

[0160] S43. Calculate local sharpness;

[0161] At the base of the Pyramid of Laplace That is, the highest resolution layer, for each pixel location Calculate the local variance within its neighborhood as a measure of the sharpness of that point:

[0162] ,

[0163] in, The local window radius, This represents the average value of the pixels within the local window.

[0164] S44. Normalize and generate a weighted pyramid;

[0165] For each pixel position The image sharpness measure at that point is normalized to obtain the underlying fusion weights:

[0166] The fusion weight quantifies the relative sharpness advantage of each image at different spatial locations; the higher the weight value, the greater the contribution of that image to the fusion result at that pixel location.

[0167] S5. Pyramid fusion reconstruction: The coefficients of each layer of the Laplacian pyramid are weighted and fused using a fusion weighted pyramid, and the final super-depth image is obtained by reconstructing layer by layer upwards.

[0168] S51, For each level of the pyramid Using each input image in the first Layer fusion weights Weighted fusion of the Laplace pyramid coefficients:

[0169] ,

[0170] in, The total number of input images, For the first The image in the first Layer fusion weights, For the first The image in the first The Laplacian pyramid coefficients of the layers; this operation preserves the most accurate local details at each scale, achieving optimal coefficient selection and fusion across multiple scales;

[0171] S52, Top-level initialization and reconstruction image, due to the top-level... No higher layers are upsampled, therefore the top layer reconstructs the image. Take the fusion Laplacian coefficient of this layer:

[0172] Through iterative layer-by-layer processing, high-frequency details and low-frequency contours are gradually integrated at various scales, ultimately restoring a complete fused image.

[0173] S53, Regarding The iterative reconstruction operation is performed sequentially, the first... Layer Reconstructed Image for:

[0174] ,

[0175] S54. When the iteration is complete and the bottom layer is reached... At this point, the final fused image is obtained:

[0176] ,

[0177] The spatial resolution is consistent with the original input image. Each pixel is formed by fusing the content of the corresponding position with the highest clarity from multiple source images. Finally, F is the super depth-of-field image after the multifocal image has been processed to enhance the full depth of field.

[0178] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the present invention. Although detailed descriptions have been provided with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments, and they should all be covered within the protection scope of the claims.

Claims

1. A super-depth-of-field image acquisition system for defect detection, characterized in that, The system includes a light source module (1), a lens (2), a prism (3), a camera (4), and a displacement drive assembly. A vertical mounting plate (5) is fixedly connected to the displacement drive assembly. The light source module (1), lens (2), prism (3), and camera (4) are connected sequentially from bottom to top and fixedly installed with the surface of the vertical mounting plate (5) as the mounting reference surface. The optical axes of the light source module (1), lens (2), prism (3), and camera (4) are collinear. The light source module (1) is composed of LEDs of different heights, colors, and angles. The camera (4) and the light source module (1) are respectively connected to an external host computer.

2. The ultra-depth-of-field image acquisition system for defect detection according to claim 1, characterized in that, The light source module (1) also includes a lampshade (6) for mounting LED lights. The closed end (7) of the lampshade (6) is located at the top for docking with the lens (2), and the flared end (8) of the lampshade (6) is located at the bottom. The lampshade (6) gradually expands outward from top to bottom into a cone shape. The inner wall of the lampshade (6) is provided with multiple levels of annular steps (9) from top to bottom. The LED lights are evenly arranged around the circumference of the steps (9).

3. The ultra-depth-of-field image acquisition system for defect detection according to claim 2, characterized in that, The LED lights on the same level of the circular staircase (9) are all the same color.

4. The ultra-depth-of-field image acquisition system for defect detection according to claim 3, characterized in that, The inner wall of the lampshade (6) has six ring steps from top to bottom. White LED lights, red LED lights, white LED lights, green LED lights, white LED lights and white-blue mixed LED lights are installed on the first to sixth ring steps from top to bottom, respectively. The green LED light on the fourth ring step is at a 45° angle to the system optical axis, the white LED light on the fifth ring step is at a 45° angle to the system optical axis, the white-blue mixed LED light on the sixth ring step is at a 60° angle to the system optical axis, and the optical axes of the LED lights on the remaining ring steps are parallel to the system optical axis.

5. A super-depth-of-field image acquisition system for defect detection according to any one of claims 1-4, characterized in that, The displacement drive assembly includes a horizontal displacement unit (10) and a vertical displacement unit (11). The vertical displacement unit (11) is fixedly connected to the movable end of the horizontal displacement unit (10), and the vertical mounting plate (5) is fixedly connected to the movable end of the vertical displacement unit (11).

6. A method for acquiring ultra-depth-of-field images for defect detection, characterized in that, It is implemented based on any one of the claims 1 to 5, a super-depth-of-field image acquisition system for defect detection, and its steps are as follows: S1, Focus stack acquisition: The camera (4) is controlled to expose sequentially at different focus positions by triggering signals from the host computer to obtain a set of images focused at different depth of field positions; S2. Image sharpness evaluation: The sharpness response value of each input image is calculated using the Laplacian variance method; S3. Image Alignment: Images with different focal points are aligned using the SIFT method to obtain registered images; S4. Constructing a sharpness weight pyramid: Construct a Gaussian pyramid and a Laplacian pyramid for the registered image. Calculate and normalize the local sharpness metric based on the bottom layer of the Laplacian pyramid to generate a fusion weight pyramid. S5. Pyramid fusion reconstruction: The coefficients of each layer of the Laplacian pyramid are weighted and fused using a fusion weighted pyramid, and the final super-depth image is obtained by reconstructing layer by layer upwards.

7. The method for acquiring ultra-depth-of-field images for defect detection according to claim 6, characterized in that, In S2, the calculation steps of the Laplace variance method are as follows: S21. Employ an eight-neighbor Laplacian convolution kernel. Its form is: ， S22, for dimensions of The original image Perform zero-fill at the boundaries, where Image height, The width of the image after padding. for; ， S23, Apply the eight-neighbor Laplacian convolution kernel With the image of Hou Each pixel position Perform discrete convolution to calculate the position of each pixel. response value ; ， in, ; S24, Response Value To perform statistical analysis, the global mean of the response map for all pixels is first calculated. : ， Then calculate its variance: ， As a metric for image sharpness, The larger the value, the richer the edge details of the image and the higher the clarity; conversely, the smaller the value, the blurrier the image.

8. The method for acquiring ultra-depth-of-field images for defect detection according to claim 7, characterized in that, The sub-steps of S3 are: S31. For the original input image Construct a Gaussian pyramid by generating a scale-space representation using Gaussian kernels of different scales: ， in, , The Gaussian blur radius; Further construct the Gaussian difference pyramid: ， Where k takes ; Then, the extremum conditions in the scale space are solved: ， Find The local extreme points are the key points; S32. For each key point, statistical gradient direction histogram is generated in its neighborhood, and a 128-dimensional feature vector is constructed as the descriptor of the point. For the key points of image one Its descriptor is Then the set of key points in image one is represented as: , This represents the total number of keypoints in Image 1. Similarly, the set of key points in image two is represented as follows: ; This represents the total number of keypoints in Image 1. S33. Employ the nearest neighbor search matching strategy for each descriptor in Image 1. In the descriptor set of image two Search within. The nearest neighbor is: ; The next nearest neighbor is: ; in Use the Euclidean distance function; When the following ratio condition is met, then and Pairing, ratio condition: ； S34. Obtain the initial set of matching points. Then, the RANSAC algorithm is used to iteratively optimize the transformation matrix: S341, from Minimum subset of random sampling , the smallest subset It contains m pairs of matching points, where m depends on the degrees of freedom of the transformation model; S342, Based on the minimum subset Estimating candidate transformation matrix ; S343. Calculate the transformation matrix The set of interior points below , ， in, Represented in homogeneous coordinates, For reprojection error, The preset error threshold is used; S344, If the set of interior points If the cardinality of the set of interior points is greater than the cardinality of the current optimal interior point set, then update the optimal transformation matrix. = And record the optimal set of interior points. = ; Repeat the iterative process from S341 to S344 until convergence, and output the final transformation matrix. Using the final transformation matrix Align the images with different focal points to obtain the registered image. .

9. A method for acquiring ultra-depth-of-field images for defect detection according to claim 8, characterized in that, The sub-steps of S4 are: S41. Construct the Gaussian pyramid; For the registered image Construct a Gaussian pyramid, with the registered image forming the base layer. Then the coefficient of the bottom layer of the Gaussian pyramid , For the upper level The first image is obtained by downsampling after applying Gaussian blur to the lower layer image. Gaussian pyramid coefficient of the layer : ， in, It is a Gaussian low-pass filter. To employ interlaced downsampling, the image size is halved layer by layer, and a Gaussian pyramid is used to capture multi-scale low-frequency information of the image; S42. Construct the Pyramid of Laplace; Based on the Gaussian pyramid, the Laplace pyramid is constructed to extract bandpass information at various scales. For the top layer of the Laplace pyramid... The top coefficient of the Plas pyramid ; For the remaining layers The first image is obtained by upsampling the upper layer image. The Plas pyramid coefficient of the layer : ； in, For upsampling operations, it is necessary to... Interpolation magnification to Same size, the first The Plas pyramid coefficient of the layer The corresponding image's edge and detail information at different scales; S43. Calculate local sharpness; At the base of the Pyramid of Laplace That is, the highest resolution layer, for each pixel location Calculate the local variance within its neighborhood as a measure of the sharpness of that point: ， in, The local window radius, This represents the average value of the pixels within the local window. S44. Normalize and generate a weighted pyramid; For each pixel position The image sharpness measure at that point is normalized to obtain the underlying fusion weights: 。 10. A method for acquiring ultra-depth-of-field images for defect detection according to claim 9, characterized in that, The sub-steps of S5 are: S51, For each level of the pyramid Using each input image in the first Layer fusion weights Weighted fusion of the Laplace pyramid coefficients: ， in, The total number of input images, For the first The image in the first Layer fusion weights, For the first The image in the first The Laplace pyramid coefficient of the layer; S52, Top-level initialization and reconstruction image, due to the top-level... No higher layers are upsampled, therefore the top layer reconstructs the image. Take the fusion Laplacian coefficient of this layer: ， S53, Regarding The iterative reconstruction operation is performed sequentially, the first... Layer Reconstructed Image for: ， S54. When the iteration is complete and the bottom layer is reached... At this point, the final fused image is obtained: ， The spatial resolution is consistent with the original input image. Each pixel is formed by fusing the content of the corresponding position with the highest clarity from multiple source images. Finally, F is the super depth-of-field image after the multifocal image has been processed to enhance the full depth of field.

Images