Large-depth-of-field microstructure three-dimensional reconstruction and measurement method based on structured light

Abstract

The invention discloses a large-depth-of-field microstructure three-dimensional reconstruction and measurement method based on structured light, and belongs to the field of fine object three-dimensional shape measurement, and the method comprises the steps: introducing an off-axis holographic technology in a digital holographic technology into a traditional microstructure light measurement system, and achieving the depth-of-field expansion of the measurement system; according to the invention, the characteristic of high tolerance of the digital holographic technology to depth-of-field change during measurement is ingeniously and organically combined with the characteristic of measuring the rough surface sample by the structured light. And compared with a traditional microstructure light system, the large-depth-of-field microstructure light measurement system can effectively improve the depth of field of the system while ensuring the measurement precision.

Description

Technical Field

[0001] The present invention belongs to the field of three-dimensional shape measurement of micro-objects, and in particular relates to a three-dimensional reconstruction and measurement method of a microstructure with a large depth of field based on structured light. Background Art

[0002] Today, with the rapid advancement of electronic science and technology, the manufacturing industry has flourished. A range of high-tech portable communication devices, such as mobile phones and computers, have profoundly impacted people's lives. Microscale devices have penetrated various fields, such as implantable microrobots in medicine and micro-array lenses in reentry optics. These devices, typically measuring in the micrometer to millimeter range, have a significant impact on precision manufacturing, industrial quality inspection, and other fields. Consequently, the demand for 3D measurement of microstructures is increasing. Numerous methods have been developed for measuring the topography of objects of this size, primarily categorized as contact and non-contact. Contact methods include atomic force microscopes and scanning electron microscopes. While they offer high precision, they often require contact with the sample, which can easily damage the sample surface.

[0003] On the other hand, in non-contact measurement, white light interferometers are representative, which can realize full-field non-contact measurement and have relatively high sensitivity and accuracy. However, its depth of field is limited, and there is a diffraction bat-wing effect. Most importantly, it is difficult to measure surfaces that are too rough and have poor reflectivity. Structured light has high precision, a large measuring range and low cost, and is suitable for most application scenarios of 3D reconstruction of fine-structured objects. At present, when using structured light methods to measure tiny samples, it is generally necessary to use a microscopic structured light measurement system. Compared with traditional structured light measurement systems, microscopic structured light systems have a smaller depth of field and are very sensitive to camera defocus, resulting in large precision errors caused by defocus that seriously affect the measurement effect.

[0004] In response to the above problems, existing research on the implementation of depth of field extension is mainly divided into two types: physical extension and virtual extension. The former requires different calibration parameters for different depths due to the dynamic change of focal length, so the calibration complexity is extremely high, and it is impossible to take into account both measurement accuracy and measurement efficiency, and a trade-off needs to be made. The latter has a complex algorithm and the depth of field improvement effect is not obvious when the depth changes greatly. Summary of the Invention

[0005] To solve the above technical problems, the present invention provides a method for three-dimensional reconstruction and measurement of microstructures with a large depth of field based on structured light, comprising:

[0006] Build a microstructured light measurement system;

[0007] Acquiring image information of the object to be measured based on the microstructured light measurement system, and reconstructing the optical wave field of the out-of-focus portion of the image information of the object to be measured based on a digital holographic method to obtain an extended depth of field image;

[0008] An optimal focus evaluation is performed on the extended depth of field image based on a grayscale gradient evaluation function, and a full focus image is acquired by a focus stacking method.

[0009] Preferably, the microstructured light measurement system comprises: a solid laser, a beam splitter, a reflector, a Michelson interferometer, a microscope objective lens and a detector.

[0010] Preferably, the process of acquiring image information of the object to be measured based on the microstructured light measurement system includes:

[0011] Based on the solid laser emitting laser light, the laser light is divided into reference light and illumination light by the beam splitter;

[0012] The reference light is incident on a beam splitter through a reflector at a calculated off-axis angle, and the illumination light is irradiated onto the object to be measured by forming interference fringes through a Michelson interferometer;

[0013] The scattered light from the surface of the object to be measured is collected by the microscope objective lens, and the scattered light is imaged onto the detector to generate image information of the object to be measured.

[0014] Preferably, the total light intensity of the detector is expressed as:

[0015] I(r)=|R(r)+O(r)| 2 =|R(r)| 2 +|O(r)| 2 +R * (r)O(r)+R(r)O * (r);

[0016] Among them, I(r) is the total light intensity, R(r) and O(r) represent the light field information of the reference light and the illumination light respectively, R * (r) represents the conjugate wave of the reference light wave, O * (r) represents the conjugate wave of the irradiating light wave.

[0017] Preferably, the off-axis angle is calculated based on the highest spatial frequency of the sample to be measured and the laser wavelength.

[0018] Preferably, the optical wave field reconstruction step includes:

[0019] Performing Fourier transform on the image acquired by the detector to separate the spectral components containing the required information;

[0020] Selecting a suitable off-axis angle to completely separate the spectral components;

[0021] extracting one of the spectral components and moving the spectral component to the center of the spectrum;

[0022] The spectral components moved to the center of the spectrum are processed according to the Fresnel diffraction formula to complete the wavefront reconstruction of the signal light.

[0023] Preferably, the z direction is set as the direction of light wave propagation, and the Fresnel diffraction formula is:

[0024]

[0025] Among them, H(k) represents the distribution of Fresnel diffraction field with the change of wave vector, k0 represents the wave number in the air, and k x represents the component of the wave vector k in the x direction, k y represents the component of the wave vector k in the y direction.

[0026] Preferably, the expression for obtaining the all-focus image by the focus stacking method is:

[0027]

[0028] Among them, C is the normalization parameter, J(μ,υ,w) represents the grayscale value of the w-th image in the image stack at (μ,υ), and Ψ(μ,υ,w) represents the weight of the corresponding pixel.

[0029] On the other hand, the present invention further provides an electronic device, comprising a memory, a processor, and a computing program stored in the memory and executable on the processor, wherein the method is implemented when the processor executes the computing program.

[0030] On the other hand, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the method when executed by a processor.

[0031] Compared with the prior art, the present invention has the following advantages and technical effects:

[0032] This invention expands the depth of field of a traditional microstructured light measurement system by integrating off-axis holography, a technique used in digital holography, into the measurement system. This method cleverly combines the high tolerance of digital holography for depth-of-field variations during measurement with the unique properties of structured light for measuring rough surface samples. Ultimately, this method creates a microstructured light measurement system with a large depth of field. Compared to traditional microstructured light systems, this invention significantly improves the depth of field while maintaining measurement accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0034] Figure 1 is a light path diagram of an embodiment of the present invention;

[0035] Figure 2 This is a flowchart of defocus fringe processing according to an embodiment of the present invention.

[0036] Figure 3 Figure 1 shows the effect of expanding the depth of field using microstructured light according to an embodiment of the present invention, where (a) shows the out-of-focus fringe pattern due to the step height, and (b) shows the fringe pattern obtained after the depth of field is expanded. DETAILED DESCRIPTION

[0037] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0038] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0039] Example 1

[0040] like Figure 1-2 As shown, this embodiment provides a method for three-dimensional reconstruction and measurement of a microstructure with a large depth of field based on structured light, including:

[0041] Build a microstructured light measurement system;

[0042] Acquiring image information of the object to be measured based on the microstructured light measurement system, and reconstructing the optical wave field of the out-of-focus portion of the image information of the object to be measured based on a digital holographic method to obtain an extended depth of field image;

[0043] An optimal focus evaluation is performed on the extended depth of field image based on a grayscale gradient evaluation function, and a full focus image is acquired by a focus stacking method.

[0044] The process of acquiring image information of the object to be measured based on the microstructured light measurement system includes:

[0045] Based on the solid laser emitting laser light, the laser light is divided into reference light and illumination light by the beam splitter;

[0046] The reference light is incident on a beam splitter through a reflector at a calculated off-axis angle, and the illumination light is irradiated onto the object to be measured by forming interference fringes through a Michelson interferometer;

[0047] This system introduces the optical path of digital holography into the microstructured light system. The specific optical path diagram is as follows: Figure 1 As shown in the figure, the solid-state laser emits a 532nm laser, which is then split into reference light and illumination light after passing through a beam splitter. The reference light passes through a series of mirrors and is incident on the beam splitter at a calculated off-axis angle. The illumination light passes through a classic Michelson interferometer, and the interference fringes formed are finally irradiated onto the sample. The scattered light on the sample surface is finally returned to the detector through a lens with a magnifying effect.

[0048] Since this embodiment utilizes a Michelson interferometer to replace the structured light projector in projecting fringes onto the sample, and since subsequent capture requires only a single image, and holographic fusion is incorporated to restore clear fringes, slight variations in the fringes caused by environmental vibrations in the Michelson interferometer itself do not affect the experiment, this embodiment does not employ active vibration isolation.

[0049] Integrating a Michelson interferometer into a microstructured light system overcomes the limitations of projected fringe periods in traditional structured light systems. Theoretically, the Michelson interferometer can produce fringes of varying wavelength widths, potentially improving measurement accuracy. The fringe generation method here can determine the fringe period by adjusting the pitch and spacing of the Michelson interferometer's reflectors.

[0050] To achieve its extended depth of field, the optical wavefield is reconstructed at different z-scales by leveraging the characteristics of digital holography. Best focus is assessed using a grayscale gradient evaluation function. Focus stacking allows for full image fusion within seconds, resulting in a fully focused image. This method can bring out-of-focus fringes back into focus with a single shot, achieving the goal of extending the depth of field. The evaluation function for best focus can be selected based on specific requirements.

[0051] At this time, the total light intensity received by the detector can be expressed as:

[0052] I(r)=|R(r)+O(r)| 2 =|R(r)| 2 +|O(r)| 2 +R * (r)O(r)+R(r)O * (r) (1)

[0053] Where R(r) and O(r) represent the light field information of the reference light and the illumination light, respectively. At this time, if the Fourier transform is performed on formula (1), the following formula can be obtained in this embodiment:

[0054]

[0055] At this time k r is the wave vector of the reference light. At this time, the first term in formula (2) is the zero-frequency component, the second term is the convolution operation of the background light, and the third and fourth terms are two non-overlapping circular bright spots in the Fourier spectrum, such as Figure 2 As shown. The third and fourth items contain the information required for this embodiment. At this time, by selecting a suitable incident angle for the reference light, the last two items can be completely separated, and two circular spots will appear on the Fourier spectrum. The specific off-axis angle needs to meet the following requirements. For the sample to be tested, its highest spatial frequency is B, and the bandwidth is 2B. At this time, it should meet the following requirements:

[0056]

[0057] In the above formula, λ is the wavelength of the laser and θ is the off-axis angle, so the above formula can be further derived as:

[0058] θ ≥ arcsin(3Bλ);

[0059] At this point, it can be considered that when the off-axis angle meets the above conditions, the real image and the virtual image can be separated. At this time, one of them is extracted, such as the third item, and moved to the center of the spectrum.

[0060] Considering that the essence of defocus is actually Fresnel diffraction, according to the Fresnel diffraction formula:

[0061]

[0062] Where τ is the defocus amount and K0 is the wave number in air. At this point, performing an inverse Fourier transform can complete the wavefront reconstruction of the signal light. The digital refocusing of the wavefront can be expressed as:

[0063]

[0064] Completing wavefront reconstruction means that this embodiment can restore the light field at z, which can refocus out-of-focus fringes. This method can restore the light field at different z planes. The sample to be tested is often uneven, which means that different parts of the image may be in focus at different z planes. Therefore, it is necessary to further determine where the focus is. Here, this embodiment uses the gradient variance within a certain pixel neighborhood as the focus evaluation value of the point. The formula can be written as:

[0065]

[0066] Where F(μ,υ,w) represents the focus evaluation value of the w-th image in the image stack at the μ-th row and the υ-th column, and φ is a scaling factor used to adjust the steepness of the function. The weighted average of the pixel values ​​in the image stack is then performed, and the output of the fused image is:

[0067]

[0068] Where C is the normalization parameter, J(μ,υ,w) represents the grayscale value of the w-th image in the image stack at (μ,υ), and Ψ(μ,υ,w) represents the weight of the corresponding pixel. After processing by the above method, it can be seen that Figure 3 (a) shows the defocused stripes captured by the microscopic structured light system, and (b) shows the clear stripes obtained after being refocused after passing through the above system.

[0069] On the other hand, this embodiment further provides an electronic device, comprising a memory, a processor, and a computing program stored in the memory and executable on the processor, wherein the method is implemented when the processor executes the computing program.

[0070] On the other hand, this embodiment further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the method when executed by a processor.

[0071] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for three-dimensional reconstruction and measurement of microstructures with a large depth of field based on structured light, characterized in that: include: Build a microstructured light measurement system; Acquiring image information of the object to be measured based on the microstructured light measurement system, and reconstructing the optical wave field of the out-of-focus portion of the image information of the object to be measured based on a digital holographic method to obtain an extended depth of field image; An optimal focus evaluation is performed on the extended depth of field image based on a grayscale gradient evaluation function, and a full focus image is acquired by a focus stacking method.

2. The method according to claim 1, characterized in that The microstructured light measurement system comprises: a solid laser, a beam splitter, a reflector, a Michelson interferometer, a microscope objective lens and a detector.

3. The method according to claim 2, characterized in that The process of acquiring image information of the object to be measured based on the microstructured light measurement system includes: Based on the solid laser emitting laser light, the laser light is divided into reference light and illumination light by the beam splitter; The reference light is incident on a beam splitter through a reflector at a calculated off-axis angle, and the illumination light is irradiated onto the object to be measured by forming interference fringes through a Michelson interferometer; The scattered light from the surface of the object to be measured is collected by the microscope objective lens, and the scattered light is imaged onto the detector to generate image information of the object to be measured.

4. The method according to claim 2, characterized in that The expression of the total light intensity of the detector is: I(r)=|R(r)+O(r)| 2 =|R(r)| 2 +|O(r)| 2 +R * (r)O(r)+R(r)O * (r) ; Among them, I(r) is the total light intensity, R(r) and O(r) represent the light field information of the reference light and the illumination light respectively, R * (r) represents the conjugate wave of the reference light wave, O * (r) represents the conjugate wave of the irradiating light wave.

5. The method according to claim 2, characterized in that The off-axis angle is calculated based on the highest spatial frequency of the sample to be measured and the laser wavelength.

6. The method according to claim 2, characterized in that The optical wave field reconstruction step includes: Performing Fourier transform on the image acquired by the detector to separate the spectral components containing the required information; Selecting a suitable off-axis angle to completely separate the spectral components; extracting one of the spectral components and moving the spectral component to the center of the spectrum; The spectral components moved to the center of the spectrum are processed according to the Fresnel diffraction formula to complete the wavefront reconstruction of the signal light.

7. The method according to claim 6, characterized in that The z direction is set as the direction of light wave propagation, and the Fresnel diffraction formula is: Among them, H(k) represents the distribution of Fresnel diffraction field with the change of wave vector, k0 represents the wave number in the air, and k x represents the component of the wave vector k in the x direction, k y represents the component of the wave vector k in the y direction.

8. The method according to claim 1, characterized in that The expression for obtaining a fully focused image by the focal stack method is: Among them, C is the normalization parameter, J(μ,υ,w) represents the grayscale value of the w-th image in the image stack at (μ,υ), and Ψ(μ,υ,w) represents the weight of the corresponding pixel.

9. An electronic device comprising a memory, a processor, and a computing program stored in the memory and executable on the processor, wherein: When the processor executes the computing program, the method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images