Automatic focusing method based on pixel-level aberration detection

By employing a pixel-level aberration detection method and utilizing multinomial fitting and neural network training, the problem of light convergence in the imaging system was solved, achieving efficient autofocus and improved image sharpness.

CN118972698AActive Publication Date: 2024-11-15WENZHOU UNIV METAVERSE & ARTIFICIAL INTELLIGENCE RES INST

Patent Information

Application Number
CN202411406181.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-10
Publication Date
2024-11-15
Estimated Expiration
2044-10-10

AI Technical Summary

Technical Problem

In existing technologies, assembly errors and incorrect focusing settings make it difficult for light to converge in the imaging system, resulting in diffuse light spots and blurred images. Existing imaging models differ significantly from the actual situation.

Method used

A pixel-level aberration detection method is adopted. By using histogram-invariant spatial confusion, multinomial fitting, Fourier transform and neural network training, the point spread function of the pixel position is estimated. Combined with PID control and lens parameter conversion, autofocus is achieved.

Benefits of technology

Generate high-quality full-focus and defocus image pairs in large batches to improve image focus quality and enhance the accuracy and efficiency of autofocus.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118972698A_ABST
    Figure CN118972698A_ABST
Patent Text Reader

Abstract

The invention discloses an automatic focusing method based on pixel-level aberration detection, which belongs to the technical field of image processing, and mainly utilizes Fraunhofer diffraction to calculate a pupil function and obtain an amplitude point spread function, and then performs Fourier transform on the amplitude point spread function to obtain an amplitude transfer function of a coherent imaging system. And finally, carrying out modular square on the amplitude transfer function to obtain an intensity point spread function of the incoherent imaging system. A point spread function model at each pixel position in a space-variant system is constructed by referring to the relationship between a pupil function and a point spread function in wavefront coding, so that an aberration estimation problem in automatic focusing is converted into an estimation problem of a point spread function polynomial coefficient at each pixel position.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to an automatic focusing method based on pixel-level aberration detection. Background Art

[0002] The number of photosensitive cells in digital sensors has increased rapidly with the improvement of light perception capabilities. However, due to inevitable aberrations such as assembly errors and incorrect focus settings, it is difficult for the light emitted from the object point to converge on the camera's photosensitive cells, which will form a diffuse light spot and cause image blur. The point spread function (PSF) describes the distribution of light emitted by a single object point on the image plane after passing through the lens. The PSF covers both the structural information of the lens and the depth information of the scene. Estimating the PSF of an imaging system can be applied to 3D microscopy, autofocus, image deblurring, adaptive optics and astronomical imaging, and monocular depth estimation. Summary of the invention

[0003] In view of the defects of the prior art, the present invention provides an auto-focusing method based on pixel-level aberration detection, which is used to solve the problem of the increasing gap between the imaging model and the actual situation in the prior art.

[0004] To achieve the above-mentioned object and other related objects, the present invention provides an automatic focusing method based on pixel-level aberration detection, comprising: The pixel arrangement of the image samples is randomly shuffled using a histogram-invariant spatial confusion method to obtain a fully focused image. For any given image I, the row and column indices of the image are randomly shuffled and recombined, and the pixel positions are arranged according to the recombined indices, while keeping the RGB channels unchanged; After setting the aberration range, polynomial fitting is performed on the aberration using, but not limited to, Zernike polynomials and Seidel aberrations; The point spread function of each pixel position is calculated based on the corresponding relationship between the aberration and the point spread function, that is, the point spread function of each pixel position is calculated based on the corresponding relationship between the aberration and the point spread function, that is, the point spread function is obtained by performing Fourier transformation on the pupil function and then taking the square of the modulus; According to the linear invariance of the optical system to the intensity of the light wave field and the superposition theorem, the image of the object can be expressed as the weighted sum of two-dimensional pulse functions using the object field; The blurred image obtained by simulation is used as input, and the polynomial coefficients of each pixel position are used as labels to train a neural network. The output feature map of the neural network is consistent with the input, and each position of the feature map records the defocus amount of the pixel at the corresponding position.

[0005] The defocus amount of each pixel position is output by the neural network, and the movement amount of the lens assembly is obtained using methods including but not limited to PID control and lens parameter conversion.

[0006] As described above, an autofocus method based on pixel-level aberration detection of the present invention has the following beneficial effects: the present invention can automatically generate fully focused and defocused image pairs with spatially varying defocus amount annotations in large quantities, and at the same time, aberrations including but not limited to distortion, spherical aberration and coma are added to the simulation to improve the quantity and annotation quality of fully focused images and defocused image pairs. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 Flowchart for modeling point spread functions;

[0008] Figure 2 This is a system workflow diagram provided for Example 1 of the present invention. DETAILED DESCRIPTION

[0009] In order to make the technical solutions and advantages of the present invention more clearly understood, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to be limiting of the present application.

[0010] The present invention provides an automatic focusing method based on pixel-level aberration detection, which specifically comprises the following steps: Step S10, obtain three-channel images of any natural scene to form a sample set of images to be processed, and use a histogram-invariant spatial obfuscation method to process the images to be processed to produce a sample set of aberration-free images. To obtain image samples, you can use a camera to take pictures or download public data sets to obtain image samples with rich styles, and use a histogram-invariant spatial obfuscation method to randomly shuffle the pixel arrangement of the image samples to obtain a fully focused image. For any given image, randomly shuffle the row and column indexes of the image and recombine them, and arrange the pixel positions according to the recombined indexes, while keeping the RGB channels unchanged; Step S20, using random numbers to generate polynomial coefficients of the aberration at each pixel position, wherein the polynomial coefficients include but are not limited to Zernike polynomials and Seidel aberrations. After setting the aberration range, polynomial fitting is performed on the aberration using including but not limited to Zernike polynomials and Seidel aberrations; Step S30: Figure 1As shown in the figure, the pupil function is calculated by Fraunhofer diffraction and the amplitude point spread function is obtained. Then, the amplitude point spread function is Fourier transformed to obtain the amplitude transfer function of the coherent imaging system. Finally, the amplitude transfer function is modulo squared to obtain the intensity point spread function of the incoherent imaging system. Drawing on the relationship between the pupil function and the point spread function in wavefront coding, the PSF model at each pixel position in the space-variant system is constructed, thereby converting the aberration estimation problem into the estimation problem of the PSF polynomial coefficients at each pixel position. Step S40, representing the image of the object point as a weighted sum of two-dimensional impulse functions, multiplying the light intensity value of each pixel by the corresponding point spread function according to the aberration-free image and the intensity point spread function of each pixel position, and finally superimposing them in sequence according to the pixel position to obtain an image of the scene; Step S50: Using the blurred image as input and the polynomial coefficients of each pixel position as labels to train a neural network, such as Figure 2 As shown in the figure, given a single image, the neural network-based model PSFNet is used to estimate the Zernike coefficient map or Seidel coefficient map of the PSF of each pixel in the input image. The number of output channels of the CNN model is equal to the order of the aberration. During the model training process, the input image is converted into a fully focused image using the histogram-invariant spatial aliasing sampling method. During the model inference test process, the image of the scene to be focused is directly input without converting it into a fully focused image. Step S60: The image of the desired focus scene is transmitted to the neural network, the channel corresponding to the defocus aberration in the output feature map is extracted, and then the value of the corresponding position of the focus area is taken out to convert the movement amount of the lens assembly.

[0011] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An autofocus method based on pixel-level aberration detection, It is characterized in that include: 1) Obtaining three-channel images of any natural scene to form a sample set of images to be processed, and using a histogram-invariant spatial confusion method to process the images to be processed to produce a sample set of images without aberration; 2) using random numbers to generate polynomial coefficients of the aberration at each pixel position, wherein the polynomial coefficients include but are not limited to Zernike polynomials and Seidel aberrations; 3) The pupil function is calculated using Fraunhofer diffraction to obtain the amplitude point spread function, which is then Fourier transformed to obtain the amplitude transfer function of the coherent imaging system. Finally, the amplitude transfer function is modulo squared to obtain the intensity point spread function of the incoherent imaging system. 4) The image of the object point is represented as a weighted sum of two-dimensional impulse functions. According to the aberration-free image and the intensity point spread function of each pixel position, the blurred image is calculated by sliding addition of the product of each pixel value of the aberration-free image and the corresponding point spread function; 5) Using the blurred image as input and the polynomial coefficients of each pixel position as labels to train a neural network; 6) The image of the desired focus scene is passed into the neural network, the numerical value of the position corresponding to the focus area in the output feature map is extracted, and the movement amount of the lens assembly is calculated.

2. The automatic focusing method based on pixel-level aberration detection according to claim 1, It is characterized in that The pixel arrangement of the image samples is randomly shuffled using a histogram-invariant spatial confusion method to obtain a fully focused image. For any given image I, the row and column indices of the image are randomly shuffled and recombined, and the pixel positions are arranged according to the recombined indices while keeping the RGB channels unchanged.

3. The automatic focusing method based on pixel-level aberration detection according to claim 1, It is characterized in that After the aberration range is set, polynomial fitting is performed on the aberration using methods including but not limited to Zernike polynomials and Seidel aberrations.

4. The automatic focusing method based on pixel-level aberration detection according to claim 1, It is characterized in that The point spread function of each pixel position is calculated based on the corresponding relationship between the aberration and the point spread function. That is, the point spread function is obtained by performing Fourier transformation on the pupil function and then taking the square of the modulus.

5. The automatic focusing method based on pixel-level aberration detection according to claim 1, It is characterized in that The image of the scene is represented by the object field as a weighted sum of two-dimensional impulse functions, that is, the light intensity value of each pixel is multiplied by the corresponding point spread function, and finally the image of the scene is obtained by superimposing them in sequence according to the pixel position.

6. The automatic focusing method based on pixel-level aberration detection according to claim 1, It is characterized in that The blurred image obtained by simulation is used as input, and the polynomial coefficients of each pixel position are used as labels to train a neural network. The output feature map of the neural network is consistent with the input, and each position of the feature map records the defocus amount of the pixel at the corresponding position.

7. The automatic focusing method based on pixel-level aberration detection according to claim 1, It is characterized in that The neural network outputs the defocus amount at each pixel position, and the defocus amount is converted into the movement amount of the lens assembly using methods including but not limited to PID control and lens parameter conversion.

Citation Information

Patent Citations

  • Self-supervised extended target aberration detection method based on convolutional neural network

    CN115019125A

  • Telescope aberration correction method based on point spread function

    CN116699838A

  • Focus setting determination

    GB202400692D0

  • Miniaturized adaptive optics with deformable phase plate for digital tilt and aberration correction

    US20240094607A1

  • Device and method for extended depth of field imaging

    WO2023113193A1

Cited By

  • System and method for optimizing focusing position of camera during OLED panel detection

    CN122476267A