Focusing method and device of optical device, electronic equipment and storage medium

Abstract

The invention relates to a focusing method and device of an optical device, electronic equipment and a storage medium, the optical device comprises a first optical element and a first detection assembly, during focusing, a first reticle is placed on an object plane of the first optical element in advance, and the first detection assembly is placed in a defocus range of the rear surface of the first optical element; the focusing method comprises the following steps: acquiring an image of a first reticle acquired by a first detection assembly; the image is input into a pre-trained focusing model, the output of the focusing model is obtained, and the output comprises the distance between the current position of the first detection assembly and the focusing position of the first optical element and the direction of the current position relative to the focusing position; and adjusting the position of the first detection assembly according to the distance and the direction, so that the first detection assembly moves to the focusing position. According to the method and the device, the pre-trained focusing model is combined, so that the defocusing related information can be obtained only by shooting the image once in the focusing process, the focusing efficiency is high, and the equipment cost is low.

Description

Technical Field

[0001] The present application relates to the field of optical technology, and in particular to a focusing method, device, electronic device and storage medium for an optical device. Background Art

[0002] In recent years, due to the continuous improvement of optical design and manufacturing technology, optical systems such as optical lenses and cameras have developed rapidly and are widely used in mobile phones, car systems and other fields. High-quality optical cameras require excellent system design, precision optical processing and manufacturing, and accurate system assembly during the production process. Optical focus is an important step in the manufacture, assembly and final testing of optical systems. Focus error can lead to a series of problems, such as the inability of the system to operate according to the established functions after assembly and the inability to correctly evaluate the imaging quality of the optical system.

[0003] The most traditional focusing method relies on manual operation, which is affected by subjective factors and experience, and cannot guarantee the accuracy and repeatability of focusing. At present, the mainstream methods for realizing autofocus can be divided into two categories: autofocus methods based on hardware and auxiliary devices and autofocus methods based on image processing. The former measures the distance between the system and the subsequent components by emitting sound, light and other signals and receiving the reflected signals, and then places the subsequent components on the quasi-focal plane by moving the components to obtain a clear image, but this type of method will introduce additional equipment, increase the cost and volume of the autofocus system, and is difficult to insert into the existing manufacturing or measurement system; the latter is currently commonly used to collect a series of images along the optical axis of the system, and determine the location of the focal plane based on the image clarity or the value of the MTF (Modulation Transfer Function) at a fixed frequency as the evaluation standard. This is also the most commonly used method at present, known as the "focus depth method". In addition, there are many other methods such as the defocus depth method, but their common disadvantages are that each time focusing is performed, multiple images need to be taken at different positions, which is time-consuming and affects the efficiency of focusing. Summary of the invention

[0004] In response to the above technical problems, the present application provides a focusing method, device, electronic device and storage medium for an optical device, which, combined with a pre-trained focusing model, allows only one image to be taken during the focusing process to obtain defocus-related information, with high focusing efficiency and low equipment cost.

[0005] In order to solve the above technical problems, the present application provides a focusing method of an optical device, wherein the optical device comprises a first optical element and a first detection assembly. During focusing, a first graticule is placed on the object plane of the first optical element in advance, and the first detection assembly is placed within the defocus range of the rear surface of the first optical element. The focusing method comprises: Acquire an image of the first reticle captured by the first detection component; Inputting the image into a pre-trained focus model to obtain an output of the focus model, wherein the output includes a distance between a current position of the first detection component and a focus position of the first optical element, and a direction of the current position relative to the focus position; The position of the first detection component is adjusted according to the distance and the direction, so that the first detection component moves to the quasi-focus position.

[0006] In some embodiments, the first graticule has cross slits or star-point holes.

[0007] In some embodiments, the training method of the focus model includes: Acquire a sample data set, each piece of data in the sample data set includes an image of the second graticule, a distance between a collection position of the second detection assembly and a focal position of the second optical element, and a direction of the collection position relative to the focal position of the second optical element; A pre-built first deep learning model is trained based on the sample data set to obtain the focus model.

[0008] In some embodiments, the focus model includes a quality data extraction algorithm, a feature extraction layer and a data regression layer, and the inputting the image into a pre-trained focus model to obtain an output of the focus model includes: Inputting the image into a pre-trained focus model, acquiring quality data of the image through the quality data extraction algorithm analysis, and extracting image feature data of the image through the feature extraction layer; The quality data and the image feature data are concatenated and used together as inputs of the data regression layer to obtain the output of the focus model.

[0009] In some embodiments, the focus model is applicable to optical elements of different models, and the step of inputting the image into a pre-trained focus model to obtain an output of the focus model further includes: The image and optical parameters of the first optical element are input into a pre-trained focus model.

[0010] In some embodiments, the method of training the focus model includes: Acquire a sample data set, each piece of data in the sample data set includes an image of the second graticule, a distance between a collection position of the second detection assembly and a focal position of the second optical element, and a direction of the collection position relative to the focal position of the second optical element; A pre-built second deep learning model is trained based on the sample data set to obtain the focus model.

[0011] In some embodiments, the method for collecting the sample data set includes: adjusting the second detection assembly to a focal position of the second optical element; placing the second reticle on the object plane of the second optical element; Adjusting the position of the second detection assembly on the plane where it is located so that the imaging area of ​​the second reticle is located at the detection center of the second detection assembly; Adjusting the position of the second detection assembly at a preset step value within the defocus range of the second optical element to collect images of the second reticle at different positions, and recording the distance between the collection position of the corresponding image and the in-focus position of the second optical element, and the direction of the collection position relative to the in-focus position of the second optical element; The data of different acquisition positions of different second optical elements are sorted to obtain the sample data set.

[0012] The present application also provides a focusing device for an optical device, the optical device comprising a first optical element and a first detection assembly. When focusing, a first graticule is placed on the object plane of the first optical element in advance, and the first detection assembly is placed within the defocus range of the rear surface of the first optical element. The focusing device comprises a moving mechanism and a controller, the moving mechanism is connected to the controller, and is used to adjust the position of the first detection assembly according to the instruction of the controller; The controller is used to execute the steps of the method described above.

[0013] The present application also provides an electronic device, including a storage medium and a controller, wherein a computer program is stored on the storage medium, and when the computer program is executed by the controller, the steps of the method described above are implemented.

[0014] The present application also provides a storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described above are implemented.

[0015] The focusing method, device, electronic device and storage medium of the optical device of the present application, the optical device includes a first optical element and a first detection component. When focusing, a first grating plate is placed on the object plane of the first optical element in advance, and the first detection component is placed within the defocus range of the rear surface of the first optical element. The focusing method includes: obtaining an image of the first grating plate captured by the first detection component; inputting the image into a pre-trained focusing model to obtain the output of the focusing model, the output including the distance between the current position of the first detection component and the quasi-focus position of the first optical element, and the direction of the current position relative to the quasi-focus position; adjusting the position of the first detection component according to the distance and direction, so that the first detection component moves to the quasi-focus position. In the present application, combined with the pre-trained focusing model, only one image needs to be taken during the focusing process to obtain the relevant information of defocus, and the focusing efficiency is high and the equipment cost is low. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The figure is a flowchart of a focusing method of an optical device according to an embodiment.

[0017] Figure 2 The figure is a schematic diagram of scene arrangement of a focusing method of an optical device according to an embodiment.

[0018] Figure 3 is a schematic diagram of the architecture of a first deep learning model according to an embodiment.

[0019] Figure 4 is a schematic diagram of the architecture of a second deep learning model according to an embodiment. DETAILED DESCRIPTION

[0020] In order to facilitate the understanding of the present application, the present application will be described more fully below with reference to the relevant drawings. The preferred embodiments of the present application are given in the drawings. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thoroughly and comprehensively understood.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used herein includes any and all combinations of one or more related listed items. In this application, "each" includes one and more than two quantities.

[0022] In the focusing method of the optical device of the present application, the optical device includes a first optical element and a first detection assembly. When focusing, a first graticule is placed on the object plane of the first optical element in advance, and the first detection assembly is placed within the defocus range of the rear surface of the first optical element. The defocus range refers to a certain range in the front-to-back direction centered on the quasi-focus position on the optical axis of the optical element, and the placement position of the first detection assembly only needs to make the collected image not too blurred.

[0023] The optical device can be any device that needs to be focused. For the scenario of active focusing between optical elements, the first optical element is the front element and the first detection component is the subsequent element. For the focusing scenario of the optical test system, the first optical element is the system to be tested and the first detection component is the detector.

[0024] Figure 1 FIG. 1 is a flow chart of a focusing method of an optical device according to an embodiment. Figure 1 As shown, the focusing method is used to focus the above optical device, comprising the following steps: S1, acquiring an image of a first graticule acquired by a first detection component; S2, inputting the image into a pre-trained focus model to obtain an output of the focus model, wherein the output includes a distance between a current position of the first detection component and a focus position of the first optical element, and a direction of the current position relative to the focus position; S3, adjusting the position of the first detection component according to the distance and direction, so that the first detection component moves to a quasi-focus position.

[0025] Please combine Figure 2 , the first detection assembly 50 is placed at a certain position in the defocus range of the first optical element 40, and the first grating plate 30 is placed on the object plane of the first optical element 40. The light emitted by the light source 10 passes through the filter 20 and the diffuser (not shown) to form an incoherent uniform illumination light. After passing through the first optical element 40, the pattern of the first grating plate 30 is imaged onto the first detection assembly 50, and the image of the first grating plate 30 is obtained.

[0026] The focusing model is trained based on a pre-built deep learning model. It can output the distance between the current position of the first detection component when the image is captured within the defocus range and the in-focus position of the first optical element, as well as the direction of the current position relative to the in-focus position as a reference for adjusting the position of the first detection component. There is no need to capture multiple images or use other detection tools, and focusing can be achieved quickly with high accuracy and low cost.

[0027] The distance is an absolute value, and the direction of the current position relative to the quasi-focus position can reflect whether the current position is on the side of the quasi-focus position close to the first optical element or on the side away from the first optical element, thereby providing an adjustment direction.

[0028] Please continue to refer to Figure 2 The first detection assembly 50 is connected to the computer 60, and the focus model is deployed in the computer 60, which can also be used to display the data in the focusing process and provide an operation interface. In addition, the first detection assembly 50 can be driven by a moving mechanism (not shown) to achieve displacement, and the controller of the moving mechanism can be arranged separately and connected to the computer 60, or integrated in the computer 60.

[0029] In some embodiments, the first graticule is preferably a graticule with cross slits or star-point holes, so as to simplify the image analysis process. The first graticule and the second graticule used when training the focus model may not be the same, but the patterns are basically the same.

[0030] In some embodiments, a method for training a focus model includes: Acquire a sample data set, each piece of data in the sample data set includes an image of the second graticule, a distance between a collection position of the second detection assembly and a focal position of the second optical element, and a direction of the collection position relative to the focal position of the second optical element; A pre-built first deep learning model is trained based on the sample data set to obtain a focus model.

[0031] Among them, in each piece of data, the distance between the collection position of the second detection component and the in-focus position of the second optical element, and the direction of the collection position relative to the in-focus position of the second optical element are used as labels corresponding to the image. Thus, the trained focusing model can output the distance between the current position of the first detection component when collecting the image and the in-focus position of the first optical element, and the direction of the current position relative to the in-focus position, only by using one image collected within the defocus range.

[0032] In some embodiments, a method for collecting a sample data set includes: adjusting the second detection assembly to a focal position of the second optical element; placing a second reticle on the object plane of the second optical element; Adjusting the position of the second detection assembly on the plane so that the imaging area of ​​the second reticle is located at the detection center of the second detection assembly; Adjusting the position of the second detection assembly at a preset step value within the defocus range of the second optical element to collect images of the second graticule at different positions, and recording the distance between the collection position of the corresponding image and the in-focus position of the second optical element, and the direction of the collection position relative to the in-focus position of the second optical element; The data from different acquisition positions of different second optical elements are sorted to obtain a sample data set.

[0033] Among them, the equipment used for scene arrangement of sample collection and Figure 2 The models or parameters of the devices shown are basically the same. Specifically, the second detection assembly is first adjusted to the quasi-focus position of the second optical element. This process can be performed using conventional methods (such as the focus depth method or the defocus depth method). A second grating is placed on the object plane of the second optical element. The light emitted by the light source 10 passes through the filter 20 and the diffuser (not shown) to form an incoherent uniform illumination light. After passing through the second optical element, the pattern of the second grating is imaged onto the second detection assembly. The position of the second detection assembly on the plane is adjusted so that the imaging area of ​​the second grating is located at the detection center of the second detection assembly. After that, the position of the second detection assembly is adjusted with a preset step value within the defocus range of the second optical element to collect images of the second grating at different positions, and the distance between the collection position of the corresponding image and the quasi-focus position of the second optical element, as well as the direction of the collection position relative to the quasi-focus position of the second optical element are recorded to obtain the current sample data set of the second optical element. In order to enrich the data set and improve the generalization ability of the model, the second optical elements with different focal lengths and different imaging characteristics can be replaced for data collection, and the collection process is the same. Finally, the sample data of all the second optical elements are taken together as training data.

[0034] Please refer to Figure 3 The first deep learning model is mainly divided into two parts: (1) Feature extraction layer. After the image is input to this part, the grayscale distribution characteristics of the image are extracted layer by layer through continuous convolution blocks or residual blocks. Through network learning, the model can identify the different degradation feature vectors of the image in the in-focus state and out-of-focus state, so that the model can understand the changes in image quality. The output of this part is a one-dimensional feature vector. (2) Data regression layer. Through the fully connected neural network, the amount of data is continuously reduced layer by layer, and finally the feature vector is mapped to the position difference and direction value between the current image position and the in-focus position.

[0035] When training the first deep learning model, the sample data set can be divided into a training set and an experience set, thereby improving the parameter accuracy of the model.

[0036] In some embodiments, in order to further improve the accuracy of focusing, the focusing model of the present application is further combined with image quality data analysis. The image quality data can reflect the transmission ability of optical elements for different frequency components, and can comprehensively reflect various factors affecting imaging quality such as diffraction, aberration, vignetting and stray light, and thus also contains relevant information about the defocus direction and value of a certain current image. Specifically, the focusing model includes a quality data extraction algorithm, a feature extraction layer and a data regression layer, and the image is input into a pre-trained focusing model to obtain the output of the focusing model, including: Input the image into a pre-trained focus model, obtain the quality data of the image through quality data extraction algorithm analysis, and extract image feature data of the image through a feature extraction layer; The quality data is concatenated with the image feature data and used together as the input of the data regression layer to obtain the output of the focus model.

[0037] Among them, when the first grating plate is a grating plate with cross slits, the quality data is the MTF sequence of the current image, and when the first grating plate is a grating plate with star point holes, the quality data is the analysis data of the star point method. After respectively acquiring the image feature data and the image quality data, the two types of data are spliced ​​and used together as the input of the data regression layer to obtain the output of the focus model, and the output includes the distance between the current position of the first detection component and the quasi-focus position of the first optical element, and the direction of the current position relative to the quasi-focus position. It can be understood that since data splicing is performed as the input of the data regression layer, the length of the data regression layer of the focus model needs to be longer than the length of the data regression layer without using quality data.

[0038] In some embodiments, when the focus model is applicable to optical elements of different models, inputting the image into a pre-trained focus model to obtain an output of the focus model further includes: The image and the optical parameters of the first optical element are input into a pre-trained focus model.

[0039] When the focus model is applicable to optical elements of different models, the optical parameters of the first optical element are used to calculate the quality data of the current first optical element. In addition, for the same type of optical element, due to the manufacturing tolerances of different batches, installation tolerances and changes in the matching between optical elements, focusing is also required. When the focus model is applicable to a single type of optical element, there is no need to input additional optical parameters into the focus model.

[0040] In some embodiments, the method focuses on a training method of a model, including: Acquire a sample data set, each piece of data in the sample data set includes an image of the second graticule, a distance between a collection position of the second detection assembly and a focal position of the second optical element, and a direction of the collection position relative to the focal position of the second optical element; A pre-built second deep learning model is trained based on the sample data set to obtain a focus model.

[0041] In some embodiments, please refer to Figure 4 The second deep learning model includes a quality data extraction algorithm, a feature extraction layer and a data regression layer, wherein the quality data extraction algorithm uses the conventional slit method to calculate the MTF sequence, or uses the star point method to extract quality data. The feature extraction layer extracts the grayscale distribution characteristics of the image layer by layer through continuous convolution blocks or residual blocks. Through network learning, the model can identify the different degradation feature vectors exhibited by the image in the in-focus state and out-of-focus state, so that the model can understand the changes in image quality. The output of this part is a one-dimensional feature vector. The data regression layer has a longer length than the first deep learning model. Based on the splicing of quality data and image feature data, the data volume is continuously reduced layer by layer through a fully connected neural network, and finally the spliced ​​data is mapped to the position difference and direction value between the current image position and the in-focus position.

[0042] When training the second deep learning model, the required sample data set is the same as the sample data set when training the first deep learning model, and the collection method can also be the same, which will not be repeated here. Due to the difference in the architecture of the learning model, the focus model obtained by training the second deep learning model can analyze image features and image quality at the same time, integrate data from more dimensions, and have higher analysis accuracy.

[0043] The focusing method of the optical device of the present application includes: obtaining an image of a first grating plate collected by a first detection component; inputting the image into a pre-trained focusing model to obtain the output of the focusing model, the output including the distance between the current position of the first detection component and the quasi-focus position of the first optical element, and the direction of the current position relative to the quasi-focus position; adjusting the position of the first detection component according to the distance and direction to move the first detection component to the quasi-focus position. In the present application, combined with the pre-trained focusing model, only one image needs to be taken during the focusing process to obtain defocus related information, with high focusing efficiency and low equipment cost.

[0044] The present application also provides a focusing device for an optical device, the optical device comprising a first optical element and a first detection assembly, when focusing, a first graticule is placed on the object plane of the first optical element in advance, and the first detection assembly is placed within the defocus range of the rear surface of the first optical element. The focusing device comprises a moving mechanism and a controller, the moving mechanism is connected to the controller, and is used to adjust the position of the first detection assembly according to the instructions of the controller.

[0045] The controller is used to execute the steps of the method described above. The specific implementation process of the method executed by the controller is detailed in the description of the method embodiment, which will not be repeated here.

[0046] The present application also provides an electronic device, including a storage medium and a controller, wherein a computer program is stored on the storage medium, and when the computer program is executed by the controller, the steps of the method described in the above embodiment are implemented.

[0047] The present application also provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in the above embodiment are implemented.

[0048] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A focusing method for an optical device, characterized in that: The optical device comprises a first optical element and a first detection assembly. When focusing, a first graticule is placed on the object plane of the first optical element in advance, and the first detection assembly is placed within the defocus range of the rear surface of the first optical element. The focusing method comprises: Acquire an image of the first reticle collected by the first detection component; Inputting the image into a pre-trained focus model to obtain an output of the focus model, wherein the output includes a distance between a current position of the first detection component and a focus position of the first optical element, and a direction of the current position relative to the focus position; The position of the first detection component is adjusted according to the distance and the direction, so that the first detection component moves to the quasi-focus position.

2. The focusing method of an optical device according to claim 1, characterized in that: The first graticule has cross slits or star-point holes.

3. The focusing method of an optical device according to claim 1, characterized in that: The training method of the focus model includes: Acquire a sample data set, each piece of data in the sample data set includes an image of the second graticule, a distance between a collection position of the second detection assembly and a focal position of the second optical element, and a direction of the collection position relative to the focal position of the second optical element; A pre-built first deep learning model is trained based on the sample data set to obtain the focusing model, wherein the first deep learning model includes a feature extraction layer and a data regression layer.

4. The focusing method of an optical device according to claim 1, characterized in that: The focusing model includes a quality data extraction algorithm, a feature extraction layer and a data regression layer. The inputting the image into the pre-trained focusing model to obtain the output of the focusing model includes: Inputting the image into a pre-trained focus model, acquiring quality data of the image through the quality data extraction algorithm analysis, and extracting image feature data of the image through the feature extraction layer; The quality data and the image feature data are concatenated and used together as inputs of the data regression layer to obtain the output of the focus model.

5. The focusing method of an optical device according to claim 4, characterized in that: The focusing model is applicable to optical elements of different models, and the image is input into a pre-trained focusing model to obtain an output of the focusing model, and further includes: The image and optical parameters of the first optical element are input into a pre-trained focus model.

6. The focusing method of an optical device according to claim 4, characterized in that: The method of training the focus model includes: Acquire a sample data set, each piece of data in the sample data set includes an image of the second graticule, a distance between a collection position of the second detection assembly and a focal position of the second optical element, and a direction of the collection position relative to the focal position of the second optical element; A pre-built second deep learning model is trained based on the sample data set to obtain the focus model, wherein the second deep learning model includes a quality data extraction algorithm, a feature extraction layer and a data regression layer.

7. The focusing method of an optical device according to claim 3 or 5, characterized in that: The method for collecting the sample data set includes: adjusting the second detection assembly to a focal position of the second optical element; placing the second reticle on the object plane of the second optical element; Adjusting the position of the second detection assembly on the plane where it is located so that the imaging area of ​​the second reticle is located at the detection center of the second detection assembly; Adjusting the position of the second detection assembly at a preset step value within the defocus range of the second optical element to collect images of the second reticle at different positions, and recording the distance between the collection position of the corresponding image and the in-focus position of the second optical element, and the direction of the collection position relative to the in-focus position of the second optical element; The data of different acquisition positions of different second optical elements are sorted to obtain the sample data set.

8. A focusing device for an optical device, characterized in that: The optical device includes a first optical element and a first detection assembly. When focusing, a first graticule is placed on the object plane of the first optical element in advance, and the first detection assembly is placed within the defocus range of the rear surface of the first optical element. The focusing device includes a moving mechanism and a controller. The moving mechanism is connected to the controller and is used to adjust the position of the first detection assembly according to the instruction of the controller. The controller is configured to execute the steps of the method according to any one of claims 1 to 7.

9. An electronic device, comprising a storage medium and a controller, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the controller, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.

Images