Focus detection method and focus detection device of imaging system and storage medium
By arranging the image sharpness of multiple imaging sensors in the imaging system, and combining the sensor position to analyze the focus state of the object to be measured, the problem of introducing miscellaneous light at the focus wavelength is solved, and the focus performance of the imaging system is improved.
Patent Information
- Application Number
- CN202411228492.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-02
- Publication Date
- 2025-07-29
AI Technical Summary
The existing imaging system introduces focus wavelengths to analyze the focus detection results of the object to be measured, resulting in additional fuzzy light and system complexity increasing, reducing the accuracy of the focus detection results.
Multiple imaging sensors are used to distribute the image along the imaging optical path of the object to be measured in sequence, and images are collected at different imaging positions respectively. The focus detection results of the object to be measured are analyzed through image sharpness and sensor position, without introducing focus wavelength and additional modules.
Improve the focus performance of the imaging system to ensure the accuracy of the focus detection results without increasing the system complexity.
Smart Images
Figure CN120390079A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of imaging technologies, and particularly to a focus detection method, a focus detection device and a storage medium for an imaging system. Background Art
[0002] An imaging system includes an imaging sensor. It is necessary to place the object to be measured within the depth of focus of the imaging sensor so that the object to be measured can be clearly imaged. Therefore, it is very important to analyze the focus detection result of the object to be measured in the imaging system.
[0003] In related technologies, the focus detection method of a line scan imaging system generally refers to: configuring a focus detection system in the line scan imaging system to introduce a focus detection wavelength and using the triangulation method to analyze the focus detection result of the object to be measured, so as to realize the focus detection of the line scan imaging system.
[0004] However, the inventors have found through research that the above method introduces a focus detection wavelength through a focus detection system to analyze the focus detection result of the object to be measured. The introduction of the focus detection wavelength by an additional module generates additional stray light, reducing the accuracy of analyzing the focus detection result of the object to be measured. Summary of the Invention
[0005] In view of this, embodiments of the present application provide a focus detection method, a focus detection device and a storage medium for an imaging system. On the basis of adding an imaging sensor without affecting the imaging function of the imaging system, there is no need to introduce a focus detection wavelength, there is no additional stray light and additional module, and there is no need to increase the complexity of the entire system. By collecting the image sharpness of multiple images to be measured of the same image by multiple imaging sensors, which represents the imaging clarity of the object to be measured when imaging separately on multiple imaging sensors, and combining the imaging positions of the imaging sensors corresponding to the multiple images to be measured respectively, the focus detection result of the object to be measured can be accurately analyzed, thereby improving the focus detection performance of the imaging system.
[0006] In a first aspect, embodiments of the present application provide a focus detection method for an imaging system. The imaging system includes multiple imaging sensors, and the multiple imaging sensors are sequentially staggered along the imaging optical path of the object to be measured and are used to collect images of the object to be measured at different imaging positions. The focus detection method includes:
[0007] Obtaining images of the object to be measured from each of the imaging sensors respectively according to a preset imaging acquisition rule, and obtaining multiple images to be measured; the preset imaging acquisition rule includes sequentially enabling time nodes for each of the imaging sensors to collect the same image; performing gradient calculation on the multiple images to be measured respectively to obtain the image sharpness corresponding to each of the multiple images to be measured; and analyzing to obtain the focus detection result of the object to be measured according to the image sharpness corresponding to each of the multiple images to be measured and the imaging positions of the imaging sensors corresponding to the multiple images to be measured respectively.
[0008] Further, analyzing the focus detection result of the object to be measured based on the image sharpness corresponding to each of the multiple images to be measured and the imaging positions of the imaging sensors corresponding to each of the multiple images to be measured includes: performing data fitting on the image sharpness corresponding to each of the multiple images to be measured based on the imaging positions of the imaging sensors corresponding to each of the multiple images to be measured to generate a sharpness curve; determining the imaging position corresponding to the maximum image sharpness in the sharpness curve as the first position; determining the standard imaging position of the object to be measured as the second position; the standard imaging position is the imaging position where the object to be measured is best imaged clearly when the object to be measured is at the front focal plane of the imaging system; and determining the focus detection result of the object to be measured according to the positional relationship between the first position and the second position.
[0009] Further, the step of determining the standard imaging position includes: determining one imaging sensor from the multiple imaging sensors as a reference sensor; the reference sensor is at the imaging plane of the imaging system and is used to best image the object to be measured clearly when the object to be measured is at the front focal plane of the imaging system; and determining the imaging position of the reference sensor as the standard imaging position.
[0010] Further, determining the focus detection result of the object to be measured according to the positional relationship between the first position and the second position includes: if the positional relationship indicates that the first position is less than the second position, determining that the focus detection result is that the object to be measured is in a positive defocus state; if the positional relationship indicates that the first position is equal to the second position, determining that the focus detection result is that the object to be measured is in an in-focus state; if the positional relationship indicates that the first position is greater than the second position, determining that the focus detection result is that the object to be measured is in a negative defocus state; obtaining the axial magnification of the imaging system; and determining the defocus amount of the object to be measured by taking the quotient of the difference between the first position and the second position divided by the axial magnification.
[0011] Further, calculating the gradient of each of the multiple images to be measured to obtain the image sharpness corresponding to each of the multiple images to be measured includes: for each image to be measured, obtaining the gray values of each pixel in the image to be measured, calculating the gradient of each pixel in the image to be measured in combination with a preset gradient operator, and obtaining the image sharpness of the image to be measured through fusion processing; wherein the preset gradient operators include Sobel operator, Tenengrad operator, and Laplacian operator.
[0012] Further, obtaining the acquired images from each of the imaging sensors according to a preset imaging acquisition rule to obtain a plurality of images to be measured includes: obtaining a plurality of time nodes and forming the preset imaging acquisition rule, each time node being configured as the quotient of the imaging position of the corresponding imaging sensor and the imaging scanning line frequency and the imaging pixel size, and the plurality of time nodes being configured to progress in the staggered distribution order of each imaging sensor on the imaging optical path of the object to be measured; enabling the corresponding imaging sensor based on each time node in the preset imaging acquisition rule, and acquiring the corresponding image to be measured until all time nodes are traversed to complete one round of image acquisition.
[0013] In a second aspect, an in-focus detection device for an imaging system provided by an embodiment of the present application, the imaging system includes a plurality of imaging sensors, and the plurality of imaging sensors are sequentially and staggeredly distributed along the imaging optical path of an object to be measured for respectively acquiring images of the object to be measured at different imaging positions; the in-focus detection device includes: an acquisition unit, a calculation unit, and a determination unit;
[0014] The acquisition unit is configured to obtain the acquired images from each of the imaging sensors according to a preset imaging acquisition rule to obtain a plurality of images to be measured; the preset imaging acquisition rule includes time nodes for sequentially enabling each of the imaging sensors to acquire the same image; the calculation unit is configured to perform gradient calculation on the plurality of images to be measured respectively to obtain the image sharpness corresponding to each of the plurality of images to be measured; the determination unit is configured to analyze the in-focus detection result of the object to be measured according to the image sharpness corresponding to each of the plurality of images to be measured and the imaging positions of the imaging sensors corresponding to each of the plurality of images to be measured.
[0015] Further, the imaging sensor is a time delay integration (TDI) sensor.
[0016] Further, the process of the acquisition unit obtaining a plurality of images to be measured specifically includes: the acquisition unit obtains a plurality of time nodes and forms the preset imaging acquisition rule, each time node being configured as the quotient of the imaging position of the corresponding imaging sensor and the imaging scanning line frequency and the imaging pixel size, and the plurality of time nodes being configured to progress in the staggered distribution order of each imaging sensor on the imaging optical path of the object to be measured; the acquisition unit enables the corresponding imaging sensor based on each time node in the preset imaging acquisition rule, and acquires the corresponding image to be measured until all time nodes are traversed to complete one round of image acquisition.
[0017] Further, the calculation unit obtains the image sharpness corresponding to each of the images to be measured, specifically including: for each of the images to be measured, the calculation unit obtains the gray values of each pixel in the image to be measured, calculates the gradient of each pixel in the image to be measured by combining a preset gradient operator, and obtains the image sharpness of the image to be measured through fusion processing; wherein, the preset gradient operators include Sobel operator, Tenengrad operator, and Laplacian operator.
[0018] Further, the determination unit obtains the focusing detection result of the object to be measured, specifically including: the determination unit performs data fitting on the image sharpness corresponding to each of the images to be measured based on the imaging positions of the imaging sensors corresponding to each of the images to be measured, and generates a sharpness curve; the determination unit determines the imaging position corresponding to the maximum image sharpness in the sharpness curve as the first position; the determination unit determines the standard imaging position of the object to be measured as the second position; the standard imaging position is the imaging position at which the object to be measured is best imaged clearly when the object to be measured is at the front focal plane of the imaging system; the determination unit determines the focusing detection result of the object to be measured according to the positional relationship between the first position and the second position.
[0019] In a third aspect, an embodiment of the present application discloses a computer-readable storage medium, which is used to store a computer program. When the computer program is executed by a processor, it is used to implement the focusing detection method mentioned in the first aspect above.
[0020] Compared with the prior art, the present application has at least the following advantages:
[0021] Adopting the technical solution of the embodiment of the present application, the imaging system includes a plurality of imaging sensors that are arranged staggeredly in sequence along the imaging optical path of the object to be measured, and are used to respectively collect images of the object to be measured at different imaging positions. Based on this, first, according to the preset imaging acquisition rule formed by the time nodes when the plurality of imaging sensors collect the same image, obtain the images of the object to be measured from the plurality of imaging sensors respectively to obtain a plurality of images to be measured; then, by respectively calculating the image gradients of the plurality of images to be measured, obtain the corresponding image sharpnesses respectively corresponding to the plurality of images to be measured; finally, analyze the image sharpnesses respectively corresponding to the plurality of images to be measured and the imaging positions of the imaging sensors respectively corresponding to the plurality of images to be measured to obtain the focusing detection result of the object to be measured. This method, without affecting the imaging function of the imaging system by adding imaging sensors, does not require introducing a focusing detection wavelength, there is no additional stray light and additional modules, and does not increase the complexity of the entire system. By using the image sharpnesses of the plurality of images to be measured collected by the plurality of imaging sensors for the same image to represent the imaging clarity when the object to be measured is imaged by the plurality of imaging sensors respectively, and combining the imaging positions of the imaging sensors respectively corresponding to the plurality of images to be measured, the focusing detection result of the object to be measured can be accurately analyzed, thereby improving the focusing detection performance of the imaging system. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the technical solution of the embodiment of the present application, the following will briefly introduce the drawings required for the description of the embodiment of the present application. Obviously, the drawings in the following description are only some embodiments recorded in the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.
[0023] Figure 1 It is a schematic diagram of the system framework involved in an application scenario in the embodiment of the present application;
[0024] Figure 2 It is a schematic flowchart of a focusing detection method for an imaging system provided by the embodiment of the present application;
[0025] Figure 3 It is a schematic diagram of the distribution of a plurality of imaging sensors in an imaging system provided by the embodiment of the present application;
[0026] Figure 4 It is a schematic diagram of a sharpness curve when the object to be measured is in a positive defocus state, in-focus state, and negative defocus state provided by the embodiment of the present application;
[0027] Figure 5 It is a schematic diagram of the structure of a focusing detection device for an imaging system provided by the embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0028] To enable those skilled in the art to better understand the solution of this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of this application.
[0029] The imaging system includes an imaging sensor. The focus detection method of the imaging system generally refers to: configuring a focus detection system in the imaging system to introduce a focus detection wavelength and using the triangulation method to analyze the focus detection results of the object to be measured, so as to achieve the focus detection of the imaging system. However, the inventor has found through research that by introducing a focus detection wavelength through the focus detection system to analyze the focus detection results of the object to be measured, the additional module introducing the focus detection wavelength generates additional stray light, increases the complexity of the entire system, and reduces the accuracy of analyzing the focus detection results of the object to be measured.
[0030] To solve this problem, in the embodiments of this application, on the basis that adding an imaging sensor does not affect the imaging function of the imaging system, there is no need to introduce a focus detection wavelength, there is no additional stray light and additional module, and there is no need to increase the complexity of the entire system. By collecting the image sharpness of multiple images to be measured of the same image by multiple imaging sensors, which represents the imaging clarity of the object to be measured when imaging respectively by multiple imaging sensors, and combining the imaging positions of the imaging sensors corresponding to the multiple images to be measured respectively, the focus detection results of the object to be measured can be accurately analyzed, thereby improving the focus detection performance of the imaging system.
[0031] For example, one of the scenarios of the embodiments of this application can be applied to, for example Figure 1 the scenario shown. This scenario includes an imaging system 101 and a focus detection system 102. The imaging system 101 conforms to the implementation manner provided in the embodiments of this application, and the focus detection system 102 executes the implementation manner provided in the embodiments of this application to analyze the focus detection results of the object to be measured in the imaging system 101.
[0032] First, in the above application scenario, although the action description of the implementation manner provided in the embodiments of this application is executed by the focus detection system 102; however, the embodiments of this application are not limited in terms of the execution subject, as long as the actions disclosed in the implementation manner provided in the embodiments of this application are executed.
[0033] Second, the above scenario is only a scenario example provided by the embodiments of this application, and the embodiments of this application are not limited to this scenario.
[0034] The following will describe in detail the specific implementation manners of the defocus amount determination method and related devices of the imaging system in the embodiments of this application in conjunction with the accompanying drawings through embodiments.
[0035] First, the specific implementation of the focus detection method for the imaging system in the embodiments of the present application will be described in detail through examples.
[0036] See Figure 2 , which shows a schematic flow chart of a focus detection method for an imaging system in the embodiments of the present application. In this embodiment, the imaging system includes a plurality of imaging sensors, and the plurality of imaging sensors are arranged in a staggered manner in sequence along the imaging optical path of the object to be measured, and are used to respectively collect images of the object to be measured at different imaging positions; the method may include, for example:
[0037] S201: Obtain images of the object to be measured collected by the plurality of imaging sensors respectively according to a preset imaging acquisition rule, and obtain a plurality of images to be measured; the preset imaging acquisition rule includes time nodes at which the plurality of imaging sensors collect the same image.
[0038] In the related art, additional modules in the imaging system introduce a focus detection wavelength to generate additional stray light, which increases the complexity of the entire system and reduces the accuracy of analyzing the focus detection results of the object to be measured. Therefore, in the embodiments of the present application, to solve this problem, on the one hand, considering that there are imaging sensors in the imaging system, adding imaging sensors does not affect the imaging function of the imaging system; on the other hand, when the plurality of imaging sensors are arranged in a staggered manner in sequence along the imaging optical path of the object to be measured, the image sharpness of the plurality of images of the object to be measured collected by the plurality of imaging sensors at different imaging positions represents the imaging clarity when the object to be measured is imaged by the plurality of imaging sensors respectively. Considering that the object to be measured has the highest imaging clarity at the imaging position corresponding to the maximum sharpness, by combining the plurality of imaging sensors at different imaging positions, it is possible to accurately analyze the focus detection results of the object to be measured without introducing a focus detection wavelength and without additional stray light and additional modules, so as to improve the focus detection performance of the imaging system.
[0039] Among them, the plurality of imaging sensors include a preset sensor originally in the imaging system and N target sensors added, and the N target sensors are symmetrically distributed about the preset sensor along the imaging optical path of the object to be measured, and the length of each target sensor is less than the length of the preset sensor.
[0040] See Figure 3 , Figure 3 is a schematic diagram of the distribution of a plurality of imaging sensors in an imaging system provided by an embodiment of the present application; among them, the plurality of imaging sensors include a preset sensor T, target sensor 1, target sensor 2, target sensor 3, and target sensor 4; Figure 3(a) shows a front view of the imaging system, including the preset sensor T, target sensor 1, target sensor 2, target sensor 3, and target sensor 4. The Y-axis is the vertical direction, with target sensors 1 and 4 symmetrically positioned about the preset sensor T. Target sensors 2 and 3 are also symmetrically positioned about the preset sensor T. The X-axis is the horizontal direction, with the length of each target sensor 1, target sensor 2, target sensor 3, and target sensor 4 along the X-axis being shorter than the length of the preset sensor T. The imaging optical path of the object under test is along the Z-axis, with the Y-axis, X-axis, and Z-axis being perpendicular to each other.
[0041] Figure 3 (b) shows a side view of the preset sensor T, target sensor 1, target sensor 2, target sensor 3 and target sensor 4 in the imaging system. On the Z axis, the preset sensor T, target sensor 1, target sensor 2, target sensor 3 and target sensor 4 are in different planes on the Z axis and staggered with each other, so as to respectively capture images of the object to be measured at different imaging positions.
[0042] Based on this, in an embodiment of the present application, first, it is necessary to acquire images of the object to be tested from multiple imaging sensors respectively to obtain multiple images to be tested; further considering that multiple imaging sensors acquire images of the object to be tested at different imaging positions respectively, so that the same pixel of the object to be tested is imaged at different time nodes by multiple imaging sensors; therefore, it is necessary to acquire images of the object to be tested from multiple imaging sensors respectively according to preset imaging acquisition rules formed at the time nodes when multiple imaging sensors acquire the same image, in order to obtain multiple images to be tested.
[0043] The time points at which the multiple imaging sensors capture the same image are related to the relative distances between the multiple imaging sensors along the imaging optical path of the object to be measured.
[0044] As an example of S201, in the above Figure 3 On the basis of the time nodes of target sensor 1, target sensor 2, preset sensor T, target sensor 3 and target sensor 4 collecting the same image, for example, t1, t2, t T , t3 and t4, obtain the images of the object to be measured collected by target sensor 1, target sensor 2, preset sensor T, target sensor 3 and target sensor 4, and obtain 5 images to be measured, for example, I1, I2, I T , I3 and I4.
[0045] S202: performing gradient calculation on the plurality of images to be tested respectively to obtain image sharpness corresponding to the plurality of images to be tested respectively.
[0046] In the embodiments of the present application, after obtaining multiple images of the object to be measured from multiple imaging sensors according to the preset imaging acquisition rule formed at the time nodes when the multiple imaging sensors collect the same image in S201, for the multiple images, it is necessary to calculate the image sharpness indicating the imaging clarity of the object to be measured when imaging on the multiple imaging sensors respectively. Further considering that the larger the image gradient, the clearer the image, and the smaller the image gradient, the blurrier the image, that is, the image gradient represents the image clarity, which is equivalent to the image sharpness; therefore, by calculating the image gradients of the multiple images of the object to be measured respectively, the image sharpness corresponding to each of the multiple images of the object to be measured can be obtained to represent the imaging clarity of the object to be measured when imaging on the multiple imaging sensors respectively.
[0047] As an example of S202, based on the example of S201 above, by calculating the image gradients of I1, I2, I T , I3 and I4, the image sharpness corresponding to I1, I2, I T , I3 and I4 respectively is obtained, for example, sharpness1, sharpness2, sharpness T , sharpness3 and sharpness4.
[0048] S203: Analyze the image sharpness corresponding to each of the multiple images of the object to be measured and the imaging positions of the imaging sensors corresponding to each of the multiple images of the object to be measured to obtain the focus detection result of the object to be measured.
[0049] In the embodiments of the present application, after calculating the image gradients of the multiple images of the object to be measured respectively in S202 to obtain the image sharpness corresponding to each of the multiple images of the object to be measured, since the image sharpness corresponding to each of the multiple images of the object to be measured represents the imaging clarity of the object to be measured when imaging on the multiple imaging sensors respectively, and the imaging positions of the imaging sensors corresponding to each of the multiple images of the object to be measured are different, considering that in this case, the object to be measured has the maximum imaging clarity when imaging at the imaging position of the imaging sensor corresponding to the maximum sharpness; therefore, by analyzing the image sharpness corresponding to each of the multiple images of the object to be measured and the imaging positions of the imaging sensors corresponding to each of the multiple images of the object to be measured, the focus detection result of the object to be measured can be obtained.
[0050] As an example of S203, based on the example of S202 above, the imaging positions of target sensor 1, target sensor 2, preset sensor T, target sensor 3 and target sensor 4 are z1, z2, z T , z3 and z4; through sharpness1, sharpness2, sharpness T , sharpness3, sharpness4, z1, z2, z T, z3, and z4 to determine the focusing detection result of the object to be measured.
[0051] Through various embodiments provided in this embodiment, the imaging system includes a plurality of imaging sensors arranged in a staggered manner in sequence along the imaging optical path of the object to be measured, and is configured to respectively acquire images of the object to be measured at different imaging positions. Based on this, first, according to a preset imaging acquisition rule formed by the time nodes when the plurality of imaging sensors acquire the same image, obtain a plurality of images to be measured by respectively acquiring images of the object to be measured from the plurality of imaging sensors; then, by respectively calculating the image gradients of the plurality of images to be measured, obtain the corresponding image sharpnesses of the plurality of images to be measured; finally, analyze the image sharpnesses corresponding to the plurality of images to be measured and the imaging positions of the imaging sensors corresponding to the plurality of images to be measured to obtain the focusing detection result of the object to be measured. This method, on the basis of adding imaging sensors without affecting the imaging function of the imaging system, does not require introducing a focusing detection wavelength without additional stray light and additional modules, does not require increasing the complexity of the entire system, and represents the imaging clarity of the object to be measured when it is imaged by the plurality of imaging sensors through the image sharpnesses of the plurality of images to be measured acquired by the plurality of imaging sensors. By combining the imaging positions of the imaging sensors corresponding to the plurality of images to be measured, the focusing detection result of the object to be measured can be accurately analyzed, thereby improving the focusing detection performance of the imaging system.
[0052] In the embodiment of the present application, when specifically implementing S203 above, first, when the image sharpnesses corresponding to the plurality of images to be measured represent the imaging clarity of the object to be measured when it is imaged by the plurality of imaging sensors respectively, and the imaging positions of the imaging sensors corresponding to the plurality of images to be measured are different, it is necessary to determine the imaging position corresponding to the maximum sharpness of the object to be measured in this case; considering that the sharpness curve with the imaging position as the horizontal axis and the image sharpness as the vertical axis can display the maximum sharpness and the imaging position corresponding to the maximum sharpness, first fit the image sharpnesses corresponding to the plurality of images to be measured to a sharpness curve through the imaging positions of the imaging sensors corresponding to the plurality of images to be measured, and then determine the imaging position corresponding to the maximum sharpness as the first position through this sharpness curve; on the basis that the imaging position when the object to be measured is at the front focal plane of the imaging system is used as the second position when the object to be measured is optimally clear, the focusing detection result of the object to be measured can be determined through the positional relationship between the first position and the second position. Based on this, in an optional embodiment of the embodiment of the present application, S203 above includes the following S2031 - S2034 (not shown in the figure):
[0053] S2031: Based on the imaging positions of the imaging sensors corresponding to the plurality of images to be measured, perform data fitting on the image sharpnesses corresponding to the plurality of images to be measured to generate a sharpness curve.
[0054] S2032: Determine the imaging position corresponding to the maximum image sharpness from the sharpness curve as the first position.
[0055] S2033: Determine the standard imaging position of the object to be measured as the second position; the standard imaging position is the position where the object to be measured is imaged most clearly when the object to be measured is in the front focal plane of the imaging system.
[0056] S2034: Determine the focusing detection result of the object to be measured according to the positional relationship between the first position and the second position.
[0057] This method fits the sharpness curve through the imaging positions and image sharpness of the imaging sensors corresponding to multiple images to be measured to determine the imaging position corresponding to the maximum sharpness, and can relatively accurately obtain the imaging position with the maximum imaging clarity. Combining the position where the object to be measured is imaged most clearly when the object to be measured is in the front focal plane of the imaging system, the focusing detection result of the object to be measured can be relatively accurately determined.
[0058] As an example of S2031 - S2034, on the basis of the above example of S203, first fit the sharpness curve through (z1, sharpness1), (z2, sharpness2), (z T , sharpness T ), (z3, sharpness3) and (z4, sharpness4); then determine the imaging position corresponding to the maximum sharpness sharpness max as the first position z max ; when the object to be measured is in the front focal plane of the imaging system, the position where the object to be measured is imaged most clearly is the second position z standard . On this basis, through the positional relationship between z max and z standard , determine the focusing detection result of the object to be measured.
[0059] Among them, considering that there is an imaging sensor among multiple imaging sensors that is in the imaging plane of the imaging system and is used to image the object to be measured most clearly when the object to be measured is in the front focal plane of the imaging system, this imaging sensor is used as the reference sensor, and the standard imaging position can be determined based on the imaging position of this reference sensor, which is the standard imaging position in the above S2033. Based on this, in an optional implementation manner of the embodiments of the present application, the steps for determining the standard imaging position in the above S2033 include the following S1 - S2 (not shown in the figure):
[0060] S1: Determine one imaging sensor from multiple imaging sensors as the reference sensor; the reference sensor is in the imaging plane of the imaging system and is used to image the object to be measured most clearly when the object to be measured is in the front focal plane of the imaging system.
[0061] S2: Determine the imaging position of the reference sensor as the standard imaging position.
[0062] Among them, when specifically implementing the above S2034, since the first position is the imaging position corresponding to the maximum sharpness of the object to be measured, and the second position is the best clear imaging position of the object to be measured when the object to be measured is in the front focal plane of the imaging system. When the first position is less than the second position, it indicates that the object to be measured is in a positive defocus state, and the focus detection result is that the object to be measured is in a positive defocus state; when the first position is equal to the second position, it indicates that the object to be measured is in the in-focus state, and the focus detection result is that the object to be measured is in the in-focus state; when the first position is greater than the second position, it indicates that the object to be measured is in a negative defocus state, and the focus detection result is that the object to be measured is in a negative defocus state.
[0063] In addition, considering the difference between the imaging position corresponding to the maximum sharpness of the object to be measured and the best clear imaging position of the object to be measured when the object to be measured is in the front focal plane of the imaging system, it can accurately represent the deviation value of the best clear imaging position of the object to be measured relative to the imaging position corresponding to the maximum sharpness of the object to be measured when the object to be measured is in the front focal plane of the imaging system. Based on this, the quotient of the difference between the first position and the second position divided by the axial magnification is determined as the defocus amount of the object to be measured.
[0064] Therefore, in an alternative implementation manner of the embodiments of the present application, the above S2033 includes the following S3 - S6 (not shown in the figure):
[0065] S3: If the position relationship indicates that the first position is less than the second position, determine that the focus detection result is that the object to be measured is in a positive defocus state.
[0066] S4: If the position relationship indicates that the first position is equal to the second position, determine that the focus detection result is that the object to be measured is in the in-focus state.
[0067] S5: If the position relationship indicates that the first position is greater than the second position, determine that the focus detection result is that the object to be measured is in a negative defocus state.
[0068] S6: Obtain the axial magnification of the imaging system; use the difference between the first position and the second position, and the quotient divided by the axial magnification is determined as the defocus amount of the object to be measured.
[0069] As an example, the calculation formula for the defocus amount d of the object to be measured is as follows:
[0070] d = (z max - z standard ) / M
[0071] Among them, M is the axial magnification of the imaging system.
[0072] SeeFigure 4 , Figure 4 is a schematic diagram of sharpness curves when the object to be measured is in the positive defocus state, in-focus state, and negative defocus state provided by the embodiment of the present application; based on the above Figure 3 and the example of the above S203, Figure 4 in (a), it shows that as z1, z2, z T , z3, and z4 increase monotonically, sharpness1, sharpness2, sharpness T , sharpness3, and sharpness4 decrease monotonically, and the object to be measured is in the positive defocus state; Figure 4 in (b), it shows that as z1, z2, z T , z3, and z4 increase monotonically, sharpness T is the largest, sharpness1 and sharpness4 are similar, and sharpness2 and sharpness3 are similar, and the object to be measured is in the in-focus state; Figure 4 in (c), it shows that as z1, z2, z T , z3, and z4 increase monotonically, sharpness1, sharpness2, sharpness T , sharpness3, and sharpness4 increase monotonically, and the object to be measured is in the negative defocus state.
[0073] In the embodiment of the present application, when the above S202 is specifically implemented, considering that the gradient of the image is determined by the gray values of each pixel in the image and a preset gradient operator for edge detection; therefore, for each of the multiple images to be measured, the gradient of each pixel in the image to be measured can be accurately calculated through the gray values of each pixel in the image to be measured and the preset gradient operator, and the image sharpness of the image to be measured is obtained through fusion processing. Based on this, in an optional implementation manner of the embodiment of the present application, the above S202 includes: for each image to be measured, obtaining the gray values of each pixel in the image to be measured, calculating the gradient of each pixel in the image to be measured in combination with the preset gradient operator, and obtaining the image sharpness of the image to be measured through fusion processing; where the preset gradient operator includes the Sobel operator, the Tenengrad operator based on the Sobel operator, and the Laplacian operator.
[0074] As an example, based on the example of the above S202, the preset gradient operator is the Sobel operator, including d x and d y ; for each of the images to be measured I1, I2, I T , I3, and I4, through each pixel point (i, j) in the image to be measured, d x and dy The gray value of the (i, j) includes I(i - 1, j - 1), I(i, j - 1), I(i + 1, j - 1), I(i - 1, j), I(i + 1, j), I(i - 1, j + 1), I(i, j + 1), and I(i + 1, j + 1). Calculate the gradient of the image to be measured, and obtain the sharpness of the image to be measured through fusion processing. Among them, the calculation formula of sharpness is as follows:
[0075]
[0076] d x (i,j) = [I(i - 1, j - 1) + 2I(i - 1, j) + I(i - 1, j + 1)] - [I(i + 1, j - 1) + 2I(i
[0077] + 1, j) + I(i + 1, j + 1)]
[0078] d y (i,j) = [I(i - 1, j + 1) + 2I(i, j + 1) + I(i + 1, j + 1)] - [I(i - 1, j - 1) + 2I(i, j
[0079] - 1) + I(i + 1, j - 1)]
[0080]
[0081] In the embodiment of the present application, when specifically implementing S202, considering the time nodes for multiple imaging sensors to collect the same image, the imaging positions of multiple imaging sensors, and their relationship with the line frequency and imaging pixel size; therefore, first, obtain the quotient of the imaging position of each imaging sensor corresponding to each time node and the imaging scan line frequency and imaging pixel size. Multiple time nodes are configured to progress in the staggered distribution order of each imaging sensor on the imaging light path of the object to be measured, forming a preset imaging acquisition rule; then, enable the corresponding imaging sensor based on each time node in the preset imaging acquisition rule to collect the corresponding image to be measured until all time nodes are traversed to complete one round of image acquisition. Based on this, in an optional implementation manner of the embodiment of the present application, the above S201 includes the following S2011 - S2012 (not shown in the figure):
[0082] S2011: Obtain multiple time nodes and form them into a preset imaging acquisition rule. Each time node is configured to be the quotient of the imaging position of the corresponding imaging sensor and the imaging scan line frequency and imaging pixel size. Multiple time nodes are configured to progress in the staggered distribution order of each imaging sensor on the imaging light path of the object to be measured.
[0083] S2012: Enable the corresponding imaging sensor according to each time node in the preset imaging acquisition rule, and acquire the corresponding image to be measured until all time nodes are traversed to complete one round of image acquisition.
[0084] As an example, in the above Figure 3 the relative distances of the imaging positions corresponding to the target sensor 1, target sensor 2, preset sensor T, target sensor 3, and target sensor 4 include y1, y2, y T , y3, and y4, the line frequency is f, and the imaging pixel size is p; based on the example of S201 above, the 5 time nodes corresponding to the target sensor 1, target sensor 2, preset sensor T, target sensor 3, and target sensor 4, namely, t1, t2, t T , t3, and t4 are shown in the following specific calculation formulas in the table:
[0085] Table 1
[0086] Target Sensor 1 Target Sensor 2 Preset Sensor T Target Sensor 3 Target Sensor 4 <![CDATA[t1 = y1 / (fp)]]> <![CDATA[t2 = y2 / (fp)]]> <![CDATA[t T = y T / (fp)]]> <![CDATA[t3 = y3 / (fp)]]> <![CDATA[t4 = y4 / (fp)]]>
[0087] As can be seen from Table 1, the time interval between the time node corresponding to the target sensor 2 and the time node corresponding to the target sensor 1 is (y2 - y1) / (fp), and the time interval between the time node corresponding to the preset sensor T and the time node corresponding to the target sensor 2 is (y T - y2) / (fp), the time interval between the time node corresponding to the target sensor 3 and the time node corresponding to the preset sensor T is (y3 - y T ) / (fp), and the time interval between the time node corresponding to the target sensor 4 and the time node corresponding to the target sensor 3 is (y4 - y3) / (fp).
[0088] In the embodiments of the present application, since the multiple imaging sensors include the preset sensor originally in the imaging system and the added N target sensors, the N target sensors are symmetrically distributed about the preset sensor along the imaging optical path of the object to be measured, and the length of each target sensor is less than the length of the preset sensor; when N target sensors are greater than or equal to 4 target sensors, the N target sensors can be arranged in a matrix along the imaging optical path of the object to be measured and symmetrically distributed about the preset sensor, so as to avoid that the N target sensors cannot be in different planes in the direction perpendicular to the imaging optical path of the object to be measured vertically. Based on this, in an optional implementation manner of the embodiments of the present application, when N ≥ 4, the N target sensors are matrix-distributed about the preset sensor along the imaging optical path of the object to be measured.
[0089] In the embodiments of the present application, since multiple imaging sensors can be time delay integration (TDI) chips with imaging characteristics, which can further improve the focus detection accuracy of the imaging system; therefore, in an optional implementation manner of the embodiments of the present application, the multiple imaging sensors are all time delay integration (TDI) sensors.
[0090] Next, the specific implementation manner of the device related to the defocus amount determination of the imaging system in the embodiments of the present application will be described in detail through embodiments.
[0091] See Figure 5 , which shows a schematic structural diagram of a focus detection device of an imaging system in the embodiments of the present application. In this embodiment, the imaging system includes multiple imaging sensors, and the multiple imaging sensors are arranged in a staggered manner in sequence along the imaging optical path of the object to be measured, and are used to respectively collect images of the object to be measured at different imaging positions; the focus detection device may specifically include, for example: an acquisition unit 501, a calculation unit 502, and a determination unit 503.
[0092] The acquisition unit 501 is configured to respectively obtain the collected images from each of the imaging sensors according to a preset imaging acquisition rule, so as to obtain multiple images to be measured; the preset imaging acquisition rule includes time nodes for sequentially enabling each of the imaging sensors to collect the same image.
[0093] The calculation unit 502 is configured to perform gradient calculation on the multiple images to be measured respectively, so as to obtain the image sharpness corresponding to each of the multiple images to be measured.
[0094] The determination unit 503 is configured to analyze and obtain the focus detection result of the object to be measured according to the image sharpness corresponding to each of the multiple images to be measured and the imaging positions of the imaging sensors corresponding to the multiple images to be measured.
[0095] In an optional implementation manner of the embodiments of the present application, the process of the acquisition unit 501 obtaining multiple images to be measured specifically includes: the acquisition unit 501 obtains multiple time nodes and forms a preset imaging acquisition rule, and each time node is configured as the quotient of the imaging position of the corresponding imaging sensor and the imaging scanning line frequency and the imaging pixel size, and the multiple time nodes are configured to progress in accordance with the staggered distribution order of each imaging sensor on the imaging optical path of the object to be measured; the acquisition unit 501 enables the corresponding imaging sensor based on each time node in the preset imaging acquisition rule, and acquires the corresponding image to be measured until all time nodes are traversed to complete one round of image acquisition.
[0096] The calculation unit 502 obtains the image sharpness corresponding to each image to be measured, specifically including: for each image to be measured, the calculation unit 502 obtains the gray values of each pixel in the image to be measured, calculates the gradient of each pixel in the image to be measured by combining a preset gradient operator, and obtains the image sharpness of the image to be measured through fusion processing; wherein, the preset gradient operators include Sobel operator, Tenengrad operator, and Laplacian operator.
[0097] The determination unit 503 obtains the focusing detection result of the object to be measured, specifically including: the determination unit 503 performs data fitting on the image sharpness corresponding to each of the multiple images to be measured based on the imaging positions of the imaging sensors corresponding to each of the multiple images to be measured, and generates a sharpness curve; the determination unit 503 determines the imaging position corresponding to the maximum image sharpness in the sharpness curve as the first position; the determination unit 503 determines the standard imaging position of the object to be measured as the second position; the standard imaging position is the imaging position when the object to be measured is at the front focal plane of the imaging system and is the best clear imaging position of the object to be measured; the determination unit 503 determines the focusing detection result of the object to be measured according to the positional relationship between the first position and the second position.
[0098] Of course, for the specific functions of the acquisition unit 501, the calculation unit 502, and the determination unit 503, reference can also be made to the corresponding processing steps of the focus detection method in the above text.
[0099] Through various implementation manners provided in this embodiment, the imaging system includes a plurality of imaging sensors arranged in a staggered manner in sequence along the imaging optical path of the object to be measured, and is used to collect images of the object to be measured at different imaging positions. Based on this, first, according to a preset imaging acquisition rule formed by the time nodes when the plurality of imaging sensors collect the same image, obtain multiple images to be measured by collecting images of the object to be measured from the plurality of imaging sensors respectively; then, by calculating the image gradients of the multiple images to be measured respectively, obtain the image sharpness corresponding to each of the multiple images to be measured; finally, analyze the image sharpness corresponding to each of the multiple images to be measured and the imaging positions of the imaging sensors corresponding to each of the multiple images to be measured, and obtain the focusing detection result of the object to be measured. This method does not affect the imaging function of the imaging system by adding imaging sensors. Without introducing a focus detection wavelength, there is no additional stray light and additional modules, and there is no need to increase the complexity of the entire system. By using the image sharpness of multiple images to be measured collected by a plurality of imaging sensors for the same image to represent the imaging clarity of the object to be measured when imaging by the plurality of imaging sensors respectively, and combining the imaging positions of the imaging sensors corresponding to each of the multiple images to be measured, the focusing detection result of the object to be measured can be accurately analyzed, thereby improving the focus detection performance of the imaging system.
[0100] In addition, an embodiment of the present application further provides a computer device, and the computer device includes a processor and a memory:
[0101] The memory is used to store a computer program and transmit the computer program to the processor;
[0102] The processor is used to execute the method described in the above embodiments according to the instructions in the computer program.
[0103] In addition, an embodiment of the present application further provides a computer-readable storage medium, which is used to store a computer program, and when the computer program is executed by a processor, it is used to implement the method described in the above embodiments.
[0104] In addition, an embodiment of the present application further provides a computer program product, including a computer program, and when the computer program runs on a computer device, it causes the computer device to execute the method described in the above embodiments.
[0105] In this specification, the various embodiments are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method part.
[0106] Those skilled in the art can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been generally described according to their functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.
[0107] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. The term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article or device including the said element.
[0108] The above are only the preferred embodiments of the present application and do not impose any formal restrictions on the present application. Although the present application has been disclosed above with the preferred embodiments, it is not intended to limit the present application. Any person skilled in the art can make many possible changes and modifications to the technical solution of the present application, or modify it into equivalent embodiments with equivalent changes, without departing from the scope of the technical solution of the present application. Therefore, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application still fall within the scope of protection of the technical solution of the present application.
Claims
1. A focusing method for an imaging system, characterized in that, The imaging system includes a plurality of imaging sensors, and the plurality of imaging sensors are arranged in a staggered manner in sequence along the imaging optical path of the object to be measured, and are used to respectively collect images of the object to be measured at different imaging positions; The defocus detection method includes: Obtaining images of the object to be measured from each of the imaging sensors according to a preset imaging acquisition rule, and obtaining a plurality of images to be measured; the preset imaging acquisition rule includes time nodes for sequentially enabling each of the imaging sensors to collect the same image; Performing gradient calculation on the plurality of images to be measured respectively, and obtaining the image sharpness corresponding to each of the plurality of images to be measured; Analyzing based on the image sharpness corresponding to each of the plurality of images to be measured and the imaging positions of the imaging sensors corresponding to each of the plurality of images to be measured, and obtaining a defocus detection result of the object to be measured.
2. The method according to claim 1, wherein The analyzing based on the image sharpness corresponding to each of the plurality of images to be measured and the imaging positions of the imaging sensors corresponding to each of the plurality of images to be measured, and obtaining a defocus detection result of the object to be measured, includes: Based on the imaging positions of the imaging sensors corresponding to each of the plurality of images to be measured, performing data fitting on the image sharpness corresponding to each of the plurality of images to be measured, and generating a sharpness curve; Determining the imaging position corresponding to the maximum image sharpness in the sharpness curve as the first position; Determining the standard imaging position of the object to be measured as the second position; the standard imaging position is the imaging position at which the object to be measured is clearest when the object to be measured is at the front focal plane of the imaging system; Determining the defocus detection result of the object to be measured according to the positional relationship between the first position and the second position.
3. The method according to claim 2, wherein The step of determining the standard imaging position includes: Determining one imaging sensor from the plurality of imaging sensors as a reference sensor; the reference sensor is located at the imaging plane of the imaging system and is used to perform the clearest imaging on the object to be measured when the object to be measured is at the front focal plane of the imaging system; Determining the imaging position of the reference sensor as the standard imaging position.
4. The method according to claim 2, wherein The determining the defocus detection result of the object to be measured according to the positional relationship between the first position and the second position, includes: If the positional relationship indicates that the first position is less than the second position, determining that the defocus detection result is that the object to be measured is in a positive defocus state; If the positional relationship indicates that the first position is equal to the second position, determining that the defocus detection result is that the object to be measured is in an in-focus state; If the positional relationship indicates that the first position is greater than the second position, determining that the defocus detection result is that the object to be measured is in a negative defocus state; Obtaining the axial magnification of the imaging system; using the quotient of the difference between the first position and the second position divided by the axial magnification as the defocus amount of the object to be measured.
5. The method according to claim 1, characterized in that The performing gradient calculation on the plurality of images to be measured respectively, and obtaining the image sharpness corresponding to each of the plurality of images to be measured, includes: For each of the images to be measured, obtain the gray values of each pixel in the image to be measured, calculate the gradient of each pixel in the image to be measured in combination with a preset gradient operator, and obtain the image sharpness of the image to be measured through fusion processing; Among them, the preset gradient operators include Sobel operator, Tenengrad operator, and Laplacian operator.
6. The method according to claim 1, characterized in that, The obtaining of the acquired images from each of the imaging sensors according to a preset imaging acquisition rule to obtain a plurality of images to be measured includes: Obtain a plurality of time nodes and form the preset imaging acquisition rule. Each time node is configured as the quotient of the imaging position of the corresponding imaging sensor and the imaging scan line frequency and the imaging pixel size. The plurality of time nodes are configured to progress in the staggered distribution order of each imaging sensor on the imaging optical path of the object to be measured; Based on each time node in the preset imaging acquisition rule, enable the corresponding imaging sensor to acquire the corresponding image to be measured until all time nodes are traversed to complete a round of image acquisition.
7. A focusing device for an imaging system, characterized in that, The imaging system includes a plurality of imaging sensors, and the plurality of imaging sensors are sequentially and staggeredly distributed along the imaging optical path of the object to be measured, and are used to acquire images of the object to be measured at different imaging positions; The focus detection device includes: an acquisition unit, a calculation unit, and a determination unit; The acquisition unit is used to obtain the acquired images from each of the imaging sensors according to a preset imaging acquisition rule to obtain a plurality of images to be measured; the preset imaging acquisition rule includes time nodes for sequentially enabling each of the imaging sensors to acquire the same image; The calculation unit is used to perform gradient calculation on the plurality of images to be measured respectively to obtain the image sharpness corresponding to each of the plurality of images to be measured; The determination unit is used to analyze the focus detection result of the object to be measured according to the image sharpness corresponding to each of the plurality of images to be measured and the imaging position of the imaging sensor corresponding to each of the plurality of images to be measured.
8. The focus detection device according to claim 7, wherein, The imaging sensor is a time delay integration TDI sensor.
9. The focus detection device according to claim 7, characterized in that The process of the acquisition unit obtaining a plurality of images to be measured specifically includes: The acquisition unit obtains a plurality of time nodes and forms the preset imaging acquisition rule. Each time node is configured as the quotient of the imaging position of the corresponding imaging sensor and the imaging scan line frequency and the imaging pixel size. The plurality of time nodes are configured to progress in the staggered distribution order of each imaging sensor on the imaging optical path of the object to be measured; The acquisition unit enables the corresponding imaging sensor based on each time node in the preset imaging acquisition rule to acquire the corresponding image to be measured until all time nodes are traversed to complete a round of image acquisition.
10. The focus detection device according to claim 7, characterized in that, The process of the calculation unit obtaining the image sharpness corresponding to each of the plurality of images to be measured specifically includes: For each of the images to be measured, the calculation unit obtains the gray values of each pixel in the image to be measured, calculates the gradient of each pixel in the image to be measured in combination with a preset gradient operator, and obtains the image sharpness of the image to be measured through fusion processing; Among them, the preset gradient operators include Sobel operator, Tenengrad operator, and Laplacian operator.
11. The focus detection device according to claim 7, characterized in that, The determination unit obtaining the focus detection result of the object to be measured specifically includes: The determination unit performs data fitting on the image sharpness corresponding to multiple images to be measured based on the imaging positions of the imaging sensors corresponding to the multiple images to be measured, and generates a sharpness curve; The determination unit determines the imaging position corresponding to the maximum image sharpness in the sharpness curve as the first position; The determination unit determines the standard imaging position of the object to be measured as the second position; the standard imaging position is the imaging position where the object to be measured is best imaged clearly when the object to be measured is in the front focal plane of the imaging system; The determination unit determines the focus detection result of the object to be measured according to the positional relationship between the first position and the second position.
12. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, it is used to implement the method according to any one of claims 1-6.