Lens quality evaluation method based on virtual wavefront aberration and storage medium

Abstract

The invention relates to the field of lens quality detection, in particular to a lens quality evaluation method based on virtual wavefront aberration and a storage medium. Comprising the following steps: acquiring a to-be-detected grey-scale map corresponding to virtual wavefront medium-high frequency information of a to-be-detected lens by using a lens quality detection device; and inputting the grey-scale map to be detected into a detection segmentation algorithm based on deep learning, and generating a virtual wavefront phase difference defect detection result corresponding to the lens to be detected. Starting from the imaging performance, the defect is found from the wavefront aberration by utilizing the characteristic that the defect can introduce high-frequency aberration. The resolution ratio is high, and finer defects which are difficult to discover based on the traditional gray scale can be identified. Meanwhile, the wavefront phase difference is identified mainly based on the change of the light direction caused by the tiny structures. Therefore, the surface dirt which does not influence the imaging quality cannot be displayed in the virtual wavefront aberration due to the fact that the surface dirt does not cause deflection of the light direction, and the dirt can be prevented from being recognized as flaws.

Description

Background Art

[0002] Optical lenses are a kind of transparent optical elements that are widely used in optical systems. Their quality has a great impact on the performance of the entire system. Traditional lens quality assessment is done manually, but due to the small size and irregular shape of optical lens defects, there are even material impurities inside the lens that are extremely difficult to detect with the naked eye. This leads to the disadvantages of manual inspection, such as long time consumption, strong subjectivity, damage to the naked eye due to long-term work, low efficiency, and inaccuracy. Therefore, it is an inevitable result of industrial development that machine vision replaces traditional manual inspection.

[0003] Traditional machine learning algorithms such as SVM still require manual design of feature extraction operators, and do not have high detection accuracy for data with multiple defects. Deep learning-based algorithms are being widely used in the detection field due to their automatic feature extraction and high detection accuracy. Maestro-Watson et al. (Maestro-Watson D, Balzategui J, Eciolaza L, et al. Deflectometric data segmentation for surface inspection: a fully convolutional neural network approach [J]. Journal of Electronic Imaging, 2020, 29 (4): 041007-041007.) proposed a U-Net-based full segmentation network architecture for the detection of mirror surface objects, and used four architectures to perform boundary segmentation on the defects of reflective surface objects, but it could not classify the defects; Yang et al. (Yang W, Chen M, Wu H, et al. Deep learning-based weak micro-defect detection on an optical lens surface with micro vision [J]. Optics Express, 2023, 31 (4): 5593-5608.) realized defect detection on the surface of optical lenses based on the YOLO network architecture, and had high real-time performance and high accuracy, but its general detection paradigm could not provide accurate boundary information for tiny defects in the lens, and the detection rate of tiny defects was extremely low. The above methods are all based on direct defect recognition based on captured images, and cannot recognize foggy defects such as water marks inside the lens.

[0004] Wang Wei et al. (Wang Wei, Wang Jie, Huang Yiyang, et al. Transparent object surface defect detection method based on polarized transmission structured light [J]. Acta Optica Sinica, 2021, 41(18): 1812002.) proposed a method of polarized transmission structured light to detect surface defects of transparent objects. This method can reduce the interference of surface dust on defect detection by adjusting polarized light, but it is still difficult to extract fog defects inside the lens by only processing the image grayscale; Patent CN 115980052 A uses stripe structured light to calculate the average grayscale value of the image under different phase stripes, and obtains the location of the defect through binarization and morphological processing, but the algorithm is too sensitive to the grayscale changes caused by light, and still cannot identify fog-like defects.

[0005] As a result, existing lens quality inspection methods are unable to more accurately and comprehensively detect tiny defects in the lens, such as knife marks, water marks, etc. Summary of the invention

[0006] In view of the above technical problems, the technical solution adopted by the present invention is:

[0007] According to one aspect of the present invention, a lens quality assessment method based on virtual wavefront aberration is provided, wherein the method is applied to a lens quality detection device, and the lens quality detection device comprises:

[0008] LCD screen, telecentric lens imaging system, CCD camera and lens placement platform;

[0009] The telecentric lens imaging system is located directly above the liquid crystal screen, and is used to obtain image information displayed on the liquid crystal screen; the optical axis of the telecentric lens imaging system is perpendicular to the liquid crystal screen; the lens placement platform is arranged between the telecentric lens imaging system and the liquid crystal screen, and the optical axis of the telecentric lens imaging system is perpendicular to the lens placement plane in the lens placement platform;

[0010] A CCD camera is connected to the telecentric lens imaging system and is used to obtain light entering the telecentric lens imaging system;

[0011] The method comprises the following steps:

[0012] Using the lens quality detection device, obtaining a grayscale image to be detected corresponding to the high-frequency information of the virtual wavefront of the lens to be detected;

[0013] The grayscale image to be detected is input into a detection and segmentation algorithm based on deep learning to generate a virtual wavefront aberration defect detection result corresponding to the lens to be detected.

[0014] Furthermore, the lens quality detection device is used to obtain a grayscale image to be detected corresponding to the high-frequency information of the virtual wavefront of the lens to be detected, including:

[0015] Controlling the LCD screen to project a phase information image for obtaining a virtual wavefront aberration;

[0016] Based on the phase information image, respectively obtain the spatial relative position relationship between the screen pixels and the camera CCD pixels before and after the lens to be detected;

[0017] According to the spatial relative position relationship between the screen pixels and the camera CCD pixels before and after the lens to be detected, the angle between the two light beams before and after entering the lens to be detected is obtained;

[0018] Decomposing the angle between the two light beams into angles α and β with two reference directions;

[0019] According to α and β, the virtual wavefront of the lens to be inspected is reconstructed.

[0020] Furthermore, after reconstructing the virtual wavefront of the lens to be inspected, the method further includes:

[0021] Obtain high-frequency information in the virtual wavefront;

[0022] The high-frequency information in the virtual wavefront is normalized to generate a grayscale image of the lens to be inspected.

[0023] Furthermore, the method further comprises:

[0024] Control the LCD screen to display a white image;

[0025] The image of the lens to be inspected acquired by the CCD camera is input into a detection and segmentation algorithm based on deep learning to generate a bright field defect detection result of the lens to be inspected.

[0026] Furthermore, the lens quality detection device further comprises: a plurality of bar-shaped light sources; the plurality of bar-shaped light sources are arranged around the lens to be detected; and are used to provide a white light beam entering from the side of the lens to be detected;

[0027] The method further comprises:

[0028] Control the LCD screen to display a black image, and control the strip light sources on both sides of the lens to be inspected to emit light in sequence;

[0029] The image of the lens to be inspected acquired by the CCD camera is input into a detection and segmentation algorithm based on deep learning to generate a dark field defect detection result of the lens to be inspected.

[0030] Furthermore, the method further comprises:

[0031] The bright field defect detection result, the dark field defect detection result and the virtual wavefront aberration defect detection result of the lens to be detected are integrated to generate a quality defect map of the lens to be detected;

[0032] According to the number of defects and the size of the defects, it is determined whether the lens to be inspected meets the production quality standard.

[0033] Furthermore, the phase information image includes: a sinusoidal phase fringe image, a square wave phase fringe image, or a triangle wave phase fringe image.

[0034] Furthermore, based on the phase information image, the spatial relative position relationship between the screen pixels and the camera CCD pixels before and after the lens to be detected is respectively obtained, including:

[0035] Use phase shift technology to obtain the spatial relative phases of screen pixels and camera CCD pixels before and after the lens to be tested;

[0036] The absolute phase is obtained from the relative phase using a phase information solution method.

[0037] Furthermore, the method for calculating the phase information includes: a time phase solution algorithm, a space phase unwrapping algorithm, a Fourier phase unwrapping algorithm, or a least squares phase unwrapping algorithm.

[0038] According to a second aspect of the present invention, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned lens quality assessment method based on virtual wavefront aberration is implemented.

[0039] According to a third aspect of the present invention, there is provided an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned lens quality assessment method based on virtual wavefront aberration when executing the computer program.

[0040] The present invention has at least the following beneficial effects:

[0041] Starting from the imaging performance, the defect will introduce high-frequency aberrations and find defects from the wavefront aberration. With high resolution, the ultra-precision turning lines of the lens can be clearly seen, and more subtle defects such as knife marks and water marks that are difficult to find based on traditional grayscale can be identified.

[0042] At the same time, since the wavefront aberration in the present invention is mainly based on the refraction of light by those tiny structures, causing the change of light direction for identification, therefore, for those surface dirt that does not affect the image quality, since it does not cause the deflection of the light direction, it will not be displayed in the virtual wavefront aberration, and will not be placed in the grayscale image, so the present application can avoid identifying dirt that does not affect the image quality as a defect. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 It is a structural schematic diagram of a lens quality detection device in one embodiment of the present invention;

[0045] Figure 2 It is a schematic flow chart of a lens quality assessment method based on virtual wavefront aberration in one embodiment of the present invention;

[0046] Figure 3 A schematic diagram of the principle of obtaining virtual wavefront aberration in an embodiment of the present invention;

[0047] Figure 4 Schematic diagram of the principle of calculating virtual wavefront aberration in an embodiment of the present invention;

[0048] Figure 5 It is a grayscale image of high-frequency information based on virtual wavefront aberration in an embodiment of the present invention;

[0049] Figure 6 4 is a structural diagram of a defect recognition network based on virtual wavefront aberration (i.e., an improved DeepLabv3+ model) in an embodiment of the present invention;

[0050] Figure 7 This is a defect recognition result diagram based on virtual wavefront aberration in one embodiment of the present invention;

[0051] Figure 8 It is a schematic diagram of the fusion determination result of bright field detection result, dark field detection result and virtual wavefront aberration detection in an embodiment of the present invention.

[0052] Reference numerals:

[0053] 1. Industrial camera, 2. Telecentric lens, 3. Bar light source, 4. LCD screen, 5. Lens to be tested, 6. Electric control translation stage, 7. Computer. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0055] As a possible embodiment of the present invention, a lens quality assessment method based on virtual wavefront aberration is provided, and the method is applied to a lens quality detection device.

[0056] like Figure 1 As shown, the lens quality detection device includes: an industrial CCD camera 1, a telecentric lens 2 connected to the CCD camera, two visible white light strip light sources 3 placed on both sides of the imaging lens 5 to be tested, a liquid crystal screen 4 placed under the imaging lens to be tested, an optical lens to be tested, an X, Y direction electric control displacement stage 6 (i.e., a lens placement platform) for moving the lens 5 to be tested, and a computer 7 for controlling the LCD screen 4 to display patterns, controlling the strip light source 3 to illuminate, controlling the electric control displacement stage 6 to adjust the position of the lens to be tested, and controlling the industrial CCD camera 1 to collect images. Among them, the electric control displacement stage 6 is used to move to different positions, and different lenses to be tested 5 can be tested. The computer 7 controls the screen to display different patterns, controls the strip light source 3 to emit light in sequence, and the light passes through the lens and is modulated by it and converges into the telecentric lens 2. The CCD camera 1 collects the modulated bright field, dark field and sinusoidal fringes, and finally the computer 7 performs calculation detection of the corresponding defects. The object field of the telecentric lens 2 is larger than that of the industrial CCD camera 1. The strip light source 3 can also be replaced by lamp beads or parallel light sources.

[0057] The lens placement platform includes a tray support, an electric control displacement stage 6 and a stepping motor. The tray support is fixed on the electric control displacement stage 6, and the stepping motor is used to drive the electric control displacement stage 6.

[0058] Before using the above-mentioned lens quality assessment method based on virtual wavefront aberration, it is necessary to build a corresponding lens quality detection device. The specific steps are as follows:

[0059] Connect the industrial CCD camera 1 to the telecentric lens 2 through the standard C port, and adjust the position by the level to keep the optical axis in the CCD camera lens in the vertical direction. The optical axis of the telecentric lens 2 is kept coincident with the optical axis in the CCD camera lens. Place the lens 5 to be tested and the tray on the electric control displacement stage 6 to keep them in the horizontal direction. Connect the computer 7 to the camera 1, the strip light source 3, the electric control displacement stage 6 and the LCD screen 4.

[0060] Lens quality assessment method based on virtual wavefront aberration, such as Figure 2As shown, it includes bright field detection, dark field detection and virtual wavefront aberration detection, and finally all the monitoring results are integrated and displayed. Specifically, the algorithm content of bright field detection, dark field detection and virtual wavefront aberration detection is as follows:

[0061] Bright field detection: In bright field detection, the computer 7 controls the LCD screen 4 to display a white image. The light emitted by the screen is modulated by the surface and the interior of the lens 5 to be detected, and converges into the telecentric optical imaging system 2. When the screen emits white light, the CCD camera 1 takes a shot and collects a bright field image, that is, the light signal modulated by the lens to be detected is finally received by the industrial CCD camera 1. Among them, mechanical defects such as defects or scratches on the lens surface will block or reflect or transmit the light to other irregular directions. The corresponding light will appear as a gray area on the target surface of the industrial CCD camera 1. The collected image is processed by grayscale based on OpenCV, and the defects on the surface or inside of the lens, such as scratches and pits on the lens surface, can be detected.

[0062] Dark field detection: Computer 7 controls LCD screen 4 to display a black image, controls the strip light sources 3 on both sides of the lens 5 to be detected to emit light in sequence, and the array camera takes pictures in sequence to collect two dark field images. When the side light passes through the inside of the lens, the tiny white spots of defects produce a diffuse reflection effect on the side light that enters. Part of the diffusely reflected light converges into the telecentric optical imaging system 2, and finally the light signal is received by the industrial CCD camera 1. Among them, the tiny white defects (i.e., tiny white spots) inside the lens can only be presented on the target surface of the camera by side lighting and using the dark field detection principle. Finally, the image collected by the camera is processed by grayscale based on OpenCV to detect the tiny white spot defects inside the lens. This defect cannot be imaged in the camera by front lighting.

[0063] Virtual wavefront aberration detection: In order to realize the measurement of the virtual wavefront aberration of the lens 5 to be detected, it is necessary to obtain the spatial relative position relationship between the screen pixels and the camera CCD pixels before and after the lens. That is, determine the angle between the direction of the light beam before it enters the lens 5 to be detected and the direction of the light beam after modulation and deflection by the lens 5 to be detected. Then, according to the angle and the divided angles α and β between the X-axis and the Y-axis respectively, the wavefront slope can be determined according to α and β. Finally, according to the calculated wavefront slope, the Southwell regional wavefront integration algorithm is used to reconstruct the wavefront phase distribution. Finally, the measurement of the virtual wavefront aberration of the lens 5 to be detected is realized by obtaining high-order information in the wavefront phase distribution.

[0064] like Figure 3 This is a schematic diagram of the principle of obtaining virtual wavefront aberration in this embodiment. Figure 3The left side shows the measurement state when no lens is placed. Before measurement, ensure that the telecentric lens 2 and the LCD screen 4 are placed vertically, and set the plane z where the LCD screen 4 is located to 0. The LCD screen 4 projects 1-16 single-frequency sinusoidal fringe images I x (x, y), the relative phase ψ(x) wrapped in [-π, π) is obtained by the four-step phase shift method, and the absolute phase is obtained using the time solution phase algorithm That is, the spatial relative position relationship between screen pixels and camera CCD pixels.

[0065] In this embodiment, the phase information used to obtain the virtual wavefront aberration can be the phase information of various waves such as sine wave phase information, square wave phase information, triangle wave phase information, etc. The method for calculating the phase information can be various unwrapping algorithms such as time phase unwrapping algorithm, spatial phase unwrapping algorithm, Fourier phase unwrapping algorithm, least squares phase unwrapping algorithm, etc.

[0066] In this embodiment, a four-step phase shift method and a time phase solution algorithm are used in combination to obtain the spatial relative position relationship between the screen pixels and the camera CCD pixels.

[0067] The relative phase ψ(x) of each group of fringes is obtained by formula (1):

[0068]

[0069] where x(p,q) is a function of the camera pixels (p,q).

[0070] The absolute position relationship between screen pixels and camera CCD pixels It can be obtained by unpacking the relative position relationship ψ(x):

[0071] x(p,q)=ψ(x(p,q))+2kπ (2)

[0072] Where k is a coefficient related to the fringe order.

[0073] Formula (2) obtains the (x, y) coordinates corresponding to the camera pixel (p, q). For the sake of simplicity, x(p, q) and y(p, q) are denoted as x, y in the following. The above formulas (1) and (2) only disclose the calculation method for the coordinates in the X-axis direction. The calculation method for the coordinates in the Y-axis direction is the same and will not be repeated here.

[0074] Since the telecentric lens only receives parallel light beams from the object, and the telecentric lens 2 is perpendicular to the LCD screen 4, the approximate principal plane 8 height z of the lens 5 to be inspected is set to l=h, the height h of the approximate principal plane 8 is used to indicate the approximate position where the light is deflected in the lens 5 to be detected. If h is set too large, the accuracy of the beam equation calculated by the two-point formula in the following embodiment can be improved, but the volume of the lens quality detection device will be increased. In this embodiment, h=10 cm is set. In this embodiment, by introducing a virtual lens approximate principal plane, the need to introduce a translation stage for accurate wavefront calculation in patent CN114295327A is avoided, which greatly accelerates the defect detection process.

[0075] In this embodiment, the following example is used to illustrate how to solve the equation of the light beam before and after entering the lens 5 to be detected by using a two-point equation:

[0076] Assume that the light beam received by the camera pixel (p, q) obtained by the phase shift method comes from the coordinate point of the LCD screen (x s ,y s ,0), and the coordinate point of the light beam passing through the approximate principal plane of the lens to be detected is (x l ,y l ,h)(plane position of the lens to be tested). Figure 2 The right side is the principle diagram after the lens is placed. The dotted line in the figure shows the light deflected by the defect. After the lens is placed, the coordinate point of the light beam modulated by the lens 5 to be tested from the LCD screen is (x d-s ,y d-s ,0). Before placing the lens, for the light beam entering the lens 5 to be detected, the two-point equation can be obtained:

[0077]

[0078] The lens 5 to be inspected has a modulating effect on the light beam entering the camera 1. After placing the lens, the two-point equation can be obtained:

[0079]

[0080] The above formulas (3) and (4) can be used to solve the light beams before and after entering the lens 5 to be detected, and thus the angle between the two light beams can be calculated. In addition, since in this embodiment, the telecentric lens 2 only receives parallel light beams from the object, and the telecentric lens 2 is perpendicular to the liquid crystal screen 4, the light beam before entering the lens 5 to be detected is only a point after being projected onto the XY plane. Therefore, in this embodiment, it is only necessary to solve the angles between the projection of the light beam after entering the lens 5 to be detected on the XY plane and the X-axis and Y-axis, respectively, to obtain α and β.

[0081] like Figure 4, assuming that the angles of the light beam before and after the lens are α and β in the x and y directions respectively. The slope of the virtual wavefront W(x,y) of the lens to be tested in the x and y directions is equal to the tangent value of the angles α and β in the two directions:

[0082]

[0083] Furthermore, a known regional wavefront reconstruction algorithm such as Southwell regional wavefront integration is used for calculation, and the virtual wavefront W(x, y) can be obtained by integrating the tangent values ​​of α and β with sampling intervals of dx and dy in the x and y directions. The virtual wavefront aberration is a rough quantification of the deflection angle of the light after passing through the lens to be measured, and the method for calculating the virtual wavefront aberration can be any regional wavefront reconstruction algorithm.

[0084] In order to detect defects in the lens, it is necessary to extract the high-frequency information of the wavefront, perform polynomial fitting on the virtual wavefront W(x, y) obtained after calculation to obtain high-order residuals, or obtain high-frequency information through two-dimensional high-pass filtering, and normalize the high-frequency information of the wavefront to make it a grayscale image. The grayscale pattern to be detected is as follows: Figure 5 shown.

[0085] The above method decodes the fringe image by the phase shift method, calculates the angle between the light beams before and after the object to be tested is placed, and integrates the angle using the known regional wavefront reconstruction algorithm to obtain the virtual wavefront aberration of the object to be tested, and then generates a grayscale image corresponding to the lens defect based on the wavefront aberration. By collecting this image, a sample set for training the following neural network can be formed, and in the later detection and use stage, a defect detection image corresponding to the lens 5 to be tested can be generated.

[0086] In order to realize lens defect detection based on virtual wavefront aberration, this embodiment uses an improved DeepLabv3+ network based on dilated separable convolution to detect the above grayscale image. The model introduces multiple dilated convolutions with different expansion rates in series or parallel to obtain more images with less obvious defects. In addition, the 3×3 convolution is degraded to 1×1 in the backbone network of continuous downsampling, which reduces the problem of inaccurate defect boundary segmentation caused by low resolution. The network principle diagram is shown in the figure below. Figure 6 :

[0087] The image to be detected passes through four 3x3 convolutions and enters the parallel hole convolution module, where the expansion rate of each hole convolution is different. In the feature map of low-resolution input, the convolution with a lower expansion rate is used for sparse feature extraction, and in the feature map of high-resolution input, the convolution with a higher expansion rate is used for dense feature extraction. Finally, the feature maps with different receptive field scales but the same feature dimensions are spliced, thereby realizing the feature extraction of defects of different scales. The result of the feature map after the hole convolution is:

[0088]

[0089] Where (u,v) is the position on the output feature map, (i,j) is the position in the convolution kernel, O is the convolution output, I is the convolution input, d is the dilation rate of the dilated convolution (usually a positive integer), and K is the convolution kernel of the dilated convolution. The introduction of convolution holes expands the receptive field of the convolution, allowing the convolution kernel to cover a wider input area without increasing the size, which helps capture features of different scales. It does not reduce the resolution of the output feature map, which is conducive to retaining accurate boundary information.

[0090] The above-mentioned hole convolution pyramid can be regarded as the encoder part. In order to accurately segment the marked defect features, it is necessary to input the extracted coded defects into the decoding part to extract the boundaries. This embodiment adopts an improved decoder model to integrate the multi-scale features extracted by the hole convolution pyramid module into each upsampling stage. Compared with the previous fusion only in the initial decoding stage, it can provide more refined edge information for defects of different scales. In order to improve the problem of boundary loss when the decoder directly uses 16 times bilinear interpolation upsampling, 4 times bilinear interpolation upsampling is used in the decoder part and the low-level features output by the cascade encoder are output in the channel dimension, which effectively reduces the loss of boundary information. Bilinear interpolation requires four points around the pixel, two single linear interpolations are performed in the u direction, and single linear interpolation is performed in the v direction. The results are fused with 1×1 convolution in the channel dimension. The decoding part of this simple initial stage can be calculated by the following formula:

[0091] f BI (u,v)=[I(u1,v1)(u2-u)(v2-v)+I(u2,v1)(u-u1)(v2-v)

[0092] +I(u1,v2)(u2-u)(v-v1)+I(u2,v2)(u-u1)(v-v1)](7)

[0093] O D (u,v)=Concat[O I (u,v),f BI (u,v)] (8)

[0094] where f BI (u,v) is the bilinear interpolation output, O D (u,v) is the decoder output, O I (u,v) is the encoder output, Concat[·] is the channel concatenation operator, which is implemented by the public third-party library torch.

[0095] Decoder part The operator represents channel addition, gradually adding the more accurate boundary information at the lower level to the decoded feature map, retaining the boundary information as much as possible, which can slightly improve the accuracy of boundary recognition in actual calculation.

[0096] This embodiment should identify all defect targets as much as possible, so during model detection, the traditional Dice loss function is replaced by the Tversky loss function to improve the recall rate of the network. The overall loss function consists of two parts: the classification loss function L for distinguishing different types of defects and the loss function L for distinguishing different types of defects. ce And the segmentation loss function L that improves the detection boundary tversky :

[0097] Loss = L ce(z,c) +L tversky (9)

[0098] The classification loss function L ce It is expressed as:

[0099]

[0100] where z=[z0,z1,…,z C-1 ] represents the network output, y c is the one-hot encoding of the sample, and the classification loss function is the cross entropy loss function, which describes the distance between the output distribution and the experimental sample distribution.

[0101] Segmentation loss function L tversky It is expressed as:

[0102]

[0103] TP represents true positive samples (predicted to be positive, but actually positive), FP represents false positive samples (predicted to be positive, but actually negative), and FN represents false negative samples (predicted to be negative, but actually positive). α and β are coefficients of the weighted denominator. This segmentation loss function is a variant of the intersection-over-union ratio of segmented regions, and is usually used to improve the recall rate.

[0104] In this embodiment, the decoder part of the existing DeepLabv3+ network based on dilated separable convolution is improved, and the multi-scale features extracted by the dilated convolution pyramid module can be integrated into each upsampling stage. As a result, the more accurate boundary information at the lower level can be gradually added to the feature map of the decoded part, and the boundary information can be retained as much as possible, which can slightly improve the accuracy of boundary recognition in actual calculations. In addition, the improved DeepLabv3+ also replaces the traditional Dice loss function with the Tversky loss function to ensure that in the later defect recognition, the model can recognize all defect targets as much as possible.

[0105] In addition, since the improved DeepLabv3+ model in the present invention is mainly used to identify and segment the image area corresponding to the defect from the collected image of the lens to be tested, the improved DeepLabv3+ model can also be replaced by other image detection and segmentation algorithms based on deep learning. There are many existing image detection and segmentation models based on deep learning, such as "U-Net, DeepLab series, SegNet, etc.

[0106] The device parameters used in this example are as follows: Intel Core i9-11900k CPU; one GeForceRTX 3070 graphics card with 8G video memory; two Samsung DDR4 memory sticks with a total RAM of 32G. Data collection, data cleaning, and data labeling are performed on the device, and the labeled data is trained in the above model. The final prediction results are as follows: Figure 7 shown.

[0107] Comprehensive results: The above bright field test results, dark field test results and virtual wavefront aberration test results are integrated. The fusion results are comprehensively considered to determine whether the lens meets the production quality standards based on the size of the defects and the number of defects in each defect type. The final evaluation results are as follows: Figure 8 As shown, the leftmost part is the setting interface of the relevant defect judgment parameters, the middle part is the diagram of the fused defect detection results corresponding to the 15 lenses 5 to be detected, and the rightmost part is the result statistics and judgment result display of the defect results in the 15 lenses 5 to be detected.

[0108] The present invention controls the LCD screen to project black and white images and structured light stripe images, and the telecentric imaging system receives the rear camera to collect images, and performs image processing and stripe decoding on the collected images. The stripe image is decoded by the phase shift method, the angle between the light beams before and after the object to be tested is calculated, and the angle is integrated using the known regional wavefront reconstruction algorithm to obtain the virtual wavefront aberration of the object to be tested. Data collection is performed on the calculated image, and the bright and dark field images and the decoded virtual wavefront aberration are detected using a segmentation model based on deep learning. Finally, the bright and dark field and segmentation detection model results are fused to obtain the quality evaluation result. The method of the present invention starts from the imaging quality of the lens and takes the virtual wavefront aberration as the primary quality evaluation indicator. The beneficial effects are: not only can it identify details that are difficult to find based on traditional grayscale (no missed detection), such as knife marks, water marks, etc.; but it can also avoid identifying dirt that does not affect the imaging quality as defects (no over-detection). The method of the present invention is used to solve the problems of long time, low efficiency and poor detection effect of manual lens detection, and can realize lens quality detection intelligently, in batches and accurately, especially can obtain lens surface and internal defect points, scratch texture and areas affecting imaging quality.

[0109] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0110] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0111] In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0112] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".

[0113] The electronic device according to this embodiment of the present invention is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0114] The electronic device is presented in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: the at least one processor mentioned above, the at least one storage device mentioned above, and a bus connecting different system components (including storage devices and processors).

[0115] The storage stores program codes, which can be executed by the processor, so that the processor executes the steps according to various exemplary embodiments of the present invention described in the above “Exemplary Method” section of this specification.

[0116] The memory may include readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory, and may further include read only memory (ROM).

[0117] The storage may also include a program / utility having a set (at least one) of program modules, such program modules including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0118] The bus may represent one or more of several types of bus structures including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures.

[0119] The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may communicate with one or more devices that enable a user to interact with the electronic device, and / or may communicate with any device (e.g., routers, modems, etc.) that enables the electronic device to communicate with one or more other computing devices. Such communication may be performed through an input / output (I / O) interface. Furthermore, the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter. The network adapter communicates with other modules of the electronic device through a bus. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0120] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0121] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present invention may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above "Exemplary Method" section of the present specification.

[0122] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0123] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0124] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0125] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0126] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0127] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0128] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A lens quality assessment method based on virtual wavefront aberration, characterized in that: The method is applied to a lens quality detection device, which comprises: LCD screen, telecentric lens imaging system, CCD camera and lens placement platform; The telecentric lens imaging system is located directly above the liquid crystal screen, and is used to obtain image information displayed on the liquid crystal screen; the optical axis of the telecentric lens imaging system is perpendicular to the liquid crystal screen; the lens placement platform is arranged between the telecentric lens imaging system and the liquid crystal screen, and the optical axis of the telecentric lens imaging system is perpendicular to the lens placement plane in the lens placement platform; A CCD camera is connected to the telecentric lens imaging system and is used to obtain light entering the telecentric lens imaging system; The method comprises the following steps: Using the lens quality detection device, obtaining a grayscale image to be detected corresponding to the high-frequency information of the virtual wavefront of the lens to be detected; The grayscale image to be detected is input into a detection and segmentation algorithm based on deep learning to generate a virtual wavefront aberration defect detection result corresponding to the lens to be detected.

2. The method according to claim 1, characterized in that Using the lens quality detection device, obtaining a grayscale image to be detected corresponding to the high-frequency information of the virtual wavefront of the lens to be detected, including: Controlling the LCD screen to project a phase information image for obtaining a virtual wavefront aberration; Based on the phase information image, respectively obtain the spatial relative position relationship between the screen pixels and the camera CCD pixels before and after the lens to be detected; According to the spatial relative position relationship between the screen pixels and the camera CCD pixels before and after the lens to be detected, the angle between the two light beams before and after entering the lens to be detected is obtained; Decomposing the angle between the two light beams into angles α and β with two reference directions; According to α and β, the virtual wavefront of the lens to be inspected is reconstructed.

3. The method according to claim 2, characterized in that After reconstructing the virtual wavefront of the lens to be inspected, the method further includes: Obtain high-frequency information in the virtual wavefront; The high-frequency information in the virtual wavefront is normalized to generate a grayscale image of the lens to be inspected.

4. The method according to claim 1, characterized in that: The method further comprises: Control the LCD screen to display a white image; The image of the lens to be inspected acquired by the CCD camera is input into a detection and segmentation algorithm based on deep learning to generate a bright field defect detection result of the lens to be inspected.

5. The method according to claim 4, characterized in that The lens quality detection device further comprises: a plurality of bar-shaped light sources; the plurality of bar-shaped light sources are arranged around the lens to be detected; and are used to provide a white light beam entering from the side of the lens to be detected; The method further comprises: Control the LCD screen to display a black image, and control the strip light sources on both sides of the lens to be inspected to emit light in sequence; The image of the lens to be inspected acquired by the CCD camera is input into a detection and segmentation algorithm based on deep learning to generate a dark field defect detection result of the lens to be inspected.

6. The method according to claim 5, characterized in that The method further comprises: The bright field defect detection result, the dark field defect detection result and the virtual wavefront aberration defect detection result of the lens to be detected are integrated to generate a quality defect map of the lens to be detected; According to the number of defects and the size of the defects, it is determined whether the lens to be inspected meets the production quality standard.

7. The method according to claim 2, characterized in that The phase information image includes: a sinusoidal phase fringe image, a square wave phase fringe image, or a triangle wave phase fringe image.

8. The method according to claim 2, characterized in that: Based on the phase information image, the spatial relative position relationship between the screen pixels and the camera CCD pixels before and after the lens to be detected is respectively obtained, including: Use phase shift technology to obtain the spatial relative phases of screen pixels and camera CCD pixels before and after the lens to be tested; The absolute phase is obtained from the relative phase using a phase information solution method.

9. The method according to claim 8, characterized in that The method for calculating the phase information includes: a time phase solution algorithm, a space phase unwrapping algorithm, a Fourier phase unwrapping algorithm, or a least squares phase unwrapping algorithm.

10. A non-transitory computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for evaluating lens quality based on virtual wavefront aberration according to any one of claims 1 to 9 is implemented.

Images