Camera scanning method and apparatus for optical element surface defect detection

Abstract

The application discloses a camera scanning method and device for optical element surface defect detection, and relates to the technical field of optical detection. The method comprises the following steps: based on the dark field microscopic imaging principle, a CMOS microscope camera is controlled to perform overall scanning on a to-be-detected optical element based on a preset rough scanning path in a rough scanning mode with a magnification of 1; a sub-aperture image collected by the CMOS microscope camera at each first sub-aperture position of the preset rough scanning path is acquired, and defect feature information in the sub-aperture image is extracted; a target defect point is determined based on the defect feature information, and the target defect point is clustered to obtain a defect point clustering result; a path of all defect point classes is planned with the shortest scanning path as the target to obtain a fine scanning path; and the CMOS microscope camera is controlled to perform scanning on the to-be-detected optical element based on the fine scanning path in a fine scanning mode with a magnification greater than 1. The application can reduce scanning time and data storage amount, and reduce image splicing errors in the later stage.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of optical detection technology, and more particularly to a camera scanning method and device for detecting surface defects of an optical element. BACKGROUND

[0002] Optical elements have been widely used in large astronomical telescope systems, X-ray laser systems, inertial confinement nuclear fusion systems and other fields. If there are defects on the surface of the optical element, the performance of the optical system will be greatly affected. In order to meet the imaging quality of the optical system, ensure the operation safety of the optical system and the use function of the optical system, the requirement for the surface quality of the optical element is becoming higher and higher. Accordingly, it is of great significance to realize fast and accurate detection of surface defects of the optical element.

[0003] In the related art, a Charge Coupled Device (CCD) is used to obtain an image of the entire optical element to be detected at one time, and then an image processing technology is used to digitally describe the defects. Although this method is simple to operate, in order to obtain fine defect information (such as microns), a CCD with high resolution is needed, which is costly and has a relatively limited detection aperture range. In view of this, in order to ensure the high resolution of the defect image, a zoom technology is used in the related art to collect magnified sub-aperture images of the entire optical element to be detected, and a full-size image of the optical element to be detected is obtained by sub-aperture stitching, and then the full-size image is subjected to defect recognition to realize defect detection. This can realize the detection of large-aperture optical elements while ensuring the optical resolution. However, this method needs to complete the overall scanning of the optical element to be detected under magnified sub-aperture, which increases the detection time, and in the case of a large number of defects and close distance between defects, a large number of sub-aperture images with repeated defects will be collected, which not only increases the storage data volume, but also easily leads to the accumulation of image stitching errors, affecting the detection effect. SUMMARY

[0004] In view of the above problems existing in the prior art, the present application provides a camera scanning method and device for detecting surface defects of an optical element.

[0005] The camera scanning method for detecting surface defects of an optical element provided by the present application comprises the following steps:

[0006] Based on the dark field microscopic imaging principle, a CMOS microscope camera is controlled to perform overall scanning on the optical element to be detected based on a preset coarse scanning path in a coarse scanning mode; wherein the magnification of the CMOS microscope camera in the coarse scanning mode is 1.

[0007] acquire a sub-aperture image collected by the CMOS microscope camera at each first sub-aperture position of the preset coarse scanning path, and extract defect feature information in the sub-aperture image;

[0008] determine a target defect point based on the defect feature information, and cluster the target defect point to obtain a defect point clustering result, the target defect point being used to represent a defect in the sub-aperture image;

[0009] determine a class center of each defect point class in the defect point clustering result, and determine the class center as a second sub-aperture position;

[0010] plan a path for all the second sub-aperture positions to obtain a fine scanning path, with the shortest scanning path as the target;

[0011] control the CMOS microscope camera to scan the optical element to be measured in a fine scanning manner based on the fine scanning path, the second sub-aperture position being a position at which the CMOS microscope camera collects a sub-aperture image in the fine scanning manner, and a magnification of the CMOS microscope camera in the fine scanning manner being greater than 1.

[0012] Optionally, the determination of a target defect point based on the defect feature information and the clustering of the target defect point to obtain a defect point clustering result comprises:

[0013] determining a defect type based on the defect feature information;

[0014] in a case where the defect type includes a pitting type and a scratch type, normalizing the defect of the scratch type into a first defect point;

[0015] determining the first defect point and a second defect point of the pitting type as target defect points, and clustering the target defect points to obtain a defect point clustering result.

[0016] Optionally, the normalization of the defect of the scratch type into a first defect point comprises:

[0017] determining a start point and an end point of the defect of the scratch type;

[0018] dividing the defect of the scratch type once every target distance from the start point to the end point, wherein the target distance is determined based on a division parameter and a second field of view range corresponding to the fine scanning manner, the division parameter being greater than 0 and less than 1;

[0019] determining the start point, the end point and all division points of the division as the normalized first defect points of the defect of the scratch type.

[0020] Optionally, the clustering of the target defect points comprises:

[0021] Step A1: classifying each of the target defect points into a category, and determining the Euclidean distance between each two categories;

[0022] Step A2: in the case that the Euclidean distance is less than or equal to a preset distance threshold, classifying the two categories corresponding to the Euclidean distance into the same category to obtain a new clustering result;

[0023] Step A3: re-determining the Euclidean distance between each two categories in the new clustering result;

[0024] Step A4: repeatedly performing the above steps A2 to A3 until the Euclidean distance between each two categories in the new clustering result is greater than the preset distance threshold, and the new clustering result is determined as the defect point clustering result.

[0025] Optionally, the extracting of the defect feature information in the sub-aperture image comprises:

[0026] filtering the sub-aperture image to obtain a first image;

[0027] performing binaryzation processing on the first image to obtain a binaryzation image;

[0028] performing digital morphological processing on the binaryzation image to obtain a second image;

[0029] extracting a target pixel point with a gray value greater than a preset gray threshold in the second image, and determining the coordinate information of the target pixel point as the defect feature information in the sub-aperture image.

[0030] Optionally, the controlling of the CMOS microscope camera to perform overall scanning on the optical element to be measured based on a preset coarse scanning path in a coarse scanning mode comprises:

[0031] controlling the CMOS microscope camera to perform overall scanning on the optical element to be measured based on the preset coarse scanning path at a magnification of 1 times and a step distance of a distance of a first field of view range;

[0032] wherein, the preset coarse scanning path comprises a U-shaped path or a grid-shaped path, and the first field of view range is determined based on the magnification, the pixels of the CMOS microscope camera and the resolution of the CMOS microscope camera.

[0033] Optionally, the controlling of the CMOS microscope camera to perform scanning on the optical element to be measured based on the fine scanning path in a fine scanning mode comprises:

[0034] control the CMOS microscope camera to perform scanning on the optical element to be tested based on the fine scanning path in a second field of view range;

[0035] The second field of view range is determined based on a multiple magnification, pixels of the CMOS microscope camera and resolution of the CMOS microscope camera, the multiple magnification is a magnification greater than 1, and the second field of view range is smaller than the first field of view range.

[0036] The embodiment of the present application provides a camera scanning device for optical element surface defect detection, comprising:

[0037] The first scanning unit is configured to control the CMOS microscope camera to perform overall scanning on the optical element to be tested based on a preset coarse scanning path in a coarse scanning mode based on a dark field microscopic imaging principle, and the magnification of the CMOS microscope camera in the coarse scanning mode is 1.

[0038] The information extraction unit is configured to acquire a sub-aperture image collected by the CMOS microscope camera at each first sub-aperture position of the preset coarse scanning path, and extract defect feature information in the sub-aperture image.

[0039] The clustering unit is configured to determine a target defect point based on the defect feature information, cluster the target defect point, and obtain a defect point clustering result, and the target defect point is used to represent a defect in the sub-aperture image.

[0040] The determination unit is configured to determine a class center of each defect point class in the defect point clustering result, and determine the class center as a second sub-aperture position.

[0041] The path planning unit is configured to perform path planning on all the second sub-aperture positions with the shortest scanning path as a target, and obtain a fine scanning path.

[0042] The second scanning unit is configured to control the CMOS microscope camera to perform scanning on the optical element to be tested based on the fine scanning path in a fine scanning mode, the second sub-aperture position is a position at which the CMOS microscope camera collects a sub-aperture image in the fine scanning mode, and the magnification of the CMOS microscope camera in the fine scanning mode is greater than 1.

[0043] The embodiment of the present application provides a computer device, which comprises a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the camera scanning method for optical element surface defect detection.

[0044] The embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to make the processor execute the camera scanning method for detecting surface defects of an optical element.

[0045] The camera scanning method and device for detecting surface defects of an optical element provided by the embodiment of the present application first controls a CMOS microscope camera to perform overall scanning on a to-be-detected optical element based on a preset coarse scanning path in a coarse scanning mode with a magnification of 1 based on a dark field microscopic imaging principle, acquires a sub-aperture image collected by the CMOS microscope camera at each first sub-aperture position of the preset coarse scanning path, and extracts defect feature information in the sub-aperture image; then determines a target defect point for representing defects in the sub-aperture image based on the defect feature information, and clusters the target defect point to obtain a defect point clustering result; then determines a class center of each defect point class in the defect point clustering result, and determines the class center as a second sub-aperture position; finally, path planning is performed on all the second sub-aperture positions to obtain a fine scanning path with the shortest scanning path as the target, controls the CMOS microscope camera to perform scanning on the to-be-detected optical element based on the fine scanning path in a fine scanning mode with a magnification greater than 1, and the CMOS microscope camera collects a sub-aperture image at the second sub-aperture position in the fine scanning process. In this way, the defect points on the to-be-detected optical element are classified through clustering, and the class center is used as the sub-aperture position in the fine scanning process, which can reduce repeated scanning of adjacent defects, reduce data storage and reduce image splicing errors in the later stage. Moreover, the shortest fine scanning path is planned according to the class center, then the CMOS microscope camera is controlled to scan the to-be-detected optical element according to the planned shortest fine scanning path in the fine scanning mode with a magnified sub-aperture and with the class center as the sub-aperture image collection position, which can ensure the optimal scanning time and scan all the defect clustering sub-apertures, thereby reducing the scanning time and the detection time. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without any creative effort.

[0047] Figure 1 The structure diagram of the dark field microscopic imaging measurement system provided by the embodiment of the present application;

[0048] Figure 2 The flowchart of the camera scanning method for detecting surface defects of an optical element provided by the embodiment of the present application;

[0049] Figure 3 A path schematic diagram of a CMOS microscope camera provided by an embodiment of the present application scanning according to a grid type path;

[0050] Figure 4 A schematic diagram of a defect point clustering result obtained after clustering of target defect points provided by an embodiment of the present application;

[0051] Figure 5 A schematic diagram of a planned fine scanning path provided by an embodiment of the present application;

[0052] Figure 6 A structural schematic diagram of a camera scanning device for optical element surface defect detection provided by an embodiment of the present application. DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0054] An optical element is an important element in an optical system, and the surface quality of the optical element directly affects the imaging quality and operation safety of the optical system and plays an important role in the optical system. Accordingly, it is of great significance to detect surface defects of the optical element.

[0055] The detection method of the surface defects of the optical element can include a manual visual detection method and a defect detection method based on machine vision. The manual visual detection method has low detection efficiency, high labor cost, and the detection effect is affected by human error. The defect detection method based on machine vision has the characteristics of non-contact, low cost, and high efficiency, and can meet different detection requirements.

[0056] In a machine vision-based defect detection method, the entire optical element to be tested can be acquired by a charge coupled device (CCD) at one time, and then the defects are digitally described by means of image processing technology. Although this method is simple to operate, in order to obtain fine defect information (such as micrometer level), a CCD with high resolution needs to be used, which is costly and the detection aperture range is relatively limited. In view of this, in order to ensure the high resolution of the defect image, a zoom technology is used in the related technology to collect magnified sub-aperture images of the entire optical element to be tested, and a full-size image of the optical element to be tested is obtained by image stitching technology, and then the full-size image is subjected to defect recognition, which can realize the detection of large-aperture optical elements while ensuring the optical resolution. However, this method needs to complete the overall scanning of the optical element to be tested under the condition of magnified sub-aperture, which increases the detection time, and in the case that the number of defects is large and the distance between defects is close, a large number of sub-aperture images with repeated defects will be collected, which not only increases the storage data volume, but also easily leads to the accumulation of image stitching errors, affecting the detection effect.

[0057] In view of this, the embodiment of the present application provides a camera scanning method for optical element surface defect detection, which can be applied to large-aperture optical elements. Based on the principle of dark-field microscopic imaging, the method first controls a complementary metal-oxide-semiconductor (CMOS) microscope camera to perform overall scanning on a to-be-detected optical element based on a preset coarse scanning path in a coarse scanning mode with a magnification of 1, so as to extract defect feature information from sub-aperture images collected in the coarse scanning process. Then, target defect points representing defects in the sub-aperture images are determined based on the defect feature information, and the target defect points are clustered. The centers of the clusters are used as positions of the sub-aperture images collected in a fine scanning mode with a magnification greater than 1 for planning of a shortest fine scanning path. Then, the CMOS microscope camera is controlled to perform scanning on the to-be-detected optical element based on the fine scanning path in the fine scanning mode, and sub-aperture images at the positions of the cluster centers are collected in the fine scanning process. Thereafter, the sub-aperture images obtained in the fine scanning can be spliced by using a sub-aperture splicing method to obtain a full-size image of the to-be-detected optical element, and then the full-size image is subjected to defect recognition to realize defect detection. In this way, by clustering the defect points and using the cluster centers as the positions of the sub-aperture in the fine scanning process of magnifying the sub-aperture, the repeated scanning of adjacent defects can be reduced, and thus the data storage amount is reduced and the image splicing error in the later stage is reduced. Moreover, by planning the shortest fine scanning path of the CMOS microscope camera according to the cluster centers, the to-be-detected optical element can be scanned according to the planned shortest fine scanning path in the fine scanning process of magnifying the sub-aperture, which can ensure that the scanning time is optimal and all the defect cluster sub-apertures are scanned, thereby reducing the scanning time and the detection time and ensuring the defect detection effect.

[0058] The camera scanning method for optical element surface defect detection provided by the embodiment of the present application can be applied to a computer device in a dark-field microscopic imaging measurement system. The computer device can include a terminal device or a server, etc. The terminal device can include a mobile phone, a computer, a tablet computer, etc. The server can include a standalone server, a cluster server, or a cloud server, etc. The camera scanning method for optical element surface defect detection can also be applied to a camera scanning device for optical element surface defect detection arranged in the computer device. The camera scanning device for optical element surface defect detection can be realized by software, hardware, or a combination of both.

[0059] Figure 1A structural schematic diagram of a dark field microscopic imaging measurement system is shown, which can measure the surface of an optical element to be measured based on a dark field microscopic imaging principle. The dark field microscopy (or dark field microscopic) is a special microscopic method in optical microscopy and electron microscopy, in which light or electrons other than the object to be measured enter the objective lens of the microscope, so that the background of the field of view observed in the ocular lens is black, and only the edge of the object to be measured is bright. The dark field microscopic imaging principle is to block direct light and only allow inclined light to irradiate on the object to be measured, and to use the scattered light of the object to be measured for imaging. Specifically, the dark field microscope uses a special condenser which produces an inverted hollow conical illumination area with the focal point located on the plane of the object to be measured. In the absence of the object to be measured, the inclined light will cross each other and form a dark background, while the inclined light passing through the object to be measured will produce diffraction, reflection or refraction to form a bright sample image.

[0060] Referring to Figure 1 The dark field microscopic imaging measurement system can include a CMOS microscope camera 11, a ring light source 12, an electrically controlled translation stage 13 and a computer device 14. Specifically, when measuring the optical element to be measured, the optical element to be measured 100 can be placed on the stage of the electrically controlled translation stage 13, and the ring light source 12 emits ring light to irradiate on the optical element to be measured 100. If there is a defect on the surface of the optical element to be measured 100, the backscattered light caused by the defect will enter the microscopic imaging system through the objective lens of the CMOS microscope camera 11 and converge on the target surface of the CMOS of the CMOS microscope camera 11, while the reflected light reflected by the optical element to be measured 100 does not enter the microscopic imaging system, forming a dark field, so that the defect image forms a bright image in a dark background. The electrically controlled translation stage 13 clamps the CMOS microscope camera 11, and the computer device 14 can drive the CMOS microscope camera 11 to move by controlling the translation mechanism of the electrically controlled translation stage 13 to move, and at the same time, the computer device 14 can also acquire the image collected by the CMOS microscope camera 11 and process the image.

[0061] Based on Figure 1 The dark field microscopic imaging measurement system is shown, and the camera scanning method for detecting the surface defect of the optical element of the present application is applied to the computer device 14 as an example, and the Figures 2-5 The camera scanning method for detecting the surface defect of the optical element of the present application is described.

[0062] Figure 2 A flowchart of the camera scanning method for detecting the surface defect of the optical element provided by the embodiment of the present application is shown, and referring to Figure 2 The camera scanning method for detecting the surface defect of the optical element can include the following steps 210-260.

[0063] Step 210: based on the dark field microscopic imaging principle, the CMOS microscope camera is controlled to perform overall scanning on the optical element to be measured based on a preset coarse scanning path in a coarse scanning mode.

[0064] The magnification of the CMOS microscope camera in the coarse scanning mode is 1.

[0065] For example, the shooting distance of the CMOS microscope camera from the optical element to be measured is set to achieve clear imaging. For example, in the coarse scanning mode, the shooting distance can be set to 53 mm.

[0066] Specifically, based on the dark field microscopic imaging principle, the computer device can control the CMOS microscope camera to perform overall scanning on the optical element to be measured based on a preset coarse scanning path at a magnification of 1 and a step distance of the first field of view range; wherein the preset coarse scanning path includes a back-shaped path or a grid-shaped path, and the first field of view range is determined based on the magnification, the pixel, and the resolution of the CMOS microscope camera.

[0067] For example, in the coarse scanning mode, the magnification of the CMOS microscope camera is 1, the pixel size is 3.45 μm x 3.45 μm, and the resolution of the sub-aperture image is 1280 pixel x 960 pixel. Therefore, in the coarse scanning mode, the first field of view range corresponding to a single sub-aperture image collected by the CMOS microscope camera is (3.45 μm x 1280 pixel) x (3.45 μm x 960 pixel) ≈ 4.4 mm x 3.3 mm. At this time, the step distance of the CMOS microscope camera is 4.4 mm x 3.3 mm, i.e. the step distance of the X axis is 4.4 mm and the step distance of the Y axis is 3.3 mm. Wherein, pixel is the basic unit of the image, which means "pixel".

[0068] Specifically, taking the grid-shaped path as an example, Figure 3 a path diagram of the CMOS microscope camera scanning according to the grid-shaped path is shown, referring to Figure 3 The origin O of the workpiece coordinate system XOY is the upper left corner of the optical element to be measured. In combination with Figure 1 As shown in the dark field microscopic imaging measurement system shown in Figure 3the dashed box in FIG. 1B), until the CMOS microscope camera 11 moves to the right edge of the optical element 100 to be measured. Then, the computer device 14 controls the motorized translation stage 13 to move the CMOS microscope camera 11 by one step along the Y-axis direction, and the CMOS microscope camera 11 captures a sub-aperture image at the position after the movement at a magnification of 1 times. Next, the computer device 14 controls the motorized translation stage 13 to continue moving the CMOS microscope camera 11 leftward along the X-axis direction, and captures a sub-aperture image at a magnification of 1 times every time the CMOS microscope camera 11 moves by one step. In this way, the entire surface of the optical element to be measured is captured. During the capturing of the sub-aperture images, the scanning step numbers (i, j) of the motorized translation stage 13 corresponding to the sub-aperture images can be saved, where i = 0, 1, 2, …, and j = 0, 1, 2, ….

[0069] For example, assuming that the first field of view range is 4.4 mm x 3.3 mm, then in the coarse scanning mode, the step distance of the CMOS microscope camera 11 along the X-axis direction is 4.4 mm, and the step distance of the CMOS microscope camera 11 along the Y-axis direction is 3.3 mm.

[0070] Step 220: Obtain the sub-aperture images captured by the CMOS microscope camera at each first sub-aperture position of the preset coarse scanning path, and extract defect feature information in the sub-aperture images.

[0071] A sub-aperture refers to a technical means of dividing an optical element with a large aperture into several small regions for independent observation and processing, and is mainly used to improve the spatial resolution and imaging quality of an optical system.

[0072] During the scanning of the optical element to be measured by the CMOS microscope camera along the preset coarse scanning path, the position of each movement by one step is taken as a first sub-aperture position for capturing a sub-aperture image, and the computer device can obtain the sub-aperture images captured by the CMOS microscope camera at each first sub-aperture position of the preset coarse scanning path. After obtaining the sub-aperture images, the computer device can extract defect feature information in the sub-aperture images by using image processing technology. The defect feature information can include defect shape feature data, defect quantity, and defect position information, etc.

[0073] For example, the computer device can determine a defect type of the defect according to the defect shape feature data. The defect type can include a pitting type and a scratch type.

[0074] In an embodiment, the extracting of the defect feature information in the sub-aperture image can include: performing filtering processing on the sub-aperture image to obtain a first image; performing binaryzation processing on the first image to obtain a binaryzation image; performing digital morphological processing on the binaryzation image to obtain a second image; extracting a target pixel point with a gray value greater than a preset gray threshold value in the second image, and determining coordinate information of the target pixel point as the defect feature information in the sub-aperture image.

[0075] The coordinate information is defect position information. The preset gray threshold value can be set according to experience or determined through experiments, for example, can be set to 240 according to experience.

[0076] For example, the median filtering can be used to perform filtering processing on the sub-aperture image to obtain the first image. Through the median filtering processing, the noise caused by dust and the like in the dark field image in the sub-aperture image can be removed. The median filtering is a nonlinear smoothing filtering method, which can remove the noise while preserving the feature information of the image.

[0077] For example, a segmentation threshold T can be determined according to experience or through experiments. For the first image, the pixel points of the first image can be traversed, if the pixel value of the pixel point is greater than the segmentation threshold T, the pixel value of the pixel point is re-set to 1; if the pixel value of the pixel point is less than or equal to the segmentation threshold T, the pixel point is determined as a background point, and its pixel value is re-set to 0. In this way, the target and the background can be distinguished, the binaryzation processing of the first image is realized, and the region of interest is maximally preserved. The binaryzation processing of the first image can be represented by the following formula (1):

[0078]

[0079] Wherein, f(x, y) represents the pixel point of the xth row and yth column in the first image, g(x, y) represents the binaryzation image obtained by performing binaryzation processing on the first image, and T represents the segmentation threshold.

[0080] For example, the digital morphological processing on the binaryzation image can include two parts of erosion and dilation, wherein the erosion is the inward contraction of the target boundary, and the dilation is the outward expansion of the target boundary. The digital morphological processing can be an opening operation of erosion and dilation, or a closing operation of erosion and dilation. The opening operation is to first erode and then dilate the image, and the closing operation is to first dilate and then erode the image. The closing operation can connect the disconnected adjacent targets and fill the small holes in the target area, so in the embodiment of the application, the closing operation can be used to perform the digital morphological processing on the binaryzation image. Specifically, the digital morphological processing based on the closing operation can be represented by the following formula (2):

[0081] F·S=(F⊕S)ΘS (2)

[0082] Wherein, F represents the binary image g(x, y), S is a structural element, and Θ represents the former is eroded by the latter.

[0083] In this way, the position information of the defects in the sub-aperture image is clearer and the edges are more full by performing the digital morphological processing on the binary image.

[0084] Step 230: determining target defect points based on the defect feature information, and clustering the target defect points to obtain defect point clustering results.

[0085] The target defect points are used to represent the defects in the sub-aperture image, that is, the target defect points are some pixel points that can be used to represent the defects. For example, for the pitting type defect, the pitting can be used as the target defect point; for the scratch type defect, some feature points on the scratch can be used to represent the defect, such as interval points and two end points can be determined as the target defect points.

[0086] For example, the defect feature information can include defect shape feature data, defect quantity, and defect position information, etc. The defect type can be determined according to the defect feature information, and then the target defect points used to represent the defect can be determined according to the characteristics of the defects of different defect types.

[0087] Specifically, the target defect points are determined based on the defect feature information, and the target defect points are clustered to obtain defect point clustering results, which can include the following steps 231-233.

[0088] Step 231: determining a defect type based on the defect feature information;

[0089] Step 232: in the case where the defect type includes the pitting type and the scratch type, the scratch type defect is normalized into a first defect point;

[0090] Step 233: determining the first defect point and a second defect point of the pitting type as the target defect points, and clustering the target defect points to obtain defect point clustering results.

[0091] In step 231, the defect feature information can include defect shape feature data, the defect type can include a pit type and a scratch type, and the defect type can be determined based on the defect feature information by using a deep learning algorithm. For example, a large number of sub-aperture sample images can be obtained, and the defect types in the sub-aperture sample images can be labeled to obtain label data. Meanwhile, the defect feature information in the sub-aperture sample images can be extracted to obtain sample defect feature information. Then, the sample defect feature information can be input into a defect classification model to be trained to obtain an output result of the defect classification model to be trained. The loss function value can be determined according to the output result and the label data. The model parameters of the defect classification model to be trained can be adjusted according to the loss function value until the model converges, and a trained defect classification model is obtained. Then, the defect feature information can be input into the trained defect classification model, and the defect type can be determined by using the trained defect classification model.

[0092] In step 232, the defects on the surface of the optical element to be measured can include only pit type defects, or only scratch type defects, or both pit type and scratch type defects. In the case where the defect type includes both pit type and scratch type, since the pit type defect and the scratch type defect have different characteristics, the length of the scratch type defect is relatively long and cannot be completely covered by one defect sub-aperture. Therefore, in the embodiments of the present application, the pit type and the scratch type can be normalized, and the scratch type defect can be converted into a series of points as the basis for selecting the defect sub-aperture by normalization.

[0093] Specifically, normalizing the scratch type defect into first defect points can include: determining the starting point and the ending point of the scratch type defect; dividing the scratch type defect once every target distance from the starting point to the ending point; and determining the starting point, the ending point and all the division points as the first defect points of the normalized scratch type defect.

[0094] The target distance is determined based on a division parameter and a second field of view range corresponding to the fine scanning mode, and the division parameter is greater than 0 and less than 1. Specifically, the target distance can be the product of the division parameter and the minimum side of the second field of view range.

[0095] For example, assuming that the division parameter is 0.5 and the second field of view range is 0.97mm×0.73mm, the target distance can be 0.5×0.73mm=0.365mm. That is, a division point is set every 0.365mm to convert the scratch type defect into a series of points to obtain the first defect points.

[0096] In this way, a plurality of defect points can be used to represent the scratch type defect, and the positions of these defect points can be used as the sub-aperture of the scratch type defect, so that the accurate image information of the scratch can be extracted subsequently.

[0097] In the embodiment of the present application, after the target defect points are determined by the sub-aperture images obtained by the coarse scanning, the target defect points need to be scanned accurately by the fine scanning mode with high magnification and small field of view, so as to extract the accurate image information of the defects. However, if the single target defect point is simply taken as the reference for path planning of the fine scanning, the adjacent defect points will be in the field of view of each other, which will likely cause a large amount of repeated scanning, thereby wasting the scanning time and data storage space. Therefore, in an embodiment of the present application, the clustering algorithm can be used to cluster the target defect points first. Through clustering, the target defect points close to each other can be classified into a category, and then the target defect points in the same category can be scanned under the same defect sub-aperture, so as to improve the scanning efficiency and reduce unnecessary repeated operations.

[0098] Specifically, in step 233, the target defect points are clustered to obtain the defect point clustering result, which can include the following steps A1 to A4.

[0099] Step A1: each target defect point is classified into a category, and the Euclidean distance between each two categories is determined;

[0100] Step A2: in the case that the Euclidean distance is less than or equal to a preset distance threshold, the two categories corresponding to the Euclidean distance are classified into the same category, and a new clustering result is obtained;

[0101] Step A3: the Euclidean distance between each two categories in the new clustering result is determined again;

[0102] Step A4: the above steps A2 to A3 are repeatedly executed until the Euclidean distance between each two categories in the new clustering result is greater than the preset distance threshold, and the new clustering result is determined as the defect point clustering result.

[0103] The Euclidean distance can represent the similarity between two categories. When the Euclidean distance between two categories is determined, the category center of each category can be determined first, and then the Euclidean distance between two categories can be determined based on the category center of each category by using the following formula (3):

[0104]

[0105] wherein L represents the Euclidean distance between two categories, (x n , y n ) represents the category center coordinates of one category, and (x m , y m ) represents the category center coordinates of another category.

[0106] The preset distance threshold can be determined according to the corresponding second field of view range under the fine scanning mode. Specifically, the preset distance threshold is less than the minimum side of the second field of view range.

[0107] The class center can be the centroid of the points in the class. Specifically, the class center of each class can be determined according to formula (4) as follows:

[0108]

[0109] where (x z , y z ) represents the centroid coordinates of each class, i.e., the class center coordinates; n is the number of points in the class, and (x i , y i ) represents the coordinates of the i-th point in the class.

[0110] It can be understood that the clustering method of steps A1-A4 can be regarded as a hierarchical clustering method.

[0111] In this embodiment, for example, Figure 4 A schematic diagram of the defect point clustering result obtained after clustering the target defect points is shown, and reference is made to FIG. 6. Figure 4 As shown in FIG. 6, the target defect points in the same dashed circle are classified into the same class. In this way, by clustering, the target defect points with close distances can be classified into one class, so that the target defect points in the same class can be placed under the same defect sub-aperture for subsequent scanning, unnecessary repeated scanning is reduced, and the scanning efficiency is improved.

[0112] In one embodiment, in the case where the defects on the surface of the optical element to be measured only include pitting-type defects, the pitting points can be determined as the target defect points. In the case where the defects on the surface of the optical element to be measured only include scratch-type defects, the starting point, the ending point, and all the division points of the scratch-type defects can be determined as the target defect points.

[0113] Step 240: Determine the class center of each defect point class in the defect point clustering result, and determine the class center as the second sub-aperture position.

[0114] After obtaining the defect point clustering result by clustering the target defect points, for each defect point class in the defect point clustering result, the class center can be determined according to formula (4) above, and then the class center of each defect point class is taken as the sub-aperture position in subsequent fine scanning, i.e., the second sub-aperture position, i.e., the scanning position in fine scanning. In this way, the sub-aperture covering all the defects can be obtained.

[0115] Step 250: Path planning is performed on all the second sub-aperture positions with the objective of the shortest scanning path, to obtain a fine scanning path.

[0116] After the second sub-aperture positions are determined, the scanning order of the second sub-aperture positions needs to be planned. Specifically, the path planning task of the second sub-aperture positions can be converted into a travelling salesman problem (TSP). That is, for n second sub-aperture positions, the limitation of the planned scanning path is that each second sub-aperture position can only be reached once, and finally returned to the starting position. At the same time, the goal of path planning is to require the planned scanning path to be the minimum value among all paths. In this way, the shortest scanning path of the CMOS microscope camera accurate scanning can be determined through path planning, ensuring that the scanning process time is optimal and all defect cluster sub-apertures are scanned, reducing the scanning time.

[0117] For example, the path of all second sub-aperture positions can be planned based on a heuristic algorithm to obtain a precise scanning path. The heuristic algorithm can include at least one of an annealing algorithm, a genetic algorithm, a particle swarm algorithm, and an ant colony algorithm.

[0118] As shown in Table 1 below, the performance of several heuristic algorithms in the path planning problem of the second sub-aperture positions is compared:

[0119] Table 1

[0120]

[0121]

[0122] In Table 1, K-means is a k-means clustering algorithm, distance refers to the path distance of scanning, and time refers to the operation time of the computing device. As can be seen from Table 1, the ant colony algorithm is optimal in terms of path distance and operation time, and therefore, as a preferred embodiment, the ant colony algorithm can be preferentially selected as the algorithm for scanning path planning.

[0123] Step 260: controlling the CMOS microscope camera to scan the optical element to be measured in a precise scanning manner based on the precise scanning path.

[0124] The second sub-aperture position is the position of the CMOS microscope camera when collecting a sub-aperture image in a precise scanning manner, and the magnification of the CMOS microscope camera in the precise scanning manner is greater than 1.

[0125] In combination with Figure 1The dark-field microscopy imaging measurement system shown can be used to plan a fine scanning path. The computer device 14 can control the translation mechanism of the electronically controlled translation stage 13 to control the CMOS microscope camera 11 to perform a precise scan of the optical element 100 under test in fine scanning mode according to the fine scanning path. During the fine scanning process, the CMOS microscope camera 11 acquires sub-aperture images at each second sub-aperture position in order to further extract precise image information of defects.

[0126] In one embodiment, controlling a CMOS microscope camera to perform a fine-scan path-based scan of the optical element under test may include: controlling the CMOS microscope camera to perform a fine-scan path-based scan of the optical element under test with a second field of view. The second field of view is determined based on a magnification factor, the number of pixels in the CMOS microscope camera, and the resolution of the CMOS microscope camera. The magnification factor is greater than 1, and the second field of view is smaller than the first field of view.

[0127] For example, Figure 5 A schematic diagram of the planned fine scanning path is shown. (Refer to...) Figure 5 As shown, positions 1-10 are the second sub-aperture positions, and position 7, enclosed by a triangle, is the starting point of the fine scan path. Following the direction of the arrows, the fine scan path passes through each second sub-aperture position sequentially before returning to position 7. In fine scan mode, assuming the CMOS microscope camera has a magnification of 4.5, a pixel size of 3.45μm × 3.45μm, and a sub-aperture image resolution of 1280 pixels × 960 pixels, then in fine scan mode, the second field of view corresponding to a single sub-aperture image acquired by the CMOS microscope camera is: (3.45μm × 1280 pixels / 4.5) × (3.45μm × 960 pixels / 4.5) ≈ 0.97mm × 0.73mm. The computer equipment can control the CMOS microscope camera along... Figure 5 The fine scanning path shown starts scanning from position 7. Each time it reaches a second sub-aperture position, it acquires a sub-aperture image from the CMOS microscope camera at 4.5x magnification within a 0.97mm × 0.73mm field of view, until it returns to position 7. This allows for the scanning of all defect cluster sub-apertures in the shortest possible time, reducing scanning time.

[0128] The camera scanning method and device for optical element surface defect detection provided by the embodiment of the present application first controls the CMOS microscope camera to perform overall scanning on the to-be-detected optical element based on a preset coarse scanning path in a coarse scanning mode with a magnification of 1 based on the dark field microscopic imaging principle, acquires a sub-aperture image collected by the CMOS microscope camera at each first sub-aperture position of the preset coarse scanning path, and extracts defect feature information in the sub-aperture image; then determines a target defect point for representing the defect in the sub-aperture image based on the defect feature information, and clusters the target defect point to obtain a defect point clustering result; then determines a class center of each defect point class in the defect point clustering result, and determines the class center as a second sub-aperture position; finally, path planning is performed on all the second sub-aperture positions to obtain a fine scanning path with the shortest scanning path as the target, and the CMOS microscope camera is controlled to perform scanning on the to-be-detected optical element based on the fine scanning path in a fine scanning mode with a magnification greater than 1, and the CMOS microscope camera collects a sub-aperture image at the second sub-aperture position in the fine scanning process. In this way, the defect points on the to-be-detected optical element are classified by clustering, and the class center is used as the sub-aperture position in the fine scanning process, which can reduce repeated scanning of adjacent defects, reduce data storage and reduce image splicing errors in the later stage. Moreover, the shortest fine scanning path is planned according to the class center, and then the CMOS microscope camera is controlled to scan the to-be-detected optical element according to the planned shortest fine scanning path in the fine scanning mode with a sub-aperture magnification and with the class center as the sub-aperture image collection position, which can ensure the optimal scanning time and scan all the defect clustering sub-apertures, thereby reducing the scanning time and the detection time.

[0129] Based on the same inventive concept, the embodiment of the present application provides a camera scanning device for optical element surface defect detection. Since the principle for solving the technical problem of the device is similar to the camera scanning method for optical element surface defect detection provided by the embodiment of the present application, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described here.

[0130] As shown in Figure 6 The camera scanning device for optical element surface defect detection provided by the embodiment of the present application can include a first scanning unit 610, an information extraction unit 620, a clustering unit 630, a determination unit 640, a path planning unit 650, and a second scanning unit 660.

[0131] The first scanning unit 610 is configured to control the CMOS microscope camera to perform overall scanning on the to-be-detected optical element based on a preset coarse scanning path in a coarse scanning mode based on the dark field microscopic imaging principle; and the magnification of the CMOS microscope camera in the coarse scanning mode is 1.

[0132] The information extraction unit 620 is configured to acquire a sub-aperture image collected by the CMOS microscope camera at each first sub-aperture position of the preset rough scanning path, and extract defect feature information in the sub-aperture image.

[0133] The clustering unit 630 is configured to determine a target defect point based on the defect feature information, and cluster the target defect point to obtain a defect point clustering result, the target defect point being used to represent a defect in the sub-aperture image.

[0134] The determination unit 640 is configured to determine a class center of each defect point class in the defect point clustering result, and determine the class center as a second sub-aperture position.

[0135] The path planning unit 650 is configured to perform path planning on all the second sub-aperture positions with the shortest scanning path as a target, to obtain a fine scanning path.

[0136] The second scanning unit 660 is configured to control the CMOS microscope camera to perform scanning on the optical element to be measured in a fine scanning manner based on the fine scanning path, the second sub-aperture position being a position at which the CMOS microscope camera collects a sub-aperture image in the fine scanning manner, and a magnification of the CMOS microscope camera in the fine scanning manner being greater than 1.

[0137] It should be understood that the units included in the camera scanning device for optical element surface defect detection above are only logical divisions according to the functions implemented by the device, and in actual application, the units can be stacked or split. The camera scanning device for optical element surface defect detection provided in the embodiment implements functions corresponding to the camera scanning method for optical element surface defect detection provided in the above embodiment. For more detailed processing procedures implemented by the device, the above method embodiment (1) has been described in detail, and will not be described in detail here.

[0138] Another embodiment of the present application also provides a computer device, which comprises a processor and a memory; the memory is configured to store computer program code, the computer program code comprising computer instructions; when the processor executes the computer instructions, the computer device performs each step of the camera scanning for optical element surface defect detection in the method procedure shown in the above method embodiment.

[0139] Another embodiment of the present application also provides a computer readable storage medium, which stores computer instructions, and when the computer instructions run on a computer device, the computer device performs each step of the camera scanning for optical element surface defect detection in the method procedure shown in the above method embodiment.

[0140] While the preferred embodiments of the application have been described, additional variations and modifications can be made to these embodiments by those skilled in the art once they have the benefit of the present disclosure without departing from the spirit and scope of the application. Accordingly, it is intended that the appended claims include all such modifications and variations as fall within the scope of the present application.

[0141] It is apparent that those skilled in the art can make various changes and modifications to the application without departing from the spirit and scope of the application. It is therefore intended that the present application cover all such changes and modifications that are within its scope.

Claims

1. A camera scanning method for optical element surface defect detection, characterized in that, The method comprises the following steps: Based on the principle of dark field microscopic imaging, a CMOS microscope camera is controlled to perform overall scanning on a to-be-tested optical element based on a preset coarse scanning path in a coarse scanning mode; wherein the magnification of the CMOS microscope camera in the coarse scanning mode is 1; Sub-aperture images collected by the CMOS microscope camera at each first sub-aperture position of the preset coarse scanning path are acquired, and defect feature information in the sub-aperture images is extracted; Based on the defect feature information, target defect points are determined, and the target defect points are clustered to obtain defect point clustering results, wherein the target defect points are used to represent defects in the sub-aperture images; The class centers of each defect point class in the defect point clustering results are determined, and the class centers are determined as second sub-aperture positions; With the shortest scanning path as the target, path planning is performed on all the second sub-aperture positions to obtain a fine scanning path; The CMOS microscope camera is controlled to perform scanning on the to-be-tested optical element based on the fine scanning path in a fine scanning mode, the second sub-aperture positions are positions at which the CMOS microscope camera collects sub-aperture images in the fine scanning mode, and the magnification of the CMOS microscope camera in the fine scanning mode is greater than 1; The method further comprises the following steps: Based on the defect feature information, a defect type is determined; In the case where the defect type includes a pit type and a scratch type, the defects of the scratch type are normalized into first defect points; The first defect points and second defect points of the pit type are determined as target defect points, and the target defect points are clustered to obtain defect point clustering results; The method further comprises the following steps: The starting point and the ending point of the defects of the scratch type are determined; From the starting point to the ending point, the defects of the scratch type are divided once every target distance; wherein the target distance is determined based on a division parameter and a second field of view range corresponding to the fine scanning mode, and the division parameter is greater than 0 and less than 1; The starting point, the ending point and all division points of the division are determined as the first defect points of the normalized defects of the scratch type.

2. The camera scanning method for optical element surface defect inspection according to claim 1, wherein, The method further comprises the following steps: Step A1: Each target defect point is classified into a class, and the Euclidean distance between each two classes is determined; Step A2: In the case where the Euclidean distance is less than or equal to a preset distance threshold, the two classes corresponding to the Euclidean distance are classified into the same class to obtain a new clustering result; Step A3: The Euclidean distance between each class in the new clustering result is re-determined; Step A4: The above steps A2 to A3 are repeatedly executed until the Euclidean distance between each class in the new clustering result is greater than the preset distance threshold, and the new clustering result is determined as the defect point clustering result.

3. The camera scan method of optical element surface defect detection according to any one of claims 1 to 2, wherein, The method further comprises the following steps: Filtering the sub-aperture image to obtain a first image; Performing binaryzation processing on the first image to obtain a binary image; Performing digital morphological processing on the binary image to obtain a second image; Extracting a target pixel point with a gray value greater than a preset gray threshold in the second image, and determining coordinate information of the target pixel point as defect feature information in the sub-aperture image.

4. The camera scan method of optical element surface defect detection according to any one of claims 1 to 2, wherein, The control CMOS microscope camera in a coarse scanning mode to perform overall scanning on the optical element to be measured based on a preset coarse scanning path, comprising: Controlling the CMOS microscope camera to perform overall scanning on the optical element to be measured based on the preset coarse scanning path with a magnification of 1 times and a step distance of a distance of a first field of view range; Wherein, the preset coarse scanning path includes a back-shaped path or a grid-shaped path, and the first field of view range is determined based on the magnification, the pixels of the CMOS microscope camera and the resolution of the CMOS microscope camera.

5. The camera scanning method for optical element surface defect detection of claim 4, wherein, The control CMOS microscope camera in a coarse scanning mode to perform overall scanning on the optical element to be measured based on a preset coarse scanning path, comprising: Controlling the CMOS microscope camera to perform overall scanning on the optical element to be measured based on the preset coarse scanning path with a magnification of 1 times and a step distance of a distance of a first field of view range; Wherein, the preset coarse scanning path includes a back-shaped path or a grid-shaped path, and the first field of view range is determined based on the magnification, the pixels of the CMOS microscope camera and the resolution of the CMOS microscope camera.

6. A camera scanning apparatus for optical element surface defect detection, characterized in that, The control CMOS microscope camera in a coarse scanning mode to perform overall scanning on the optical element to be measured based on a preset coarse scanning path, comprising: Controlling the CMOS microscope camera to perform overall scanning on the optical element to be measured based on the preset coarse scanning path with a magnification of 1 times and a step distance of a distance of a first field of view range; Wherein, the preset coarse scanning path includes a back-shaped path or a grid-shaped path, and the first field of view range is determined based on the magnification, the pixels of the CMOS microscope camera and the resolution of the CMOS microscope camera. Including: The first scanning unit is configured to control the CMOS microscope camera to perform overall scanning on the optical element to be measured based on a preset coarse scanning path in a coarse scanning mode based on the dark field microscopic imaging principle; wherein the magnification of the CMOS microscope camera in the coarse scanning mode is 1; The information extraction unit is configured to acquire a sub-aperture image collected by the CMOS microscope camera at each first sub-aperture position of the preset coarse scanning path, and extract defect feature information in the sub-aperture image; The clustering unit is configured to determine target defect points based on the defect feature information, and cluster the target defect points to obtain a defect point clustering result, wherein the target defect points are used to represent defects in the sub-aperture image; The determination unit is configured to determine a class center of each defect point class in the defect point clustering result, and determine the class center as a second sub-aperture position; The path planning unit is configured to plan a path for all the second sub-aperture positions to obtain a fine scanning path, with the shortest scanning path as the target; The second scanning unit is configured to control the CMOS microscope camera to perform scanning on the optical element to be measured based on the fine scanning path in a fine scanning mode, wherein the second sub-aperture position is a position at which the CMOS microscope camera collects a sub-aperture image in the fine scanning mode, and the magnification of the CMOS microscope camera in the fine scanning mode is greater than 1; The clustering unit is specifically configured to: Determine a defect type based on the defect feature information; In a case where the defect types include the pit type and the scratch type, the defects of the scratch type are normalized as first defect points; The first defect points and second defect points of the pit type are determined as target defect points, and the target defect points are clustered to obtain a defect point clustering result; The clustering unit is specifically configured to: determine a start point and an end point of the defects of the scratch type; divide the defects of the scratch type every target distance from the start point to the end point; wherein the target distance is determined based on a division parameter and a second field of view range corresponding to the fine scanning mode, and the division parameter is greater than 0 and less than 1; determine the start point, the end point and all division points of the division as the first defect points of the defects of the scratch type after normalization.

7. A computer device, comprising: The computer device includes a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the camera scanning method for detecting the surface defects of the optical element according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer program is stored and executed by the processor to make the processor execute the camera scanning method for detecting the surface defects of the optical element according to any one of claims 1 to 5.

Images