Camera scanning method and device for detecting surface defects of optical element

Abstract

The invention discloses a camera scanning method and device for optical element surface defect detection, and relates to the technical field of optical detection.The method comprises the steps that based on the dark field microscopic imaging principle, a CMOS microscope camera is controlled to conduct overall scanning based on a preset coarse scanning path on a to-be-detected optical element in a coarse scanning mode with the magnification factor being 1; acquiring a sub-aperture image acquired by the CMOS microscope camera at each first sub-aperture position of the preset coarse scanning path, and extracting defect feature information in the sub-aperture image; determining target defect points based on the defect feature information, and clustering the target defect points to obtain a defect point clustering result; performing path planning on class centers of all defect point classes by taking the shortest scanning path as a target to obtain a fine scanning path; and controlling the CMOS microscope camera to scan the optical element to be detected based on the fine scanning path in a fine scanning mode with the magnification times greater than 1. According to the invention, the scanning time and the data storage amount can be reduced, and the later image splicing error is reduced.

Description

Technical Field

[0001] The present invention relates to the field of optical detection technology, and more specifically to a camera scanning method and device for detecting surface defects of optical elements. Background Art

[0002] Optical components have been widely used in large astronomical telescope systems, X-ray laser systems, inertial confinement nuclear fusion systems and other fields. If there are defects on their surfaces, it will greatly affect the performance of the optical system. In order to meet the imaging quality of the optical system, ensure the operation safety of the optical system and the use function of the optical system, the requirements for the surface quality of optical components are becoming higher and higher. Accordingly, it is of great significance to realize the rapid and accurate detection of surface defects of optical components.

[0003] In the related art, a charge coupled device (CCD) can be used to obtain an image of the entire optical element to be tested at one time, and then the defects are digitally described with the help of image processing technology. Although this method is simple to operate, in order to obtain fine defect information (such as micrometer level), it is necessary to use a CCD with higher resolution, which is costly and has a relatively limited detection aperture range. In view of this, in order to ensure the high resolution of the defect image, the related art uses zoom technology to collect the enlarged sub-aperture image of the entire optical element to be tested, and obtains the full-size image of the optical element to be tested by the sub-aperture splicing method, and then the full-size image is used to identify defects and realize defect detection, which can realize the detection of large-aperture optical elements while ensuring optical resolution. However, this method requires the overall scanning of the optical element to be tested to be completed under the condition of enlarged sub-aperture, which increases the detection time. Moreover, when the number of defects is large and the distance between defects is close, a large number of sub-aperture images with repeated defects will be collected, which not only increases the amount of stored data, but also easily leads to the accumulation of subsequent image splicing errors, affecting the detection effect. Summary of the invention

[0004] In view of the above problems existing in the prior art, an embodiment of the present invention provides a camera scanning method and device for detecting surface defects of optical components.

[0005] The camera scanning method for detecting surface defects of an optical element provided by an embodiment of the present invention comprises:

[0006] Based on the principle of dark field microscopic imaging, the CMOS microscope camera is controlled to perform an overall scan of the optical element to be measured based on a preset rough scanning path in a rough scanning mode; wherein the magnification of the CMOS microscope camera in the rough scanning mode is 1;

[0007] Acquire a sub-aperture image captured by the CMOS microscope camera at each first sub-aperture position of the preset rough scanning path, and extract defect feature information in the sub-aperture image;

[0008] Determine target defect points based on the defect feature information, and cluster the target defect points to obtain defect point clustering results, wherein the target defect points are used to characterize defects in the sub-aperture image;

[0009] Determine the class center of each defect point class in the defect point clustering result, and determine the class center as the second sub-aperture position;

[0010] Taking the shortest scanning path as the goal, path planning is performed for all the second sub-aperture positions to obtain a precise scanning path;

[0011] The CMOS microscope camera is controlled to scan the optical element to be measured based on the fine scanning path in a fine scanning manner, the second sub-aperture position is the position where the CMOS microscope camera collects a sub-aperture image in the fine scanning manner, and the magnification of the CMOS microscope camera in the fine scanning manner is greater than 1.

[0012] Optionally, determining target defect points based on the defect feature information, and clustering the target defect points to obtain defect point clustering results includes:

[0013] Determining the defect type based on the defect characteristic information;

[0014] In the case where the defect types include pitting type and scratching type, normalizing the scratching type defect to the first defect point;

[0015] The first defect point and the second defect point of the pitting type are determined as target defect points, and the target defect points are clustered to obtain a defect point clustering result.

[0016] Optionally, normalizing the scratch-type defect to a first defect point includes:

[0017] Determine the starting point and the ending point of the scratch-type defect;

[0018] From the starting point to the end point, the scratch-type defect is divided once at each target distance; wherein the target distance is determined based on a division parameter and a second field of view range corresponding to the fine scanning mode, and the division parameter is greater than 0 and less than 1;

[0019] The starting point, the ending point and all the divided points are determined as the first defect point after the scratch-type defect is normalized.

[0020] Optionally, clustering the target defect points to obtain defect point clustering results includes:

[0021] Step A1: classify each target defect point into a class, and determine the Euclidean distance between every two classes;

[0022] Step A2: when the Euclidean distance is less than or equal to a preset distance threshold, classifying the two classes corresponding to the Euclidean distance into the same class to obtain a new clustering result;

[0023] Step A3: re-determine the Euclidean distance between each cluster in the new clustering result;

[0024] Step A4: Repeat the above steps A2 to A3 until the Euclidean distances between the various clusters in the new clustering result are greater than the preset distance threshold, and determine the new clustering result as the defect point clustering result.

[0025] Optionally, extracting defect feature information from the sub-aperture image includes:

[0026] Performing filtering processing on the sub-aperture image to obtain a first image;

[0027] Performing binarization processing on the first image to obtain a binarized image;

[0028] Performing digital morphological processing on the binary image to obtain a second image;

[0029] Target pixel points whose grayscale values ​​are greater than a preset grayscale threshold value are extracted from the second image, and coordinate information of the target pixel points is determined as defect feature information in the sub-aperture image.

[0030] Optionally, controlling the CMOS microscope camera to perform an overall scan of the optical element to be measured in a rough scanning manner based on a preset rough scanning path includes:

[0031] Controlling the CMOS microscope camera to perform an overall scan of the optical element to be measured based on the preset rough scanning path at a magnification of 1 and a stepping distance within the first field of view;

[0032] Wherein, the preset rough scanning path includes a U-shaped path or a grid-shaped path, and the first field of view range is determined based on the magnification, the pixels of the CMOS microscope camera and the resolution of the CMOS microscope camera.

[0033] Optionally, controlling the CMOS microscope camera to scan the optical element to be measured in a fine scanning manner based on the fine scanning path includes:

[0034] Controlling the CMOS microscope camera to scan the optical element to be measured based on the precise scanning path in a second field of view;

[0035] Among them, the second field of view range is determined based on the multiple magnification, the pixels of the CMOS microscope camera and the resolution of the CMOS microscope camera, the multiple magnification is a magnification greater than 1, and the second field of view range is smaller than the first field of view range.

[0036] An embodiment of the present invention provides a camera scanning device for detecting surface defects of an optical element, comprising:

[0037] A first scanning unit is used to control the CMOS microscope camera to perform an overall scan of the optical element to be measured based on a preset rough scanning path in a rough scanning mode based on the principle of dark field microscopic imaging; wherein the magnification of the CMOS microscope camera in the rough scanning mode is 1;

[0038] An information extraction unit, used for acquiring a sub-aperture image collected by the CMOS microscope camera at each first sub-aperture position of the preset rough scanning path, and extracting defect feature information in the sub-aperture image;

[0039] A clustering unit, used for determining target defect points based on the defect feature information, and clustering the target defect points to obtain defect point clustering results, wherein the target defect points are used to characterize defects in the sub-aperture image;

[0040] A determination unit, used to determine the class center of each defect point class in the defect point clustering result, and determine the class center as the second sub-aperture position;

[0041] A path planning unit, configured to perform path planning for all the second sub-aperture positions with the goal of minimizing the scanning path, to obtain a precise scanning path;

[0042] A second scanning unit is used to control the CMOS microscope camera to scan the optical element to be measured based on the fine scanning path in a fine scanning manner, the second sub-aperture position is the position where the CMOS microscope camera collects the sub-aperture image in the fine scanning mode, and the magnification of the CMOS microscope camera in the fine scanning mode is greater than 1.

[0043] An embodiment of the present invention provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes any one of the above-mentioned camera scanning methods for detecting surface defects of optical elements.

[0044] An embodiment of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes any one of the above-mentioned camera scanning methods for detecting surface defects of optical elements.

[0045] The camera scanning method and device for detecting defects on the surface of an optical element provided by an embodiment of the present invention first control a CMOS microscope camera to perform an overall scan of the optical element to be tested based on a preset coarse scanning path in a coarse scanning mode with a magnification of 1 based on the principle of dark field microscopic imaging, obtain a sub-aperture image collected by the CMOS microscope camera at each first sub-aperture position of the preset coarse scanning path, and extract defect feature information in the sub-aperture image; then determine a target defect point for characterizing the defect in the sub-aperture image based on the defect feature information, cluster the target defect points, and obtain a defect point clustering result; then determine the class center of each defect point class in the defect point clustering result, and determine the class center as the second sub-aperture position; finally, with the shortest scanning path as the goal, perform path planning for all second sub-aperture positions to obtain a fine scanning path, control the CMOS microscope camera to scan the optical element to be tested based on the fine scanning path in a fine scanning mode with a magnification greater than 1, and during the fine scanning process, the CMOS microscope camera collects sub-aperture images at the second sub-aperture position. In this way, the defect points on the optical element to be tested are classified by clustering, and the cluster center is used as the sub-aperture position in the fine scanning process, which can reduce the repeated scanning of adjacent defects, reduce the amount of data storage and reduce the subsequent image stitching error; moreover, by planning the shortest fine scanning path according to the cluster center, and then controlling the CMOS microscope camera to scan the optical element to be tested according to the planned shortest fine scanning path in a fine scanning manner of enlarging the sub-aperture and using the cluster center as the acquisition position of the sub-aperture image, it can ensure that the scanning time is optimal and all defect clustering sub-apertures are scanned, thereby reducing the scanning time and detection time. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 A schematic diagram of the structure of a dark field microscopic imaging measurement system provided by an embodiment of the present invention;

[0048] Figure 2 A schematic flow chart of a camera scanning method for detecting surface defects of an optical element provided by an embodiment of the present invention;

[0049] Figure 3 A schematic diagram of a path for scanning by a CMOS microscope camera according to an embodiment of the present invention in a grid-type path;

[0050] Figure 4 A schematic diagram of a defect point clustering result obtained after clustering target defect points provided by an embodiment of the present invention;

[0051] Figure 5 A schematic diagram of a planned precise scanning path provided by an embodiment of the present invention;

[0052] Figure 6 A schematic structural diagram of a camera scanning device for detecting surface defects of optical components provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] Optical elements are important components in optical systems. Their surface quality directly affects the imaging quality and operational safety of optical systems, and plays an important role in optical systems. Accordingly, it is of great significance to detect surface defects of optical elements.

[0055] The detection methods of optical component surface defects can include artificial visual detection methods and defect detection methods based on machine vision. Among them, artificial visual detection has low detection efficiency, high labor costs, and the detection effect is affected by human errors; while the defect detection method based on machine vision has the characteristics of non-contact, low cost, and high efficiency, which can meet different detection needs.

[0056] In the defect detection method based on machine vision, a charge coupled device (CCD) can be used to obtain the image of the entire optical element to be tested at one time, and then the defects are digitally described with the help of image processing technology. Although this method is simple to operate, in order to obtain fine defect information (such as micrometer level), it is necessary to use a CCD with higher resolution, which is costly and the detection aperture range is relatively limited. In view of this, in order to ensure the high resolution of the defect image, the related technology uses zoom technology to collect the enlarged sub-aperture image of the entire optical element to be tested, and obtains the full-size image of the optical element to be tested through image stitching technology, and then the full-size image is used to identify defects, which can realize the detection of large-aperture optical elements while ensuring optical resolution. However, this method requires the overall scanning of the optical element to be tested to be completed under the condition of enlarged sub-aperture, which increases the detection time. Moreover, when the number of defects is large and the distance between defects is close, a large number of sub-aperture images with repeated defects will be collected, which not only increases the amount of stored data, but also easily leads to the accumulation of subsequent image stitching errors, affecting the detection effect.

[0057] In view of this, an embodiment of the present invention provides a camera scanning method for detecting defects on the surface of an optical element, which can be applied to large-aperture optical elements. Based on the principle of dark-field microscopic imaging, the method first controls a complementary metal oxide semiconductor (CMOS) microscope camera to perform an overall scan of the optical element to be tested based on a preset rough scanning path in a rough scanning mode with a magnification of 1, so as to extract defect feature information using the sub-aperture image collected during the rough scanning process; then, based on the defect feature information, the target defect points that can represent the defects in the sub-aperture image are determined, and the target defect points are clustered; then, each type of center is used as the position for collecting sub-aperture images in a fine scanning mode with a magnification greater than 1 to plan the shortest fine scanning path; then, the CMOS microscope camera is controlled to scan the optical element to be tested based on the fine scanning path in a fine scanning mode, and the sub-aperture images at the positions of the cluster centers are collected during the fine scanning process. After that, the sub-aperture images obtained by fine scanning can be spliced ​​using the sub-aperture splicing method to obtain a full-size image of the optical element to be tested, and then the full-size image can be used to identify defects and realize defect detection. In this way, by clustering defect points and using the cluster center as the sub-aperture position in the fine scanning process of the enlarged sub-aperture, the repeated scanning of adjacent defects can be reduced, thereby reducing the amount of data storage and reducing the image splicing error in the later stage; moreover, by planning the shortest fine scanning path of the CMOS microscope camera according to the cluster center, the optical element to be tested can be scanned according to the planned shortest fine scanning path during the fine scanning of the enlarged sub-aperture, which can ensure the optimal scanning time and scan all defect clustering sub-apertures, reducing the scanning time and detection time, and ensuring the defect detection effect.

[0058] The camera scanning method for detecting surface defects of optical elements provided in the embodiment of the present invention can be applied to computer equipment in a dark field microscopic imaging measurement system, and the computer equipment may include a terminal device or a server, etc. Among them, the terminal device may include a mobile phone, a computer, a tablet computer, etc.; the server may include an independent server, a cluster server or a cloud server, etc. The camera scanning method for detecting surface defects of optical elements can also be applied to a camera scanning device for detecting surface defects of optical elements set in a computer device, and the camera scanning device for detecting surface defects of optical elements can be implemented by software, hardware or a combination of both.

[0059] Figure 1The schematic diagram of the structure of the dark field microscopic imaging measurement system is shown, and the dark field microscopic imaging measurement system can measure the surface of the optical element to be measured based on the dark field microscopic imaging principle. Among them, dark field microscopy (or dark field microscopy) describes a special microscopic technique in optical microscopy and electron microscopy, which removes light or electrons other than the object to be measured from entering the objective lens of the microscope, so that the background of the field observed in the eyepiece is black, and only the edge of the object to be measured is bright. The principle of dark field microscopic imaging is to block direct light, only allow oblique light to illuminate the object to be measured, and use the scattered light of the object to be measured for imaging. Specifically, the dark field microscope uses a special condenser, which produces an inverted hollow conical illumination area with a focus on the plane of the object to be measured. In places where there is no object to be measured, the oblique light will cross each other and form a dark background, while the oblique light passing through the object to be measured will produce diffraction, reflection or refraction to form a bright sample image.

[0060] Reference Figure 1 As shown, the dark field microscopic imaging measurement system may include a CMOS microscope camera 11, an annular light source 12, an electrically controlled translation stage 13 and a computer device 14. Specifically, when measuring the optical element to be measured, the optical element to be measured 100 may be placed on the stage of the electrically controlled translation stage 13, and the annular light source 12 emits annular light to illuminate the optical element to be measured 100. If there is a defect on the surface of the optical element to be measured 100, the backscattered light caused by the defect will enter the microscopic imaging system through the objective lens of the CMOS microscope camera 11 and converge on the target surface of the CMOS of the CMOS microscope camera 11, while the reflected light reflected by the optical element to be measured 100 does not enter the microscopic imaging system, forming a dark field, so that the defect image forms a bright image under a dark background. The electrically controlled translation stage 13 clamps the CMOS microscope camera 11, and the computer device 14 can drive the CMOS microscope camera 11 to move by controlling the translation mechanism of the electrically controlled translation stage 13. At the same time, the computer device 14 can also obtain the image captured by the CMOS microscope camera 11 and process the image.

[0061] based on Figure 1 The dark field microscopic imaging measurement system shown in FIG. 1 is used as an example to apply the camera scanning method for detecting surface defects of an optical element of the present invention to a computer device 14. Figure 2 to Figure 5 The camera scanning method for detecting surface defects of optical elements of the present invention is described.

[0062] Figure 2 A schematic diagram showing a flow chart of a camera scanning method for detecting surface defects of an optical element provided by an embodiment of the present invention is shown. Figure 2 As shown, the camera scanning method for detecting surface defects of an optical element may include the following steps 210 to 260.

[0063] Step 210: Based on the dark field microscopic imaging principle, the CMOS microscope camera is controlled to perform a rough scanning on the optical element to be measured based on a preset rough scanning path in a rough scanning manner.

[0064] Among them, the magnification of the CMOS microscope camera in the coarse scanning mode is 1.

[0065] For example, the shooting distance between the CMOS microscope camera and the optical element to be measured is set with the goal of clear imaging. For example, in the coarse scanning mode, the shooting distance can be set to 53 mm.

[0066] Specifically, based on the principle of dark-field microscopy, the computer device can control the CMOS microscope camera to perform an overall scan of the optical element to be tested based on a preset rough scanning path with a magnification of 1 and a step distance of the first field of view; wherein the preset rough scanning path includes a U-shaped path or a grid path, and the first field of view is determined based on the magnification, pixels and resolution of the CMOS microscope camera.

[0067] For example, in the coarse scanning mode, the magnification of the CMOS microscope camera is 1, the pixel size is 3.45μm×3.45μm, and the resolution of the sub-aperture image is 1280pixel×960pixel. Then, in the coarse scanning mode, the first field of view corresponding to the single sub-aperture image collected by the CMOS microscope camera is: (3.45μm×1280pixel)×(3.45μm×960pixel)≈4.4mm×3.3mm. At this time, the step distance of the CMOS microscope camera is 4.4mm×3.3mm, that is, the step distance of the X axis is 4.4mm, and the step distance of the Y axis is 3.3mm. Among them, pixel is the basic unit of the image, which means "pixel".

[0068] Specifically, taking the preset rough scanning path as a grid-type path as an example, Figure 3 A schematic diagram of the path of a CMOS microscope camera scanning along a grid-type path is shown, referring to Figure 3 , set the origin O of the workpiece coordinate system XOY to the upper left corner of the optical element to be measured. Figure 1 In the dark field microscopic imaging measurement system shown in FIG. 1 , the annular light source 12 emits annular light to illuminate the surface of the optical element 100 to be measured. The computer device 14 can control the translation mechanism of the electric translation stage 13 to control the CMOS microscope camera 11 to scan rightward along the X-axis starting from the upper left corner of the optical element to be measured according to a grid-type path. For each step, the CMOS microscope camera 11 collects a sub-aperture image (e.g., Figure 3), until the CMOS microscope camera 11 moves to the right edge of the optical element 100 to be tested. Then, the computer device 14 controls the electric-controlled translation stage 13 to clamp the CMOS microscope camera 11 and move it one step along the Y-axis direction. The CMOS microscope camera 11 collects a sub-aperture image at a magnification of 1 at the moved position. Next, the computer device 14 controls the electric-controlled translation stage 13 to clamp the CMOS microscope camera 11 and continue to move left along the X-axis. For each step, a sub-aperture image is collected at a magnification of 1. This process is repeated and repeated until the entire surface of the optical element to be tested is collected. In the process of collecting the sub-aperture image, the scanning step number (i, j) of the electric-controlled translation stage 13 corresponding to the sub-aperture image can be saved, where i=0, 1, 2..., j=0, 1, 2...

[0069] For example, assuming that the first field of view range is 4.4 mm×3.3 mm, then in the coarse scanning mode, the stepping distance of the CMOS microscope camera 11 along the X-axis direction is 4.4 mm, and the stepping distance along the Y-axis direction is 3.3 mm.

[0070] Step 220: Obtain a sub-aperture image captured by the CMOS microscope camera at each first sub-aperture position of a preset rough scanning path, and extract defect feature information in the sub-aperture image.

[0071] Subaperture refers to a technical means of dividing a large-aperture optical element into several small areas for independent observation and processing. It is mainly used to improve the spatial resolution and imaging quality of the optical system.

[0072] When the CMOS microscope camera scans the optical element to be measured along the preset rough scanning path, the position of each step length is used as the first sub-aperture position for collecting the sub-aperture image, and the computer device can obtain the sub-aperture image collected by the CMOS microscope camera at each first sub-aperture position of the preset rough scanning path. After the computer device obtains the sub-aperture image, the image processing technology can be used to extract the defect feature information in the sub-aperture image, and the defect feature information may include defect shape feature data, defect quantity and defect position information, etc.

[0073] For example, the computer device may determine the defect type of the defect based on the defect shape feature data, and the defect type may include a pitting type and a scratching type.

[0074] In one embodiment, extracting defect feature information in a sub-aperture image may include: filtering the sub-aperture image to obtain a first image; binarizing the first image to obtain a binarized image; performing digital morphological processing on the binarized image to obtain a second image; extracting target pixel points in the second image whose grayscale values ​​are greater than a preset grayscale threshold, and determining the coordinate information of the target pixel points as the defect feature information in the sub-aperture image.

[0075] The coordinate information is the defect position information. The preset grayscale threshold can be set based on experience or determined through experiments, for example, it can be set to 240 based on experience.

[0076] For example, the sub-aperture image may be filtered by median filtering to obtain the first image. The noise of the dark field image in the sub-aperture image caused by dust or the like may be removed by median filtering. Median filtering is a nonlinear smoothing filtering method that can eliminate noise while retaining the characteristic information of the image.

[0077] For example, a segmentation threshold T can be determined based on experience or through experiments. For the first image, the pixels of the first image can be traversed. If the pixel value of the pixel is greater than the segmentation threshold T, the pixel value of the pixel is reset to 1; if the pixel value of the pixel is less than or equal to the segmentation threshold T, the pixel is determined as a background point and its pixel value is reset to 0. In this way, the target and the background can be distinguished, the binarization processing of the first image is realized, and the region of interest is retained to the maximum extent. The binarization processing of the first image can be expressed by the following formula (1):

[0078]

[0079] Wherein, f(x, y) represents the pixel at the xth row and yth column in the first image, g(x, y) represents the binarized image obtained after binarization of the first image, and T represents the segmentation threshold.

[0080] For example, digital morphological processing of a binary image may include two parts: erosion and dilation, wherein erosion is the inward contraction of the target boundary, and dilation is the outward expansion of the target boundary. Digital morphological processing may be an open operation of erosion and dilation, or a closed operation of erosion and dilation, wherein the open operation is to erode the image first and then dilate it, and the closed operation is to dilate the image first and then erode it. The closed operation can connect disconnected adjacent targets and fill small holes in the target area. Therefore, in an embodiment of the present invention, the binary image may be digitally morphologically processed using a closed operation. Specifically, the digital morphological processing based on the closed operation may be expressed as the following formula (2):

[0081] F·S=(F⊕S)ΘS (2)

[0082] Among them, F represents the binary image g(x, y), S is the structural element, ⊕ means that the former is expanded by the latter, and Θ means that the former is eroded by the latter.

[0083] In this way, by performing digital morphological processing on the binary image, the position information of the defect in the sub-aperture image is made clearer and the edge is fuller.

[0084] Step 230: Determine target defect points based on the defect feature information, and cluster the target defect points to obtain defect point clustering results.

[0085] Among them, the target defect points are used to characterize the defects in the sub-aperture image, that is, the target defect points are some pixel points that can be used to represent the defects. For example, for pitting-type defects, pitting can be used as target defect points; for scratch-type defects, some characteristic points on the defect scratch can be used to characterize the defect, such as taking points at intervals from the scratch, and determining the interval points and the two endpoints as target defect points.

[0086] For example, defect feature information may include defect shape feature data, defect quantity and defect location information, etc. The defect type of the defect can be determined based on the defect feature information, and then the target defect point used to characterize the defect can be determined based on the characteristics of defects of different defect types.

[0087] Specifically, determining the target defect points based on the defect feature information, clustering the target defect points, and obtaining the defect point clustering results may include the following steps 231 to 233.

[0088] Step 231: Determine the defect type based on the defect feature information;

[0089] Step 232: When the defect types include pitting type and scratching type, normalize the scratching type defect to the first defect point;

[0090] Step 233: determine the first defect point and the second defect point of the pitting type as target defect points, and cluster the target defect points to obtain defect point clustering results.

[0091] In step 231, the defect feature information may include defect shape feature data, and the defect type may include pitting type and scratch type. The defect type may be determined based on the defect feature information using a deep learning algorithm. For example, a large number of sub-aperture sample images may be obtained, and the defect types in the sub-aperture sample images may be marked to obtain label data; at the same time, the defect feature information in the sub-aperture sample images may be extracted to obtain sample defect feature information; then, the sample defect feature information may be input into the defect classification model to be trained to obtain the output result of the defect classification model to be trained, and the loss function value may be determined based on the output result and the label data; the model parameters of the defect classification model to be trained may be adjusted based on the loss function value until the model converges to obtain a trained defect classification model. Afterwards, the defect feature information may be input into the trained defect classification model, and the defect type may be determined using the trained defect classification model.

[0092] In step 232, the defects on the surface of the optical element to be tested may include only pitting defects, or only scratch defects, or both pitting and scratch defects. In the case where the defect types include pitting and scratch types, due to the different characteristics of pitting defects and scratch defects, the length of the scratch defect is longer and cannot be completely covered by one defect sub-aperture. In view of this, in the embodiment of the present invention, the pitting and scratch defects can be normalized, and the scratch defect can be converted into a series of points as the basis for selecting the defect sub-aperture.

[0093] Specifically, normalizing the scratch-type defect to the first defect point may include: determining the starting point and the ending point of the scratch-type defect; dividing the scratch-type defect once at each target distance from the starting point to the ending point; and determining the starting point, the ending point and all the division points as the first defect point after the normalization of the scratch-type defect.

[0094] The target distance is determined based on the division parameter and the second field of view range corresponding to the fine scanning mode, and the division parameter is greater than 0 and less than 1. Specifically, the target distance may be the product of the division parameter and the minimum side of the second field of view range.

[0095] For example, assuming that the division parameter is 0.5 and the second field of view is 0.97 mm × 0.73 mm, the target distance can be 0.5 × 0.73 mm = 0.365 mm. That is, a division point is set every 0.365 mm to convert the scratch-type defect into a series of points to obtain the first defect point.

[0096] In this way, several defect points can be used to characterize the scratch-type defect, and the positions of these defect points can be used as the sub-aperture of the scratch-type defect, so as to subsequently extract the precise image information of the scratch.

[0097] In an embodiment of the present invention, after the target defect points are determined using the sub-aperture images obtained by coarse scanning, it is necessary to use a high-magnification, small-field-of-view fine scanning method to accurately scan these target defect points in order to extract accurate image information of the defect. However, if a single target defect point is used as a reference for path planning for fine scanning, then adjacent defect points will appear in each other's field of view, which may lead to a large number of repeated scans, thereby wasting scanning time and data storage space. Therefore, in one embodiment of the present invention, a clustering algorithm can be used to cluster the target defect points. Through clustering, target defect points with similar distances can be classified into one category, and then the target defect points of the same category can be placed under the same defect sub-aperture for scanning, thereby improving scanning efficiency and reducing unnecessary repeated operations.

[0098] Specifically, in step 233, the target defect points are clustered to obtain defect point clustering results, which may include the following steps A1 to A4.

[0099] Step A1: Classify each target defect point into a class and determine the Euclidean distance between every two classes;

[0100] Step A2: when the Euclidean distance is less than or equal to a preset distance threshold, the two classes corresponding to the Euclidean distance are classified into the same class to obtain a new clustering result;

[0101] Step A3: Re-determine the Euclidean distance between each cluster in the new clustering result;

[0102] Step A4: Repeat the above steps A2 to A3 until the Euclidean distances between the various clusters in the new clustering result are greater than the preset distance threshold, and determine the new clustering result as the defect point clustering result.

[0103] Among them, the Euclidean distance can represent the similarity between two classes. When determining the Euclidean distance between two classes, the class center of each class can be determined first, and then based on the class center of each class, the Euclidean distance between the two classes can be determined using the following formula (3):

[0104]

[0105] Among them, L represents the Euclidean distance between two classes, (x n ,y n ) represents the class center coordinates of a class, (x m ,y m ) represents the class center coordinates of another class.

[0106] The preset distance threshold can be determined according to the second field of view range corresponding to the fine scanning mode. Specifically, the preset distance threshold is smaller than the minimum edge of the second field of view range.

[0107] The class center can be the centroid of each point in the class. Specifically, the class center of each class can be determined according to the following formula (4):

[0108]

[0109] Among them, (x z ,y z ) represents the centroid coordinates of each class, that is, the class center coordinates; n is the number of points in the class, (x i ,y i ) represents the coordinates of the i-th point in the class.

[0110] It can be understood that the clustering method of steps A1 to A4 can be regarded as a hierarchical clustering method.

[0111] In this embodiment, for example, Figure 4 A schematic diagram showing the defect point clustering results obtained after clustering the target defect points is shown. Figure 4 As shown, the target defect points within the same dotted circle are classified into the same category. In this way, through clustering, the target defect points with similar distances can be classified into one category, so that the target defect points of the same category can be subsequently scanned under the same defect sub-aperture, reducing unnecessary repeated scanning and improving scanning efficiency.

[0112] In one embodiment, when the defects on the surface of the optical element to be tested only include pitting defects, these pitting defects can be determined as target defect points. When the defects on the surface of the optical element to be tested only include scratch defects, the starting point, the end point and all the division points of the scratch defect can be determined as target defect points.

[0113] Step 240: Determine the cluster center of each defect point cluster in the defect point clustering result, and determine the cluster center as the second sub-aperture position.

[0114] After clustering the target defect points to obtain the defect point clustering result, for each defect point class in the defect point clustering result, the class center can be determined according to the above formula (4), and then the class center of each defect point class is used as the sub-aperture position in the subsequent fine scanning, that is, the second sub-aperture position, that is, the scanning position in the fine scanning. In this way, a sub-aperture covering all defects can be obtained.

[0115] Step 250: With the goal of minimizing the scanning path, path planning is performed for all second sub-aperture positions to obtain a precise scanning path.

[0116] After determining the second sub-aperture position, it is necessary to plan the scanning order of the second sub-aperture position. Specifically, the path planning task of the second sub-aperture position can be transformed into a traveling salesman problem (TSP). That is, for n second sub-aperture positions, the restriction of the planned scanning path is that each second sub-aperture position can only be reached once, and finally return to the starting position. At the same time, the goal of path planning is to require the planned scanning path to be the minimum value among all paths. In this way, the shortest scanning path for accurate scanning of the CMOS microscope camera can be determined through path planning, ensuring that the scanning process time is optimal and all defective clustered sub-apertures are scanned, reducing the scanning time.

[0117] For example, path planning can be performed on all second sub-aperture positions based on a heuristic algorithm to obtain a precise scanning path, wherein the heuristic algorithm can include at least one of an annealing algorithm, a genetic algorithm, a particle swarm algorithm, and an ant colony algorithm.

[0118] Table 1 below shows the performance comparison of several heuristic algorithms in the path planning problem at the second sub-aperture position:

[0119] Table 1

[0120]

[0121]

[0122] In Table 1, K-means is a k-means clustering algorithm, distance refers to the scanning path distance, and time refers to the computing time of the computing device. As can be seen from Table 1, the ant colony algorithm is optimal in both path distance and computing time. Therefore, as a preferred embodiment, the ant colony algorithm can be preferentially selected as the scanning path planning algorithm.

[0123] Step 260: Control the CMOS microscope camera to scan the optical element to be tested in a fine scanning manner based on the fine scanning path.

[0124] The second sub-aperture position is the position of the CMOS microscope camera when collecting the sub-aperture image in the fine scanning mode, and the magnification of the CMOS microscope camera in the fine scanning mode is greater than 1.

[0125] Combination Figure 1In the dark field microscopic imaging measurement system shown, after planning the precise scanning path, the computer device 14 can control the translation mechanism of the electric control translation stage 13 to control the CMOS microscope camera 11 to accurately scan the optical element 100 to be measured along the precise scanning path in the precise scanning mode. During the precise scanning process, the sub-aperture image at each second sub-aperture position is collected by the CMOS microscope camera 11 to further extract the precise image information of the defect.

[0126] In one embodiment, controlling the CMOS microscope camera to scan the optical element to be measured in a fine scanning manner based on the fine scanning path may include: controlling the CMOS microscope camera to scan the optical element to be measured based on the fine scanning path in a second field of view. The second field of view is determined based on the multiple magnification, the pixels of the CMOS microscope camera, and the resolution of the CMOS microscope camera, the multiple magnification is a magnification greater than 1, and the second field of view is smaller than the first field of view.

[0127] For example, Figure 5 A schematic diagram showing the planned precise scanning path is shown, referring to Figure 5 As shown, positions 1 to 10 are the second sub-aperture positions, and position 7 framed by a triangle is the starting point of the fine scanning path. Along the direction of the arrow, the fine scanning path passes through each second sub-aperture position in turn and returns to position 7. In the fine scanning mode, assuming that the magnification of the CMOS microscope camera is 4.5, the pixel size is 3.45μm×3.45μm, and the resolution of the sub-aperture image is 1280pixel×960pixel, then in the fine scanning mode, the second field of view range corresponding to the single sub-aperture image collected by the CMOS microscope camera is: (3.45μm×1280pixel / 4.5)×(3.45μm×960pixel / 4.5)≈0.97mm×0.73mm. The computer device can control the CMOS microscope camera along Figure 5 The fine scanning path shown starts scanning from position 7, and each time it reaches a second sub-aperture position, it acquires a sub-aperture image collected by the CMOS microscope camera at a magnification of 4.5 and a field of view of 0.97mm×0.73mm, until it returns to position 7. In this way, the scanning of all defect clustering sub-apertures can be completed in the shortest time, reducing the scanning time.

[0128] The camera scanning method and device for detecting defects on the surface of an optical element provided by an embodiment of the present invention first control a CMOS microscope camera to perform an overall scan of the optical element to be tested based on a preset coarse scanning path in a coarse scanning mode with a magnification of 1 based on the principle of dark field microscopic imaging, obtain a sub-aperture image collected by the CMOS microscope camera at each first sub-aperture position of the preset coarse scanning path, and extract defect feature information in the sub-aperture image; then determine a target defect point for characterizing the defect in the sub-aperture image based on the defect feature information, cluster the target defect points, and obtain a defect point clustering result; then determine the class center of each defect point class in the defect point clustering result, and determine the class center as the second sub-aperture position; finally, with the shortest scanning path as the goal, perform path planning for all second sub-aperture positions to obtain a fine scanning path, control the CMOS microscope camera to scan the optical element to be tested based on the fine scanning path in a fine scanning mode with a magnification greater than 1, and during the fine scanning process, the CMOS microscope camera collects sub-aperture images at the second sub-aperture position. In this way, the defect points on the optical element to be tested are classified by clustering, and the cluster center is used as the sub-aperture position in the fine scanning process, which can reduce the repeated scanning of adjacent defects, reduce the amount of data storage and reduce the subsequent image stitching error; moreover, by planning the shortest fine scanning path according to the cluster center, and then controlling the CMOS microscope camera to scan the optical element to be tested according to the planned shortest fine scanning path in a fine scanning manner of enlarging the sub-aperture and using the cluster center as the acquisition position of the sub-aperture image, it can ensure that the scanning time is optimal and all defect clustering sub-apertures are scanned, thereby reducing the scanning time and detection time.

[0129] Based on the same inventive concept, an embodiment of the present invention provides a camera scanning device for detecting surface defects of optical components. Since the principle of solving the technical problem by the device is similar to the camera scanning method for detecting surface defects of optical components provided by an embodiment of the present invention, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0130] like Figure 6 As shown, the camera scanning device for optical element surface defect detection provided by the embodiment of the present invention may include a first scanning unit 610, an information extraction unit 620, a clustering unit 630, a determination unit 640, a path planning unit 650 and a second scanning unit 660.

[0131] The first scanning unit 610 is used to control the CMOS microscope camera to perform a rough scanning on the optical element to be measured based on a preset rough scanning path in a rough scanning mode based on the principle of dark field microscopic imaging; wherein the magnification of the CMOS microscope camera in the rough scanning mode is 1;

[0132] The information extraction unit 620 is used to obtain the sub-aperture image collected by the CMOS microscope camera at each first sub-aperture position of the preset rough scanning path, and extract the defect feature information in the sub-aperture image;

[0133] A clustering unit 630 is used to determine target defect points based on defect feature information, and cluster the target defect points to obtain defect point clustering results. The target defect points are used to characterize defects in the sub-aperture image.

[0134] A determination unit 640, configured to determine a cluster center of each defect point cluster in the defect point clustering result, and determine the cluster center as the second sub-aperture position;

[0135] A path planning unit 650 is used to perform path planning for all second sub-aperture positions with the goal of minimizing the scanning path to obtain a precise scanning path;

[0136] The second scanning unit 660 is used to control the CMOS microscope camera to scan the optical element to be measured based on the fine scanning path in a fine scanning manner. The second sub-aperture position is the position where the CMOS microscope camera collects the sub-aperture image in the fine scanning mode. The magnification of the CMOS microscope camera in the fine scanning mode is greater than 1.

[0137] It should be understood that the units included in the above camera scanning device for detecting surface defects of optical components are only logical divisions based on the functions implemented by the device. In practical applications, the above units can be superimposed or split. In addition, the functions implemented by the camera scanning device for detecting surface defects of optical components provided in this embodiment correspond one-to-one to the camera scanning method for detecting surface defects of optical components provided in the above embodiment. The more detailed processing flow implemented by the device has been described in detail in the above method embodiment 1 and will not be described in detail here.

[0138] Another embodiment of the present invention also provides a computer device, which includes: a processor and a memory; the memory is used to store computer program code, and the computer program code includes computer instructions; when the processor executes the computer instructions, the computer device executes each step of camera scanning for optical element surface defect detection in the method flow shown in the above method embodiment.

[0139] Another embodiment of the present invention further provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on a computer device, the computer device executes each step of camera scanning for optical element surface defect detection in the method flow shown in the above method embodiment.

[0140] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0141] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A camera scanning method for detecting surface defects of optical components, characterized in that: include: Based on the principle of dark field microscopic imaging, the CMOS microscope camera is controlled to perform an overall scan of the optical element to be measured based on a preset rough scanning path in a rough scanning mode; wherein the magnification of the CMOS microscope camera in the rough scanning mode is 1; Acquire a sub-aperture image captured by the CMOS microscope camera at each first sub-aperture position of the preset rough scanning path, and extract defect feature information in the sub-aperture image; Determine target defect points based on the defect feature information, and cluster the target defect points to obtain defect point clustering results, wherein the target defect points are used to characterize defects in the sub-aperture image; Determine the class center of each defect point class in the defect point clustering result, and determine the class center as the second sub-aperture position; Taking the shortest scanning path as the goal, path planning is performed for all the second sub-aperture positions to obtain a precise scanning path; The CMOS microscope camera is controlled to scan the optical element to be measured based on the fine scanning path in a fine scanning manner, the second sub-aperture position is the position where the CMOS microscope camera collects a sub-aperture image in the fine scanning manner, and the magnification of the CMOS microscope camera in the fine scanning manner is greater than 1.

2. The camera scanning method for detecting surface defects of optical components according to claim 1, characterized in that: The step of determining target defect points based on the defect feature information and clustering the target defect points to obtain defect point clustering results includes: Determining the defect type based on the defect characteristic information; In the case where the defect types include pitting type and scratching type, normalizing the scratching type defect to the first defect point; The first defect point and the second defect point of the pitting type are determined as target defect points, and the target defect points are clustered to obtain a defect point clustering result.

3. The camera scanning method for detecting surface defects of optical components according to claim 2, characterized in that: The step of normalizing the scratch-type defect to a first defect point comprises: Determine the starting point and the ending point of the scratch-type defect; From the starting point to the end point, the scratch-type defect is divided once at each target distance; wherein the target distance is determined based on a division parameter and a second field of view range corresponding to the fine scanning mode, and the division parameter is greater than 0 and less than 1; The starting point, the ending point and all the divided points are determined as the first defect point after the scratch-type defect is normalized.

4. The camera scanning method for detecting surface defects of optical components according to claim 2, characterized in that: The step of clustering the target defect points to obtain a defect point clustering result includes: Step A1: classify each target defect point into a class, and determine the Euclidean distance between every two classes; Step A2: when the Euclidean distance is less than or equal to a preset distance threshold, classifying the two classes corresponding to the Euclidean distance into the same class to obtain a new clustering result; Step A3: re-determine the Euclidean distance between each cluster in the new clustering result; Step A4: Repeat the above steps A2 to A3 until the Euclidean distances between the various clusters in the new clustering result are greater than the preset distance threshold, and determine the new clustering result as the defect point clustering result.

5. The camera scanning method for detecting surface defects of an optical element according to any one of claims 1 to 4, characterized in that: The step of extracting defect feature information from the sub-aperture image comprises: Performing filtering processing on the sub-aperture image to obtain a first image; Performing binarization processing on the first image to obtain a binarized image; Performing digital morphological processing on the binary image to obtain a second image; Target pixel points whose grayscale values ​​are greater than a preset grayscale threshold value are extracted from the second image, and coordinate information of the target pixel points is determined as defect feature information in the sub-aperture image.

6. The camera scanning method for detecting surface defects of an optical element according to any one of claims 1 to 4, characterized in that: The method of controlling the CMOS microscope camera to perform an overall scan of the optical element to be measured in a rough scanning manner based on a preset rough scanning path comprises: Controlling the CMOS microscope camera to perform an overall scan of the optical element to be measured based on the preset rough scanning path at a magnification of 1 and a stepping distance within the first field of view; Wherein, the preset rough scanning path includes a U-shaped path or a grid-shaped path, and the first field of view range is determined based on the magnification, the pixels of the CMOS microscope camera and the resolution of the CMOS microscope camera.

7. The camera scanning method for detecting surface defects of optical components according to claim 6, characterized in that: The controlling the CMOS microscope camera to scan the optical element to be tested in a fine scanning manner based on the fine scanning path comprises: Controlling the CMOS microscope camera to scan the optical element to be measured based on the precise scanning path in a second field of view; Among them, the second field of view range is determined based on the multiple magnification, the pixels of the CMOS microscope camera and the resolution of the CMOS microscope camera, the multiple magnification is a magnification greater than 1, and the second field of view range is smaller than the first field of view range.

8. A camera scanning device for detecting surface defects of optical components, characterized in that: include: A first scanning unit is used to control the CMOS microscope camera to perform an overall scan of the optical element to be measured based on a preset rough scanning path in a rough scanning mode based on the principle of dark field microscopic imaging; wherein the magnification of the CMOS microscope camera in the rough scanning mode is 1; An information extraction unit, used for acquiring a sub-aperture image collected by the CMOS microscope camera at each first sub-aperture position of the preset rough scanning path, and extracting defect feature information in the sub-aperture image; A clustering unit, used for determining target defect points based on the defect feature information, and clustering the target defect points to obtain defect point clustering results, wherein the target defect points are used to characterize defects in the sub-aperture image; A determination unit, used to determine the class center of each defect point class in the defect point clustering result, and determine the class center as the second sub-aperture position; A path planning unit, configured to perform path planning for all the second sub-aperture positions with the goal of minimizing the scanning path, to obtain a precise scanning path; A second scanning unit is used to control the CMOS microscope camera to scan the optical element to be measured based on the fine scanning path in a fine scanning manner, the second sub-aperture position is the position where the CMOS microscope camera collects the sub-aperture image in the fine scanning mode, and the magnification of the CMOS microscope camera in the fine scanning mode is greater than 1.

9. A computer device, characterized in that: The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the camera scanning method for detecting surface defects of optical elements as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the processor executes a camera scanning method for detecting surface defects of an optical element as claimed in any one of claims 1 to 7.

Images