Efficient multi-light-field contact lens flaw detection method and device based on cloud-machine vision

Abstract

The invention relates to the technical field of cloud and machine vision, in particular to an efficient multi-light-field contact lens flaw detection method and device based on cloud-machine vision, and the method comprises the steps: preprocessing a multi-light-field original image to obtain a point light source image, a bright field image, a dark field image and a preprocessed image; performing glass bowl detection on the preprocessed image, and extracting geometric features of the glass bowl; contour feature extraction is performed on the printed contact lenses in the glass bowl area based on the geometric features of the glass bowl; and carrying out multi-light-field-based surface flaw detection on the basis of the plurality of feature images of the printed contact lens. According to the method, the multi-light-field original image is preprocessed, so that various feature images are obtained, and flaw detection of edges and printing areas is carried out from multiple aspects. The contact lens image acquired by the high-resolution camera is further utilized, and the image is automatically identified, so that the purposes of reducing the labor cost, saving the time and improving the production yield are achieved.

Description

Technical Field

[0001] The present invention relates to the field of cloud and machine vision technology, and in particular to a cloud-machine vision-based efficient multi-light field contact lens defect detection method and device. Background Art

[0002] The main manufacturing methods of contact lenses include turning, casting, etc. Due to the different lens materials, various types of defects are prone to occur during the production process. For example, abnormal lens size, various surface defects, etc.

[0003] In the prior art, printed areas and edges of printed contact lenses are generally detected manually, which is prone to various false detections and missed detections, and has high efficiency and low cost. Summary of the invention

[0004] In view of this, an object of the present invention is to provide an efficient multi-light field contact lens defect detection method and device based on cloud-machine vision to solve the problems in the background technology.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] The present invention provides an efficient multi-light field contact lens defect detection method based on cloud-machine vision, comprising the steps of:

[0007] Acquire a multi-light field original image of a printed contact lens immersed in water in a glass bowl, and pre-process the multi-light field original image to obtain a point light source image, a bright field image, a dark field image, and a pre-processed image, wherein the pre-processed image is obtained by filtering the bright field image;

[0008] Performing glass bowl detection on the preprocessed image, and extracting geometric features of the glass bowl when the glass bowl exists in the preprocessed image; performing contour feature extraction on the printed contact lens in the glass bowl area based on the geometric features of the glass bowl;

[0009] Obvious defect detection is performed based on the contour features of the printed contact lenses, wherein the obvious defects include lens edge adhesion, deformation, multiple lenses, and abnormal size; and surface defect detection based on multi-light fields is performed based on multiple characteristic images of the printed contact lenses to obtain detection results, wherein the multiple characteristic images include contour features, point light source images, bright field images, dark field images, and pre-processed images.

[0010] The present application also provides an efficient multi-light field contact lens defect detection device based on cloud-machine vision, comprising:

[0011] An acquisition and preprocessing module, used for acquiring a multi-light field original image of a printed contact lens immersed in water in a glass bowl, and preprocessing the multi-light field original image to obtain a point light source image, a bright field image, a dark field image, and a preprocessed image, wherein the preprocessed image is obtained by filtering the bright field image;

[0012] A feature extraction module, used to detect a glass bowl in the preprocessed image, and extract geometric features of the glass bowl when the glass bowl exists in the preprocessed image; and extract contour features of the printed contact lens in the glass bowl area based on the geometric features of the glass bowl;

[0013] The defect detection module is used to detect obvious defects based on the contour features of the printed contact lenses, wherein the obvious defects include lens edge contact, deformation, multiple lenses, and abnormal size; and to perform surface defect detection based on multiple light fields based on multiple feature images of the printed contact lenses to obtain detection results, wherein the multiple feature images include contour features, point light source images, bright field images, dark field images, and pre-processed images.

[0014] The beneficial effects of the present invention are as follows: an efficient multi-light field contact lens defect detection method and device based on cloud-machine vision of the present invention obtains a multi-light field original image of a printed contact lens immersed in water in a glass bowl, and pre-processes the multi-light field original image to obtain a point light source image, a bright field image, a dark field image and a pre-processed image; performs glass bowl detection on the pre-processed image, and extracts the geometric features of the glass bowl when there is a glass bowl in the pre-processed image; extracts contour features of the printed contact lens in the glass bowl area based on the geometric features of the glass bowl; and performs surface defect detection based on multi-light fields based on multiple feature images of the printed contact lens. The present application pre-processes the multi-light field original image to obtain multiple feature images, thereby performing defect detection on the edge and printing area from multiple aspects. The present application further utilizes the contact lens image captured by a high-resolution camera, automatically identifies the image, and determines whether the corresponding product is normal or not, so as to reduce labor costs, save time, and improve production yield. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0016] Figure 1 This is a topological diagram of the technical solution in this application;

[0017] Figure 2 This is a schematic diagram of the computing algorithm in the cloud service of this application;

[0018] Figure 3 This is a schematic diagram of the parameter configuration process in the implementation of this application;

[0019] Figure 4 is a flow chart of an efficient multi-light field contact lens defect detection method based on cloud-machine vision shown in an embodiment of the present application;

[0020] Figure 5 This is a schematic diagram of the preprocessing process in one embodiment of the present application;

[0021] Figure 6 This is a schematic diagram of the process of glass bowl detection in one embodiment of the present application;

[0022] Figure 7 A schematic diagram of the process of extracting the edge profile of the lens in this application;

[0023] Figure 8 It is a schematic diagram of the process of detecting obvious defects of the lens in this application. DETAILED DESCRIPTION

[0024] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0025] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show the layers related to the present invention rather than being drawn according to the number, shape and size of the layers in actual implementation. In actual implementation, the type, quantity and proportion of each layer may be changed arbitrarily, and the layer layout may also be more complicated.

[0026] In the following description, numerous details are discussed to provide a more thorough explanation of embodiments of the present invention; however, it is apparent to one skilled in the art that embodiments of the present invention may be practiced without these specific details.

[0027] Figure 1 This is the topological diagram of the technical solution in this application, such as Figure 1 As shown, in this application, the printed contact lens product is photographed by an industrial camera to obtain a corresponding grayscale image; the image is transmitted to an industrial computer and forwarded to a cloud server via the industrial computer for defect detection; the product inspection result is obtained and pushed to the industrial computer and the subsequent data processing center.

[0028] Figure 2 This is a schematic diagram of the computing algorithm in the cloud service of this application, such as Figure 2 As shown, the algorithm of this application is mainly composed of 6 modules, including:

[0029] A. Initialization configuration module: responsible for initializing the configuration of the detection system, and reading, loading, color space conversion, noise reduction and other operations on the contact lens images collected by the camera to reduce the interference caused by environmental noise, and finally generate the Process Image;

[0030] B. Glass bowl detection module: responsible for extracting the small bowl ROI and determining the size of the small bowl.

[0031] C. Lens rough inspection module: responsible for using edge extraction algorithm on process image to extract lens edge contour and detect obvious defects of lens, including lens edge adhesion, deformation, multiple lenses, size abnormality, etc.

[0032] D. Lens surface detection module: responsible for two functions. First, ROI division and extraction of each area on the lens surface, and second, lens surface defect detection based on combining each ROI and multi-light field image;

[0033] E. Result analysis and generation module: Integrate the results in modules B to D to obtain the final lens test results.

[0034] F. Output module: Package and output the results in step E.

[0035] Before conducting formal testing, this application needs to load relevant configuration parameters. Figure 3 This is a schematic diagram of the parameter configuration process in the implementation of this application. Figure 3 As shown in the figure, the parameter configuration process includes:

[0036] A1. The initialization configuration process is as follows.

[0037] A11. Import the configuration table file. This parameter configuration file contains lens system parameters, light source configuration parameters, algorithm configuration parameters, system setting parameters, etc. Among them, load the system configuration parameters to set whether to enable the surface detection module and whether to output images; load the algorithm parameters, including extraction parameters and analysis parameters, etc. The details are as follows:

[0038] For module B: small bowl area BlisterArea; effective small bowl radius (BlisterRL, BlisterRU), the former is the lower bound and the latter is the upper bound;

[0039] For module C: lens edge determination threshold T touch , the major and minor axis difference judgment threshold T abdif , lens aspect ratio threshold T edge_asp, lens size diameter setting threshold range LensDiaRange;

[0040] For module D: minimum surface detection size SMinL, minimum bright field surface detection grayscale BSMinG, minimum dark field surface detection grayscale DSMinG, minimum point light source surface detection grayscale PSMinG;

[0041] For module E: defect priority configuration table Priority;

[0042] Read and load the light source configuration parameters and verify them, load the light source parameters used by the system, and compare the parameters with the industrial computer light source system. If the light source settings are inconsistent, return the corresponding error code and exit the system, otherwise enter A14.

[0043] Initialize the image object related data structure used by the algorithm. In this embodiment, it includes the following:

[0044] Input image group, ImagePacket;

[0045] Apply the image group, LensImageGroup;

[0046] Polar coordinate image group, LensPolarImageGroup;

[0047] Defect data group, Set;

[0048] Initialize the relevant result objects used by the algorithm. This embodiment includes the following:

[0049] Step B, glass bowl detection result, BlisterResult;

[0050] Step C, edge extraction result, LensEdgeEctRes;

[0051] Step C, lens rough inspection result, LensRoughRes;

[0052] Step D, lens surface detection result, LensSurfaceInsRes;

[0053] Step E, detecting the final result, LensResult;

[0054] A12, read and load the light source configuration parameters and verify, load the light source parameters used by the system, and compare the parameters with the industrial computer light source system. If the light source settings are inconsistent, return the corresponding error code and exit the system, otherwise enter A14.

[0055] A13. Initialize the image object related data structure used by the algorithm. In this embodiment, the following is included:

[0056] Input image group, ImagePacket;

[0057] Apply the image group, LensImageGroup;

[0058] Polar coordinate image group, LensPolarImageGroup;

[0059] Defect data group, Set;

[0060] A14. Initialize the relevant result objects used by the algorithm. In this embodiment, the following is included:

[0061] Step B, glass bowl test result, BlisterResult

[0062] Step C, edge extraction result, LensEdgeExtRes;

[0063] Step C, lens rough inspection results, LensRoughRes

[0064] Step D, lens surface detection results, LensSurfaceInsRes

[0065] Step E: Detect the final result, LensResult

[0066] Figure 4 is a flow chart of an efficient multi-light field contact lens defect detection method based on cloud-machine vision shown in an embodiment of the present application, such as Figure 4 As shown: An efficient multi-light field contact lens defect detection method based on cloud-machine vision in this embodiment may include the following steps:

[0067] S410, acquiring a multi-light field original image of a printed contact lens immersed in water in a glass bowl, and preprocessing the multi-light field original image to obtain a point light source image, a bright field image, a dark field image, and a preprocessed image, wherein the preprocessed image is obtained by filtering the bright field image;

[0068] Figure 5 Schematic diagram of the preprocessing process in one embodiment of the present application. Figure 5 As shown, the process of preprocessing the multi-light field original image in the present application includes:

[0069] A2, pre-processing:

[0070] A21 reads the image data transmitted by the camera, divides the image data according to the multi-light field settings of bright field, dark field, and point light source, that is, the multi-light field original image, converts the image data from RAW to RGB format, and temporarily stores it in ImagePacket.

[0071] A22, transforms all light field image data in ImagePackett in A21 into grayscale space, obtains grayscale images and stores them in the corresponding array in LensImageGroup, specifically including bright field image BrightImage, dark field image DarkImage, and point light source image.

[0072] A23 performs Gaussian filtering with a kernel of 3×3 and normalization stretching processing on the bright field image BrightImage in the LensImageGroup in A22 to obtain a denoised preprocessed image ProcessImage, the purpose of which is to reduce the interference caused by environmental noise.

[0073] S420, performing glass bowl detection on the preprocessed image, and extracting geometric features of the glass bowl when the glass bowl exists in the preprocessed image; and extracting contour features of the printed contact lens in the glass bowl area based on the geometric features of the glass bowl;

[0074] Figure 6 FIG. 1 is a flow chart of glass bowl detection in an embodiment of the present application. Figure 6 As shown, the glass bowl test includes:

[0075] B. Glass bowl detection module: It is responsible for applying the segmentation algorithm to the preprocessed image ProcessImage and determining whether there is a glass bowl and whether the size is consistent based on the extracted relevant pixel content. Specifically, it includes:

[0076] B1, load the pre-processed image ProcessImage obtained in A23, the effective small bowl radius (BlisterRL, BlisterRU) and the glass bowl area BlisterArea configured in A11;

[0077] B2, binarizing the preprocessed image and extracting the contour features of the glass bowl in the binarized preprocessed image; applying a segmentation algorithm to the preprocessed image ProcessImage in B1 to perform binarization segmentation and extract the corresponding region of interest ROI;

[0078] B3. Fit the minimum inscribed circle of the glass bowl contour whose area is larger than the pre-configured glass bowl area BlisterArea to obtain the radius R and center C of the minimum inscribed circle. First, filter the contours extracted in B2 in turn by area. If it is larger than BlisterArea in B1, fit the minimum inscribed circle to the contour and obtain the radius R and center C of the circle, and then proceed to B4. If none of them are satisfied, end.

[0079] B4. Compare the radius R of the minimum inscribed circle with a preconfigured radius range (BlisterRL, BlisterRU), and when the radius of the minimum inscribed circle falls within the preconfigured radius range, determine that a glass bowl exists in the preprocessed image, and use the radius and center of the minimum inscribed circle as geometric features of the glass bowl.

[0080] Specifically, the radius R obtained in B3 is judged using (BlisterRL, BlisterRU) in B1. If R is not within the range, the result item in the glass bowl detection result BlisterResult is assigned "No Blister"; otherwise, it is considered that this module has passed and BlisterResult is assigned "Pass" and enters B5. The expression is as follows:

[0081]

[0082] B5. Store the radius R and center C obtained in B3 in BlisterResult.

[0083] S430, performing obvious defect detection based on the contour features of the printed contact lens, wherein the obvious defects include lens edge contact, deformation, multiple lenses, and abnormal size; and performing surface defect detection based on multi-light fields based on multiple feature images of the printed contact lens to obtain a detection result, wherein the multiple feature images include contour features, point light source images, bright field images, dark field images, and pre-processed images.

[0084] Finally, detection is performed based on the various features extracted above. The detection content is as follows:

[0085] C. Lens rough inspection, responsible for: extraction of lens edge contour and detection of obvious lens defects, including lens edge adhesion, deformation, multiple lenses, abnormal size, etc., including:

[0086] C1. Contour extraction, Figure 7 The schematic diagram of the process of extracting the edge profile of the lens in this application is as follows: Figure 7 As shown, including:

[0087] C11, filtering and binarizing the preprocessed image to obtain a binary image C1a; specifically, performing two low-pass filtering processes on the preprocessed image ProcessImage generated by A, wherein the first filtering kernel size is 5×5, and the second filtering kernel size is a Gaussian kernel of 3×3, to obtain a filtered image Filtered; then performing an adaptive algorithm on the filtered image Filtered to obtain a binary image C1a;

[0088] C12. Constructing a glass bowl region BlisterROI based on the geometric features of the glass bowl; specifically, constructing a small bowl ROI, BlisterROI, using the small bowl fitting circle information (radius R and center C) stored in BlisterResult in B;

[0089] C13, constructing a first feature image C1fea based on the glass bowl region BlisterROI and the binary image C1a, wherein the mathematical expression of the first feature image C1fea is:

[0090]

[0091] Where, C1fea i,j is the pixel with coordinates (i, j) in the first feature image C1fea, C1a i,j is the pixel with coordinates (i, j) in the binary image C1a, BlisterROI i,j is the pixel with coordinates (i, j) in the glass bowl region BlisterROI;

[0092] C14, extracting the contour features in the first feature image C1fea, and extracting the inner contour inner and the outer contour outer of the printed contact lens based on the pre-constructed lens parameters, wherein the lens parameters include the major and minor axis difference determination threshold T abdif , lens aspect ratio threshold T edge_asp , the lens size diameter sets the threshold interval LensDiaRange; fit the outer contour of the printed contact lens to obtain the ellipse contour ellipse. And save it to the data structure corresponding to LensEdgeExtRes, enter C15; if both the inner and outer contours do not exist, it is considered that there is no lens, and the result item in LensEdgeExtRes is assigned the value "NoLens" and exit;

[0093] C15, verify and integrate the internal and external contours saved in C14, and supplement the relevant data in LensEdgeExtRes, including the effective ROI area BlisterROI, the lens edge fitting ellipse ellipse, the number of detected contours edge_num, the edge feature data C1fea, etc.;

[0094] C2, lens obvious defect detection, Figure 8 It is a schematic diagram of the process of obvious defects of the lens in this application, such as Figure 8 As shown, including:

[0095] C21, using BlisterROI and ellipse integrated by LensEdgeExtRes in C15 to judge the edge of the lens. It includes generating a lens area image based on the ellipse outline ellipse, and extracting the intersection image C21Img of the glass bowl area BlisterROI and the lens area image by "and" operation. When the number of pixels in the intersection image C21Img is greater than a preset number threshold, it is determined that there is a lens edge defect;

[0096] If the number of pixel values ​​at the intersection is greater than the lens edge parameter threshold T in A12 touch , then the lens is considered to be attached to the edge, the result item in LensRoughRes is assigned the value "Touvh" and exit, otherwise the next step, the expression is as follows;

[0097]

[0098] C22. When the printed contact lens only has an outer contour, it is determined that the lens has a large-scale edge defect;

[0099] Use edge_num in C15 to identify large-scale defects in the lens. If edge_num is not 2, that is, only the outer edge exists but not the inner edge, the lens is considered to have large-scale edge defects, and the result item in LensRoughRes is assigned the value "EdgeDefect" and exits. Otherwise, the next step is as follows:

[0100]

[0101] C23, extracting the major axis a and the minor axis b of the ellipse profile ellipse, calculating the difference between the major axis a and the minor axis b, and calculating the aspect ratio of the major axis a and the minor axis b; when the difference is greater than a preset major-minor axis difference threshold, or when the aspect ratio is less than a preset aspect ratio threshold, determining that the lens has a deformation defect;

[0102] The lens deformation is judged by using the major axis a and minor axis b in the ellipse in C15. If the difference between the major axis and the minor axis is greater than the difference between the major axis and the minor axis in A13, the threshold T is set. abdif , the lens is considered deformed; in addition, the aspect ratio of a and b is checked. If the aspect ratio is less than the preset lens aspect ratio threshold T edge_asp , the lens is considered deformed. If the lens is deformed, the result item in lensRoughRes is assigned the value "Distort" and exits, otherwise the next step is as follows:

[0103]

[0104] C24, extracting the ellipse center and assumed radius r of the ellipse outline, wherein the assumed radius Based on the ellipse center and the assumed radius r, the contour features in the first feature image Clfea are expanded in polar coordinates, and the average value and variance of the polar diameters of all contour pixel points in the polar coordinates are calculated, and a filtering range based on the 3sigma principle is constructed based on the average value and variance of the polar diameters, and contour points that do not fall into the filtering range are regarded as abnormal points. When the number of abnormal points is greater than a preset multiple-piece determination number threshold, it is determined that the lens has multiple defects;

[0105] Use C1fea and ellipse in C15 to detect whether the lens is "multiple". For C1fea, use the center EC of the ellipse in ellipse and the assumed radius of the major and minor axes. Polar coordinates are expanded, and the "multi-piece" detection is performed using an outlier detection algorithm based on mathematical statistics, that is, the mean and variance of the polar diameters of all contour pixel points in polar coordinates are used to construct a filtering range based on the 3sigma principle, and contour points that do not fall into the filtering range are regarded as outliers. When the number of outliers is greater than the preset multi-piece judgment threshold, the "multi-piece" feature is met. If the detection meets the "multi-piece" feature, the result item in LensRoughRes is assigned to "MutilLens" and exits, otherwise the next step is the expression as follows:

[0106]

[0107] C25, extract the major axis a and minor axis b of the ellipse, and use the major axis a and minor axis b in ellipse in C15 to determine the lens size. Calculate the mean d=mean(a,b) of the major axis a and the minor axis b, consider d as the lens diameter, and compare the mean d of the major axis a and the minor axis b with the preset lens size diameter setting threshold interval LensDiaRange. When the mean of the major axis a and the minor axis b is not within the preset lens size diameter setting threshold interval LensDiaRange, it is determined that the lens has a size non-compliant defect. If it is not within this size range, it is considered that the size does not match, and the result item in LensRoughRes is assigned to "offsixe" and exit, otherwise the next step, the expression is as follows:

[0108]

[0109] D. Surface defect detection based on multi-light field: It is mainly responsible for two functions. First, it divides the ROI of each area on the lens surface and extracts relevant area parameters. Second, it detects lens surface defects based on combining each ROI and multi-light field image and outputs the corresponding surface detection results. Specifically, it includes:

[0110] D1, input the pre-processed image ProcessImage in A, and judge whether there is a pattern in it. If it is a transparent lens, use the configuration method based on empirical parameters to set and extract the lens ROI, otherwise enter D2;

[0111] D2. Extraction of printed pattern area is as follows:

[0112] D21, downsampling and sharpening the preprocessed image ProcessImage to obtain a downsampled image sub_image; using downsampling to eliminate the interference of the singular pattern, and performing image sharpening on the downsampled image to highlight the pattern area information;

[0113] D22, performing binarization and morphological processing on the downsampled image sub_image to obtain a downsampled contour image sub_th;

[0114] D23, based on the pre-configured empirical area threshold, the contours in the downsampled contour image sub_th are screened to obtain the inner contour and the outer contour of the printing area; sub_th is contour extracted and filtered, and the inner and outer contours of the corresponding printing area are obtained according to the empirical area threshold sub_area. The outer contour outer is used as the end area of ​​the printing area, and the inner contour inner is used as the start area of ​​the printing area;

[0115] D24, fitting the inner contour of the printing area and the outer contour of the printing area respectively to obtain printing area features, wherein the printing area features include the inner ellipse of the printing area print_inner_ellipse, the outer ellipse of the printing area print_outer_ellipse and the printing area surface_1st_region_roi_img;

[0116] D3, central optical zone extraction is as follows:

[0117] D31, multiplying the pre-configured central optical zone ratio optical_pct by the ellipse profile ellipse to obtain a scaled ellipse profile opt_ellipse after major and minor axis compression;

[0118] D32, drawing a first contour image cfg_opt based on the scaled ellipse contour opt_ellipse, and drawing a second contour image print_opt based on the printing area inner ellipse print_inner_ellipse;

[0119] D33, performing an AND operation on the first contour image cfg_opt and the second contour image print_opt to obtain central optical zone features of the real central optical zone, wherein the central optical zone features include the central optical zone fitting ellipse opt_ellipse and the optical zone image of interest optical_region_roi_img;

[0120] D4, the surrounding transparent area is extracted as follows:

[0121] D41, use the ellipse outline in C to draw the image (grayscale 255) to obtain the global image whole_surface_roi_img of the lens;

[0122] D42, using the outer contour of the printed area to draw (with a grayscale of 0) the contour of the transparent area around the lens surface_2nd_region_roi_img in the global image whole_surface_roi_img;

[0123] D43, fitting the outline of the peripheral transparent region of the lens to obtain the peripheral transparent region ellipse surface_2nd_region_outer_ellipse;

[0124] D5. Surface defect detection. Surface defect detection includes defect feature extraction, weak feature filtering, optical area defect detection, printing area defect detection, peripheral transparent area defect detection and surface result integration.

[0125] The specific algorithm flow is as follows:

[0126] D51. Defect feature extraction, including:

[0127] D511, combining the optical zone image of interest and the peripheral transparent area ellipse to generate an optical detection region of interest ROI B ;

[0128] Specifically, the effective detection areas in the bright field image, dark field image and point light source image are generated using the ROI of each area obtained in D2-D4. For example, if it is a "printed color film", the corresponding bright field effective detection area is the optical zone ROI combined with the central optical area ROI and the peripheral transparent area ROI, and the expression is as follows:

[0129] ROI B =(optical)or(surface 2nd )

[0130] D512, the optical detection region of interest ROI BPerforming an AND operation with the preprocessed image ProcessImage to obtain valid image data ImageDate;

[0131] D513, performing adaptive binarization processing on the effective image data to obtain effective image binarization data; extracting contours in the effective image binarization data, and removing contours whose contour areas are smaller than a preset area threshold, to obtain remaining contours; specifically, using an adaptive binarization algorithm to binarize ImageDate in D512, and performing contour extraction and filtering, filtering out weak contours whose contour areas are less than A;

[0132] D514, drawing the remaining contour to an eighth feature image feature;

[0133] D52, central optical area defect detection, including:

[0134] D521, initialization of regional detection results: Initialize the defect storage dictionary Defect and the regional detection result opt_result to NG;

[0135] D522, regional feature acquisition: performing an AND operation on the optical region image of interest optical_region_roi_img and the eighth feature image feature to obtain a first optical region image feature_opt; performing a morphological "closing" operation with a kernel of k on the first optical region image feature_opt to obtain a fourth optical region image feture_opt2, in order to filter subtle features and improve computational efficiency.

[0136] D523, feature filtering: extract the contour features of the fourth optical zone image feature_opt2, and extract the geometric information and grayscale information of the contour features of the fourth optical zone image feature_opt2, wherein the geometric information includes the length, width, aspect ratio, roundness, contour area and density of the circumscribed rotated rectangle of the contour, and the grayscale information includes the maximum grayscale value, minimum grayscale value, grayscale value mean, center grayscale, grayscale variance within the contour and gradient within the contour; based on the pre-configured geometric inspection parameters, grayscale inspection parameters, geometric information and grayscale information of the contour features, the surface defects of the contact lens are detected to obtain the surface defect detection results. If the corresponding defect type is met, the relevant contour data and defect type are retained to Defect, otherwise discarded, and the relevant expressions are as follows:

[0137] Geometry feature filtering:

[0138]

[0139] Grayscale feature filtering:

[0140]

[0141] Defect related:

[0142] Defect={f1,f2,f3,…f i}

[0143]

[0144]

[0145] f i_gray = {gray i ,var i ,min_g i ,max_g i ,gradient i ,center_g i}

[0146] D524, feature filtering: update the regional detection results: make a judgment on Defect. If the dictionary is empty, it is considered that the central optical area has no defects, and opt_rDsult is set to "Pass";

[0147] D53, printing area defect detection is similar to D52, no further description is given;

[0148] D54, peripheral transparent area defect detection is similar to D52, no further description is given;

[0149] D55. Summary of surface test results, including:

[0150] D551. Initialization of surface detection results: Initialize the surface defect storage set SurfaceDefectSet and the surface detection result surface_result to "Pass";

[0151] D552, comprehensively judge the detection results of the above surface areas. If the detection result of any area is NG, surface_result is updated to "NG", otherwise no update is made;

[0152] D553, make a comprehensive judgment on the Defect dictionary of each area detection, if the Defect dictionary of the area is not empty, copy the corresponding data to SurfaceDefectSet;

[0153] D554. Output the final comprehensive surface defect storage set SurfaceDefectSet and the surface detection result surface_result.

[0154] E. Result analysis and generation module: It is mainly responsible for making a comprehensive judgment on the above-mentioned glass bowl inspection results, lens rough inspection results, lens edge inspection and surface inspection results by combining the project defect priority file, and outputting the final result of the lens and the corresponding annotation information, including:

[0155] E1. Initialize the result data in LensResult to "Pass";

[0156] E2. Search BlisterResult in module B, LensEdgeExtRes and LensRough Res in module C. If any of the results in the above three modules is not "Pass", assign the same defect code to LensResult.result, copy the relevant annotation data to the data structure corresponding to LensResult, and enter E5, otherwise enter E3;

[0157] E3. Determine the length of SurfaceDefectSet in module D. If the length is 0, end this module, otherwise enter E4;

[0158] E4. Load the Priority data in A11, build a priority queue, prioritize the defect codes in Set, assign the defect code with the highest priority to LensResult.result, and copy the related annotation data to the data structure corresponding to LensResult.

[0159] E5. According to the corresponding detection results, output the corresponding detection information and annotation information, and end this module.

[0160] F. Result output: Package and output the result in step E.

[0161] F1. Combine the system parameters in A14 to package the output results. If it is necessary to draw an image to mark the location of the defect, use the relevant position coordinates of module F, such as min_rect.

[0162] F2. Output the product detection result Res corresponding to the image analyzed in step F through the relevant API, and present the result to the corresponding front-end display interface and related data processing center, such as physical software, Web, mobile APP, etc.

[0163] An efficient multi-light field contact lens defect detection method based on cloud-machine vision is invented. The method obtains a multi-light field original image of a printed contact lens immersed in water in a glass bowl, and pre-processes the multi-light field original image to obtain a point light source image, a bright field image, a dark field image, and a pre-processed image; performs glass bowl detection on the pre-processed image, and extracts the geometric features of the glass bowl when there is a glass bowl in the pre-processed image; extracts contour features of the printed contact lens in the glass bowl area based on the geometric features of the glass bowl; and performs multi-light field-based surface defect detection based on multiple feature images of the printed contact lens. The present application pre-processes the multi-light field original image to obtain multiple feature images, thereby performing defect detection on the edge and printed area from multiple aspects. The present application further utilizes the contact lens image captured by a high-resolution camera, performs automatic identification on the image, and determines whether the corresponding product is normal or not, so as to reduce labor costs, save time, and improve production yield.

[0164] The present application also provides an efficient multi-light field contact lens defect detection device based on cloud-machine vision, comprising:

[0165] An acquisition and preprocessing module, used for acquiring a multi-light field original image of a printed contact lens immersed in water in a glass bowl, and preprocessing the multi-light field original image to obtain a point light source image, a bright field image, a dark field image, and a preprocessed image, wherein the preprocessed image is obtained by filtering the bright field image;

[0166] A feature extraction module, used to detect a glass bowl in the preprocessed image, and extract geometric features of the glass bowl when the glass bowl exists in the preprocessed image; and extract contour features of the printed contact lens in the glass bowl area based on the geometric features of the glass bowl;

[0167] The defect detection module is used to detect obvious defects based on the contour features of the printed contact lenses, wherein the obvious defects include lens edge contact, deformation, multiple lenses, and abnormal size; and to perform surface defect detection based on multiple light fields based on multiple feature images of the printed contact lenses to obtain detection results, wherein the multiple feature images include contour features, point light source images, bright field images, dark field images, and pre-processed images.

[0168] The present invention discloses an efficient multi-light field contact lens defect detection device based on cloud-machine vision. The device obtains a multi-light field original image of a printed contact lens immersed in water in a glass bowl, and pre-processes the multi-light field original image to obtain a point light source image, a bright field image, a dark field image, and a pre-processed image; performs glass bowl detection on the pre-processed image, and extracts the geometric features of the glass bowl when the glass bowl exists in the pre-processed image; extracts contour features of the printed contact lens in the glass bowl area based on the geometric features of the glass bowl; and performs multi-light field-based surface defect detection based on multiple feature images of the printed contact lens. The present application pre-processes the multi-light field original image to obtain multiple feature images, thereby performing defect detection on the edge and printed area from multiple aspects. The present application further utilizes the contact lens image captured by a high-resolution camera, performs automatic identification on the image, and determines whether the corresponding product is normal or not, so as to reduce labor costs, save time, and improve production yield.

[0169] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0170] The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes any one of the methods in this embodiment.

[0171] The computer-readable storage medium in this embodiment can be understood by ordinary technicians in this field: all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0172] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication with each other. The memory is used to store computer programs, the communication interface is used to communicate, and the processor and the transceiver are used to run computer programs so that the electronic terminal executes each step of the above method.

[0173] In this embodiment, the memory may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0174] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0175] In the above-mentioned embodiments, although the present invention has been described in conjunction with the specific embodiments of the present invention, many replacements, modifications and variations of these embodiments will be apparent to those of ordinary skill in the art based on the foregoing description. The embodiments of the present invention are intended to cover all such replacements, modifications and variations falling within the broad scope of the appended claims.

[0176] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. An efficient multi-light field contact lens defect detection method based on cloud-machine vision, characterized in that: Includes steps: Acquire a multi-light field original image of a printed contact lens immersed in water in a glass bowl, and pre-process the multi-light field original image to obtain a point light source image, a bright field image, a dark field image, and a pre-processed image, wherein the pre-processed image is obtained by filtering the bright field image; Performing glass bowl detection on the preprocessed image, and extracting geometric features of the glass bowl when the glass bowl exists in the preprocessed image; performing contour feature extraction on the printed contact lens in the glass bowl area based on the geometric features of the glass bowl; Obvious defect detection is performed based on the contour features of the printed contact lenses, wherein the obvious defects include lens edge contact, deformation, multiple lenses, and abnormal size; and surface defect detection based on multi-light fields is performed based on multiple characteristic images of the printed contact lenses to obtain detection results, wherein the multiple characteristic images include contour features, point light source images, bright field images, dark field images, and pre-processed images.

2. The method for detecting defects of contact lenses using multi-light fields based on cloud-machine vision according to claim 1, characterized in that: Performing glass bowl detection on the preprocessed image and extracting geometric features of the glass bowl when the glass bowl exists in the preprocessed image includes: Binarizing the preprocessed image and extracting the contour features of the glass bowl in the preprocessed image after the binarization process; Fitting the minimum inscribed circle of the glass bowl contour whose area is larger than the pre-configured glass bowl area to obtain the radius and center of the minimum inscribed circle; The radius of the minimum inscribed circle is compared with a preconfigured radius range, and when the radius of the minimum inscribed circle falls within the preconfigured radius range, it is determined that a glass bowl exists in the preprocessed image, and the radius and center of the minimum inscribed circle are used as geometric features of the glass bowl.

3. The method for detecting defects of contact lenses using multi-light fields based on cloud-machine vision according to claim 1, characterized in that: Based on the geometric features of the glass bowl, contour features of the printed contact lens in the glass bowl area are extracted, including: Filtering and binarizing the preprocessed image to obtain a binarized image C1a; Constructing a glass bowl region BlisterROI based on the geometric features of the glass bowl; A first feature image C1fea is constructed based on the glass bowl region BlisterROI and the binary image C1a, wherein the mathematical expression of the first feature image C1fea is: Where, C1fea i，j is the pixel with coordinates (i, j) in the first feature image C1fea, C1a i，j is the pixel with coordinates (i, j) in the binary image C1a, BlisterROI i，j is the pixel with coordinates (i, j) in the glass bowl region BlisterROI; Extract the contour features in the first feature image Clfea, and extract the inner contour and outer contour of the printed contact lens based on the pre-constructed lens parameters, wherein the lens parameters include the major and minor axis difference determination threshold T abdif , lens aspect ratio threshold T edge_asp , lens size diameter setting threshold range LensDiaRange; The outer contour of the printed contact lens is fitted to obtain an ellipse contour.

4. The method for detecting defects of contact lenses using multi-light fields based on cloud-machine vision according to claim 3, characterized in that: Detect obvious defects based on the contour features of printed contact lenses, including: Generate a lens area image based on the ellipse outline, and extract an intersection image C21Img between the glass bowl area BlisterROI and the lens area image, and determine that there is a lens edge defect when the number of pixel points in the intersection image C21Img is greater than a preset number threshold; When the printed contact lens only has an outer contour, it is determined that the lens has a large-scale edge defect; Extracting the major axis a and the minor axis b of the ellipse profile ellipse, calculating the difference between the major axis a and the minor axis b, and calculating the aspect ratio of the major axis a and the minor axis b; when the difference is greater than a preset major-minor axis difference threshold, or when the aspect ratio is less than a preset aspect ratio threshold, determining that the lens has a deformation defect; Extract the ellipse center and assumed radius r of the ellipse outline ellipse, where the assumed radius Based on the ellipse center and the assumed radius r, the contour features in the first feature image Clfea are expanded in polar coordinates, and the average value and variance of the polar diameters of all contour pixel points in the polar coordinates are calculated, and a filtering range based on the 3sigma principle is constructed based on the average value and variance of the polar diameters, and contour points that do not fall into the filtering range are regarded as abnormal points. When the number of abnormal points is greater than a preset multiple-piece determination number threshold, it is determined that the lens has multiple defects; The major axis a and the minor axis b of the ellipse outline ellipse are extracted, the mean of the major axis a and the minor axis b are calculated, and the mean of the major axis a and the minor axis b are compared with a preset lens size diameter setting threshold interval LensDiaRange. When the mean of the major axis a and the minor axis b is not within the preset lens size diameter setting threshold interval LensDiaRange, it is determined that the lens has a size non-compliance defect.

5. The method for detecting defects of contact lenses using multi-light fields based on cloud-machine vision according to claim 4, characterized in that: Based on the multiple characteristic images of the printed contact lens, surface defect detection based on multi-light fields is performed to obtain detection results, including: Extracting printing area features, central optical zone features and lens peripheral transparent area contours from the preprocessed image; The surface defects of the contact lens are detected based on the printed area features, the central optical zone features and the peripheral transparent area ellipse to obtain a detection result.

6. The method for detecting defects of contact lenses using multi-light fields based on cloud-machine vision according to claim 5, characterized in that: Extracting printing area features from the pre-processed image includes: Downsampling and sharpening the preprocessed image to obtain a downsampled image; Binarizing and morphologically processing the downsampled image to obtain a downsampled contour image; Screening the contours in the downsampled contour image based on a pre-configured empirical area threshold to obtain an inner contour of the printing area and an outer contour of the printing area; The inner contour of the printing area and the outer contour of the printing area are fitted respectively to obtain the printing area characteristics, wherein the printing area characteristics include the inner ellipse of the printing area, the outer ellipse of the printing area and the printing area.

7. The method for detecting defects of contact lenses using multi-light fields based on cloud-machine vision according to claim 6, characterized in that: Extracting the central optical zone features and the peripheral transparent area contour of the lens from the pre-processed image includes: Multiplying the pre-configured central optical zone ratio by the ellipse profile ellipse to obtain a scaled ellipse profile; Drawing a first contour image based on the scaled ellipse contour, and drawing a second contour image based on the ellipse in the printing area; Performing an AND operation on the first contour image and the second contour image to obtain central optical zone features, wherein the central optical zone features include a central optical zone fitting ellipse and an optical zone image of interest; A global image of the lens is drawn based on the ellipse outline, the outer contour of the printed area is used in the global image to draw the outline of the peripheral transparent area of ​​the lens, and the outline of the peripheral transparent area of ​​the lens is fitted to obtain a peripheral transparent area ellipse.

8. The method for detecting defects of contact lenses using multi-light fields based on cloud-machine vision according to claim 7, characterized in that: Detecting surface defects of a contact lens based on the printed area features, the central optical zone features, and the peripheral transparent area ellipse, comprising: generating an optical detection region of interest of a target feature image by combining the optical zone image of interest and the peripheral transparent region ellipse, wherein the target feature image is one of the bright field image, the dark field image and the point light source image; Performing an AND operation on the optical detection region of interest and the preprocessed image to obtain valid image data; Adaptively binarizing the effective image data to obtain effective image binarized data; extracting contours from the effective image binarized data, and removing contours whose contour areas are smaller than a preset area threshold to obtain remaining contours; drawing the remaining contour to an eighth feature image; Performing an AND operation on the optical zone image of interest and the eighth characteristic image to obtain a first optical zone image; performing an AND operation on the printed area and the eighth characteristic image to obtain a second optical zone image; and performing an AND operation on the peripheral transparent area ellipse and the eighth characteristic image to obtain a third optical zone image; Performing a morphological closing operation on the first optical zone image to obtain a fourth optical zone image; performing a morphological closing operation on the second optical zone image to obtain a fifth optical zone image; performing a morphological closing operation on the third optical zone image to obtain a sixth optical zone image; Surface defects of the contact lens are detected based on preconfigured inspection parameters, the fourth optical zone image, the fifth optical zone image, and the sixth optical zone image to obtain a surface defect detection result.

9. The method for detecting defects of contact lenses using multi-light fields based on cloud-machine vision according to claim 8, characterized in that: Detecting surface defects of the contact lens based on pre-configured inspection parameters, the fourth optical zone image, the fifth optical zone image, and the sixth optical zone image to obtain a surface defect detection result, including: Extracting contour features of the fourth optical zone image, the fifth optical zone image and the sixth optical zone image, and extracting geometric information and grayscale information of the contour features of the fourth optical zone image, the fifth optical zone image and the sixth optical zone image, wherein the geometric information includes the length, width, aspect ratio, roundness, contour area and density of a circumscribed rotated rectangle of the contour, and the grayscale information includes the maximum grayscale value, the minimum grayscale value, the mean grayscale value, the central grayscale, the grayscale variance within the contour and the gradient within the contour; The surface defects of the contact lens are detected based on the pre-configured geometric inspection parameters, grayscale inspection parameters, geometric information and grayscale information of the contour features to obtain a surface defect detection result.

10. An efficient multi-light field contact lens defect detection device based on cloud-machine vision, characterized in that: include: An acquisition and preprocessing module, used for acquiring a multi-light field original image of a printed contact lens immersed in water in a glass bowl, and preprocessing the multi-light field original image to obtain a point light source image, a bright field image, a dark field image, and a preprocessed image, wherein the preprocessed image is obtained by filtering the bright field image; A feature extraction module, used to detect a glass bowl in the preprocessed image, and extract geometric features of the glass bowl when the glass bowl exists in the preprocessed image; and extract contour features of the printed contact lens in the glass bowl area based on the geometric features of the glass bowl; The defect detection module is used to detect obvious defects based on the contour features of the printed contact lenses, wherein the obvious defects include lens edge contact, deformation, multiple lenses, and abnormal size; and to perform surface defect detection based on multiple light fields based on multiple feature images of the printed contact lenses to obtain detection results, wherein the multiple feature images include contour features, point light source images, bright field images, dark field images, and pre-processed images.

Images