Computer-implemented method of enhancing ophthalmic lens edge defect detection and other defects

Abstract

The present invention relates to a computer-implemented method for enhancing the detection of edge defects and other defects in ophthalmic lenses. More specifically, the present invention aims at improving the reliability and classification accuracy of edge inspection in contact lenses. The present invention is a computer-implemented method representing a software architecture comprising software components representing core functional modules of an application program and their interdependencies. The system and method of the present invention captures high resolution images, transforms the circular edge of the lens into a horizontal line representing the circular edge, limits the image size by eliminating the pixel information around the edge, divides the horizontal edge image into overlapping sections, and stacks the extracted images vertically to form a single high resolution image that is well suited for processing and analysis by a convolutional neural network after the original image dataset has been augmented with new images generated with a generative adversarial network for enabling accurate classification of defects.

Description

Technical Field

[0001] This invention relates to improving the efficiency of the image classification process in terms of accuracy and speed by extracting specific relevant pixel data and removing irrelevant data to achieve and maintain a high-resolution image of the contact lens. The efficiency improvement is achieved through image rearrangement and reconstruction, which facilitates the rapid training of the inspection system using neural networks and the application of the trained decision model during contact lens inspection in automated systems. Background Technology

[0002] Contact lens defect inspection systems and methods continue to evolve towards improving efficiency while reducing costs. Pattern recognition, image correlation, histogram equalization, and jitter correction are some commonly used image processing algorithms in inspection methods. As defect inspection standards become more stringent, additional algorithms have been implemented to improve inspection efficiency, but at the cost of time. Each additional algorithm increases inspection time, leading to lower productivity. Technological improvements in CPU speed, advanced GPUs (Graphics Processing Units), and high-speed memory have helped improve image processing speed, but the ever-increasing size of images acquired from high-resolution cameras only exacerbates the decline in productivity. Therefore, it is crucial to carefully study software methods for reconfiguring high-resolution images to assist the inspection process in achieving high accuracy and repeatability.

[0003] Neural networks offer a new approach to improving inspection quality without sacrificing the quality and productivity of the inspection system. However, with the significant increase in image size, image compression algorithms are introduced to minimize pixel data, which leads to the degradation of critical defect data. Therefore, inspection quality is affected, and the efficiency of subsequent processes is also compromised. Neural networks are more efficient for medium-sized images. Increased image size negatively impacts the performance of neural networks and deep learning methods.

[0004] Current technology lacks the ability to use software methods without compromising inspection quality, especially when micro-defect identification and feature extraction are fundamental requirements for inspection systems when processing high-resolution images. Summary of the Invention

[0005] This invention proposes a computer-implemented method that combines a high-resolution camera supported by a high-performance CPU and a high-speed memory assisted by a GPU to analyze and process optimized images by identifying and discarding unimportant pixel data and retaining only important regions for defect detection (i.e., edges). The computer-implemented method provides optimized images that are uncompressed but preprocessed to minimize redundant pixel data, and applies smoothing techniques to the uncompressed image to enhance defect candidates, thereby enabling better detection. In the case of identifying defects within a lens, the high-resolution image is divided into several optimal and predetermined-size images, which are more suitable for rapid processing by a deep learning module including several neural network modules for feature extraction and feature classification. It is important to note that even in the case of defect identification within a lens, image compression is not applied when rearranging or extracting images from the original image.

[0006] One advantage of reducing or rearranging the image is the elimination of redundant pixels around the lens edges. Subsequently, by simultaneously dividing the image into predetermined sizes that are easily processed by deep learning modules, enhanced defect detection can be achieved in terms of both speed and accuracy.

[0007] One of the objectives of this invention is to provide high-resolution and optimized pre-processed images of the circular edges of a contact lens without compressing or distorting pixel data.

[0008] Another aspect of the invention is a method for reconstructing and rearranging the circular edge of a contact lens to aid in faster image processing. This is achieved by detecting the edge of the circular contact lens, unfolding the edge, and converting the edge into a horizontal edge. The image is further optimized by removing redundant pixel data around the edge to minimize the image size.

[0009] Arranging pixels as horizontal and vertical edges has one advantage: it improves processing and helps the algorithm process images faster. In other words, the processing speed is significantly increased. This algorithm can be used for edge detection.

[0010] Another advantage of vertically stacking images (image regions) of a predetermined size to form a square image is that the square image has the same length and width. For example, when scanning a square image pixel by pixel, less computation is required compared to scanning a rectangular image. The advantage is the speed of computation, not the number of computations. For example, processing a square image is significantly faster than processing a rectangular image.

[0011] Another object of the present invention is to further rearrange the horizontal edges that are divided into smaller lengths that overlap each other and stack the horizontal edges on top of each other to generate a square image with edge data that is suitable for the needs of the input layer of any neural network and avoids using redundant black pixel data to fill the image to achieve the square size.

[0012] Another object of the present invention is to create a set of analysis modules to help train a computer-implemented method during the configuration of an inspection system for machine learning.

[0013] Another object of this invention is to apply a generative adversarial network algorithm to further transform the segmented image to generate several new, realistic defect features similar to the original defect. The newly generated images are then used to further enhance the training of the computer-implemented method during the configuration of the inspection system for machine learning.

[0014] Other aspects of the invention include various combinations of one or more of the foregoing aspects of the invention, as well as one or more combinations of various embodiments of the invention as found or derived from the following detailed description. It should be understood that the foregoing aspects of the invention also have corresponding computer-implemented methods, which are also aspects of the invention. It should also be understood that those skilled in the art can derive other embodiments of the invention from the following detailed description of specific embodiments of the invention, as well as from the description of the system according to the invention and specific embodiments. Attached Figure Description

[0015] The specific features, aspects, and advantages of the invention will be better understood with reference to the following description, the appended claims, and the accompanying drawings, in which:

[0016] Figure 1 This is a flowchart describing the steps of a training contact lens inspection system for implementing one embodiment of the present invention to identify defects around the edge of a contact lens and their subsequent classification according to different criteria.

[0017] Figure 1a This is a flowchart describing the steps of a training contact lens inspection system for implementing another embodiment of the present invention to identify defects within a contact lens and their subsequent classification according to different criteria.

[0018] Figure 2 This is an illustration of the image at the edge of the contact lens.

[0019] Figure 2a This is a diagram of the image of the contact lens edge after polar coordinate transformation.

[0020] Figure 3 It is stacked in a specific sequence. Figure 2aAn illustration of the image showing the edge extraction region.

[0021] Figure 4 This is an illustration of a high-resolution image of a contact lens suitable for inspecting defects within a lens.

[0022] Figure 5 After identifying the region to be extracted Figure 4 The illustration is shown in the image.

[0023] Figure 6 After extracting the overlapping regions to be used as input for the next process Figure 5 An illustration of the image. Detailed Implementation

[0024] In the following description of preferred embodiments of the invention, reference is made to the accompanying drawings, which form part of the invention, and specific embodiments in which the invention can be practiced are illustrated by way of illustration. It should be understood that other embodiments and structural changes may be utilized without departing from the scope of the invention.

[0025] exist Figure 1 A general flowchart of the system and method according to the present invention is shown. System 10 begins by acquiring a high-resolution image of the contact lens in acquisition process action 12. The acquired image is then processed in extraction process action 14 to extract the circular edge of the lens. The extracted circular edge is then converted into a horizontal image in transformation process action 16. In elimination process action 18, irrelevant pixel data around the horizontal edge is eliminated to minimize the image size. Subsequently, the horizontal edge image is divided into several overlapping images in segmentation process action 20. In stacking process step 22, the segmented image groups are sequentially stacked one on top of the other to form a high-resolution image of the contact lens edge for easy analysis. The reconstructed and rearranged image in stacking process step 22 can then be used as input to a deep learning module for training and analysis. The process flow ends in step 24.

[0026] exist Figure 1aThe diagram illustrates a general flowchart of another embodiment of the system and method according to the invention. System 30 begins by acquiring a high-resolution image of the contact lens in acquisition process step 32. The image is then processed in positioning process step 33 to locate the circular edge of the lens. In drawing process step 34, a square tightly surrounding the circular edge of the contact lens is drawn. In filling process step 35, dark pixels are used to fill pixel data outside a predetermined area of ​​the circular edge of the lens. The predetermined area can be from 16×16 pixels to 128×128 pixels. The contact lens image is then divided into predetermined sizes with overlapping images in programmable partitioning process step 36. The predetermined sizes are equal in size. Alternatively, the predetermined sizes can be different. In deep learning extraction process step 38, labeled regions in the image are extracted and stored separately as input to a deep learning module for training and analysis. The process flow ends in step 40.

[0027] Having described the general system and method according to the invention, the following paragraphs provide details of the above-described process operations.

[0028] Figure 2 yes Figure 1 A graphical representation of the process flow diagram. Figure 2 In the image, lens 50 represents the high-resolution image from the contact lens. Figure 2 In this context, defect 51 represents a tearing defect and is strategically positioned to enable those skilled in the art to understand the concept of the unfolding of the circular edge of the lens, which will be discussed below. Figure 2a As shown, starting from A1 to A2, lens 50 is unfolded or converted into a horizontal image. This is lens 50 and... Figure 2 The area between the inner circle 510 and the black shadow. Figure 2 The outer circle in the middle, like Figure 2a Pixel regions 500, 502, 504, 506, and 508 are horizontally transformed as shown. Pixel regions can be of equal size or have different (but equal) sizes. Figure 2a As shown, Figure 2a Image region segment 500 begins before positions A1 to A2 and ends after position 52, which overlaps with image region segment 502. Figure 2a Image region segment 502 begins before position 52 in image region segment 500 and ends after overlapping to position 53 in image region segment 504. Figure 2a Image region segment 504 begins before position 53 in image region segment 502 and ends after overlapping to position 54 in image region segment 506. Figure 2aImage region segment 506 begins before position 54 in image region segment 504 and ends after overlapping to position 55 in image region segment 508. Figure 2a Image region segment 508 begins before position 55 in image region segment 506 and ends after overlapping to positions A1 to A2 in image region segment 500. One advantage of this method of overlapping to adjacent segments is that it avoids losing or ignoring areas around the edges of the image. It is important to note the defect 51 in… Figure 2a The location shown in the unexpanded image. Figure 3 In the image 58, image region segments 500, 502, 504, 506, and 508 are included, wherein image region segments 500, 502, 504, 506, and 508 contain a large area in the vertical direction due to overlap. For example... Figure 3 As shown, image regions 500, 502, 504, 506, and 508 are stacked sequentially, one on top of the other, to form a square image 58. The number of segments and the width of each segment are automatically calculated to produce a square image during stacking. It is important to note that the square shape of image 58 is determined by subsequent process steps. If the next process step requires a different configured image shape or size, the same requirement will be met during image arrangement. Figure 3 The image in the image is a high-resolution reconstruction of the edge of the contact lens. Figure 3 Images in the image can be used as input images and applied to deep learning modules composed of neural networks, which facilitate rapid training and analysis. Furthermore, it is important to note the tearing defect 51 in... Figure 3 The location shown in the stacked image 58.

[0029] exist Figure 4 The image shown is of the contact lens. Figure 4 yes Figure 1a A graphical representation of the process flow diagram. Figure 4 In the image, 66 represents a high-resolution image of the contact lens positioned within a square image 65. First, the outer edge of the contact lens is detected, and then the outer boundary is drawn. Subsequently, dark pixels are filled between the outer boundary and the outer edge of the lens. The area defined by the circular edge of lens 66 and square 65 is filled with dark pixels because it is not important for inspection. Once the lens edge is identified by the edge detection algorithm, then... Figure 4 The size of is Figure 5 The high-resolution images of the contact lenses X1 and Y1 are divided into multiple overlapping image segments. These images can be of equal or different sizes. Image segmentation can be achieved in several ways. One method is to define and draw the boundaries of the image; once the boundaries are drawn, the size of the entire image can be calculated. For illustrative purposes, Figure 5A high-resolution image of size X1, Y1 is divided into nine equal parts: 600, 602, 604, 606, 608, 610, 612, 614, and 616. Each divided image is preferably a specific size suitable for high-speed processing by the graphics processing unit and deep learning software module. The divided image sizes X' and Y' are predetermined to facilitate better analysis of defect features and improve efficiency before training the neural network. Figure 6 Image 70 shows a typical partitioned image of size X', Y'.

[0030] Some embodiments of the invention have now been described. It will be apparent to those skilled in the art that the foregoing is merely exemplary and not restrictive, and is presented by way of example only. Many modifications and other embodiments are within the scope of those skilled in the art and are considered to fall within the scope of the invention as defined by the appended claims and their equivalents.

Claims

1. A computer implemented method for training an inspection system for contact lens defect inspection, the method comprising: acquiring uncompressed high resolution images of contact lenses; pre-processing the images by applying a smoothing algorithm to enhance defect information without compressing the pixel data; segmenting the pre-processed images and performing size normalization by reconstructing them into square regions best suited for input to a graphics processing unit; applying a deep learning algorithm to the segmented images to extract feature information in each of the segmented images to aid in machine learning and training; classifying the extracted and calibrated feature information into several categories by applying a neural network; reclassifying the extracted features from a new image generated by applying a generative adversarial network to the segmented images; creating a broad database of extracted features detected in the segmented images for micro-defect inspection in uncompressed images; applying a neural network in conjunction with a domain knowledge database to quickly and efficiently inspect new devices.

2. The computer-implemented method of claim 1, further comprising: removing redundant data that is not relevant to the defects being inspected.

3. The computer-implemented method of claim 1, further comprising: removing redundant data and replacing it with dark pixels to enhance the accuracy of the neural network.

4. The computer implemented method of claim 1, further comprising: extracting the circular edges of the contact lens images; horizontally transforming and unrolling the circular edges into horizontal lines; and dividing the horizontal lines into segments with overlapping regions so that no pixel information is lost when two adjacent edges are combined.

5. The computer implemented method of claim 4, further comprising the process of stacking each horizontal line on top of each other to create a normalized square image for further analysis.

6. The computer-implemented method of claim 5, wherein, processing the contact lens images using a deep learning module for inspection, analysis, or training of the contact lens depending on the processing step at any given time.

7. The computer-implemented method of claim 1, further comprising: tracking the edges of the contact lens images and dividing the contact lens images into several overlapping squares optimized for input to a graphics processing unit.

8. The computer-implemented method of claim 7, further comprising: extracting the overlapping squares and filling all redundant pixels outside the edges with black pixels.

9. The computer-implemented method of claim 8, further comprising: analyzing and classifying the extracted features and defects for further enhancing the efficacy of the accumulated domain knowledge database.

Images