Apparatus and method for measuring center deviation of contact lens using artificial intelligence

Abstract

To provide an apparatus and method for measuring a center deviation of a contact lens capable of quickly and accurately detecting an off-center defect of a contact lens, thereby reducing a defect rate and increasing production efficiency.SOLUTION: An apparatus for measuring a center deviation of a contact lens includes a data augmentation unit configured to augment original contact lens image data photographed during a contact lens manufacturing process, an artificial intelligence learning unit configured to use a dataset augmented by the data augmentation unit as an input to conduct learning through an artificial intelligence learning model, and detect a center point of a colored area and a center point of a frame area of the contact lens through learning, and a measuring unit configured to measure the center deviation using the center point of the colored area and the center point of the frame area detected through the artificial intelligence learning model in the artificial intelligence learning unit.SELECTED DRAWING: Figure 13

Description

[Technical Field]

[0001] The present invention relates to a technique for measuring the deviation distance from the center point of a contact lens using artificial intelligence. [Background technology]

[0002] Digital transformation (DX) is bringing about fundamental changes in the structure of modern industry. This revolutionary change is largely due to the rapid development of information technology (IT) and advances in data analysis technology, which are redefining existing work methods and processes in a wide range of fields. Among these, the concept of smart factories plays a particularly important role in the manufacturing industry. Smart factories integrate cutting-edge technologies such as the Internet of Things (IoT), big data, and artificial intelligence (AI) to automate and optimize manufacturing processes, dramatically improving product quality, productivity, and cost efficiency, and revolutionizing traditional manufacturing methods.

[0003] The key to smart factories is data-driven decision-making and process optimization. To achieve this, technologies such as high-level data analysis, predictive modeling, and real-time monitoring are utilized. For example, automated defect inspection systems using image processing technology play an important role in continuously monitoring product quality and quickly sorting out and removing defective products. The development of smart factory technology will strengthen quality control during the manufacturing process, reduce defect rates, and improve the reliability of final products.

[0004] The importance of quality control is also emphasized in the field of contact lens manufacturing, where the products come into direct contact with the eye, and the field of contact lens manufacturing is one that is greatly affected by digital transformation.

[0005] In the manufacturing process of contact lenses, the sandwich method is a method of manufacturing lenses by adding dye between lens layers. The typical defects that occur when manufacturing lenses using this sandwich method are as follows:

[0006] Figure 1 illustrates a normal color contact lens product.

[0007] Figure 2 illustrates a tint defect in a colored contact lens. A tint defect occurs when even one layer is not tinted during multiple tinting processes.

[0008] Figure 3 illustrates a line defect in a colored contact lens, where a line object is created by scratching during the coloring process.

[0009] FIG. 4 illustrates a center-off defect in a colored contact lens.

[0010] 4, the red circle (a) is the colored area (CA) and the blue circle (b) is the frame area (FA). The deviation from the center point refers to the colored area being deviated from the center point.

[0011] Figure 5 shows an example of a dot defect in a color contact lens. A dot defect occurs when part of the dot is missing.

[0012] Figure 6 illustrates cosmetic breakage of colored contact lenses, where the colored area is partially uncolored.

[0013] Figure 7 illustrates an example of an unmolded defect in a colored contact lens. An unmolded defect is a defect in which part of the lens is not molded.

[0014] Figure 8 illustrates a crack defect in a color contact lens, which is a defect in which the inside or periphery of the lens cracks.

[0015] Figure 9 illustrates a foreign body defect in a colored contact lens. A foreign body defect is a defect that occurs when a foreign body enters the lens.

[0016] Figure 10 illustrates an air bubble defect in a colored contact lens. An air bubble defect is a defect caused by air bubbles forming in the lens during molding.

[0017] Figure 11 illustrates dust defects in color contact lenses. Dust defects are defects caused by dust during molding.

[0018] Figure 12 illustrates a printing defect on a colored contact lens, where part of the iris image is not printed.

[0019] Among these defects of contact lenses, center deviation defects go beyond simple pass / fail judgment and require accurate measurement of how far the lens is off the center. This measurement is performed by adjusting the position of the printing equipment to ensure proper printing on the center. Center deviation (CD) is measured using the following formula:

[0020]

number

[0021] where (x CA ,y CA ) indicates the center point of the colored area (CA), and (x FA ,y FA ) indicates the center point of the frame area (FA).

[0022] Since contact lenses are products that come into direct contact with the eyes, even the smallest defects can have a significant impact on the health and comfort of the user's eyes. Therefore, manufacturers must apply strict quality standards and, to meet these standards, must constantly carry out precise inspection processes. Currently, when a defect occurs during the contact lens production process, all defective products produced at the facility within a certain period of time are discarded, which results in increased production costs and reduced quality.

[0023] Conventional techniques for determining the quality of contact lenses have had the following problems:

[0024] First, there is a problem that there is a limit to the judgment of the quality of contact lenses. Conventional methods propose classification models for all types of defects, including good products and center-offset defects, but they cannot measure the distance CD from the center, making it difficult to make fine adjustments to the printing equipment.

[0025] Second, there is a problem of limitations in data augmentation. Conventional data augmentation methods have been traditional methods such as color conversion, position conversion, and image rotation, but these methods cannot overcome the limitations of fixed printing patterns and may cause overfitting, especially for lens types with insufficient data.

[0026] Third, there is a problem of consumption of computing resources: the conventional technique uses both image segmentation and Hough circle detection, which consumes a considerable amount of computing resources.

[0027] Fourth, there is a problem of limited accuracy. In the conventional technology, there is an error of 2.902 pixels between the predicted value and the actual measured value in a 512 x 512 pixel image, and the accuracy needs to be further improved. [Prior art documents] [Patent documents]

[0028] [Patent Document 1] Korean Patent Registration No. 10-2504785 Summary of the Invention [Problem to be solved by the invention]

[0029] The present invention has been created to solve the above-mentioned problems, and its purpose is to provide an apparatus and method for measuring the distance of deviation from the center point of a contact lens using artificial intelligence, which can quickly and accurately detect and measure deviation defects from the center point of a contact lens, thereby reducing the defect rate and increasing production efficiency.

[0030] The objects of the present invention are not limited to those mentioned above, and other objects not mentioned will be apparent to those skilled in the art from the following description. [Means for solving the problem]

[0031] In order to achieve the above-mentioned object, the device for measuring the deviation distance from the center point of a contact lens according to the present invention includes a data augmentation unit for augmenting original contact lens image data captured during the contact lens manufacturing process; an artificial intelligence learning unit that uses the augmented data set in the data augmentation unit as an input value to conduct learning through an artificial intelligence learning model and detects the center points of the colored area and the frame area of ​​the contact lens through learning; and a measurement unit that measures the deviation distance from the center point using the center points of the colored area and the frame area detected in the artificial intelligence learning unit through the artificial intelligence learning model.

[0032] The data augmenter may augment the original contact lens image data using a diffusion model.

[0033] The data augmenter may enhance the original contact lens image data using Denoising Diffusion Probabilistic Models (DDPM).

[0034] The artificial intelligence learning unit may proceed with learning using an object detection model.

[0035] The artificial intelligence learning unit may perform learning using an AC-YOLO (Asymmetric Convolution-You Only Look Once) model that applies an asymmetric convolutional neural network.

[0036] The method for measuring the deviation distance from the center point of a contact lens in the device for measuring the deviation distance from the center point of a contact lens of the present invention includes: a data augmentation step of augmenting original contact lens image data captured during the contact lens manufacturing process; an artificial intelligence learning step of using the augmented data set in the data augmentation step as input values ​​to conduct learning through an artificial intelligence learning model and detecting the center points of the colored region and the frame region of the contact lens through learning; and a measurement step of measuring the deviation distance from the center point using the center points of the colored region and the frame region detected through the artificial intelligence learning model in the artificial intelligence learning step.

[0037] In the data augmentation step, the original contact lens image data can be augmented using a diffusion model.

[0038] In the data augmentation step, the original contact lens image data can be augmented using Denoising Diffusion Probabilistic Models (DDPM).

[0039] In the artificial intelligence learning step, learning can be carried out using an object detection model.

[0040] In the artificial intelligence learning step, learning can be carried out using an AC-YOLO (Asymmetric Convolution-You Only Look Once) model that applies an asymmetric convolutional neural network. [Effects of the Invention]

[0041] According to the present invention, by measuring the deviation distance from the center point of a contact lens using artificial intelligence, it is possible to quickly and accurately detect deviation defects from the center point of a contact lens, thereby reducing the defect rate and improving production efficiency.

[0042] Furthermore, the present invention has the effect of contributing to innovatively improving quality control in contact lens manufacturing, increasing production efficiency, and enhancing the reliability of final products. Furthermore, it is expected that the present invention can be applied to defect detection and quality control in various manufacturing industries beyond the color contact lens industry. [Brief explanation of the drawings]

[0043] [Figure 1] This is an example of a normal color contact lens product. [Figure 2] This illustrates defective coloring of color contact lenses. [Figure 3] This illustrates a line defect in a color contact lens. [Figure 4] 1 illustrates a center-off defect in a colored contact lens. [Figure 5] This illustrates a dot defect on a color contact lens. [Figure 6] This is an example of cosmetic wear of colored contact lenses. [Figure 7] This illustrates an example of a non-molding defect in a color contact lens. [Figure 8] This illustrates a crack defect in a color contact lens. [Figure 9]This illustrates a foreign body defect in a color contact lens. [Figure 10] This illustrates a bubble defect in a color contact lens. [Figure 11] This illustrates a dust defect in colored contact lenses. [Figure 12] This illustrates printing defects on colored contact lenses. [Figure 13] 1 is a block diagram showing the schematic configuration of a device for measuring a deviation distance from the center point of a contact lens according to an embodiment of the present invention. [Figure 14] 1 is a flowchart illustrating a method for measuring a deviation distance from the center point of a contact lens according to an embodiment of the present invention. [Figure 15] 1 conceptually illustrates the overall flow of a method for measuring a deviation distance from the center point of a contact lens according to an embodiment of the present invention. [Figure 16] This shows the diffusion process and de-diffusion process of the denoising diffusion stochastic model (DDPM). [Figure 17] This illustrates the diffusion process in the denoising diffusion stochastic model (DDPM). [Figure 18] This illustrates the learning process through U-Net in the denoising diffusion probabilistic model (DDPM). [Figure 19] 1 shows the overall structure of AC-YOLO proposed in this invention. [Figure 20] (a) The existing YOLO convolution module and (b) the asymmetric convolutional neural network (AC-CNN) proposed in this invention are shown. [Figure 21] This is a comparison of the asymmetric convolutional neural network (AC-CNN) kernel and the standard CNN kernel when passing through the center point of an object. [Figure 22] 1 is a diagram summarizing the experimental environment for learning a denoising diffusion probability model (DDPM) model in an experiment of the present invention. [Figure 23] 1 is a diagram summarizing the experimental environment for YOLO model learning in the experiment of the present invention. [Figure 24] 1 is a diagram summarizing the configuration of the denoising diffusion probability model (DDPM) training data in an experiment of the present invention. [Figure 25] 1 is an image of data augmented through a denoising diffusion probability model (DDPM) in an experiment of the present invention. [Figure 26] 1 shows the data distribution of an enhanced image in an experiment of the present invention. [Figure 27] 1 is a table summarizing the data configuration for AC-YOLO learning in the experiments of the present invention. [Figure 28] 1 is a table summarizing the composition of a dataset augmented by existing methods in an experiment of the present invention. [Figure 29] 10 is a graph showing data loss in training using data augmented by existing augmentation methods and data augmented by a denoising diffusion probability model (DDPM) in an experiment of the present invention. [Figure 30] 10 is a graph showing the lowest data loss results in training using data augmented by existing augmentation methods and data augmented by the denoising diffusion probability model (DDPM) in an experiment of the present invention. [Figure 31] 10 is a table showing the number of weighting values ​​for each comparison model in an experiment of the present invention. [Figure 32] This shows the results when the average loss in the learning process for each model was the lowest in the experiments of the present invention. [Figure 33] 10 shows the difference in center coordinates between the colored area CA and the frame area FA in the actual area (a) and the difference in center coordinates between the colored area CA and the frame area FA in the predicted area (b) in an experiment of the present invention. [Figure 34] This shows the lowest MCPsE (Mean Center Points Error) results for each learning model in the experiments of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0044] Since the present invention can be modified in various ways and can have various embodiments, specific embodiments are illustrated in the drawings and will be described in detail. However, it is not intended to limit the present invention to the specific embodiments, and it should be understood that the present invention includes all modifications, equivalents, and alternatives that fall within the spirit and technical scope of the present invention.

[0045] The terms used in this application are merely used to describe specific embodiments and are not intended to limit the present invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, the terms "comprise" or "have" and the like merely specify the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, and should be understood not to preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0046] Unless otherwise indicated herein or clearly contradicted by context, all terms used in this disclosure, including technical and scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which this invention pertains. In addition, commonly used and dictionary-defined terms should be interpreted as having a meaning consistent with the meaning they have in the context of the relevant art, and should not be interpreted as idealized or overly formal unless expressly defined in this application.

[0047] In addition, in the description based on the accompanying drawings, the same reference numerals are used for the same components regardless of the drawing numerals, and redundant description thereof will be omitted. In describing the present invention, if it is recognized that a detailed description of well-known technologies related to the present invention may obscure the gist of the present invention, the detailed description thereof will be omitted.

[0048] The present invention focuses on detecting and measuring off-center defects, thereby improving the quality of contact lens products and increasing production efficiency.

[0049] First, the present invention aims to go beyond simple pass / fail judgment and measure the deviation distance CD from the center point, thereby enabling adjustment of printing equipment and optimizing the production process.

[0050] In the present invention, a highly accurate standard is set, judging a contact lens to be defective when the deviation from the center point is 0.4 mm or more (approximately 1.9% of the diameter) compared to the overall diameter of 21 mm. This is an essential element for accurate quality control of contact lenses.

[0051] In the present invention, in order to accommodate various lens types, a method is sought that can quickly accommodate various lens types by utilizing limited data.

[0052] The present invention aims to achieve fast and efficient defect detection even with limited computing resources. The system proposed in this invention must be able to process one image within 0.3 seconds, which contributes to real-time inspection and fast decision-making.

[0053] The present invention relates to a device for measuring the deviation distance from the center point of a contact lens, the device comprising: a data augmentation unit for augmenting original contact lens image data captured during the contact lens manufacturing process; an artificial intelligence learning unit that uses the augmented data set in the data augmentation unit as input values ​​to conduct learning through an artificial intelligence learning model and detects the center points of the colored area and the frame area of ​​the contact lens through learning; and a measurement unit that measures the deviation distance from the center point using the center points of the colored area and the frame area detected in the artificial intelligence learning unit through the artificial intelligence learning model.

[0054] The data augmenter can augment the original contact lens image data using a diffusion model.

[0055] The data augmenter may enhance the original contact lens image data using Denoising Diffusion Probabilistic Models (DDPM).

[0056] The artificial intelligence learning unit can perform learning using an object detection model.

[0057] The artificial intelligence learning unit can perform learning using an AC-YOLO (Asymmetric Convolution-You Only Look Once) model that applies an asymmetric convolutional neural network.

[0058] The present invention also relates to a computer-readable recording medium having recorded thereon a program for causing a computer to execute a method for measuring the distance of deviation from the center point of a contact lens, the method including: a data augmentation step of augmenting original contact lens image data captured during the contact lens manufacturing process; an artificial intelligence learning step of using the augmented data set in the data augmentation step as input values ​​to conduct learning through an artificial intelligence learning model and detecting the center points of the colored regions and frame regions of the contact lens through learning; and a measurement step of measuring the distance of deviation from the center point using the center points of the colored regions and frame regions detected through the artificial intelligence learning model in the artificial intelligence learning step.

[0059] The data augmentation step may include using a diffusion model to augment the original contact lens image data. The data augmentation step may include using denoising diffusion probabilistic models (DDPMs) to augment the original contact lens image data.

[0060] In the artificial intelligence learning step, learning can be performed using an object detection model. In the artificial intelligence learning step, learning can be performed using an AC-YOLO (Asymmetric Convolution-You Only Look Once) model that applies an asymmetric convolutional neural network.

[0061] FIG. 13 is a block diagram showing the outline of the configuration of an apparatus for measuring the deviation distance from the center point of a contact lens according to one embodiment of the present invention.

[0062] Referring to FIG. 13, the device for measuring the deviation distance from the center point of a contact lens according to an embodiment of the present invention includes a data augmentation unit 110, an artificial intelligence learning unit 120, and a measurement unit .

[0063] The data augmenter 110 augments the original contact lens data using a diffusion model.

[0064] Since existing enhancement methods have limitations for fixed patterns, the data enhancement unit 110 of the present invention enhances contact lens data using a diffusion model.

[0065] The data augmentation unit 110 of the present invention augments data on a portion of the original contact lens dataset using brightness variation and image rotation image processing, and then finally augments the dataset using a diffusion model, a denoising diffusion probabilistic model (DDPM). The dataset generated through this process contains high diversity and abundant information, which significantly improves the data representation compared to existing methods. In addition, the efficiency of the augmentation method is further demonstrated by presenting an in-depth analysis of the generalization performance during deep learning between the characteristics of the dataset generated through the denoising diffusion probabilistic model (DDPM) and the original dataset. The final augmented dataset and the dataset used to train the denoising diffusion probabilistic model (DDPM) are used as training datasets of the present invention, and the original dataset not used in training is set as a validation dataset.

[0066] The AI ​​learning unit 120 uses the data set augmented by the data augmentation unit 110 as input values ​​to perform learning through an AI learning model, and detects the center points of the colored area and the frame area of ​​the contact lens through the learning.

[0067] In an embodiment of the present invention, the AI ​​learning unit 120 may perform learning using an object detection model. More specifically, the AI ​​learning unit 120 may perform learning using an AC-YOLO (Asymmetric Convolution-You Only Look Once) algorithm that applies an asymmetric convolutional neural network to the object detection model.

[0068] The measurement unit 130 measures the deviation distance from the center point using the center point of the colored area and the center point of the frame area detected by the AI ​​learning unit 120 through the AI ​​learning model.

[0069] FIG. 14 is a flowchart showing a method for measuring the deviation distance from the center point of a contact lens according to one embodiment of the present invention.

[0070] FIG. 15 conceptually shows the overall flow of a method for measuring the deviation distance from the center point of a contact lens according to one embodiment of the present invention.

[0071] 14 and 15, the method for measuring the deviation distance from the center point of a contact lens in the device for measuring the deviation distance from the center point of a contact lens of the present invention includes a data augmentation step (S110) of augmenting original contact lens image data captured during the contact lens manufacturing process, an artificial intelligence learning step (S120) of using the augmented data set in the data augmentation step (S110) as an input value to learn through an artificial intelligence learning model and detecting the center points of the colored area and the frame area of ​​the contact lens through learning, and a measurement step (S130) of measuring the deviation distance from the center point using the center points of the colored area and the frame area detected through the artificial intelligence learning model in the artificial intelligence learning step (S120).

[0072] In one embodiment of the present invention, the data augmentation step (S110) can augment the original contact lens image data using a diffusion model. More specifically, the data augmentation step (S110) can augment the original contact lens image data using denoising diffusion probabilistic models (DDPMs).

[0073] In one embodiment of the present invention, the AI ​​learning step (S120) may use an object detection model for learning. More specifically, the AI ​​learning step (S120) may use an AC-YOLO (Asymmetric Convolution-You Only Look Once) model, which employs an asymmetric convolutional neural network.

[0074] In this invention, we go beyond traditional data enhancement methods, such as brightness change and image rotation, and use a denoising diffusion stochastic model (DDPM), which is a diffusion model, to enhance data that deviates from a fixed pattern.

[0075] The denoising diffusion probabilistic model (DDPM) is a deep learning-based generative model that models the probabilistic process of generating data. One of the features of this model is that it generates data incrementally. For this reason, it can demonstrate superior performance compared to generative adversarial networks (GANs), which generate images themselves in high-dimensional data sections.

[0076] FIG. 16 shows the diffusion process and de-diffusion process of the denoising diffusion stochastic model (DDPM).

[0077] Referring to FIG. 16, the denoising diffusion stochastic model (DDPM) is divided into two steps: a diffusion process and a de-diffusion process.

[0078] FIG. 17 illustrates the diffusion process in the denoising diffusion stochastic model (DDPM).

[0079] As shown in Figure 17, the diffusion process in the denoising diffusion stochastic model (DDPM) gradually adds noise from the original data (χ0) through each time step (t) to the final time step (T = 1000). The diffusion process follows a Markov chain as shown in the following equation.

[0080]

number

[0081] where ε is the noise sampled from a standard normal distribution and β t is a coefficient that determines the noise level.

[0082] The de-diffusion process is a process of restoring from time t to time t-1, and follows the following formula:

[0083]

number

[0084] where ε θ (χ t ,t) is the noise estimated through the trained neural network. The key point of the denoising diffusion probabilistic model (DDPM) is that it uses a neural network to predict the noise ε in the dediffusion process. Through this, it learns a process that gradually approaches the original data χ0.

[0085] Figure 18 illustrates the learning process through U-Net in the denoising diffusion probabilistic model (DDPM).

[0086] As shown in Figure 18, the denoising diffusion probabilistic model (DDPM) can be trained based on U-Net in the dediffusion process.

[0087] In the denoising diffusion probabilistic model (DDPM), a loss function based on the variational lower bound (ELBO) is mainly used. This loss function is calculated by multiplying the original data χ0 of the model by the generated data χ t The loss function L(θ) used in the denoising diffusion probability model (DDPM) has the following formula:

[0088]

number

[0089] In Equation 4, θ is a parameter of the model, ε is a noise sampled from a standard normal distribution, and ε θ (χ t , t) is the noise estimated by the neural network at time step t, and λ t denotes the weighting value for each time step.

[0090] The denoising diffusion probabilistic model (DDPM) differs from traditional data augmentation methods. While traditional methods increase the diversity of a dataset by adjusting image brightness, rotation, etc., the denoising diffusion probabilistic model (DDPM) gradually adds noise to the original data and then works inversely to generate a more refined image from the noise. This allows the deep learning model to augment more complex data patterns, ultimately augmenting data with a variety of patterns.

[0091] In the present invention, the center coordinates of the colored area CA and the frame area FA of the contact lens are detected based on YOLO (You Only Look Once), which is a type of object detection deep learning model, and the deviation distance CD from the center point is measured.

[0092] YOLO is a deep learning-based algorithm for real-time object detection.

[0093] YOLO detects and classifies various objects by viewing the entire image only once. Unlike traditional object detection methods, this model processes the entire image through a single neural network without a separate region proposal step, divides the image into an S*S grid, and predicts a bounding box, corresponding confidence score, and conditional class probabilities for each cell.

[0094] Each bounding box is used to predict the position and size of an object. The position of the bounding box is (x,y) and the size is (w,h), which is predicted by a neural network. The error between the actual box and the predicted box can be expressed as:

[0095]

number

[0096] where 1 ij obj is 1 if the object is in the jth bounding box of the ith cell, and 0 otherwise.

[0097] The confidence score for each bounding box is calculated as the product of the probability that the box contains the object and the Intersection Over Union (IOU), which can be expressed as the following formula:

[0098]

number

[0099] The conditional class probability indicates the probability that a grid cell contains an object of a particular class and is expressed as follows:

[0100]

number

[0101] YOLO also considers class-specific confidence, which is the product of the probability that an object belongs to a particular class, the probability that the object exists, and the IOU of the box, as shown in the following formula:

[0102]

number

[0103] YOLO's loss function takes into account location error, confidence score error, and class prediction error. To minimize this, the model is trained to detect objects of various sizes and types and predict their accurate location and class.

[0104] YOLO has clear advantages over the existing method of applying a Hough circle detection algorithm after semantic segmentation in detecting the center coordinates of the colored area CA and the frame area FA and measuring the deviation distance CD from the center point.

[0105] First, YOLO is a region-based detection method. While semantic segmentation is a method of classifying classes for each pixel, object detection uses anchor boxes to classify specific regions within an image. This is advantageous in grasping the exact position and size of an object, and as a result, it can detect the center coordinates of the object more accurately.

[0106] Second, object instance segmentation. While semantic segmentation is useful for identifying object contours, it does not distinguish between individual object instances. In contrast, object detection can identify each object individually and determine its center coordinates.

[0107] Third, computational complexity: Object detection can be computationally more efficient than semantic segmentation because it targets only objects of interest in the entire image. Semantic segmentation requires processing for every pixel of the image, which requires relatively more computational resources, especially for high-resolution images.

[0108] Fourth, the object detection model is flexible for various object sizes. It can detect objects of various sizes using anchor boxes of various sizes. This helps to detect objects of various sizes more effectively and accurately grasp their center coordinates compared to semantic segmentation.

[0109] Finally, intuitive result analysis. Object detection can intuitively indicate the location of an object through a bounding box, which provides an intuitive and clear way to determine its center coordinates. While semantic segmentation results indicate the exact boundary of an object, determining its center coordinates requires further calculations, such as a Hough circle detection algorithm. In the case of Hough circle detection, which was used in previous research, the accuracy of center coordinate detection decreases if the semantic segmentation results are not predicted as perfect circles.

[0110] In this invention, we propose AC-YOLO (Asymmetric CNN-YOLO), which applies an asymmetric convolutional neural network (CNN) to the YOLO network structure.

[0111] Convolutional neural networks (CNNs) have an outstanding ability to recognize local patterns in images, through which they extract features such as image edges, textures, and colors. Kernel size plays an important role in CNNs. CNNs are composed of various layers, which are essential elements for extracting and understanding image features. Among these, CNN layers use filters (kernels) of various sizes and shapes to grasp image features.

[0112] Generally, the kernels used in convolutional neural networks (CNNs) are often square, such as (3x3) or (5x5), but recently, the use of asymmetric kernels, such as (1x3) or (3x1), has also been attracting attention. Such asymmetric kernels are useful for better capturing specific orientations in an image. For example, a (1x3) kernel is suited to detecting horizontal features in an image, while a (3x1) kernel better captures vertical features.

[0113] These asymmetric kernels are particularly useful when directionality is an important factor in an image, such as detecting road lines or analyzing the vertical structure of a building. They provide broad coverage in one direction while focusing on a narrower area in the other, allowing for better capture of directional details.

[0114] The use of asymmetric kernels improves the computational efficiency of the model and allows for the extraction of more specialized features, which helps to better understand different aspects of the image and make more accurate predictions for specific tasks. Therefore, choosing the size and shape of the kernel is very important when designing a convolutional neural network (CNN).

[0115] FIG. 19 shows the overall structure of AC-YOLO proposed in this invention.

[0116] FIG. 20 shows (a) the existing YOLO convolution module and (b) the asymmetric convolutional neural network (AC-CNN) proposed in this invention.

[0117] Referring to Figures 19 and 20, AC-YOLO proposed in the present invention is trained by replacing the convolutional neural network layers before the C2f layer located in the YOLOv8n backbone with asymmetric convolutional neural network layers.

[0118] Figure 21 compares the asymmetric convolutional neural network (AC-CNN) kernel with the standard convolutional neural network (CNN) kernel when passing through the center point of an object.

[0119] In FIG. 21, (a) shows the AC-CNN kernel, and (b) shows the standard CNN kernel.

[0120] As shown in Figure 21, the reason why AC-CNN is used in the present invention is that when continuous (1x3) and (3x1) kernels pass through the center point of an object, it is determined that features can be extracted more concentrated on the center point than in the (3x3) kernel area. This is based on the principle that the size of the kernel is defined as a probability space, and when passing through the center point of an object, the probability of the center point in the probability space gradually increases as the kernel size becomes smaller.

[0121] In addition, asymmetric convolutional neural networks (CNNs) do not work well in the early layers and are better applied after a certain number of features have been extracted, so they are not applied in the early layers.

[0122] FIG. 22 is a diagram summarizing the experimental environment for learning the denoising diffusion probability model (DDPM) in the experiment of the present invention.

[0123] As shown in Figure 22, the denoising diffusion probabilistic model (DDPM) uses high-spec computing resources capable of processing high-performance parallel graphics processing units (GPUs) due to the large number of model weight values.

[0124] FIG. 23 is a diagram summarizing the experimental environment for YOLO model learning in the experiment of the present invention.

[0125] As shown in Figure 23, the YOLO model generally utilizes random seeds to ensure reproducibility due to the small number of model weight values, and uses the computing resources of a single graphics processing unit (GPU).

[0126] In existing methods, the original dataset is augmented using traditional methods to compensate for the lack of data. However, such augmentation methods may be at risk of overfitting due to fixed patterns. To address this issue, in our experiments, we use a denoising diffusion probability model (DDPM) to augment the data.

[0127] The dataset for training the denoising diffuse probabilistic model (DDPM) was sampled for each lens type to avoid biased enhancement of a specific lens type. The sampling method is as follows:

[0128] Original data set O for lens type i i ,o i It consists of pieces of data, which are expressed as follows:

[0129]

number

[0130] In this set, select K indices y according to a discrete uniform probability distribution. K is the number of i is 10 if is greater than 10, otherwise oi The selected index set is expressed as follows:

[0131]

number

[0132] Selected index {y ik} K K=1 Located in i Sampling data from S i This is expressed as follows:

[0133]

number

[0134] In the present experiment, data o In this case, o3=6, and for the remaining i, o i The total number of extracted sampling data can be calculated as follows, and in this data set, the total is 116 according to the following formula:

[0135]

number

[0136] After this, S i In the above, all lens types i are augmented to 50 by Algorithm 1. In the present experiment, Algorithm 1 is applied to all i to construct a total of 600 training datasets for the denoising diffusion probabilistic model (DDPM). This is calculated using the following formula:

[0137]

number

[0138] FIG. 24 is a diagram summarizing the configuration of the training data for the denoising diffusion probability model (DDPM) in the experiment of the present invention.

[0139] In Fig. 24, the set of data augmented by the denoising diffusion probability model (DDPM) is data D and the image size is 400x400.

[0140] FIG. 25 is an image of data augmented through a denoising diffusion probability model (DDPM) in an experiment of the present invention.

[0141] Referring to FIG. 25, the datasets augmented through the denoising diffusion probabilistic model (DDPM) were generally augmented with similar patterns depending on the lens type, but various results were observed, such as loss of lens illumination, additional generation, deviation from the center point, and slight deformation of the lens pattern.

[0142] FIG. 26 shows the data distribution of the enhanced image in the experiment of the present invention.

[0143] In Figure 26, the image on the left shows the distribution of center points, and the image on the right shows the distribution of heights and widths. In the image on the right, the bottom region shows the distribution of colored areas CA, and the top right region shows the distribution of frame areas FA.

[0144] FIG. 27 is a table summarizing the data structure for learning AC-YOLO in the experiments of the present invention.

[0145] As shown in Figure 27, 1000 data sets were augmented through the denoising diffusion probability model (DDPM). D and sample data data s A total of 1116 image data sets, including 116 of the above, were used as training data for AC-YOLO. T The validation data set data VThe dataset consists of 300 data sets that have not been used in training the denoising diffusion probabilistic model (DDPM), and among these, there are 180 lens types with untrained patterns.

[0146] In the present experiment, the data was augmented through a diffusion model. T and data augmented by existing methods t data V We will verify generalization and experiment to see what datasets can relatively further prevent overfitting.

[0147] FIG. 28 is a table summarizing the composition of the dataset augmented by the existing method in the experiment of the present invention.

[0148] Figure 28 shows the data t The structure of the dataset is illustrated.

[0149] In the present experiment, data augmented by existing augmentation methods t and data augmented by denoising diffusion probability model (DDPM) T Train the DeepLabV3+ model using data V The comparison and evaluation will be carried out in the following manner. The image size will be adjusted to 416 x 416.

[0150] FIG. 29 is a graph showing data loss in training using data augmented by existing augmentation methods and data augmented by the denoising diffusion probability model (DDPM) in an experiment of the present invention.

[0151] Figure 29 shows the data T and data t When we proceed with each of these as training data, the data V (a) shows the loss of the denoising diffusion stochastic model (DDPM) method, and (b) shows the loss of the existing enhancement method.

[0152] As shown in Figure 29, data T data t More than data V The loss of is even lower, and the data V The loss dissipation was slow.

[0153] FIG. 30 is a graph showing the lowest data loss results in training using data augmented by existing augmentation methods and data augmented by the denoising diffusion probability model (DDPM) in the experiments of the present invention.

[0154] Thus, the data augmented through the denoising diffusion probability model (DDPM) T is an existing augmentation method, data t It can be seen that, despite the small amount of data compared to (1), it is effective in preventing overfitting to fixed print patterns. This means that the denoising diffusion probabilistic model (DDPM) enhancement method can improve the diversity of data and enable the model to perform more generalized learning. In the experiments of the present invention, data that has the effect of preventing such overfitting was T The process of learning and evaluating this will be carried out at AC-YOLO.

[0155] In our experiments, we use AC-YOLO to measure the deviation distance CD from the center point and compare its performance with existing methods and various object detection models. In this experiment, the image size of the training and validation datasets was adjusted to 416 x 416, and each YOLO model was trained for 1000 epochs. The number of weights for each model is as follows:

[0156] FIG. 31 is a chart showing the number of weighting values ​​for each comparison model in the experiment of the present invention.

[0157] FIG. 32 shows the results of the experiment of the present invention when the average loss in the learning process for each model was the lowest.

[0158] As shown in Figure 32, the AC-YOLO proposed in this invention has higher box loss and distribution focal loss (DFL) than the existing YOLO. This is because the YOLO loss function is calculated based on IOU, which takes into account both height and width. However, in this invention, height and width are not taken into account, and only the accurate center coordinates of the object are required, so the YOLO loss and the deviation distance CD from the center point are not necessary and sufficient conditions.

[0159] Figure 33 shows (a) the difference in center coordinates between the colored area CA and the frame area FA in the actual area, and (b) the difference in center coordinates between the colored area CA and the frame area FA in the predicted area in an experiment of the present invention.

[0160] Referring to FIG. 33, there may be a large difference between the predicted center coordinates of the colored area CA and the frame area FA and the actual center coordinates of the colored area CA and the frame area FA. However, if the colored area CA and the frame area FA are predicted to be in the same direction and distance, CD x and CDs y can be accurately measured.

[0161] Therefore, in the present invention, not only the deviation distance CD from the center point but also the error in the center coordinates of each colored area CA and the frame area FA are taken into consideration for evaluation.

[0162] The center coordinates of the colored area CA and the frame area FA are (x CA ,y CA ), (x FA ,y FA ), the predicted value is defined as ^(hat), and the number of validation data is defined as ν. The axis error of each center coordinate (Center mean absolute error of axis in area: MAE (axis,area) ) is defined using the following formula:

[0163]

number

[0164] The error in the deviation distance CD from the center point (Mean center deviation error: MCDE) is defined as the average of the Euclidean distance differences as shown in the following formula.

[0165]

number

[0166] In the present invention, the best model is defined as the model with the lowest MCPsE (Mean Center Points Error) during the learning process, and can be expressed by the following formula:

[0167]

number

[0168] FIG. 34 shows the lowest MCPsE (Mean Center Points Error) results for each learning model in the experiments of the present invention.

[0169] As shown in Figure 34, the AC-YOLO proposed in this invention outperforms existing methods in predicting the center coordinates and the offset distance CD from the center point of the colored area CA and the frame area FA. This confirms that measuring the offset distance CD from the center point through object detection is more effective than existing methods. In addition, by applying asymmetric convolution instead of the convolutional neural network (CNN) used in the existing YOLO backbone, we show that more accurate prediction of object center points for the contact lens dataset can be achieved while using fewer weights.

[0170] In this way, in the experiments of this invention, we proposed and verified a new model called AC-YOLO to measure the deviation distance CD from the center point of a contact lens. The experimental results show that AC-YOLO has better performance than the existing YOLO model.

[0171] Furthermore, the data augmentation method through the diffusion model proposed in this invention was effective in preventing overfitting.

[0172] The technology proposed in this invention is expected to play an important role in improving quality control in the contact lens manufacturing process, reducing the defect rate, and increasing production efficiency.

[0173] Meanwhile, the method for measuring the deviation distance from the center point of a contact lens according to the embodiment of the present invention can be realized as computer-readable code on a computer-readable recording medium, which encompasses any kind of recording device that stores data that can be read by a computer system.

[0174] For example, computer-readable recording media include read-only memory (ROM), random access memory (RAM), compact disc read-only memory (CD-ROM), magnetic tape, hard disk (HD), floppy disk, portable storage device, non-volatile memory (Flash memory), and optical data storage device.

[0175] The computer-readable recording medium can also be distributed among computer systems connected through a computer communication network, and the computer-readable code can be stored and executed in a distributed manner.

[0176] Although the present invention has been described above using several preferred embodiments, these embodiments are merely illustrative and not limiting, and it should be understood that those skilled in the art can make various modifications and variations without departing from the spirit of the present invention and the scope of rights disclosed in the appended claims. [Explanation of symbols]

[0177] 100. Apparatus for measuring the distance of deviation from the center point of a contact lens 110 Data Augmentation Department 120 Artificial Intelligence Learning Department 130 Measuring section

Claims

1. a data enhancement unit that enhances original contact lens image data captured during the contact lens manufacturing process; an artificial intelligence learning unit that uses the augmented data set from the data augmentation unit as an input value to perform learning through an artificial intelligence learning model, and detects the center point of the colored area and the center point of the frame area of ​​the contact lens through learning; a measuring unit for measuring a deviation distance from the center point using the center point of the colored area and the center point of the frame area detected through the AI ​​learning model in the AI ​​learning unit; A device for measuring a deviation distance from the center point of a contact lens, comprising:

2. The device for measuring the deviation distance from the center point of a contact lens according to claim 1 , wherein the data augmenting unit augments the original contact lens image data using a diffusion model.

3. 3. The device for measuring a deviation distance from a center point of a contact lens according to claim 2, wherein the data augmentation unit augments the original contact lens image data using denoising diffusion probabilistic models (DDPM).

4. The device for measuring the deviation distance from the center point of a contact lens according to claim 1 , wherein the artificial intelligence learning unit performs learning using an object detection model.

5. 5. The device for measuring the deviation distance from the center point of a contact lens according to claim 4, wherein the artificial intelligence learning unit performs learning using an AC-YOLO (Asymmetric Convolution-You Only Look Once) model to which an asymmetric convolutional neural network is applied.

6. 1. A method for measuring a distance of deviation from the center point of a contact lens using a device for measuring a distance of deviation from the center point of a contact lens, comprising: a data enhancement step of enhancing original contact lens image data captured during the contact lens manufacturing process; an artificial intelligence learning step of using the augmented data set from the data augmentation step as an input value to learn through an artificial intelligence learning model, and detecting the center points of the colored area and the frame area of ​​the contact lens through learning; a measuring step of measuring a deviation distance from the center point using the center point of the colored area and the center point of the frame area detected through the AI ​​learning model in the AI ​​learning step; A method for measuring a deviation distance from the center point of a contact lens, comprising:

7. 7. The method for measuring the deviation distance from the center point of a contact lens according to claim 6, wherein the data augmentation step augments the original contact lens image data using a diffusion model.

8. 8. The method for measuring the deviation distance from the center point of a contact lens according to claim 7, wherein the data enhancement step uses denoising diffusion probabilistic models (DDPM) to enhance the original contact lens image data.

9. 7. The method for measuring the deviation distance from the center point of a contact lens according to claim 6, wherein the artificial intelligence learning step uses an object detection model to proceed with learning.

10. 10. The method for measuring the deviation distance from the center point of a contact lens according to claim 9, wherein the artificial intelligence learning step uses an AC-YOLO (Asymmetric Convolution-You Only Look Once) model to which an asymmetric convolutional neural network is applied to learn.

11. A computer-readable recording medium storing a program for causing a computer to execute a method for measuring a deviation distance from the center point of a contact lens, a data enhancement step of enhancing original contact lens image data captured during the contact lens manufacturing process; an artificial intelligence learning step of using the augmented data set from the data augmentation step as an input value to learn through an artificial intelligence learning model, and detecting the center points of the colored area and the frame area of ​​the contact lens through learning; a measuring step of measuring a deviation distance from the center point using the center point of the colored area and the center point of the frame area detected through the AI ​​learning model in the AI ​​learning step; A computer-readable recording medium having recorded thereon a program for causing a computer to execute a method for measuring a deviation distance from the center point of a contact lens, comprising:

12. 12. A computer-readable recording medium having a program recorded thereon that can cause a computer to execute the method for measuring the deviation distance from the center point of a contact lens described in claim 11, characterized in that in the data augmentation step, the original contact lens image data is augmented using a diffusion model.

13. 13. A computer-readable recording medium having a program recorded thereon that can cause a computer to execute the method for measuring the deviation distance from the center point of a contact lens according to claim 12, wherein the data enhancement step enhances the original contact lens image data using denoising diffusion probabilistic models (DDPM).

14. A computer-readable recording medium having a program recorded thereon that can cause a computer to execute the method for measuring the deviation distance from the center point of a contact lens described in claim 11, characterized in that in the artificial intelligence learning step, learning is carried out using an object detection model.

15. 15. A computer-readable recording medium having recorded thereon a program capable of causing a computer to execute the method for measuring the deviation distance from the center point of a contact lens according to claim 14, wherein the artificial intelligence learning step proceeds with learning using an AC-YOLO (Asymmetric Convolution-You Only Look Once) model to which an asymmetric convolutional neural network is applied.

Images