System and method for analysis and inspecting image employing AI deep-learning of YOLO Resnet

Abstract

This invention relates to an image analysis inspection method based on YOLO Resnet AI deep learning. It performs customized vision inspection for each process of precision machined parts and provides an inspection processor and an AI solution for each vision inspection. In particular, it enables customized determination of good products based on geometric feature information corresponding to the precision machined parts through machine vision based on AI algorithms. This facilitates the easy acquisition of manufacturing data for the AI ​​solution and allows for the utilization of the advantage of the AI ​​solution's significant effectiveness in a mass production system. Furthermore, in this vision inspection processor, the K-means method is applied to identify the geometric information of the precision machined part, thereby increasing the accuracy of good product determination using geometric information that represents valid values. Additionally, to prevent bias associated with the K-means method, the present invention applies (ROI) image preprocessing and (maximum area) contour extraction to the geometric information of the precision machined part, thereby effectively performing computer vision-based defect detection.

Description

Technology Field

[0001] The contents disclosed in this specification relate to a vision inspection method and a system for precision machined parts, such as aerospace components. More specifically, the invention relates to a method for custom-analyzing and detecting the causes of defects for each process using AI technology in such vision inspection. Background Technology

[0002] Unless otherwise indicated in this specification, the contents described in this section are not prior art for the claims of this application, and are not to be recognized as prior art simply because they are included in this section.

[0003] Generally, the main production process for precision machined parts, such as automotive parts, is carried out as follows: raw material receiving -> processing -> post-processing -> product inspection -> shipment.

[0004] The present invention aims to perform customized vision inspection for each of these processes and to develop and widely disseminate AI solutions for each process.

[0005] To this end, video monitoring devices are customized for each of the multiple processes of the precision machined product.

[0006] For example, the first vision inspection_dimensions use an area image monitoring device, the second vision inspection_2D appearance uses a line scan image monitoring device, the third vision inspection_3D appearance uses a 3D scan image monitoring device, and the fourth vision inspection_characteristics uses a multispectral image monitoring device.

[0007] Furthermore, the possession of multiple CNC machining facilities facilitates the acquisition of manufacturing data for the development of AI solutions for these precision machined parts. In other words, the company generally owns a number of CNC machining facilities (MCTs, CNC lathes), which makes it easy to secure and analyze the big data necessary for AI solution development.

[0008] In addition, the above-mentioned precision machined products are mass-produced, and the utilization of AI-based vision systems is highly effective. For example, a specific company has a production system of about 300 units per month for Cover HUBs, which corresponds to an annual production volume of about 3,600 units.

[0009] Due to the mass production method, there is a constant possibility of mass defects occurring depending on processing conditions.

[0010] Currently, due to the implementation of visual inspections by workers, there is a constant risk of customer claims resulting from the deployment of a large workforce and human errors caused by the workers' condition.

[0011] Therefore, it can be expected that the utilization effect will be significant when applying the AI ​​solution according to the present invention.

[0012] Looking at the prior art with this background, it includes the following literature. Prior art literature

[0013] Document 1 European Registration No. 04102404, SYSTEM AND METHOD FOR DYNAMICALLY GENERATING COMPOSABLE WORKFLOW FOR MACHINE VISION APPLICATION-BASED ENVIRONMENTS

[0014] Reference 2 European Register No. 0185558, Process and device to assist the positioning of work pieces by superpositioning images

[0015] Reference 3 European Register No. 03482347, Video-Based Data Collection, Image Capture and Analysis Confirmation

[0016] Reference 3 European Register No. 00243253, Method for automatically recognizing objects susceptible to overlapping one another The problem to be solved

[0017] The disclosed invention provides a vision inspection customized for each process of precision machined parts and offers an inspection processor and an AI solution for each vision inspection. In particular, it enables the customized determination of good parts based on geometric feature information corresponding to the machined parts using AI algorithm-based machine vision for such precision machined parts. Through this, the invention aims to provide a YOLO Resnet AI deep learning-based image analysis inspection system and method that facilitates the acquisition of manufacturing data for the AI ​​solution and offers the advantage of significant effectiveness of the AI ​​solution in a mass production system.

[0018] In addition, in this vision inspection processor, the K-means technique is applied to identify the geometric information of the precision machined part, thereby enabling the accuracy of good product judgment to be improved using the geometric information of the precision machined part that represents valid values.

[0019] In addition, to prevent bias associated with such K-means techniques, the present invention applies (ROI) image preprocessing and (maximum area) contour extraction to the geometric information of a precision machined part, thereby effectively performing computer vision-based defect detection. means of solving the problem

[0020] The YOLO Resnet AI deep learning-based image analysis inspection system and method according to the present invention is,

[0021] In a YOLO Resnet AI deep learning-based image analysis inspection method for precision machined parts of a management information processing device,

[0022] A first step of custom-configuring video surveillance devices for each of the multiple processes of the above-mentioned precision machined product to collect multiple pieces of video surveillance information per corresponding work unit of the precision machined product;

[0023] In the first step above, the custom setting of the video surveillance device is,

[0024] 1) 1st Vision Inspection_Dimensions: Area Video Monitoring Device;

[0025] 2) Second Vision Inspection_2D Appearance: Line Scan Video Monitoring Device;

[0026] 3) Third Vision Inspection_3D Appearance: 3D Scan Video Monitoring Device;

[0027] 4) 4th Vision Surveillance_Characteristics: Multispectral Video Surveillance Device;

[0028] A second step of collecting multiple pieces of geometric information within a set ROI area of ​​the precision machined product using the video surveillance information from the first step above;

[0029] Step 3-1, which involves extracting a plurality of random K points from the geometric information of Step 2 above using the K-means technique and setting them as center points;

[0030] Step 3-2 for calculating the difference in distance between the center point of Step 3-1 and the remaining points;

[0031] Step 3-3, which performs clustering by assigning the remaining points to the nearest center point based on the distance calculated in Step 3-2 above;

[0032] Step 3-4, which updates the point corresponding to the center in the cluster of Step 3-3 above as the center point;

[0033] Step 3-5, which repeatedly performs steps 3-2 through 3-4 until the updated center point of step 3-4 does not change;

[0034] Step 3-6, which sets the center point obtained through Step 3-5 above as a valid value and specifies the geometric information of the precision machined product using only the center point;

[0035] Step 4, approximating the geometric information of the precision machined product of Steps 3-6 above with a set polygon;

[0036] A fifth step of calculating the number of corners, filling ratio, convexity, and perimeter information as first geometric feature information from the polygon approximation information of the fourth step above;

[0037] Step 6, which compares the information of Step 5 above with each set reference value;

[0038] Step 7, which determines the precision machined product to be in a shape-deformed state if, as a result of the comparison in Step 6 above, the information in Step 5 above differs from the corresponding reference value by one or more set values;

[0039] Step 8, which calculates information on the presence or absence of holes, corner angles, and edge continuity as second geometric feature information from the polygon approximation information of Step 4 above;

[0040] A ninth step of comparing the information of the eighth step above with each set reference value; and

[0041] It includes a 10th step of determining the precision machined product as having a shape defect if, as a result of the comparison in the 9th step above, the information in the 8th step above differs from the corresponding reference value by one or more set values.

[0042] And, the above second step is,

[0043] Step 2-1, which extracts the set ROI area of ​​the precision machined product from the video surveillance information of Step 1 above, converts the ROI image to grayscale, and binarizes it using a set Gaussian blur to separate the object and the background;

[0044] Step 2-2, which detects the outline for the object of Step 2-1 above and specifies the main outline as the maximum area contour;

[0045] Step 2-3, which specifies and extracts geometric information of the corresponding precision part using the main contour of Step 2-2 above;

[0046] Step 2-4-1, which extracts color, texture, and edge from the geometric information of Step 2-3 above and converts them into a high-dimensional feature vector based on the vector conversion information of the setting first reference value_color, texture, and edge of the precision part; and

[0047] It includes a 2-4-2 step of extracting pattern information from the high-dimensional feature vector of the above 2-4-1 step and normalizing it based on the set 2nd reference value_pattern quantification information;

[0048] In the above 2-4-2 step, the normalization is,

[0049] The feature scale difference for the above pattern quantification information is characterized by normalizing the mean to 0 and the standard deviation to 1.

[0050] And, the above 6th step is,

[0051] Step 6-1, which involves collecting and accumulating geometric information within a set ROI area for each precision machined part and first geometric feature information for each geometric information from the corresponding video surveillance device to secure a dataset;

[0052] Step 6-2, which detects reference value information representing cluster characteristics from the dataset of Step 6-1 above;

[0053] Step 6-3, which normalizes the reference value information of Step 6-2 above;

[0054] Step 6-4, which sets independent and dependent variables for calculating first geometric feature information for each geometric information using the normalization information of Step 6-3 above;

[0055] Step 6-5, which creates training data and performs training using the independent and dependent variables of the above Step 6-4; and

[0056] A 6-6 step for calculating the setting reference value information of the 6th step as a learning result of the 6-5 step; or including,

[0057] The above 9th step is,

[0058] Step 9-1, which involves collecting and accumulating geometric information within a set ROI area for each precision machined part and second geometric feature information for each geometric information from the corresponding video surveillance device to secure a dataset;

[0059] Step 9-2, which detects reference value information representing cluster characteristics from the dataset of Step 9-1 above;

[0060] Step 9-3, which normalizes the reference value information of Step 9-2 above;

[0061] Step 9-4, which sets independent and dependent variables for calculating second geometric feature information for each geometric information using the normalization information of Step 9-3 above;

[0062] Step 9-5, which creates training data and performs training using the independent and dependent variables of Step 9-4 above; and

[0063] It is characterized by including a 9-6 step of calculating the setting reference value information of the 9th step as a learning result of the 9-5 step above. Effects of the invention

[0064] According to the present invention, vision inspection is customized for each process of a precision machined product, and an inspection processor and an AI solution for each vision inspection are provided. In particular, machine vision based on an AI algorithm for such precision machined products enables the customized determination of good products using appropriate geometric feature information. Through this, it facilitates the acquisition of manufacturing data for the AI ​​solution and enables the utilization of the advantage of the AI ​​solution's significant effectiveness in a mass production system.

[0065] Furthermore, in this vision inspection processor, the K-means technique is applied to identify the geometric information of the precision machined part, thereby increasing the accuracy of good product judgment based on the geometric information of the precision machined part representing valid values.

[0066] In addition, to prevent bias associated with such K-means techniques, the present invention applies (ROI) image preprocessing and (maximum area) contour extraction to the geometric information of a precision machined part, thereby effectively performing computer vision-based defect detection. Brief explanation of the drawing

[0067] FIG. 1 is a diagram conceptually illustrating a YOLO Resnet AI deep learning-based image analysis inspection system and method according to the present invention. FIG. 2 is a drawing illustrating the system of FIG. 1 in its entirety. Figure 3 is the system configuration diagram of Figure 1. Figure 4 is a flowchart illustrating the method of Figure 1 in sequence. Specific details for implementing the invention

[0068] FIG. 1 is a diagram conceptually illustrating a YOLO Resnet AI deep learning-based image analysis inspection system and method according to the present invention.

[0069] As illustrated in FIG. 1, the system according to the present invention performs customized vision inspection for each process of a precision machined product and provides an inspection processor and an AI solution for each vision inspection. In particular, machine vision based on an AI algorithm for such precision machined products determines whether the product is good based on the corresponding geometric feature information.

[0070] For reference, such precision machined parts may be, for example, aircraft components such as cover hubs.

[0071] To this end, the system according to the present invention includes a management information processing device (100), and the above-described method is applied thereto.

[0072] As described above, the precision machined product according to the present invention, for example, an automobile part, has the following stages as a major production process form: raw material receiving -> processing -> post-processing -> product inspection -> shipment.

[0073] The present invention provides customized vision inspection for each of these processes in the management information processing device (100), and provides a vision inspection processor and an AI solution for each process.

[0074] To this end, the system according to the present invention customizes an image monitoring device, a vision inspection processor, and an AI solution for each of the multiple processes for the precision machined product.

[0075] For example, the first vision inspection_dimensions uses an area image monitoring device, the second vision inspection_2D appearance uses a line scan image monitoring device, the third vision inspection_3D appearance uses a 3D scan image monitoring device, and the fourth vision inspection_characteristics uses a multispectral image monitoring device. And, a vision inspection processor is provided for each.

[0076] The above first vision inspection_dimension uses an area camera to derive an FOV optimized for inspection. It is suitable for a Cover HUB as a precision machined part, and image capture and inspection processing are possible therefor.

[0077] For reference, the above second vision inspection_2D appearance uses a line scan camera. While an area camera captures images in planar (horizontal × vertical) units, a line scan camera captures images in line units. By combining and visualizing each line, the appearance can be accurately represented.

[0078] The above third vision inspection_3D exterior uses a 3D scan camera. 3D scanning is a technology that precisely acquires shape information of precision machined parts and generates three-dimensional data including depth information. Examples include the structured light method and the laser triangulation method. The structured light method projects a patterned light source onto a precision machined part and extracts depth information by analyzing the deformed pattern with a camera. Its advantages include high resolution and precision, while its disadvantage is the possibility of performance degradation in bright environments.

[0079] The above-mentioned fourth vision inspection characteristic uses a multispectral camera. This analyzes the detailed characteristics of precision machined parts by using light of various wavelengths, such as infrared, in addition to visible light. It contains more information than a standard RGB image. A filter-based multispectral camera uses a specific wavelength filter and can acquire images of multiple wavelengths. An advantage is that various wavelengths can be selected, while a disadvantage is the possibility of speed degradation due to filter changes.

[0080] In the system according to the present invention, the vision inspection processor of the management information processing device (100) that can be commonly used in each of these processes has the following configuration.

[0081] 1) Multiple video surveillance information of the precision machined product is collected from the corresponding video surveillance device.

[0082] 2) Using this video surveillance information, multiple pieces of geometric information of the precision machined product are collected within a specific ROI area of ​​the precision machined product.

[0083] To elaborate, the ROI image is converted to grayscale, Gaussian Blur is applied, and then preprocessed with Otsu binarization to remove noise and separate the object and background.

[0084] Afterward, all outlines are detected and their areas are calculated, and the contour with the maximum area is selected, for example, as the main contour of the Cover HUB component.

[0085] This rule-based algorithm is computationally lightweight and highly interpretable, making it suitable for automatic identification of defect candidates prior to deep learning.

[0086] Through these main contours, the geometric information of the precision machined part can be identified and extracted.

[0087] In particular, to prevent bias caused by the K-means technique in the next step, the present invention applies (ROI) image preprocessing and (maximum area) contour extraction to the geometric information of a precision machined part, thereby effectively performing computer vision-based defect detection.

[0088] 3) Next, the geometric information of the precision machined part is determined using the K-means technique. This involves extracting only valid values ​​from point cloud information representing the shape of the precision machined part and using them to determine its geometric information.

[0089] 4) In this way, specific geometric information is approximated by the corresponding polygon. The polygon can be determined differently depending on the shape of the precision machined part.

[0090] 5) From the above polygon approximation information, the number of corners (e.g., whether there are 4), the filling ratio (e.g., based on 0.7), the convexity (e.g., based on 0.85), and the perimeter area regularity (e.g., proximity to 1.27) are calculated as the first geometric feature information and compared with each reference value. Thus, if there is a difference between the two values ​​as a result of the comparison, it is determined to be in a shape deformation state. Through this, it is possible to determine whether the shape of the precision machined part is deformed.

[0091] The above reference value is derived using an AI algorithm according to the present invention, and specific details will be described later.

[0092] 6) From the polygon approximation information above, the presence or absence of (internal) holes (based on RETR_TREE), corner angles (e.g., 90° deviation), and edge continuity (based on Canny) are calculated as second geometric feature information and compared with their respective reference values. Therefore, if there is a difference between the two values ​​as a result of the comparison, it is determined to be a shape defect state. Through this, shape defects, such as defects, distortions, and cracks, can be detected in the precision machined part. In such cases, different weighted scores can be assigned to each criterion. High weights are applied to key indicators such as internal holes and filling ratios.

[0093] The present invention can effectively determine whether a precision machined product is a good product by using information on these seven representative geometric features. That is, it provides defect judgment criteria based on seven geometric features.

[0094] The entire data is processed rapidly using multiprocessing, and boundary cases are marked separately for use in subsequent labeling and quality control decisions.

[0096] Figure 2 is a diagram illustrating the system of Figure 1 in its entirety.

[0097] As illustrated in FIG. 2, the system according to the present invention is largely installed at a plurality of different manufacturing locations. This refers to multiple manufacturing locations of a specific company. To this end, the system has a customized video surveillance device for each process and a management information processing device (100) for vision inspection therefrom.

[0098] The above-mentioned management information processing device (100) performs vision inspections tailored to each process of a precision machined product and provides an inspection processor and an AI solution for each vision inspection. In particular, it uses machine vision based on an AI algorithm for such precision machined products to determine whether the product is good based on geometric feature information that fits the product.

[0099] Each device can be connected to a private or leased network, and can use various existing wired and wireless communication networks.

[0100] Figure 3 is a block diagram of the system of Figure 1.

[0101] As illustrated in FIG. 3, in the system according to the present invention, the management information processing device (100) mainly comprises a key signal input unit (101), a storage unit (102), an interface unit (103), a signal processing unit (104), a control unit (105), a display unit (106), and a voice output unit (107).

[0102] The above key signal input unit (101) receives user setting information according to user key operation, and includes registration information for each process image monitoring device, vision inspection processing information, and defect judgment criteria information for seven geometric feature information according to the present invention.

[0103] The above storage unit (102) registers setting information of the key signal input unit (101), video surveillance information of the video surveillance device, etc.

[0104] The above interface unit (103) transmits and receives various information by connecting to the above video surveillance device or sensor, etc.

[0105] The above signal processing unit (104) processes and converts information transmitted and received with the interface unit (103) according to a predetermined format, and includes signal transmission according to the transmission signal format, noise removal, filtering, digital signal conversion, and processing.

[0106] The above control unit (105) controls each of the above units to perform a defect detection operation according to the present invention, specifically as shown in FIG. 4.

[0107] The above display unit (106) displays the defect detection result according to the present invention on the screen under the control of the above control unit (105).

[0108] The voice output unit (107) outputs a defect detection result according to the present invention by voice under the control of the control unit (105).

[0109] Figure 4 is a flowchart illustrating the method of Figure 1 in sequence.

[0110] As illustrated in FIG. 4, the method according to the present invention first performs customized vision inspection for each process of a precision machined product and provides an inspection processor and an AI solution for each vision inspection. In particular, machine vision based on an AI algorithm for such precision machined products determines whether the product is good based on the corresponding geometric feature information.

[0111] This facilitates the easy acquisition of manufacturing data for AI solutions and enables the leverage of the significant benefits of AI solutions within a mass production system.

[0112] Furthermore, in this vision inspection processor, the K-means technique is applied to identify the geometric information of the precision machined part, thereby increasing the accuracy of good product judgment based on the geometric information of the precision machined part representing valid values.

[0113] For reference, the aforementioned K-means technique is, simply put, an algorithm that groups data into K clusters. A cluster is a group of data with similar characteristics. Similarly, clustering means grouping data into clusters.

[0114] In the K-means algorithm, K represents the number of clusters to be grouped, and means represents the average. It means that data are grouped into K clusters by utilizing the mean of each cluster. Here, the mean refers to the average distance between the center of each cluster and the data points.

[0115] Specifically, the above-mentioned management information processing device (100) first customizes the image monitoring device for each of the multiple processes for the precision machined product and collects multiple image monitoring information per corresponding work unit of the precision machined product (Step 1).

[0116] In the first step above, the custom settings of the video surveillance device are as follows.

[0117] 1) 1st Vision Inspection_Dimensions: Area Video Monitoring Device;

[0118] 2) Second Vision Inspection_2D Appearance: Line Scan Video Monitoring Device;

[0119] 3) Third Vision Inspection_3D Appearance: 3D Scan Video Monitoring Device;

[0120] 4) 4th Vision Surveillance_Characteristics: Multispectral Video Surveillance Device;

[0121] Multiple geometric information within the set ROI area of ​​the precision machined product is collected using the video surveillance information from the first step (second step).

[0122] To elaborate, the ROI image is converted to grayscale, Gaussian Blur is applied, and then preprocessed with Otsu binarization to remove noise and separate the object and background.

[0123] Afterward, all outlines are detected and their areas are calculated, and the contour with the maximum area is selected, for example, as the main contour of the Cover HUB component.

[0124] This rule-based algorithm is computationally lightweight and highly interpretable, making it suitable for automatic identification of defect candidates prior to deep learning.

[0125] Through these main contours, the geometric information of the precision machined part can be identified and extracted.

[0126] In particular, to prevent bias caused by the K-means technique in the next step, the present invention applies (ROI) image preprocessing and (maximum area) contour extraction to the geometric information of a precision machined part, thereby effectively performing computer vision-based defect detection.

[0127] Next, the geometric information of the precision machined part is determined using the K-means technique. This involves extracting only valid values ​​from the point cloud information representing the shape of the precision machined part and using them to determine its geometric information.

[0128] To do this, first, among the geometric information of the above Step 2, a plurality of random K points are extracted using the K-means technique and set as center points (Step 3-1).

[0129] Calculate the difference in distance between the center point of Step 3-1 and the remaining points (Step 3-2).

[0130] The remaining points are clustered by assigning them to the nearest center point using the distance calculated in Step 3-2 above (Step 3-3).

[0131] The point corresponding to the center in the cluster of Step 3-3 above is updated as the center point (Step 3-4).

[0132] Steps 3-2 through 3-4 are repeated until the updated center point of Step 3-4 does not change (Step 3-5).

[0133] The center point obtained through the above 3-5 steps is set as a valid value, and the geometric information of the precision machined part is specified using only the center point (3-6 steps).

[0134] The geometric information of the precision machined product from steps 3-6 above is approximated by a set polygon (step 4).

[0135] In the polygon approximation information of the above 4th step, the number of corners, filling ratio, convexity, and perimeter information are calculated as the first geometric feature information (5th step).

[0136] The information from the above Step 5 is compared with each setting reference value (Step 6).

[0137] As a result of the comparison in Step 6 above, if the information in Step 5 above differs from the corresponding reference value by one or more set values, the precision machined product is determined to be in a shape-deformed state (Step 7).

[0138] For example, in the above polygon approximation information, the number of corners (e.g., whether there are 4), the filling ratio (e.g., based on 0.7), the convexity (e.g., based on 0.85), and the perimeter area regularity (e.g., proximity to 1.27) are calculated as first geometric feature information and compared with each reference value. Thus, if there is a difference between the two values ​​as a result of the comparison, it is determined to be in a shape deformation state. Through this, it is possible to determine whether the shape of the precision machined part is deformed.

[0139] The above reference value is derived using an AI algorithm according to the present invention, and specific details will be described later.

[0140] From the polygon approximation information of the above 4th step, information regarding the presence or absence of holes, corner angles, and edge continuity is calculated as the second geometric feature information (8th step).

[0141] The information from the above Step 8 is compared with each setting reference value (Step 9).

[0142] As a result of the comparison in Step 9 above, if the information in Step 8 above differs from the corresponding reference value by one or more set values, the precision machined product is determined to be in a shape defect state (Step 10).

[0143] For example, from the polygon approximation information, the presence or absence of (internal) holes (based on RETR_TREE), corner angles (e.g., 90° deviation), and edge continuity (based on Canny) are calculated as second geometric feature information and compared with their respective reference values. Therefore, if the two values ​​differ as a result of the comparison, it is determined to be a shape defect state. Through this, shape defects, such as defects, distortions, and cracks, can be detected in the corresponding precision machined part. In such cases, different weighted scores can be assigned to each criterion. High weights are applied to key indicators such as internal holes and filling ratios.

[0144] The present invention can effectively determine whether a precision machined product is a good product by using information on these seven representative geometric features. That is, it provides defect judgment criteria based on seven geometric features.

[0145] In such cases, another embodiment applies (ROI) image preprocessing and (maximum area) contour extraction to the geometric information of the precision machined part to prevent bias according to the K-means technique as described above, thereby effectively performing computer vision-based defect detection.

[0146] Specifically, the present embodiment performs the following operations for the second step.

[0147] First, the above 2nd step extracts the set ROI area of ​​the precision machined product from the video surveillance information of the above 1st step, converts the ROI image to grayscale, and separates the object and background by binarizing it with a set Gaussian blur (2-1st step).

[0148] For the object of the above Step 2-1, the outline is detected and the main outline is specified as the maximum area contour (Step 2-2).

[0149] Geometric information of the corresponding precision part is specifically extracted using the main contour of the above Step 2-2 (Step 2-3).

[0150] Color, texture, and edges are extracted from the geometric information of the above 2-3 steps and converted into high-dimensional feature vectors based on the vector transformation information of the first reference value_color, texture, and edges of the precision part (2-4-1 step).

[0151] For example, based on color, texture, and edge shape, it is converted into a feature vector of 300 dimensions or more, and the pattern is quantified using histograms, GLCM, Hu Moments, etc. That is, pattern information of reference values ​​(vector conversion information based on color, texture, and edge) is set and registered for the precision machined part, and based on this, the color, texture, and edge information according to the above 2-3 steps is quantified as a pattern.

[0152] Pattern information is extracted from the high-dimensional feature vector of Step 2-4-1 above and normalized based on the set second reference value_pattern quantification information (Step 2-4-2).

[0153] In the above 2-4-2 step, the normalization includes normalizing the feature scale difference for the pattern quantification information to a mean of 0 and a standard deviation of 1.

[0154] Through this, K-means bias in the next step can be effectively prevented.

[0155] For reference, normalized high-dimensional vectors can be utilized for unsupervised grouping and pre-classification automation of large ROIs.

[0156] Meanwhile, according to the present invention, the aforementioned reference value can be derived using the following AI algorithm.

[0157] To this end, the above 6th step operates as follows.

[0158] First, a dataset is obtained by collecting and accumulating geometric information within the set ROI area for each precision machined part from the corresponding video surveillance device, and first geometric feature information for each geometric information, namely the number of corners, filling ratio, convexity, and perimeter (Step 6-1).

[0159] Reference value information representing cluster characteristics is detected from the dataset of Step 6-1 above (Step 6-2).

[0160] The reference value information from Step 6-2 above is normalized (Step 6-3).

[0161] Using the normalization information of Step 6-3 above, independent_precision_machined_product_identification_information, geometric_information, and dependent_dependent_first_geometric_feature_information are set to calculate the first geometric feature information for each geometric information (Step 6-4).

[0162] Training data is created and trained using the independent and dependent variables from Step 6-4 above (Step 6-5).

[0163] Based on the learning result of the above Step 6-5, the setting reference value information of the above Step 6 is calculated and registered as the setting value of the above Step 6 (Step 6-6).

[0164] The above 9th step is as follows.

[0165] First, a dataset is obtained by collecting and accumulating geometric information within the set ROI area for each precision machined part and second geometric feature information for each geometric information, namely the presence or absence of holes, corner angles, and edge continuity, from the corresponding video surveillance device (Step 9-1).

[0166] Reference value information representing cluster characteristics is detected from the dataset of Step 9-1 above (Step 9-2).

[0167] The reference value information from Step 9-2 above is normalized (Step 9-3).

[0168] Using the normalization information of Step 9-3 above, independent_precision_machined_product_identification_information, geometric_information, and dependent_dependent_second_geometric_feature_information are set to calculate second geometric feature information for each geometric information (Step 9-4).

[0169] Training data is created and trained using the independent and dependent variables from Step 9-4 above (Step 9-5).

[0170] Based on the learning result of the above Step 9-5, the setting reference value information of the above Step 9 is calculated and registered as the setting value of the above Step 9 (Step 9-6).

[0171] The aforementioned AI algorithm is explained in detail.

[0172] The above AI algorithm adopts YOLOv11n-seg (3.4M parameters) for real-time performance and lightweighting to perform segmentation and detection based on 640×640 inputs, and secures performance suitable for inspecting machined parts with simple shapes.

[0173] By utilizing the Backbone-Neck-Head structure of YOLOv11 to extract and fuse multi-scale features and configuring an architecture that performs segmentation and detection simultaneously, the balance of accuracy, speed, and scalability is optimized.

[0174] A stable and consistent ROI generation pipeline is implemented by denormalizing the polygon coordinates generated by YOLOv11 to calculate and clip the minimum bounding box, extracting the ROI using OpenCV slicing, and performing automatic rotation and exception logging if necessary.

[0175] The entire ROI processing process is parallelized through multiprocessing to optimize rapid processing of large-scale datasets.

[0176] All ROIs are saved in data / processed / roi_extraction / while retaining their original filenames.

[0177] Transfer learning is performed by adopting ResNet18 based on ImageNet pre-training, stable training and high representation are secured through the Residual Block and Skip Connection structure, and the last FC layer is redesigned for 2 classes (OK / NG).

[0178] Through a fine-tuning strategy, the entire network is optimized to match the characteristics of the Cover-HUB inspection process data, the input is standardized to a 224×224×3 format, and classification performance and learning efficiency are maximized by utilizing pre-trained features.

[0179] It applies the Adam optimizer, ReduceLROnPlateau, and Early Stopping to induce stable convergence and automatically saves the best-performing model.

[0180] Augmentation such as rotation, flipping, and ColorJitter is applied to Train, while Valid / Test uses only normalization to ensure fairness in evaluation.

[0181] In addition, epoch-specific metrics and the Confusion Matrix are analyzed to check for overfitting and recall-centric performance, and the final model (.pth) is verified for field applicability using a test set.

[0182] Based on supervised learning, OK / NG label data is input, and weights are updated in a direction that minimizes CrossEntropyLoss, and stable learning and verification are performed with a DataLoader, mini-batch (32), and 30 Epoch configuration.

[0183] In the event of class imbalance, weight adjustments are applied to mitigate bias, and Accuracy, Precision, Recall, and F1-Score are used as evaluation metrics to precisely analyze the model's overall classification performance and error types.

[0184] Meanwhile, another embodiment of the aforementioned K-means technique calculates valid values ​​for intervals between each point, thereby enabling the generation of metadata for temperature, voltage, and current signals for each interval between each point.

[0185] This finds valid values ​​between intervals through linear regression-based trend line analysis.

[0186] To this end, Step 3-6 first sets the valid values ​​of Step 3-6, and then additionally generates valid values ​​for each interval between points using Equation 1 below (Step 3-6-1).

[0187] In the above Step 3-6-1, Equation 1 is Y1 j = α1X1 i T + β1 i (i = 1, ... , n) is.

[0188] Here, Y1 j is the valid value for each interval between each point, X1 i T is the predicted geometric information for each interval between points (T is the transpose), α1 is the coefficient, β1 i is an error variable.

[0189] In Step 3-6-1 above, additional valid values ​​are generated for each interval between each center point_valid point using Equation 2 below (Step 3-6-2).

[0190] In the above Step 3-6-2, Equation 2 is Y2 j = α2X2 i T + β2 i (i = 1, ... , n) is.

[0191] Here, Y2 j is the valid value for each interval between valid points, the said valid point is the centroid_valid value obtained through the K-means technique, X2 i T is the interval-by-interval prediction geometric information between valid points (T is the transpose), α2 is the coefficient, β2i is an error variable.

[0192] From the multiple valid values ​​for each interval obtained through Step 3-6-2 above, multiple random K values ​​are extracted using the K-means technique and set as center points (Step 3-6-3).

[0193] Calculate the difference in distance between the center point of Step 3-6-3 and the remaining points (Step 3-6-4).

[0194] The remaining points are clustered by assigning them to the nearest center point using the distance calculated in Step 3-6-4 above (Step 3-6-5).

[0195] The point corresponding to the center in the cluster of Step 3-6-5 above is updated as the center point (Step 3-6-6).

[0196] Steps 3-6-4 through 3-6-6 are repeated until the updated center point of Step 3-6-6 above does not change (Step 3-6-7).

[0197] The center point obtained through Step 3-6-7 above is set as a valid value, and valid values ​​for each interval between valid points are extracted (Step 3-6-8).

[0198] For the effective values ​​per interval between the effective points of the above Step 3-6-8, the set weight is Y1 of Equation 1. j Among them, the corresponding weight is assigned to each valid value in the corresponding interval (Step 3-6-9).

[0199] The final value is obtained using the valid value from Step 3-6-9 above (Step 3-6-10).

[0200] Meanwhile, in the above-described embodiment, when the precision machined product is determined to be in a shape deformation state or a shape defect state, the present invention reduces power through cycle control to save power according to image monitoring for each process.

[0201] The above cycle control controls the power applied to the load by repeating the on / off cycle at a constant rate according to the input control signal during a constant arbitrary period.

[0202] In this embodiment, for such a period, that is, a cycle, a period showing a power change equal to a first set value is measured for the set reference power value of the video surveillance device, the number obtained by leveling the power change period in units of a set level is calculated, and the value is obtained by dividing the set reference period of the reference power value by the number.

[0203] Therefore, by frequency modulating the corresponding reference power with this cycle information and continuously performing subsequent video surveillance operations, video surveillance is performed efficiently.

[0204] Specifically, it is as follows.

[0205] In this embodiment, first, after step 7 or step 10, if the precision machined product is determined to be in a shape-deformed state or a shape-defective state in step 7 or step 10, the difference between the information of step 5 and each reference value is calculated, and the difference between the information of step 8 and each reference value is calculated (step 11).

[0206] The difference values ​​of the above Step 11 are grouped by Step 5 and Step 8, and multiplied by each group (Step 12). That is, the difference values ​​of Step 5—number of corners, filling ratio, convexity, and perimeter—and the difference values ​​of Step 8—presence or absence of holes, corner angle, and edge continuity are grouped and multiplied by each group.

[0207] The setting reference power of the corresponding video surveillance device is frequency modulated by cycle control by the difference value of the above 12th step and proceed to the above 1st step (13th step).

[0208] In the above 13th step, the cycle control measures a period representing a power change of a first set value for the reference power, calculates the number of such power change periods by leveling them in units of set levels, detects cycle information by dividing the set reference period of the reference power by the number, and frequency modulates the reference frequency of the reference power using the cycle information.

[0209] In another embodiment, this cycle control can be performed a second time to further prevent power consumption due to continuous video surveillance.

[0210] To this end, in this embodiment, after step 13, if the precision machined product is secondarily determined to be in a shape-deformed state or shape-defective state through steps 7 and 10 according to step 13, the actual current power consumption of the image surveillance device is detected with respect to the reference power according to step 13 (step 14).

[0211] Compare the current power consumption of the above Step 14 with the reference power according to the above Step 13 (Step 15).

[0212] In the above 15th step, the difference between the current power consumption of the above 14th step and the reference power according to the above 13th step is calculated based on the comparison result (16th step).

[0213] The reference power according to the 13th step is frequency modulated by the 2nd cycle control by the difference value of the 16th step, and the process proceeds to the 1st step (17th step).

[0214] In the above 17th step, the second cycle control measures a cycle representing a power change of a second set value for the current power consumption, calculates the number of such cycles obtained by leveling the corresponding power change cycles in the level units, detects the second cycle information by dividing the corresponding cycle of the reference power according to the above 13th step by the number, and frequency modulates the reference frequency of the reference power according to the above 13th step using the corresponding cycle information.

[0215] Meanwhile, in the aforementioned immediately preceding embodiment, regarding the power change cycle, that is, the power change cycle required to perform secondary cycle control to prevent power consumption due to video surveillance, only the effective value can be extracted to obtain accurate cycle information.

[0216] To this end, the present embodiment first collects a plurality of periodic information indicating a power change equal to the second set value with respect to the reference power of the 17th step (17-1 step).

[0217] Among the multiple power change cycle information of Step 17-1 above, multiple random K points are extracted using the K-means technique and set as center points (Step 17-2).

[0218] Calculate the difference in distance between the center point of Step 17-2 and the remaining points (Step 17-3).

[0219] The remaining points are clustered by assigning them to the nearest center point based on the distance from Step 17-3 above (Step 17-4).

[0220] The point corresponding to the center in the cluster of Step 17-4 above is updated as the center point (Step 17-5).

[0221] Steps 17-3 through 17-5 are repeated until the center point of Step 17-5 does not change (Step 17-6).

[0222] The center point of the above Step 17-6 is set as a valid value (Step 17-7).

[0223] Calculate the average value of the valid values ​​from Step 17-7 above (Step 17-8).

[0224] The average value of the above Step 17-8 is set as the power change cycle (Step 17-9).

[0225] Meanwhile, another embodiment can calculate valid values ​​for each interval between each point, and generate valid values ​​of power change cycle information, i.e., metadata, for each interval between each point.

[0226] This finds valid values ​​between intervals through linear regression-based trend line analysis.

[0227] Specifically, first, valid values ​​for each segment are calculated through a linear regression-based trend line analysis according to Equation 1 below, and additional metadata for power change cycle information is generated for each segment. Next, a set weight is assigned to the segment metadata of the valid point within the segment metadata, and the resulting final value is obtained as the corresponding valid value.

[0228] The specific actions are as follows.

[0229] That is, the above Step 17-7 is configured as follows.

[0230] First, Step 17-7 sets the center point of Step 17-6 as a valid value, and then additionally generates valid values ​​for power change period information for each interval between points using Equation 1 below (Step 17-7-1).

[0231] In the above Step 17-7-1, Equation 1 is Y1 j = α1X1 i T + β1 i (i = 1, ... , n) is.

[0232] Here, Y1 j is the valid value for each interval between each point, X1 iT is the periodic information for predicted power change by interval between each point (T is the prefix), α1 is the coefficient, β1 i is an error variable.

[0233] In the above Step 17-7-1, additional valid values ​​of power change period information are generated for each interval between each center point_valid point using Equation 2 below (Step 17-7-2).

[0234] In the above Step 17-7-2, Equation 2 is Y2 j = α2X2 i T + β2 i (i = 1, ... , n) is.

[0235] Here, Y2 j is the valid value for each interval between valid points, the said valid point is the centroid_valid value obtained through the K-means technique, X2 i T is the periodic information for predicted power change by interval between valid points (T is the prefix), α2 is the coefficient, β2 i is an error variable.

[0236] From the multiple valid values ​​for each interval obtained through Step 17-7-2 above, multiple random K values ​​are extracted using the K-means technique and set as center points (Step 17-7-3).

[0237] Calculate the difference in distance between the center point of Step 17-7-3 and the remaining points (Step 17-7-4).

[0238] The remaining points are clustered by assigning them to the nearest center point using the distance calculated in Step 17-7-4 above (Step 17-7-5).

[0239] The point corresponding to the center in the cluster of Step 17-7-5 above is updated as the center point (Step 17-7-6).

[0240] Steps 17-7-4 through 17-7-6 are repeated until the updated center point of Step 17-7-6 does not change (Step 17-7-7).

[0241] The center point obtained through the above Step 17-7-7 is set as a valid value, and valid values ​​for each interval between valid points are extracted (Step 17-7-8).

[0242] For the valid values ​​per interval between the valid points of Step 17-7-8 above, the set weight is Y1 of Equation 1 above. j Among them, the corresponding weight is assigned to each valid value of the corresponding interval (Step 17-7-9).

[0243] The final value is obtained using the valid value from Step 17-7-9 above (Step 17-7-10).

[0244] For reference, the above weighting factors can be determined differently depending on the material, specifications, and usage of the precision machined part, and additionally, they can be customized for each driving voltage. Explanation of the symbols

[0245] 100 : Management information processing device

Claims

Claim 1 A method for YOLO Resnet AI deep learning-based image analysis inspection of a precision machined product of a management information processing device (100), comprising: a first step of custom-setting image monitoring devices for each of the multiple processes of the precision machined product to collect multiple image monitoring information per work unit of the precision machined product; wherein, in the first step, the custom setting of the image monitoring devices comprises: 1) a first vision inspection_dimension: area image monitoring device; 2) a second vision inspection_2D appearance: line scan image monitoring device; 3) a third vision inspection_3D appearance: 3D scan image monitoring device; 4) a fourth vision monitoring_characteristics: multispectral image monitoring device; a second step of collecting multiple geometric information within a set ROI area of ​​the precision machined product using the image monitoring information of the first step; a third-1 step of extracting multiple arbitrary K points from the geometric information of the second step using the K-means technique and setting them as center points; and the Step 3-2, which calculates the difference in distance between the center point of Step 3-1 and the remaining points; Step 3-3, which performs clustering by assigning the remaining points to the closest center point based on the distance calculated in Step 3-2; Step 3-4, which updates the point corresponding to the center in the cluster of Step 3-3 as the center point; Step 3-5, which repeatedly performs Steps 3-2 through 3-4 until the updated center point of Step 3-4 no longer changes; Step 3-6, which sets the center point obtained through Step 3-5 as a valid value to specify the geometric information of the precision machined part using only that center point; Step 4, which approximates the geometric information of the precision machined part from Step 3-6 with a set polygon; Step 5, which calculates the number of corners, filling ratio, convexity, and perimeter information as the first geometric feature information from the polygon approximation information of Step 4; and Step 5, which compares the information from Step 5 with the respective set reference values. Step 6;A 7th step of determining the precision machined product to be in a shape-deformed state if, as a result of the comparison in the 6th step, one or more of the information in the 5th step differs from the corresponding reference value by more than a set value; an 8th step of calculating information on the presence or absence of holes, corner angles, and edge continuity as second geometric feature information from the polygon approximation information of the 4th step; and a 9th step of comparing the information in the 8th step with each set reference value. and, if, as a result of the comparison in the above-mentioned 9th step, one or more of the information in the above-mentioned 8th step differs from the corresponding reference value by more than a set value, the above-mentioned precision machined product is determined to be in a shape defect state; the above-mentioned 2nd step comprises: 2-1 step of extracting the set ROI area of ​​the above-mentioned precision machined product from the image surveillance information of the above-mentioned 1st step, converting the corresponding ROI image to grayscale, and binarizing it with a set Gaussian blur to separate the object and the background; 2-2 step of detecting the outline of the object in the above-mentioned 2-1 step and specifying the main outline as a maximum area contour; 2-3 step of specifying and extracting the geometric information of the above-mentioned precision machined product using the main outline of the above-mentioned 2-2 step; and 2-4-1 step of extracting color, texture, and edge from the geometric information of the above-mentioned 2-3 step and converting them into a high-dimensional feature vector based on the vector conversion information of the above-mentioned precision machined product's set 1st reference value_color, texture, and edge. and a YOLO Resnet AI deep learning-based image analysis inspection method comprising: a 2-4-2 step of extracting pattern information from a high-dimensional feature vector of the 2-4-1 step and normalizing it based on the set 2nd reference value_pattern quantification information; wherein, in the 2-4-2 step, the normalization is characterized by normalizing the feature scale difference for the pattern quantification information to a mean of 0 and a standard deviation of 1. Claim 2 In claim 1, the 6th step comprises: a 6-1 step of securing a dataset by collecting and accumulating geometric information within a set ROI area for each precision machined product and first geometric feature information for each geometric information in the corresponding image surveillance device; a 6-2 step of detecting reference value information representing clustering characteristics in the dataset of the 6-1 step; a 6-3 step of normalizing the reference value information of the 6-2 step; a 6-4 step of setting independent variables and dependent variables for calculating first geometric feature information for each geometric information using the normalized information of the 6-3 step; and a 6-5 step of creating training data and learning using the independent variables and dependent variables of the 6-4 step. and a 6-6 step of calculating the set reference value information of the 6th step as the learning result of the 6-5 step; or the 9th step comprises: a 9-1 step of securing a dataset by collecting and accumulating geometric information within a set ROI area for each precision machined product and second geometric feature information for each geometric information in the corresponding video surveillance device; a 9-2 step of detecting reference value information representing cluster characteristics in the dataset of the 9-1 step; a 9-3 step of normalizing the reference value information of the 9-2 step; a 9-4 step of setting independent and dependent variables to calculate the second geometric feature information for each geometric information using the normalized information of the 9-3 step; and a 9-5 step of creating training data and learning using the independent and dependent variables of the 9-4 step. A YOLO Resnet AI deep learning-based image analysis inspection method comprising: a 9-6 step of calculating the setting reference value information of the 9th step as a learning result of the 9-5 step; Claim 3 In claim 1, the 3-6 step comprises a 3-6-1 step of additionally generating valid values ​​for intervals between each point using the following Equation 1 after setting the valid values ​​of the 3-6 step; wherein in the 3-6-1 step, Equation 1 is Y1 j = α1X1 i T + β1 i (i = 1, ... , n) and, (here, Y1 j is the valid value for each interval between each point, X1 i T is the predicted geometric information for each interval between points (T is the transpose), α1 is the coefficient, β1 i (is an error variable) Step 3-6-2, in which valid values ​​are additionally generated for each interval between each center point_valid point using Equation 2 below, in Step 3-6-1 above; in Step 3-6-2 above, Equation 2 is, Y2 j = α2X2 i T + β2 i (i = 1, ... , n) and, (here, Y2 j is the valid value for each interval between valid points, the said valid point is the centroid_valid value obtained through the K-means technique, X2 i T is the interval-by-interval prediction geometric information between valid points (T is the transpose), α2 is the coefficient, β2 i (is an error variable) Step 3-6-3, which sets multiple random K points as center points by extracting them using the K-means technique from multiple valid values ​​per interval obtained through Step 3-6-2 above; Step 3-6-4, which calculates the difference in distance between the center point of Step 3-6-3 and the remaining points; Step 3-6-5, which performs clustering by assigning the remaining points to the closest center point based on the distance calculated in Step 3-6-4; Step 3-6-6, which updates the point corresponding to the center in the cluster of Step 3-6-5 as the center point; Step 3-6-7, which repeatedly performs Steps 3-6-4 through 3-6-6 until the updated center point of Step 3-6-6 no longer changes; Step 3-6-8, which sets the center point obtained through Step 3-6-7 as a valid value and extracts valid values ​​per interval between valid points; the valid For the valid values ​​for each interval between points, the set weight is Y1 of Equation 1 above. j A YOLO Resnet AI deep learning-based image analysis inspection method comprising: a step 3-6-9 of assigning a corresponding weight to each valid value of the corresponding section; and a step 3-6-10 of obtaining a final value from the valid values ​​of the step 3-6-9. Claim 4 In any one of claims 1-3, if the precision machined product is determined to be in a shape-deformed state or a shape-defect state in the 7th step or the 10th step, the 11th step calculates the difference between the information of the 5th step and each reference value, and calculates the difference between the information of the 8th step and each reference value; the 12th step groups the difference values ​​of the 11th step by the 5th step and the 8th step and multiplies them by each group; A YOLO Resnet AI deep learning-based image analysis inspection method comprising: a 13th step of proceeding to the 1st step by frequency modulating the set reference power of the corresponding video surveillance device by cycle control by the difference value of the 12th step; wherein, in the 13th step, the cycle control is characterized by measuring a period indicating a power change by a first set value for the reference power, calculating the number obtained by leveling the power change period in units of a set level, detecting cycle information with the value obtained by dividing the set reference period of the reference power by the number, and frequency modulating the reference frequency of the reference power with the cycle information. Claim 5 In claim 4, after the 13th step, if the precision machined product is secondarily determined to be in a shape-deformed state or shape-defective state through the 7th step and the 10th step according to the 13th step, a 14th step of detecting the actual current power consumption of the corresponding image surveillance device with respect to the reference power according to the 13th step; a 15th step of comparing the current power consumption of the 14th step with the reference power according to the 13th step; and a 16th step of calculating the difference value between the current power consumption of the 14th step and the reference power according to the 13th step in the 15th step; A YOLO Resnet AI deep learning-based image analysis inspection method comprising: a 17th step of frequency modulating the reference power according to the 13th step by a second cycle control by a difference value of the 16th step, and proceeding to the 1st step; wherein in the 17th step, the second cycle control is characterized by measuring a cycle representing a power change by a second set value for the current power consumption, calculating the number of such cycles obtained by leveling the corresponding power change cycles in the level units, detecting second cycle information with the value obtained by dividing the corresponding cycle of the reference power according to the 13th step by the number, and frequency modulating the reference frequency of the reference power according to the 13th step with the corresponding cycle information.

Images