A new AI-based learning model generation system for visual inspection on product production lines

Abstract

In the AI-based new learning model generation system for visual inspection on product production lines of the present invention, the supplementary set extraction module extracts two or more supplementary data sets from a plurality of learning data sets used in the learning of an existing learning model generated for visual inspection on product production lines based on judgment type information. Furthermore, the supplementary set determination module calculates the similarity between the new learning data and the learning images of the supplementary data sets, and determines any one above the baseline value as the supplementary learning data. Furthermore, the new model generation module causes the pre-trained model to learn the supplementary learning data set and the new learning data to generate a new learning model. Thus, when a new learning model is generated using a small amount of new learning data, previous learning data sets similar to the new learning data are additionally used for learning, thereby improving the judgment accuracy of the new learning model.

Description

Technical Field

[0001] The present invention relates to an AI-based new learning model generation system for visual inspection on product production lines, and more specifically, to an AI-based new learning model generation system for visual inspection on product production lines that uses a learning dataset used in the generation of a registered existing learning model for the generation of a new learning model. Background Art

[0002] Deep learning, a form of AI (artificial intelligence), is a machine learning method based on artificial neural networks (ANNs), which mimics the way human neurons learn. In recent years, deep learning has garnered significant attention due to its contributions to image recognition, speech recognition, and natural language processing.

[0003] Such deep learning technology has also been applied to visual inspection on product production lines in recent years. As an example, Korean Patent Publication No. 10-2019-0063839 discloses "Quality Inspection Method and System Based on Machine Vision Using Deep Learning in Manufacturing Processes."

[0004] In the technology disclosed in the above-mentioned Korean Patent Gazette, a product image for learning is generated, a classifier for distinguishing good products from defective products is used to learn the generated product image for learning, and the product is judged as a good product or a defective product using the learned classifier.

[0005] In deep learning technology such as the technology disclosed in the above-mentioned Korean patent publication, image data recording good products and defective products and label information indicating whether the image is an image of a defective product or a good product are input into a learning model such as a classification machine for learning, thereby generating a new learning model for good and defective inspection.

[0006] Generally, various types of defect inspections are performed on production lines. A production line for a single product includes multiple processes, such as injection molding, sheet metal forming, subassembly, and final assembly. Visual inspections are performed on the results of each process, such as components manufactured by injection molding or sheet metal forming, subassembly, and final assembly.

[0007] Figure 1 This is a diagram schematically showing an example of a general product production line.

[0008] Reference Figure 1 The product production line includes injection molding, sheet forming, sub-assembly, and final assembly. The injection molding and sheet forming processes are used to manufacture the components required for product production. A single product production line can include multiple injection molding or sheet forming processes.

[0009] Components manufactured through injection molding or sheet metal forming processes may have various forms of defects, such as dents, cracks, and scratches. Therefore, the dent inspection, crack inspection, and scratch inspection processes are included to detect such defects.

[0010] Subassembly process is a process of producing subcomponents as components for assembly, e.g. Figure 1 As shown, the sub-assemblies are further inspected for dents, cracks, and scratches, and additionally inspected for assembly defects such as missing or misaligned components.

[0011] The final assembly process, where subcomponents are assembled to produce a product, involves visual inspections for foreign matter, assembly defects, and other issues. Inspections for dents, cracks, and scratches are also performed during the final assembly process.

[0012] However, the results of each process are different, so the form of the image used as learning data is different. Even if the defect type is the same, the form of the defect will be different depending on the result, and the result itself as its background will also be different, so the form of the result image will be different. In theory, it is necessary to generate each learning model according to the result of each process and the type of defect for application.

[0013] Furthermore, for manufacturers producing diverse product lines, each product will have a different production line. Even with the same injection molding process, the resulting product will differ, necessitating the creation of a separate learning model for each product. Similarly, when changing production lines to produce new products, a new learning model must be created that matches the individual processes and defect types of the new production line.

[0014] Here, the judgment accuracy of the new learning model is determined by various factors. The amount of learning data, the selection of hyperparameters set in the initial learning stage, the selection of the pre-training model (pre-training model, the same below) and other important factors that determine the judgment accuracy.

[0015] However, in the case of a new learning model applied to a new product production line, the amount of learning data, especially the amount of bad data, can only be relatively small, and the judgment accuracy of the new learning model generated by learning with a small amount of learning data can only be low. Summary of the Invention

[0016] Technical issues

[0017] The purpose of the present invention is to provide a new AI-based learning model generation system for visual inspection on product production lines: the learning data set used in the generation of registered existing learning models is used in the generation of new learning models, so that the judgment accuracy can be improved when generating new learning models with a small amount of learning data.

[0018] The purpose of the present invention is to make judgments based on the similarity between learning images when deciding to add previous learning data sets to the generation of a new learning model, and to extract learning data sets that are close to the learning images that constitute the new learning data from the previous learning data sets to be added.

[0019] The purpose of the present invention is to improve the similarity between the new learning data and the new learning data by using the feature map and the feature point distribution when judging the similarity, thereby improving the judgment accuracy when generating a new learning model.

[0020] The purpose of the present invention is to determine the pre-trained model used in the generation of a new learning model and the determination of an additional learning data set as the best learning model among the registered existing learning models, thereby improving the judgment accuracy of the new learning model.

[0021] The purpose of the present invention is to reduce the time required to determine the pre-training model, improve the efficiency of the decision process, and recommend the best pre-training model.

[0022] Means of solving technical problems

[0023] In the AI-based new learning model generation system for visual inspection on product lines of the present invention, a supplementary set extraction module extracts two or more supplementary set data sets from multiple learning data sets used to learn an existing learning model generated for visual inspection on product lines based on judgment type information. The additional set determination module calculates the similarity between the new learning data and the learning images in the supplementary set data sets and determines any set that exceeds a baseline value as additional learning data. Furthermore, the new model generation module causes a pre-trained model to learn the additional learning data sets and the new learning data to generate a new learning model.

[0024] The judgment type information includes at least one of defect type information regarding the type of defect, product type information regarding the type of the inspected product, and component type information regarding the type of the inspected component. Each of the learning data sets stored in the data storage includes model information including at least one of the defect type information, product type information, and component type information. In this case, the complement set extraction module extracts the complement data set by referring to the model information.

[0025] The supplementary set extraction module extracts the supplementary data set according to a priority order set in the order of the defect type information, product type information, and component type information.

[0026] The additional set determination module generates a feature map for the new learning data and calculates a feature point distribution, generates a feature map for the supplementary data set and calculates a feature point distribution, and calculates a difference between the feature point distributions as a similarity.

[0027] The AI-based new learning model generation system of the present invention also includes: a learning model storage device, which stores a plurality of the above-mentioned existing learning models; a candidate model extraction module, which extracts two or more candidate models based on the above-mentioned judgment type information; and a model determination module, which applies new learning data to two or more candidate models to calculate the judgment accuracy, and determines the highest candidate model with a judgment accuracy above a preset first benchmark value as the above-mentioned pre-trained model.

[0028] In the case where there is a similar candidate model whose difference in judgment accuracy from the above-mentioned top candidate model is within a preset similarity range, the above-mentioned model determination module extracts the feature point distribution of the feature graph generated for the above-mentioned new learning data during the judgment process of the above-mentioned top candidate model and the above-mentioned similar candidate model, and determines any one with a larger difference in feature point distribution as the above-mentioned pre-trained model.

[0029] The above-mentioned model determination module uses the average value of the difference in the above-mentioned feature point distribution between a preset number of bad learning data and normal learning data arbitrarily selected from multiple learning data applied in the judgment process of the above-mentioned top-level candidate model and the above-mentioned similar candidate model to determine the pre-trained model.

[0030] When the above-mentioned judgment accuracy is below the above-mentioned first benchmark value, the above-mentioned model determination module extracts the top n as re-inspection candidate models based on the above-mentioned judgment accuracy, enables each of the above-mentioned re-inspection candidate models to learn a preset number of learning data among the multiple learning data, and then inputs the learning data into the above-mentioned re-inspection candidate model that has learned the preset number of learning data among the multiple learning data to judge whether it is good or bad, and determines the top re-inspection candidate model whose judgment accuracy of each of the above-mentioned re-inspection candidate models is above the preset second benchmark value as the above-mentioned pre-trained model.

[0031] When extracting the re-inspection candidate models, the model decision module removes the candidate models whose judgment accuracy is below a preset lower limit from the re-inspection candidate models.

[0032] In the case that there is a similar re-inspection candidate model among the multiple re-inspection candidate models, the difference between the above-mentioned judgment accuracy and the above-mentioned top re-inspection candidate model is within a preset similarity range, the above-mentioned model determination module extracts the feature point distribution of the feature graph generated for each of the above-mentioned learning data during the judgment process of the above-mentioned top re-inspection candidate model and the above-mentioned similar re-inspection candidate model, and among the above-mentioned top re-inspection candidate model and the above-mentioned similar re-inspection candidate model, any one with a larger difference in the above-mentioned feature point distribution between the bad learning data and the normal learning data is determined as the above-mentioned pre-trained model.

[0033] The above-mentioned model determination module calculates the average value of the difference in the distribution of the above-mentioned feature points between a preset number of bad learning data and normal learning data, and determines any model with a larger above-mentioned average value among the above-mentioned top re-inspection candidate model and the above-mentioned similar re-inspection candidate model as the above-mentioned pre-training model.

[0034] The difference between the distributions of the above feature points can be calculated using the KL-Divergence algorithm.

[0035] Effects of the Invention

[0036] The AI-based new learning model generation system for visual inspection on a product production line of the present invention has one or more of the following effects.

[0037] First, the learning data set used in the generation of the registered existing learning model is used in the generation of the new learning model. When generating a new learning model with a small amount of learning data, a large amount of learning data can be ensured, which can improve the judgment accuracy of the newly generated new learning model.

[0038] Second, when deciding to add a learning dataset to the previous learning dataset, the similarity between learning images is used as a basis, and the additional learning dataset to be added in the generation of the new learning model is determined to be the image most similar to the new learning data.

[0039] Third, when judging the similarity between learning data, the similarity is calculated using the feature map and feature point distribution calculated by the pre-training model, which can improve the new learning data and similarity and significantly improve the judgment accuracy when generating a new learning model.

[0040] Fourth, candidate data sets are extracted using the judgment type information including any one of defect type information, product type information, and component type information, and a learning data set having a high similarity to the new learning data is extracted as a candidate.

[0041] Fifth, the pre-trained model used in the generation of a new learning model and the determination of an additional learning data set is determined to be the best learning model among the existing registered learning models, thereby improving the judgment accuracy of the new learning model.

[0042] Sixth, when determining the pre-trained model, the judgment type information including one of the defect type information, product type information, and component type information is used to extract the backup model used to determine the pre-trained model, and the learning model that has similarity with the new learning model is extracted as the backup model.

[0043] Seventh, when determining the pre-training model, the backup model only judges a part of the new learning data, and determines the pre-training model based on the judgment accuracy, thereby reducing the time required for determining the pre-training model and recommending the best pre-training model.

[0044] Eighth, when determining the pre-trained model, when the judgment accuracy of all the candidate models is below the first benchmark value, the top n candidate models are extracted as re-examination candidate models, and the pre-trained model is determined after the re-examination candidate models learn part of the learning data, thereby improving the efficiency of the pre-trained model determination process and recommending the best pre-trained model.

[0045] Ninth, when determining a pre-training model, if the accuracy is within a similar range, an additional evaluation process is performed using the feature point distribution to recommend the best pre-training model. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a diagram schematically showing an example of a general product production line.

[0047] Figure 21 is a diagram showing the structure of a new learning model generation system based on AI according to an embodiment of the present invention.

[0048] Figure 3 and Figure 4 This is a diagram illustrating a new learning model generation process of the AI-based new learning model generation system according to an embodiment of the present invention.

[0049] Figure 5 This figure shows an example of a feature map and a feature point distribution generated during the similarity calculation process in the AI-based new learning model generation system according to an embodiment of the present invention.

[0050] Figures 6 to 11 This is a diagram illustrating another example of a new learning model generation process of the AI-based new learning model generation system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In the AI-based new learning model generation system for visual inspection on a product production line of the present invention, a data storage device storing multiple learning data sets and multiple new learning data collected for generating a new learning model stores the above-mentioned learning data sets. Each of the above-mentioned learning data sets is applied to the learning of an existing learning model that has been generated for visual inspection on a product production line. The new learning model generation system includes: a supplementary set extraction module, which, upon receiving judgment type information for generating a new learning model, extracts at least two or more supplementary data sets from the multiple learning data sets based on the above-mentioned judgment type information; a supplementary set determination module, which calculates the similarity between the new learning image included in the above-mentioned new learning data and the supplementary learning image included in the above-mentioned supplementary data set, and determines a supplementary data set whose similarity is greater than a registered benchmark value as a supplementary learning data set; and a new model generation module, which applies the above-mentioned supplementary learning data set determined by the above-mentioned supplementary set determination module and the above-mentioned new learning data to a pre-trained model to generate the above-mentioned new learning model.

[0052] The advantages and features of the present invention, as well as the methods for achieving the aforementioned advantages and features, can be more clearly understood by referring to the following embodiments, which are described in detail with reference to the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and can be embodied in various different forms. These embodiments are provided to fully disclose the present invention and clearly inform those skilled in the art of the scope of the invention. The present invention is to be defined within the scope of the claims. Throughout this specification, the same symbols represent the same components.

[0053] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0054] Figure 21 is a diagram showing the structure of a new learning model generation system 100 based on AI according to an embodiment of the present invention. Figure 2 The AI-based new learning model generation system 100 according to an embodiment of the present invention includes a data memory 130, a candidate model extraction module 161, an additional set determination module 120, and a new model generation module 140. Furthermore, the AI-based new learning model generation system 100 according to an embodiment of the present invention includes the data memory 130, the candidate model extraction module 161, the additional set determination module 120, and a main processor 150 that controls the operation of the new model generation module 140. The main processor 150 includes hardware components such as a CPU and RAM for executing the operations of the various components, and software components such as an operating system.

[0055] A plurality of learning data sets and a plurality of new learning data are stored in the data storage 130. In each learning data set, learning data used for learning an existing learning model generated for visual inspection on the product production line 300 is stored in correspondence with each learning model.

[0056] More specifically, when an existing learning model is generated through learning, a plurality of learning data used in the learning is stored as one learning data set relative to the existing learning model. Figure 1 The product production line 300 includes various visual inspection processes such as dent inspection, crack inspection, and scratch inspection processes for inspecting the results of each process, i.e., defective parts or products. The learning data used when generating learning models applied to each visual inspection process, such as a learning model for dent inspection, a learning model for crack inspection, and a learning model for scratch inspection, are stored as a set relative to each model.

[0057] The plurality of new learning data stored in the data storage 130 is collected for generating a new learning model to be generated by the AI-based new learning model generation system 100 of the present invention. For example, when establishing a new product production line 300 or changing the line on an existing product production line 300 to produce a new product, learning data including defective and normal images and label information indicating whether the images are defective or normal is collected for visual inspection of the new products or parts.

[0058] Here, the present invention describes the situation where multiple learning data sets and new learning data are stored in the data storage device 130, but multiple learning data sets and new learning data can also be stored separately in a physically separated data storage device 130, which is also included in the technical idea of ​​the present invention.

[0059] Upon receiving the judgment type information for generating a new learning model, the complement set extraction module 110 extracts at least two complement data sets from the plurality of learning data sets based on the judgment type information.

[0060] The judgment type information includes information about what kind of product or component the new learning model is applied to for visual inspection and what kind of defective form. In the present invention, the case of including at least one of defect type information about the type of defect, product type information about the type of the inspection object product, and component type information about the type of the inspection object component is taken as an example.

[0061] Accordingly, each learning dataset stored in data storage 130 includes model information containing at least one of defect type information, product type information, and component type information. Upon receiving the judgment type information, the complement set extraction module 110 extracts a complement set from the learning dataset by referring to the model information of each learning dataset.

[0062] In the present invention, the following scenario is used as an example: the complement set extraction module 110 extracts a preset number of learning datasets as complement datasets according to a priority order defined by the order of defect type information, component type information, and product type information. For example, upon receiving judgment type information including a dent as defect type information, a smartphone as product type information, and a housing as component type information, the complement set extraction module 110 extracts the learning dataset corresponding to the dent in one go. If the number of extracted learning datasets exceeds the preset number, the module extracts the preset number of complement datasets by re-extracting the learning dataset corresponding to the smartphone.

[0063] The additional set determination module 120 calculates the similarity between the new learning images included in the new learning data and the candidate learning images included in the candidate datasets, and determines as an additional learning dataset a candidate dataset having a similarity greater than a registered reference value.

[0064] The new model generation module 140 applies the learning data constituting the additional learning data set determined by the additional set determination module 120 and the new learning data to the pre-trained model, and generates a new learning model by learning the pre-trained model.

[0065] According to the structure as described above, when a learning data set used in the generation of a registered existing learning model is used in the generation of a new learning model, a supplementary data set is extracted using judgment type information, and a supplementary data set with a high similarity to the new learning data among the learning data included in the supplementary data set is determined as an additional learning data set and used in the generation of the new learning model, thereby improving the judgment accuracy when generating a new learning model with a small amount of learning data.

[0066] Below, refer to Figure 3 and Figure 4 , the new learning model generation process of the AI-based new learning model generation system 100 in an embodiment of the present invention is described in detail.

[0067] Reference Figure 3 First, new learning data for generating a new learning model is collected and registered (S30). Then, judgment type information for generating the new learning model is input (S31). Here, as described above, the judgment type information includes at least one of defect type information, product type information, and component type information.

[0068] When the judgment type information is input ( S31 ), the complement set extraction module 110 performs a process of determining an additional learning data set based on the judgment type information ( S32 ).

[0069] Reference Figure 4 The complement set extraction module 110 refers to the model information of each learning data set stored in the data storage 130 according to the judgment type information, and extracts a preset number of complement data sets from the multiple learning data sets (S321).

[0070] Then, the complement set extraction module 110 generates a feature map for calculating similarity (S322). In the present invention, the following case is taken as an example: the complement set extraction module 110 generates a new image feature map from a preset number of new learning images in the plurality of new learning data, and generates a complement image feature map from a preset number of complement learning images in the plurality of learning data constituting the complement data set.

[0071] In the present invention, a preset feature map generation algorithm is applied in the generation of feature maps, and the feature maps are generated by learning a pre-trained model, which will be described in detail later.

[0072] Therefore, only a part of the supplementary learning images in the supplementary data set and a part of the new learning images in the new learning data are used for calculating the similarity, thereby minimizing the time required for the new learning model generation process, especially the process of determining the additional learning data set.

[0073] When the new image feature map and the supplementary image feature map are generated ( S322 ), the new feature point distribution of each new image feature map is calculated, and the supplementary feature point distribution of each supplementary image feature map is calculated ( S323 ).

[0074] Figure 5 1 and 2 are diagrams showing examples of feature maps and feature point distributions generated in the similarity calculation process in the AI-based new learning model generation system 100 according to the embodiment of the present invention. Figure 5 The left image of (a) is the bad learning data, and the right image is the normal learning data. The left image of (b) is the feature map of the bad learning data, and the right image is the feature map of the normal learning data. The left image of (c) is the feature point distribution of the feature map of the bad learning data, and the right image is the feature point distribution of the feature map of the normal learning data.

[0075] When comparing the similarity between the candidate learning images in the candidate dataset and the new learning images in the new learning data, the similarity between the defective learning data or the similarity between the normal learning data is compared. For example, when the images of normal products are similar, the similarity between the normal learning data is calculated, and when the images of defective products are similar, the similarity between the defective learning data is compared to determine the additional learning dataset.

[0076] In the present invention, the difference between the new feature point distribution and the candidate feature point distribution is calculated as the similarity ( S324 ). In the present invention, the calculation is performed using the KL-Divergence algorithm as an example.

[0077] After calculating the similarities between each candidate dataset and the new learning data (S324), the complement set extraction module 110 determines a candidate dataset with a similarity exceeding a preset threshold value as the additional learning dataset (S326). If there are multiple candidate datasets exceeding the threshold value, the one with the highest similarity is determined as the additional learning dataset. Conversely, if no candidate dataset exists with a similarity exceeding the preset threshold value, the complement set extraction module 110 considers the determination a failure (S327), terminating the additional learning dataset determination process.

[0078] When calculating the similarity between the supplementary data set and the new learning data based on the structure as described above, the feature point distribution of the feature map generated during the learning process of the actual learning model is applied to the similarity comparison, so that the additional learning data set used when learning the new learning model can be selected to be closer to the image of the new learning data, thereby improving the judgment accuracy of the new learning model.

[0079] When an additional learning dataset is determined through the above process, the new model generation module 140 inputs the learning data constituting the additional learning dataset and the new learning data into the pre-trained model to cause the pre-trained model to learn ( S33 ). Finally, a new learning model is generated through learning of the pre-trained model ( S34 ).

[0080] Re-reference Figure 2 , the AI-based new learning model generation system 100 of an embodiment of the present invention also includes a pre-training module determination module 160 and a training model storage 170.

[0081] The learning model memory stores multiple existing learning models used for visual inspection on product production line 300. The existing learning models stored in the learning model memory are stored in association with the respective learning data sets stored in data memory 130. Specifically, when a dent inspection learning model for a specific component is stored in the learning model memory, the learning data used to generate the dent inspection learning model is stored in the learning data set. The existing learning models and learning data sets correspond to model information containing at least one of defect type information, component type information, and product type information. This will be described in detail later.

[0082] The pre-trained module determination module 160 determines one of the existing learning models stored in the training model storage 170 as the pre-trained model to be used in the new model generation module 140 and the additional set determination module 120. In the present invention, the case where the pre-trained module determination module 160 includes the candidate model extraction module 161 and the model determination module 162 is taken as an example.

[0083] Based on the judgment type information, the candidate model extraction module 161 extracts at least two candidate models from the plurality of existing learning data. Furthermore, the model determination module 162 inputs a predetermined number of new learning data stored in the data memory 130 into each candidate model to determine whether the candidate model is good or bad. The top candidate model whose judgment accuracy among the candidate models exceeds a predetermined first reference value is determined as the pre-trained model.

[0084] More specifically, in order to generate a new learning model, multiple learning data are collected. When judgment type information such as bad type information is received, the supplementary model extraction module 161 retrieves the existing learning model generated to check the same bad type as the bad type information from the existing learning models stored in the learning model memory, and extracts it as a supplementary model.

[0085] Here, the candidate model extraction module 161 extracts a preset number of existing learning models as candidate models according to the priority set in the order of defect type information, component type information, and product type information, as described above, based on the model information registered for each existing learning model.

[0086] Through the process as described above, the defective existing learning model with the most similar morphology to the new learning model can be extracted as a backup model from the multiple existing learning models stored in the learning model memory, which can improve the judgment accuracy of the new learning model generated using the final selected pre-trained model in the subsequent process.

[0087] Model determination module 162 extracts a predetermined amount of learning data from the learning data stored in data storage 130. For example, if 1,000 new learning images are stored as new learning data, calculating the judgment accuracy using all 1,000 new learning images would require a significant amount of time to determine the pre-trained model. Therefore, extracting a predetermined amount, for example, 20%, or 200 new learning images, for use in calculating the judgment accuracy can reduce the time required to determine the pre-trained model.

[0088] Model determination module 162 inputs 200 new learning data items into each candidate model, determines whether they are good or bad, and calculates the accuracy of the judgment based on the label information of each new learning data item. Furthermore, model determination module 162 selects the top candidate model among those with an accuracy exceeding a first threshold value as the pretrained model used to generate the new learning model. While the present invention uses the first threshold value set to 80% as an example, the technical concept of the present invention is not limited to this.

[0089] Through the above process, among the existing learning models applied to various product production lines 300, the learning model most similar to the new learning model can be extracted as a backup model, and the extracted backup model can be used to judge the quality of the learning data. Among the backup models with a judgment accuracy above the first benchmark value, the highest backup model can be recommended as a pre-trained model, thereby improving the judgment accuracy of the new learning model generated later.

[0090] In addition, instead of using all the collected learning data, the pre-training model can be determined using only a certain amount of learning data, which can significantly reduce the time required.

[0091] Below, refer to Figures 6 to 11 , another example of the new learning model generation process of the AI-based new learning model generation system 100 according to an embodiment of the present invention is described in detail. Figures 6 to 11In the embodiment shown in FIG, a pre-training model is determined by the pre-training module determination module 160, and the determined pre-training model is applied to the additional learning data set determination process and the new learning model generation process.

[0092] In the pair Figure 6 When describing the embodiment shown in FIG. Figure 3 The detailed description of the structure corresponding to the embodiment shown in the figure is omitted. Figure 3 The description of the embodiment shown in FIG.

[0093] Reference Figure 6 , collect new learning data and register (S60), when receiving the judgment type information (S61), execute the process of determining the pre-training model (S70). Here, the steps (S60) and (S61) correspond to Figure 3 The detailed description of the steps (S30) and (S31) shown in the figure is omitted.

[0094] Figures 7 to 9 This is a diagram illustrating an example of a process of determining a pre-trained model in the AI-based new learning model generation system 100 according to an embodiment of the present invention.

[0095] Reference Figure 7 Based on the judgment type information input in step (S61), the candidate model extraction module 161 refers to the model information of the existing learning model stored in the learning model memory to extract a candidate model (S71). Here, the method for extracting a predetermined number of candidate models from the existing learning data by the candidate model extraction module 161 is as described above.

[0096] When the candidate models are extracted, the model decision module 162 performs a judgment accuracy evaluation process on each candidate model ( S72 ). Figure 8 This is a diagram for explaining the first (1st) judgment accuracy evaluation process executed by the model determination module 162.

[0097] The model determination module 162 extracts a predetermined number (i) of new learning data from the new learning data stored in the data storage 130 ( S721 ). The model determination module 162 then causes each candidate model to determine whether it is good or bad based on the extracted i learning data ( S722 ). The model determination module 162 calculates the accuracy of each candidate model based on the determination results and the label information of each new learning data ( S723 ).

[0098] Re-reference Figure 7 , determine whether the judgment accuracy of each candidate model is above the first reference value (S73), and determine the highest candidate model among the candidate models whose judgment accuracy is above the first reference value as the pre-trained model (S74).

[0099] On the other hand, when the judgment accuracy of all candidate models is less than or equal to the first reference value in step ( S73 ), the model determination module 162 executes a second (2nd) judgment accuracy evaluation process ( S75 ). Figure 5 This is a diagram for explaining the second judgment accuracy evaluation process performed by the model determination module 162.

[0100] Reference Figure 9 The model determination module 162 extracts the top n candidate models as re-inspection candidate models based on the judgment accuracy calculated in step (S72) (S751). For example, if ten candidate models are extracted and all ten candidate models have a judgment accuracy of less than 80%, the top five candidate models are extracted as re-inspection candidate models.

[0101] When extracting the top n candidate re-inspection models, it is determined whether any of the n candidate re-inspection models have a judgment accuracy below a preset lower limit (S752). The candidate models with a judgment accuracy below the lower limit are removed from the candidate re-inspection models (S753). For example, if the lower limit is set to 60%, and two of the five candidate re-inspection models have a judgment accuracy below 60%, only three candidate models are extracted as candidate re-inspection models.

[0102] Therefore, the candidate models with a judgment accuracy less than the lower limit value will be excluded in the second judgment accuracy evaluation process, thereby reducing the time required in the second judgment accuracy evaluation process, and the candidate models with a low possibility of being pre-trained models will be excluded in advance, thereby improving the efficiency of the model decision process.

[0103] When the re-inspection candidate models are extracted through the above process, the model determination module 162 extracts a preset number (j) of new learning data from the data storage 130 (S754). Then, the model determination module 162 learns each re-inspection candidate model using the extracted j new learning data (S755).

[0104] When the re-inspection candidate model completes learning, the model determination module 162 extracts a preset number (k) of new learning data from the data storage 130 (S756), and enables each re-inspection candidate model that has completed learning to judge the quality of the extracted k new learning data (S757), and calculates the judgment accuracy of each re-inspection candidate model based on the judgment result and the label information of each new learning data (S758).

[0105] Through the above process, in the first judgment accuracy evaluation process (S72), the candidate model below the first benchmark value is made to learn a part of the new learning data actually used to generate the new learning model, and then the judgment accuracy is re-evaluated, thereby increasing the possibility of extracting the existing learning model suitable for the new learning data as a pre-trained model.

[0106] Re-reference Figure 7 When the judgment accuracy of the re-inspection candidate models is calculated through the second judgment accuracy evaluation process, it is determined whether the judgment accuracy of each re-inspection candidate model is greater than or equal to a second reference value (S76). Among the re-inspection candidate models with a judgment accuracy greater than or equal to the second reference value, the highest ranking re-inspection candidate model is determined as the pre-trained model (S77). Conversely, if no re-inspection candidate model has a judgment accuracy greater than or equal to the second reference value, the recommendation is determined to have failed (S78).

[0107] Here, the second reference value is set relatively higher than the first reference value. For example, as described above, if the first reference value is set to 80%, the second reference value can be set to 90%. This means that the re-inspection candidate model, which receives the second reference value and is applied, learns using the new learning data used when actually generating the new learning model, and the accuracy of the judgment based on the new learning data is improved.

[0108] Through the process as described above, the pre-trained model is determined at one time from the candidate models with high judgment accuracy. When the judgment accuracy is lower than the first benchmark value, a secondary judgment process is added to calculate the judgment accuracy after the top n candidate models learn new learning data, so that the existing learning model closest to the new learning model is determined as the pre-trained model.

[0109] Figure 10 and Figure 11 This is a diagram illustrating another example of the process of determining a pre-trained model in the AI-based new learning model generation system 100 according to an embodiment of the present invention. Figure 10 and Figure 11 The embodiment shown in FIG. Figures 7 to 9 Regarding the modified examples of the embodiment shown in the figure, descriptions of corresponding structures are omitted.

[0110] When the judgment type information is received (refer to Figure 6 In S61), the candidate model extraction module 161 extracts a candidate model based on the judgment type information by referring to the model information of the existing learning model stored in the learning model memory (S12). The candidate model extraction method performed by the candidate model extraction module 161 is as described above.

[0111] When the candidate models are extracted, the model decision module 162 performs a judgment accuracy evaluation process (1st) (S13) on each candidate model. Figure 8 The embodiment shown in the figure is corresponding and its description is omitted.

[0112] After the accuracy of each candidate model is calculated through the one-time accuracy evaluation process, it is determined whether the accuracy of each candidate model is greater than a first reference value (S14). If a similar candidate model exists whose accuracy differs from the highest candidate model greater than the first reference value by a value within a predetermined similarity range, a comparison process is performed with the similar candidate models (S91).

[0113] Reference Figure 11 More specifically, when there is a similar candidate model with a difference in judgment accuracy between the top candidate model and the similar candidate model within a similar range (S911), feature graphs of the top candidate model and the similar candidate model are extracted (S912). Furthermore, feature point distributions of the feature graphs are extracted (S913).

[0114] In order to calculate the judgment accuracy of the top candidate model and similar candidate models, a feature map is generated for each learning data in the process of the candidate model judging the learning data based on deep learning. In addition, the probability distribution, that is, the feature point distribution, can be extracted from the feature map. Here, the feature map and feature point distribution are extracted from the learning data for normal learning data and bad learning data. Examples of feature maps and feature point distributions for normal learning data and bad learning data are as follows: Figure 5 shown.

[0115] like Figure 8 As shown in (c), there is a difference in the distribution of feature points between the bad learning data and the normal learning data. When the difference in the distribution of the two feature points becomes larger, it can be evaluated that the learning model extracts the feature map from the original image in a distinguishable manner, and the result can improve the judgment accuracy of the final learning model.

[0116] If the similarity range for accuracy is set to 2%, and the accuracy of the top candidate model is calculated to be 85%, and the accuracy of the next candidate model is calculated to be 83%, then that candidate model is extracted as a similar candidate model. However, the accuracy varies depending on the sampling process and amount of new learning data extracted for accuracy calculation. Therefore, it may be impossible to determine the optimal learning model, either the top candidate model or the similar candidate model within the similarity range. Therefore, the optimal pre-trained model is determined using feature point distribution.

[0117] In the present invention, the case of calculating the difference value of the feature point distribution between the bad learning data and the normal learning data in the top learning model and the similar candidate model (S914), and determining any one with the larger difference as the pre-trained model is taken as an example. At this time, any preset number of bad learning data and normal learning data are extracted from the multiple new learning data used in the judgment process of the top candidate model and the similar candidate model, and the difference value of the feature point distribution between the bad learning data and the normal learning data is calculated (S914), and then the average value is calculated (S915). In addition, the average value of the top candidate model is compared with the average value of the similar candidate model (S916). When the average value of the top candidate model is larger, the top candidate model is selected (S917). When the average value of the similar candidate model is larger, the similar candidate model is selected (S918), thereby determining it as the pre-trained model ( Figure 10 S15).

[0118] Here, an algorithm capable of calculating the difference in probability distribution is applied to the difference value of the feature point distribution. In the present invention, a case where the difference value is calculated using the KL-Divergence algorithm is taken as an example.

[0119] Re-reference Figure 10 , the model decision module 162 performs a second (2nd) judgment accuracy evaluation process (S16) when the judgment accuracy of all candidate models in step (S14) is below the first reference value. Figure 9 The detailed description is omitted for the sake of simplicity.

[0120] After the accuracy of each re-inspection candidate model is calculated through the second accuracy evaluation process, it is determined whether the accuracy of each re-inspection candidate model is greater than or equal to a second reference value (S17). If a similar re-inspection candidate model exists whose accuracy differs from the highest re-inspection candidate model greater than or equal to the second reference value by a predetermined similarity range, a comparison process is performed with the similar re-inspection candidate models (S92).

[0121] Here, the comparison process is similar to re-examining the supplementary model. Figure 11 The process of comparing similar candidate models shown in the figure corresponds to this process. Specifically, if there is a similar candidate model whose judgment accuracy differs from that of the top candidate model by a similar range (see S911), feature maps of the top candidate model and the similar candidate model are extracted (see S912). Furthermore, the distribution of feature points in the feature maps is extracted (see S913).

[0122] At this time, as described above, the difference in feature point distribution between the defective learning data and the normal learning data is calculated (see S914), and then the average is calculated (see S915). Furthermore, the average of the top re-inspection candidate model and the average of the similar re-inspection candidate model are compared (see S916). If the average of the top re-inspection candidate model is greater, the top re-inspection candidate model is selected (see S917). If the average of the similar re-inspection candidate model is greater, the similar re-inspection candidate model is selected (see S918) and selected as the pre-trained model (S18).

[0123] As mentioned above, if there is a top candidate model or a top recheck candidate model and there is also a similar candidate model or a similar recheck candidate model with a judgment accuracy within a similar range, the final pre-trained model is determined using the feature point distribution of the feature graph, thereby determining the learning model that is closer to the new learning model as the pre-trained model.

[0124] Re-reference Figure 6 When the pre-training model is determined through the above process, the process of determining the additional learning data set is performed (S62). Here, the process of determining the additional learning data set is the same as Figure 4 The same as the embodiment shown in FIG. Figure 4 In the embodiment shown in FIG, the feature map of the supplementary learning data set and the new learning data is generated. Figure 6 That is, the pre-trained model extracted as the most suitable for generating a new learning model is applied to the generation of feature maps in the process of determining the learning data set to be added in order to generate the new learning model, thereby determining the learning data set that is most similar to the new learning data.

[0125] And, when deciding to add additional learning data sets, use Figure 6 A new learning model is generated based on the pre-training model determined in step (S70). As described above, the learning data constituting the new learning data and the additional learning data set are learned through the pre-training model, thereby solving the problem of decreased judgment accuracy that occurs when learning with a small amount of new learning data.

[0126] While the embodiments of the present invention have been described above with reference to the accompanying drawings, the present invention is not limited to the embodiments described above and can be manufactured in various different forms. Those skilled in the art can implement the present invention in various specific forms without changing the technical concept or essential features of the present invention. Therefore, the embodiments described above are illustrative in all respects and the present invention is not limited thereto.

[0127] (Explanation of Symbols)

[0128] 100: New AI-based learning model generation system

[0129] 110: Post-complement set extraction module 120: Supplement set determination module

[0130] 130: Data storage 140: New model generation module

[0131] 150: Main processor 160: Pre-trained model decision module

[0132] 161: Supplementary model extraction module 162: Model determination module

[0133] 170: Training model storage 300: Product production line

[0134] Industrial Application Possibilities

[0135] The present invention is applied to the field of visual inspection of results of various production processes in a product production line.

Claims

1. A new AI-based learning model generation system for visual inspection on product production lines, characterized by: A data storage device storing a plurality of different types of learning data sets and a plurality of new learning data, wherein each of the learning data sets is applied to learning existing learning models that are different from each other and have been generated for visual inspection on a product production line, each of the learning data sets includes a plurality of images, and the plurality of new learning data are collected to generate a new learning model. The new learning model generation system includes: a complement set extraction module, which, upon receiving the judgment type information for generating a new learning model, extracts at least two or more complement data sets from the plurality of learning data sets based on the judgment type information; an additional set determination module that calculates a similarity between the new learning images included in the new learning data and the supplementary learning images included in the supplementary dataset, and determines a supplementary dataset having the similarity greater than or equal to a registered reference value as the additional learning dataset; and A new model generation module applies the additional learning data set determined by the additional set determination module and the new learning data to the pre-trained model to generate the new learning model.

2. The AI-based new learning model generation system for visual inspection on product production lines according to claim 1 is characterized in that The judgment type information includes at least one of defect type information on the type of defect, product type information on the type of the inspection target product, and component type information on the type of the inspection target component. Each of the learning data sets stored in the data storage includes model information having at least one of the defect type information, the product type information, and the component type information. The complement set extraction module extracts the complement data set by referring to the model information.

3. The AI-based new learning model generation system for visual inspection on product production lines according to claim 2, characterized in that: The supplementary set extraction module extracts a preset number of learning data sets as the supplementary data sets according to a priority order set in the order of the defect type information, the component type information, and the product type information.

4. The AI-based new learning model generation system for visual inspection on a product production line according to claim 1, characterized in that: The additional set determination module generates a new image feature map from a preset number of new learning images in the new learning data, and calculates the new feature point distribution of each new image feature map. Generate a supplementary image feature map from a preset number of supplementary learning images in the supplementary data set, calculate the distribution of supplementary feature points of each of the supplementary image feature maps, The difference between the new feature point distribution and the candidate feature point distribution is calculated as the similarity.

5. The AI-based new learning model generation system for visual inspection on a product production line according to claim 4, characterized in that: The difference between the new feature point distribution and the supplementary feature point distribution is calculated using the KL divergence algorithm.

6. The AI-based new learning model generation system for visual inspection on a product production line according to claim 4, characterized in that: The new image feature map and the supplementary image feature map are generated by learning a preset number of the new learning images and a preset number of the supplementary learning images through the pre-training model respectively.

7. The AI-based new learning model generation system for visual inspection on a product production line according to claim 6, characterized in that: It includes: A learning model memory storing a plurality of the above-mentioned existing learning models; a candidate model extraction module, which extracts at least two candidate models from the plurality of existing learning models based on the judgment type information; and A model determination module inputs a preset number of new learning data from the new learning data stored in the data storage device into each of the candidate models to judge whether they are good or bad, and determines the highest candidate model among the multiple candidate models whose judgment accuracy is above a preset first benchmark value as the pre-trained model.

8. The AI-based new learning model generation system for visual inspection on a product production line according to claim 7, characterized in that: If there is a similar candidate model among the plurality of candidate models whose difference in judgment accuracy from the highest candidate model is within a preset similarity range, the model determination module extracts a distribution of feature points of a feature graph generated for the new learning data during the judgment process between the highest candidate model and the similar candidate model. The model determination module determines, as the pre-trained model, whichever of the top candidate model and the similar candidate model has the largest difference in the distribution of the feature points between the bad learning data and the normal learning data.

9. The AI-based new learning model generation system for visual inspection on a product production line according to claim 8, characterized in that: The model determination module calculates the average value of the difference in the distribution of the feature points between a preset number of bad learning data and normal learning data arbitrarily selected from a plurality of learning data used in the judgment process of the top candidate model and the similar candidate model. The model having the larger average value among the top candidate model and the similar candidate model is determined as the pre-trained model.

10. The AI-based new learning model generation system for visual inspection on a product production line according to claim 8, characterized in that: The difference in the distribution of the above feature points is calculated using the KL divergence algorithm.

11. The AI-based new learning model generation system for visual inspection on a product production line according to claim 7, characterized in that: When the judgment accuracy of the plurality of candidate models is less than the first reference value, the model determination module extracts the top n candidate models as re-examination candidate models based on the judgment accuracy. The model determination module causes each of the re-inspection candidate models to perform learning using a preset number of learning data among the plurality of learning data. The model determination module inputs a preset number of learning data from the plurality of learning data into the learned re-inspection supplementary model to determine whether the data is good or bad. The highest-ranked re-inspection candidate model whose judgment accuracy among the re-inspection candidate models is greater than a preset second reference value is determined as the pre-trained model.

12. The AI-based new learning model generation system for visual inspection on a product production line according to claim 11, characterized in that: When extracting the re-inspection candidate models, the model decision module removes the candidate models whose judgment accuracy is below a preset lower limit from the re-inspection candidate models.

13. The AI-based new learning model generation system for visual inspection on a product production line according to claim 11, characterized in that: If there is a similar re-inspection candidate model among the plurality of re-inspection candidate models whose difference in judgment accuracy from the highest re-inspection candidate model is within a preset similarity range, the model determination module extracts the feature point distribution of the feature graph generated for each of the learning data during the judgment process of the highest re-inspection candidate model and the similar re-inspection candidate model. The model determination module determines, as the pre-trained model, either the top re-inspection candidate model or the similar re-inspection candidate model, whichever model has a larger difference in the distribution of the feature points between the bad learning data and the normal learning data.

14. The AI-based new learning model generation system for visual inspection on a product production line according to claim 13, characterized in that: The model determination module calculates the average value of the difference in the distribution of the feature points between a preset number of bad learning data and normal learning data arbitrarily selected from a plurality of learning data applied to the judgment process of the top re-inspection candidate model and the similar re-inspection candidate model, The model determination module determines, as the pre-trained model, the model having the larger average value between the top re-inspection candidate model and the similar re-inspection candidate model.

15. The AI-based new learning model generation system for visual inspection on a product production line according to claim 13, characterized in that: The difference in the distribution of the above-mentioned feature points is calculated using the KL divergence algorithm.

Images