AI-based pre-trained model decision system and ai-based visual inspection management system for product production line using the same

Abstract

The present application relates to an AI-based pre-training model decision system that extracts a post-filling model based on judgment type information among a plurality of learning models when the judgment type information is input, judges new learning data by the post-filling model. The highest post-filling model of which a preset 1st reference value or more is decided as a pre-training model for generating a new learning model is decided based on the judgment accuracy of the post-filling model, and the judgment accuracy of the new learning model is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to an AI-based pre-training model decision system and an AI-based vision inspection management system for a product production line using the same, and more particularly, to an AI-based pre-training model decision system and an AI-based vision inspection management system for a product production line using the same, which generates a new learning model using a registered learning model as a pre-training model. BACKGROUND

[0002] Deep learning technology, which is one of AI (Artificial intelligence) technologies, refers to a machine learning method based on an artificial neural network that mimics a biological neuron of a human to enable a machine to learn. In recent years, deep learning technology has attracted much attention as it has contributed to the development of image recognition, speech recognition, and natural language processing.

[0003] Such deep learning technology has also been applied to vision inspection on a product production line in recent years. As an example, a "quality inspection method and system based on machine vision using deep learning in a manufacturing process" (hereinafter, referred to as 'prior art 1') is disclosed in Korean Patent Laid-Open Publication No. 10-2019-0063839.

[0004] In the technology disclosed in the above Korean Patent Laid-Open Publication, a learning product image is generated, the generated learning product image is learned using a classification machine that distinguishes between good products and defective products, and the product is judged to be a good product or a defective product using the learned classification machine.

[0005] In the deep learning technology disclosed in the above Korean Patent Laid-Open Publication, image data of good products and defective products and tag information recording whether the image is a defective product image or a good product image are input to a learning model such as a classification machine to learn, thereby generating a new learning model for performing good and defective inspection.

[0006] In general, various types of defective inspection are performed on a product production line. A product production line for producing one product includes a plurality of processes such as an injection molding process, a sheet molding process, a sub-assembly process, a final assembly process, etc., and a result of each process, for example, a part manufactured by injection molding or sheet molding, a sub-part produced by a component assembly process, or a product produced by a final assembly process is subjected to vision inspection.

[0007] Figure 1 is a diagram schematically showing an example of a general product production line.

[0008] Referring to Figure 1 , the product production line includes an injection molding process, a sheet molding process, a sub-assembly process, and a final assembly process. The injection molding process and the sheet molding process manufacture components required in product production in an injection molding or sheet molding manner, and a plurality of injection molding processes or sheet molding processes are included in one product production line.

[0009] The components manufactured through the injection molding process or the sheet molding process include a dent inspection, a crack inspection, and a scratch inspection process for performing various types of defects, such as a dent, a crack, a scratch, and the like.

[0010] The sub-assembly process is a process of producing sub-components as components for assembly, and as shown in Figure 1 , the sub-components also undergo the dent inspection, the crack inspection, and the scratch inspection process, and additionally include an assembly defect inspection process such as a component omission or miss alignment.

[0011] The final assembly process is a process of producing products by assembling the sub-components, and performs an appearance inspection process such as foreign matter attachment, assembly defects, and the like. In the final assembly process, the dent inspection, the crack inspection, and the scratch inspection process are also performed.

[0012] Depending on the type of defects such as assembly defects and foreign matter attachment, additional visual inspection is required.

[0013] However, the resulting product differs for each process, and thus the form of the image used as learning data differs, and even for the same type of defect, the form of the defect differs depending on the resulting product, and the resulting product itself as a background thereof also differs, and thus the form of the resulting image differs, and in theory, each learning model needs to be generated depending on the resulting product of each process and the type of defect to be applied.

[0014] In addition, in the case of a manufacturing company that produces various product groups, the product production line also differs depending on each product, and even if the same injection molding process exists, the resulting product of each product also differs, and thus a separate learning model needs to be generated for each product. Similarly, when the product production line is changed for new product production, a new learning model for each process and the type of defect that conforms to the new product production line needs to be generated.

[0015] Here, the judgment accuracy of the new learning model is determined depending on each element, and the amount of learning data, the selection of the hyperparameters set at the beginning of learning, the selection of the learning model, and the like are important elements that determine the judgment accuracy.

[0016] However, in the case of a new learning model applied to a new product line, the amount of learning data, particularly the amount of defective data, can only be relatively small, and which model is selected as a pre-training model becomes an important factor in improving the accuracy of the judgment. SUMMARY

[0017] PROBLEM TO BE SOLVED BY THE INVENTION

[0018] An object of the present application is to provide an AI-based pre-training model decision system capable of using the best pre-training model when a registered learning model is used as a pre-training model to generate a new learning model, and an AI-based visual inspection management system for a product line using the system.

[0019] An object of the present application is to determine the best pre-training model to improve the accuracy of the judgment of a new learning model when a registered learning model is used as a pre-training model to generate a new learning model.

[0020] An object of the present application is to reduce the time required to determine a pre-training model, improve the efficiency of the determination process, and recommend the best pre-training model.

[0021] An object of the present application is to provide an AI-based visual inspection system that can quickly generate a new learning model and apply it when a new product line is established.

[0022] MEANS FOR SOLVING THE PROBLEM

[0023] In the AI-based pre-training model decision system of the present application, when receiving judgment type information for generating a new learning model, the post-model extraction module extracts at least two or more post models from among a plurality of learning models stored in the learning model storage based on the judgment type information, and the model decision module inputs a predetermined amount of learning data from among the learning data stored in the data storage to the extracted post models to determine good and bad, and determines the highest post model having a judgment accuracy of 1st reference value or more as a pre-training model for generating a new learning model.

[0024] When there is a similar post model having a difference in judgment accuracy from the above highest post model within a similar range, the above model decision module extracts the feature point distribution of the feature map generated for the learning data in the judgment process of the above highest post model and the above similar post model, and determines any one of the models having a large difference in the above feature point distribution between the defective learning data and the normal learning data as the above pre-training model.

[0025] The model decision module calculates an average of differences in the feature point distribution between the predetermined number of defective learning data and normal learning data, and decides either of the models having a larger average value as the pre-training model among the top model and the similar model.

[0026] In the case where the judgment accuracy is lower than the first reference value, the model decision module extracts n top models as recheck models based on the average judgment accuracy, causes the recheck models to learn the predetermined number of learning data, judges good and defective based on the predetermined number of learning data input to the learned recheck models, and decides the top recheck model having a judgment accuracy of a second reference value or more as the pre-training model.

[0027] The model decision module removes the recheck models having a judgment accuracy lower than the lower limit value from the recheck models.

[0028] When there is a similar recheck model having a difference in judgment accuracy from the top recheck model within a similar range, the model decision module extracts feature point distributions of feature maps generated in the judgment processes of the top recheck model and the similar recheck model, and decides either of the models having a larger difference in the feature point distribution between defective learning data and normal learning data as the pre-training model.

[0029] The difference in the feature point distribution is calculated by a KL divergence algorithm.

[0030] The judgment type information includes at least one of defective type information about a type of defective, product type information about a type of a product to be inspected, and component type information about a type of a component to be inspected, the learning model includes model information including at least one of the defective type information, the product type information, and the component type information, and the recheck model extraction module extracts the recheck model with reference to the model information.

[0031] The recheck model extraction module extracts the recheck models in a priority order set in the order of the defective type information, the component type information, and the product type information.

[0032] The learning data includes learning images and label information recording information about defects or normality of the learning images, and the AI-based pre-trained model decision system further includes a label conversion module that converts the label information of the learning data into a registered format, and the model decision module decides the pre-trained model using the learning data whose label information is converted by the label conversion module.

[0033] The AI-based visual inspection management system according to the present application includes a plurality of visual AI clients respectively provided in a plurality of product production lines geographically separated from each other to inspect defects using AI-based learning models, and a visual AI cloud that generates a new learning model registered to the visual AI clients.

[0034] The visual AI cloud includes a pre-trained model decision system that decides the pre-trained model using a plurality of learning data and a model decision module, and a new model generation system that generates the new learning model by learning the pre-trained model decided by the model decision module.

[0035] The visual AI cloud includes a cloud control module that registers the new learning model to at least one of the plurality of visual AI clients.

[0036] The plurality of learning data are transmitted from the plurality of visual AI clients and stored, and the pre-trained model decision system decides the pre-trained model according to a model generation request from the visual AI cloud in which the new learning model is registered.

[0037] Effects of the Invention

[0038] The AI-based pre-trained model decision system according to the present application and the AI-based visual inspection management system for product production lines using the same have one or more of the following effects.

[0039] First, the present application decides the best pre-trained model when generating a new learning model using a registered learning model as a pre-trained model, thereby improving the judgment accuracy of the new learning model.

[0040] Second, a post-filling model for deciding a pre-trained model is extracted using judgment type information including one of defect type information, product type information, and component type information, and a learning model having similarity with the new learning model is extracted as the post-filling model.

[0041] Third, the post-filling model judges only a part of the new learning data, and the pre-trained model is decided based on the judgment accuracy, thereby being able to recommend the best pre-trained model while reducing the time required for deciding the pre-trained model.

[0042] Fourth, when the accuracy of all subsequent models is below the first baseline value, the upper n subsequent models are extracted as re-examined subsequent models, so that the re-examined subsequent models learn a portion of the learning data, and then the pre-trained model is determined. This can improve the efficiency of the decision-making process of the pre-trained model while recommending the best pre-trained model.

[0043] Fifth, when the accuracy is within a similar range, an additional evaluation process using feature point distribution can be used to recommend the best pre-trained model.

[0044] Sixth, when applying the pre-trained model decision system to an AI-based visual inspection management system, learning data can be collected from multiple visual AI clients, and new learning models can be quickly applied when setting up new product production lines. Attached Figure Description

[0045] Figure 1 It is a diagram that roughly illustrates an example of a typical product production line.

[0046] Figure 2 This is a diagram illustrating how an AI-based pre-trained model determines the structure of a system according to an embodiment of the present invention.

[0047] Figures 3 to 5 This is a diagram illustrating one embodiment of the process of determining a pre-trained model through a pre-trained model determination system according to an embodiment of the present invention.

[0048] Figure 6 and Figure 7 This is a diagram illustrating another embodiment of the process of determining a pre-trained model through a pre-trained model determination system according to an embodiment of the present invention.

[0049] Figure 8 This is a diagram illustrating an example of the feature map and feature point distribution generated during the judgment or learning process of the learning model in the pre-learning model determination system of an embodiment of the present invention.

[0050] Figure 9 and Figure 10 This is a diagram illustrating the structure of an AI-based visual management system according to an embodiment of the present invention. Detailed Implementation

[0051] The AI-based pre-training model decision system of the present application is characterized by including: a learning model storage storing a plurality of learning models for performing visual inspection on a product production line; a data storage storing a plurality of learning data collected for generating a new learning model; a post-filling model extraction module extracting at least two or more post-filling models from among the plurality of learning models based on judgment type information for generating the new learning model, upon receipt of the judgment type information; and a model decision module inputting a predetermined amount of learning data among the learning data stored in the data storage to each of the post-filling models to judge good and bad, and deciding a top post-filling model having a judgment accuracy of the plurality of post-filling models of a predetermined first reference value or more as a pre-training model for generating the new learning model.

[0052] The advantages and features of the present application, the method of achieving the advantages and features, can be more clearly understood by referring to the following embodiments described in detail with reference to the accompanying drawings. However, the present application is not limited to the embodiments disclosed below, and can be embodied in various forms different from each other, and the embodiments are provided to completely disclose the present application and clearly inform the scope of the invention to those skilled in the art, and the present application should be defined according to the scope of the claims. Throughout the specification, the same reference numerals denote the same constituent elements.

[0053] Hereinafter, embodiments of the present application will be specifically described with reference to the accompanying drawings.

[0054] Figure 2 is a diagram showing the structure of an AI-based pre-training model decision system 100 according to an embodiment of the present application. Referring to Figure 2 , the AI-based pre-training model decision system 100 according to the present application includes a learning model storage 110, a data storage 140, a post-filling model extraction module 120, a model decision module 130, and a main processor 150 controlling the same. Here, the main processor 150 includes a hardware structure such as a CPU, a RAM, and a software structure such as an operating system, for performing the operation of each structure of the pre-training model decision system 100 according to the present application.

[0055] The learning model storage 110 stores a plurality of learning models for performing visual inspection on a product production line 300. Referring to Figure 1 , the product production line 300 includes a product production line 310, a camera 320, and a product 330. The product production line 310 is a line for producing a product, and the camera 320 is a camera for performing visual inspection on the product 330 produced on the product production line 310. Figure 1As described above, the product line 300 includes various visual inspection processes such as a dent inspection, a crack inspection, a scratch inspection process for performing an inspection of a result of each process, that is, a component or a product, and the learning model storage 110 stores learning models applied to the visual inspection processes of the product line 300, that is, a dent inspection learning model, a crack inspection learning model, a scratch inspection learning model, and the like. In the present application, a case where the learning model is a learning model based on deep learning is exemplified.

[0056] The data storage 140 stores a plurality of learning data collected for generating a new learning model. Here, the plurality of learning data includes learning images that are images of a result of a certain process of the new product line 300, that is, a component or a product, and tag information in which information on defects or normality of each learning image is recorded.

[0057] In a case where the judgment type information for generating a new learning model is input through the input unit 14, the post-filling model extraction module 120 extracts at least two or more learning models stored in the learning model storage 110 as post-filling models based on the judgment type information.

[0058] The model determination module 130 inputs a predetermined number of learning data stored in the data storage 140 to each post-filling model and determines whether it is good or bad. Also, the model determination module 130 calculates a judgment accuracy of each post-filling model from the judgment results of the plurality of post-filling models, and determines a top post-filling model having a judgment accuracy of a predetermined first reference value or more as a pre-training model for generating a new learning model.

[0059] More specifically, a plurality of learning data is collected for generating a new learning model. As an example, in the generation of a new learning model applied to a dent inspection after an injection molding process applied to a manufacturing process of a certain product, an image of a component in which a dent occurs and an image of a normal component are respectively collected as learning images, and tag information in which a tag is recorded as NG or OK in each learning image is stored as learning data.

[0060] At this time, as the judgment type information, for example, when defect type information on a type of a defect is input, the post-filling model extraction module 120 searches for a learning model generated to inspect the same type of defect as the defect type information among the learning models stored in the learning model storage 110 and extracts it as a post-filling model.

[0061] In the present application, a case where the judgment type information includes at least one of information on a type of a defect described above, product type information on a type of a product to be inspected, and component type information on a type of a component to be inspected is exemplified.

[0062] The defect type information indicates a form of a defect such as a dent, a crack, a scratch, a miss alignment, a missing component, a foreign matter, etc. The product type information indicates a type of a product to be inspected such as a type of an object product such as a smartphone, a refrigerator, a TV, etc. The component type information indicates a type of a component to be inspected such as a case, a frame, a PCB, a door, etc.

[0063] Here, the learning model stored in the learning model storage includes model information having at least one of the defect type information, the product type information, and the component type information, in correspondence with the judgment type information, and the post-filling model extraction module 120 extracts, with reference to the model information, a learning model corresponding to the judgment type information input through the input unit 14 as the post-filling model.

[0064] At this time, the model extraction module 120 extracts the post-filling models of a predetermined number in a priority order set in the order of the defect type information, the component type information, and the product type information. For example, when the judgment type information including a dent as the defect type information, a smartphone as the product type information, and a case as the component type information is received, the model extraction module 120 extracts a learning model corresponding to the dent at one time, and when the extracted learning model is more than a set number, extracts a learning model corresponding to the smartphone among them at two times.

[0065] Through the above-described process, among the plurality of learning models stored in the learning model storage 110, a learning model checking a defect of a form most similar to a new learning model can be extracted as the post-filling model, and a possibility of improving a judgment accuracy of the new learning model generated using the finally selected pre-trained model in a subsequent process can be increased.

[0066] On the other hand, as described above, the model decision module 130 extracts a predetermined number of learning data from the learning data stored in the data storage 140. For example, in a case where 1000 pieces of learning images are stored as the learning data, if all of the 1000 pieces of learning data are used to calculate the judgment accuracy, a lot of time is required to decide the pre-learning model. Therefore, a predetermined number of learning images are extracted, for example, 20% or 200 pieces of learning images are extracted to be used to calculate the judgment accuracy, and thus a time required to decide the pre-learning model can be reduced.

[0067] Here, the model determination module 130 inputs 200 training data images into each supplementary model to determine whether they are good or bad, and calculates the judgment accuracy based on the label information of each training data image. Furthermore, among the supplementary models whose judgment accuracy is above the first benchmark value, the model determination module 130 determines the top-ranked supplementary model as the pre-trained model for generating the new learning model. This invention uses the case where the first benchmark value is set to 80% as an example, but the technical concept of this invention is not limited to this.

[0068] Through the process described above, in the learning model applied to various product production lines 300, the learning model most similar to the new learning model can be extracted as a supplementary model. The extracted supplementary model is used to judge the quality of the learning data. Thus, among the supplementary models with an accuracy of more than the first benchmark value, the top-ranked supplementary model is recommended as the pre-trained model, thereby improving the judgment accuracy of the newly generated learning model.

[0069] In addition, by using only a certain amount of training data instead of all the collected training data to determine the pre-trained model, the required time can be significantly reduced.

[0070] Below, refer to Figures 3 to 5 An embodiment of the process by which a pre-trained model is determined by a pre-trained model determination system 100 according to an embodiment of the present invention will be described in detail.

[0071] First, when new learning data for generating new learning models is collected, it is stored in data storage 140 (S30). Here, multiple learning models are stored in learning model storage 110, and as described above, each learning model includes model information.

[0072] Then, upon receiving the judgment type information for generating a new learning model (S31), the supplementary model extraction module 120 extracts the supplementary model based on the judgment type information and with reference to the model information of the learning model stored in the learning model memory 110 (S32). The method for extracting the supplementary model by the supplementary model extraction module 120 is as described above.

[0073] When a complement model is extracted, the model determination module 130 performs an accuracy evaluation process for each complement model (S33). Figure 4 This is a diagram illustrating the accuracy evaluation process performed by the model decision module 130.

[0074] The model decision module 130 extracts a predetermined number (i) of learning data from the learning data stored in the data storage 140 (S331). Then, the post-processing decision module 130 causes each post-processing model to judge the extracted i learning data as good or bad (S332), and calculates the judgment accuracy of each post-processing model based on the judgment result and the label information of each learning data (S333).

[0075] Referring back to Figure 3 , it is determined whether the judgment accuracy of each post-processing model is the first reference value or more (S34), and the highest post-processing model is decided as the pre-training model in the post-processing model whose judgment accuracy is the first reference value or more (S35).

[0076] On the other hand, in the case where the judgment accuracy of all post-processing models in the step (S34) is less than the first reference value, the model decision module 130 performs a secondary judgment accuracy evaluation process (S36). Figure 5 FIG. 7 is a diagram illustrating the secondary judgment accuracy evaluation process performed by the model decision module 130.

[0077] Referring back to Figure 5 , the model decision module 130 extracts the upper n post-processing models as re-examination post-processing models based on the judgment accuracy calculated in the step (S33) (S361). For example, in the state where 10 post-processing models are extracted, when the 10 post-processing models all indicate a judgment accuracy of 80% or less, the upper 5 post-processing models are extracted as re-examination post-processing models.

[0078] Here, when the upper n re-examination post-processing models are extracted, it is determined whether there is a post-processing model whose judgment accuracy is less than a predetermined lower limit value among the n re-examination post-processing models (S362), and the post-processing model having a judgment accuracy less than the lower limit value is removed from the re-examination post-processing models (S363). For example, when it is assumed that the lower limit value is set to 60%, if the judgment accuracy of 2 of the 5 re-examination post-processing models is 60% or less, only 3 post-processing models are extracted as re-examination post-processing models.

[0079] Thus, the post-processing model whose judgment accuracy is less than the lower limit value is excluded in the secondary judgment accuracy evaluation process, thereby reducing the time required in the secondary judgment accuracy evaluation process, and the efficiency of the model decision process is improved by previously excluding the post-processing model having a low possibility of being decided as the pre-training model.

[0080] When the re-examination post-processing models are extracted through the above process, the model decision module 130 extracts a predetermined number (j) of learning data from the data storage 140 (S364). Then, the model decision module 130 causes each re-examination post-processing model to learn the extracted j learning data (S365).

[0081] When the learning of the re-checked supplementary model is completed, the model decision module 130 extracts a preset number (k) of learning data from the data storage 140 (S366), so that each re-checked supplementary model that has completed learning judges the extracted k learning data as good or bad (S367), and calculates the judgment accuracy of each re-checked supplementary model based on the judgment result and the label information of each learning data (S368).

[0082] Through the above process, in a judgment accuracy evaluation process (S33), the supplementary model below the first benchmark value learns a portion of the learning data used to actually generate the new learning model, and then the judgment accuracy is re-evaluated, thereby increasing the possibility of extracting the learning model suitable for the new learning data as a pre-training model.

[0083] Re-reference Figure 3 When the accuracy of the re-checked supplementary models is calculated through the secondary accuracy evaluation process, it is determined whether the accuracy of each re-checked supplementary model is above the second benchmark value (S37). Among the re-checked supplementary models with an accuracy of above the second benchmark value, the top-ranked re-checked supplementary model is determined as the pre-trained model (S38). Conversely, if there is no re-checked supplementary model with an accuracy of above the second benchmark value, it can be determined as a recommendation failure (S39).

[0084] Here, the second benchmark value can be set relatively higher than the first benchmark value. For example, as mentioned above, if the first benchmark value is set to 80%, the second benchmark value can be set to 90%. This means that the learning data applied when the re-examination of the supplementary model learning, which accepts the application of the second benchmark value, is actually generated when generating a new learning model, thereby improving the accuracy of the judgment of the new learning data.

[0085] Through the process described above, the pre-trained model is determined from the supplementary model with high judgment accuracy in one go. When the judgment accuracy is lower than the first benchmark value, the upper n supplementary models learn new learning data. Then, a quadratic judgment process is added to calculate the judgment accuracy, so that the learning model that is closest to the new learning model is determined as the pre-trained model.

[0086] Below, refer to Figure 6 and Figure 7 Another embodiment of the process by which a pre-trained model is determined by the pre-trained model determination system 100 according to an embodiment of the present invention will be described in detail. Figure 6 and Figure 7 The embodiment shown in the figure is as follows Figures 3 to 5 The embodiments illustrated in the figure are variations, and descriptions of their corresponding structures are omitted.

[0087] As with the above-described embodiment, when new learning data for generating a new learning model is collected, it is stored to the data storage 140 (S60). Here, the plurality of learning models are in a state of being stored to the learning model storage 110, and each of the learning models includes model information.

[0088] When the judgment type information for generating a new learning model is input (S61), the post-model extraction module 120 extracts a post-model based on the judgment type information, with reference to the model information of the learning models stored in the learning model storage 110 (S62). The method of extracting a post-model by the post-model extraction module 120 is as described above.

[0089] When the post-model is extracted, the model decision module 130 performs a judgment accuracy evaluation process on each of the post-models (S63). Here, the judgment accuracy evaluation process performed by the model decision module 130 corresponds to the embodiment illustrated in Figure 4 , and thus the description thereof is omitted.

[0090] When the judgment accuracy of each of the post-models is calculated through the judgment accuracy evaluation process, it is determined whether the judgment accuracy of each of the post-models is the first reference value or more (S64). At this time, in a case where there is a similar post-model in which the difference between the judgment accuracy of the highest post-model and the first reference value is within a predetermined similar range, a comparison process of the similar post-model is performed (S71).

[0091] Referring to Figure 7 , specifically, in a case where there is a similar post-model in which the difference in the judgment accuracy of the highest post-model is within a similar range (S711), feature maps of the highest post-model and the similar post-model are extracted (S712). Also, feature point distributions of the feature maps are extracted (S713).

[0092] In order to calculate the judgment accuracy of the highest post-model and the similar post-model, a feature map is generated for each of the learning data in a process in which the post-model judges the learning data based on AI. Also, a probability distribution, that is, a feature point distribution, can be extracted in the feature map. Here, the feature map and the feature point distribution are extracted for normal learning data and abnormal learning data in the learning data.

[0093] Figure 8 is a diagram illustrating an example of a feature map and a feature point distribution generated in a judgment or learning process of a learning model in a pre-learning model decision system of an embodiment of the present disclosure. Figure 8The left image of (a) is the poor learning data, and the right image is the normal learning data. The left image of (b) is the feature map regarding the poor learning data, and the right image is the feature map regarding the normal learning data. The left image of (c) is the feature point distribution of the feature map regarding the poor learning data, and the right image is the feature point distribution of the feature map regarding the normal learning data.

[0094] Figure 8 (c) of FIG. 8 indicates the difference in the feature point distribution between the poor learning data and the normal learning data. As the difference between the two feature point distributions becomes larger, it can be evaluated that the learning model well extracts the feature map from the original image in a distinguishable manner, and as a result, the judgment accuracy of the final learning model can be improved.

[0095] In a case where the similar range for the judgment accuracy is set to 2%, the judgment accuracy of the top-ranking post-training model is calculated to be 85%, and in a case where the judgment accuracy of the next-ranking post-training model is calculated to be 83%, the post-training model is extracted as the similar post-training model, and the judgment accuracy can vary depending on the sampling process and the number of learning data extracted in order to calculate the judgment accuracy. Therefore, a case where it is not possible to determine which one of the top-ranking post-training model and the similar post-training model within the similar range is the best can occur, and thus the best pre-training model is determined using the feature point distribution.

[0096] In the present application, a case where the difference value (S714) of the feature point distribution between the poor learning data and the normal learning data is calculated in the top-ranking learning model and the similar post-training model is exemplified, and any one of the difference values is determined as the pre-training model. At this time, the predetermined number of poor learning data and normal learning data is arbitrarily extracted from a plurality of learning data applied to the judgment process of the top-ranking post-training model and the similar post-training model, the difference value of the feature point distribution between the poor learning data and the normal learning data is calculated (S714), and then the average value is calculated (S715). Also, the average value of the top-ranking post-training model and the average value of the similar post-training model are compared (S716), when the average value of the top-ranking post-training model is large, the top-ranking post-training model is selected (S717), when the average value of the similar post-training model is large, the similar post-training model is selected (S718), and is determined as the pre-training model (S65).

[0097] Here, the difference value of the feature point distribution can apply an algorithm for calculating the difference of the probability distribution, and in the present application, a case where the difference value is calculated by the KL divergence (KL-Divergence) algorithm is exemplified.

[0098] Referring back to Figure 6In a case where the judgment accuracy of all the post-filling models in the step (S64) is lower than the first reference value, the model decision module 130 executes a secondary judgment accuracy evaluation process (S66). Here, the secondary judgment accuracy evaluation process corresponds to the similar post-filling model comparison process illustrated in FIG. 7 of the Embodiment 2, and thus a detailed explanation thereof is omitted. Figure 5

[0099] When the judgment accuracy of each re-examination post-filling model is calculated through the secondary judgment accuracy evaluation process, it is judged whether the judgment accuracy of each re-examination post-filling model is equal to or higher than a second reference value (S67). At this time, in a case where there is a similar re-examination post-filling model whose difference from the judgment accuracy of the highest-ranking re-examination post-filling model is within a preset similar range, a similar re-examination post-filling model comparison process (S72) is performed.

[0100] Here, the similar re-examination post-filling model comparison process corresponds to the similar post-filling model comparison process illustrated in FIG. 7 of the Embodiment 2. Specifically, in a case where there is a similar re-examination post-filling model whose difference from the judgment accuracy of the highest-ranking re-examination post-filling model is within a similar range (see S711), feature maps of the highest-ranking re-examination post-filling model and the similar re-examination post-filling model are extracted (see S712). Also, feature point distributions of the feature maps are extracted (see S713). Figure 7

[0101] At this time, as described above, a difference value of the feature point distributions between the defective learning data and the normal learning data is selected (see S714), and then an average value is calculated (see S715). Also, the average value of the highest-ranking re-examination post-filling model and the average value of the similar re-examination post-filling model are compared (see S716), and when the average value of the highest-ranking re-examination post-filling model is greater, the highest-ranking re-examination post-filling model is selected (see S717), and when the average value of the similar re-examination post-filling model is greater, the similar re-examination post-filling model is selected (see S718) to decide a pre-training model (S68).

[0102] As described above, in a case where there is a highest-ranking post-filling model or a highest-ranking re-examination post-filling model, if there is a similar post-filling model or a similar re-examination post-filling model whose judgment accuracy is within a similar range, a final pre-training model is decided using the feature point distributions of the feature maps, so that a learning model closer to a new learning model is decided as the pre-training model.

[0103] On the other hand, as described above, in a case where there is no highest-ranking post-filling model or highest-ranking re-examination post-filling model, if there is a similar post-filling model or a similar re-examination post-filling model whose judgment accuracy is within a similar range, a final pre-training model is decided using the feature point distributions of the feature maps, so that a learning model closer to a new learning model is decided as the pre-training model. Figure 2 ​​As illustrated, the pre-training model decision system 100 of the embodiment of the present application further includes a label conversion module 160. As described above, the learning data includes learning images and label information for each learning image, and the label conversion module 160 converts the label information of the learning data into a registered format. The learning data is collected on the actual product line 300, and depending on the difference in the manner of recording the distinction between defective and normal on each product line 300, the format of the label information can differ.

[0104] In this regard, in the present application, the label conversion module 160 converts the label information of the learning data into a registered format of NG and OK, and the model decision module 130 decides the pre-training model using the learning data whose label information has been converted by the label conversion module 160.

[0105] Next, the AI-based visual management system of the embodiment of the present application will be described in detail with reference to Figure 9 and Figure 10 .

[0106] The AI-based visual management system of the embodiment of the present application includes a plurality of visual AI clients 30 and a visual AI cloud 10.

[0107] As Figure 9 illustrated, the visual AI clients 30 are respectively provided on a plurality of product lines 300 that are geographically separated, and register AI-based learning models for visual inspection to inspect whether or not defective. The product lines 300 can be separated according to each product, or the same product line 300 can be divided at multiple places. Also, as Figure 1 illustrated, one product line 300 includes a plurality of production processes, including an inspection process for visually inspecting a result of each production process, i.e., a component or a product. Here, the visual AI clients 30 are individually provided in each inspection process, or can be provided in a form of collectively managing each inspection process of one product line 300.

[0108] The visual AI cloud 10 communicates with each visual AI client 30 to manage each visual AI client 30. Also, the visual AI cloud 10 generates a new learning model registered to the visual AI client 30.

[0109] Figure 10 is a diagram illustrating an example of the structure of the visual AI cloud 10 and the visual AI client 30 of the AI-based visual management system of the embodiment of the present application.

[0110] Referring to Figure 10 , the visual AI client 30 includes a photographing module 31, a client storage 32, a deep learning inspection module 33, a client communication module 34, and a client control module 35.

[0111] The photography module 31 photographs the finished products (i.e., the inspection objects) to perform visual inspection of the finished products in each process of the product production line 300. The deep learning inspection module 33 includes a learning model based on deep learning, which receives images photographed by the photography module 31 through the learning model to determine whether they are defective or normal.

[0112] Images captured by the imaging module 31 are stored in the client memory 32. Here, the images stored in the client memory 32 are labeled with information indicating whether they are normal or defective, as determined by the deep learning inspection module 33.

[0113] The client control module 35 communicates with the visual AI cloud 10 through the client communication module 34. The client control module 35 controls the entire visual inspection through the deep learning inspection module 33, and updates the currently registered learning model or registers a new learning model through communication with the visual AI cloud 10.

[0114] Furthermore, under preset conditions, the client control module 35 requests the visual AI cloud 10 to relearn the currently registered learning model. For example, if a manager confirms a false negative image as a defective image by the deep learning inspection module 33, and the false negative rate reaches a certain level, the client control module 35 requests the visual AI cloud 10 to relearn. At this time, the client control module 35 transmits new image and label information stored in the client memory 32 as new learning data to the visual AI cloud 10.

[0115] like Figure 10 As shown, the visual AI cloud 10 includes a pre-trained model decision system 100, a new model generation system 11, a cloud communication module 12, and a cloud control module 13.

[0116] The pre-trained model determination system 100 determines the pre-trained model used to generate the new learning model from among multiple learning models stored in the learning model memory 110 through the above process, therefore a detailed description of it is omitted.

[0117] When the new model generation system 11 generates a new learning model, it teaches the pre-trained model determined in the pre-trained model determination system 100 to learn new learning data and generate a new learning model. Here, as described above, the new learning data is stored in the data storage 140 of the pre-trained model determination system 100.

[0118] In addition, the new model generation system 11, when generating a new learning model, in order to generate learning of a pre-training model of the new learning model, applies learning data applied in generation of the pre-training model decided by the model decision module 130, that is, learning data used when the learning model decided as the pre-training model is generated (may include learning data additionally collected after generation for re-learning) and the above new learning data together, and performs learning. Thus, in a case where the amount of learning data collected for generation of the new learning model is relatively small, learning data of a pre-learning model having characteristics closest to the new learning model is also included, and thus the judgment accuracy can be improved by learning using more learning data.

[0119] On the other hand, the cloud control module 13 of the visual AI cloud 10 transmits the new learning model generated by the new model generation system 11 to the visual AI client 30 through the cloud communication module 12.

[0120] In the present application, a case where a new learning model is generated according to a request of the visual AI client 30 is exemplified. For example, when a product production line 300 is newly set up for production of a new product, a new learning model required on the product production line 300 needs to be generated. At this time, on the product production line 300, a manager or other visual inspection machine judges good and bad over a certain period, and collects bad images and normal images, label information as learning data according to new parts or products and bad types.

[0121] Also, when a certain amount of learning data is collected, the client control module 35 of the visual AI client 30 transmits the collected learning data and requests generation of a new learning model to the visual AI cloud 10, at this time the client control module 35 transmits the judgment type information together, and thus the pre-training model decision system 100 of the visual AI cloud 10 decides a pre-training model. Also, the cloud control module 13 of the visual AI cloud 10 transmits a new learning model through the cloud communication module 12, to register the new learning model in the visual AI client 30 which transmitted the model generation request.

[0122] Through the above structure, when the same new product is produced on a plurality of product production lines 300 geographically separated, collection of initial learning data is dispersed, but is concentrated and transmitted to the visual AI cloud 10 to generate a new learning model, and thus a new learning model applied to the product production line 300 can be generated in a short time after the new product production line 300 is set up.

[0123] The above-described embodiments of the present application are explained with reference to the accompanying drawings, the present application is not limited to the above-described embodiments, and can be manufactured in various forms different from each other, and a person skilled in the art can implement various embodiments different from each other without changing the technical idea or essential characteristics of the present application. Therefore, the above-described embodiments are illustrative in all aspects, and the present application is not limited thereto.

[0124] (Symbol explanation)

[0125] 10: visual AI cloud 11: new model generation system

[0126] 12: cloud communication module 13: cloud control module

[0127] 14: input unit 30: visual AI client

[0128] 31: photographing module 32: client storage

[0129] 33: deep learning inspection module 34: client communication module

[0130] 35: client control module 100: pre-trained model decision system

[0131] 110: learning model storage 120: post-filling model extraction module

[0132] 130: model decision module 140: data storage

[0133] 150: main processor 160: label conversion module

[0134] Industrial utilization possibility

[0135] The present application can be applied to the field of visual inspection of the result of each production process of a product production line.

Claims

1. An AI-based pre-training model decision system, characterized in that, It includes: a learning model storage storing a plurality of learning models for visual inspection on a product production line; a data storage storing a plurality of learning data collected for generating a new learning model; a post-filling model extraction module extracting at least two or more post-filling models from the plurality of learning models based on judgment type information for generating the new learning model when the judgment type information is received; and a model decision module inputting a predetermined number of learning data from the learning data stored in the data storage to each of the post-filling models to judge good and bad, deciding a top post-filling model having a judgment accuracy of the plurality of post-filling models of a predetermined first reference value or more as a pre-training model for generating the new learning model, in the case where there is a similar post-filling model having a difference in the judgment accuracy from the top post-filling model within a predetermined similar range among the plurality of post-filling models, the model decision module extracts a feature point distribution of a feature map generated for the learning data in the judgment process of the top post-filling model and the similar post-filling model, the model decision module decides any one of the top post-filling model and the similar post-filling model having a large difference in the feature point distribution between bad learning data and normal learning data as the pre-training model.

2. The AI-based pre-training model decision system according to claim 1, wherein the model decision module calculates an average value of the difference in the feature point distribution between a predetermined number of bad learning data and normal learning data selected at random from the plurality of learning data applied in the judgment process of the top post-filling model and the similar post-filling model, the model decision module decides any one of the top post-filling model and the similar post-filling model having a large average value as the pre-training model.

3. The AI-based pre-training model decision system according to claim 1, wherein the difference in the feature point distribution is calculated by a KL divergence algorithm.

4. The AI-based pre-training model decision system according to claim 1, wherein in the case where the judgment accuracy of the plurality of post-filling models is a predetermined first reference value or less, the model decision module extracts top n as re-inspection post-filling models based on the judgment accuracy, the model decision module causes each of the re-inspection post-filling models to learn using a predetermined number of learning data from the plurality of learning data, the model decision module judges good and bad by inputting a predetermined number of learning data from the plurality of learning data to the learned re-inspection post-filling models, the model decision module decides a top re-inspection post-filling model having a judgment accuracy of each of the re-inspection post-filling models of a predetermined second reference value or more as the pre-training model.

5. The AI-based pre-training model decision system according to claim 4, wherein The model determination module removes the post-filling model whose determination accuracy is lower than a predetermined lower limit value from the post-filling models after the re-examination. 6.The AI-based pre-trained model determination system according to claim 4, wherein In a case where there is a similar post-filling model whose difference from the determination accuracy of the highest-level post-filling model after the re-examination is within a predetermined similar range among the plurality of post-filling models after the re-examination, the model determination module extracts feature point distributions of feature maps generated for each of the learning data in determination processes of the highest-level post-filling model after the re-examination and the similar post-filling model, The model determination module determines either one of the highest-level post-filling model after the re-examination and the similar post-filling model, in which a difference in the feature point distribution between the defective learning data and the normal learning data is large, as the pre-trained model. 7.The AI-based pre-trained model determination system according to claim 6, wherein The model determination module calculates an average value of the difference in the feature point distribution between the defective learning data and the normal learning data selected at random from a predetermined number of learning data used in the determination processes of the highest-level post-filling model after the re-examination and the similar post-filling model, The model determination module determines either one of the highest-level post-filling model after the re-examination and the similar post-filling model, in which the average value is large, as the pre-trained model. 8.The AI-based pre-trained model determination system according to claim 6, wherein The difference in the feature point distribution is calculated by a KL divergence algorithm. 9.The AI-based pre-trained model determination system according to claim 1, wherein The determination type information includes at least one of defective type information about a type of defect, product type information about a type of product to be inspected, and component type information about a type of component to be inspected, Each of the learning models stored in the learning model storage includes model information including at least one of the defective type information, the product type information, and the component type information, The post-filling model extraction module extracts the post-filling models with reference to the model information. 10.The AI-based pre-trained model determination system according to claim 9, wherein The post-filling model extraction module extracts a predetermined number of learning models as the post-filling models in a priority order set in the order of the defective type information, the component type information, and the product type information. 11.The AI-based pre-trained model determination system according to claim 1, wherein The learning data includes learning images and label information in which information about defectiveness or normality of the learning images is recorded, The AI-based pre-trained model determination system further includes a label conversion module that converts the label information of the learning data into a registered format, The model decision module decides the pre-training model using the learning data whose label information is transformed by the label transformation module.

12. An AI-based visual inspection management system, characterized by, It includes: a plurality of visual AI clients that are respectively provided on a plurality of product production lines that are geographically separated, register AI-based learning models for visual inspection, and detect whether or not a defect exists; and a visual AI cloud that communicates with each of the visual AI clients, manages each of the visual AI clients, and generates a new learning model registered to the visual AI clients, The visual AI cloud includes: an AI-based pre-training model decision system according to any one of claims 1 to 11; and a new model generation system that generates the new learning model by causing the pre-training model decided by the model decision module of the pre-training model decision system using a plurality of the learning data stored in the data storage of the pre-training model decision system to learn.

13. The AI-based visual inspection management system according to claim 12, wherein The visual AI cloud further includes: a cloud control module that registers the new learning model generated by the new model generation system to at least one of the plurality of visual AI clients.

14. The AI-based visual inspection management system according to claim 13, wherein The plurality of learning data stored in the data storage are transmitted and stored from at least one of the plurality of visual AI clients, The pre-training model decision system decides the pre-training model according to a model generation request from the visual AI cloud that transmitted the learning data, The cloud control module registers the new learning model to the visual AI cloud that transmitted the model generation request.

15. The AI-based visual inspection management system according to claim 13, wherein The new model generation system applies the learning data used in the generation of the pre-training model decided by the model decision module to learning of the pre-training model used to generate the new learning model.

Images