Apparatus and method for updating learning data for continuous learning
By selecting and updating the learning dataset in the buffer, the problems of decreased accuracy and limited storage space in radiation penetration inspection are solved, achieving efficient learning model updates and sustained high accuracy.
Patent Information
- Application Number
- CN202510376884.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-10-16
- Filing Date
- 2025-03-27
- Publication Date
- 2025-11-28
AI Technical Summary
In existing AI-based industrial radiation penetration inspections, the accuracy of object detection decreases over time, storage space becomes limited, and transfer learning may lead to forgetting.
By selecting learning datasets that affect the prediction performance of the current learning model in the buffer, calculating the degree of change and the degree of influence, updating the buffered learning dataset, and continuously learning the learning model using the buffered learning data and the current dataset, the data with the highest degree of influence is selected for updating.
While overcoming the forgetting phenomenon, it minimizes storage space and continuously provides a highly accurate learning model.
Smart Images
Figure CN121033570A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to a defect diagnosis technology, and more particularly, to an apparatus for diagnosing defects using a learning model based on continuous learning and a method thereof.
[0002] The present application relates to a learning data selection technology, and more particularly, to an apparatus for selecting learning data for continuous learning and a method thereof.
[0003] The present application relates to a learning data update technology, and more particularly, to an apparatus for updating learning data for continuous learning and a method thereof. BACKGROUND
[0004] Radiographic Test (RT) is a method of selecting radiation such as X-rays and γ-rays according to use conditions and purposes, and making the selected radiation penetrate a test object, thereby imaging on an x-ray film and detecting defects inside the test object. It is the most widely used method among non-destructive inspection methods for detecting internal defects.
[0005] For defect diagnosis based on artificial intelligence-based industrial radiographic test (RT), the main task is object detection, and it is necessary to maintain the accuracy of object detection as time passes. In addition, as a common method of re-learning the characteristics of new data sets generated as time passes while operating an artificial intelligence model, a transfer learning method is used, but there can be a forgetting phenomenon in which the accuracy related to previous data decreases. In the case of retaining all previous learning data sets in order to prevent the above-described problem, a storage space problem occurs. SUMMARY
[0006] The present application relates to a defect diagnosis technology, and more particularly, to an apparatus for diagnosing defects using a learning model based on continuous learning and a method thereof.
[0007] The present application relates to a learning data selection technology, and more particularly, to an apparatus for selecting learning data for continuous learning and a method thereof.
[0008] The present application relates to a learning data update technology, and more particularly, to an apparatus for updating learning data for continuous learning and a method thereof.
[0009] To achieve the above object, a method of diagnosing defects according to a preferred embodiment of the present application includes the steps of learning, by a learning part, a current learning model using a buffer learning data set previously stored in a buffer and a current learning data set, and detecting, by a detection part, defects from a radiographic image using the current learning model when the radiographic image is input.
[0010] The step of learning the current learning model includes a step of loading, by the learning unit, learning data selected from past learning data sets according to degrees of influence on predictive performance of the current learning model in the continuous learning process, i.e., a buffer learning data set, and a step of learning the current learning model using the buffer learning data set and the current learning data set.
[0011] The method further includes, after the step of learning the current learning model, a step of calculating, by the switching unit, degrees of change from a plurality of past learning models of previous learning rounds that are two or more rounds apart from the current learning model to the current learning model in the continuous learning, a step of determining, by the switching unit, whether the degrees of change reach a reference value or more, a step of deciding, by the switching unit, to update the buffer learning data set in a case where the degrees of change reach the reference value or more, a step of calculating, by the updating unit, degrees of influence of a plurality of learning data in the current learning data set on predictive performance of a future learning model that is to be learned in a subsequent learning round of the current learning model when the future learning model is learned using the current learning data set in the continuous learning process, and a step of updating, by the updating unit, the buffer learning data set from a part of the plurality of learning data according to an order from high to low of the degrees of influence.
[0012] The step of calculating the degrees of influence includes a step of calculating, by the updating unit, a plasticity score indicating a probability that a predictive value of the current learning model and a predictive value of the future learning model differ for learning data in the current learning data set, a step of calculating, by the updating unit, a stability score indicating a probability that a predictive value of the current learning model and a predictive value of a first past learning model learned in a previous learning round of the current learning model, i.e., a first past learning round, differ for learning data in the current learning data set, and a step of calculating, by the updating unit, a weighted average of the plasticity score and the stability score as the degree of influence.
[0013] The step of calculating the stability score is characterized in that:
[0014] According to a mathematical formula
[0015]
[0016] The calculation is performed,
[0017] The is a stability score, the is learning data of the current learning data set, and the is a predicted value of the first past learning model, and is a predicted value of the current learning model.
[0018] The step of calculating the plasticity score, characterized in that:
[0019] According to the mathematical formula
[0020]
[0021] is calculated,
[0022] The is a plasticity score, the is learning data of a current learning dataset, the is a predicted value of the future learning model, and the is a predicted value of the current learning model.
[0023] The step of calculating the plasticity score, characterized in that:
[0024] According to the mathematical formula
[0025]
[0026] The predicted value of the future learning model is derived,
[0027] The is a predicted value of the future learning model, the is learning data of a current learning dataset, and the GP(θ t ) is a gradient vector predicted by the gradient prediction model from the future learning model.
[0028] The step of calculating the influence degree, characterized in that:
[0029] According to the mathematical formula
[0030] S i = λ · Plasticity i + (1 - λ) · Stability i
[0031] is calculated,
[0032] The S is an influence degree, the i is an index of a current learning data, the λ is a weight, the Plasticity is a plasticity score, and the Stability is a stability score.
[0033] The step of calculating the degree of change is characterized by deriving the degree of change from a degree of similarity of a first gradient vector representing a change between a weight vector of a current learning model and a weight vector of a first past learning model that learned in a previous learning round of the current learning model and a second gradient vector representing a change between the weight vector of the first past learning model and a weight vector of a second past learning model that learned in a previous learning round of the first past learning model.
[0034] The degree of similarity is characterized by:
[0035] According to a mathematical expression
[0036]
[0037] The calculation is performed,
[0038] The CS is a degree of similarity, the g t is a first gradient vector between a weight vector of a current learning model and a weight vector of a first past learning model, the g t-1 is a second gradient vector between the weight vector of the first past learning model and a weight vector of a second past learning model, and the is a transposed vector of the second gradient vector.
[0039] To achieve the above object, a device for diagnosing a defect according to a preferred embodiment of the present application includes a learning unit that learns a current learning model using a buffer learning dataset selected from past learning datasets according to a degree of influence on a prediction performance of the current learning model in a continuous learning process and a current learning dataset, and a detection unit that detects a defect from a radiographic image using the current learning model when the radiographic image is input.
[0040] The learning unit is characterized by loading the buffer learning dataset selected from the past learning datasets according to the degree of influence on the prediction performance of the current learning model in the continuous learning process and learning the current learning model using the buffer learning dataset and the current learning dataset.
[0041] The device further includes a switching unit that calculates a degree of change from a plurality of past learning models of previous learning rounds that are two or more rounds apart from the current learning model to the current learning model in a learning round of the continuous learning, and determines whether the degree of change reaches a reference value
[0042]
[0043] If the value is above a certain threshold, and the degree of change reaches a threshold value or above a certain threshold, it is decided to update the buffer learning dataset; and the updating unit calculates the degree to which multiple learning data in the current learning dataset affect the prediction performance of the future learning model when the current learning dataset is used to learn the future learning model in subsequent learning rounds of the current learning model during continuous learning, and selects a portion of the multiple learning data to update the buffer learning dataset according to the order of the degree of influence from high to low.
[0044] The updating unit is characterized by the following steps: calculating a plasticity score for the learning data in the current learning dataset, representing the probability that the predicted value of the current learning model differs from the predicted value of the future learning model; calculating a stability score for the learning data in the current learning dataset, representing the probability that the predicted value of the current learning model differs from the predicted value of a first past learning model that was learned in the previous learning rounds of the current learning model, i.e., the first past learning rounds; and calculating a weighted average of the plasticity score and the stability score as the degree of influence.
[0045] The updating unit is characterized in that:
[0046] According to the mathematical formula
[0047]
[0048] Calculate the stability score.
[0049] The As the stability score, the The learning data for the current learning dataset, the The predicted value is the first past learning model, and the... This is the predicted value of the current learning model.
[0050] The updating unit is characterized in that:
[0051] According to the mathematical formula
[0052]
[0053] Calculate the plasticity fraction.
[0054] The As the plasticity fraction, the The learning data for the current learning dataset, the The predicted value of the future learning model, and the This is the predicted value of the current learning model.
[0055] The update section is characterized by:
[0056] According to the mathematical formula
[0057]
[0058] deriving a prediction value of the future learning model,
[0059] The update section is characterized by: being a prediction value of a future learning model, the update section is characterized by: being learning data of a current learning dataset, and the GP(θ t ) being a gradient vector predicted by the future learning model through the gradient prediction model.
[0060] The update section is characterized by:
[0061] According to the mathematical formula
[0062] S i = λ · Plasticity i + (1 - λ) · Stability i
[0063] calculating the degree of influence,
[0064] The S is a degree of influence, the i is an index of current learning data, the λ is a weight, the Plasticity is a plasticity score, and the Stability is a stability score.
[0065] The switching section is characterized by deriving the degree of change from a degree of similarity of a first gradient vector representing a change between a weight vector of a current learning model and a weight vector of a first past learning model that learned in a previous learning round of the current learning model and a second gradient vector representing a change between the weight vector of the first past learning model and a weight vector of a second past learning model that learned in a previous learning round of the first past learning model.
[0066] The switching section is characterized by:
[0067] According to the mathematical formula
[0068]
[0069] calculating the degree of similarity,
[0070] The CS is a degree of similarity, the g tis a first gradient vector between a weight vector of the current learning model and a weight vector of a first past learning model, the g t-1 is a second gradient vector between the weight vector of the first past learning model and a weight vector of a second past learning model, and the g is a transposed vector of the second gradient vector.
[0071] In order to achieve the above-mentioned object, the learning data selection method according to the preferred embodiment of the present application comprises the following steps: a step of learning, by a learning unit, a current learning model using a buffer learning data set stored in a buffer and a current learning data set; a step of calculating, by an updating unit, degrees of influence of a plurality of learning data in the current learning data set on a prediction performance of a future learning model to be learned in a subsequent learning round of the current learning model when the future learning model is learned using the current learning data set in a continuous learning process; and a step of updating, by the updating unit, the buffer learning data set according to an order from high to low of the degrees of influence to select a part of the plurality of learning data.
[0072] The step of calculating the degrees of influence comprises the following steps: a step of calculating, by the updating unit, a plasticity score representing a probability of a difference between a prediction value of the current learning model and a prediction value of the future learning model for learning data in the current learning data set; a step of calculating, by the updating unit, a stability score representing a probability of a difference between the prediction value of the current learning model and a prediction value of a first past learning model learned in a previous learning round of the current learning model, i.e., a first past learning round, for the learning data in the current learning data set; and a step of calculating, by the updating unit, a weighted average of the plasticity score and the stability score as the degree of influence.
[0073] The step of calculating the stability score is characterized in that the updating unit calculates the stability score according to a mathematical formula
[0074]
[0075] The stability score is calculated according to a mathematical formula is a stability score, the g is learning data of the current learning data set, the g is a prediction value of the first past learning model, and the g is a prediction value of the current learning model.
[0076] The step of calculating the plasticity score is characterized in that the updating unit calculates the plasticity score according to a mathematical formula
[0077]
[0078] calculating the plasticity score, the is a plasticity score, the is learning data of a current learning dataset, the is a predicted value of a future learning model, and the is a predicted value of a current learning model.
[0079] The method, before the step of calculating the influence degree, further comprises: a step of constructing, by the learning unit, learning data derived from the current learning dataset, i.e. derived learning data; and a step of learning, by the learning unit, the gradient prediction model using the derived learning data.
[0080] The step of calculating the plasticity score, characterized in that: the switching unit calculates the plasticity score using the gradient prediction model.
[0081] The derived learning data, characterized in that: it comprises a modified weight vector of an object layer of the current learning model each time the current learning dataset is divided in a round unit and the current learning model is iteratively learned by the learning unit, and a target gradient vector representing the difference between the weight vector of the previous round and the weight vector of the current round each time the current learning dataset is divided in a round unit and the current learning model is iteratively learned.
[0082] The step of calculating the plasticity score, characterized in that: the updating unit,
[0083] According to the mathematical formula
[0084]
[0085] deriving a predicted value of the future learning model; the is a predicted value of a future learning model, and the is learning data of a current learning dataset, and the GP(θ t ) is a gradient vector predicted by the future learning model through the gradient prediction model.
[0086] The step of calculating the influence degree, characterized in that: the updating unit,
[0087] According to the mathematical formula
[0088] S i = λ · Plasticity i + (1-λ) · Stability i
[0089] The influence degree is calculated, S is the influence degree, i is the index of the current learning data, λ is the weight, Plasticity is the plasticity score, and Stability is the stability score.
[0090] To achieve the above-mentioned purpose, the learning data selection device according to the preferred embodiment of the present application comprises: a learning unit that learns a current learning model using a buffer learning data set stored in a buffer and a current learning data set; and an updating unit that calculates an influence degree of each of a plurality of learning data in the current learning data set on a prediction performance of a future learning model to be learned in a subsequent learning round of the current learning model, and updates the buffer learning data set using a part of the plurality of learning data in descending order of the influence degree.
[0091] The updating unit is characterized in that the updating unit calculates a plasticity score representing a probability that a prediction value of the current learning model and a prediction value of the future learning model differ from each other, and a stability score representing a probability that a prediction value of the current learning model and a prediction value of a first past learning model learned in a first past learning round of the current learning model differ from each other, and calculates a weighted average of the plasticity score and the stability score as the influence degree.
[0092] The updating unit is characterized in that the updating unit calculates the influence degree according to the mathematical expression
[0093]
[0094] The stability score is calculated, S is the stability score, i is the index of the current learning data set, λ is the weight, Plasticity is the plasticity score, and Stability is the stability score. is the stability score, i is the index of the current learning data set, λ is the weight, Plasticity is the plasticity score, and Stability is the stability score. is the learning data of the current learning data set, λ is the weight, Plasticity is the plasticity score, and Stability is the stability score. is the prediction value of the first past learning model, and λ is the weight. is the prediction value of the current learning model.
[0095] The updating unit is characterized in that the updating unit calculates the influence degree according to the mathematical expression
[0096] The updating unit is characterized in that the updating unit calculates the influence degree according to the mathematical expression
[0097]
[0098] The plasticity score is calculated, S is the plasticity score, i is the index of the current learning data set, λ is the weight, Plasticity is the plasticity score, and Stability is the stability score. is the plasticity score, i is the index of the current learning data set, λ is the weight, Plasticity is the plasticity score, and Stability is the stability score. is learning data of a current learning dataset, the is a predicted value of a future learning model, and the is a predicted value of a current learning model.
[0099] The learning unit constructs learning data derived from the current learning dataset, i.e., derived learning data, and learns the gradient prediction model using the constructed derived learning data, and the switching unit calculates the plasticity score using the gradient prediction model.
[0100] The derived learning data includes a modified weight vector of an object layer of the current learning model each time the current learning dataset is divided in a round unit and the current learning model is iteratively learned by the learning unit, and a target gradient vector representing a difference between a weight vector of a previous round and a weight vector of a current round each time the current learning dataset is divided in a round unit and the current learning model is iteratively learned.
[0101] The updating unit includes:
[0102] According to a mathematical expression
[0103]
[0104] derives a predicted value of the future learning model, the is a predicted value of a future learning model, and the is learning data of a current learning dataset, and the GP(θ t ) is a gradient vector predicted by the gradient prediction model from the future learning model.
[0105] The updating unit includes:
[0106] According to a mathematical expression
[0107] S i = λ · Plasticity i + (1 - λ) · Stability i
[0108] calculates the influence degree, S, the index of the current learning data, i, the weight, λ, the plasticity score, Plasticity, and the stability score, Stability.
[0109] To achieve the above object, the learning data updating method according to the preferred embodiment of the present application includes: a step of learning, by a learning unit, a current learning model using a buffer learning data set stored in advance into a buffer and a current learning data set; a step of calculating, by a switching unit, a degree of change from a past learning model of a previous learning round, which is two rounds or more apart from the current learning model, to the current learning model in a learning round of continuous learning; a step of determining, by the switching unit, whether the degree of change is equal to or more than a reference value; a step of deciding, by the switching unit, to update the buffer learning data set in a case where the degree of change is equal to or more than the reference value; and a step of extracting, by an updating unit, at least a part of learning data in the current learning data set and updating the buffer learning data set.
[0110] The step of calculating the degree of change is characterized by deriving the degree of change by the switching unit from a degree of similarity between a first gradient vector representing a change between a weight vector of a current learning model and a weight vector of a first past learning model learned in a previous learning round of the current learning model and a second gradient vector representing a change between the weight vector of the first past learning model and a weight vector of a second past learning model learned in a previous learning round of the first past learning model.
[0111] The degree of similarity is characterized by:
[0112] According to a mathematical expression
[0113]
[0114] The CS is a degree of similarity, g t is a first gradient vector between a weight vector of a current learning model and a weight vector of a first past learning model, g t-1 is a second gradient vector between the weight vector of the first past learning model and a weight vector of a second past learning model, and the is a transposed vector of the second gradient vector.
[0115] The step of determining whether the degree of change is equal to or more than a reference value is characterized by determining, by the switching unit, that the degree of change is equal to or more than the reference value in a case where the degree of similarity between the first gradient vector and the second gradient vector is less than the reference value.
[0116] The method further includes, after the step of determining whether the degree of change is equal to or more than a reference value, a step of erasing, by the switching unit, a current learning data set in a case where the degree of change is less than the reference value.
[0117] The buffered learning data set stored in advance in the buffer is characterized by being learning data selected from past learning data sets in accordance with a degree of influence on the prediction performance of the current learning model in a continuous learning process.
[0118] To achieve the above object, a learning data updating device according to a preferred embodiment of the present application includes: a learning section that learns a current learning model using a buffered learning data set stored in advance in a buffer and a current learning data set; a switching section that calculates a degree of change from a past learning model at a previous learning round that is two or more learning rounds apart from the current learning model in a continuous learning to the current learning model, and determines to update the buffered learning data set if the degree of change is equal to or greater than a reference value after determining whether the degree of change is equal to or greater than the reference value; and an updating section that extracts at least a part of the learning data in the current learning data set and updates the buffered learning data set.
[0119] The switching section is characterized by deriving the degree of change from a degree of similarity between a first gradient vector representing a change between a weight vector of the current learning model and a weight vector of a first past learning model that learned at a previous learning round of the current learning model, and a second gradient vector representing a change between the weight vector of the first past learning model and a weight vector of a second past learning model that learned at a previous learning round of the first past learning model.
[0120] The switching section is characterized by:
[0121] According to mathematical expression
[0122]
[0123] The degree of similarity is calculated, the CS is a degree of similarity, the g t is a first gradient vector between a weight vector of the current learning model and a weight vector of the first past learning model, the g t-1 is a second gradient vector between the weight vector of the first past learning model and a weight vector of the second past learning model, and the is a transposed vector of the second gradient vector.
[0124] The switching section is characterized by determining that the degree of change is equal to or greater than the reference value if the degree of similarity between the first gradient vector and the second gradient vector is less than the reference value.
[0125] The switching section is characterized by erasing the current learning data set by the switching section if the degree of change is less than the reference value.
[0126] the buffer learning dataset stored in advance into the buffer, is characterized by being learning data selected from past learning data sets according to a degree of influence on the prediction performance of the future learning model in the continuous learning process.
[0127] The present application can minimize storage space while overcoming the catastrophic forgetting phenomenon by updating the buffer learning dataset according to the prediction performance of the future learning model and simultaneously performing continuous learning of the learning model using the buffer learning dataset and the new learning dataset, thereby continuously providing a learning model achieving high accuracy at the lowest cost. BRIEF DESCRIPTION OF DRAWINGS
[0128] Figure 1 is a schematic diagram for explaining the configuration of a system for diagnosing defects using a learning model based on continuous learning according to an embodiment of the present application.
[0129] Figure 2 is a schematic diagram for explaining the configuration of an apparatus for diagnosing defects using a learning model based on continuous learning according to an embodiment of the present application.
[0130] Figure 3 is a schematic diagram for explaining learning data sets and learning models according to learning rounds in a continuous learning process according to an embodiment of the present application.
[0131] Figure 4 is a flowchart for explaining a selection method of learning data for continuous learning according to an embodiment of the present application.
[0132] Figure 5 is a flowchart for explaining a learning method of a gradient prediction model according to an embodiment of the present application.
[0133] Figure 6 is a flowchart for explaining a calculation method of a degree of influence of learning data on the prediction performance of a future learning model according to an embodiment of the present application.
[0134] Figure 7 is a flowchart for explaining a method of diagnosing defects using a learning model based on continuous learning according to an embodiment of the present application.
[0135] Figure 8 is a screen example for explaining a method of diagnosing defects using a learning model based on continuous learning according to an embodiment of the present application.
[0136] Figure 9 is a schematic diagram illustrating a computing device according to an embodiment of the present application. DETAILED DESCRIPTION
[0137] The present application can be modified in various ways and have various embodiments, and specific embodiments will be illustrated and described in detail below. However, this is not intended to limit the present application to specific embodiments, but should be understood to include all modifications, equivalents, and alternatives included in the spirit and scope of the present application.
[0138] The terms used in the present application are used only to describe specific embodiments, and are not intended to limit the present application. Unless explicitly stated otherwise in the context, singular forms also include plural forms. In the present application, terms such as "include" or "have" are used only to indicate that the features, numbers, steps, actions, components, parts or combinations described in the specification are present, and should not be understood as excluding the possibility of existence or addition of one or more other features, numbers, steps, actions, components, parts or combinations.
[0139] In particular, the terms or words used in the specification and claims described below should not be interpreted as limited to general or dictionary meanings, but should be interpreted based on the concept of the inventor who can appropriately define the concepts of the terms to best explain the present application in the best way. In particular, in the embodiments of the present application, "estimation" or "inference" means deriving a result calculated by a learning model (LM: Machine Learning Model / Deep Learning Model) according to learning content.
[0140] First, a system for diagnosing defects using a learning model based on continuous learning according to an embodiment of the present application will be described. Figure 1 is a schematic diagram for explaining the configuration of a system for diagnosing defects using a learning model based on continuous learning according to an embodiment of the present application.
[0141] Figure 2 is a schematic diagram for explaining the configuration of an apparatus for diagnosing defects using a learning model based on continuous learning according to an embodiment of the present application. Figure 3 is a schematic diagram for explaining the learning data set and the learning model according to the learning round in the continuous learning process according to an embodiment of the present application.
[0142] Referring to Figure 1, a system for diagnosing a defect using a learning model based on continuous learning according to an embodiment of the present invention includes a radiography device RTA, a scanner SC, an inspection device 10, and a storage device 40.
[0143] The radiography device RTA can perform radiography on an object body to which an image quality indicator (IQI) is attached, thereby obtaining a radiography film on which the object body to which the image quality indicator is attached is photographed. Here, the object body can be exemplified by a pipe and a hose.
[0144] The scanner SC can scan the radiography film, thereby generating a digitized radiography image. After the digitized radiography image is generated in the manner as described above, the generated radiography image is input to the inspection device 10. At this time, the radiography image can be directly input from the scanner SC, or can be input from a corresponding storage medium after being stored in the storage medium.
[0145] The inspection device 10 can learn a current learning model using a buffer learning dataset previously stored in a buffer area BF and a current learning dataset through continuous learning, and detect a defect from the radiography image using the current learning model which is the latest learning model among the learning models learned through the continuous learning. In addition, the inspection device 10 can output a report including the radiography image in which the defect is detected by a bounding box (BB).
[0146] The storage device 40 can be a cloud server or a database server. The buffer area BF of the storage device 40 can store data, and the buffer learning dataset according to an embodiment of the present invention can be stored in the buffer area BF as described above. Here, the buffer learning dataset is learning data selected from past learning datasets used in a learning process of a past learning round based on a current learning round, according to a degree of influence on a prediction performance of a current learning model in a continuous learning process. In addition, although a case in which the buffer area BF is a storage medium included in the storage device 40 is described, in other embodiments of the present invention, the buffer area BF can be a storage medium included in the inspection device 10.
[0147] Referring to Figure 2 , the inspection device 10 includes a collection part 100, a learning part 200, a switching part 300, an update part 400, and a detection part 500.
[0148] The collection part 100 can continuously collect learning data and construct a current learning dataset for continuous learning. The collection part 100 provides the constructed current learning dataset to the learning part 200 after constructing the current learning dataset.
[0149] The learning unit 200 can generate a learning model by learning (Deep Learning or Machine Learning). The learning model according to the embodiment of the present application can detect a learned object from an image. The learning model according to the embodiment of the present application detects a learned object, i.e., a defect, from a radiographic image. At this time, the learning model can detect an area occupied by the defect from the radiographic image through a bounding box (BB). The learning model as described above can be exemplified by, for example, a convolutional neural network (CNN), YOLO, a region convolutional neural network (RCNN), and a faster region convolutional neural network (Faster RCNN). The learning model includes a plurality of layers (or modules) connected to each other, and the plurality of layers (or modules) is constituted by a plurality of operations. In addition, the plurality of layers (or modules) is connected by a weight (W). That is, an operation result output from a certain layer (or module) is input to an operation of a next layer after applying a weight. The learning model performs a plurality of operations applying a weight between a plurality of layers (or modules) on input data and generates an output. In other words, a plurality of operations connected by a weight between a plurality of layers (or modules) is performed. Next, the plurality of operations connected by a weight between a plurality of layers (or modules) of the learning model as described above is referred to as a "weight operation".
[0150] In particular, the learning unit 200 according to the embodiment of the present application learns a learning model through continuous learning.
[0151] Referring to Figure 3 , the continuous learning in the embodiment of the present application refers to a method of continuously collecting new learning data and learning a learning model using learning data sets of different learning rounds after constructing a learning data set including a plurality of learning data by dividing the continuously collected learning data according to learning rounds. In the embodiment of the present application, the learning rounds can be divided according to at least one of time, the number of collected learning data, and a specific event. Thereby, the present application can continuously learn according to different learning rounds using a learning data set including a plurality of learning data divided according to learning rounds, and thus can continuously learn a learning model, thereby continuously generating a learning model different from a learning model derived in a previous learning round.
[0152] In the embodiment of the present application, the learning model that is to be learned in the current learning round (t) is referred to as the current learning model when the reference time point is referred to as the current learning round (t). Further, the learning data set that is used to learn the current learning model is referred to as the current learning data set. The current learning data set includes a plurality of learning data that is collected from the previous learning round of the current learning round (t), i.e., the first past learning round (t-1), to the current learning round (t).
[0153] Further, the learning model that is learned in the previous learning round of the current learning model, i.e., the first past learning round (t-1), is referred to as the first past learning model. Further, the learning data set that is used to learn the first past learning model is referred to as the first past learning data set. The first past learning data set includes a plurality of learning data that is collected from the previous learning round of the first past learning round (t-1), i.e., the second past learning round (t-2), to the first past learning round (t-1).
[0154] Further, the learning model of all the previous learning rounds (t-1, t-2,...) with reference to the current learning round (t) is referred to as the past learning model. Further, the learning data set of all the past learning rounds (t-1, t-2,...) with reference to the current learning round (t) is referred to as the past learning data set.
[0155] Further, the learning model that is to be learned in the subsequent learning round of the current learning model, i.e., the future learning round (t+1), is referred to as the future learning model. The learning data set that is used to learn the future learning model is referred to as the future learning data set. The future learning data set includes a plurality of learning data that is collected from the current learning round (t) to the future learning round (t+1).
[0156] Referring back to Figure 2 , the switching unit 300 is used to determine whether to update the buffer learning data set according to the embodiment of the present application and the time point of the update.
[0157] The switching section 300 derives the degree of change from a plurality of past learning models of the previous learning round that is two or more rounds apart from the current learning model to the current learning model in the learning round of the continuous learning. To this end, the switching section 300 derives a first gradient vector that indicates the change between the weight vector of the current learning model and the weight vector of the first past learning model that is learned in the previous learning round of the current learning model. Next, the switching section 300 generates a second gradient vector that indicates the change between the weight vector of the first past learning model and the weight vector of the second past learning model that is learned in the previous learning round of the first past learning model. Next, the switching section 300 derives the degree of change from the similarity between the first gradient vector and the second gradient vector. Next, the switching section 300 determines whether the degree of change is equal to or greater than a reference value. That is, the switching section 300 can determine that the degree of change is equal to or greater than the reference value in the case where the similarity between the first gradient vector and the second gradient vector is less than the reference value, and determine that the degree of change is less than the reference value in the case where the similarity is equal to or greater than the reference value. According to the determination result as described above, in the case where the degree of change is equal to or greater than the reference value (the similarity is less than the reference value), the switching section 300 decides to update the buffer learning dataset. In contrast, the switching section 300 erases the current learning dataset in the case where the degree of change is less than the reference value (the similarity is equal to or greater than the reference value).
[0158] The updating section 400 calculates the degree to which the plurality of learning data in the current learning dataset respectively affects the prediction performance of the future learning model. That is, the updating section 400 calculates the degree to which the plurality of learning data in the current learning dataset respectively affects the prediction performance of the future learning model that is the learning model of the subsequent learning round of the current learning model when the future learning model is learned using the current learning dataset in the process of the continuous learning. Further, the updating section 400 selects a part of the plurality of learning data of the current learning dataset in the order of the calculated degrees of the effect from high to low, and updates the buffer learning dataset. At this time, the updating section 400 deletes the buffer learning data that is previously stored in the buffer BF, and stores the learning data selected from the plurality of learning data in the order of the degrees of the effect from high to low in the buffer, thereby updating the buffer learning dataset. Further, the updating section 400 erases the remaining learning data that is not selected from the plurality of learning data in the order of the degrees of the effect from high to low.
[0159] Next, the selection method of the learning data for the continuous learning according to the embodiment of the present application will be described. Figure 4 The selection method of the learning data for the continuous learning according to the embodiment of the present application will be described.
[0160] Referring to Figure 4The collection section 100 continuously collects learning data and constructs a current learning data set in step S110.
[0161] In step S120, the learning section 200 generates a current learning model by learning using the buffered learning data set stored in advance in the buffer BF and the current learning data set after loading the buffered learning data set stored in advance in the buffer BF. The buffered learning data set stored in advance in the buffer BF is learning data selected from past learning data sets used in past learning processes of learning rounds according to the degree of influence on the prediction performance of the current learning model in a continuous learning process.
[0162] The switching section 300 derives the degree of change from a plurality of past learning models of learning rounds preceding the current learning model by two or more learning rounds in the continuous learning in step S130. Next, step S130 will be described in more detail as described above.
[0163] First, the switching section 300 derives a first gradient vector indicating the change between the weight vector of the current learning model and the weight vector of a first past learning model learned in a past learning round of the current learning model.
[0164] Next, the switching section 300 derives a second gradient vector indicating the change between the weight vector of the first past learning model and the weight vector of a second past learning model learned in a past learning round of the first past learning model. Next, the switching section 300 derives the degree of change from the similarity between the first gradient vector and the second gradient vector.
[0165] At this time, the switching section 300 can calculate the similarity according to the following mathematical expression 1.
[0166] [Mathematical Expression 1]
[0167]
[0168] where CS represents the similarity. g t is the first gradient vector between the weight vector of the current learning model and the weight vector of the first past learning model. Further, g t-1 is the second gradient vector between the weight vector of the first past learning model and the weight vector of the second past learning model. Further, g is the transposed vector of the second gradient vector.
[0169] Next, the switching section 300 determines whether the degree of change is equal to or greater than the reference value in step S140. At this time, the switching section 300 determines that the degree of change is equal to or greater than the reference value in the case where the degree of similarity between the first gradient vector and the second gradient vector is less than the reference value, according to mathematical expression 1. In contrast, the switching section 300 determines that the degree of change is less than the reference value in the case where the degree of similarity between the first gradient vector and the second gradient vector is equal to or greater than the reference value, according to mathematical expression 1.
[0170] According to the determination result of step S140, in the case where the degree of change is less than the reference value (the degree of similarity is equal to or greater than the reference value), the switching section 300 erases the current learning dataset by executing step S150, and repeats steps S110 to S140 as described above.
[0171] In contrast, according to the determination result of step S140, in the case where the degree of change is equal to or greater than the reference value (the degree of similarity is less than the reference value), it is decided to update the buffer learning dataset and execute step S160.
[0172] The learning section 200 learns the gradient prediction model (GPM) in step S160. The gradient prediction model (GPM) can collect derivative learning data from the learning data on which the current learning model is learned, and predict the gradient vector of the future learning model using the collected derivative learning data. The learning of the gradient prediction model (GPM) will be described in more detail in the following content.
[0173] Next, the updating section 400 calculates the degree to which each of the plurality of learning data in the current learning dataset affects the prediction performance of the future learning model in step S170. Specifically, the degree to which each of the plurality of learning data in the current learning dataset affects the prediction performance of the future learning model is calculated when the future learning model, which is the learning model of the subsequent learning round of the current learning model, is learned using the current learning dataset in the continuous learning process. The step of calculating the degree of influence as described above will be described in more detail in the following content.
[0174] Next, the updating section 400 updates the buffer learning dataset by selecting a part of the plurality of learning data of the current learning dataset in order of the degree of influence from high to low according to the degree of influence in step S180. At this time, the updating section 400 deletes the buffer learning data stored in the buffer BF in advance, and stores the learning data selected from the plurality of learning data in order of the degree of influence from high to low in the buffer BF, thereby updating the buffer learning dataset. In addition, the remaining learning data not selected in order of the degree of influence from high to low among the plurality of learning data is erased in step S180.
[0175] By updating the buffer learning dataset according to the method as described above and simultaneously performing continuous learning of the learning model using the buffer learning dataset and the new learning dataset, it is possible to minimize the storage space while overcoming the catastrophic forgetting phenomenon, thereby continuously providing a learning model achieving high accuracy at the lowest cost.
[0176] Next, a learning method of a gradient prediction model (GPM) according to an embodiment of the present application will be described. Figure 5 is a flowchart for describing a learning method of a gradient prediction model according to an embodiment of the present application. Specifically, Figure 5 is a detailed description of step S160 in Figure 4
[0177] Referring to Figure 5 In step S210, the learning unit 200 is in a state of selecting an object layer of the current learning model. For example, it is assumed that the object layer is a convolution layer. In addition, in step S210, the learning unit 200 is in a state of dividing and learning a plurality of learning data in the current learning dataset in an epoch unit while learning the current learning model in step S120, and storing the modified weight vector of the object layer of the current learning model each time the division and learning in the epoch unit are performed.
[0178] By doing so, the learning unit 200 constructs learning data derived from the current learning dataset, i.e., a plurality of derived learning data, in step S220.
[0179] The derived learning data includes the modified weight vector of the object layer of the current learning model each time the current learning dataset is divided in an epoch unit and the iterative learning of the current learning model is performed, and the target gradient vector representing the difference between the weight vector of the previous epoch and the weight vector of the current epoch each time the current learning dataset is divided in an epoch unit and the iterative learning of the current learning model is performed.
[0180] The modified weight vector of the object layer is changed in the manner shown in the following mathematical expression 2 each time the plurality of learning data is divided in an epoch unit and the iterative learning is performed.
[0181]
Mathematical Expression 2
[0182] W(t+1) = W(t) + LR x G(t)
[0183] where t is an index of an epoch. LR represents a learning rate. Further, G(t) represents a gradient vector. The present application approximates a product of the learning rate LR and the gradient vector G(t) as a gradient vector. By this, the weight vector can be expressed as [W(1), W(2), W(3), …, W(E-1)] (E is an arbitrary positive constant). Further, the target gradient vector corresponding to [W(1), W(2), W(3), …, W(E-1)] can be expressed as [W(2)-W(1), [W(3)-W(2), [W(4)-W(3), …, [W(E)-W(E-1)].
[0184] Next, the learning unit 200 inputs the weight vector in the derived learning data to the gradient prediction model (GPM) having the weight for which the learning is not completed in step S230. Further, the gradient prediction model (GPM) performs the weight operation of applying the weight for which the learning is not completed to the learning radiological image in step S240, thereby generating the weight difference vector that predicts the difference between the weight vector of the previous epoch and the weight vector of the object layer of the current epoch.
[0185] Next, the learning unit 200 calculates the loss that represents the difference between the target gradient vector and the weight difference vector by the loss function in step S250. Next, the learning unit 200 performs the optimization of modifying the weight of the gradient prediction model (GPM) in step S260, thereby minimizing the loss derived by the loss function.
[0186] Next, the learning unit 200 determines whether the learning completion condition is satisfied in step S270. In an embodiment, the learning completion condition can be a case where the loss calculated in the step S250 converges and reaches below a target value set in advance. According to the determination result of the step S270 as described above, in a case where the learning completion condition is not satisfied, the steps S230 to S270 as described above are repeatedly performed using different multiple learning data. In contrast, according to the determination result of the step S270, in a case where the learning completion condition is satisfied, the learning unit 200 completes the learning of the gradient prediction model (GPM) in step S280.
[0187] Next, a method of calculating the degree to which each of the multiple learning data in the current learning data set according to the embodiment of the present application affects the prediction performance of the future learning model will be described. Figure 6 is a flowchart for describing a method of calculating the degree to which the learning data according to the embodiment of the present application affects the prediction performance of the future learning model. That is, Figure 6 is a detailed description of the step S170 in Figure 4 .
[0188] The update unit 400 calculates, in step S310, a plasticity score indicating a probability that a prediction value of the current learning model and a prediction value of the future learning model differ, for the learning data in the current learning data set.
[0189] At this time, the update unit 400 can calculate the plasticity score in accordance with the following mathematical expression 3.
[0190] [Mathematical expression 3]
[0191]
[0192] wherein, is a conditional probability indicating the plasticity score. n is an index of the learning data in the current learning data set. A superscript such as t is an index of a learning round in the continuous learning process (may also be an index of the learning model), t indicating the current learning round and t-1 indicating the first past learning round. is the learning data in the current learning data set. Further, is the prediction value of the future learning model, and indicates the prediction value of the current learning model.
[0193] Further, because the prediction value of the future learning model is generated before the future learning model is generated, it is estimated using a gradient prediction model (GPM) through a predicted gradient of the future learning model. By this, the prediction value of the future learning model in the mathematical expression 3 will be estimated in accordance with the following mathematical expression 4.
[0194] [Mathematical expression 4]
[0195]
[0196] wherein, is the learning data in the current learning data set. Further, is the prediction value of the future learning model, and the GP(θ t ) is a gradient vector predicted by the gradient prediction model (GPM) for the future learning model.
[0197] Next, the update unit 400 calculates, in step S320, a stability score indicating a probability that a prediction value of the current learning model and a prediction value of a first past learning model learned in a previous learning round of the current learning model, i.e., the first past learning round, differ, for the learning data in the current learning data set.
[0198] At this time, the update unit 400 can calculate the stability score in accordance with the following mathematical expression 5.
[0199] [Mathematical expression 5]
[0200]
[0201] wherein, is a conditional probability representing a stability score.
[0202] n is an index of learning data in a current learning dataset. Further, a superscript such as t is an index of a learning round in a continuous learning process, t indicating a current learning round and t-1 indicating a first past learning round. represents learning data in a current learning dataset. Further, is a predicted value of a first past learning model, and the represents a predicted value of a current learning model.
[0203] Next, the update section 400 calculates a weighted average of the plasticity score and the stability score as the influence degree in step S330.
[0204] At this time, the update section 400 can calculate the influence degree according to the following mathematical expression 6.
[0205]
Mathematical Expression 6
[0206] S i = λ · Plasticity i + (1 - λ) · Stability i
[0207] wherein, S represents the influence degree. i is an index of learning data in a current learning dataset. λ is a weight set in advance. Plasticity represents a plasticity score, and Stability represents a stability score.
[0208] Next, a method of diagnosing a defect using a learning model based on continuous learning according to an embodiment of the present application will be described. Figure 7 is a flowchart for describing a method of diagnosing a defect using a learning model based on continuous learning according to an embodiment of the present application. Figure 8 is a screen example for describing a method of diagnosing a defect using a learning model based on continuous learning according to an embodiment of the present application.
[0209] In Figure 7 , the learning model as described in the step S120 is a current learning model that is learned using a previously stored buffer learning dataset and a current learning dataset.
[0210] Referring to Figure 7The radiographic apparatus RTA performs radiography on the object body on which the image quality indicator (IQI) is installed in step S410, thereby generating a radiographic film on which the object body on which the image quality indicator is installed is photographed. The object body can be exemplified by a pipe, a hose, and the like.
[0211] Next, the scanner SC scans the radiographic film in step S420, thereby generating a digitized radiographic image.
[0212] After the digitized radiographic image is generated as described above, the generated radiographic image is input to the inspection apparatus 10. At this time, the radiographic image can be input directly from the scanner SC or input from a corresponding storage medium after being stored in the storage medium.
[0213] The detection section 500 of the inspection apparatus 10 receives input of the radiographic image in step S430. Next, the detection section 500 detects a defect from the radiographic image using a current learning model, which is the latest model in the learning models trained by the continuous learning practice, in step S440 in the manner described in the above context. At this time, the current learning model can detect an area occupied by the defect from the radiographic image by a bounding box (BB) as shown in Figure 8
[0214] Next, the detection section 500 outputs a report including the radiographic image in which the defect is detected by the bounding box (BB) in step S450.
[0215] Figure 9 is a schematic diagram illustrating a computing device according to an embodiment of the present application. Figure 9 The computing device TN100 in
[0216] In the embodiment of Figure 9 The computing device TN100 can include at least one processor TN110, a transceiver TN120, and a memory TN130. In addition, the computing device TN100 can further include a storage device TN140, an input interface device TN150, and an output interface device TN160. The constituent elements included in the computing device TN100 can be connected by a bus TN170 and perform communication with each other.
[0217] The processor TN110 can execute program commands stored in at least one of the memory TN130 and the storage TN140. The processor TN110 can be a central processing device (CPU), a graphics processing device (GPU), or a dedicated processor that executes a method according to an embodiment of the present application. The processor TN110 can be configured to implement the steps, functions, and methods described in connection with the embodiment of the present application. The processor TN110 can control each constituent element of the computing device TN100.
[0218] The memory TN130 and the storage TN140 can each store various information related to the operation of the processor TN110. The memory TN130 and the storage TN140 can each be configured by at least one of a volatile storage medium and a non-volatile storage medium. For example, the memory TN130 can be configured by at least one of a read only memory (ROM) and a random access memory (RAM).
[0219] The transceiver TN120 can transmit or receive a wired signal or a wireless signal. The transceiver TN120 can perform communication by being connected to a network.
[0220] In particular, the collecting part 100, the learning part 200, the switching part 300, the updating part 400, and the detecting part 500 of the inspection device 10 according to the embodiment of the present application can be implemented in a program form readable by a computer means and stored in the memory TN130, and further executed by the processor TN110. Alternatively, the collecting part 100, the learning part 200, the switching part 300, the updating part 400, and the detecting part 500 can be sub-modules of the processor TN110.
[0221] Further, the buffer area BF according to the embodiment of the present application can be the memory TN130 or the storage TN140 of the inspection device 10 or the storage device 40.
[0222] Further, the various methods according to the embodiments of the present application described above can be implemented in a program code of a computer readable program and recorded in a computer readable recording medium. The recording medium can include a program instruction, a data file, a data structure, and the like, or a combination thereof. The program instruction recorded in the recording medium can be specially designed and configured for the present application, or can be known and used by a computer software professional. For example, the recording medium includes a magnetic medium such as a hard disk and a floppy disk, a magnetic tape, an optical medium such as a compact disk read only memory (CD-ROM) and a digital versatile disk (DVD), a magneto-optical medium such as a floptical disk, a read only memory (ROM), a random access memory (RAM), and a flash memory, and the like, which are specially configured to store and execute the program instruction. Examples of the program instruction can include not only a machine code generated by a compiler but also a high level language code that can be executed by an interpreter or the like on a computer. The hardware device described above can be configured to work with one or more software modules in order to perform the operation of the present application, and vice versa.
[0223] In the above, an embodiment of the present application has been described, but a person having ordinary knowledge in the related technical field can make various modifications and changes to the present application by adding, changing, deleting, or adding elements, etc. within the scope of the idea of the present application described in the claims, and such modifications and changes should be understood to be included in the scope of the claims of the present application.
Claims
1. A method for updating learning data, characterized in that, include: The steps involve the learning department using a buffered learning dataset stored in a buffer beforehand and the current learning dataset to learn the current learning model; The step of calculating the degree of change from the previous learning model, which is more than two learning rounds away from the current learning model in the continuous learning rounds, to the current learning model; The step of determining whether the degree of change reaches or exceeds a reference value by the switching unit; The step of the switching unit deciding to update the buffered learning dataset when the degree of change reaches or exceeds the benchmark value; as well as, The step of the update unit extracting at least a portion of the learning data from the current learning dataset and updating the buffer learning dataset.
2. The method for updating learning data according to claim 1, characterized in that: The steps to calculate the degree of change include: The switching unit derives the degree of change based on the similarity between a first gradient vector representing the change between the weight vector of the current learning model and the weight vector of a first past learning model learned in a previous learning round of the current learning model, and a second gradient vector representing the change between the weight vector of the first past learning model and the weight vector of a second past learning model learned in a previous learning round of the first past learning model.
3. The method for updating learning data according to claim 2, characterized in that: The degree of similarity is based on the mathematical formula Perform calculations. The CS refers to the degree of similarity. The g t The first gradient vector is the gradient between the weight vector of the current learning model and the weight vector of the first past learning model. The g t-1 The second gradient vector is the gradient between the weight vector of the first past learning model and the weight vector of the second past learning model. The is the transpose of the second gradient vector.
4. The method for updating learning data according to claim 2, characterized in that: The steps for determining whether the degree of change reaches or exceeds the benchmark value include: If the similarity between the first gradient vector and the second gradient vector is less than the reference value, the switching unit determines that the degree of change reaches or exceeds the reference value.
5. The method for updating learning data according to claim 1, characterized in that: After determining whether the degree of change reaches or exceeds a benchmark value, the method further includes: If the degree of change is insufficient compared to the baseline value, The step of erasing the current learning dataset by the switching unit.
6. The method for updating learning data according to claim 1, characterized in that: The buffered learning dataset, which is stored in the buffer beforehand, is learning data selected from past learning datasets during continuous learning based on the degree to which it affects the predictive performance of the current learning model.
7. A learning data updating device, characterized in that, include: The learning department uses the buffered learning dataset stored in the buffer beforehand and the current learning dataset to learn the current learning model; The switching unit calculates the degree of change from the past learning model (which is more than two learning rounds away from the current learning model in consecutive learning rounds) to the current learning model. After determining whether the degree of change has reached or exceeded the benchmark value... If the degree of change reaches or exceeds the baseline value, it is decided to update the buffered learning dataset; and, The updating unit extracts at least a portion of the learning data from the current learning dataset and updates the buffered learning dataset.
8. The learning data updating device according to claim 7, characterized in that: The switching unit derives the degree of change based on the similarity between a first gradient vector representing the change between the weight vector of the current learning model and the weight vector of a first past learning model learned in a previous learning round of the current learning model, and a second gradient vector representing the change between the weight vector of the first past learning model and the weight vector of a second past learning model learned in a previous learning round of the first past learning model.
9. The learning data updating device according to claim 8, characterized in that: The switching unit is based on the mathematical formula Calculate the degree of similarity. The CS refers to the degree of similarity. The g t The first gradient vector is the gradient between the weight vector of the current learning model and the weight vector of the first past learning model. The g t-1 The second gradient vector is the gradient between the weight vector of the first past learning model and the weight vector of the second past learning model. The is the transpose of the second gradient vector.
10. The learning data updating device according to claim 8, characterized in that: If the similarity between the first gradient vector and the second gradient vector is less than the reference value, the switching unit determines that the degree of change reaches or exceeds the reference value.
11. The learning data updating device according to claim 7, characterized in that: If the degree of change is insufficient for the baseline value, the switching unit erases the current learning dataset.
12. The learning data updating device according to claim 7, characterized in that: The buffered learning dataset, which is stored in the buffer beforehand, is learning data selected from past learning datasets during continuous learning based on the degree to which it affects the predictive performance of the current learning model.