Method for product inspection using artificial neural network model based on unsupervised and supervised learning

Abstract

According to one embodiment of the present disclosure, a method is disclosed in which a computing device performs to inspect a product using an artificial neural network model. Specifically, according to the present disclosure, the computing device identifies input data including an image of a product, generates a common feature vector based on the input data using an encoder module, generates a first inspection result data based on inputting at least a portion of the common feature vector to a first artificial neural network model based on supervised learning, generates a second inspection result data based on inputting at least a portion of the common feature vector to a second artificial neural network model based on unsupervised learning, and generates a third inspection result data based on the first inspection result data and the second inspection result data.

Description

Technology Field

[0001] The present disclosure relates to a product inspection method utilizing unsupervised learning and supervised learning-based artificial neural network models. Specifically, it relates to a method of generating a common feature vector that reflects global and local features of input data including an image of a product by utilizing an encoder module, and generating product inspection result data by utilizing both a first artificial neural network model based on supervised learning and a second artificial neural network model based on unsupervised learning. Background Technology

[0002] With the advancement of computer vision and artificial intelligence technologies, there is an increasing number of attempts to address the process of detecting product defects in manufacturing environments by utilizing computer vision. Conventional segmentation models that identify defect regions in product images primarily operate using a closed-set segmentation method, which predicts only the classes within a given dataset. While this approach may be effective for predefined defect types, it has limitations in real-world environments like manufacturing, where new, unpredicted defect patterns (unseen defects) can continuously occur. It is difficult for existing models to accurately detect these unlearned defects.

[0003] One approach proposed to address this is an unsupervised learning-based reconstruction technique; however, since this method learns only from normal images and utilizes reconstruction errors for anomaly detection, it suffers from issues such as a lack of harmony with existing classes or low segmentation accuracy. Furthermore, operating supervised and unsupervised learning models as separate structures imposes limitations on computational efficiency and consistency in result interpretation between models.

[0004] Therefore, there is a demand in the industry for Open-Set based inspection systems capable of effectively detecting even undefined defects.

[0005] Korean registered patent KR 2804390 discloses a product inspection system capable of artificial intelligence self-learning by combining quality inspection technology using inspection standards and quality inspection technology based on artificial intelligence. The problem to be solved

[0006] The present disclosure is devised in response to the aforementioned background technology and, specifically, relates to a method for generating a common feature vector that reflects global and local features of input data including an image of a product by utilizing an encoder module, and generating product inspection result data by utilizing both a first artificial neural network model based on supervised learning and a second artificial neural network model based on unsupervised learning.

[0007] Meanwhile, the technical problem that the present disclosure aims to solve is not limited to the technical problem mentioned above, and various technical problems may be included within the scope obvious to a person skilled in the art from the contents described below. means of solving the problem

[0008] According to an embodiment of the present disclosure for realizing the aforementioned task, a method is disclosed that is performed by a computing device to inspect a product using an artificial neural network model. The method may include the steps of: identifying input data including an image of a product; generating a common feature vector based on the input data using an encoder module; generating a first inspection result data based on inputting at least a portion of the common feature vector to a first artificial neural network model based on supervised learning; generating a second inspection result data based on inputting at least a portion of the common feature vector to a second artificial neural network model based on unsupervised learning; and generating a third inspection result data based on the first inspection result data and the second inspection result data.

[0009] In one embodiment, the encoder module may include a first submodule based on a convolutional neural network; and a second submodule based on a vision transformer.

[0010] In one embodiment, the step of generating a common feature vector based on the input data using an encoder module may include: a step of generating a local feature vector based on the input data using the first submodule; a step of generating a global feature vector based on the local feature vector using the second submodule; and a step of generating the common feature vector based on fusing the local feature vector and the global feature vector.

[0011] In one embodiment, the step of generating the common feature vector based on fusing the local feature vector and the global feature vector may include: identifying the type of artificial neural network model to which the common feature vector is input; setting weights for each of the local feature vector and the global feature vector based on the type of artificial neural network model to which the common feature vector is input; and generating the common feature vector based on the local feature vector, the global feature vector, and the weights.

[0012] In one embodiment, the step of setting weights for each of the local feature vector and the global feature vector based on the type of artificial neural network model to which the common feature vector is input may include: increasing the weight of the local feature vector when the artificial neural network model to which the common feature vector is input is a supervised learning-based artificial neural network model, and increasing the weight of the global feature vector when the artificial neural network model to which the common feature vector is input is an unsupervised learning-based artificial neural network model.

[0013] In one embodiment, the first artificial neural network model may correspond to an artificial neural network model trained to classify product images containing defects and perform segmentation on defect regions.

[0014] In one embodiment, the second artificial neural network model may correspond to an artificial neural network model trained to classify product images containing defects using product images not containing defects as training data.

[0015] In one embodiment, the first inspection result data includes segmentation data for the image of the product; and class classification data for the image of the product, and the second inspection result data may include class classification data for the image of the product.

[0016] In one embodiment, the step of generating the third inspection result data based on the first inspection result data and the second inspection result data may include: determining the first inspection result data as the third inspection result data when the class classification data included in the first inspection result data and the second inspection result data match; determining the first inspection result data as the third inspection result data when the class classification data included in the first inspection result data includes a defect class; and determining the second inspection result data as the third inspection result data when the class classification data included in the first inspection result data does not include a defect class and the class classification data included in the second inspection result data includes a defect class.

[0017] In one embodiment, if the class classification data included in the first inspection result data does not include a defect class and the class classification data included in the second inspection result data includes a defect class, the method may further include the step of additionally training the first artificial neural network model based on the input data after the step of determining the second inspection result data as the third inspection result data.

[0018] According to an embodiment of the present disclosure for realizing the aforementioned task, a computer program stored in a computer-readable storage medium is disclosed, which enables a computing device to perform operations for inspecting a product using an artificial neural network model. The operations may include: an operation of identifying input data including an image of a product; an operation of generating a common feature vector based on the input data using an encoder module; an operation of generating a first inspection result data based on inputting at least a portion of the common feature vector to a first artificial neural network model based on supervised learning; an operation of generating a second inspection result data based on inputting at least a portion of the common feature vector to a second artificial neural network model based on unsupervised learning; and an operation of generating a third inspection result data based on the first inspection result data and the second inspection result data.

[0019] According to an embodiment of the present disclosure for realizing the aforementioned objectives, a computing device for inspecting a product using an artificial neural network model is disclosed. The computing device may be configured to include one or more processors and memory, wherein the one or more processors identify input data including an image of a product, generate a common feature vector based on the input data using an encoder module, generate a first inspection result data based on inputting at least a portion of the common feature vector to a first artificial neural network model based on supervised learning, generate a second inspection result data based on inputting at least a portion of the common feature vector to a second artificial neural network model based on unsupervised learning, and generate a third inspection result data based on the first inspection result data and the second inspection result data. Effects of the invention

[0020] According to the present disclosure, by combining a supervised learning-based model and an unsupervised learning-based model through a single common encoder, it is possible to effectively detect not only previously defined defects but also new types of defects that have not been learned. Furthermore, by generating a common feature vector that fuses local and global features to provide information optimized for each learning method, a significant improvement in segmentation precision and anomaly detection performance occurs. In addition, by integrating supervised and unsupervised learning results to derive inspection results, there is an advantage of increasing the reliability of defect identification and enabling efficient application to real-time quality management.

[0021] Meanwhile, the effects of the present disclosure are not limited to those mentioned above, and various effects may be included within the scope obvious to a person skilled in the art from the contents described below. Brief explanation of the drawing

[0022] FIG. 1 is a block diagram of a computing device for inspecting a product using an artificial neural network model according to one embodiment of the present disclosure. FIG. 2 is a conceptual diagram showing an artificial neural network according to one embodiment of the present disclosure. FIG. 3 is a flowchart illustrating the process of inspecting a product using an artificial neural network model according to one embodiment of the present disclosure. FIG. 4 is a conceptual diagram illustrating a process of inspecting a product using an artificial neural network model according to one embodiment of the present disclosure. FIG. 5 is a conceptual diagram showing the detailed configuration of an encoder module according to one embodiment of the present disclosure. FIG. 6 is a conceptual diagram illustrating the process of generating third inspection result data of a product according to one embodiment of the present disclosure. FIG. 7 is a brief and general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented. Specific details for implementing the invention

[0023] The present disclosure aims to generate a common feature vector that reflects global and local features of input data including an image of a product by utilizing an encoder module, and to generate product inspection result data by utilizing both a first artificial neural network model based on supervised learning and a second artificial neural network model based on unsupervised learning.

[0024] Various embodiments are now described with reference to the drawings. In this specification, various descriptions are provided to provide an understanding of the present disclosure. However, it is evident that these embodiments can be practiced without such specific descriptions.

[0025] As used herein, terms such as “component,” “module,” “system,” etc. refer to computer-related entities, hardware, firmware, software, combinations of software and hardware, or executions of software. For example, a component may be a procedure executed on a processor, a processor, an object, an execution thread, a program, and / or a computer, but is not limited thereto. For example, both an application executed on a computing device and the computing device itself may be a component. One or more components may reside within a processor and / or an execution thread. One component may be localized within a single computer. In this disclosure, one or more components may be distributed between two or more computers, i.e., between execution environments. Additionally, these components may be executed from various computer-readable media having various data structures stored therein. Components may communicate through local and / or remote processes, for example, according to signals having one or more data packets (e.g., data from one component interacting with another component in a local system or distributed system, and / or data transmitted through signals to other systems and networks such as the Internet).

[0026] Furthermore, the term "or" is intended to mean an implicit "or" rather than an exclusive "or." That is, unless otherwise specified or evident from the context, "X uses A or B" is intended to mean one of the natural implicit substitutions. In other words, if X uses A; if X uses B; or if X uses both A and B, "X uses A or B" may apply to any of these cases. Additionally, the term "and / or" as used herein should be understood to refer to and include all possible combinations of one or more of the enumerated related items.

[0027] Additionally, the terms “comprising” and / or “comprising” should be understood to mean that such features and / or components are present. However, the terms “comprising” and / or “comprising” should be understood not to exclude the presence or addition of one or more other features, components and / or groups thereof. Furthermore, unless otherwise specified or clearly evident from the context to indicate a singular form, the singular in this specification and claims should generally be interpreted to mean “one or more.”

[0028] And, the term “at least one of A or B” should be interpreted to mean “a case including only A,” “a case including only B,” or “a combination of A and B.”

[0029] Those skilled in the art should recognize that the various exemplary logical blocks, configurations, modules, circuits, means, logics, and algorithmic steps described in connection with the embodiments disclosed herein may be implemented in electronic hardware, computer software, or a combination of both. To clearly exemplify the interchangeability of hardware and software, various exemplary components, blocks, configurations, means, logics, modules, circuits, and steps have been generally described above in terms of their functionality. Whether such functionality is implemented in hardware or software depends on the specific application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in various ways for each specific application. However, such decisions regarding implementation should not be construed as going beyond the scope of this disclosure.

[0030] Description of the presented embodiments is provided to enable those skilled in the art to use or practice the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art. The general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure is not limited to the embodiments presented herein. The present disclosure should be interpreted in the broadest possible scope consistent with the principles and novel features presented herein.

[0031] FIG. 1 is a block diagram of a computing device for inspecting a product using an artificial neural network model according to one embodiment of the present disclosure.

[0032] The configuration of the computing device (100) illustrated in FIG. 1 is merely a simplified example. In one embodiment of the present disclosure, the computing device (100) may include other configurations for performing the computing environment of the computing device (100), and only some of the disclosed configurations may constitute the computing device (100).

[0033] The computing device (100) may include a processor (110), memory (130), and a network unit (150).

[0034] The processor (110) may be composed of one or more cores and may include a processor for data analysis and deep learning, such as a central processing unit (CPU) of a computing device, a general purpose graphics processing unit (GPGPU), or a tensor processing unit (TPU). The processor (110) may read a computer program stored in memory (130) and perform data processing for machine learning according to one embodiment of the present disclosure. According to one embodiment of the present disclosure, the processor (110) may perform calculations for learning a neural network. The processor (110) may perform calculations for learning a neural network, such as processing input data for learning in deep learning (DL), extracting features from input data, calculating errors, and updating the weights of the neural network using backpropagation.

[0035] At least one of the CPU, GPGPU, and TPU of the processor (110) can process the learning of a network function. For example, the CPU and GPGPU can together process the learning of a network function and data classification using the network function. In addition, in one embodiment of the present disclosure, processors of a plurality of computing devices can be used together to process the learning of a network function and data classification using the network function. In addition, a computer program executed in a computing device according to one embodiment of the present disclosure may be a CPU, GPGPU, or TPU executable program.

[0036] In the present disclosure, "supervised learning" may refer to a method of training an artificial neural network model by utilizing input data and label information for that input together, thereby enabling the model to learn the relationship between the input and the label. Supervised learning requires label data during the training process, and the trained model can output a prediction result for a given input when a new input is provided. For example, if product images and label masks for defect regions corresponding to each image are provided together, training a segmentation model that accurately predicts defect locations using such data constitutes supervised learning.

[0037] In the present disclosure, "unsupervised learning" may refer to a learning method in which a model learns based on similarities or anomalies between inputs, without explicit ground truth information regarding the input data. Unsupervised learning is useful for processing new types of data or unlabeled data because it allows a model to be trained without ground truth data.

[0038] The processor (110) can identify input data including an image of a product. In the present disclosure, the input data may include a high-resolution image of the product, or may include a set of image patches created by dividing the high-resolution image of the product into units.

[0039] The processor (110) can generate a common feature vector based on input data by utilizing an encoder module. The encoder module of the present disclosure may include a first submodule based on a convolutional neural network and a second submodule based on a Vision Transformer. "Vision Transformer" may refer to a transformer-based neural network structure for receiving image data as input and learning global contextual information of the entire image.

[0040] The processor (110) can generate a local feature vector based on input data by first utilizing a first submodule in order to generate a common feature vector by utilizing an encoder module. The local feature vector may refer to data in the form of a vector extracted to reflect the local features of the input data well.

[0041] The processor (110) can generate a global feature vector based on a local feature vector by utilizing a second submodule. In the present disclosure, a global feature vector may refer to data in the form of a vector extracted to better reflect the features of the entire image rather than detailed regions of the image. Specifically, the global feature vector and the local feature vector may each have a real value between 0 and 1. For example, for an input image, the local feature vector may be extracted as {1, 0, 0, 1, 0} and the global feature vector as {0, 0, 0, 1, 1}.

[0042] That is, the processor (110) can generate local feature vectors by utilizing a first submodule based on a convolutional neural network and generate global feature vectors by utilizing a second submodule based on a vision transformer.

[0043] The processor (110) can generate a common feature vector based on fusing a local feature vector and a global feature vector. In the present disclosure, a common feature vector may refer to a vector obtained by appropriately weighting the sum or concatenating the two vectors to simultaneously utilize the local features and global features of the input data. For example, when the local feature vector is {1, 0, 0, 1, 0} and the global feature vector is {0, 0, 0, 1, 1}, the processor (110) can generate a common feature vector {1, 0, 0, 1, 0, 0, 0, 0, 1, 1} based on concatenating the two vectors. As another example, the processor (110) can generate a common feature vector {0.5, 0, 0, 1, 0.5} based on setting the weights of the two vectors to 0.5 each and performing a weighted sum when the local feature vector is {1, 0, 0, 1, 0} and the global feature vector is {0, 0, 0, 1, 1}.

[0044] When generating a common feature vector by weighted summing local feature vectors and global feature vectors, the processor (110) can generate two versions of the common feature vector and input each version of the common feature vector into different artificial neural network models. For example, the processor (110) can generate a first common feature vector and a second common feature vector by setting the weights of the local feature vector and the global feature vector differently. If the artificial neural network model to which the first common feature vector is input is a first artificial neural network model based on supervised learning, the processor (110) can increase the weight of the local feature vector during the process of generating the first common feature vector. Conversely, if the artificial neural network model to which the second common feature vector is input is a second artificial neural network model based on unsupervised learning, the processor (110) can increase the weight of the global feature vector during the process of generating the second common feature vector.

[0045] Specifically, when the local feature vector is {1, 0, 0, 1, 0} and the global feature vector is {0, 0, 0, 1, 1} and the artificial neural network model to which the common feature vector is input is a first artificial neural network model based on supervised learning, the processor (110) can generate the common feature vector based on weighted summing the two vectors by setting the weights of the local feature vector and the global feature vector to 0.7 and 0.3. That is, in this case, the common feature vector may be {0.7, 0, 0, 1, 0.3}.

[0046] Meanwhile, if the artificial neural network model to which the common feature vector is input is a second artificial neural network model based on unsupervised learning, the processor (110) can generate the common feature vector based on weighted summing the two vectors by setting the weights of the local feature vector and the global feature vector to 0.3 and 0.7. That is, in this case, the common feature vector may be {0.3, 0, 0, 1, 0.7}.

[0047] The processor (110) can generate first inspection result data based on inputting at least a portion of a common feature vector into a first artificial neural network model based on supervised learning. For example, if the processor (110) generates a common feature vector based on concatenating a local feature vector and a global feature vector, it can generate first inspection result data based on inputting only the portion of the common feature vector corresponding to the local feature vector into the first artificial neural network model. As another example, if the processor (110) generates a common feature vector based on weighted summing a local feature vector and a global feature vector, it can generate first inspection result data based on inputting the entire common feature vector into the first artificial neural network model.

[0048] Meanwhile, the processor (110) can generate second inspection result data based on inputting at least a portion of the common feature vector into a second artificial neural network model based on unsupervised learning. For example, if the processor (110) generates a common feature vector based on concatenating a local feature vector and a global feature vector, it can generate second inspection result data based on inputting only the portion of the common feature vector corresponding to the global feature vector into the second artificial neural network model. As another example, if the processor (110) generates a common feature vector based on weighted summing a local feature vector and a global feature vector, it can generate second inspection result data based on inputting the entire common feature vector into the second artificial neural network model.

[0049] In the present disclosure, the first artificial neural network model may be a supervised learning-based artificial neural network model. The first artificial neural network model may be trained to classify product images containing defects and to perform segmentation on defect regions. In the case of supervised learning, since each data includes label data containing information not only on whether defects are included but also on the type of defect, the first artificial neural network model may classify product images by reflecting the type of defect when the input data contains defects.

[0050] In the present disclosure, the second artificial neural network model may be an unsupervised learning-based artificial neural network model. The second artificial neural network model may be an artificial neural network model trained using only product images that do not contain defects, i.e., normal product images, as training data. The second artificial neural network model may be trained to classify input data based on reconstruction error, thereby classifying data of unlearned types as defective images.

[0051] The first inspection result data generated using a first artificial neural network model based on supervised learning may include class classification data for product images, and if the product contains defects, it may include segmentation data for defective areas.

[0052] The second inspection result data generated using a second artificial neural network model based on unsupervised learning may include class classification data for product images.

[0053] The processor (110) can generate third test result data based on first test result data and second test result data. In the present disclosure, third test result data may refer to data including a final test result for input data generated by combining first test result data and second test result data. A specific method for generating third result data will be described later with reference to FIG. 6.

[0054] According to the present disclosure, by using a first artificial neural network model, which is a supervised learning-based artificial neural network model, and a second artificial neural network model, which is an unsupervised learning-based artificial neural network model in parallel, it is possible to effectively detect not only previously defined defects but also new types of defects that have not been learned. Furthermore, by generating a common feature vector that fuses local and global features through an encoder module and providing information optimized for each learning method, a significant effect occurs in improving segmentation precision and anomaly detection performance.

[0056] According to one embodiment of the present disclosure, the memory (130) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory), RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, a magnetic disk, and an optical disk. The computing device (100) may operate in conjunction with web storage that performs the storage function of the memory (130) on the internet. The description of the memory described above is merely an example and the present disclosure is not limited thereto.

[0057] A network unit (150) according to one embodiment of the present disclosure can use various wired communication systems such as a public switched telephone network (PSTN), xDSL (x Digital Subscriber Line), RADSL (Rate Adaptive DSL), MDSL (Multi Rate DSL), VDSL (Very High Speed ​​DSL), UADSL (Universal Asymmetric DSL), HDSL (High Bit Rate DSL), and a local area network (LAN).

[0058] Additionally, the network unit (150) presented in this disclosure may use various wireless communication systems such as CDMA (Code Division Multi Access), TDMA (Time Division Multi Access), FDMA (Frequency Division Multi Access), OFDMA (Orthogonal Frequency Division Multi Access), SC-FDMA (Single Carrier-FDMA), and other systems.

[0059] In the present disclosure, the network unit (150) can be configured regardless of the mode of communication, such as wired and wireless, and can be configured as various communication networks, such as a Personal Area Network (PAN) or a Wide Area Network (WAN). In addition, the network may be a known World Wide Web (WWW) and may utilize wireless transmission technology used for short-range communication, such as Infrared Data Association (IrDA) or Bluetooth. The technologies described in the present disclosure may also be used in other networks mentioned above.

[0061] FIG. 2 is a conceptual diagram showing an artificial neural network according to one embodiment of the present disclosure.

[0062] Throughout this disclosure, neural network, neural network model, and neural network may be used interchangeably. A neural network may consist of a set of interconnected computational units, which may generally be referred to as nodes. These nodes may also be referred to as neurons. A neural network is composed of at least one node. The nodes (or neurons) constituting the neural networks may be interconnected by one or more links.

[0063] In a neural network, one or more nodes connected via links can form relative input and output node relationships. The concepts of input and output nodes are relative; any node in an output node relationship with respect to one node may be in an input node relationship with respect to another node, and vice versa. As described above, the input node versus output node relationship can be generated based on links. One or more output nodes may be connected to a single input node via links, and vice versa.

[0064] In a relationship between an input node and an output node connected through a single link, the value of the output node's data can be determined based on the data input to the input node. Here, the link interconnecting the input node and the output node may have a weight (at this time, parameters and weights may be used interchangeably throughout this disclosure). The weight may be variable and may be varied by a user or an algorithm to enable the neural network to perform the desired function. For example, if one or more input nodes are interconnected to a single output node by respective links, the output node may determine its value based on the values ​​input to the input nodes connected to the output node and the weights set on the links corresponding to each input node.

[0065] As described above, a neural network consists of one or more nodes interconnected through one or more links, forming input-output node relationships within the network. The characteristics of a neural network can be determined by the number of nodes and links within the network, the relationships between the nodes and links, and the weight values ​​assigned to each link. For example, if two neural networks exist with the same number of nodes and links but different weight values ​​for the links, the two neural networks may be recognized as different from each other.

[0066] A neural network can be composed of a set of one or more nodes. A subset of nodes constituting a neural network can form a layer. Some of the nodes constituting a neural network can form a layer based on their distances from an initial input node. For example, a set of nodes with a distance of n from an initial input node can form n layers. The distance from the initial input node can be defined by the minimum number of links that must be traversed to reach that node from the initial input node. However, this definition of a layer is arbitrary for illustrative purposes, and the degree of a layer within a neural network can be defined in a way different from that described above. For example, a layer of nodes may be defined by its distance from a final output node.

[0067] Initial input nodes may refer to one or more nodes within a neural network to which data is directly input without passing through links in their relationships with other nodes. Alternatively, in terms of link-based relationships between nodes within the neural network, they may refer to nodes that do not have other input nodes connected by links. Similarly, final output nodes may refer to one or more nodes within a neural network that do not have output nodes in their relationships with other nodes. Furthermore, hidden nodes may refer to nodes constituting the neural network that are neither initial input nodes nor final output nodes.

[0068] A neural network model including a neural network can be trained in at least one of supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. Training of a neural network model may be a process of applying knowledge to the neural network model to perform a specific action.

[0069] Neural network models can be trained to minimize the error in their output. The training process involves repeatedly inputting training data into the model, calculating the error between the model's output and the target for the training data, and updating the weights of each node in the model by backpropagating the error from the output layer to the input layer in a direction that reduces the error. In the case of supervised learning, training data is used where the correct answer is labeled for each data point (i.e., labeled training data), whereas in the case of unsupervised learning, the correct answer may not be labeled for each training data point. For instance, in the case of supervised learning for data classification, the training data may consist of data where each training point is labeled with a category. Labeled training data is input into the neural network model, and the error can be calculated by comparing the model's output (category) with the labels of the training data. As another example, in the case of unsupervised learning for data classification, the error can be calculated by comparing the input training data with the neural network model's output. The calculated error is backpropagated in the neural network model (i.e., from the output layer to the input layer), and through backpropagation, the connection weights of each node in each layer of the neural network model can be updated. The amount of change in the connection weights of each updated node can be determined by the learning rate. The computation of the neural network model on input data and the backpropagation of the error can constitute an epoch. The learning rate can be applied differently depending on the number of iterations of the neural network model's epoch. For example, a high learning rate can be used in the early stages of training to quickly achieve a certain level of performance and increase efficiency, while a low learning rate can be used in the later stages to improve accuracy.

[0070] In the training of neural network models, training data is generally a subset of real-world data (i.e., the data intended to be processed using the trained neural network model). Consequently, there may exist epochs where errors decrease on the training data but increase on the real data. Overfitting is a phenomenon where the model learns excessively on the training data, leading to an increase in errors on real-world data. For example, a neural network model trained on yellow cats might fail to recognize cats other than yellow ones as cats, which can be considered a type of overfitting. Overfitting can act as a cause for increased errors in machine learning algorithms. Various optimization methods can be used to prevent this overfitting. To prevent overfitting, methods such as increasing the training data, regularization, dropout (which disables some nodes in the network during training), and batch normalization can be applied.

[0072] FIG. 3 is a flowchart illustrating the process of inspecting a product using an artificial neural network model according to one embodiment of the present disclosure.

[0073] According to FIG. 3, a process of inspecting a product using an artificial neural network model according to one embodiment of the present disclosure may include the steps of: identifying input data including an image of the product (S310); generating a common feature vector based on the input data using an encoder module (S320); generating a first inspection result data based on inputting at least a portion of the common feature vector to a first artificial neural network model based on supervised learning (S330); generating a second inspection result data based on inputting at least a portion of the common feature vector to a second artificial neural network model based on unsupervised learning (S340); and generating inspection result data of the product based on the first inspection result data and the second inspection result data (S350).

[0074] In step S310, the processor (110) can identify input data including an image of a product. The type of input data has been described above with reference to FIG. 1.

[0075] In step S320, the processor (110) can generate a common feature vector based on input data by utilizing an encoder module. The encoder module of the present disclosure may include a first submodule based on a convolutional neural network and a second submodule based on a Vision Transformer. "Vision Transformer" may refer to a transformer-based neural network structure for receiving image data as input and learning global contextual information of the entire image. To generate a common feature vector by utilizing the encoder module, the processor (110) may first generate a local feature vector based on input data by utilizing the first submodule. A local feature vector may refer to data in the form of a vector extracted to effectively reflect the local features of the input data. The processor (110) can generate a global feature vector based on a local feature vector by utilizing a second submodule. In the present disclosure, a global feature vector may refer to data in the form of a vector extracted to better reflect the features of the entire image rather than detailed regions of the image. That is, the processor (110) can generate a local feature vector by utilizing a first submodule based on a convolutional neural network and generate a global feature vector by utilizing a second submodule based on a vision transformer.

[0076] The processor (110) can generate a common feature vector based on fusing a local feature vector and a global feature vector. In this disclosure, the common feature vector may refer to a vector obtained by appropriately weighting the sum or concatenating the two vectors to simultaneously utilize the local features and global features of the input data. When generating a common feature vector by weighting the local feature vector and the global feature vector, the processor (110) can generate two versions of the common feature vector and input each version of the common feature vector into different artificial neural network models. For example, the processor (110) can generate a first common feature vector and a second common feature vector by setting the weights of the local feature vector and the global feature vector differently. If the artificial neural network model to which the first common feature vector is input is a first artificial neural network model that is a supervised learning-based artificial neural network model, the processor (110) can increase the weight of the local feature vector during the process of generating the first common feature vector. Conversely, if the artificial neural network model to which the second common feature vector is input is a second artificial neural network model that is an unsupervised learning-based artificial neural network model, the processor (110) can increase the weight of the global feature vector during the process of generating the second common feature vector.

[0077] In step S330, the processor (110) can generate first inspection result data based on inputting at least a portion of the common feature vectors into a first artificial neural network model based on supervised learning.

[0078] In step S340, the processor (110) can generate second inspection result data based on inputting at least a portion of the common feature vector into a second artificial neural network model based on unsupervised learning. By steps S330 and S340, both the unsupervised learning-based artificial neural network model and the supervised learning-based artificial neural network model can be used to inspect whether the product contains defects for the same input data.

[0079] In step S350, the processor (110) can generate third inspection result data of the product based on the first inspection result data and the second inspection result data. A specific method for generating the third inspection result data will be described later with reference to FIG. 6.

[0081] FIG. 4 is a conceptual diagram illustrating a process of inspecting a product using an artificial neural network model according to one embodiment of the present disclosure.

[0082] The processor (110) can identify input data (410) containing an image of a product and generate a common feature vector based on inputting the input data (410) to an encoder module (420).

[0083] The processor (110) can input a common feature vector into a supervised learning-based first artificial neural network model (430) and an unsupervised learning-based second artificial neural network model (440), and then generate first test result data and second test result data using each artificial neural network model.

[0084] Afterward, the processor (110) can generate a third test result data (460) based on the first test result data and the second test result data by utilizing the test module (450). The third test result data may be a module that outputs a highly reliable test result by synthesizing the results obtained from an unsupervised learning-based model and a supervised learning-based model.

[0086] FIG. 5 is a conceptual diagram showing the detailed configuration of an encoder module according to one embodiment of the present disclosure.

[0087] The processor (110) can input input data (510) to a first submodule (521) based on a convolutional neural network and a second submodule (522) based on a Vision Transformer. The first submodule (521) can generate a local feature vector (523), which is a feature vector that reflects the local features of the input data (510).

[0088] The second submodule (522) can generate a global feature vector (524) which is a feature vector reflecting the global features of the input data (510). In one embodiment of the present disclosure, during the process in which the second submodule (522) generates the global feature vector (524), a local feature vector (523) generated by the first submodule (512) may be optionally input.

[0089] The processor (110) can generate a common feature vector (525) based on a local feature vector (523) and a global feature vector (524). In the present disclosure, the common feature vector (525) corresponds to a feature vector containing all important features of the input data, including the local feature vector (523) and the global feature vector (524), and can be generated based on a weighted sum or concatenation of the local feature vector (523) and the global feature vector (524). A specific method for generating the common feature vector (525) has been described above with reference to FIG. 1.

[0091] FIG. 6 is a conceptual diagram illustrating the process of generating third inspection result data of a product according to one embodiment of the present disclosure.

[0092] The processor (110) can generate a third result data (660) based on the first test result data (651) generated using the first artificial neural network model (630) and the second test result data (652) generated using the second artificial neural network model (640) by utilizing the test module (650).

[0093] The first inspection result data (651) generated using a first artificial neural network model (630) based on supervised learning may include class classification data (6511) for an image of a product, and if the product contains defects, it may include segmentation data (6512) for defective areas. Meanwhile, the second inspection result data (652) generated using a second artificial neural network model (640) based on unsupervised learning may include class classification data (6522) for an image of a product.

[0094] The processor (110) can determine the first test result data (651) as the third test result data (660) if the class classification data included in the first test result data (651) and the second test result data (652) match.

[0095] Meanwhile, the processor (110) can determine the first inspection result data (651) as the third inspection result data (660) if the class classification data included in the first inspection result data (651) includes a defect class.

[0096] Meanwhile, if the class classification data included in the first inspection result data (651) does not include a defect class and the class classification data included in the second inspection result data (652) includes a defect class, the processor (110) can determine the second inspection result data (652) as the third inspection result data (660).

[0097] To summarize the above, the processor (110) prioritizes the result output by the first artificial neural network model (630), i.e., the first inspection result data (651), when the first artificial neural network model (630), which is a supervised learning-based artificial neural network model, detects a defect from the input data; however, if the first artificial neural network model classifies it as a normal image but the second artificial neural network model (640) classifies it as a defective image, the processor prioritizes the second inspection result data (652) to determine the input data as a defective image containing a new type of defect.

[0098] The processor (110) can further train the first artificial neural network model (630) based on input data when the class classification data (6511) included in the first inspection result data (651) does not include a defect class and the class classification data (6522) included in the second inspection result data (652) includes a defect class. That is, according to the present disclosure, the first artificial neural network model (630) based on supervised learning detects defects of a type that it has not learned by utilizing the second artificial neural network model (640) based on unsupervised learning, and by continuously training the first artificial neural network model (630) based on images containing a new type of defect, the effect of improving the performance of the artificial neural network model in performing the task of detecting defects over the long term occurs.

[0101] Meanwhile, a computer-readable medium storing a data structure is disclosed according to an embodiment of the present disclosure.

[0102] A data structure can refer to the organization, management, and storage of data that enables efficient access and modification of data. A data structure can refer to the organization of data for solving specific problems (e.g., data retrieval, data storage, data modification in the shortest possible time). A data structure may also be defined by physical or logical relationships between data elements designed to support specific data processing functions. Logical relationships between data elements may include connections between user-defined data elements. Physical relationships between data elements may include actual relationships between data elements physically stored on a computer-readable storage medium (e.g., a permanent storage device). Specifically, a data structure may include sets of data, relationships between data, and functions or instructions applicable to the data. Through an effectively designed data structure, a computing device can perform operations while minimizing the use of its resources. Specifically, through an effectively designed data structure, a computing device can increase the efficiency of operations, reading, insertion, deletion, comparison, exchange, and retrieval.

[0103] Data structures can be classified into linear and non-linear data structures based on their form. A linear data structure is one where only one piece of data is connected to the next. Linear data structures can include lists, stacks, queues, and deques. A list can refer to a set of data that maintains an internal order. Lists can include linked lists. A linked list is a data structure where data is connected in a line, with each piece of data possessing a pointer. In a linked list, the pointer can contain information regarding the connection to the next or previous data. Depending on its form, a linked list can be represented as a singly linked list, a doubly linked list, or a circular linked list. A stack is a data arrangement structure that allows for restricted access to data. A stack can be a linear data structure where data can be processed (e.g., insertion or deletion) only at one end. Data stored in a stack can be a Last-In, First-Out (LIFO) data structure, meaning that the later an item is entered, the sooner it is retrieved. A queue is a data sequence structure that allows for limited access to data; unlike a stack, it can be a FIFO (First in First Out) data structure where data stored later is retrieved later. A deque is a data structure that can process data at both ends.

[0104] Non-linear data structures can be structures where multiple data are connected after a single piece of data. Non-linear data structures may include graph data structures. A graph data structure can be defined by vertices and edges, and an edge may include a line connecting two different vertices. Graph data structures may include tree data structures. A tree data structure may be a data structure where there is only one path connecting two different vertices among the multiple vertices included in the tree. In other words, it may be a data structure that does not form a loop in a graph data structure.

[0105] Throughout this specification, computational model, neural network, network function, and neural network may be used interchangeably. Hereinafter, the term neural network will be used consistently. A data structure may include a neural network. Furthermore, a data structure including a neural network may be stored on a computer-readable medium. A data structure including a neural network may also include data preprocessed for processing by the neural network, data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, activation functions associated with each node or layer of the neural network, loss functions for learning the neural network, etc. A data structure including a neural network may include any of the components disclosed above. That is, a data structure including a neural network may be configured to include all or any combination thereof, such as data preprocessed for processing by the neural network, data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, activation functions associated with each node or layer of the neural network, and loss functions for learning the neural network. In addition to the configurations described above, a data structure including a neural network may include any other information that determines the characteristics of the neural network. Furthermore, the data structure may include any form of data used or generated during the computational process of the neural network, and is not limited to the foregoing. A computer-readable medium may include a computer-readable recording medium and / or a computer-readable transmission medium. A neural network may be composed of a set of interconnected computational units that may generally be referred to as nodes. These nodes may also be referred to as neurons. A neural network is composed of at least one node.

[0106] A data structure may include data input to a neural network. A data structure including data input to a neural network may be stored on a computer-readable medium. Data input to a neural network may include training data input during the neural network learning process and / or input data input to a neural network after training is complete. Data input to a neural network may include pre-processed data and / or data subject to pre-processing. Pre-processing may include a data processing process for inputting data into a neural network. Accordingly, a data structure may include data subject to pre-processing and data generated by pre-processing. The aforementioned data structure is merely an example, and the present disclosure is not limited thereto.

[0107] The data structure may include weights of the neural network. (In this specification, weights and parameters may be used interchangeably.) The data structure including the weights of the neural network may be stored on a computer-readable medium. The neural network may include multiple weights. The weights may be variable and may be varied by a user or an algorithm to enable the neural network to perform a desired function. For example, if one or more input nodes are interconnected to a single output node by respective links, the output node may determine the data value output from the output node based on values ​​input to the input nodes connected to the output node and weights set on the links corresponding to each input node. The aforementioned data structure is merely an example and the present disclosure is not limited thereto.

[0108] As an example rather than a limitation, weights may include weights that vary during the neural network learning process and / or weights for which neural network learning is completed. Weights that vary during the neural network learning process may include weights at the start of the learning cycle and / or weights that vary during the learning cycle. Weights for which neural network learning is completed may include weights for which the learning cycle is completed. Accordingly, a data structure containing the weights of a neural network may include a data structure containing weights that vary during the neural network learning process and / or weights for which neural network learning is completed. Therefore, the weights and / or combinations of each weight described above are included in the data structure containing the weights of a neural network. The aforementioned data structure is merely an example and the present disclosure is not limited thereto.

[0109] Data structures containing the weights of a neural network may be stored on a computer-readable storage medium (e.g., memory, hard disk) after undergoing a serialization process. Serialization may be a process of converting a data structure into a form that can be stored on the same or different computing devices and later reconstructed for use. A computing device may serialize the data structure to transmit and receive data over a network. A serialized data structure containing the weights of a neural network may be reconstructed on the same or different computing devices through deserialization. Data structures containing the weights of a neural network are not limited to serialization. Furthermore, data structures containing the weights of a neural network may include data structures designed to increase computational efficiency while minimizing the use of computing device resources (e.g., B-Tree, Trie, m-way search tree, AVL tree, Red-Black Tree in non-linear data structures). The foregoing is merely an example and the present disclosure is not limited thereto.

[0110] The data structure may include hyperparameters of the neural network. The data structure including the neural network hyperparameters may be stored on a computer-readable medium. The hyperparameters may be variables that are varied by the user. The hyperparameters may include, for example, a learning rate, a cost function, the number of learning cycle iterations, weight initialization (e.g., setting the range of weight values ​​subject to weight initialization), and the number of hidden units (e.g., the number of hidden layers, the number of nodes in the hidden layers). The aforementioned data structure is merely an example, and the present disclosure is not limited thereto.

[0112] FIG. 7 is a brief and general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented.

[0113] Although the present disclosure has been described as generally being implementable by a computing device, a person skilled in the art will be well aware that the present disclosure may be implemented in combination with computer-executable instructions and / or other program modules that can be executed on one or more computers and / or as a combination of hardware and software.

[0114] Generally, a program module includes routines, programs, components, data structures, etc., that perform a specific task or implement a specific abstract data type. Furthermore, a person skilled in the art will be well aware that the method of the present disclosure can be implemented in other computer system configurations, including single-processor or multi-processor computer systems, minicomputers, mainframe computers, as well as personal computers, handheld computing devices, microprocessor-based or programmable consumer electronics, etc. (each of which may be connected to and operated with one or more associated devices).

[0115] The embodiments described in this disclosure may also be implemented in a distributed computing environment in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

[0116] Computers typically include various computer-readable media. Any medium accessible by a computer may be a computer-readable medium, and such computer-readable media include volatile and non-volatile media, transitory and non-transitory media, and removable and non-removable media. By example, but not limiting, computer-readable media may include computer-readable storage media and computer-readable transmission media. Computer-readable storage media include volatile and non-volatile media, transitory and non-transitory media, and removable and non-removable media implemented by any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, DVD (digital video disk) or other optical disk storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, or any other media that can be accessed by a computer and used to store desired information.

[0117] Computer-readable transmission media typically include all information transmission media that implement computer-readable instructions, data structures, program modules, or other data, etc., on a modulated data signal, such as a carrier wave or other transport mechanism. The term modulated data signal means a signal in which one or more of the characteristics of the signal are set or modified to encode information within the signal. By example, not limiting, computer-readable transmission media include wired media, such as wired networks or direct-wired connections, and wireless media, such as acoustic, RF, infrared, and other wireless media. Any combination of the media described above is also considered to be within the scope of computer-readable transmission media.

[0118] An exemplary environment for implementing various aspects of the present disclosure, including a computer (1102), is shown, wherein the computer (1102) includes a processing unit (1104), system memory (1106), and a system bus (1108). The system bus (1108) connects system components, including system memory (1106) (but not limited thereto), to the processing unit (1104). The processing unit (1104) may be any processor among various commercial processors. Dual processors and other multiprocessor architectures may also be used as the processing unit (1104).

[0119] The system bus (1108) may be any of several types of bus structures that can be additionally interconnected to a local bus using any of the memory bus, peripheral bus, and various commercial bus architectures. System memory (1106) includes read-only memory (ROM) (1110) and random access memory (RAM) (1112). The basic input / output system (BIOS) is stored in non-volatile memory (1110), such as ROM, EPROM, EEPROM, etc., and this BIOS includes basic routines that help transfer information between components within the computer (1102) at times such as during startup. The RAM (1112) may also include high-speed RAM, such as static RAM, for caching data.

[0120] The computer (1102) also includes an internal hard disk drive (HDD) (1114) (e.g., EIDE, SATA)—this internal hard disk drive (1114) may also be configured for external use within a suitable chassis (not shown)—a magnetic floppy disk drive (FDD) (1116) (e.g., for reading from or writing to a removable diskette (1118)), and an optical disk drive (1120) (e.g., for reading from a CD-ROM disk (1122) or reading from or writing to other high-capacity optical media such as a DVD). The hard disk drive (1114), the magnetic disk drive (1116), and the optical disk drive (1120) may each be connected to the system bus (1108) by a hard disk drive interface (1124), a magnetic disk drive interface (1126), and an optical drive interface (1128). The interface (1124) for implementing an external drive includes at least one or both of USB (Universal Serial Bus) and IEEE 1394 interface technologies.

[0121] These drives and associated computer-readable media provide non-volatile storage of data, data structures, computer-executable instructions, etc. In the case of a computer (1102), the drives and media correspond to storing any data in a suitable digital format. Although the description of computer-readable media above refers to HDDs, removable magnetic disks, and removable optical media such as CDs or DVDs, a person skilled in the art will know that other types of computer-readable media, such as zip drives, magnetic cassettes, flash memory cards, cartridges, etc., may also be used in exemplary operating environments and that any of these media may contain computer-executable instructions for performing the methods of the present disclosure.

[0122] A number of program modules, including an operating system (1130), one or more application programs (1132), other program modules (1134), and program data (1136), may be stored in the drive and RAM (1112). All or part of the operating system, application, module and / or data may also be cached in RAM (1112). It will be well known that the present disclosure may be implemented in various commercially available operating systems or combinations of operating systems.

[0123] The user can input commands and information into the computer (1102) through one or more wired / wireless input devices, such as a pointing device like a keyboard (1138) and a mouse (1140). Other input devices (not shown) may include a microphone, an IR remote control, a joystick, a game pad, a stylus pen, a touch screen, etc. These and other input devices are often connected to the processing unit (1104) via an input device interface (1142) connected to the system bus (1108), but may also be connected via other interfaces such as a parallel port, an IEEE 1394 serial port, a game port, a USB port, an IR interface, etc.

[0124] A monitor (1144) or other type of display device is also connected to the system bus (1108) via an interface such as a video adapter (1146). In addition to the monitor (1144), the computer generally includes other peripheral output devices (not shown), such as speakers, a printer, and so on.

[0125] The computer (1102) may operate in a networked environment using a logical connection to one or more remote computers, such as remote computer(s) (1148), via wired and / or wireless communication. The remote computer(s) (1148) may be a workstation, a computing device computer, a router, a personal computer, a portable computer, a microprocessor-based entertainment device, a peer device, or other conventional network node, and generally include many or all of the components described for the computer (1102), but for brevity, only the memory storage device (1150) is illustrated. The illustrated logical connection includes a wired / wireless connection to a local area network (LAN) (1152) and / or a larger network, e.g., a wide area network (WAN) (1154). Such LAN and WAN networking environments are common in offices and companies and facilitate enterprise-wide computer networks, such as intranets, all of which can be connected to a global computer network, e.g., the Internet.

[0126] When used in a LAN networking environment, the computer (1102) is connected to a local network (1152) via a wired and / or wireless communication network interface or adapter (1156). The adapter (1156) may facilitate wired or wireless communication to the LAN (1152), and the LAN (1152) may also include a wireless access point installed therein to communicate with the wireless adapter (1156). When used in a WAN networking environment, the computer (1102) may include a modem (1158), be connected to a communication computing device on the WAN (1154), or have other means to establish communication through the WAN (1154), such as through the Internet. The modem (1158), which may be an internal or external and a wired or wireless device, is connected to the system bus (1108) via a serial port interface (1142). In a networked environment, the program modules described for the computer (1102) or parts thereof may be stored in a remote memory / storage device (1150). It will be well known that the illustrated network connection is exemplary and that other means of establishing a communication link between computers may be used.

[0127] The computer (1102) operates to communicate with any wireless device or object that is deployed and operated via wireless communication, for example, a printer, scanner, desktop and / or portable computer, PDA (portable data assistant), communication satellite, any equipment or place associated with a wireless detectable tag, and a telephone. This includes at least Wi-Fi and Bluetooth wireless technologies. Accordingly, the communication may be a predefined structure as in a conventional network, or simply ad hoc communication between at least two devices.

[0128] Wi-Fi (Wireless Fidelity) enables connectivity to the Internet and other sources without wires. Wi-Fi is a wireless technology, similar to a cell phone, that allows devices, such as computers, to transmit and receive data indoors and outdoors—that is, anywhere within the coverage area of ​​a base station. Wi-Fi networks use a wireless technology called IEEE 802.11 (a, b, g, etc.) to provide secure, reliable, and high-speed wireless connections. Wi-Fi can be used to connect computers to each other, to the Internet, and to wired networks (using IEEE 802.3 or Ethernet). Wi-Fi networks can operate in unlicensed 2.4 and 5 GHz wireless bands, for example, at data rates of 11 Mbps (802.11a) or 54 Mbps (802.11b), or in products that include both bands (dual band).

[0129] Those skilled in the art of the present disclosure will understand that information and signals may be represented using any various different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced in the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0130] Those skilled in the art will understand that the various exemplary logic blocks, modules, processors, means, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented by electronic hardware, various forms of programs or design code (referred to herein as software for convenience), or a combination of all such. To clearly illustrate this interoperability between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been generally described above in relation to their functions. Whether such functions are implemented as hardware or software depends on the design constraints imposed on the specific application and the overall system. Those skilled in the art may implement the functions described in various ways for each specific application, but such implementation decisions should not be interpreted as being outside the scope of this disclosure.

[0131] The various embodiments presented herein may be implemented as methods, devices, or articles manufactured using standard programming and / or engineering techniques. The term "article manufactured" includes a computer program, a carrier, or a medium accessible from any computer-readable storage device. For example, computer-readable storage media include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, etc.), optical discs (e.g., CDs, DVDs, etc.), smart cards, and flash memory devices (e.g., EEPROMs, cards, sticks, key drives, etc.). Additionally, the various storage media presented herein include one or more devices and / or other machine-readable media for storing information.

[0132] It should be understood that the specific order or hierarchy of steps in the presented processes is an example of exemplary approaches. It should be understood that the specific order or hierarchy of steps in the processes may be rearranged within the scope of this disclosure based on design priorities. The appended method claims provide elements of various steps in a sample order, but do not imply being limited to the specific order or hierarchy presented.

[0133] Description of the presented embodiments is provided so that a person skilled in the art may use or practice the present disclosure. Various modifications to these embodiments will be apparent to a person skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure is not limited to the embodiments presented herein, but should be interpreted in the broadest possible scope consistent with the principles and novel features presented herein.

Claims

Claim 1 A method performed by a computing device to inspect a product using an artificial neural network model, comprising: a step of identifying input data including an image of a product; a step of generating a common feature vector based on the input data using an encoder module including a first submodule and a second submodule; a step of generating first inspection result data based on inputting at least a portion of the common feature vector into a first artificial neural network model based on supervised learning; and a step of generating second inspection result data based on inputting at least a portion of the common feature vector into a second artificial neural network model based on unsupervised learning. The method comprises the step of generating a third test result data based on the first test result data and the second test result data; and the step of generating a common feature vector based on the input data using the encoder module comprises: the step of generating a local feature vector based on the input data using the first submodule; the step of generating a global feature vector based on the local feature vector using the second submodule; the step of identifying the type of artificial neural network model to which the common feature vector is to be input; the step of setting weights for each of the local feature vector and the global feature vector based on the type of artificial neural network model to which the common feature vector is to be input; and the step of generating the common feature vector based on the local feature vector, the global feature vector and the weights. Claim 2 A method according to claim 1, wherein the first submodule is based on a convolutional neural network and the second submodule is based on a Vision Transformer. Claim 3 delete Claim 4 delete Claim 5 A method according to claim 1, wherein the step of setting weights for each of the local feature vector and the global feature vector based on the type of artificial neural network model to which the common feature vector is input comprises: increasing the weight of the local feature vector if the artificial neural network model to which the common feature vector is input is a supervised learning-based artificial neural network model; and increasing the weight of the global feature vector if the artificial neural network model to which the common feature vector is input is an unsupervised learning-based artificial neural network model. Claim 6 A method according to claim 1, wherein the first artificial neural network model corresponds to an artificial neural network model trained to classify product images containing defects and perform segmentation on defect regions. Claim 7 In claim 1, the second artificial neural network model corresponds to an artificial neural network model trained to classify product images containing defects using product images not containing defects as training data. Claim 8 A method according to claim 1, wherein the first inspection result data comprises segmentation data for the image of the product; and class classification data for the image of the product; and the second inspection result data comprises class classification data for the image of the product. Claim 9 In claim 8, the step of generating the third inspection result data based on the first inspection result data and the second inspection result data comprises: determining the first inspection result data as the third inspection result data when the class classification data included in the first inspection result data and the second inspection result data match; determining the first inspection result data as the third inspection result data when the class classification data included in the first inspection result data includes a defect class; and determining the second inspection result data as the third inspection result data when the class classification data included in the first inspection result data does not include a defect class and the class classification data included in the second inspection result data includes a defect class. Claim 10 In claim 9, the method further comprises the step of additionally training the first artificial neural network model based on the input data, after the step of determining the second inspection result data as the third inspection result data when the class classification data included in the first inspection result data does not include a defect class and the class classification data included in the second inspection result data includes a defect class. Claim 11 A computer program stored on a computer-readable storage medium, which enables a computing device to perform operations for inspecting a product using an artificial neural network model, wherein the operations include: an operation of identifying input data including an image of a product; an operation of generating a common feature vector based on the input data using an encoder module including a first submodule and a second submodule; an operation of generating first inspection result data based on inputting at least a portion of the common feature vector to a first artificial neural network model based on supervised learning; and an operation of generating second inspection result data based on inputting at least a portion of the common feature vector to a second artificial neural network model based on unsupervised learning. A computer program stored on a computer-readable storage medium, comprising: an operation to generate a third test result data based on the first test result data and the second test result data; and an operation to generate a common feature vector based on the input data using the encoder module, wherein the operation to generate a local feature vector based on the input data using the first submodule; an operation to generate a global feature vector based on the local feature vector using the second submodule; an operation to identify the type of artificial neural network model to which the common feature vector is to be input; an operation to set weights for each of the local feature vector and the global feature vector based on the type of artificial neural network model to which the common feature vector is to be input; and an operation to generate the common feature vector based on the local feature vector, the global feature vector and the weights. Claim 12 A computer program stored on a computer-readable storage medium, wherein, in claim 11, the first submodule is based on a convolutional neural network and the second submodule is based on a Vision Transformer. Claim 13 delete Claim 14 delete Claim 15 A computer program stored on a computer-readable storage medium, wherein, in claim 11, the operation of setting weights for each of the local feature vector and the global feature vector based on the type of artificial neural network model to which the common feature vector is input comprises: an operation of increasing the weight of the local feature vector when the artificial neural network model to which the common feature vector is input is a supervised learning-based artificial neural network model; and an operation of increasing the weight of the global feature vector when the artificial neural network model to which the common feature vector is input is an unsupervised learning-based artificial neural network model. Claim 16 A computing device for inspecting products using an artificial neural network model, comprising one or more processors; and memory; wherein the one or more processors are configured to identify input data including an image of a product, generate a common feature vector based on the input data by utilizing an encoder module including a first submodule and a second submodule, generate a first inspection result data based on inputting at least a portion of the common feature vector to a first artificial neural network model based on supervised learning, generate a second inspection result data based on inputting at least a portion of the common feature vector to a second artificial neural network model based on unsupervised learning, and generate a third inspection result data based on the first inspection result data and the second inspection result data, and generating a common feature vector based on the input data by utilizing the encoder module comprises: generating a local feature vector based on the input data by utilizing the first submodule; generating a global feature vector based on the local feature vector by utilizing the second submodule; identifying the type of artificial neural network model to which the common feature vector will be input; and the artificial neural network to which the common feature vector will be input A computing device comprising: setting weights for each of the local feature vector and the global feature vector based on the type of neural network model; and generating the common feature vector based on the local feature vector, the global feature vector and the weights. Claim 17 A computing device according to claim 16, wherein the first submodule is based on a convolutional neural network and the second submodule is based on a Vision Transformer. Claim 18 delete Claim 19 delete Claim 20 A computing device according to claim 16, wherein, based on the type of artificial neural network model to which the common feature vector is input, the weights of the local feature vector and the global feature vector are each: increasing the weight of the local feature vector when the artificial neural network model to which the common feature vector is input is a supervised learning-based artificial neural network model; and increasing the weight of the global feature vector when the artificial neural network model to which the common feature vector is input is an unsupervised learning-based artificial neural network model.

Images