Apparatus, method, and recording medium having instructions recorded thereon for determining the class of data

Abstract

A technique for determining a class of data is disclosed. An apparatus according to one aspect of the present disclosure includes one or more processors and one or more memories storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, wherein the one or more processors acquire first data regarding a component to be placed on a board, determine whether a model for a first class corresponding to the first data is stored in the one or more memories, determine whether the first data corresponds to the first class using the model in response to determining that the model for the first class is stored, determine whether the first data corresponds to the first class using second data corresponding to the first class in response to determining that the model for the first class is not stored, and transmit information indicating whether the first data corresponds to the first class to an external device.

Description

[Technical Field]

[0001] This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 439,727, filed January 18, 2023, which is hereby incorporated by reference into this disclosure.

[0002] The present disclosure relates to techniques for determining classes of data. [Background technology]

[0003] Surface Mount Technology (SMT) is used to place one or more components on a board. Many components connect to pads on the board, and each component must be placed in a precise position on the board to ensure accurate connection between the component and the pad. Automated Optical Inspection (AOI) equipment uses optical measurement techniques to inspect whether each component has been placed in the correct position on the board. Once the components have been placed in the correct position on the board, other parts can be placed on the board.

[0004] However, if a component is not positioned correctly on the board, the board may be sent to a review station for re-inspection depending on the inspection results of the automated optical inspection device. At the review station, an operator can re-inspect the component that the automated optical device determined to be incorrectly coupled to the board, and make a final decision on whether or not an error occurred. Summary of the Invention [Problem to be solved by the invention]

[0005] At least one embodiment of the present disclosure provides a technique that allows for more precise determination of the class of data.

[0006] At least one embodiment of the present disclosure can reduce false calls that determine that a non-defective substrate is defective.

[0007] At least one embodiment of the present disclosure can reduce the number of substrates that are re-inspected at the review station.

[0008] At least one embodiment of the present disclosure can generate a model for each class and use the generated model to determine the class of input data.

[0009] At least one embodiment of the present disclosure can determine the class of input data even when a model for the class corresponding to the input data has not been generated. [Means for solving the problem]

[0010] An apparatus according to one aspect of the present disclosure includes one or more processors and one or more memories storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, wherein the one or more processors acquire first data regarding a component to be placed on a board, determine whether a model for a first class corresponding to the first data is stored in the one or more memories, and, in response to a determination that the model for the first class is stored, determine using the model whether the first data corresponds to the first class, in response to a determination that the model for the first class is not stored, determine using second data corresponding to the first class whether the first data corresponds to the first class, and transmit information indicating whether the first data corresponds to the first class to an external device.

[0011] In one embodiment, when the one or more processors use the model to determine whether the first data belongs to the first class, the one or more processors may use the model to calculate a probability that the first data belongs to the first class, determine whether the probability is equal to or greater than a predetermined reference probability, and determine that the first data belongs to the first class if the probability is equal to or greater than the predetermined reference probability, or determine that the first data does not belong to the first class if the probability is less than the predetermined reference probability.

[0012] In one embodiment, the predetermined reference probability may be determined using the average value of the minimum probability of one or more data determined to belong to the first class and the maximum probability of one or more data determined not to belong to the first class.

[0013] In one embodiment, in response to determining that the model for the first class is not stored, the one or more processors may calculate a similarity between the first data and the second data using a machine learning algorithm, and determine whether the first data corresponds to the first class based on the similarity between the first data and the second data.

[0014] In one embodiment, the machine learning algorithm may include ArcFace (Additive Angular Margin Loss).

[0015] In one embodiment, the machine learning algorithm may be trained using a plurality of arbitrarily generated virtual data and the second data.

[0016] In one embodiment, the plurality of virtual data may be generated by adjusting the size, font, and weight of characters in one or more actual data stored in the one or more memories, and adding blur and defects.

[0017] In one embodiment, the plurality of virtual data may be generated using at least one algorithm of Generative Adversarial Networks (GAN) and stable diffusion.

[0018] In one embodiment, the one or more processors may receive a learning request for the first data; in response to receiving the learning request, determine whether the model for the first class is stored; in response to determining that the model for the first class is not stored, add the first data to a learning queue for generating or training the model; and in response to determining that a model for the first class is stored, add the first data to the learning queue if the model for the first class satisfies a predetermined condition.

[0019] In one embodiment, when the one or more processors add the first data to the learning queue in response to a determination that the model for the first class is not stored, the one or more processors may add the first data to the learning queue if the number of one or more data corresponding to the first class stored in the one or more memories is greater than a predetermined value and the first data is not present in the learning queue.

[0020] In one embodiment, the predetermined condition may be a condition that is satisfied when the number of times the model determines that data that does not belong to the first class belongs to the first class is greater than a first threshold, or the number of times the model determines that data that belongs to the first class does not belong to the first class is greater than a second threshold.

[0021] In one embodiment, the one or more processors may train the model using a dataset including data that falls into one or more classes similar to the first class and arbitrarily selected data.

[0022] In one embodiment, a proportion of data in the dataset that falls into one or more classes similar to the first class may be equal to or less than a proportion of the arbitrarily selected data in the dataset.

[0023] In one embodiment, the external device is a display, and in response to determining that a model for the first class and the second data corresponding to the class are not stored, the one or more processors may output a message to the display indicating that it is impossible to determine whether the first data corresponds to the first class.

[0024] In one embodiment, the model may include a feature extractor and a classifier.

[0025] In one embodiment, the feature extractor may be commonly applied to the models stored in the one or more memories, and the classifier may be trained based on different information for each model stored in the one or more memories.

[0026] In one embodiment, the feature extractor may be trained using a number of arbitrarily generated virtual data.

[0027] In one embodiment, the feature extractor and classifier run on a Graphic Processing Unit (GPU), and the one or more processors load classifiers included in models of classes that are used more frequently than a predetermined value from the memory into a cache on the GPU, and use the feature extractor and the classifier loaded into the cache to determine whether data corresponds to a class.

[0028] In one embodiment, the one or more processors may select a least used classifier from the one or more classifiers loaded into the cache to remove from the cache.

[0029] In one embodiment, in response to determining that the first data does not fall within the first class, the one or more processors may transmit information indicating that the first data does not fall within the first class to a review station.

[0030] An electronic device according to one embodiment of the present disclosure may include one or more processors and one or more memories storing instructions to be executed by the one or more processors, and when the instructions are executed by the one or more processors, the one or more processors may be configured to execute a method according to the present disclosure.

[0031] In one embodiment of the present disclosure, in a non-transitory computer-readable recording medium having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to perform operations, the instructions may be configured to cause the one or more processors to perform a method according to the present disclosure. [Effects of the Invention]

[0032] According to at least one embodiment of the present disclosure, a technique can be provided that can determine the class of data more precisely.

[0033] At least one embodiment of the present disclosure can reduce false positives, such as determining that a non-defective substrate is defective.

[0034] At least one embodiment of the present disclosure can reduce the number of substrates that need to be re-inspected at the review station.

[0035] According to at least one embodiment of the present disclosure, a model for each class can be generated and the generated model can be used to determine the class of input data.

[0036] According to at least one embodiment of the present disclosure, even if a model for a class corresponding to the input data is not generated, the class of the input data can be determined.

[0037] The effects of the present disclosure are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the specification. [Brief explanation of the drawings]

[0038] [Figure 1] FIG. 1 is a block diagram of an electronic device according to an embodiment of the present disclosure. [Figure 2] FIG. 1 is a block diagram of a model according to an embodiment of the present disclosure. [Figure 3A] FIG. 1 illustrates a process for learning a machine learning algorithm according to one embodiment of the present disclosure. [Figure 3B] FIG. 10 illustrates a process for determining whether input data corresponds to a class according to an embodiment of the present disclosure. [Figure 4] FIG. 10 is a diagram illustrating virtual data according to an embodiment of the present disclosure. [Figure 5] FIG. 10 is a diagram illustrating a comparison dataset generated for learning input data according to one embodiment of the present disclosure. [Figure 6] FIG. 10 illustrates a re-inspection process at a review station according to one embodiment of the present disclosure. [Figure 7] FIG. 1 is a block diagram of a model running on a GPU according to an embodiment of the present disclosure. [Figure 8] FIG. 10 is a diagram illustrating an example of determining a reference probability in an electronic device according to an embodiment of the present disclosure. [Figure 9] 1 is a flowchart illustrating a method for determining whether first data corresponds to a first class in an electronic device according to an embodiment of the present disclosure. [Figure 10] 1 is a flowchart illustrating a method for generating or retraining a model for a first class in an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0039] The various examples described in this document are provided for the purpose of clearly explaining the technical idea of ​​this document and are not intended to limit the technical idea to specific embodiments. The technical idea of ​​this disclosure includes various modifications, equivalents, alternatives, and embodiments selectively combining all or part of the examples described in this document. Furthermore, the scope of the technical idea of ​​this disclosure is not limited to the various examples presented below or the specific descriptions thereof.

[0040] Unless otherwise defined, terms used in this document, including technical or scientific terms, may have the meaning commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0041] As used in this document, terms such as "comprise," "may include," "comprise," "can comprise," "have," "may have," and the like, imply the presence of a feature (e.g., a function, operation, or component) in question, but do not exclude the presence of additional features. That is, such terms should be understood as open-ended terms that include the possibility of including other embodiments.

[0042] As used in this document, the singular terms "a," "an," and "the" may also include the plural meaning unless the context clearly dictates otherwise, and this also applies to the singular terms used in the claims.

[0043] As used in this document, unless the context clearly indicates otherwise, the terms "first," "second," "first," "second," and the like, when referring to multiple similar objects, are used to distinguish one object from another and do not limit the order or importance of the objects among themselves.

[0044] As used herein, phrases such as "A, B, and C," "A, B, or C," "A, B, and / or C," "at least one of A, B, and C," "at least one of A, B, or C," "at least one of A, B, and / or C," etc., can refer to each listed item or any possible combination of the listed items. For example, "at least one of A or B" can refer to (1) at least one A, (2) at least one B, or (3) all of at least one A and at least one B.

[0045] As used in this document, the phrase "based on" is used to describe one or more factors that influence the decision, act of judgment, or behavior described in the phrase or sentence in which it appears, and does not exclude additional factors that influence that decision, act of judgment, or behavior.

[0046] As used in this document, the expression "coupled" or "connected" to one component (e.g., a first component) to another component (e.g., a second component) can mean that the one component is directly coupled or connected to the other component, or that the one component is further coupled or connected via another component (e.g., a third component).

[0047] As used in this document, the expression "configured to" may have the meanings of "set to," "capable of," "modified to," "made to," "capable of," etc., depending on the context. This expression is not limited to the meaning of "specially designed in terms of hardware." For example, a processor configured to perform a specific operation may refer to a generic purpose processor that can perform that specific operation by executing software.

[0048] Hereinafter, various embodiments of the present disclosure will be described with reference to the accompanying drawings. In the accompanying drawings and descriptions relating to the drawings, identical or substantially equivalent components may be designated by the same reference numerals. In addition, in the following descriptions of various embodiments, repeated descriptions of identical or corresponding components may be omitted, but this does not mean that the components are not included in the embodiments.

[0049] 1 is a block diagram of an electronic device according to one embodiment of the present disclosure. According to one embodiment, electronic device 100 may include memory 110, processor 120, and / or communication interface 130. In some embodiments, at least one of these components of electronic device 100 may be omitted, or other components may be added to electronic device 100. In some embodiments, additionally or alternatively, some components may be embodied as an integrated unit, or may be further embodied as one or more individual components.

[0050] At least some of the internal and external components of the electronic device 100 may be connected to each other via a bus, a GPIO (general purpose input / output), a SPI (serial peripheral interface), or a MIPI (mobile industry processor interface), and may exchange data and / or signals.

[0051] The processor 120 can run software and control at least one component of the electronic device 100 coupled to the processor 120. The processor 120 can also perform various operations related to the present disclosure, such as calculations, processing, data generation, and processing. The processor 120 can also load data from or store data in the memory 110. In one embodiment, the processor 120 can receive, as input data, an image of each component placed on a board and determine whether the component is placed in the correct location on the board. A component can refer to one or more parts placed on a board.

[0052] The memory 110 may store a variety of data, including software, that may be retrieved, processed, or utilized by at least one component of the electronic device 100. The memory 110 may include volatile and / or non-volatile memory.

[0053] The communication interface 130 may perform wireless or wired communication between the electronic device 100 and a server or between the electronic device 100 and another external electronic device. For example, the communication interface 130 may perform wireless communication using a method such as long-term evolution (LTE), LTE Advance (LTE-A), code division multiple access (CDMA), wideband CDMA (WCDMA), wireless broadband (WiBro), wireless fidelity (WiFi), Bluetooth (Bluetooth), near field communication (NFC), global positioning system (GPS), or global navigation satellite system (GNSS). For example, the communication interface 130 may perform wired communication using a method such as universal serial bus (USB), high definition multimedia interface (HDMI), recommended standard 232 (RS-232), or plain old telephone service (POTS). Depending on the embodiment, the communication interface 130 may be omitted.

[0054] In one embodiment, the electronic device 100 can communicate with the external device 101 directly or via a server. The external device 101 may be the same type of device as the electronic device 100 or a different type of device. All or some of the operations performed by the electronic device 100 may be performed by the external device 101. In one embodiment, the electronic device 100 can transmit the results of the operations it has performed to the external device 101. The external device 101 can perform subsequent operations using the transmitted data. According to one embodiment, the external device 101 may be a server in a review station. According to one embodiment, if the electronic device is an automatic optical inspection device, an image of the inspection object can be obtained by irradiating light or patterned light formed by an illumination unit in the automatic optical inspection device onto an inspection object and receiving light reflected from the inspection object (e.g., a PCB board or a component) with an imaging unit. Thereafter, the automatic optical inspection device can perform a first inspection of the image of the inspection object based on preset reference information and determine whether the inspection object is good or bad. At this time, if the object to be inspected is determined to be good, it is transferred to the next line in the SMT line, and if the object to be inspected is determined to be bad, the image of the object to be inspected can be re-inspected in the automatic optical inspection device or transmitted to the server of the review station, where it can be re-inspected.

[0055] For example, during the first inspection, optical character recognition (OCR) or optical character verification is used to determine whether the characters to be inspected can be recognized from the inspection image and judged as good or bad, and then the re-inspection can determine whether the images are similar or not.

[0056] That is, the review station server receives the inspection results for the data from the electronic device 100 and can output the inspection results to a display so that an operator in the review station can re-inspect the inspection results. The review station server can ultimately determine whether the data corresponds to a class based on the re-inspection results for the data. The re-inspection of the data performed at the review station can be performed manually by an operator.

[0057] The processor 120 can acquire input data regarding components to be placed on the board. A component can refer to one or more parts to be placed on the board. The position where each component is to be placed on the board can be determined. For example, a first component should be placed at a first position on the board, and a second component should be placed at a second position on the board. The data can refer to an image including information about the shape and size of the component, letters, numbers, or symbols written on the component or an object to be inspected, such as a PCB. Alternatively, the data can refer to an image including crack shapes, foreign objects, solder fillets, solder bridges, lead lift, component or lead offset, etc.

[0058] According to one embodiment, the processor 120 may perform an operation of determining a class of the acquired data. Each piece of data may correspond to one class. A class is a data classification corresponding to a specific component, and may be generated for each component. The class may include one or more pieces of data corresponding to one component. For example, a first class corresponding to a first component may include first and second data for the first component, and a second class corresponding to a second component may include third and fourth data for the second component. According to one embodiment, the data may be an image for a component. For example, the first and second data may be images for the first component, and the third and fourth data may be images for the second component. For convenience of explanation, the input data will be described below as data corresponding to the first class.

[0059] The processor 120 may determine whether a model for a class corresponding to the acquired input data is stored in the memory 110. In response to acquiring input data, the processor 120 may determine whether a model for a first class is stored in the memory 110. The model may refer to an algorithm (e.g., One Class Convolutional Neural Network (OC-CNN)) for determining whether the input data corresponds to a specific class, and different models may be generated and stored for each class. For example, if a first model for a first class is stored in the memory 110, the processor 120 may use the first model to determine that the first data belongs to the first class and that the third data does not belong to the first class.

[0060] According to one embodiment, the model for each class may calculate a probability that input data corresponds to the class. For example, the probability that first input data corresponds to the first class may be calculated as 0.6, and the probability that second input data corresponds to the first class may be calculated as 0.9. The processor 120 may determine a reference probability for determining whether data corresponds to a class and compare the probability calculated by the model with the reference probability to determine whether the input data corresponds to the class. The processor 120 may determine that the input data corresponds to the class if the probability calculated by the model is equal to or greater than the reference probability, and may determine that the input data does not correspond to the class if the probability calculated by the model is less than the reference probability. For example, if the processor 120 determines the reference probability to be 0.8, the processor 120 may determine that the first input data does not correspond to the first class and that the second input data corresponds to the first class. The processor 120 may accumulate and store the determination results for the multiple input data as described above. That is, the probability calculated by the model for each input data and the result of the determination of whether the input data corresponds to the class may be accumulated and stored.

[0061] According to one embodiment, processor 120 may determine a reference probability for each class based on the accumulated determination results. Processor 120 may calculate the average of the minimum value of the probabilities calculated for data corresponding to the class and the maximum value of the probabilities calculated for data not corresponding to the class. The processor may determine the relatively higher value between the average value and a predetermined critical probability as the reference probability. For example, when determining whether input data corresponds to a first class, the probability of the first input data not corresponding to the first class may be 0.4, the probability of the second input data being determined to not correspond to the first class may be 0.6, the probability of the third input data being determined to correspond to the first class may be 0.9, and the probability of the fourth input data being determined to correspond to the first class may be 0.98, and the critical probability may be determined to be 0.7. In this case, the highest value of the probabilities calculated for the input data not corresponding to the first class may be 0.6 for the second input data, and the lowest value of the probabilities calculated for the input data determined to correspond to the first class may be 0.9 for the third input data. The average of these two values ​​is 0.75, which is higher than the critical probability of 0.7, so processor 120 can determine the base probability as 0.75. On the other hand, if the average of the determined two values ​​is lower than 0.7, processor 120 can determine the base probability as 0.7.

[0062] If the first model is not stored in the memory 110, the processor 120 may check whether one or more pieces of data corresponding to the first class are stored in the memory 110. If one or more pieces of data corresponding to the first class are stored, the processor 120 may use the stored data to determine whether the input data corresponds to the first class. In other words, even if the first model has not been generated, the processor 120 may use the stored data to determine whether the input data corresponds to the first class. According to one embodiment, the processor 120 may use a machine learning algorithm (e.g., Metric Learning, ArcFace (Additive Angular Margin Loss)) to determine whether the input data corresponds to the first class. The metric learning algorithm is a machine learning algorithm that calculates the similarity between two pieces of input data and determines whether the two pieces of input data are similar. The processor 120 may use the metric learning algorithm to determine whether the input data corresponds to the first class even if the first model has not been generated. For example, the processor 120 can calculate the similarity between the first data or second data corresponding to the first class and the input data, and if the calculated similarity is greater than or equal to a predetermined value, determine that the input data corresponds to the first class.

[0063] On the other hand, if data corresponding to the first class is not stored, the processor 120 may output information indicating that the class of the first data cannot be determined. For example, the processor 120 may output a message indicating that the class of the first data cannot be determined (e.g., "Model not found") to an external device, which may be a display.

[0064] A method for generating a model for each class will be described below. For ease of explanation, it is assumed that data corresponding to the first class is received. The processor 120 may receive a learning request for input data from a learning server. The learning server is a component installed in the electronic device 100 for learning input data, and may transmit a request for initial learning or a request for fine-tuning to the processor in response to receiving input data.

[0065] In response to receiving the learning request, the processor 120 may check whether a first model is stored in the memory 110. If the first model is not stored, the processor 120 may add input data to a learning queue to generate a model. According to one embodiment, the processor 120 may sequentially learn data included in the learning queue to generate a model for the class. This model generation process corresponds to an initial training process. When not many models have been generated because not many data have been input, most of the learning requests from the learning server may be requests related to the initial training process. According to one embodiment, if a predetermined number (e.g., 50 or 100) or more of data corresponding to the first class is stored and no data corresponding to the first class already exists in the learning queue, the processor 120 may add the input data to a learning queue for generating the first model. This is because a certain number of data or more is required to generate a highly accurate model, and if input data for generating the first model already exists in the learning queue, duplicate learning is not necessary. According to one embodiment, if a predetermined number or more of data corresponding to the first class is stored but data corresponding to the first class already exists in the learning queue, or if there is no data corresponding to the first class in the learning queue but the number of data corresponding to the first class is less than a predetermined number, the processor may not add input data to the learning queue even if it receives a learning request.

[0066] If the first model is saved, the processor 120 may add input data to a training queue to retrain the currently generated model using the input data. This process may correspond to fine tuning. According to one embodiment, the processor 120 may determine to retrain the current model based on the number of errors made in determining whether each piece of data corresponds to the first class using the first model. For example, the processor 120 may determine to retrain the first model based on the number of first errors made when determining whether data that does not correspond to the first class corresponds to the first class and the number of second errors made when determining whether data that corresponds to the first class does not correspond to the first class, among the results of determining whether the input data corresponds to the first class using the first model. For example, the processor 120 may determine to retrain the first model when the number of first errors exceeds a predetermined value (e.g., 20) or the number of second errors exceeds a predetermined value (e.g., 100).

[0067] The processor 120 may generate a comparison dataset for training the input data. The processor 120 may generate a comparison dataset including data having similar characteristics to the input data (hereinafter, "similar data") to improve the accuracy of the generated model. Similar data refers to data in which the difference between the input data and the overall size and the size of the characters included in each data is less than a predetermined value, or the difference between the input data and the number of strokes in the characters included in each data is less than a predetermined number. For example, if the input data is "188," the comparison dataset may be generated as similar data having similar visual characteristics, such as "100," "168," "186," and "180." The processor 120 may train the input data using the comparison dataset, allowing the generated model to more accurately determine whether the input data corresponds to a class.

[0068] According to one embodiment, the processor 120 can determine the weight of similar data in the comparison data set. For example, the processor 120 can determine the proportion of similar data in the comparison data set to a predetermined value (e.g., 25%), and the remainder to be random data. The processor 120 can extract features of the input data using the randomly determined data, while comparing the input data with similar data to extract features of the input data more accurately. This can generate a model that can more accurately determine data corresponding to a class.

[0069] The electronic device 100 according to various embodiments of the present disclosure may be a device in various forms. For example, the electronic device 100 may be a portable communication device, a computer device, a portable multimedia device, a wearable device, a home appliance device, an AOI placed after a mounter or reflow in an SMT line, a review station linked to an AOI, a smart factory monitoring system linked to an SMT line, or a combination of one or more of the above-mentioned devices. The electronic device 100 according to the present disclosure is not limited to the above-mentioned devices.

[0070] FIG. 2 is a block diagram of a model according to one embodiment of the present disclosure. Referring to FIG. 2, the model may include a feature extractor 210, a classifier 220, and a softmax 230. A different model may be generated for each class, and the models may be generated based on input data 200 as previously described with reference to FIG. 1. According to one embodiment, multiple models may share the same feature extractor 210 but include separate classifiers 220. For example, a first model for a first class may include the common feature extractor 210 and a first classifier, and a second model for a second class may include the common feature extractor 210 and a second classifier.

[0071] The feature extractor 210 can extract features from the input data 200 to determine whether the input data 200 corresponds to the first class. The feature extractor 210 may be trained using multiple input data 200 or virtual data. The feature extractor 210 is configured to extract features from the input data 200 and needs to be trained using a large amount of data. The feature extractor 210 may be trained using multiple virtual data generated based on the input data. The virtual data may not be actual data, but may be data generated for training the feature extractor 210. According to one embodiment, the virtual data used to train the feature extractor 210 may be generated by a processor.

[0072] The classifier 220 can output a value calculated based on the features of the data 200 extracted by the feature extractor 210, indicating whether the input data 200 corresponds to the first class. The classifier 220 may be trained using a plurality of data corresponding to the first class. For example, the classifier 220 can output a value indicating whether the features of the input data 200 are similar to the features of the first class.

[0073] The softmax 230 is an activation function that normalizes the values ​​received from the classifier 220 to values ​​ranging from 0 to 1 and outputs the normalized values. The softmax 230 processes the values ​​obtained from the classifier 220 and can output the probability that the input data 200 corresponds to a specific class.

[0074] FIG. 3A illustrates a process for training a machine learning algorithm according to an embodiment of the present disclosure. FIG. 3B illustrates a process for determining whether input data corresponds to a class according to an embodiment of the present disclosure. FIGS. 3A and 3B illustrate a process for determining the class of input data 320 using a machine learning algorithm when a model for the first class has not been generated. Referring to FIG. 3A, the machine learning algorithm may be trained based on data 300 corresponding to the first class. For example, the machine learning algorithm may train on the features of the first class by training on the first data 300 corresponding to the first class. Referring to FIG. 3B, the machine learning algorithm may generate a representative image 310 based on the learned features of the first class, compare the input image 320 with the representative image 310, and determine whether the input image 320 corresponds to the first class. For example, the machine learning algorithm may determine whether the representative image 310 of the first class and the input image 320 share similar features. For example, machine learning algorithms 312 and 322 can receive input image 320 and representative image 310, extract features of representative image 310, and extract features of input image 320. Softmax 330 can obtain the features of representative image 310 and input image 320 from machine learning algorithms 312 and 322 and determine whether input image 320 corresponds to the first class.

[0075] FIG. 4 illustrates virtual data according to an embodiment of the present disclosure. The virtual data may be used to train a feature extractor commonly included in each model and a machine learning algorithm for determining whether data corresponds to a class before a model is generated. According to an embodiment, the virtual data may be generated by adjusting the size, font, and weight of characters in real data and adding blur and defects. The virtual data may be generated using at least one algorithm of generative adversarial networks (GAN) and stable diffusion. According to an embodiment, the virtual data may be generated by a processor of the electronic device 100. Referring to FIG. 4, multiple pieces of virtual data 400a, 400b, 400c, and 400d may be generated by changing the size and font of characters in an image actually stored in memory or adding blur and defects.

[0076] FIG. 5 is a diagram illustrating a comparison dataset generated for training input data according to an embodiment of the present disclosure. A processor may generate the comparison datasets 510 and 520 for training the input data 500. The processor may generate the comparison datasets 510 and 520 including similar data 510 having characteristics similar to the input data 500 and randomly selected data 520. For example, referring to FIG. 5, the processor may generate the comparison datasets 510 and 520 including similar data 510 "9670SG" having visual characteristics similar to the input data 500 "96700G." Examples of the similar data 510 included in the comparison datasets 510 and 520 are not limited to those shown in FIG. 5. In the comparison datasets 510 and 520, the data 520 excluding the similar data 510 may be randomly selected from data stored in memory. By training the model using the comparison datasets 510 and 520, it is possible to more accurately determine whether the input data 500 corresponds to a class.

[0077] FIG. 6 illustrates a re-inspection process at a review station according to one embodiment of the present disclosure. The processor determines whether the input data corresponds to the first class based on the details described above with reference to FIGS. 1 to 5, and if the input data does not correspond to the first class, it can transmit the inspection results for the input data to the review station 610. The review station 610 can re-inspect the processor's inspection results 600 for the input data based on a sample 612 for the data corresponding to the first class. According to one embodiment, the review station 610 may allow an operator to re-inspect the processor's inspection results 600 by directly comparing the sample 612 with the input data.

[0078] A review station 610 may receive inspection results 600 for input data from a processor. For example, inspection results 600 may determine that first input data 602 and second input data 604 do not correspond to a first class, and that third input data 606 corresponds to the first class. The review station 610 may re-examine the determination based on data samples 612 to see if the determination is correct. For example, the re-examination result (620) may show that first input data 622 is defective but matches sample 612, so it corresponds to the first class. Second input data 624 also matches sample 612, so it corresponds to the first class. Furthermore, unlike the inspection results 600 from the processor that determined that third data 626 corresponds to the first class, third data 626 does not match sample 612, so it may be determined that it does not correspond to the first class.

[0079] FIG. 7 is a block diagram of a model running on a GPU according to one embodiment of the present disclosure. A model for determining whether input data corresponds to a class can run on a GPU 700. That is, the model can output a determination result 740 indicating whether the input data corresponds to a class using a feature extractor 710 and a classifier 720 running on the GPU 700. However, due to the limited capacity of the GPU 700, it is difficult to load and use classifiers for all models at once. Therefore, to reduce the capacity of the GPU 700, classifiers for models used more frequently than a predetermined frequency may be loaded from memory 730 to a cache 722 on the GPU 700. For example, if 100 classifiers can be stored in the cache 722 on the GPU 700, the processor can load the 100 classifiers from memory 730 to the cache 722 in descending order of frequency of use, starting with the most frequently used model among the multiple models.

[0080] According to one embodiment, the processor may remove the least used classifier of a model from cache 722 depending on the frequency of use of the classifier. For example, if all 100 classifiers are loaded into cache 722, but a classifier that is not loaded into cache 722 is needed, the processor may remove the least used classifier from among the classifiers loaded into cache 722 and load the needed classifier from memory 730 into cache 722.

[0081] FIG. 8 is a diagram showing an example of determining a reference probability in an electronic device according to an embodiment of the present disclosure. In the graph of FIG. 8, the horizontal axis represents the class, and the vertical axis represents the probability calculated when multiple pieces of input data are input to a model of each class. The vertical lines in the graph represent the probability values ​​calculated by the model of each class for multiple pieces of input data, and the bars at the top and bottom of the vertical lines may represent intervals where the probability values ​​are highly distributed (for example, intervals where the top 25% to 75% of data are distributed). Furthermore, the bubbles displayed for each class represent the reference probability for the model of that class. That is, the reference probability for the model of the first class 800 is the value (P x ), the baseline probability for the model of the second class 810 may be the value (0.9) indicated by the second bubble 816.

[0082] The processor may determine the reference probability of each model based on the output values ​​of the plurality of input data for each model. The probability values ​​output when the plurality of input data are input to the model may be accumulated and stored in the memory. The processor may calculate the reference probability of each model by calculating the average value of the minimum probability value of the input data determined to correspond to the first class 800 and the maximum probability value of the input data determined not to correspond to the first class 800, and a predetermined critical probability (P x ), a relatively high value can be determined as the reference probability. For example, referring to FIG. 8, among the data input to the first model, the maximum value 802 of the probability values ​​of input data determined not to correspond to the first class 800 is 0.1, and the minimum value 804 of the probability values ​​of input data determined to correspond to the first class 800 is 0.99, so the average value of the two values ​​is approximately 0.5. Among the data input to the second model, the maximum value 812 of the probability values ​​of input data determined not to correspond to the second class 810 is 0.86, and the minimum value 814 of the probability values ​​of input data determined to correspond to the second class 810 is 0.98, so the average value 816 of the two values ​​is approximately 0.9. The critical probability (P x) is approximately 0.7, the processor may determine that the base probability for the model of the first class 800 is 0.7 and the base probability for the model of the second class 810 is 0.9. According to one embodiment, the processor may reset the base probabilities each time a model is generated or retrained. According to another embodiment, the processor may reset the base probabilities at predetermined time intervals.

[0083] 9 is a flowchart illustrating a method for determining whether first data corresponds to a first class in an electronic device according to an embodiment of the present disclosure. The electronic device may obtain input data in operation 900. For example, the input data may be an image of a component to be placed on a board.

[0084] At operation 910, electronic device 100 may check whether a model for a first class corresponding to the first data is stored in memory 110. If the first model is stored, electronic device 100 may determine whether the first data corresponds to the first class using the first model at operation 912. If the first model is not stored, electronic device 100 may determine whether the input data corresponds to the first class based on the first data corresponding to the first class at operation 914.

[0085] In operation 920, the electronic device 100 can transmit information indicating whether the first data corresponds to the first class to the external device 101. The electronic device 100 can determine whether the first data corresponds to the first class and transmit the inspection results to the review station 610.

[0086] 10 is a flowchart of a method for generating or retraining a model for a first class in an electronic device according to an embodiment of the present disclosure. The electronic device 100 may receive a learning request for input data in operation 1000. In response to receiving the learning request, the electronic device 100 may check whether a model for the first class corresponding to the input data is stored in the memory 110 in operation 1010.

[0087] If the first model is stored in memory 110, electronic device 100 may check whether a predetermined condition is met in operation 1020. According to one embodiment, electronic device 100 may decide to retrain the current model depending on the number of errors that occurred in the result of determining whether each piece of data corresponds to the first class using the first model. In operation 1022, electronic device 100 may decide to retrain the first model based on the number of first errors that determined that data that does not correspond to the first class corresponds to the first class and the number of second errors that determined that data that corresponds to the first class does not correspond to the first class, among the results of determining whether input data corresponds to the first class using the first model. If the predetermined condition is not met, electronic device 100 may not need to retrain the first model.

[0088] If the first model is not stored in memory 110, electronic device 100 may add the input data to a learning queue for generating the first model at operation 1030. According to one embodiment, electronic device 100 may add the input data to the learning queue in response to a predetermined number (e.g., 100) or more of data corresponding to the first class being stored and data corresponding to the first class not already being present in the learning queue. Electronic device 100 may generate a model for the first class at operation 1032.

[0089] Although the steps of the method or algorithm according to the present disclosure are described sequentially in the flowcharts shown in Figures 9 and 10, the steps may be performed sequentially or in any combinable order according to the present disclosure. The description using the flowcharts does not preclude variations or modifications to the method or algorithm, and does not imply that any step is essential or preferred. In some embodiments, at least some steps may be performed in parallel, iteratively, or heuristically. In some embodiments, at least some steps may be omitted, and other steps may be added.

[0090] Various embodiments of the present disclosure may be embodied as software on a machine-readable storage medium. The software may be software for implementing various embodiments of the present disclosure. The software can be inferred from various embodiments of the present disclosure by a programmer skilled in the art. For example, the software may be a program including machine-readable instructions (e.g., code or code segments). The device may be a device capable of operating in accordance with instructions retrieved from a storage medium, such as a computer. In one embodiment, the device may be electronic device 100 according to an embodiment of the present disclosure. In one embodiment, a processor of the device executes the retrieved instructions and causes components of the device to perform the functions corresponding to the instructions. In one embodiment, the processor may be processor 120 according to an embodiment of the present disclosure. The storage medium may refer to any type of recording medium on which data is stored that is machine-readable. The storage medium may include, for example, a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc. In one embodiment, the storage medium may be memory 110. In one embodiment, the storage medium may be embodied in a distributed form, such as in computer systems connected via a network. Software may be stored and executed in a distributed manner in the computer systems. The storage medium may be a non-transitory storage medium. A non-transitory storage medium refers to a tangible medium in which data is stored semi-permanently or temporarily, and does not include a transitory signal.

[0091] Although the technical idea of ​​the present disclosure has been described above using various embodiments, the technical idea of ​​the present disclosure includes various substitutions, modifications, and alterations that can be made within the scope of what can be understood by a person having ordinary skill in the art to which the present disclosure pertains. Furthermore, it should be understood that these substitutions, modifications, and alterations are included within the scope of the appended claims.

Claims

1. one or more processors; one or more memories storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations; The one or more processors: Obtaining first data regarding components to be placed on the substrate; determining whether a model for a first class corresponding to the first data is stored in the one or more memories; responsive to determining that the model for the first class has been saved, using the model to determine whether the first data falls within the first class; In response to determining that the model for the first class is not saved, determining whether the first data corresponds to the first class using second data corresponding to the first class; The device transmits information indicating whether the first data falls within the first class to an external device.

2. When the one or more processors use the model to determine whether the first data falls into the first class, Using the model, calculate the probability that the first data falls into the first class; determining whether the probability is greater than or equal to a predetermined reference probability; If the probability is equal to or greater than a predetermined reference probability, it is determined that the first data belongs to the first class; The apparatus of claim 1 , wherein the apparatus determines that the first data does not belong to the first class if the probability is less than a predetermined reference probability.

3. 3. The apparatus of claim 2, wherein the predetermined reference probability is determined using an average value of a minimum probability of one or more data determined to belong to the first class and a maximum probability of one or more data determined not to belong to the first class.

4. The one or more processors: responsive to determining that the model for the first class is not stored, calculating a similarity between the first data and the second data using a machine learning algorithm; The apparatus of claim 1 , further comprising: determining whether the first data belongs to the first class based on the similarity between the first data and the second data.

5. The apparatus of claim 4 , wherein the machine learning algorithm comprises ArcFace (Additive Angular Margin Loss).

6. The machine learning algorithm: The device according to claim 4 , wherein the device is trained using a plurality of arbitrarily generated virtual data and the second data.

7. The plurality of virtual data are The device according to claim 6, wherein the image is generated by adjusting the size, font, and weight of characters in the one or more actual data stored in the one or more memories, and adding blur and defects.

8. The plurality of virtual data are The apparatus of claim 6 , wherein the signal is generated using at least one of the following algorithms: Generative Adversarial Networks (GAN) and Stable Diffusion.

9. The one or more processors: Obtaining a learning request for the first data; In response to receiving the training request, determining whether the model for the first class is saved; responsive to determining that the model for the first class is not stored, adding the first data to a training queue for generating or training the model; 2. The apparatus of claim 1, wherein, in response to determining that a model for the first class is saved, the apparatus adds the first data to the training queue if the model for the first class satisfies a predetermined condition.

10. When the one or more processors add the first data to the training queue in response to determining that the model for the first class is not saved, 10. The device of claim 9, wherein if the number of one or more pieces of data corresponding to the first class stored in the one or more memories is greater than a predetermined value and the first data is not present in the learning queue, the device adds the first data to the learning queue.

11. The predetermined condition is: The device of claim 9, wherein the condition is satisfied when the number of times the model determines that data that does not belong to the first class belongs to the first class is greater than a first threshold, or the number of times the model determines that data that belongs to the first class does not belong to the first class is greater than a second threshold.

12. The one or more processors: The apparatus of claim 9 , wherein the model is trained using a dataset including data that falls into one or more classes similar to the first class and arbitrarily selected data.

13. The apparatus of claim 12 , wherein a proportion of data in the dataset that falls into one or more classes similar to the first class is equal to or less than a proportion of the arbitrarily selected data in the dataset.

14. the external device is a display; The one or more processors:

2. The device of claim 1, wherein, in response to determining that a model for the first class and the second data corresponding to the class are not stored, a message indicating that it is impossible to determine whether the first data corresponds to the first class is output to the display.

15. The apparatus of claim 1 , wherein the model includes a feature extractor and a classifier.

16. 16. The apparatus of claim 15, wherein the feature extractor is commonly applied to the models stored in the one or more memories, and the classifier is trained based on information that differs for each model stored in the one or more memories.

17. The apparatus of claim 15 , wherein the feature extractor is trained using a plurality of arbitrarily generated virtual data.

18. The feature extractor and classifier run on a GPU (Graphical Processing Unit); The one or more processors: Loading a classifier included in a model of a class that is used at a frequency equal to or greater than a predetermined value from the memory into a cache on the GPU; The apparatus of claim 15 , wherein the feature extractor and the cached classifier are used to determine whether data falls into a class.

19. The one or more processors: The apparatus of claim 18 , further comprising: selecting a least used classifier from the one or more classifiers loaded into the cache to remove from the cache.

20. The one or more processors:

10. The apparatus of claim 1, wherein, in response to determining that the first data does not fall within the first class, the apparatus transmits information to a review station indicating that the first data does not fall within the first class.

21. 1. A method performed on an apparatus including one or more processors and one or more memories storing instructions for execution by said one or more processors, comprising: obtaining first data indicative of components to be placed on the substrate; determining whether a model for a first class corresponding to the first data is stored in one or more memories; responsive to determining that the model for the first class has been saved, using the model to determine whether the first data corresponds to the first class; In response to determining that a model for the first class is not stored, determining whether the first data corresponds to the first class using second data corresponding to the first class; and transmitting information indicating whether the first data falls within the first class to an external device.

22. 1. A non-transitory computer-readable storage medium having instructions recorded thereon that, when executed by one or more processors, cause the one or more processors to perform operations, comprising: The instructions may cause the one or more processors to: obtaining first data indicative of components to be placed on the substrate; determining whether a model for a first class corresponding to the first data is stored in one or more memories; responsive to determining that the model for the first class has been saved, using the model to determine whether the first data corresponds to the first class; In response to determining that the model for the first class is not stored, determining whether the first data corresponds to the first class using second data corresponding to the first class; and transmitting information indicating whether the first data corresponds to the first class to an external device.

Images