Recording medium containing devices, methods, and instructions for determining the class of data

Abstract

A technique for determining a class of data is disclosed. An apparatus according to one aspect of the present disclosure includes one or more processors and one or more memories storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, wherein the one or more processors acquire first data regarding a component to be placed on a board, determine whether a model for a first class corresponding to the first data is stored in the one or more memories, determine whether the first data corresponds to the first class using the model in response to determining that the model for the first class is stored, determine whether the first data corresponds to the first class using second data corresponding to the first class in response to determining that the model for the first class is not stored, and transmit information indicating whether the first data corresponds to the first class to an external device.

Description

Technical Field

[0001] This application claims the benefit of priority of U.S. Provisional Patent Application No. 63 / 439,727, filed on January 18, 2023. Accordingly, Provisional Application No. 63 / 439,727 is incorporated herein by reference.

[0002] This disclosure relates to techniques for determining classes of data.

Background Art

[0003] Surface Mount Technology (SMT) is used to place one or more components on a substrate. Although many components are coupled to pads on the substrate, each component needs to be placed at an exact position on the substrate in order for the component and the pad to be accurately coupled. An Auto Optical Inspection (AOI) device can use optical measurement technology to inspect whether each component is placed at an exact position on the substrate. If the components are placed at exact positions on the substrate, other parts may be placed on the substrate.

[0004] However, if a component is not placed at an exact position on the substrate, the substrate may be transmitted to a review station for re-inspection based on the inspection result of the auto optical inspection device. At the review station, an operator can re-inspect a component determined by the auto optical device to have an error in coupling to the substrate and finally determine whether an error has occurred.

Summary of the Invention

Problems to be Solved by the Invention

[0005] At least one embodiment of this disclosure provides a technique that can more precisely determine classes of data.

[0006] At least one embodiment of the present disclosure can reduce false calls in which a defect-free substrate is identified as defective.

[0007] At least one embodiment of the present disclosure can reduce the number of substrates that need to be re-inspected at the review station.

[0008] At least one embodiment of this disclosure can generate a model for each class and use the generated model to determine the class of the input data.

[0009] At least one embodiment of the present disclosure can determine the class of input data even when a model has not been generated for the class corresponding to the input data. [Means for solving the problem]

[0010] An apparatus relating to one aspect of the present disclosure includes one or more processors and one or more memories in which instructions are stored that cause the one or more processors to perform calculations when executed by the one or more processors, the one or more processors can acquire first data relating to components arranged on a board, determine whether a model for a first class corresponding to the first data is stored in the one or more memories, determine whether the first data belongs to the first class using the model in response to the determination that the model for the first class is stored, determine whether the first data belongs to the first class using second data belonging to the first class in response to the determination that the model for the first class is not stored, and transmit information to an external device indicating whether the first data belongs to the first class.

[0011] In one embodiment, when one or more processors use the model to determine whether the first data belongs to the first class, they use the model to calculate the probability that the first data belongs to the first class, determine whether the probability is greater than or equal to a predetermined reference probability, and if the probability is greater than or equal to the predetermined reference probability, they determine that the first data belongs to the first class, and if the probability is less than the predetermined reference probability, they determine that the first data does not belong to the first class.

[0012] In one embodiment, the predetermined reference probability may be determined using the average value of the minimum probability of one or more data determined to belong to the first class and the maximum probability of one or more data determined not to belong to the first class.

[0013] In one embodiment, one or more processors may, in response to a decision that the model for the first class is not saved, use a machine learning algorithm to calculate the similarity between the first data and the second data, and determine whether the first data belongs to the first class based on the similarity between the first data and the second data.

[0014] In one embodiment, the machine learning algorithm may include ArcFace(Additive Angular Margin Loss).

[0015] In one embodiment, the machine learning algorithm may be trained using a plurality of arbitrarily generated virtual data and the second data.

[0016] In one embodiment, the plurality of virtual data may be generated by adjusting the character size, font, and thickness of one or more actual data stored in one or more memories, and by adding blur and defects.

[0017] In one embodiment, the plurality of virtual data may be generated using at least one algorithm from among GAN (Generative Adversarial Networks) and stable diffusion.

[0018] In one embodiment, one or more processors acquire a learning request for the first data, determine whether or not the model for the first class is stored in response to acquiring the learning request, add the first data to the learning queue for generating or training the model in response to the determination that the model for the first class is not stored, and add the first data to the learning queue in response to the determination that the model for the first class is stored if the model for the first class satisfies predetermined conditions.

[0019] In one embodiment, when one or more processors add the first data to the learning queue in response to a decision that the model for the first class is not stored, if the number of one or more data items corresponding to the first class stored in the one or more memory is greater than a predetermined value and the first data is not present in the learning queue, the first data can be added to the learning queue.

[0020] In one embodiment, the predetermined condition may be a condition that is satisfied when the number of times the model determines that data that does not belong to the first class belongs to the first class is greater than a first threshold, or when the number of times the model determines that data that does belong to the first class does not belong to the first class is greater than a second threshold.

[0021] In one embodiment, the one or more processors may train the model using a dataset (set) that includes data belonging to one or more classes similar to the first class and arbitrarily selected data.

[0022] In one embodiment, the ratio of the data corresponding to one or more classes similar to the first class in the dataset may be less than or equal to the ratio of the arbitrarily selected data in the dataset.

[0023] In one embodiment, the external device is a display, and the one or more processors can output, to the display, a statement indicating that it is impossible to determine whether the first data corresponds to the first class in response to a determination that the model for the first class and the second data corresponding to the class are not stored.

[0024] In one embodiment, the model may include a feature extractor and a classifier.

[0025] In one embodiment, the feature extractor may be commonly applied to the models stored in the one or more memories, and the classifier may be learned based on different information for each of the models stored in the one or more memories.

[0026] In one embodiment, the feature extractor may be learned using a plurality of arbitrarily generated virtual data.

[0027] In one embodiment, the feature extractor and the classifier operate on a GPU (Graphic Processing Unit), and the one or more processors load, from the memory to the cache on the GPU, the classifier included in the model of the class used at a frequency of a predetermined value or more, and can determine whether the data corresponds to the class using the feature extractor and the classifier loaded in the cache.

[0028] In one embodiment, the one or more processors can select the least used classifier among the one or more classifiers loaded in the cache and remove it from the cache.

[0029] In one embodiment, one or more processors may, in response to a determination that the first data does not belong to the first class, transmit information to a review station indicating that the first data does not belong to the first class.

[0030] An electronic device according to one embodiment of the present disclosure includes one or more processors and one or more memories in which instruction words to be executed by the one or more processors are stored, and when the instruction words are executed by the one or more processors, the one or more processors may be configured to execute the method according to the present disclosure.

[0031] In one embodiment of the present disclosure, a non-temporary computer-readable recording medium that stores instruction words causing one or more processors to operate when executed by one or more processors, the instruction words may be configured to cause the one or more processors to execute the method according to the present disclosure. [Effects of the Invention]

[0032] According to at least one embodiment of this disclosure, a technique can be provided that enables more precise determination of data classes.

[0033] According to at least one embodiment of this disclosure, it is possible to reduce the misjudgment of determining a defect-free substrate as defective.

[0034] According to at least one embodiment of this disclosure, the number of substrates to be re-inspected at the review station can be reduced.

[0035] According to at least one embodiment of this disclosure, a model can be generated for each class, and the class of the input data can be determined using the generated model.

[0036] According to at least one embodiment of this disclosure, the class of the input data can be determined even when a model for the class corresponding to the input data is not generated.

[0037] The effects relating to this disclosure are not limited to those mentioned above, and other effects not mentioned will be clearly understood by a person of ordinary skill from the description in the specification. [Brief explanation of the drawing]

[0038] [Figure 1] This is a block diagram of an electronic device according to one embodiment of the present disclosure. [Figure 2] This is a block diagram of a model relating to one embodiment of the present disclosure. [Figure 3A] This figure shows the process of training a machine learning algorithm according to one embodiment of the present disclosure. [Figure 3B] This figure shows the process of determining whether input data belongs to a class according to one embodiment of the present disclosure. [Figure 4] This figure shows virtual data relating to one embodiment of the present disclosure. [Figure 5] This figure shows a comparison dataset generated to train input data according to one embodiment of the present disclosure. [Figure 6] This figure shows the re-examination process at the review station according to one embodiment of the present disclosure. [Figure 7] This is a block diagram of a model that operates on a GPU according to one embodiment of the present disclosure. [Figure 8] This figure shows an example of determining a reference probability in an electronic device according to one embodiment of the present disclosure. [Figure 9] This flowchart shows a method for determining whether or not first data falls under the first class in an electronic device according to one embodiment of the present disclosure. [Figure 10] This flowchart shows a method for generating or retraining a model for a first class in an electronic device according to one embodiment of the present disclosure. [Modes for carrying out the invention]

[0039] The various embodiments described herein are illustrative for the purpose of clearly illustrating the technical concept of this document and are not intended to limit it to any particular embodiment. The technical concept of this disclosure includes various modifications, equivalents, alternatives, and embodiments selectively combined from all or part of the embodiments described herein. Furthermore, the scope of rights to the technical concept of this disclosure is not limited to the various embodiments or specific descriptions thereof presented below.

[0040] Unless otherwise defined, terms used in this document, including technical or scientific terms, may have meanings that are generally understood by a person with ordinary skill in the art to which this disclosure pertains.

[0041] Expressions such as "includes," "may include," "equip," "may equip," "possess," and "may possess" used in this document mean that the feature in question (e.g., function, operation, or component) exists, but do not exclude the existence of other further features. In other words, such expressions should be understood as open-ended terms that imply the possibility of including other embodiments.

[0042] In this document, singular expressions may also have plural meanings unless otherwise specified in the context, and this applies equally to singular expressions in claims.

[0043] In this document, expressions such as "first," "second," or "first," "second," etc., are used to distinguish one object from others when referring to multiple objects of the same kind, unless otherwise specified in the context. They do not limit the order or importance of the objects.

[0044] Expressions used in this document such as "A, B, and C," "A, B, or C," "A, B, and / or C," "at least one of A, B, and C," "at least one of A, B, or C," and "at least one of A, B, and / or C" can refer to each of the listed items, or any possible combination of the listed items. For example, "at least one of A or B" can refer to (1) at least one A, (2) at least one B, or (3) all of at least one A and at least one B.

[0045] The expression "based on" as used in this document is used to describe one or more factors that influence an act of decision, judgment, or action described in the phrase or sentence containing this expression, and this expression does not exclude any further factors that influence such act of decision, judgment, or action.

[0046] The expression used in this document that one component (e.g., the first component) is "linked" or "connected" to another component (e.g., the second component) can mean not only that the first component is directly linked or connected to the other component, but also that it is linked or connected via yet another component (e.g., the third component).

[0047] The expression "configured to" as used in this document may mean, depending on the context, "set to do," "capable of doing," "modified to do," "made to do," or "capable of doing." This expression is not limited to meaning "specifically designed in hardware." For example, a processor configured to perform a specific operation can mean a generic-purpose processor that can perform that specific operation by running software.

[0048] Various embodiments of this disclosure will be described below with reference to the attached drawings. In the attached drawings and descriptions relating to the drawings, identical or substantially equivalent components may be denoted by the same reference numeral. Furthermore, in the descriptions of the various embodiments below, redundant descriptions of identical or corresponding components may be omitted, but this does not mean that such components are not included in that embodiment.

[0049] Figure 1 is a block diagram of an electronic device according to one embodiment of the present disclosure. According to one embodiment, the electronic device 100 may include a memory 110, a processor 120, and / or a communication interface 130. In one embodiment, at least one of these components of the electronic device 100 may be omitted, or other components may be added to the electronic device 100. In one embodiment, as an addition or alternative, some components may be embodied as a combined entity, or further embodied as one or more individuals.

[0050] At least some of the internal and external components of the electronic device 100 are connected to each other via a bus, GPIO (general purpose input / output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface), etc., and can exchange data and / or signals.

[0051] The processor 120 can drive software and control at least one component of the electronic device 100 connected to the processor 120. The processor 120 can also perform various operations related to this disclosure, such as calculations, processing, data generation, and manipulation. Furthermore, the processor 120 can load data from or store data in the memory 110. In one embodiment, the processor 120 can receive images of each component placed on the substrate as input data and determine whether the component is positioned in the correct location on the substrate. A component can mean one or more parts placed on the substrate.

[0052] The memory 110 can store various types of data. The data stored in the memory 110 is data acquired, processed, or used by at least one component of the electronic device 100, and may include software. The memory 110 may include volatile and / or non-volatile memory.

[0053] The communication interface 130 can perform wireless or wired communication between the electronic device 100 and the server, or between the electronic device 100 and other external electronic devices. For example, the communication interface 130 can perform wireless communication using methods such as LTE (Long-Term Evolution), LTE-A (LTE Advance), CDMA (Code Division Multiple Access), WCDMA (Wideband CDMA), WiBro (Wireless Broadband), WiFi (Wireless Fidelity), Bluetooth (Registered Trademark), NFC (Near Field Communication), GPS (Global Positioning System), or GNSS (Global Navigation Satellite System). For example, the communication interface 130 can perform wired communication using methods such as USB (Universal Serial Bus), HDMI (High Definition Multimedia Interface), RS-232 (Recommended Standard 232), or POTS (Plain Old Telephone Service). Depending on the embodiment, the communication interface 130 may be omitted.

[0054] In one embodiment, the electronic device 100 can communicate with an external device 101 via a server or directly. The external device 101 may be the same type of device as the electronic device 100 or a different type of device. All or part of the operations performed by the electronic device 100 may be performed by the external device 101. In one embodiment, the electronic device 100 can transmit the results of the operations performed to the external device 101. The external device 101 can then perform subsequent operations using the transmitted data, etc. According to one embodiment, the external device 101 may be a server in a review station. According to one embodiment, if the electronic device is an automated optical inspection device, it can irradiate the object to be inspected with light or patterned light formed by the illumination unit in the automated optical inspection device, and receive the light reflected from the object to be inspected (e.g., a PCB substrate or component) with the imaging unit to acquire an image of the object to be inspected. Subsequently, the automated optical inspection device can perform a primary inspection of the image of the object to be inspected based on pre-set reference information and determine whether the object to be inspected is good or bad. If the object being inspected is deemed satisfactory, it is transferred to the next line in the SMT line. If the object is deemed unsatisfactory, its image is either re-inspected within the automated optical inspection device or transmitted to the server at the review station, where a re-inspection can be performed.

[0055] For example, during the initial inspection, Optical Character Recognition (OCR) or Optical Character Verification can be used to determine whether the target character is recognized from the inspection image, and the result can be judged as good or bad. A re-inspection can then determine whether or not the images are similar.

[0056] In other words, the server in the review station receives the inspection results for the data from the electronic device 100 and can output them to a display so that an operator in the review station can perform a re-inspection of the inspection results. Based on the re-inspection results of the data, the review station server can finally determine whether or not the data belongs to a class. The re-inspection of the data performed in the review station may be performed manually by an operator.

[0057] The processor 120 can acquire input data regarding components to be placed on the circuit board. A component can mean one or more parts to be placed on the circuit board. The position of each component on the circuit board may be determined. For example, the first component should be placed at the first position on the circuit board, and the second component should be placed at the second position on the circuit board. The data can mean an image containing information about the shape and size of the component, and letters, numbers, or symbols written on the component or the object being inspected, such as a PCB. Alternatively, the data can mean an image containing information about the crack shape of the component, foreign matter, solder fillets, solder bridges, lead lifting, component or lead offset, etc.

[0058] According to one embodiment, the processor 120 can perform an operation to determine the class of the acquired data. Each piece of data may correspond to one class. A class is a data classification corresponding to a specific component, and may be generated separately for each component, and may contain one or more pieces of data corresponding to one component. For example, a first class corresponding to a first component may include first and second pieces of data for the first component, and a second class corresponding to a second component may include third and fourth pieces of data for the second component. According to one embodiment, the data may be an image of a component. For example, the first and second pieces of data may be an image of the first component, and the third and fourth pieces of data may be an image of the second component. For the sake of explanation, the input data will be described below assuming that it corresponds to the first class.

[0059] The processor 120 can check whether a model for the class corresponding to the acquired input data is stored in memory 110. In response to acquiring input data, the processor 120 can check whether a model for the first class is stored in memory 110. The model can mean an algorithm (e.g., OC-CNN (One Class Convolutional Neural Network)) for determining whether the input data corresponds to a specific class, and different models may be generated and stored for each class. For example, if the first model for the first class is stored in memory 110, the processor 120 can use the first model to determine that the first data belongs to the first class and that the third data does not belong to the first class.

[0060] In one embodiment, the model for each class can calculate the probability that the input data corresponds to the class. For example, the probability that the first input data corresponds to the first class can be calculated as 0.6, and the probability that the second input data corresponds to the first class can be calculated as 0.9. The processor 120 can determine a reference probability for determining whether the data corresponds to a class, and can determine whether the input data corresponds to the class by comparing the probability calculated by the model with the reference probability. The processor 120 can determine that the input data corresponds to the class if the probability calculated by the model is equal to or greater than the reference probability, and that the input data does not correspond to the class if the probability calculated by the model is less than the reference probability. For example, if the processor 120 determines the reference probability to be 0.8, it can determine that the first input data does not correspond to the first class, and the second input data corresponds to the first class. The processor 120 can accumulate and store the judgment results for multiple input data as described above. That is, it can accumulate and store the probability calculated by the model for each input data and the result of determining whether or not it corresponds to the class.

[0061] According to one embodiment, the processor 120 can determine a reference probability for each class based on the accumulated judgment results. The processor 120 can calculate the average of the minimum probability calculated for data corresponding to the class and the maximum probability calculated for data not corresponding to the class. The processor can determine the reference probability to be a relatively high value among the predetermined critical probabilities, along with the average value. For example, when determining whether input data corresponds to a first class, the probability of the first input data determined not to correspond to the first class is 0.4, the probability of the second input data is 0.6, the probability of the third input data determined to correspond to the first class is 0.9, the probability of the fourth input data is 0.98, and the critical probability may be determined to be 0.7. In this case, the highest value among the probabilities calculated for input data determined not to correspond to the first class is 0.6 for the second input data, and the lowest value among the probabilities calculated for input data determined to correspond to the first class is 0.9 for the third input data. The average of these two values ​​is 0.75, which is higher than the critical probability of 0.7, so the processor 120 can determine the reference probability to be 0.75. On the other hand, if the average of the two values ​​determined above is lower than 0.7, the processor 120 can determine the reference probability to be 0.7.

[0062] If the first model is not stored in memory 110, the processor 120 can check whether one or more data corresponding to the first class are stored in memory 110. If one or more data corresponding to the first class are stored, the processor 120 can use the stored data to determine whether the input data corresponds to the first class. In other words, even if the first model has not been generated, the processor 120 can use the stored data to determine whether the input data corresponds to the first class. According to one embodiment, the processor 120 can use a machine learning algorithm (e.g., Metric Learning, ArcFace (Additive Angular Margin Loss)) to determine whether the input data corresponds to the first class. The Metric Learning algorithm is a machine learning algorithm for calculating the similarity between two input data and determining whether the two input data are similar. Even if the first model has not been generated, the processor 120 can use the Metric Learning algorithm to determine whether the input data corresponds to the first class. For example, the processor 120 can calculate the similarity between the input data and the first or second data corresponding to the first class, and if the calculated similarity is greater than or equal to a predetermined value, it can determine that the input data corresponds to the first class.

[0063] On the other hand, if no data corresponding to the first class is stored, the processor 120 can output information indicating that it cannot determine the class of the first data. For example, the processor 120 can output a message indicating that it cannot determine the class of the first data (e.g., "Model not found") to an external device, which may be a display.

[0064] The following describes how to generate models for each class. For the sake of explanation, we will assume that data corresponding to the first class has been received. The processor 120 can obtain a learning request for the input data from the learning server. The learning server is a configuration introduced in the electronic device 100 for learning the input data, and in response to receiving the input data, it can send a request for initial learning or a request for fine tuning to the processor.

[0065] Upon receiving a training request, the processor 120 can check whether a first model is stored in memory 110. If a first model is not stored, the processor 120 can add input data to the training queue in order to generate a model. In one embodiment, the processor 120 can sequentially train the data included in the training queue and generate a model for each class. This model generation process corresponds to initial training, and if not many models have been generated because not much data has been input, then most of the training requests from the training server may be requests related to the initial training process. In one embodiment, if a predetermined number (e.g., 50 or 100) or more of data corresponding to a first class are stored, and no data corresponding to a first class already exists in the training queue, the processor 120 can add input data to the training queue for generating a first model. This is because a certain number of data points are necessary to generate a highly accurate model, and if input data for generating a first model already exists in the training queue, there is no need to train the data again. According to one embodiment, if a predetermined number or more of data corresponding to the first class are stored, but data corresponding to the first class already exists in the learning queue, or if there is no data corresponding to the first class in the learning queue, but the number of data corresponding to the first class is less than a predetermined number, the processor does not need to add input data to the learning queue even if it receives a learning request.

[0066] If the first model is saved, the processor 120 can add the input data to the training queue in order to retrain the currently generated model using the input data. This process may be considered fine tuning. In one embodiment, the processor 120 can decide whether to retrain the current model based on the number of errors that occurred in determining whether each data corresponds to a first class using the first model. For example, the processor 120 can decide to retrain the first model based on the number of first errors in determining whether data that does not correspond to a first class corresponds to a first class, and the number of second errors in determining whether data that does correspond to a first class does not. For example, the processor 120 can decide to retrain the first model if the number of first errors exceeds a predetermined value (e.g., 20) or if the number of second errors exceeds a predetermined value (e.g., 100).

[0067] The processor 120 can generate a comparison dataset to train on the input data. To improve the accuracy of the generated model, the processor 120 can generate a comparison dataset that includes data with similar features to the input data (hereinafter referred to as similar data). Similar data means data in which the difference in overall size and the size of the characters contained in each data from the input data is less than or equal to a predetermined value, or the difference in the number of strokes of the characters contained in each data is less than or equal to a predetermined number. For example, if the input data is "188", the comparison dataset may be generated as similar data with similar visual features, such as "100", "168", "186", and "180". The processor 120 can use the comparison dataset to train on the input data so that the generated model can more accurately determine whether the input data corresponds to a class or not.

[0068] In one embodiment, the processor 120 can determine the proportion of similar data in the comparison dataset. For example, the processor 120 can determine that the proportion of similar data in the comparison dataset is a predetermined value (e.g., 25%), and the remainder is determined to be random data. The processor 120 extracts features of the input data using the randomly determined data, while also comparing the similar data with the input data to extract features of the input data with greater accuracy. This makes it possible to generate a model that can more accurately determine which data corresponds to which class.

[0069] The electronic device 100 in various embodiments of this disclosure may be of various forms. For example, the electronic device 100 may be a portable communication device, a computer device, a portable multimedia device, a wearable device, a home appliance device, an AOI located downstream of a mounter or reflow in an SMT line, a review station linked to an AOI, a smart factory monitoring system linked to an SMT line, or a combination of one or more of the above-mentioned devices. The electronic device 100 of this disclosure is not limited to the above-mentioned devices.

[0070] Figure 2 is a block diagram of a model according to one embodiment of the present disclosure. Referring to Figure 2, the model may include a feature extractor 210, a classifier 220, and a softmax 230. Different models may be generated for each class, and the models may be generated based on input data 200, as described in Figure 1 above. According to one embodiment, multiple models may share the same feature extractor 210 but include separate classifiers 220. For example, a first model for a first class may include a common feature extractor 210 and a first classifier, and a second model for a second class may include a common feature extractor 210 and a second classifier.

[0071] The feature extractor 210 can extract features from the input data 200 to determine whether the input data 200 corresponds to a first class. The feature extractor 210 may be trained using multiple input data 200 or virtual data. The feature extractor 210 is configured to extract features from the input data 200 and needs to be trained using a large amount of data. The feature extractor 210 may be trained using multiple virtual data generated based on the input data. The virtual data is not actual data, but may be data generated for the training of the feature extractor 210. In one embodiment, the virtual data used to train the feature extractor 210 may be generated by a processor.

[0072] The classifier 220 can output a value calculated based on the features of the data 200 extracted by the feature extractor 210, indicating whether the input data 200 corresponds to the first class. The classifier 220 may be trained using multiple data corresponding to the first class. For example, the classifier 220 can output a value indicating whether the features of the input data 200 are similar to the features of the first class.

[0073] Softmax 230 is an activation function that normalizes the values ​​received from the classifier 220 to a range of 0 to 1 and outputs them. Softmax 230 can process the values ​​obtained from the classifier 220 and output the probability that the input data 200 corresponds to a specific class.

[0074] Figure 3A shows the process of training a machine learning algorithm according to one embodiment of the present disclosure. Figure 3B shows the process of determining whether input data belongs to a class according to one embodiment of the present disclosure. Figures 3A and 3B show the process of determining the class of input data 320 using a machine learning algorithm when a model for the first class is not generated. Referring to Figure 3A, the machine learning algorithm may be trained based on data 300 corresponding to the first class. For example, the machine learning algorithm can learn the features of the first class by training the first data 300 corresponding to the first class. Referring to Figure 3B, the machine learning algorithm can generate a representative image 310 based on the trained features of the first class, compare the input image 320 with the representative image 310, and determine whether the input image 320 belongs to the first class. For example, the machine learning algorithm can determine whether the representative image 310 of the first class and the input image 320 share similar features. For example, machine learning algorithms 312 and 322 can receive an input image 320 and a representative image 310, extract features from the representative image 310, and extract features from the input image 320. Softmax 330 can obtain features from the machine learning algorithms 312 and 322 for the representative image 310 and the input image 320, and determine whether the input image 320 belongs to the first class.

[0075] Figure 4 shows virtual data according to one embodiment of the present disclosure. The virtual data may be used to train a feature extractor that is included in common to each model, and a machine learning algorithm for determining whether the data corresponds to a class before the model is generated. According to one embodiment, the virtual data may be generated by adjusting the size, font, and weight of characters in the actual data and adding blur and defects. The virtual data may be generated using at least one algorithm from GAN (Generative Adversarial Networks) and stable diffusion. According to one embodiment, the virtual data may be generated by the processor of the electronic device 100. Referring to Figure 4, the multiple virtual data 400a, 400b, 400c, and 400d may be generated by changing the size and font of characters or adding blur and defects to the image actually stored in memory.

[0076] Figure 5 shows a comparison dataset generated for training input data according to one embodiment of the present disclosure. The processor can generate comparison datasets 510 and 520 for training input data 500. The processor can generate comparison datasets 510 and 520 that include similar data 510 having similar features to the input data 500 and randomly selected data 520. For example, referring to Figure 5, the processor can generate comparison datasets 510 and 520 that include similar data 510 "9670SG" having similar visual features to the input data 500 "96700G". Examples of similar data 510 included in comparison datasets 510 and 520 are not limited to those shown in Figure 5. In comparison datasets 510 and 520, the data 520 excluding the similar data 510 may be randomly selected from data stored in memory. By training with comparison datasets 510 and 520, the model can more accurately determine whether the input data 500 corresponds to a class or not.

[0077] Figure 6 shows the re-inspection process at the review station according to one embodiment of the present disclosure. The processor determines whether the input data corresponds to a first class as described in Figures 1 to 5 above, and if the input data does not correspond to a first class, it can send the inspection result for the input data to the review station 610. The review station 610 can re-inspect the processor's inspection result 600 for the input data based on the sample 612 for data corresponding to a first class. According to one embodiment, the review station 610 may be configured such that an operator directly compares the sample 612 with the input data to re-inspect the processor's inspection result 600.

[0078] The review station 610 can receive inspection results 600 from the processor regarding the input data. For example, it can receive inspection results 600 that determine that the first input data 602 and the second input data 604 do not correspond to the first class, and that the third input data 606 corresponds to the first class. The review station 610 can re-examine whether the above determination is correct based on the data sample 612. For example, as a result of the re-examination (620), it can be determined that the first input data 622 is defective but matches sample 612, so it corresponds to the first class, and the second input data 624 also matches sample 612, so it corresponds to the first class. Also, unlike the processor's inspection result 600 which determined that the third data 626 corresponds to the first class, it does not match sample 612, so it can be determined that it does not correspond to the first class.

[0079] Figure 7 is a block diagram of a model running on a GPU according to one embodiment of the present disclosure. The model for determining whether input data corresponds to a class can run on the GPU 700. That is, the model can output a determination result 740 indicating whether input data corresponds to a class using a feature extractor 710 and a classifier 720 running on the GPU 700. However, since the capacity of the GPU 700 is limited, it is difficult to load and use the classifiers of all models on the GPU 700 at once. Therefore, in order to conserve the capacity of the GPU 700, classifiers of models that are used more than a certain frequency may be loaded from memory 730 into the cache 722 on the GPU 700. For example, if 100 classifiers can be stored in the cache 722 on the GPU 700, the processor can load 100 classifiers from memory 730 into the cache 722 in order of frequency of use, starting with the model that is used most frequently among the multiple models.

[0080] In one embodiment, the processor can remove the least used classifier from the cache 722 based on how often the classifiers are used. For example, if all 100 classifiers are loaded into the cache 722, but a classifier not loaded into the cache 722 is needed, the processor can remove the least used classifier from the classifiers loaded into the cache 722 and load the required classifier from memory 730 into the cache 722.

[0081] Figure 8 shows an example of determining a reference probability using an electronic device according to one embodiment of the present disclosure. In the graph of Figure 8, the horizontal axis represents the class, and the vertical axis represents the probability calculated when multiple input data are input to the model of each class. The vertical lines in the graph represent the probability values ​​calculated by the model of each class for multiple input data, and the bars located at the lower and upper ends of the vertical lines may represent intervals where probability values ​​are most frequently distributed (for example, intervals where the top 25% to 75% of the data are distributed). In addition, the bubbles indicated for each class represent the reference probability for the model of that class. That is, the reference probability for the model of the first class 800 is the value indicated in the first bubble 806 (P x ), the baseline probability for the second class 810 model may be the value (0.9) indicated for the second bubble 816.

[0082] The processor can determine the baseline probability for each model based on the output values ​​of multiple input data for each model. The probability values ​​output when multiple input data are input to the model may be accumulated and stored in memory. The processor calculates the baseline probability by taking the minimum probability value of the input data determined to correspond to the first class 800 and the maximum probability value of the input data determined not to correspond to the first class 800, and a predetermined critical probability (P x Of these, the relatively higher value can be determined as the reference probability. For example, referring to Figure 8, among the data input to the first model, the maximum probability value of input data determined not to correspond to the first class 800 is 802, which is 0.1, and the minimum probability value of input data determined to correspond to the first class 800 is 804, which is 0.99, so the average of the two values ​​is approximately 0.5. Among the data input to the second model, the maximum probability value of input data determined not to correspond to the second class 810 is 812, which is 0.86, and the minimum probability value of input data determined to correspond to the second class 810 is 814, which is 0.98, so the average of the two values ​​is 816, which is approximately 0.9. Critical probability (P xSince ) is approximately 0.7, the processor can determine that the reference probability for the first class 800 model is 0.7 and the reference probability for the second class 810 model is 0.9. According to one embodiment, the processor can reset the reference probability each time the model is generated or retrained. According to another embodiment, the processor can reset the reference probability at predetermined time intervals.

[0083] Figure 9 is a flowchart showing a method for determining whether a first data falls under a first class in an electronic device according to one embodiment of the present disclosure. The electronic device can acquire input data in operation 900. For example, the input data may be an image of a component placed on a substrate.

[0084] In operation 910, the electronic device 100 can check whether a model for the first class corresponding to the first data is stored in memory 110. If the first model is stored, in operation 912, the electronic device 100 can use the first model to determine whether the first data corresponds to the first class. If the first model is not stored, in operation 914, the electronic device 100 can determine whether the input data corresponds to the first class based on the first data corresponding to the first class.

[0085] In operation 920, the electronic device 100 can transmit information to the external device 101 indicating whether the first data corresponds to the first class. The electronic device 100 can determine whether the first data corresponds to the first class and transmit the inspection result to the review station 610.

[0086] Figure 10 is a flowchart of a method for generating or retraining a model for a first class in an electronic device according to one embodiment of the present disclosure. The electronic device 100 can acquire a learning request for input data in operation 1000. In response to acquiring the learning request, the electronic device 100 can check in operation 1010 whether or not a model for a first class corresponding to the input data is stored in memory 110.

[0087] If the first model is stored in memory 110, the electronic device 100 can check in operation 1020 whether predetermined conditions are met. According to one embodiment, the electronic device 100 can decide whether to retrain the current model based on the number of errors that occurred in determining whether each piece of data corresponds to a first class using the first model. In operation 1022, the electronic device 100 can decide to retrain the first model based on the number of first errors in determining whether data that does not correspond to a first class corresponds to a first class, and the number of second errors in determining that data that does correspond to a first class does not correspond to a first class, among the results of determining whether input data corresponds to a first class using the first model. If predetermined conditions are not met, the electronic device 100 does not need to retrain the first model.

[0088] If the first model is not stored in memory 110, the electronic device 100 may, in operation 1030, add the input data to the learning queue for generating the first model. According to one embodiment, the electronic device 100 may add the input data to the learning queue if a predetermined number (e.g., 100) or more of data corresponding to the first class are stored and data corresponding to the first class does not already exist in the learning queue. The electronic device 100 may, in operation 1032, generate a model for the first class.

[0089] While the steps of the method or algorithm relating to this disclosure are sequentially described in the flowcharts shown in Figures 9 and 10, the steps may be performed in any order that can be combined as arbitrarily as permitted by this disclosure, in addition to being performed sequentially. This flowchart description does not preclude changes or modifications to the method or algorithm, nor does it imply that any particular step is essential or preferred. In one embodiment, at least some steps may be performed in parallel, iteratively, or heuristically. In one embodiment, at least some steps may be omitted, and other steps may be added.

[0090] Various embodiments of this disclosure may be embodied as software on a machine-readable storage medium. The software may be software for embodying various embodiments of this disclosure. The software can be inferred from various embodiments of this disclosure by a programmer in the art to which this disclosure belongs. For example, the software may be a program containing machine-readable instructions (e.g., codes or code segments). The machine is a device capable of operating in response to instructions called from the storage medium, and may be, for example, a computer. In one embodiment, the machine may be an electronic device 100 according to an embodiment of this disclosure. In one embodiment, the processor of the machine can execute a called instruction and cause the components of the machine to perform a function corresponding to the instruction. In one embodiment, the processor may be a processor 120 according to an embodiment of this disclosure. The storage medium can mean any type of recording medium on which data is stored and is readable by the machine. The storage medium may include, for example, ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. In one embodiment, the storage medium may be memory 110. In one embodiment, the storage medium may be implemented in a distributed form, such as a network-connected computer system. The software may be stored and executed in a distributed manner on the computer system. The storage medium may be a non-transitory storage medium. A non-transitory storage medium means a tangible medium that exists regardless of whether the data is stored semi-permanently or temporarily, and does not include signals that are transmitted transiently.

[0091] While the technical concept of this disclosure has been explained through various embodiments, the technical concept of this disclosure includes various substitutions, modifications, and alterations that can be understood by a person with ordinary skill in the art to which this disclosure pertains. Furthermore, such substitutions, modifications, and alterations should be understood to be included within the scope of the attached claims.

Claims

1. One or more processors, Includes one or more memories in which instructions are stored that cause the one or more processors to perform calculations when the execution is performed by the one or more processors, The one or more processors described above are: First data is obtained regarding the components to be placed on the circuit board. Determine whether a model for the first class corresponding to the first data is stored in one or more of the memory. In response to the determination that the model for the first class is saved, the model is used to determine whether the first data corresponds to the first class. In response to the determination that the model for the first class is not saved, the second data corresponding to the first class is used to determine whether the first data corresponds to the first class. A device that transmits information to an external device indicating whether the first data corresponds to the first class.

2. When one or more processors use the model to determine whether the first data corresponds to the first class, Using the aforementioned model, calculate the probability that the first data belongs to the first class. Determine whether the aforementioned probability is greater than or equal to a predetermined reference probability. If the aforementioned probability is greater than or equal to a predetermined reference probability, the first data is determined to belong to the first class. The apparatus according to claim 1, wherein if the probability is less than a predetermined reference probability, it is determined that the first data does not belong to the first class.

3. The apparatus according to claim 2, wherein the predetermined reference probability is determined using the average value of the minimum probability of one or more data determined to belong to the first class and the maximum probability of one or more data determined not to belong to the first class.

4. The one or more processors described above are: In response to the determination that the model for the first class is not saved, the similarity between the first data and the second data is calculated using a machine learning algorithm. The apparatus according to claim 1, wherein the first data is determined to be in the first class based on the similarity between the first data and the second data.

5. The apparatus according to claim 4, wherein the machine learning algorithm includes ArcFace (Additive Angular Margin Loss).

6. The aforementioned machine learning algorithm is The apparatus according to claim 4, which is trained using a plurality of arbitrarily generated virtual data and the second data.

7. The aforementioned multiple virtual data are The apparatus according to claim 6, which adjusts the size, font, and thickness of characters in one or more actual data stored in one or more of the aforementioned memories, and adds blur and defects to generate the data.

8. The aforementioned multiple virtual data are The apparatus according to claim 6, which is generated using at least one algorithm from among GAN (Generative Adversarial Networks) and stable diffusion.

9. The one or more processors described above are: Obtain a learning request for the first data mentioned above, Upon obtaining the aforementioned learning request, it is determined whether or not the model for the first class is saved. In response to the determination that the model for the first class is not saved, the first data is added to the learning queue for generating or training the model. The apparatus according to claim 1, wherein, in response to the determination that a model for the first class is stored, if the model for the first class satisfies predetermined conditions, the first data is added to the learning queue.

10. When one or more processors add the first data to the learning queue in response to a decision that the model for the first class is not saved, The apparatus according to claim 9, wherein if the number of one or more data items corresponding to the first class stored in one or more memory locations is greater than a predetermined value, and the first data is not present in the learning waiting queue, the first data is added to the learning waiting queue.

11. The aforementioned predetermined conditions are: The apparatus according to claim 9, wherein the condition is satisfied when the model determines that data that does not belong to the first class belongs to the first class, and the number of times it determines that data that belongs to the first class does not belong to the first class, is greater than a first threshold, or the number of times it determines that data that belongs to the first class does not belong to the first class, is greater than a second threshold.

12. The one or more processors described above are: The apparatus according to claim 9, which trains the model using a dataset (set) that includes data belonging to one or more classes similar to the first class and arbitrarily selected data.

13. The apparatus according to claim 12, wherein the ratio of data in the dataset that corresponds to one or more classes similar to the first class is less than or equal to the ratio of arbitrarily selected data in the dataset.

14. The aforementioned external device is a display, The one or more processors described above are: The apparatus according to claim 1, wherein, in response to the determination that a model for the first class and the second data corresponding to the class are not stored, a statement indicating that it is impossible to determine whether the first data corresponds to the first class is output to the display.

15. The apparatus according to claim 1, wherein the model includes a feature extractor and a classifier.

16. The apparatus according to claim 15, wherein the feature extractor is applied in common to the models stored in one or more memories, and the classifier is learned based on different information for each of the models stored in the one or more memories.

17. The apparatus according to claim 15, wherein the feature extractor is trained using a plurality of arbitrarily generated virtual data.

18. The feature extractor and classifier mentioned above operate on a GPU (Graphics Processing Unit), The one or more processors described above are: Classifiers included in the model of a class used at a frequency greater than a predetermined value are loaded from the memory into the cache on the GPU. The apparatus according to claim 15, which determines whether data belongs to a class using the feature extractor and the classifier loaded into the cache.

19. The one or more processors described above are: The apparatus according to claim 18, which selects the least used classifier from among the one or more classifiers loaded into the cache and removes it from the cache.

20. The one or more processors described above are: The apparatus according to claim 1, wherein, in response to a determination that the first data does not belong to the first class, it transmits information indicating that the first data does not belong to the first class to a review station.

21. A method performed in a device including one or more processors and one or more memories in which instructions to be executed by the one or more processors are stored, The first step involves acquiring data that specifies which components to place on the circuit board, The steps include determining whether or not a model for the first class corresponding to the first data is stored in one or more memory locations, In response to the decision that the model for the first class is saved, the step of using the model to determine whether the first data corresponds to the first class, In response to the determination that a model for the first class is not saved, the steps include determining whether the first data corresponds to the first class using the second data corresponding to the first class, A method comprising the step of transmitting to an external device information indicating whether the first data corresponds to the first class.

22. A non-temporary computer-readable recording medium that records instructions causing one or more processors to perform calculations when executed by one or more processors, The instruction is performed by one or more processors, The first step involves acquiring data that specifies which components to place on the circuit board, The steps include determining whether or not a model for the first class corresponding to the first data is stored in one or more memory locations, In response to the decision that the model for the first class is saved, the step of using the model to determine whether the first data corresponds to the first class, In response to the determination that the model for the first class is not saved, the steps include determining whether the first data corresponds to the first class using the second data corresponding to the first class, A non-temporary computer-readable recording medium that performs the steps of: transmitting information to an external device indicating whether the first data corresponds to the first class; and

Images