Apparatus and method for determining category of data, and recording medium recording instructions

Abstract

Techniques for determining a category of data are disclosed. An apparatus of one aspect of the present disclosure may include: one or more processors; and one or more memories storing instructions that, when executed by means of the one or more processors, cause the one or more processors to perform operations; wherein the one or more processors may obtain first data regarding a component configured on a substrate, determine whether a model of a first category corresponding to the first data is stored in the one or more memories, and, when it is determined that the model of the first category is stored, determine whether the model of the first category is stored in the one or more memories. Determining whether the first data corresponds to the first category by using the model when the model of the first category is determined to be stored, and determining whether the first data corresponds to the first category by using second data corresponding to the first category when the model of the first category is determined not to be stored, information indicating whether the first data corresponds to the first category is transmitted to an external device.

Description

Technical Field

[0001] This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 439,727, filed on January 18, 2023. Provisional Application No. 63 / 439,727 is hereby incorporated by reference into this disclosure.

[0002] The present disclosure relates to techniques for determining the category of data. Background Art

[0003] Surface Mount Technology (SMT) is used to place one or more components on a substrate. A large number of components are bonded to the pads of the substrate. To ensure proper bonding between the components and the pads, each component must be positioned correctly on the substrate. Automated Optical Inspection (AOI) devices use optical measurement technology to detect whether each component is positioned correctly on the substrate. Once the components are positioned correctly on the substrate, other components can be placed on the substrate.

[0004] However, if a component is not correctly positioned on the substrate, the substrate can be transferred to a review station for re-inspection based on the inspection results of the automated optical inspection system. At the review station, an operator can re-inspect the component that was determined by the automated optical inspection system to be incorrectly bonded to the substrate and make a final determination as to whether an error has occurred. Summary of the Invention

[0005] Technical issues

[0006] At least one embodiment of the present disclosure may provide a technology that can more accurately determine the category of data.

[0007] At least one embodiment of the present disclosure can reduce false calls that may occur when a non-defective substrate is identified as defective.

[0008] At least one embodiment of the present disclosure can reduce the number of substrates to be re-inspected at an inspection station.

[0009] At least one embodiment of the present disclosure can generate models for various categories and use the generated models to determine the category of input data.

[0010] At least one embodiment of the present disclosure can determine the category of input data even when a model of the category corresponding to the input data has not been generated.

[0011] Technical Solution

[0012] An apparatus according to one aspect of the present disclosure may include: one or more processors; and one or more memories, wherein the memories store instructions which, when run with the aid of the one or more processors, enable the one or more processors to perform operations; wherein the one or more processors may obtain first data about components configured on a substrate, determine whether a model of a first category corresponding to the first data is stored in the one or more memories, and when it is determined that the model of the first category is stored, use the model to determine whether the first data corresponds to the first category, and when it is determined that the model of the first category is not stored, use second data corresponding to the first category to determine whether the first data corresponds to the first category, and transmit information indicating whether the first data corresponds to the first category to an external device.

[0013] In one embodiment, when the one or more processors use the model to determine whether the first data corresponds to the first category, they can use the model to calculate the probability of the first data corresponding to the first category, and determine whether the probability is above a predetermined baseline probability. If the probability is above a predetermined baseline probability, it is determined that the first data corresponds to the first category; if the probability does not reach the predetermined baseline probability, it is determined that the first data does not correspond to the first category.

[0014] In one embodiment, the predetermined baseline probability can be determined using the average of the minimum value of the probability of more than one data being determined to correspond to the first category and the maximum value of the probability of more than one data not corresponding to the first category.

[0015] In one embodiment, when it is determined that the model of the first category is not stored, the one or more processors may use a machine learning algorithm to calculate the similarity between the first data and the second data, and determine whether the first data corresponds to the first category based on the similarity between the first data and the second data.

[0016] In one embodiment, the machine learning algorithm may include ArcFace (Additive Angular MarginLoss).

[0017] In one embodiment, the machine learning algorithm may be trained using a plurality of arbitrarily generated virtual data and the second data.

[0018] In one embodiment, the plurality of virtual data may be generated by adjusting the size, font, and thickness of characters and adding blur and defects to the one or more actual data stored in the one or more memories.

[0019] In one embodiment, the plurality of virtual data may be generated using at least one of a Generative Adversarial Networks (GAN) and a stable diffusion algorithm.

[0020] In one embodiment, the one or more processors may obtain a training request for the first data, and when obtaining the training request, determine whether the model of the first category has been stored; when it is determined that the model of the first category has not been stored, add the first data to a training queue for generating or training the model; when it is determined that the model of the first category has been stored, if the model of the first category meets a predetermined condition, add the first data to the training queue.

[0021] In one embodiment, when the one or more processors add the first data to the training queue as it is determined that the model of the first category is not stored, the first data may be added to the training queue if the number of one or more data corresponding to the first category stored in the one or more memories is greater than a predetermined value and the first data is not in the training queue.

[0022] In one embodiment, the predetermined condition may be a condition that is satisfied when the model judges data that does not correspond to the first category as corresponding to the first category a number of times greater than a first critical value, or judges data that corresponds to the first category as not corresponding to the first category a number of times greater than a second critical value.

[0023] In one embodiment, the one or more processors may train the model using a data set including data corresponding to one or more categories similar to the first category and arbitrarily selected data.

[0024] In one embodiment, the ratio of data corresponding to one or more categories similar to the first category in the dataset may be less than the ratio of the arbitrarily selected data in the dataset.

[0025] In one embodiment, the external device may be a display device. When it is determined that the model of the first category and the second data corresponding to the category are not stored, the one or more processors may output a statement to the display device indicating that it is impossible to determine whether the first data corresponds to the first category.

[0026] In one embodiment, the model may include a feature extractor and a classifier.

[0027] In one embodiment, the feature extractor may be commonly applied to the models stored in the one or more memories, and the classifier may be trained based on different information for each model stored in the one or more memories.

[0028] In one embodiment, the feature extractor can be trained using a plurality of arbitrarily generated virtual data.

[0029] In one embodiment, the feature extractor and classifier may run on a graphics processing unit (GPU), and the one or more processors may load the classifier included in the model of the category used at a frequency above a predetermined value from the memory into a cache on the GPU, and use the feature extractor and the classifier loaded into the cache to determine whether the data corresponds to the category.

[0030] In one embodiment, the one or more processors may select a least used classifier from among the one or more classifiers loaded into the cache and remove it from the cache.

[0031] In one embodiment, when the one or more processors determine that the first data does not correspond to the first category, they may transmit information indicating that the first data does not correspond to the first category to a review station.

[0032] An electronic device according to an embodiment of the present disclosure includes one or more processors and one or more memories, wherein the one or more memories store instructions to be executed by means of the one or more processors, and the one or more processors are configured to execute the method of the present disclosure when the instructions are executed by means of the one or more processors.

[0033] A non-transitory computer-readable recording medium according to an embodiment of the present disclosure records instructions that, when executed by one or more processors, enable the one or more processors to perform actions. The instructions are configured to enable the one or more processors to execute the method of the present disclosure.

[0034] Effects of the Invention

[0035] According to at least one embodiment of the present disclosure, a technology capable of more accurately determining the category of data can be provided.

[0036] According to at least one embodiment of the present disclosure, the misjudgment of a non-defective substrate as defective can be reduced.

[0037] According to at least one embodiment of the present disclosure, the number of substrates to be re-inspected at an inspection station can be reduced.

[0038] According to at least one embodiment of the present disclosure, models of various categories may be generated and the category of input data may be determined using the generated models.

[0039] According to at least one embodiment of the present disclosure, even when a model of a category corresponding to the input data is not generated, the category of the input data can be determined.

[0040] The effects of the present disclosure are not limited to the effects mentioned above, and a person skilled in the art can clearly understand other effects not mentioned from the description in the specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 FIG. 4 is a block diagram of an electronic device according to an embodiment of the present disclosure.

[0042] Figure 2 A block diagram of a model according to an embodiment of the present disclosure.

[0043] Figure 3a According to an embodiment of the present disclosure, a process of training a machine learning algorithm is shown.

[0044] Figure 3b According to an embodiment of the present disclosure, a process of determining whether input data corresponds to a category is shown.

[0045] Figure 4 Dummy data according to an embodiment of the present disclosure is shown.

[0046] Figure 5 The following illustrates a comparison data set generated for learning input data according to an embodiment of the present disclosure.

[0047] Figure 6 According to an embodiment of the present disclosure, a re-inspection process in a detection station is shown.

[0048] Figure 7 A block diagram of a model running on a GPU according to an embodiment of the present disclosure.

[0049] Figure 8 An embodiment of determining a baseline probability by an electronic device according to an embodiment of the present disclosure is shown.

[0050] Figure 9 A sequence diagram of a method for an electronic device to determine whether first data corresponds to a first category according to an embodiment of the present disclosure.

[0051] Figure 10 A sequence diagram of a method for generating or retraining a first category model for an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0052] The various embodiments described in this document are provided for the purpose of clearly describing the technical concept of the present disclosure and are not intended to limit them to specific implementation forms. The technical concept of the present disclosure includes various modifications, equivalents, alternatives, and embodiments that selectively combine all or part of the various embodiments described in this document. Furthermore, the scope of the technical concept of the present disclosure is not limited to the various embodiments set forth below or their specific descriptions.

[0053] Including technical or scientific terms, as long as they are not defined differently, the terms used in this document may have the meanings that are generally understood by one of ordinary skill in the art to which the present disclosure belongs.

[0054] As used in this document, expressions such as "including," "may include," "have," "may have," "have," and "may have" imply the presence of the subject feature (e.g., function, action, or component), and do not preclude the presence of additional features. In other words, such expressions should be understood as open-ended terms that include the possibility of other embodiments.

[0055] Singular expressions used in this document may include plural expressions unless the context indicates otherwise. This also applies to singular expressions in the claims.

[0056] As used in this document, expressions such as "first", "second", or "the first", "the second", etc. are used to distinguish one object from other objects when referring to multiple objects of the same kind, unless the context indicates otherwise, and do not limit the order or importance of the corresponding objects.

[0057] As used herein, expressions such as "A, B, and C," "A, B or C," "A, B and / or C," or "at least one of A, B, and C," "at least one of A, B, or C," or "at least one of A, B, and / or C" may refer to each listed item or all possible combinations of the listed items. For example, "at least one of A or B" may refer to (1) at least one A, (2) at least one B, or (3) at least one A and at least one B.

[0058] The expression "based on..." as used in this document is used to describe one or more factors that influence the determination, judgment, behavior, or action described in the sentence or article containing the corresponding expression. Such expression does not exclude additional factors that influence the corresponding determination, judgment, behavior, or action.

[0059] As used in this document, the expression that a certain component (e.g., a first component) is “connected” or “connected” to another component (e.g., a second component) not only means that the component is directly connected or connected to the other component, but also means that the component is connected or connected through a new other component (e.g., a third component).

[0060] As used in this document, the expression "configured to" may have the meanings of "provided to," "capable of," "changed to," "formed to," "capable of performing," etc., depending on the context. Such expressions are not limited to the meaning of "specifically designed in hardware." For example, a processor configured to perform a specific action may refer to a generic-purpose processor capable of performing that specific action by running software.

[0061] Various embodiments of the present disclosure are described below with reference to the accompanying drawings. In the drawings and their descriptions, identical or substantially equivalent components are given the same reference numerals. In the following descriptions of various embodiments, repeated descriptions of identical or corresponding components may be omitted, but this does not mean that such components are not included in the embodiments.

[0062] Figure 1 1 is a block diagram of an electronic device according to an embodiment of the present disclosure. According to one embodiment, device 100 may include a memory 110, a processor 120, and / or a communication interface 130. In certain embodiments, at least one of these components of electronic device 100 may be omitted, or other components may be added to electronic device 100. In certain embodiments, some components may be integrated or implemented as a single or multiple components, either additionally or alternatively.

[0063] At least some of the components inside and outside the electronic device 100 can be interconnected via a bus, GPIO (general purpose input / output), SPI (serial peripheral interface) or MIPI (mobile industry processor interface) to send and receive data and / or signals.

[0064] The processor 120 can drive software to control at least one component of the electronic device 100 connected to the processor 120. Furthermore, the processor 120 can perform various operations, such as calculations, processing, data generation, and processing, associated with the present disclosure. Furthermore, the processor 120 can load data from the memory 110 or store data in the memory 110. In one embodiment, the processor 120 can receive images of various components configured on a substrate as input data and determine whether the components are configured in the correct locations on the substrate. A component can refer to one or more components configured on a substrate.

[0065] The memory 110 may store various data. The data stored in the memory 110 may include software, as data obtained, processed, or used by at least one component of the electronic device 100. The memory 110 may include volatile and / or nonvolatile memory.

[0066] The communication interface 130 can perform wireless or wired communication between the electronic device 100 and a server or between the electronic device 100 and other external electronic devices. For example, the communication interface 130 can perform wireless communication based on LTE (long-term evolution), LTE-A (LTE Advance), CDMA (code division multiple access), WCDMA (wideband CDMA), WiBro (wireless broadband), WiFi (wireless fidelity), Bluetooth, NFC (near field communication), GPS (Global Positioning System), or GNSS (global navigation satellite system). For example, the communication interface 130 can perform wired communication based on USB (universal serial bus), HDMI (high definition multimedia interface), RS-232 (recommended standard 232), or POTS (plain old telephone service). The communication interface 130 may be omitted depending on the embodiment.

[0067] In one embodiment, the electronic device 100 can communicate with the external device 101 through a server or directly. The external device 101 can be the same or a different type of device as the electronic device 100. All or part of the actions performed by the electronic device 100 can also be performed by the external device 101. In one embodiment, the electronic device 100 can transmit the results of the executed actions to the external device 101. The external device 101 can use the received data to perform subsequent actions. According to one embodiment, the external device 101 can be a server within a review station. According to one embodiment, when the electronic device is an automated optical inspection device, light or patterned light generated by an illumination unit within the automated optical inspection device can be irradiated onto a test object, and a camera unit can receive light reflected from the test object (e.g., a PCB substrate or component) to obtain an image of the test object. The automated optical inspection device can then perform a test on the image of the test object based on pre-set benchmark information to determine whether the test object is defective, that is, to determine whether the test object is good (GOOD) or bad (NG). At this time, if the inspection object is judged to be good, the inspection object is transferred to the next production line in the SMT production line. If the inspection object is judged to be defective, the image of the corresponding inspection object is re-inspected in the automatic optical inspection device, or transmitted to the server of the inspection station for re-inspection through the server.

[0068] For example, during a first inspection, optical character recognition (OCR) or optical character verification (OCR) is used to determine whether the target character is recognized in the inspection image, thereby determining whether it is good or bad. A re-inspection can determine whether the image is similar.

[0069] That is, the server at the testing station can receive the test results of the data from the electronic device 100 and output them to a display device so that an operator at the testing station can review the test results. Based on the review results of the data, the testing station server can ultimately determine whether the data corresponds to the category. The review of the data performed by the testing station can be performed manually by the operator.

[0070] The processor 120 may obtain input data regarding components configured on a substrate. A component may refer to one or more components configured on a substrate. The positions of each component configured on the substrate may have been determined. For example, a first component may be configured at a first position on the substrate, and a second component may be configured at a second position on the substrate. The data may refer to an image including information regarding the shape and size of the component, characters, numbers, or symbols recorded on the component or a test object such as a PCB. Alternatively, the data may refer to an image including the shape of cracks, foreign matter, fillets, solder bridges, lifted leads, component or lead offsets, and the like of the component.

[0071] According to one embodiment, the processor 120 may perform an action of determining the category of the obtained data. Each data may correspond to a category. A category is a classification of data corresponding to a specific component, which may be generated for each component and may include more than one data corresponding to one component. For example, a first category corresponding to a first component may include first data and second data of the first component, and a second category corresponding to a second component may include third data and fourth data of the second component. According to one embodiment, the data may be an image of a component. For example, the first data and the second data may be images of the first component, and the third data and the fourth data may be images of the second component. For ease of description below, the input data is described as data corresponding to the first category.

[0072] The processor 120 may confirm whether a model of a category corresponding to the obtained input data is stored in the memory 110. In response to the obtained input data, the processor 120 may confirm whether a model of a first category is stored in the memory 110. A model may refer to an algorithm for determining whether the input data is data corresponding to a specific category (e.g., a single-class classification based on a convolutional neural network (OC-CNN)), and different models may be generated and stored by category. For example, when a first model of a first category is stored in the memory 110, the processor 120 may use the first model to determine that the first data corresponds to the first category and the third data does not correspond to the first category.

[0073] According to one embodiment, the model for each category can calculate the probability that the input data corresponds to the category. For example, the probability that the first input data corresponds to the first category can be calculated to be 0.6, and the probability that the second input data corresponds to the first category can be calculated to be 0.9. The processor 120 can determine a baseline probability for determining whether data corresponds to a category and compare the probability calculated by the model with the baseline probability to determine whether the input data corresponds to the corresponding category. If the probability calculated by the model is above the baseline probability, the processor 120 can determine that the input data corresponds to the corresponding category. If the probability calculated by the model does not reach the baseline probability, the processor 120 can determine that the input data does not correspond to the corresponding category. For example, when the baseline probability is determined to be 0.8, the processor 120 can determine that the first input data does not correspond to the first category, and the second input data does correspond to the first category. The processor 120 can accumulate and store the judgment results of the multiple input data as described above. That is, the probability calculated by the model for each input data and the result of determining whether it corresponds to the corresponding category can be accumulated and stored.

[0074] According to one embodiment, processor 120 may determine a baseline probability for each category based on the accumulated judgment results. Processor 120 may calculate the average of the minimum value among the probabilities calculated for data corresponding to the corresponding category and the maximum value among the probabilities calculated for data not corresponding to the corresponding category. The processor may determine the higher value between the average value and a predetermined threshold probability as the baseline probability. For example, when determining whether input data corresponds to the first category, the probability of the first input data determined not to correspond to the first category may be determined as follows: the probability of the first input data determined not to correspond to the first category is 0.4, the probability of the second input data is 0.6, the probability of the third input data determined to correspond to the first category is 0.98, and the threshold probability is 0.7. In this case, the highest value among the probabilities calculated for the input data determined not to correspond to the first category is 0.6 for the second input data, and the lowest value among the probabilities calculated for the input data determined to correspond to the first category is 0.9 for the third input data. The average of these two values ​​is 0.75, which is higher than the threshold probability of 0.7, so processor 120 may determine the baseline probability as 0.75. On the contrary, when the average of the two determined values ​​is lower than 0.7, the processor 120 may determine the base probability as 0.7.

[0075] When the first model is not stored in the memory 110, the processor 120 may confirm whether the memory 110 stores at least one data corresponding to the first category. When at least one data corresponding to the first category is stored, the processor 120 may use the stored data to determine whether the input data corresponds to the first category. In other words, even if the first model has not been generated, the processor 120 may use the stored data to determine whether the input data corresponds to the first category. According to one embodiment, the processor 120 may use a machine learning algorithm (e.g., metric learning, ArcFace (Additive Angular Margin Loss)) to determine whether the input data corresponds to the first category. A metric learning algorithm is a machine learning algorithm used to calculate the similarity between two input data to determine whether the two input data are similar. Even if the first model has not been generated, the processor 120 may use the metric learning algorithm to determine whether the input data corresponds to the first category. For example, the processor 120 may calculate the similarity between the first data or second data corresponding to the first category and the input data. If the calculated similarity is above a predetermined value, it may be determined that the input data corresponds to the first category.

[0076] Conversely, when no data corresponding to the first category is stored, processor 120 may output information indicating that the category of the first data cannot be determined. For example, processor 120 may output a statement indicating that the category of the first data cannot be determined (e.g., "Model not found") to an external device, which may be a display device.

[0077] The following describes methods for generating models for each category. For ease of description, this description assumes that data corresponding to the first category is received. Processor 120 can obtain a training request for input data from a training server. The training server is a component introduced to electronic device 100 for learning input data. Upon receiving input data, it can transmit an initial training request or a fine-tuning request to the processor.

[0078] As the training request is obtained, the processor 120 may confirm whether the first model is stored in the memory 110. When the first model is not stored, the processor 120 may add input data to the training queue in order to generate the model. According to one embodiment, the processor 120 may sequentially learn the data included in the training queue to generate a model of the category. This model generation process is equivalent to the initial training process (initial training). When there is not much data input and not much model generation, most of the training requests from the training server are requests related to the initial training process. According to one embodiment, if more than a predetermined number (e.g., 50 or 100) of data corresponding to the first category are stored, and there is no data corresponding to the first category in the training queue, the processor 120 may add the input data to the training queue for generating the first model. This is because, in order to generate a highly accurate model, more than a predetermined number of data is required. If the input data for generating the first model already exists in the training queue, there is no need to repeat the training. According to one embodiment, more than a predetermined amount of data corresponding to the first category has been stored, but data corresponding to the first category already exists in the training queue, or although data corresponding to the first category does not exist in the training queue, the amount of data corresponding to the first category is less than a predetermined amount. In this case, the processor may not add the input data to the training queue even if it receives a training request.

[0079] When the first model is stored, the processor 120 may add the input data to the training queue in order to retrain the currently generated model using the input data. This process may be equivalent to fine tuning. According to one embodiment, the processor 120 may make a decision to retrain the current model based on the number of errors in the results of determining whether each data corresponds to the first category using the first model. For example, the processor 120 may determine to retrain the first model based on the number of first errors and the number of second errors in the results of determining whether the input data corresponds to the first category using the first model. The first error is that data that does not correspond to the first category is judged to correspond to the first category, and the second error is that data corresponding to the first category is judged to not correspond to the first category. For example, if the number of first errors exceeds a predetermined value (e.g., 20) or the number of second errors exceeds a predetermined value (e.g., 100), the processor 120 may determine to retrain the first model.

[0080] The processor 120 may generate a comparison data set in order to learn the input data. The processor 120 may generate a comparison data set in order to improve the accuracy of the generated model, and the comparison data set includes data with similar features to the input data (hereinafter referred to as similar data). Similar data means data whose difference in overall size and the size of characters included in each data is less than a predetermined value from the input data, or whose difference in the number of strokes of the characters included in each data is less than a predetermined number of times. For example, when the input data is "188", the comparison data set may be generated by similar data with similar visual features, such as "100", "168", "186", and "180". The processor 120 may use the comparison data set to learn the input data so that the generated model can more accurately determine whether the input data corresponds to the category.

[0081] According to one embodiment, processor 120 may determine the proportion of similar data in the comparison dataset. For example, processor 120 may determine the proportion of similar data in the comparison dataset to be a predetermined value (e.g., 25%), and determine the remainder as random data. Processor 120 may use the randomly determined data to extract features of the input data and, on the other hand, compare the similar data with the input data to more accurately extract features of the input data. This may generate a model that more accurately determines which data corresponds to a category.

[0082] The electronic device 100 of various embodiments of the present disclosure can be a device of various forms. For example, the electronic device 100 can be a portable communication device, a computer device, a portable multimedia device, a wearable device, a household appliance, a placement machine in an SMT production line, an AOI configured at the back end of a reflow oven, an inspection station linked to the AOI, a smart factory monitoring system linked to an SMT production line, or a device comprising one or more of the above devices. The electronic device 100 of the present disclosure is not limited to the aforementioned devices.

[0083] Figure 2 This is a block diagram of a model of an embodiment of the present disclosure. Figure 2 , the model may include a feature extractor 210, a classifier 220 and a softmax 230. Different models may be generated for each category. Figure 1 , may be generated based on the input data 200. According to one embodiment, multiple models may share the same feature extractor 210 but include different classifiers 220. For example, a first model of a first category may include a general feature extractor 210 and a first classifier, and a second model of a second category may include a general feature extractor 210 and a second classifier.

[0084] The feature extractor 210 can extract features from the data 200 to determine whether the input data 200 corresponds to the first category. The feature extractor 210 can be trained using multiple input data 200 or virtual data. The feature extractor 210 is a component for extracting features from the input data 200 and requires training using massive amounts of data. The feature extractor 210 can be trained using multiple virtual data generated based on the input data. Virtual data is not actual data and can be data generated for the purpose of training the feature extractor 210. According to one embodiment, the virtual data used to train the feature extractor 210 can be generated by a processor.

[0085] Classifier 220 may output a value calculated based on the features of data 200 extracted by feature extractor 210 to determine whether input data 200 corresponds to the first category. Classifier 220 may be trained using a plurality of data corresponding to the first category. For example, classifier 220 may output a value indicating whether the features of input data 200 are similar to the features of the first category.

[0086] The softmax 230 is an activation function that normalizes the value received from the classifier 220 to a value between 0 and 1 and outputs the value. The softmax 230 processes the value obtained from the classifier 220 and outputs the probability that the input data 200 corresponds to a specific category.

[0087] Figure 3a According to an embodiment of the present disclosure, a process of training a machine learning algorithm is shown. Figure 3b According to an embodiment of the present disclosure, a process of determining whether input data corresponds to a category is shown. Figure 3a and Figure 3b FIG. 3 shows a process of determining the category of input data 320 using a machine learning algorithm when a model of the first category is not generated. Figure 3a , the machine learning algorithm may be trained based on the data 300 corresponding to the first category. For example, the machine learning algorithm may learn the first data 300 corresponding to the first category to train on the features of the first category. Figure 3b The machine learning algorithm may generate a representative image 310 based on the learned features of the first category, and compare the input image 320 with the representative image 310 to determine whether the input image 320 corresponds to the first category. For example, the machine learning algorithm may determine whether the representative image 310 of the first category and the input image 320 share similar features. For example, the machine learning algorithms 312 and 322 may receive the input image 320 and the representative image 310, extract features of the representative image 310, and extract features of the input image 320. Softmax 330 may obtain the features of the representative image 310 and the input image 320 from the machine learning algorithms 312 and 322, and determine whether the input image 320 corresponds to the first category.

[0088] Figure 4 Virtual data of an embodiment of the present disclosure is shown. The virtual data can be used to train feature extractors commonly included in each model, as well as machine learning algorithms for determining whether data corresponds to a category before generating a model. According to one embodiment, virtual data can be generated by adjusting the size, font, and thickness of characters in actual data and adding blur and defects. The virtual data can be generated using at least one algorithm of Generative Adversarial Networks (GAN) and stable diffusion. According to one embodiment, the virtual data can be generated by a processor of the electronic device 100. Reference Figure 4 , a plurality of virtual data 400a, 400b, 400c, 400d can be generated by changing the size and font of characters in the image actually stored in the memory, or adding blur and defects.

[0089] Figure 5 FIG2 shows a comparative data set generated for learning input data according to an embodiment of the present disclosure. The processor may generate comparative data sets 510 and 520 for learning input data 500. The processor may generate comparative data sets 510 and 520 including similar data 510 having similar features to the input data 500 and randomly selected data 520. For example, referring to FIG2 Figure 5 , the processor may generate the following comparison data sets 510, 520, that is, including similar data 510 "9670SG" having similar visual features to the input data 500 "96700G". Examples of similar data 510 included in the comparison data sets 510, 520 are not limited to Figure 5 In the comparison data sets 510 and 520, data 520 other than the similar data 510 can be randomly selected from the data stored in the memory. The model is trained using the comparison data sets 510 and 520 and can more accurately determine whether the input data 500 corresponds to a category.

[0090] Figure 6 According to an embodiment of the present disclosure, the re-inspection process in the inspection station is shown. The processor can Figures 1 to 5 6. The processor determines whether the input data corresponds to the first category based on the content described in

[15] . When the input data does not correspond to the first category, the detection result of the input data can be transmitted to the detection station 610. The detection station 610 can recheck the detection result 600 of the input data by the processor based on the sample 612 of the data corresponding to the first category. According to one embodiment, the detection station 610 can allow an operator to directly compare the sample 612 with the input data to recheck the detection result 600 of the processor.

[0091] Testing station 610 may receive a test result 600 of input data from a processor. For example, testing station 610 may receive a test result 600 that determines that first input data 602 and second input data 604 do not correspond to the first category, while third input data 606 does. Testing station 610 may review the determination based on data sample 612 to determine whether the determination is correct. For example, review result 620 may show that first input data 622 has defects but is consistent with sample 612, and therefore may be determined to correspond to the first category, and second input data 624 is also consistent with sample 612, and therefore may be determined to correspond to the first category. Furthermore, unlike test result 600 in which the processor determines that third data 626 corresponds to the first category, third data 626 may be determined not to correspond to the first category because it is inconsistent with sample 612.

[0092] Figure 7 7 is a block diagram of a model running on a GPU according to an embodiment of the present disclosure. A model for determining whether input data corresponds to a category can be run on a GPU 700. That is, the model can utilize a feature extractor 710 and a classifier 720 running on the GPU 700 to output a determination result 740 indicating whether the input data corresponds to a category. However, the capacity of the GPU 700 is limited, making it difficult to load the classifiers of all models into the GPU 700 for use at once. Therefore, in order to conserve the capacity of the GPU 700, the classifiers of models used more frequently than a predetermined frequency can be loaded from the memory 730 into the cache 722 on the GPU 700. For example, if 100 classifiers are stored in the cache 722 on the GPU 700, the processor can load the 100 classifiers from the memory 730 into the cache 722 in descending order of usage frequency, starting from the most frequently used model among the multiple models.

[0093] According to one embodiment, the processor may remove the least used classifier from the cache 722 based on the classifier's usage frequency. For example, if all 100 classifiers are already loaded into the cache 722 and a classifier not loaded into the cache 722 is needed, the processor may remove the least used classifier from the classifiers loaded into the cache 722 and load the required classifier from the memory 730 into the cache 722.

[0094] Figure 8 FIG1 shows an embodiment of an electronic device determining a reference probability according to an embodiment of the present disclosure. Figure 8In the chart, the horizontal axis refers to the category, and the vertical axis refers to the probability calculated by inputting multiple input data into the model of each category. The vertical line in the chart is the probability value calculated by the model of each category for multiple input data. The bars at the lower and upper ends of the vertical line can be intervals with a larger distribution of probability values ​​(for example, the interval where the data is distributed between the first 25% and 75%). In addition, the bubble marked with each category means the baseline probability of the model of the corresponding category. That is, the baseline probability of the model of the first category 800 can be the value P displayed by the first bubble 806. x , the baseline probability of the model for the second category 810 may be the value 0.9 displayed by the second bubble 816 .

[0095] The processor may determine the baseline probability of each model based on the output values ​​of the multiple input data of each model. The probability values ​​output by the multiple input data input to the model may be accumulated and stored in the memory. The processor may calculate the average value of the minimum probability value of the input data determined to correspond to the first category 800 and the maximum probability value of the input data determined not to correspond to the first category 800 and the predetermined critical probability P. x The higher value in is determined as the base probability. For example, referring to Figure 8 In the data input to the first model, the maximum probability value 802 of the input data determined not to correspond to the first category 800 is 0.1, and the minimum probability value 804 of the input data determined to correspond to the first category 800 is 0.99, so the average of the two values ​​is approximately 0.5. In the data input to the second model, the maximum probability value 812 of the input data determined not to correspond to the second category 810 is 0.86, and the minimum probability value 814 of the input data determined to correspond to the second category 810 is 0.98, so the average of the two values ​​816 is approximately 0.9. Since the critical probability P x The processor may set the baseline probability of the model of the first category 800 to be 0.7 and the baseline probability of the model of the second category 810 to be 0.9. According to one embodiment, the processor may reset the baseline probability each time a model is generated or retrained. According to another embodiment, the processor may reset the baseline probability at predetermined time intervals.

[0096] Figure 9 A sequence diagram of a method for an electronic device to determine whether first data corresponds to a first category according to an embodiment of the present disclosure. In act 900, the electronic device may obtain input data. For example, the input data may be an image of a component configured on a substrate.

[0097] In action 910, the electronic device 100 may confirm whether a model of a first category corresponding to the first data is stored in the memory 110. If the first model is stored, in action 912, the electronic device 100 may use the first model to determine whether the first data corresponds to the first category. If the first model is not stored, in action 914, the electronic device 100 may determine whether the input data corresponds to the first category based on the first data corresponding to the first category.

[0098] In action 920 , the electronic device 100 may transmit information indicating whether the first data corresponds to the first category to the external device 101 . The electronic device 100 may determine whether the first data corresponds to the first category and transmit the detection result to the detection station 610 .

[0099] Figure 10 A sequence diagram of a method for generating or retraining a first category model for an electronic device according to an embodiment of the present disclosure. In act 1000, the electronic device 100 may obtain a training request for input data. Upon obtaining the training request, in act 1010, the electronic device 100 may confirm whether a first category model corresponding to the input data is stored in the memory 110.

[0100] When the first model is stored in the memory 110, in action 1020, the electronic device 100 may confirm whether a predetermined condition is met. According to one embodiment, the electronic device 100 may decide to retrain the current model based on the number of errors in the results of determining whether each data corresponds to the first category using the first model. In action 1022, the electronic device 100 may determine to retrain the first model based on the number of first errors and the number of second errors in the results of determining whether the input data corresponds to the first category using the first model, the first error being that data that does not correspond to the first category is judged to correspond to the first category, and the second error being that data corresponding to the first category is judged to not correspond to the first category. When the predetermined condition is not met, the electronic device 100 may not retrain the first model.

[0101] If the first model is not stored in memory 110, in act 1030, the electronic device 100 may add the input data to a training queue for generating the first model. According to one embodiment, a predetermined number (e.g., 100) or more of data corresponding to the first category is stored. Since data corresponding to the first category does not already exist in the training queue, the electronic device 100 may add the input data to the training queue. In act 1032, the electronic device 100 may generate a model for the first category.

[0102] exist Figure 9 and Figure 10The sequence diagrams shown depict each step of the method or algorithm of the present disclosure in sequence. In addition to being performed sequentially, the steps may also be performed in any order that is compatible with the present disclosure. The description based on this flowchart does not exclude the possibility of applying changes or revisions to the method or algorithm, and does not imply that any step is required or preferred. In one embodiment, at least some of the steps may be performed in parallel, repeatedly, or heuristically. In one embodiment, at least some of the steps may be omitted, or additional steps may be added.

[0103] Various embodiments of the present disclosure may be implemented in software on a machine-readable storage medium. Software may be software used to implement various embodiments of the present disclosure. Software may be derived from various embodiments of the present disclosure by a programmer skilled in the art. For example, software may be a program comprising machine-readable instructions (e.g., code or code segments). A machine, for example, a computer, is a device that operates according to instructions retrieved from a storage medium. In one embodiment, the machine may be the electronic device 100 of the present disclosure. In one embodiment, the processor of the machine may execute the retrieved instructions, and the components of the machine may perform functions corresponding to the corresponding instructions. In one embodiment, the processor may be the processor 120 of the present disclosure. A storage medium may refer to any type of machine-readable recording medium that stores data. Examples of storage media include ROM (read-only memory), RAM (random access memory), CD-ROM (compact disc drive), magnetic tape, floppy disk, optical data storage device, etc. In one embodiment, the storage medium may be memory 110. In one embodiment, the storage medium may also be implemented in a distributed form across computer systems connected via a network. Software can be distributed, stored, and executed in computer systems. The storage medium can be non-transitory. Non-transitory storage media refers to tangible media that exist independently of the semi-permanent or temporary storage of data and does not include signals that are transmitted transiently.

[0104] While the technical concepts of the present disclosure have been described above based on various embodiments, the technical concepts of the present disclosure encompass various permutations, modifications, and variations that are achievable within the scope of understanding by those skilled in the art. Furthermore, it should be understood that such permutations, modifications, and variations are encompassed by the appended claims.

Claims

1. A device for determining a category of data, comprising: More than one processor; and One or more memories storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations; wherein the one or more processors obtain first data about components configured on the substrate, determining whether a model of a first category corresponding to the first data is stored in the one or more memories, When it is determined that the model of the first category is stored, determining whether the first data corresponds to the first category using the model, When it is determined that the model of the first category is not stored, determining whether the first data corresponds to the first category using second data corresponding to the first category, Information indicating whether the first data corresponds to the first category is transmitted to an external device.

2. The device according to claim 1, wherein When the one or more processors use the model to determine whether the first data corresponds to the first category, Calculating the probability of the first data corresponding to the first category using the model, Determine whether the probability is above a predetermined baseline probability, If the probability is greater than a predetermined reference probability, it is determined that the first data corresponds to the first category. If the probability does not reach a predetermined reference probability, it is determined that the first data does not correspond to the first category.

3. The device according to claim 2, wherein The predetermined reference probability is determined by using the average value of the minimum value of the probability of one or more data being determined to correspond to the first category and the maximum value of the probability of one or more data being determined to not correspond to the first category.

4. The device according to claim 1, wherein When determining that the model of the first category is not stored, the one or more processors use a machine learning algorithm to calculate the similarity between the first data and the second data, Based on the similarity between the first data and the second data, it is determined whether the first data corresponds to the first category.

5. The device according to claim 4, wherein The machine learning algorithm includes ArcFace.

6. The device according to claim 4, wherein The machine learning algorithm is trained using a plurality of arbitrarily generated virtual data and the second data.

7. The device according to claim 6, wherein The plurality of virtual data are generated by adjusting the size, font, and thickness of characters and adding blur and defects to the one or more actual data stored in the one or more memories.

8. The device according to claim 6, wherein The plurality of virtual data are generated using at least one algorithm selected from among a generative adversarial network and a steady-state diffusion algorithm.

9. The device according to claim 1, wherein The one or more processors obtain a training request for the first data, When the training request is obtained, determining whether the model of the first category has been stored, When it is determined that the model of the first category is not stored, adding the first data to a training queue for generating or training the model, When it is determined that the model of the first category is stored, if the model of the first category meets a predetermined condition, the first data is added to the training queue.

10. The device according to claim 9, wherein When the one or more processors determine that the model of the first category is not stored, the first data is added to the training queue. If the number of the one or more data corresponding to the first category stored in the one or more memories is greater than a predetermined value and the first data is not in the training queue, the first data is added to the training queue.

11. The device according to claim 9, wherein The predetermined condition is a condition that is satisfied when the model judges data that does not correspond to the first category as corresponding to the first category a number of times greater than a first critical value, or judges data that corresponds to the first category as not corresponding to the first category a number of times greater than a second critical value.

12. The device according to claim 9, The one or more processors train the model using a data set including data corresponding to one or more categories similar to the first category and arbitrarily selected data.

13. The device according to claim 12, wherein The ratio of data corresponding to one or more categories similar to the first category in the data set is less than the ratio of the arbitrarily selected data in the data set.

14. The device according to claim 1, wherein The external device is a display device, When it is determined that the model of the first category and the second data corresponding to the category are not stored, the one or more processors output a statement to the display device indicating that it cannot be determined whether the first data corresponds to the first category.

15. The device according to claim 1, wherein The model includes a feature extractor and a classifier.

16. The device according to claim 15, wherein The feature extractor is commonly applied to the models stored in the one or more memories, and the classifier is trained based on different information for each model stored in the one or more memories.

17. The device according to claim 15, wherein The feature extractor is trained using a plurality of arbitrarily generated virtual data.

18. The device according to claim 15, wherein The feature extractor and classifier run on a graphics processor, The one or more processors load the classifier included in the model of the category used with a frequency greater than a predetermined value from the memory into the cache on the graphics processor, The feature extractor and the classifier loaded into the cache are used to determine whether the data corresponds to a category.

19. The device according to claim 18, wherein The one or more processors select a least used classifier from among the one or more classifiers loaded into the cache and remove it from the cache.

20. The device according to claim 1, wherein When determining that the first data does not correspond to the first category, the one or more processors transmit information indicating that the first data does not correspond to the first category to a detection station.

21. A method for determining a category of data, performed in a device comprising one or more processors and one or more memories storing instructions for execution by the one or more processors, comprising: a step of obtaining first data indicating components arranged on a substrate; a step of determining whether a model of a first category corresponding to the first data is stored in one or more memories; When it is determined that the model of the first category is stored, determining whether the first data corresponds to the first category by using the model; When it is determined that no model of the first category is stored, using second data corresponding to the first category to determine whether the first data corresponds to the first category; and a step of transmitting information indicating whether the first data corresponds to the first category to an external device.

22. A non-transitory computer-readable recording medium for determining a category of data, the medium recording instructions for causing one or more processors to perform operations when executed by the medium, wherein: The instructions cause the one or more processors to execute: a step of obtaining first data indicating components arranged on a substrate; a step of determining whether a model of a first category corresponding to the first data is stored in one or more memories; When it is determined that the model of the first category is stored, determining whether the first data corresponds to the first category by using the model; When it is determined that the model of the first category is not stored, using second data corresponding to the first category to determine whether the first data corresponds to the first category; and a step of transmitting information indicating whether the first data corresponds to the first category to an external device.

Images