Method and system for monitoring judgment records for outlier detection guides

Abstract

A judgment record monitoring method for an outlier detection guide according to an embodiment of the present invention is characterized by detecting a predetermined inspection image and at least one similar image having a similarity of a predetermined standard or higher, providing judgment details which are information that specifies the content of the determination of whether or not there is an outlier, obtaining first judgment result information which is information that specifies the determination result of whether or not there is an outlier for the inspection image, and then generating the inspection image and the first judgment result information according to the similarity between the inspection image and the similar image, respectively.

Description

[Technical Field]

[0001] The present invention relates to a method and system for monitoring judgment records for outlier detection guides. More specifically, it relates to a method and system for providing guidance for real-time outlier detection based on past outlier detection records. [Background technology]

[0002] Vision inspection equipment aims to automatically detect visible defects in products using computer vision technologies, including deep learning.

[0003] Based on data obtained from sensors such as cameras, good products and defective products containing flaws must be classified with very high accuracy, and for product quality control, the equipment in the inspection process must operate uniformly and without problems.

[0004] However, new types of defects that did not occur during the training of the classification model may arise, and due to the influence of noise generated by process environmental conditions, it is not possible to detect defects with 100% accuracy using vision inspection equipment. Therefore, in order to prevent situations where defects are released, some of the manufactured products will ultimately undergo a monitoring process by human workers.

[0005] Workers make visual judgments based on image data captured by the equipment, but in the case of vision inspection data, there is a large amount of data where the boundary between normal products and defective products is ambiguous. In such cases of ambiguous shapes, the accuracy of defect detection is greatly reduced.

[0006] In particular, in the case of inexperienced workers, the likelihood of making incorrect judgments is even higher, and because inspection results can vary depending on the inspector's skill level, this can lead to problems in maintaining consistency in inspection quality.

[0007] While efforts are being made to reduce the probability of errors by new workers by having them learn certain rules, such as those found in guidebooks, there are limitations because it is impossible to learn for every possible case.

[0008] On the other hand, anomaly detection can be defined as the process of identifying abnormal patterns, outliers, and / or exceptional values ​​from given data.

[0009] In other words, anomaly detection can be a process that detects elements that deviate from the attributes of normal data.

[0010] Systems that embody this type of anomaly detection are actively used in various application fields where the identification of abnormal patterns is crucial, such as process monitoring, security intrusion detection, fraud identification, and / or medical diagnosis.

[0011] However, there is a problem in that the ability to realize anomaly detection task processing performance based on this is limited when the data required for training a model for outlier detection is relatively rare, diverse, or insufficient, such as when it is difficult to collect abnormal data containing a predetermined defect, when the data with specified labels is limited, or when attempting to train a model with a large amount of unlabeled data.

[0012] Furthermore, in conventional methods, particularly in the field of vision testing, anomaly detection based on specific images is actively performed. However, such images belong to high-dimensional data, and there is a problem in that the data processing and computational costs are inefficient when all the data for the entire image is used at once to detect outliers.

[0013] In most cases, outliers are observed as abnormal patterns that appear in small parts of an image in various sizes and forms. However, according to the conventional method, there is a problem that the discrimination ability for the above local patterns on the entire image is very low.

[0014] Therefore, it is necessary to develop a new technology that can further improve the accuracy and efficiency of anomaly detection and at the same time improve the accompanying task processing performance even under limited environmental requirements.

Prior Art Documents

Patent Documents

[0015]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0016] One embodiment of the present invention aims to implement a method and system for providing a guide for real-time outlier determination based on past outlier determination records.

[0017] At this time, one embodiment of the present invention can provide a guide for outlier determination by utilizing a deep learning model specialized in anomaly detection based on a patch feature-based learning method.

Means for Solving the Problems

[0018] A method for monitoring a determination record for an outlier determination guide according to an embodiment of the present invention includes the steps of: acquiring a predetermined inspection image; detecting at least one similar image having a similarity of a predetermined standard or higher with the acquired inspection image; providing determination details, which are information that specifies the determination content of whether or not there are outliers for the detected similar image and each of the similar images; acquiring first determination result information, which is information that specifies the determination result of whether or not there are outliers for the inspection image; and storing the inspection image and the first determination result information in a database according to the similarity between the inspection image and each of the similar images.

[0019] In other respects, the step of providing the detailed determination information includes providing at least one of the following pieces of information: worker information, which is information identifying the worker who made the determination regarding the presence or absence of outliers in the similar image; and second determination result information, which is information identifying the result of the determination regarding the presence or absence of outliers in the similar image.

[0020] In other respects, the step of detecting similar images involves at least one of the images pre-stored in the database. Image of the determined test The process includes measuring the raw data similarity between the image and the inspection image.

[0021] In other aspects, the step of detecting similar images involves, based on a predetermined image feature extraction model, a test feature vector which is a feature vector for the test image and the Image of the determined test The feature vector for Tests that have been determined A step to obtain a feature vector, and the obtained inspection feature vector and the Tests that have been determined The method further includes the step of measuring the similarity between feature vectors, which is the similarity between feature vectors.

[0022] In other respects, the image feature extraction model includes an outlier detection model, which is a deep learning model specialized for anomaly detection using a patch feature-based learning method.

[0023] In other respects, the step of detecting similar images is to use the raw data similarity and the feature vector similarity to determine the inspection image and the Image of the determined test The process further includes the step of calculating a decision similarity to determine the final similarity between the two.

[0024] In other respects, the step of detecting similar images further includes the step of detecting similar images based on the determination similarity and a previously set similarity threshold.

[0025] In other respects, the step of detecting similar images further includes determining whether the number of detected similar images meets a previously set number, and if it does not meet the previously set number, detecting additional similar images.

[0026] In other aspects, the step of detecting additional similar images includes the step of adjusting the similarity threshold and the step of detecting the similar images based on the adjusted similarity threshold.

[0027] In other respects, the step of providing the similar images and determination details further includes the step of providing the determination similarity.

[0028] In other respects, the step of providing the similar images and determination details includes the step of detecting additional similar images according to the ratio between the second determination result information for at least one similar image, and the step of further providing additional images which are the additionally detected similar images.

[0029] In other respects, the step of providing the similar images and detailed determination information includes aligning the detected similar images based on at least one of the information of the person in charge and the degree of similarity between the inspection image and the similar images, respectively.

[0030] In other respects, the step of storing in the database further includes the step of storing the inspection image and the first judgment result information in the database, depending on whether the first judgment result information and the second judgment result information are identical.

[0031] In other aspects, the determination record monitoring method for an outlier determination guide according to embodiments of the present invention further includes the step of filtering the detected similar images according to pre-set conditions.

[0032] In other respects, the step of acquiring the inspection image includes acquiring the inspection image based on an outlier detection model, which is a deep learning model specialized in anomaly detection using a patch feature-based learning method.

[0033] On the other hand, a computing device according to an embodiment of the present invention includes at least one Vision Inspection Unit, at least one Inspection Monitoring Processing Unit, at least one Monitoring Interface Unit, and at least one Inspection Database, and controls the Vision Inspection Unit to acquire a predetermined inspection image, controls the Inspection Monitoring Processing Unit to detect at least one similar image having a similarity of a predetermined standard or higher with respect to the acquired inspection image, controls the Monitoring Interface Unit to provide detailed determination information which is information that specifies the determination content of whether or not there are outliers for the detected similar image and each of the similar images, controls the Monitoring Interface Unit to acquire first determination result information which is information that specifies the determination result of whether or not there are outliers for the inspection image, and controls the Inspection Monitoring Processing Unit to store the inspection image and the first determination result information in the inspection database according to the similarity between the inspection image and each of the similar images. [Effects of the Invention]

[0034] A method and system for monitoring judgment records for outlier detection guidance according to one embodiment of the present invention can help improve the consistency and minimize errors of real-time outlier detection by providing guidance for real-time outlier detection based on past outlier detection records, thereby enhancing the overall quality and performance of the vision inspection process.

[0035] In this case, the method and system for monitoring the determination record of an outlier detection guide according to one embodiment of the present invention has the effect of improving the quality of the provided guide and simultaneously improving the efficiency of data processing for providing the guide, by utilizing a deep learning model specialized in anomaly detection using a patch feature-based learning method to provide a guide for outlier detection. [Brief explanation of the drawing]

[0036] [Figure 1] An example block diagram of a computing system embodying an outlier detection guide and monitoring service according to one embodiment of the present invention is shown. [Figure 2] An example block diagram of a computing device embodying an outlier detection guide and monitoring service according to one embodiment of the present invention is shown. [Figure 3] An example of a block diagram of another aspect of a computing device embodying an outlier detection guide and monitoring service according to one embodiment of the present invention is shown. [Figure 4] An example of a block diagram of a computing system in another aspect according to one embodiment of the present invention is shown. [Figure 5] This is an example of a block flow diagram for explaining an outlier detection model according to one embodiment of the present invention. [Figure 6] This is a flowchart illustrating a patch feature learning method for anomaly detection according to one embodiment of the present invention. [Figure 7] This is a flowchart illustrating a patch feature-based feature representation learning method according to one embodiment of the present invention. [Figure 8] This is an example of a figure illustrating an example of measuring the similarity between patch features using one embodiment of the present invention. [Figure 9] This is an example of a diagram illustrating an application example of the ReConPatch Process according to one embodiment of the present invention. [Figure 10] This is a flowchart illustrating a method for monitoring the determination record for an outlier detection guide according to one embodiment of the present invention. [Figure 11] This is a flowchart illustrating a similar image detection method according to one embodiment of the present invention. [Figure 12] This is a conceptual diagram illustrating a similar image detection method according to one embodiment of the present invention. [Figure 13] This is an example of how a similar image is provided by one embodiment of the present invention. [Figure 14] This is an example of how an added-value image is provided by one embodiment of the present invention. [Figure 15] This is an example of a diagram illustrating the alignment method when providing similar images according to one embodiment of the present invention. [Modes for carrying out the invention]

[0037] The present invention can be modified in various ways and has several embodiments; therefore, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention, and how to achieve them, will become clear when you refer to the embodiments described in detail below with the drawings. However, the present invention is not limited to the embodiments disclosed below and can be embodied in various forms. In the following embodiments, terms such as "first," "second," etc., are used not in a restrictive sense but to distinguish one component from another. Also, singular expressions include plural expressions unless they are clearly different in context. Also, terms such as "includes" or "has" mean that the features or components described in the specification exist, and do not preclude the possibility that one or more other features or components may be added. Also, in the drawings, the size of components may be exaggerated or reduced for the sake of illustration. For example, the size and thickness of each component shown in the drawings are arbitrarily shown for the sake of illustration, and the present invention is not necessarily limited to those shown.

[0038] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. When describing with reference to the drawings, identical or corresponding components will be given the same reference numerals, and redundant descriptions thereof will be omitted.

[0039] [An exemplary system providing outlier detection guidance and monitoring services]

[0040] The following describes in detail, with reference to the attached diagrams, an exemplary system that embodies an outlier detection guide and monitoring service that performs patch feature-based training on a predetermined pretrained model based on an image dataset of the target of outlier detection.

[0041] Figure 1 shows an example block diagram of a computing system that embodies an outlier detection guide and monitoring service according to one embodiment of the present invention.

[0042] Referring to Figure 1, the computing system 1000 embodying the outlier detection guide and monitoring service of the present invention includes a user computing device 110, a server computing system 130, and a training computing system 150, the devices being able to communicate via a network 170.

[0043] The patch feature learning method for anomaly detection according to embodiments of the present invention can be 1) implemented and provided locally by a user computing device 110, 2) implemented and provided in the form of a web service by a server computing system 130 that communicates with the user computing device 110, or 3) implemented and provided by the user computing device 110 and the server computing system 130 working together.

[0044] In this embodiment, the user computing device 110 and / or the server computing system 130 can train the machine learning models 120 and / or 140 through interaction with the training computing system 150, which is communicatively connected via the network 170. The training computing system 150 may be separate from the server computing system 130 or may be part of the server computing system 130.

[0045] In this case, the artificial intelligence model (in this embodiment, such as an outlier detection model) can be trained in three ways: 1) directly locally by the user computing device 110; 2) by the server computing system 130 and the user computing device 110 interacting with each other via the network 170; or 3) by another training computing system 150 using a variety of training and learning techniques. Furthermore, the artificial intelligence model trained by the training computing system 150 can be transferred and provided / updated to the user computing device 110 and / or the server computing system 130 via the network 170.

[0046] In some embodiments, the training computing system 150 may be part of the server computing system 130 or part of the user computing device 110.

[0047] The user computing device 110 may include smartphones, mobile phones, digital broadcasting devices, PDAs (personal digital assistants), PMPs (portable multimedia players), desktops, wearable devices, embedded computing devices, and / or tablet PCs, as well as all other types of computing devices.

[0048] Such a user computing device 110 includes at least one processor 111 and memory 112. Here, the processor 111 may consist of at least one or more electrically connected processors from among a central processing unit (CPU), graphics processing unit (GPU), ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers, microcontrollers, microprocessors, and / or other electrical units for performing functions.

[0049] The memory 112 may include one or more non-temporary / temporary computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, and combinations thereof, and may include web storage of a server that performs memory storage functions over the internet. Such memory 112 can store data 113 and instruction words 114 necessary for at least one or more processors 111 to perform functional operations such as training an artificial intelligence model or performing outlier detection through the artificial intelligence model.

[0050] In one embodiment, the user computing device 110 can store at least one or more machine learning models 120. In detail, the machine learning model 120 can be a variety of machine learning models, such as multiple neural networks (e.g., deep neural networks), or different types of machine learning models including nonlinear and / or linear models, and can be composed of combinations thereof.

[0051] In this case, the neural network may include at least one of the following: feed-forward neural networks, recurrent neural networks (e.g., long-term memory recurrent neural networks), convolutional neural networks, and / or other forms of neural networks.

[0052] In one embodiment, the user computing device 110 receives at least one or more machine learning models 120 from the server computing system 130 via the network 170, stores them in the memory 112, and then executes the stored machine learning models 120 using the processor 111 to perform outlier detection and the like.

[0053] In another embodiment, the server computing system 130 includes at least one machine learning model 140 and operates through the machine learning model 140, communicating with the user computing device 110 and related data, thereby providing the user with an outlier detection guide and monitoring service in conjunction with the user computing device 110.

[0054] For example, the user computing device 110 can perform outlier detection guidance and monitoring services in a manner in which the server computing system 130 provides output for user input using a machine learning model 140 via the web.

[0055] Furthermore, the artificial intelligence model can also be realized in a manner in which at least a portion of the machine learning models 120 and / or 140 are executed on the user computing device 110, and the remainder is executed on the server computing system 130.

[0056] Furthermore, the user computing device 110 may include at least one input component 121 that senses user input. For example, the user input component 121 may include a touch sensor (e.g., a touchscreen and / or touchpad) that senses touch from the user's input medium (e.g., a finger or stylus), an image sensor that senses user motion input, a microphone that senses user voice input, buttons, a mouse and / or keyboard, etc. The user input component 121 may also include an interface and an external controller (e.g., a mouse and / or keyboard) if it receives input to an external controller through the interface.

[0057] The server computing system 130 includes at least one processor 131 and memory 132. Here, the processor 131 may consist of at least one or more electrically connected processors from among central processing units (CPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), digital signal processors (DSSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, and / or other electrical units for performing functions.

[0058] The memory 132 may include one or more non-temporary / temporary computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, and magnetic disks, or combinations thereof. Such a memory 132 can store data 133 and instruction words 134 necessary for the processor 131 to perform functional operations, such as training an artificial intelligence model or performing outlier detection through the artificial intelligence model.

[0059] In one embodiment, the server computing system 130 may be embodied by including at least one computing device. For example, the server computing system 130 may include multiple computing devices operating according to a sequential computing architecture, a parallel computing architecture, or a combination thereof. The server computing system 130 may also include multiple computing devices connected by a network 170.

[0060] Furthermore, the server computing system 130 can store at least one or more machine learning models 140. For example, the server computing system 130 may include neural networks and / or other multi-layer nonlinear models as machine learning models 140. Exemplary neural networks may include feedforward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks.

[0061] The training computing system 150 includes at least one processor 151 and memory 152. Here, the processor 151 may consist of at least one or more electrically connected processors from among central processing units (CPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), digital signal processors (DSSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, and / or other electrical units for performing functions.

[0062] The memory 152 may include one or more non-temporary / temporary computer-readable storage media and combinations thereof, such as RAM, ROM, EEPROM, EPROM, flash memory devices, and magnetic disks. Such a memory 152 can store data 153 and instruction words 154 necessary for the processor 151 to perform tasks such as training an artificial intelligence model.

[0063] For example, the training computing system 150 may include a model trainer 160 that trains machine learning models 120 and / or 140 stored in the user computing device 110 and / or server computing system 130 using a variety of training or learning techniques, such as backward propagation of errors (according to the framework shown in Figure 3).

[0064] For example, such a model trainer 160 can perform backpropagation-based updates to one or more parameters of the machine learning model 120 and / or 140 based on a defined loss function. In some concrete examples, error backpropagation may include truncated backpropagation through time. The model trainer 160 can perform a number of generalization techniques (e.g., weight reduction, dropout, knowledge distillation, etc.) to improve the generalization ability of the machine learning models 120 and / or 140 being trained.

[0065] In particular, the model trainer 160 can train machine learning models 120 and / or 140 based on a series of training data 161. Here, the training data 161 can include data of different forms, such as images, audio samples and / or text. Examples of image types that can be used can include video frames, LiDAR point clouds, X-ray images, computed tomography scans, hyperspectral images and / or various other forms of images.

[0066] Such training data 161 may be provided by the user computing device 110 and / or the server computing system 130. When the training computing device trains the machine learning models 120 and / or 140 on specific data from the user computing device 110, the machine learning models 120 and / or 140 may be characterized as personalized models.

[0067] The model trainer 160 also includes computer logic that is utilized to provide the desired functionality.

[0068] Furthermore, the model trainer 160 may be embodied in hardware, firmware, and / or software that control a general-purpose processor. In one embodiment, the model trainer 160 includes a program file stored in a storage device, which is loaded into memory 152 and executed by one or more processors 151. In another embodiment, the model trainer 160 includes one or more sets of computer-executable data 153 and instruction words 154 stored in a tangible computer-readable storage medium such as a RAM hard disk or an optical or magnetic medium.

[0069] Network 170 includes, but is not limited to, 3GPP (3rd Generation Partnership Project) networks, LTE (Long Term Evolution) networks, WiMAX (World Interoperability for Microwave Access) networks, the Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), Bluetooth® networks, satellite broadcasting networks, analog broadcasting networks, and / or DMB (Digital Multimedia Broadcasting) networks.

[0070] In general, communication over network 170 can be conducted using any type of wired and / or wireless connection, via a variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, and / or FTP), encodings or formats (e.g., HTML and / or XML), and / or protection schemes (e.g., VPN, secure HTTP, and / or SSL).

[0071] Figure 2 shows an example block diagram of a computing device that embodies an outlier detection guide and monitoring service according to one embodiment of the present invention.

[0072] As shown in Figure 2, the computing device 100, which is included in the user computing device 110, the server computing system 130, and the training computing system 150, contains a number of applications (e.g., Application 1 to Application N). Each application may include a machine learning library and one or more machine learning models. For example, an application may include an image processing application (e.g., Detection, Classification, and / or Segmentation), a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, and / or a chatbot application.

[0073] In one embodiment, the computing device 100 may include a model trainer 160 for training an artificial intelligence model, and by storing and operating the trained artificial intelligence model, it can provide output data corresponding to predetermined input data (such as an image sequence in one embodiment).

[0074] Each application of the computing device 100 can communicate with a number of other components of the computing device 100, such as at least one sensor, a context manager, a device state component, and / or additional components. In one embodiment, each application can communicate with each device component using an API (e.g., a public API). In one embodiment, the API used by each application may be specific to that application.

[0075] Figure 3 shows an example of a block diagram of another aspect of a computing device 100 that embodies an outlier detection guide and monitoring service according to one embodiment of the present invention.

[0076] Referring to Figure 3, the computing device 300 includes a number of applications (e.g., Application 1 to Application N). Each application can communicate with the central intelligence layer. For example, applications may include image processing applications, text messaging applications, email applications, dictation applications, virtual keyboard applications, and / or browser applications. In one embodiment, each application can communicate with the central intelligence layer (and the models stored therein) using an API (e.g., a common API across all applications).

[0077] The central intelligence layer can include multiple machine learning models. For example, as shown in Figure 3, at least a portion of each machine learning model can be provided to each application and managed by the central intelligence layer. In other embodiments, two or more applications may share a single machine learning model. For example, in some embodiments, the central intelligence layer may provide a single model to all applications. In some embodiments, the central intelligence layer may be contained within the operating system of the computing device 300, or it may be implemented differently.

[0078] The central intelligence layer can communicate with the central device data layer. The central device data layer may be a centralized data storage for the computing device 300. As shown in Figure 3, the central device data layer can communicate with many other components of the computing device 300, such as one or more sensors, a context manager, a device state component, and / or additional components. In some embodiments, the central device data layer can communicate with each device component using an API (e.g., a private API).

[0079] Figure 4 shows an example of a block diagram of computing system 1000 in another aspect according to one embodiment of the present invention.

[0080] Referring to Figure 4, in other aspects, the computing system 1000 according to an embodiment of the present invention may include at least one Vision Inspection Unit (VIU), an Inspection Monitoring Processing Unit (MPU), an Inspection Database (IDU), a Monitoring Interface Unit (MIU), and / or an Inspection Post-processing Unit (PPU).

[0081] In detail, the vision inspection unit (VIU) according to the embodiment can automatically determine the presence or absence of outliers in a given image (e.g., a specific product image) based on a model (hereinafter referred to as a defect detection model) learned through various methodologies such as deep learning and / or machine learning.

[0082] Here, the defect detection model according to the embodiment may include the outlier detection model according to the embodiment of the present invention.

[0083] In other words, in this embodiment, the vision inspection unit (VIU) can perform anomaly detection based on a predetermined image using the outlier detection model described later, and can automatically determine whether or not there are outliers in the image according to the results of the anomaly detection performed.

[0084] In this embodiment, if the confidence level of the outlier detection result does not meet a predetermined standard, the vision inspection unit VIU can provide the corresponding image (i.e., an image in which the confidence level of the outlier detection result is below the standard value) to the inspection monitoring processing unit MPU according to the embodiment of the present invention.

[0085] In one embodiment, the vision inspection unit (VIU) can calculate the error range of the outlier detection result for a predetermined first image.

[0086] Furthermore, if the calculated error range does not meet a preset reference value (for example, within the tolerance range), the vision inspection unit (VIU) can transmit the corresponding first image (in this embodiment, the inspection image) to the inspection monitoring processing unit (MPU).

[0087] Furthermore, the inspection monitoring processing unit MPU according to the embodiment can search and extract from the inspection database IDU according to the embodiment of the present invention at least one image (a similar image, according to the embodiment) that has a similarity of the image received from the vision inspection unit VIU (an inspection image, according to the embodiment) to a preset standard or higher.

[0088] The inspection monitoring processing unit (MPU) can then provide the extracted at least one similar image to the monitoring interface (I / F) unit (MIU) according to an embodiment of the present invention.

[0089] Furthermore, in this embodiment, the inspection monitoring processing unit MPU can receive and acquire various data (such as data indicating the presence or absence of outliers) from the monitoring interface (I / F) unit MIU in response to input from a predetermined first user (such as an operator performing outlier detection in this embodiment).

[0090] The inspection monitoring processing unit (MPU) can store and manage various data acquired from the monitoring interface (I / F) unit (MIU) and / or the vision inspection unit (VIU) in the inspection database (IDU).

[0091] Furthermore, the inspection database IDU according to the embodiment can store and manage various data, information, and / or algorithms necessary for the outlier detection guide and monitoring service according to the embodiment of the present invention.

[0092] Furthermore, the monitoring interface (I / F) unit MIU according to the embodiment can provide various data and / or information related to the outlier detection guide and monitoring service by displaying them as a predetermined graphic image.

[0093] In this embodiment, the monitoring interface (I / F) unit MIU can configure and provide a screen so that a predetermined first user (hereinafter referred to as "operator") can view various data received from the inspection monitoring processing unit MPU (for example, inspection images, similar images, and / or detailed judgment information).

[0094] Furthermore, in this embodiment, the monitoring interface (I / F) unit MIU can provide various data to the inspection monitoring processing unit MPU in response to operator input (for example, input indicating the presence or absence of outliers in the inspection image).

[0095] Furthermore, the post-inspection processing unit PPU according to the embodiment can update predetermined data and / or information stored in the inspection database IDU based on additional data (such as judgment result feedback information in the embodiment) acquired after the outlier determination process (i.e., vision inspection process) according to the embodiment of the present invention has been performed on a predetermined inspection image.

[0096] In Figure 4 of the present invention, the computing system 1000 is described as including the functional components described above, in order to prevent the features of the embodiments of the present invention from being diminished.

[0097] However, it is obvious to a person of ordinary skill in the art that, depending on the embodiment, other general-purpose components may be included in addition to the components shown in Figure 4, or some of the components shown in Figure 4 may be omitted.

[0098] On the other hand, the technologies described herein may refer not only to servers, databases, software applications, and other computer-based systems, but also to actions performed and information transmitted to or from such systems. The inherent flexibility of computer-based systems will be recognized as allowing a wide range of possible configurations, combinations, and divisions of work, as well as functionality between and from components. For example, the processes described herein can be implemented using a single device or component, or multiple devices or components operating in combination. Databases and applications can be implemented by a single system or by systems distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0099] [Outlier Detection Model (ODM)]

[0100] Figure 5 is an example of a block flow diagram illustrating an outlier detection model (ODM) according to one embodiment of the present invention.

[0101] Referring to Figure 5, an outlier detection model (ODM) according to one embodiment of the present invention can be said to be an image deep learning model that performs anomaly detection based on a predetermined input image and classifies and / or recognizes the image based on this.

[0102] For reference, anomaly detection can be defined as the process of identifying abnormal patterns, outliers, and / or exceptional values ​​from specific data.

[0103] In other words, anomaly detection can be a process that detects elements that deviate from the attributes of normal data.

[0104] As an embodiment, anomaly detection can be implemented based on a method that groups predetermined data into clusters and considers points separated from the clusters as outliers.

[0105] Therefore, in this embodiment, the outlier detection model (ODM) can determine whether a given input image contains specific abnormal attributes and classify and / or recognize the image according to the determination result.

[0106] In more detail, in the embodiment, the outlier detection model (ODM) may include a first network (TN: Teacher Network) and a second network (SN: Student Network).

[0107] More specifically, the first network TN in the embodiment can mean a neural network that calculates similarity between predetermined features (patch features in the embodiment).

[0108] In one embodiment, such a first network TN may include a first feature representation layer (f(·)) and a first spatial projection layer (g(·)).

[0109] Here, the feature representation layer according to the embodiment can mean a layer that reconfigures (adjusts) a feature in a way that improves the performance of the feature representation corresponding to a predetermined feature (a patch feature in the embodiment).

[0110] To this end, the feature representation layer can be trained to extract meaningful features from a given feature (in an embodiment, a patch feature) with greater accuracy and to reconstruct (adjust) the relevant feature based on this.

[0111] Furthermore, the spatial projection layer according to the embodiment can mean a layer that projects a feature representation corresponding to a predetermined feature (a patch feature in the embodiment) into a predetermined feature representation space.

[0112] In an embodiment, such a spatial projection layer can be trained to project feature representations corresponding to predetermined features (in an embodiment, patch features) into a feature representation space in which the goals of model learning can be applied more effectively.

[0113] On the other hand, the second network SN in the embodiment can be said to be a neural network that embodies Feature Representation Learning.

[0114] For reference, feature representation learning can be defined as the process by which a deep learning model automatically detects and learns useful features from given data.

[0115] Through feature representation learning, deep learning models can effectively encode useful information contained in data, generate meaningful features that can be used in a variety of deep learning tasks, understand the complex structure and patterns of the data, and make more accurate predictions based on this understanding.

[0116] In one embodiment, the second network SN that embodies the feature representation learning described above may include a second feature representation layer (f(·)) and a second spatial projection layer (g(·)).

[0117] In this case, the first feature representation layer (f(·)) and the second feature representation layer (f(·)) and the first spatial projection layer (g(·)) and the second spatial projection layer (g(·)) according to the embodiment are intended to distinguish between the feature representation layer and spatial projection layer included in the first network TN and the feature representation layer and spatial projection layer included in the second network SN. Therefore, the explanation of the feature representation layer and spatial projection layer of the second network SN follows the explanation of the feature representation layer and spatial projection layer of the first network TN described above.

[0118] Furthermore, a more detailed explanation of the first network TN and the second network SN according to the embodiment will be provided later in the section on patch feature learning methods for anomaly detection.

[0119] On the other hand, in the embodiment, the outlier detection model (ODM) can work in conjunction with a model pre-trained to perform concept learning (hereinafter referred to as the pre-trained model) to perform various functional operations necessary for outlier detection guidance and monitoring services and / or patch feature training services for anomaly detection.

[0120] Here, the pre-trained model according to the embodiment may be an Image Deep-learning Model that has been pre-trained to perform concept learning based on a predetermined training dataset (e.g., a predetermined natural image dataset and / or a predetermined normal image dataset).

[0121] For reference, concept learning can be defined as the process of inferring general rules, concepts, or patterns from given data and classifying them. In other words, in this embodiment, the pre-trained model may be an image deep learning model that takes a predetermined image as input data, learns common features of objects and patterns in the input image, groups image components with similar features based on this, and then helps to classify or recognize a specific image based on this.

[0122] Specifically, a pre-trained model can 1) extract feature maps for a given input image.

[0123] In detail, a pre-trained model can automatically extract feature maps based on the original pixel data of an input image using a given image deep learning neural network (e.g., a Convolutional Neural Network (CNN)).

[0124] Such feature maps can show various visual attributes of the image, such as edges, color, and / or texture.

[0125] Furthermore, pre-trained models can perform 2) clustering based on feature space.

[0126] In detail, the pre-trained model can classify the input image and / or objects within the input image into groups with similar features, based on the feature maps extracted as described above.

[0127] In one embodiment, a pre-trained model can classify extracted feature maps according to the feature space using a predetermined clustering algorithm (e.g., K-means, DBSCAN, and / or hierarchical clustering algorithms) or dimensionality reduction algorithm (e.g., t-SNE and / or UMAP).

[0128] Furthermore, the pre-trained model can 3) assign a label to each cluster.

[0129] In other words, a pre-trained model can define a representative concept (hypothesis) for each cluster and set it as a label for that cluster.

[0130] In this case, the pre-trained model can manually or semi-automatically assign labels to each cluster and learn the features of images belonging to specific concepts or categories.

[0131] Furthermore, the pre-trained model can 4) test and adjust the assigned hypothesis.

[0132] In more detail, the pre-trained model can validate the initially defined concepts (clustered feature groups) as described above and adjust the relevant hypotheses if necessary.

[0133] At this point, the pre-trained model can use the new image data to perform the process of detecting and correcting mis-clustered data.

[0134] Furthermore, the pre-trained model can repeat the process described above (5).

[0135] In other words, a pre-trained model can learn to classify or recognize images more accurately by continuously improving its clustered features and associated concepts with new image data and additional feedback.

[0136] In this embodiment, the pre-trained model may be directly included in the outlier detection model (ODM) or may be embodied as a separate device and / or server from the outlier detection model (ODM).

[0137] In the following explanation, we will assume that the pre-trained model is included in and implemented within the outlier detection model (ODM), but we are not limited to this.

[0138] Furthermore, in Figure 5, in order to prevent the features of the embodiment of the present invention from being diminished, the outlier detection model (ODM) is described as including the components described above. However, it is obvious to any ordinary person in the art that, depending on the embodiment, other general-purpose components may be included in addition to those shown in Figure 5, or some of the components shown in Figure 5 may be omitted.

[0139] [Patch Feature Learning Method for Anomaly Detection]

[0140] The following describes in detail how a computing system 1000 according to an embodiment of the present invention implements a patch feature training service for anomaly detection, which performs patch feature-based training on a predetermined pretrained model based on an image dataset of the target for outlier detection.

[0141] The patch feature learning method for anomaly detection of a computing system 1000 according to an embodiment of the present invention can improve the performance and quality of various anomaly detection-based services by utilizing the trained outlier detection model (ODM) according to the embodiment of the present invention.

[0142] In this case, the patch feature learning method for anomaly detection of the computing system 1000 according to an embodiment of the present invention can effectively provide an outlier detection model (ODM) with improved performance by performing patch feature-based learning that reduces the variance of similar patch features and increases the difference between dissimilar patch features.

[0143] The patch feature learning method for anomaly detection according to embodiments of the present invention will be described in more detail below with reference to the attached drawings.

[0144] Figure 6 is a flowchart illustrating a patch feature learning method for anomaly detection according to one embodiment of the present invention.

[0145] Referring to Figures 5 and 6, an embodiment of the present invention of a patch feature learning method for anomaly detection may include the steps of: acquiring a feature map based on a pretrained model (S101); extracting a plurality of patch features based on the acquired feature map (S103); performing feature representation learning based on the extracted plurality of patch features (S105); acquiring a reconpatch feature corresponding to the performed feature representation learning (S107); performing coreset sampling based on the acquired reconpatch features (S109); acquiring a test sample image (S111); acquiring a reconpatch feature corresponding to the acquired test sample image (S113); and performing outlier detection based on the acquired reconpatch features (S115).

[0146] Specifically, the computing system 1000 according to an embodiment of the present invention can acquire a feature map based on a pretrained model (S101).

[0147] In detail, in the embodiment, the computing system 1000 can acquire feature maps corresponding to an image dataset for a predetermined outlier detection target (hereinafter referred to as the target image dataset) through a model that has been pre-trained to perform concept learning (i.e., a pre-trained model).

[0148] To reiterate, the pre-trained model according to this embodiment may be an Image Deep-learning Model that has been pre-trained to perform concept learning based on a predetermined training dataset (e.g., a predetermined natural image dataset and / or a predetermined normal image dataset).

[0149] In other words, in the embodiment, the computing system 1000 can work in conjunction with the pre-trained model described above to acquire a feature map based on a predetermined target image dataset (i.e., an image dataset containing multiple images of a predetermined target for outlier detection).

[0150] In more detail, in the embodiment, the computing system 1000 can input a target image dataset (for example, an image dataset containing multiple images of a given electronic circuit element) into a pre-trained model.

[0151] As a result, the pre-trained model can output a feature map corresponding to the input target image dataset and provide it to the computing system 1000.

[0152] This allows the computing system 1000 to acquire a feature map corresponding to the target image dataset.

[0153] In addition, in this embodiment, the computing system 1000 can extract multiple patch features based on the acquired feature map (S103).

[0154] Here, a patch feature according to this embodiment can mean a feature extracted from a patch that represents a small portion of a given image.

[0155] Specifically, the patch can be a rectangular area that represents a particular part of a given image. Such a patch can be considered a subset of the information of the overall image, and may mainly include local information and texture information.

[0156] Furthermore, the features are characteristic information extracted from a predetermined image or patch, and can summarize or represent important attributes in the image (e.g., pattern, texture, color, and / or shape).

[0157] Therefore, the patch feature can be data that indicates local attributes within a given image on a patch-by-patch basis.

[0158] In detail, in the embodiment, the computing system 1000 can extract multiple patch features based on the feature map obtained as described above.

[0159] More specifically, in one embodiment, the computing system 1000 can divide a predetermined image (hereinafter referred to as the target training image) contained in the target image dataset into predetermined patch size units before inputting it into the pre-trained model described above.

[0160] The computing system 1000 can then input each divided patch into a pre-trained model to obtain a corresponding feature map for each patch.

[0161] In other words, the computing system 1000 can divide the target training image into predetermined patch size units, input them into a pre-trained model, and obtain a feature map for each patch.

[0162] In this embodiment, the computing system 1000 can perform coreset sampling on the acquired patch-specific feature maps.

[0163] For reference, core set sampling can be defined as a method for efficiently processing large datasets, which involves extracting a representative set of samples that preserves the statistical characteristics and structure of the original dataset to the greatest extent possible while reducing the size of the dataset.

[0164] As an embodiment, the computing system 1000 can perform core set sampling using an approximation algorithm that, while considering the distribution of the given data, maintains the characteristics of the original dataset within a predetermined error range and simultaneously selects a portion of the samples that can represent the entire dataset.

[0165] Alternatively, the computing system 1000 can perform core set sampling using an importance sampling method that assigns sampling probabilities based on the importance of each given data point and preferentially selects data points with high importance.

[0166] This allows the computing system 1000 to acquire multiple patch features on which core set sampling has been performed.

[0167] On the other hand, in another embodiment, the computing system 1000 can acquire a feature map (hereinafter referred to as the overall feature map) for the entire region of the target training image.

[0168] The computing system 1000 can then divide the acquired overall feature map into predetermined patch size units.

[0169] This allows the computing system 1000 to extract multiple patch features from the target training image.

[0170] Thus, in this embodiment, the computing system 1000 can extract multiple patch features corresponding to the target training image using at least one of the methods described above.

[0171] In this embodiment, the computing system 1000 can aggregate surrounding feature vectors within a specific patch size and extract each patch feature.

[0172] Alternatively, depending on the embodiment, the computing system 1000 may use the pixel values ​​themselves within each patch as features.

[0173] Alternatively, depending on the embodiment, the computing system 1000 may use a statistical summary (e.g., mean, variance, and / or histogram, etc.) for the pixel values ​​within each patch as a feature.

[0174] Alternatively, depending on the embodiment, the computing system 1000 may analyze the texture patterns within each patch and use them as features. For example, the computing system 1000 may extract texture-based patch features using techniques such as Gabor filters, LBP (Local Binary Patterns), and / or HOG (Histogram of Oriented Gradients).

[0175] Alternatively, depending on the embodiment, the computing system 1000 may use a deep learning algorithm such as a convolutional neural network (CNN) to automatically learn and extract high-dimensional features within each patch for use as features.

[0176] Thus, in this embodiment, the computing system 1000 can extract features at the patch level and use them to support outlier detection, thereby improving processing efficiency in the data learning and analysis process, while also being able to more accurately detect abnormal local patterns that mainly appear in small parts of the image.

[0177] In addition, in this embodiment, the computing system 1000 can perform feature representation learning based on the extracted patch features (S105).

[0178] To reiterate, feature representation learning can be defined as the process by which a deep learning model (in this embodiment, an outlier detection model (ODM)) automatically detects and learns useful features from given data.

[0179] In more detail, in the embodiment, the computing system 1000 can learn feature representations for the multiple patch features extracted as described above based on the outlier detection model (ODM) according to the embodiment of the present invention.

[0180] In other words, the computing system 1000 can perform feature representation learning for an outlier detection model (ODM) that performs anomaly detection based on the extracted patch features.

[0181] To reiterate, anomaly detection can be defined as the process of identifying abnormal patterns, outliers, and / or exceptional values ​​from specific data; in other words, the process of detecting components that deviate from the attributes of normal data.

[0182] Therefore, in this embodiment, the computing system 1000 can determine whether a given input image contains specific abnormal attributes and perform feature representation learning (such as concept learning in this embodiment) to classify and / or recognize the image based on the determination result, using the aforementioned multiple patch features.

[0183] In this embodiment, the computing system 1000 can perform the feature representation learning described above based on a semi-supervised learning method.

[0184] In other words, computing system 1000 can build an outlier detection model (ODM) that embodies semi-supervised learning-based anomaly detection.

[0185] For reference, semi-supervised learning can be defined as a deep learning method that trains a model using not only data with labels (i.e., training data) but also data without labels (i.e., untrained data).

[0186] Generally, when collecting base data to build an outlier detection system, it is difficult to obtain a sufficient amount of abnormal data for smooth learning (for example, image data of abnormal states targeted for outlier detection). This can limit anomaly detection learning, which attempts to recognize abnormal states (outliers) of various shapes with high accuracy.

[0187] Therefore, in embodiments of the present invention, a pre-trained model (i.e., a pre-trained model) is constructed mainly using normal data (for embodiment, image data of the normal state of the target of outlier detection), and a semi-supervised learning-based anomaly detection is realized that performs outlier detection based on pseudo-labels using the said pre-trained model.

[0188] For reference, pseudo-labels can be defined as labels predicted by a trained model for unlabeled data.

[0189] Such pseudo-labels can be used primarily when the labeled data is restrictive, or when a large amount of unlabeled data is used to train a model.

[0190] This allows the computing system 1000 to easily achieve model training and performance improvement for building an outlier detection process, even when the labeled data is relatively rare or diverse and restrictive.

[0191] More specifically, in the embodiment, the computing system 1000 can perform feature representation learning corresponding to multiple patch features based on the first network TN and the second network SN of the outlier detection model (ODM).

[0192] Figure 7 is a flowchart illustrating a patch feature-based feature representation learning method according to one embodiment of the present invention.

[0193] Specifically, referring to Figure 7, in one embodiment, the computing system 1000 can project any patch feature pair into a predetermined feature representation space (S201).

[0194] In detail, in an embodiment, the computing system 1000 processes any first patch feature (P i ) and the second patch feature (P j A patch feature pair consisting of a pair of ) (hereinafter referred to as the first patch feature pair) can be projected onto the feature representation space.

[0195] As an embodiment, the computing system 1000 represents the first patch feature (Pi) projected into the feature representation space as shown in [Equation 1(a)] below, and the second patch feature (P j ) can be expressed as shown in [Number 1 (b)] below.

[0196]

number

[0197] In addition, in this embodiment, the computing system 1000 can calculate pairwise similarity based on patch feature pairs projected onto the feature representation space (S203).

[0198] Here, pairwise similarity according to the embodiment is defined as the first patch feature (P) that any patch feature pair contains. i ) and the second patch feature (P j This can mean data that measures the similarity between )

[0199] In other words, in the embodiment, the computing system 1000 includes the first patch feature (P i ) and the second patch feature (P j It is possible to measure pairwise similarity, which indicates the similarity between ) and .

[0200] In this embodiment, the computing system 1000 can calculate the above-mentioned pairwise similarity according to the following [Equation 2].

[0201]

number

[0202] Figure 8 is an example of a diagram showing an example of similarity measurement between patch features using one embodiment of the present invention.

[0203] However, referring to FIG. 8, when measuring similarity only in the relationship between the first patch feature (P i ) and the second patch feature (P j ) included in any patch feature pair, the pairwise similarity is the same. However, when the first feature and the second feature must be classified into different labels from each other as shown in (a) of FIG. 8 (that is, when approaching the correct label only when being further separated from each other) and when the first feature and the second feature must be classified into the same label from each other as shown in (b) of FIG. 8 (that is, when approaching the correct label only when being closer to each other), the discrimination accuracy can decrease.

[0204] In other words, when only measuring the pairwise similarity, for the K nearest neighbors (N i (i)) of the first patch feature (P k ) and the K nearest neighbors (N j ) of the second patch feature (P k )(j)) in the group relationship, without considering the mutual similarity, label prediction is performed, so the accuracy can decrease.

[0205] Therefore, in an embodiment, the computing system 1000 can calculate the contextual similarity based on the patch feature pairs projected into the feature representation space (S205).

[0206] Here, the contextual similarity according to the embodiment can mean data obtained by measuring the bidirectional similarity between the K nearest neighbors (N i )(i)) of the first patch feature (P k ) and the K nearest neighbors (N j ) of the second patch feature (P k )(j)) included in any patch feature pair.

[0207] At this time, in the embodiment, the bidirectional similarity is the first patch feature (Pi K nearest neighbors (N) k (i) Feature and second patch feature (P j K nearest neighbors (N) k (j) It can be calculated based on the average similarity between the feature and the other feature.

[0208] In detail, in the embodiment, the computing system 1000 can calculate the above-mentioned context similarity according to the K-Nearest Neighbors (K-NN) algorithm, which makes predictions based on the distance between data points as shown in [Equation 3] below, and [Equation 4].

[0209]

number

[0210]

number

[0211] In other words, in the embodiment, the computing system 1000 includes the first patch feature (P i ) and the second patch feature (P j ) The more neighbors they share in common, the higher the contextual similarity can be calculated.

[0212] Thus, the computing system 1000 has a first patch feature (P i ) and the second patch feature (P j ) can be learned to recognize feature representations in group relationships and incorporate them into the pseudo-label prediction process.

[0213] In other words, the computing system 1000 is the first patch feature (P i ) and the second patch feature (P jBy extracting the K closest feature samples from the set, calculating context similarity to measure how many samples intersect, and using this along with pairwise similarity to train an outlier detection model (ODM), the trained outlier detection model (ODM) can be made to extract features of better quality.

[0214] This allows the computing system 1000 to more accurately determine whether the pairwise similarity between the first and second features is the same, but the first and second features must be classified into different labels (i.e., they need to be further apart to approach the correct label), or whether the first and second features must be classified into the same label (i.e., they need to be closer to each other to approach the correct label), and to reflect this in pseudo-label prediction. Therefore, the computing system 1000 can directly improve the task processing quality and performance of outlier detection models (ODMs) that perform semi-supervised learning-based anomaly detection.

[0215] Returning to the previous embodiment, the computing system 1000 can calculate Integrated Similarity based on the pairwise similarity and contextual similarity calculated as described above (S207).

[0216] Here, the integrated similarity according to the embodiment is the first patch feature (P) that any patch feature pair contains. i ) and the second patch feature (P j This can be interpreted as data that combines pairwise similarity and contextual similarity between two points.

[0217] In detail, in the embodiment, the computing system 1000 determines the first patch feature (P) that the first patch feature pair contains according to [Equation 5] as shown below. i ) and the second patch feature (Pj The pairwise similarity and contextual similarity between ) can be linearly combined.

[0218]

number

[0219] In this case, the integrated similarity according to the embodiment can be defined as a linear combination of two similarities satisfying 'α∈[0,1]'.

[0220] In addition, in this embodiment, the computing system 1000 can train the second network SN of the outlier detection model (ODM) based on the calculated integrated similarity (S209).

[0221] In other words, in this embodiment, the computing system 1000 can use integrated similarity to train a second network SN that embodies feature representation learning.

[0222] In detail, in the embodiment, the computing system 1000 uses the relaxed contrast loss (L) to calculate the combined similarity for the first patch feature pair. RC This can be applied to train the second network SN.

[0223] Here, the relaxation contrast loss (L) according to the embodiment RC ) is as shown in [Equation 6] below.

[0224]

number

[0225] Here, in [Equation 6], 'z' is the embedding vector inferred by 'g(f(p))', 'N' is the number of mini-batches, i.e., patch instances, 'm' is the repelling margin, and 'ω' ij' can be a parameter that determines the weights of the induction and rebound loss terms.

[0226] Figure 9 is an example of a diagram illustrating an application example of the ReConPatch Process according to one embodiment of the present invention.

[0227] Referring to Figure 9, in this embodiment, the computing system 1000 uses integrated similarity to determine the first patch feature (P i ) and the second patch feature (P j If it is determined that the patch feature pair (hereinafter referred to as the positive feature pair) must be classified into different labels from each other, the first patch feature (P) contained in the positive feature pair i ) and the second patch feature (P j The second feature representation layer (f(·)), the embedding function of the second network SN, can be trained to map the features (f(·)) to each other while separating them in the feature representation space.

[0228] On the other hand, in one embodiment, the computing system 1000 uses integrated similarity to determine the first patch feature (P i ) and the second patch feature (P j If it is determined that a patch feature pair (hereinafter referred to as a negative feature pair) must be classified under the same label, the first patch feature (P) contained in the negative feature pair i ) and the second patch feature (P j The second feature representation layer (f(·)), the embedding function of the second network SN, can be trained to map the features (f(·)) to each other in the feature representation space while keeping them close together.

[0229] Thus, in this embodiment, the computing system 1000 can train the second feature representation layer (f(·)) to reduce the variance of similar patch features and increase the differences between heterogeneous patch features.

[0230] In other words, the computing system 1000 can be trained so that the second feature representation layer (f(·)) extracts arbitrary patch features in a way that is closer to the correct pseudo-label.

[0231] Thus, in this embodiment, the computing system 1000 can directly improve the feature representation performance of the outlier detection model (ODM) by training the second feature representation layer (f(·)) to more accurately extract meaningful features from arbitrary patch features, and can also improve the processing quality of various tasks based on this (such as anomaly detection in this embodiment).

[0232] Furthermore, in this embodiment, the computing system 1000 that has trained the second network SN based on integrated similarity can also train the first network TN of the outlier detection model (ODM).

[0233] In detail, in the embodiment, the computing system 1000 calculates the parameters (θ) of the second network SN according to the following [Equation 7] using the exponential moving average (EMA) method. f,g The data corresponding to the parameters (θ) of the first network TN is used. f￣,g￣ The data can be gradually distilled according to the specified criteria.

[0234]

number

[0235] In other words, in this embodiment, the computing system 1000 can train the first network TN of the outlier detection model (ODM) by gradually distilling the information learned in the second network SN into the first network TN according to [Equation 7] described above.

[0236] In this embodiment, the computing system 1000 can further apply an update rate adjustment variable, which is a variable that adjusts the rate of information distillation, to train the first network TN described above.

[0237] As a result, in this embodiment, the computing system 1000 can implement multiple patch feature-based feature representation learning based on the first network TN and the second network SN of the outlier detection model (ODM).

[0238] Thus, in this embodiment, the computing system 1000 can perform a process (in this embodiment, a ReConPatch Process) to construct discriminant features for outlier detection by distilling the main features of the dataset for the target of outlier detection into a pre-trained model based on semi-supervised learning as described above.

[0239] This allows the computing system 1000 to build a high-performance outlier detection model (ODM) that is trained to more accurately classify the relevant features at a given patch level into ground truth pseudo-labels.

[0240] Therefore, the computing system 1000 can directly and effectively improve the processing performance and quality of various tasks (such as anomaly detection in the embodiment) that utilize the outlier detection model (ODM).

[0241] In addition, in this embodiment, the computing system 1000 can perform coreset sampling based on the acquired recompatch features (S109).

[0242] To reiterate, core set sampling according to this embodiment is a method for efficiently processing large datasets, and can be described as a process of extracting a representative set of samples that preserves the statistical characteristics and structure of the original dataset to the greatest extent possible while reducing the size of the dataset.

[0243] In detail, as an embodiment, the computing system 1000 can perform core set sampling using an approximation algorithm that takes into account the distribution of the acquired recompatched training dataset and maintains the characteristics of the original recompatched training dataset within a predetermined error range, while simultaneously selecting a portion of the samples that can represent the entire recompatched training dataset.

[0244] In another embodiment, the computing system 1000 can perform core set sampling using an importance sampling method that assigns sampling probabilities based on the importance of each of the acquired recompatch feature data and preferentially selects data points with high importance.

[0245] This allows the computing system 1000 to obtain a recompatched training dataset (hereinafter referred to as the recompatched sampling dataset) that has undergone core set sampling.

[0246] In addition, in this embodiment, the computing system 1000 can store and manage the acquired recompatch sampling dataset on a predetermined database.

[0247] This allows the computing system 1000 to detect outliers with high accuracy while reducing data processing costs.

[0248] In addition, in this embodiment, the computing system 1000 can acquire a test sample image (S111).

[0249] Here, the test sample image according to the embodiment can mean an image for detecting the presence or absence of outliers, that is, image data obtained by taking a picture of the object of outlier detection.

[0250] Specifically, in the embodiment, the computing system 1000 can obtain the test sample image as described above based on a predetermined user input and / or interaction with an external server.

[0251] Also, in the embodiment, the computing system 1000 can obtain a recon patch feature corresponding to the obtained test sample image (S113).

[0252] In the following, the content overlapping with the above description may be summarized or omitted.

[0253] Specifically, in the embodiment, the computing system 1000 can input the obtained test sample image into a pre-trained model.

[0254] Also, in the embodiment, the computing system 1000 can obtain a feature map for the test sample image from the pre-trained model that has received the test sample image. For a detailed description of this, the description of step S101 above is applied by reference.

[0255] Also, in the embodiment, the computing system 1000 can extract a plurality of patch features based on the obtained feature map. For a detailed description of this, the description of step S103 above is applied by reference.

[0256] Also, in the embodiment, the computing system 1000 can input the extracted plurality of patch features into the above-mentioned recon patch layer.

[0257] Then, the recompatch layer can output a plurality of recompatch features corresponding to the plurality of input patch features.

[0258] Thereby, in the embodiment, the computing system 1000 can obtain a recompatch feature dataset (hereinafter, the recompatch target dataset) in a form that reduces the dispersion of similar patch features and increases the difference between heterogeneous patch features for the plurality of input patch features.

[0259] Also, in the embodiment, the computing system 1000 can perform outlier detection based on the obtained recompatch features (S115).

[0260] That is, in the embodiment, the computing system 1000 can perform outlier detection for the test sample image based on the recompatch sampling dataset obtained based on each training image used for learning and the recompatch target dataset obtained based on the test sample image.

[0261] In other words, in the embodiment, the computing system 1000 can perform anomaly detection for the test sample image based on the recompatch sampling dataset and the recompatch target dataset.

[0262] Specifically, in the embodiment, the computing system 1000 can calculate the similarity (hereinafter, the outlier detection similarity) between at least a part of the recompatch sampling dataset stored in the database and the recompatch target dataset.

[0263] In addition, in this embodiment, the computing system 1000 can generate an anomaly score map based on the calculated outlier detection similarity.

[0264] For reference, an anomaly score map can be defined as an index that shows how far a state deviates from a predetermined normal state, based on a value (score) assigned by the model.

[0265] Furthermore, in this embodiment, the computing system 1000 can determine that the higher the score according to the generated anomaly score map, the closer the test sample image is to an abnormal state, and the lower the score according to the anomaly score map, the closer the test sample image is to a normal state.

[0266] This allows the computing system 1000 to perform outlier detection on a test sample image in the embodiment.

[0267] As described above, in embodiments of the present invention, the computing system 1000 can perform a process (in embodiments, a ReConPatch Process) to construct discriminant features for outlier detection by distilling the main features of the dataset for the target of outlier detection into a pre-trained model based on semi-supervised learning as described above, and can perform anomaly detection using the outlier detection model (ODM) thus learned.

[0268] As a result, in the embodiment, the computing system 1000 can directly and significantly improve the outlier detection performance and quality based on the outlier detection model (ODM) according to the embodiment of the present invention.

[0269] As described above, the patch feature learning method and system for anomaly detection according to one embodiment of the present invention has the effect of providing an outlier detection model (ODM) that performs patch feature-based learning on a predetermined pretrained model based on an image dataset of the target of outlier detection, thereby enabling more efficient data processing and further improving the task processing performance and quality for anomaly detection.

[0270] Furthermore, the patch feature learning method and system for anomaly detection according to one embodiment of the present invention has the effect of improving the accuracy and efficiency of anomaly detection even under restrictive learning environments, while simultaneously enhancing task processing performance, by performing patch feature-based learning that reduces the variance of similar patch features and increases the differences between dissimilar patch features.

[0271] [Monitoring method for outlier detection guide]

[0272] The following describes in detail how a computing system 1000 according to an embodiment of the present invention embodies an outlier detection guide and monitoring service that provides guidance for real-time outlier detection based on past outlier detection records.

[0273] The judgment record monitoring method for an outlier detection guide of a computing system 1000 according to an embodiment of the present invention can detect at least one image (in an embodiment, a similar image) similar to the image (in an embodiment, an inspection image) for which the presence or absence of an outlier is to be determined from past outlier detection records, and provide the detected similar image and various data related to the determination of the presence or absence of an outlier previously determined therein as a predetermined graphic image on a display.

[0274] At this time, the determination record monitoring method for the outlier determination guide of the computing system 1000 according to the embodiment of the present invention can utilize the learned outlier detection model (ODM) according to the embodiment of the present invention, thereby improving the data processing efficiency for the detection of the above-mentioned inspection image and / or similar images, and at the same time, enhancing its accuracy and reliability.

[0275] Hereinafter, the determination record monitoring method for the outlier determination guide according to the embodiment of the present invention will be described in more detail with reference to the accompanying drawings.

[0276] FIG. 10 is a flowchart for explaining the determination record monitoring method for the outlier determination guide according to an embodiment of the present invention.

[0277] Referring to FIG. 10, the determination record monitoring method for the outlier determination guide according to an embodiment of the present invention may include a step of acquiring a predetermined inspection image (S301), a step of detecting a similar image for the acquired inspection image (S303), a step of filtering the detected similar images (S305), a step of providing the filtered similar images (S307), a step of acquiring determination result information for the inspection image (S309), and a step of storing and managing the acquired determination result information (S311).

[0278] Specifically, the computing system 1000 according to the embodiment of the present invention can acquire a predetermined inspection image (S301).

[0279] Here, the inspection image according to the embodiment of the present invention can mean a target image for which the presence or absence of an outlier is to be determined manually.

[0280] That is, the inspection image can be an image for which a determination regarding the presence or absence of an outlier is required by the manual operation of an operator.

[0281] In detail, in the embodiment, the computing system 1000 can acquire the above-mentioned inspection image in conjunction with a vision inspection unit VIU or the like.

[0282] More specifically, the vision inspection unit (VIU) according to the embodiment can automatically determine the presence or absence of outliers in a given image (e.g., a specific product image) using the defect detection model described above.

[0283] Here, the defect detection model according to the embodiment may include the outlier detection model (ODM) described above.

[0284] In other words, in this embodiment, the vision inspection unit (VIU) can use the aforementioned outlier detection model (ODM) to perform anomaly detection based on a predetermined image, and can automatically determine whether or not there are outliers in the image according to the results of the anomaly detection performed.

[0285] In this embodiment, if the confidence level of the outlier detection result does not meet a predetermined standard, the vision inspection unit VIU can provide the corresponding image (i.e., an image in which the confidence level of the outlier detection result is below the standard value) as a separate output.

[0286] In one embodiment, the vision inspection unit (VIU) can calculate the error range of the outlier detection result for a predetermined first image.

[0287] Furthermore, if the calculated error range does not meet a preset standard value (for example, within the acceptable error range), the vision inspection unit (VIU) can output the corresponding first image (i.e., inspection image) separately and provide it to the computing system 1000.

[0288] In this embodiment, the computing system 1000 can acquire an inspection image to provide to an operator for manual determination of the presence or absence of outliers.

[0289] In addition, in this embodiment, the computing system 1000 can detect similar images to the acquired inspection image (S303).

[0290] Here, a similar image according to an embodiment of the present invention is stored in the inspection database IDU. Image of the determined test Among these, it can mean an image that has a similarity to the aforementioned test image that exceeds a predetermined standard.

[0291] At this time, according to the embodiment Image of the determined test This can be interpreted as an image of an inspection process where outliers were previously determined manually.

[0292] In other words, Image of the determined test This could be an inspection image where a determination for the presence or absence of outliers has already been made by a specific worker prior to the present time.

[0293] In one embodiment, such Image of the determined test is the relevant Image of the determined test It can include detailed information for the corresponding determination.

[0294] Here, the detailed determination information according to the embodiment refers to a predetermined inspection image (here, Image of the determined test This can mean information including the determination of whether or not there are outliers, that is, information including various metadata related to the determination of whether or not there are outliers.

[0295] In one embodiment, the detailed judgment information may include information about the operator in charge, judgment result information, judgment result feedback information, and / or defect type information.

[0296] At this time, the information of the worker in charge according to the embodiment is a predetermined inspection image (here, Image of the determined testThis information could identify the worker who performed the determination of whether or not there were outliers in the given value.

[0297] In one embodiment, the information on the assigned worker may include the worker's name, job title, work history, and / or skill level information.

[0298] Here, the proficiency information according to the embodiment may be information that identifies the degree of skill of an operator in manual vision inspection based on an inspection image (i.e., the manual determination of the presence or absence of outliers performed on the inspection image).

[0299] In other words, skill level information can be information that indicates the accuracy of determining whether or not a worker is an outlier, using a predetermined method.

[0300] In one embodiment, the computing system 1000 can calculate the error rate for the judgment result of the worker in question in order to obtain the above-mentioned proficiency information.

[0301] Specifically, the computing system 1000 can calculate the error rate (for example, a predetermined percentage (%)) for the worker's judgment result based on the worker's judgment result information and the judgment result feedback information that matches it.

[0302] The computing system 1000 can then identify the skill level information of the worker in question based on the calculated error rate.

[0303] In this case, the judgment result information according to the embodiment is a predetermined inspection image (here, Image of the determined test This information could identify the result of the worker's judgment regarding the presence or absence of outliers.

[0304] In one embodiment, the judgment result information may be OK (i.e., no outliers) or NG (i.e., outliers present), etc.

[0305] Furthermore, the judgment result feedback information according to the embodiment refers to a predetermined inspection image (here, Image of the determined test This information may be a judgment on whether the result of the determination is correct or incorrect.

[0306] In other words, the judgment result feedback information is a predetermined test image (here, Image of the determined test This information could represent whether the judgment made by the worker in charge of the task is correct or incorrect.

[0307] In one embodiment, the computing system 1000 can acquire the judgment result feedback information described above in conjunction with predetermined user input and / or an external server (for example, a defective product management server).

[0308] Then, the computing system 1000 processes the acquired judgment result feedback information in the corresponding Image of the determined test The data can be matched and stored and managed in the IDU (Inspection Database).

[0309] Furthermore, the defect type information according to the embodiment refers to a predetermined inspection image (here, Image of the determined test This information may identify a specific type of defect (e.g., the absence of a given part).

[0310] In the embodiment, the defect type information is provided to a predetermined user (as an embodiment, the applicable Image of the determined test This can be set based on input from workers (such as those who determined whether or not there were outliers).

[0311] Returning to Figure 10, in detail, in this embodiment, the computing system 1000, in conjunction with an inspection monitoring processing unit (MPU) and the like, can detect at least one similar image to the inspection image acquired as described above from the inspection database (IDU).

[0312] Figure 11 is a flowchart illustrating a similar image detection method according to one embodiment of the present invention, and Figure 12 is a conceptual diagram illustrating a similar image detection method according to one embodiment of the present invention.

[0313] Referring to Figures 11 and 12 in more detail, in this embodiment, the computing system 1000 includes an inspection image IIM and Image of the determined test Raw data similarity (RDS) between PIIs can be measured (S401).

[0314] For reference, here, the raw data similarity RDS is defined as a given dataset (in this embodiment, the inspection image IIM and at least one Image of the determined test This can refer to information that quantitatively measures the similarity between individual data points within a dataset (including PII).

[0315] As an embodiment, the computing system 1000 uses raw data similarity RDS measurement indices such as Structural Similarity Index Map (SSIM), Mean Square Error (MSE), Euclidean Distance, Cosine Similarity, Jaccard Similarity, Pearson Correlation Coefficient, and / or Manhattan Distance to analyze the inspection image IIM and Image of the determined test The raw data similarity ratio (RDS) between PII and other data can be measured.

[0316] Furthermore, in this embodiment, the computing system 1000 includes an inspection image IIM and Image of the determined test The feature vectors for each PII can be obtained (S403).

[0317] In detail, in the embodiment, the computing system 1000 works in conjunction with a deep learning model (hereinafter referred to as the image feature extraction model) that takes a predetermined image as input data and outputs a feature vector corresponding to the input image as output data, to extract a feature vector (IFV: hereinafter referred to as the inspection feature vector) for the inspection image IIM and Image of the determined test Feature vector for PII (PFV: hereafter, Tests that have been determined It is possible to obtain a feature vector.

[0318] In this embodiment, the image feature extraction model may be a deep learning model (hereinafter referred to as the first feature extraction model) that includes a pre-trained convolutional neural network (CNN) based on a predetermined image dataset (e.g., ImageNet, COCO (Common Objects in Context), and / or a natural image dataset such as PASCAL VOC) that is unrelated to the inspection image IIM.

[0319] In another embodiment, the image feature extraction model is specialized in determining the presence or absence of outliers in the inspection image IIM (e.g., multiple image datasets). Image of the determined test This could be a deep learning model (hereinafter referred to as the second feature extraction model) that includes a pre-trained convolutional neural network (CNN) based on an image dataset (such as one containing PII).

[0320] In this embodiment, the image feature extraction model may include an outlier detection model (ODM) (hereinafter referred to as the third feature extraction model) that has already been trained on a predetermined image dataset (in this embodiment, the target image dataset) specifically for determining the presence or absence of outliers in the inspection image IIM.

[0321] More specifically, in an embodiment, the computing system 1000 works in conjunction with the first feature extraction model and / or the second feature extraction model to perform the above-mentioned inspection feature vector IFV and Tests that have been determined The feature vector (PFV) can be obtained.

[0322] Specifically, the computing system 1000 uses a first feature extraction model and / or a second feature extraction model to perform an inspection image IIM and Image of the determined test You can input PII.

[0323] Then, the first feature extraction model and / or the second feature extraction model will use the input inspection image IIM and Image of the determined test It is possible to extract and output each feature vector of PII.

[0324] As a result, the computing system 1000 obtains the inspection feature vector (IFV) from the first feature extraction model and / or the second feature extraction model. Tests that have been determined The feature vector (PFV) can be obtained.

[0325] In another embodiment, the computing system 1000 works in conjunction with a third feature extraction model (i.e., an outlier detection model (ODM)) to obtain the above-mentioned inspection feature vector IFV and Tests that have been determined The feature vector (PFV) can be obtained.

[0326] The following information may be summarized or omitted if it overlaps with the explanation given above. In detail, in this embodiment, the computing system 1000 uses a third feature extraction model to obtain an inspection image IIM and Image of the determined test You can input PII.

[0327] Then, the third feature extraction model uses the input inspection image IIM and Image of the determined test For each of the PIIs, the aforementioned pre-trained model-based feature maps can be obtained.

[0328] Furthermore, the third feature extraction model uses the feature map of the acquired inspection image IIM (hereinafter referred to as the inspection feature map) and Image of the determined test PII feature map (hereinafter, Tests that have been determinedThe feature map can be input into the reconpatch layer mentioned above.

[0329] As a result, the third feature extraction model extracts a recompatch feature dataset (hereinafter referred to as the recompatch inspection dataset) from the recompatch layer according to the inspection feature map and Tests that have been determined Recompatch feature dataset according to feature map (hereinafter referred to as recompatch) Tests that have been determined You can obtain the dataset.

[0330] In this embodiment, the third feature extraction model is used to examine the recompatch inspection dataset and / or recompatch Tests that have been determined Core set sampling can also be performed based on the dataset.

[0331] As a result, the third feature extraction model uses the acquired recompatch inspection dataset to generate inspection feature vectors IFV and recompatch. Tests that have been determined Based on the dataset Tests that have been determined The feature vector (PFV) can be obtained.

[0332] Thus, according to this embodiment, the computing system 1000 can acquire feature vectors based on an outlier detection model (ODM) whose performance has been improved through learning optimized by anomaly detection.

[0333] Therefore, the computing system 1000 is an inspection image IIM and Image of the determined test By utilizing feature vectors extracted with higher accuracy from each feature data point of PII, the similarity between the corresponding images can be measured.

[0334] This allows the computing system 1000 to further improve the quality of similar images determined later, based on the measured similarity.

[0335] In addition, in this embodiment, the computing system 1000 can measure the similarity between the acquired feature vectors (S405).

[0336] In other words, the computing system 1000 uses the inspection feature vector IFV and acquired as described above. Tests that have been determined It is possible to measure the similarity (distance) between feature vectors (PFVs).

[0337] As an embodiment, the computing system 1000 uses measurement metrics such as cosine similarity, L1 distance, L2 distance, and / or Earth Mover distance to perform inspection of feature vectors (IFV) and Tests that have been determined It is possible to measure the similarity between feature vectors (PFVs) (FVS: hereinafter referred to as feature vector similarity).

[0338] In addition, in this embodiment, the computing system 1000 can calculate decision similarity based on the measured feature vector similarity FVS and / or raw data similarity RDS (S407).

[0339] Here, the similarity determined by the embodiment is the same as the inspection image IIM. Image of the determined test This can refer to the final similarity score, which determines the degree of similarity with respect to PII.

[0340] In detail, in the embodiment, the computing system 1000 can calculate the decision similarity by combining the feature vector similarity (FVS) and raw data similarity (RDS) described above in a predetermined manner.

[0341] In one embodiment, the computing system 1000 can calculate a predetermined weighted sum based on feature vector similarity (FVS) and raw data similarity (RDS), and calculate the decision similarity based on this sum.

[0342] Alternatively, in one embodiment, the computing system 1000 can select either feature vector similarity (FVS) or raw data similarity (RDS) to calculate the decision similarity.

[0343] In other words, depending on the embodiment, the computing system 1000 can also calculate the decision similarity using either the feature vector similarity (FVS) or the raw data similarity (RDS) on its own.

[0344] In addition, in this embodiment, the computing system 1000 can detect at least one similar image based on the calculated determination similarity (S409).

[0345] In detail, in the embodiment, the computing system 1000 can compare the decision similarity calculated as described above with a preset threshold (hereinafter referred to as the similarity threshold).

[0346] In this embodiment, if the determination similarity is equal to or greater than the similarity threshold, the computing system 1000 will determine the corresponding inspection image IIM and Image of the determined test It can be determined that PIIs are similar to each other.

[0347] In other words, the computing system 1000 matches if the decision similarity is equal to or greater than the similarity threshold. Image of the determined test PII can be detected as a similar image.

[0348] On the other hand, in one embodiment, if the determination similarity is less than the similarity threshold, the computing system 1000 will determine the corresponding inspection image IIM and Image of the determined test It can be determined that PIIs are different from each other.

[0349] In other words, computing system 1000 matches if the decision similarity is less than the similarity threshold. Image of the determined test PII can be excluded from similar images.

[0350] As a result, in this embodiment, the computing system 1000 can detect at least one similar image based on the determination similarity score.

[0351] In other words, in this embodiment, the computing system 1000 stores the inspection database IDU Image of the determined test Among the PIIs, at least one image that has a similarity to the inspection image IIM that is above a predetermined criterion can be detected as a similar image.

[0352] Returning to Figure 10, in another embodiment, the computing system 1000 can filter the detected similar images (S305).

[0353] In detail, in the embodiment, the computing system 1000 can obtain user input to set whether or not to perform filtering on at least one detected similar image.

[0354] In this embodiment, when the computing system 1000 receives user input to configure the execution of similar image filtering, it can proceed with a filtering process for at least one similar image.

[0355] More specifically, in the embodiment, the computing system 1000 can perform the similar image filtering process described above based on preset filtering conditions.

[0356] Here, the filtering conditions according to the embodiment may include a condition that the skill level of the worker who determined the presence or absence of outliers in a given similar image is less than or equal to a preset standard value (i.e., the skill level information matching the relevant similar image is less than or equal to a preset standard value).

[0357] Furthermore, the filtering conditions may include a condition where the presence or absence of an outlier determined for a given similar image is incorrect (i.e., the judgment result feedback information matching the relevant similar image is incorrect).

[0358] Furthermore, filtering conditions can include criteria that do not satisfy the similar image search conditions set by the user.

[0359] In this case, the similar image search conditions according to the embodiment may include conditions for matching a specific defect type, conditions for matching a specific worker, and / or conditions for matching a specific piece of equipment (e.g., a specific imaging sensor).

[0360] Furthermore, in this embodiment, the computing system 1000 can perform filtering to delete the corresponding similar images if the above-described filtering conditions are met.

[0361] Thus, in this embodiment, the computing system 1000 can filter out similar images detected from the search database that do not meet the user's needs or whose confidence in quality is below a certain threshold.

[0362] This enables the computing system 1000 to implement outlier detection guides and monitoring services that select and efficiently utilize more meaningful data.

[0363] On the other hand, in one embodiment, the computing system 1000 can determine whether the number of at least one similar image detected and / or filtered as described above (hereinafter referred to as the number of similar image detections) is equal to or greater than a preset number (N: hereinafter referred to as the minimum number of image detections).

[0364] Here, the minimum number of image detections (N) according to the embodiment may be set manually according to user input or automatically according to a similarity threshold.

[0365] In the embodiment where the minimum number of detected images (N) is automatically set according to a similarity threshold, the computing system 1000 can set the minimum number of detected images (N) inversely proportional to the similarity threshold.

[0366] In other words, the computing system 1000 can acquire more image data the more similar the image is detected according to a lower similarity criterion.

[0367] This allows the computing system 1000 to examine more past judgment records and verify a larger number of judgment opinions, especially when there are many cases where identifying outliers is ambiguous.

[0368] At the same time, the computing system 1000 can perform more efficient data processing by reducing the amount of data as there are more examples useful for identifying outliers.

[0369] Returning to the previous step, the computing system 1000, having determined whether the number of similar images detected is equal to or greater than the minimum number of images detected (N) as described above, can perform the similar image additional detection process if the number of similar images detected is less than the minimum number of images detected (N) (i.e., if it does not meet the minimum number of images detected (N) or greater).

[0370] Specifically, in this embodiment, the computing system 1000 has already detected similar images (i.e., images that have been determined to be similar to the inspection image IIM and have been previously detected). Image of the determined test The remaining PII) Image of the determined test Based on PII, the processes of steps S303 and / or S305 described above can be repeated.

[0371] In this embodiment, the computing system 1000 can perform the similar image addition detection process by adjusting the similarity threshold value described above downward by a user-defined and / or preset numerical value.

[0372] As a result, in this embodiment, the computing system 1000 can acquire at least one additionally detected similar image (hereinafter referred to as the additionally detected image) according to step S303, and further include this in the similar images corresponding to the inspection image IIM, thereby repeatedly performing the filtering process according to step S305.

[0373] In other words, if the number of similar images detected by the computing system 1000 does not meet a preset number, it can relax the similar image detection criteria and acquire and use additional similar images.

[0374] Therefore, the computing system 1000 can easily secure a sufficient amount of data necessary for the smooth operation of the outlier detection guide and monitoring service.

[0375] On the other hand, in this embodiment, if the number of similar images detected is equal to or greater than the minimum number of images detected (N) (i.e., if the minimum number of images detected (N) is satisfied), the computing system 1000 can determine at least one similar guide image based on the determination similarity of each detected similar image.

[0376] Here, the similar guide image according to the embodiment can mean the similar image that is ultimately determined to be provided to the user (in the embodiment, the operator) via display output from among the at least one similar image detected from the search database.

[0377] In detail, as an embodiment, the computing system 1000 can determine whether the determination similarity of each detected similar image meets a predetermined criterion (for example, the top M (M > 0%)).

[0378] In addition, in this embodiment, the computing system 1000 can determine at least one similar image that satisfies the predetermined criteria as a similar guide image.

[0379] In an embodiment, the computing system 1000 can perform the processes described below based on at least one determined similar guide image.

[0380] In this embodiment, the computing system 1000 can also automatically perform the filtering process described above only when the number of similar images detected is equal to or greater than the minimum number of images detected (N).

[0381] In addition, in this embodiment, the computing system 1000 can provide filtered similar images (S307).

[0382] In other words, in the embodiment, the computing system 1000 can provide at least one similar guide image determined as described above.

[0383] In detail, in the embodiment, the computing system 1000 can, in conjunction with a monitoring interface (I / F) unit MIU or the like, display and provide the similar guide image and the corresponding detailed judgment information as a predetermined graphic image.

[0384] Figure 13 shows an example of how a similar image is provided by one embodiment of the present invention. Referring to Figure 13, in this embodiment, the computing system 1000 can match each of at least one similar guide image SGI with the corresponding determination details JDI.

[0385] Furthermore, in this embodiment, the computing system 1000 can also provide the corresponding inspection image IIM in a matching manner.

[0386] Specifically, in the embodiment, the computing system 1000 can match and provide at least some metadata information from among at least one similar guide image SGI corresponding to the inspection image IIM and various metadata related to the determination of whether or not there are outliers in each similar guide image SGI (in the embodiment, such as information on the person in charge, judgment result information, judgment result feedback information and / or defect type information).

[0387] In this embodiment, the computing system 1000 may also further match and provide the decision similarity corresponding to the relevant similar guide image SGI.

[0388] In other words, the computing system 1000 can further match and provide a specific determination similarity score that indicates how similar the relevant similar guide image SGI and inspection image IIM are.

[0389] Thus, in this embodiment, the computing system 1000 is similar to the inspection image IIM that an operator currently attempts to manually determine whether or not there are outliers. Image of the determined test PII can be detected, and the detected similar images and data on whether or not there are outliers previously identified can be provided to the current operator for easy review.

[0390] In other words, the computing system 1000 can provide the current worker with the results of a defect determination made by an existing worker for images of a similar type to the image that currently needs to be determined for defects.

[0391] This allows the computing system 1000 to provide guidance that helps current operators more easily and consistently determine the presence or absence of outliers based on meaningful historical data, thereby reducing errors.

[0392] This allows the computing system 1000 to effectively improve the overall quality and performance of the vision inspection process.

[0393] Figure 14 shows an example of how an added-value image is provided according to one embodiment of the present invention.

[0394] Referring to Figure 14, in this embodiment, the computing system 1000 can provide at least one additional provided image API according to the determination information for each provided similar guide image SGI.

[0395] Here, the additional image API provided by the embodiment can mean an image that is provided in addition when the ratio of the judgment results for each similar guide image SGI provided to the inspection image IIM does not meet a predetermined standard.

[0396] In detail, in the embodiment, the computing system 1000 can calculate the ratio between the judgment results (in the embodiment, OK (i.e., no outliers) or NG (i.e., outliers)) for each similar guide image SGI provided to the inspection image IIM.

[0397] For example, the computing system 1000 can calculate the ratio between judgment results, such as "OK:NG=2:1".

[0398] Furthermore, in this embodiment, the computing system 1000 can determine whether the ratio between the calculated judgment results satisfies a preset ratio (for example, "OK:NG=1:1").

[0399] In this embodiment, if the computing system 1000 determines that a preset ratio (hereinafter referred to as the "required judgment result ratio") is not met, it can set the number of judgment result information and images (hereinafter referred to as the "required image attribute information") necessary to satisfy the said required judgment result ratio.

[0400] For example, if the ratio between the calculated judgment results is "OK:NG=2:1", there are "2" similar guide images SGI for OK and "1" similar guide image SGI for NG, and the requested judgment result ratio is "OK:NG=1:1", the computing system 1000 can set the necessary image attribute information to request one similar guide image SGI for NG.

[0401] Furthermore, in this embodiment, the computing system 1000 can extract at least one similar image (hereinafter referred to as a selected similar image) that has been filtered (i.e., deleted) in step S305 described above.

[0402] Depending on the embodiment, if the computing system 1000 determines that a preset ratio is not met, it may perform the similar image detection process described above in step S305. A detailed explanation of this will be provided by applying mutatis mutandis to the explanation in step S305.

[0403] Furthermore, in this embodiment, the computing system 1000 can determine an additional image API based on the selected similar images extracted as described above and / or additional detection images obtained according to the similar image detection process, and the required image attribute information set as described above.

[0404] In detail, in the embodiment, the computing system 1000 can determine, based on the required image attribute information, at least one image from among the selected similar images and / or additional detection images (i.e., next-ranked images) as an additional provided image API.

[0405] More specifically, in the embodiment, the computing system 1000 can select at least one image from the selected similar images and / or additional detection images that matches the required image attribute information.

[0406] For example, if a computing system 1000 is required to have "one NG similar guide image SGI" according to the required image attribute information, it can select "one" image from the sorted similar images and / or additional detection images whose judgment result information is "NG".

[0407] In this case, if there are multiple images that match the required image attribute information, the computing system 1000 can select the images in order of their determination similarity, from highest to lowest.

[0408] In another embodiment, the computing system 1000 can determine at least one selected image as an additional provided image API.

[0409] Furthermore, the computing system 1000 can provide at least one determined additional providing image API in addition to the similar guide image SGI.

[0410] In other words, the computing system 1000, in conjunction with a monitoring interface (I / F) unit MIU or the like, can provide at least one additional image API determined as described above and its corresponding determination details JDI by displaying them as a predetermined graphic image.

[0411] In other words, the computing system 1000 can further provide, through a display, the matching of each of at least one additional image API with the corresponding determination details JDI for each additional image API.

[0412] Thus, in this embodiment, if the judgment results for each similar guide image SGI provided to the worker are biased or do not meet a preset ratio, the computing system 1000 can further provide additional guide images to complement them.

[0413] This allows the computing system 1000 to assist workers in determining the presence or absence of outliers by providing additional data to supplement biased data, such as when only images judged as OK are provided, only images judged as NG are provided, or the proportion of images judged as OK is excessively high.

[0414] Figure 15 is an example of a diagram illustrating the alignment method when providing similar images according to one embodiment of the present invention.

[0415] Referring to Figure 15, in this embodiment, the computing system 1000 can provide at least one similar guide image SGI arranged according to a predetermined alignment criterion.

[0416] As an embodiment, the computing system 1000 can arrange and provide at least one similar guide image SGI in descending order of its determination similarity.

[0417] Furthermore, as an embodiment, the computing system 1000 can arrange and provide at least one similar guide image SGI in descending order of proficiency information of the similar guide image SGI.

[0418] Therefore, the computing system 1000 can prioritize referring to a similar guide image SGI that is more closely related to the inspection image IIM or corresponds to the judgment result of a more skilled worker.

[0419] This allows the computing system 1000 to provide a guide that can further improve the efficiency and accuracy of the operator in determining the presence or absence of outliers.

[0420] On the other hand, depending on the embodiment, the computing system 1000 can further cooperate with the post-inspection processing unit PPU to provide judgment result feedback information corresponding to the similar guide image SGI.

[0421] In other words, the computing system 1000 can provide for each similar guide image SGI whether the judgment of whether or not there are outliers made by past workers is correct or incorrect.

[0422] This allows the computing system 1000 to guide the current operator to more carefully consider the presence or absence of outliers in the inspection image IIM if there is a similar guide image SGI that has a history of being misjudged in the past.

[0423] In addition, in this embodiment, the computing system 1000 can acquire judgment result information for the inspection image IIM (S309).

[0424] To reiterate, the judgment result information according to this embodiment may be information that identifies the result of the operator's judgment on the presence or absence of outliers in a predetermined inspection image IIM (here, the current inspection image).

[0425] In one embodiment, the judgment result information may be OK (i.e., no outliers) or NG (i.e., outliers present), etc.

[0426] In other words, in this embodiment, the computing system 1000 can acquire data on whether or not an outlier is present, as determined by the current operator, for the inspection image IIM that is currently being tested for the presence or absence of outliers.

[0427] Referring further to Figure 13, in one embodiment, the computing system 1000 can provide a user interface (OJI: hereinafter referred to as the outlier judgment input interface) that can input judgment result information for the inspection image IIM in conjunction with a monitoring interface (I / F) unit MIU or the like.

[0428] Furthermore, the computing system 1000 can obtain judgment result information for the inspection image IIM described above based on user (in this case, the current worker) input based on the provided outlier judgment input interface OJI.

[0429] This allows the computing system 1000 to obtain judgment result data on whether or not there are outliers, which the current operator determined for the current inspection image IIM, by referring to past judgment records for cases similar to the current inspection image IIM.

[0430] In addition, in this embodiment, the computing system 1000 can store and manage the acquired judgment result information (S311).

[0431] In detail, in this embodiment, the computing system 1000 can match the judgment result information (hereinafter referred to as real-time judgment result information) acquired for the inspection image IIM as described above with the corresponding inspection image IIM, and store and manage it on the inspection database IDU.

[0432] In this embodiment, the computing system 1000 can determine whether or not to store the corresponding inspection image IIM according to the determination similarity between the inspection image IIM and the similar guide image SGI.

[0433] In detail, in the embodiment, the computing system 1000 can compare the determination similarity between the inspection image IIM and each corresponding similar guide image SGI with a preset threshold (hereinafter referred to as the image recording threshold).

[0434] In this embodiment, if a similar guide image SGI exists that has a determination similarity score equal to or greater than the image recording threshold, the computing system 1000 may choose not to store the corresponding inspection image IIM in the inspection database IDU.

[0435] On the other hand, in this embodiment, if the determination similarity between the inspection image IIM and each similar guide image SGI is all below the image recording threshold, the computing system 1000 can store the corresponding inspection image IIM in the inspection database IDU.

[0436] In other words, in this embodiment, the computing system 1000 has a similarity of a certain threshold or higher to the inspection image IIM. Image of the determined test If a PII exists, it is determined that the corresponding inspection image IIM is likely to be an image with overlapping tendencies with an existing image already stored in the inspection database IDU. To eliminate this duplication, the corresponding inspection image IIM may not be stored in the inspection database IDU.

[0437] This enables the computing system 1000 to implement outlier detection guides and monitoring services that perform data processing and operation more efficiently.

[0438] In this embodiment, if a similar guide image SGI (hereinafter referred to as a suspected duplicate image) with a determination similarity score equal to or greater than the image recording threshold exists, the computing system 1000 can also determine whether or not to store the corresponding inspection image IIM based on the determination result information of the corresponding suspected duplicate image and the inspection image IIM, respectively.

[0439] In detail, in this embodiment, the computing system 1000 can compare the judgment result information of the inspection image IIM (hereinafter referred to as judgment result inspection information) with the judgment result information of the suspected duplicate image (hereinafter referred to as judgment result suspect information).

[0440] In this embodiment, if the judgment result inspection information and the judgment result suspected information are different, the computing system 1000 can store the corresponding inspection image IIM in the inspection database IDU, even if duplicate suspected images exist.

[0441] For example, if the computing system 1000 has a judgment result inspection information of "OK" and a judgment result suspicion information of "NG", it can store the corresponding inspection image IIM in the inspection database IDU.

[0442] Thus, in this embodiment, the computing system 1000 has a similarity of a certain threshold or higher to the inspection image IIM. Image of the determined test Even if PII exists, the determination of whether or not there are outliers in the corresponding inspection image IIM and the similarity above a certain threshold are made. Image of the determined test If the determination of whether or not there are outliers in the PII (i.e., suspected duplicate images) differs, the corresponding test image IIM is already stored in the test database IDU. Image of the determined test If it is determined that the IIM contains meaningful information different from PII, the corresponding IIM can be stored in the IDU (Inspection Database).

[0443] In this way, the computing system 1000 selectively decides whether or not to store the inspection image IIM based on the determination similarity between the inspection image IIM and the similar guide image SGI, thereby minimizing the problem of unlimited data growth on the inspection database IDU and enabling the construction of the inspection database IDU based on valuable data containing more meaningful information.

[0444] As described above, the judgment record monitoring method and system for outlier detection guidance according to one embodiment of the present invention can help improve the consistency and minimize errors of real-time outlier detection by providing guidance for real-time outlier detection based on past outlier detection records, thereby enhancing the overall quality and performance of the vision inspection process.

[0445] In this case, the method and system for monitoring the determination record for an outlier detection guide according to one embodiment of the present invention has the effect of improving the quality of the provided guide and simultaneously improving the efficiency of data processing for providing the guide by utilizing a deep learning model specialized in anomaly detection using a patch feature-based learning method to provide a guide for outlier detection.

[0446] On the other hand, the embodiments of the present invention described above can be embodied in the form of program instructions that can be executed through a variety of computer components and can be recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., individually or in combination. The program instructions recorded on the computer-readable recording medium may be specifically designed and configured for the present invention or may be publicly known and available to those skilled in the field of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memory. Examples of program instructions include not only machine code, such as that produced by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. Hardware devices can be modified into one or more software modules to perform the processing according to the present invention, and vice versa.

[0447] The specific executions described herein are merely embodiments and do not in any way limit the scope of the invention. For the sake of brevity, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of such systems may be omitted. Furthermore, the wire connections or connecting members between components shown in the drawings are illustrative examples of functional and / or physical or circuit connections and may be substituted or shown as additional, diverse functional, physical, or circuit connections in actual devices. Also, components may not be necessary for the application of the invention unless specifically mentioned, such as "essential" or "important."

[0448] Furthermore, while the detailed description of the present invention has been provided with reference to preferred embodiments, a person skilled in the art or with ordinary knowledge in the art will understand that the present invention can be modified and altered in various ways without departing from the spirit and technical scope of the invention as described in the claims below. Therefore, the technical scope of the present invention is not limited to what is described in the detailed description of the specification, but must be determined by the claims. [Industrial applicability]

[0449] The present invention relates to a method and apparatus for executing a program that determines whether or not there are abnormalities in products produced by photographing them using a machine learning model, and therefore has industrial applicability.

Claims

1. A method for a computing system to perform judgment record monitoring for an outlier detection guide, Steps include obtaining a predetermined inspection image, The steps include detecting at least one similar image having a similarity (similarity) of the acquired inspection image or higher than a predetermined standard, The steps include providing detailed determination information which is information that identifies the detected similar images and the determination of whether or not there are outliers in each of the similar images, The steps include obtaining first judgment result information, which is information that identifies the result of determining whether or not there are outliers in the aforementioned inspection image, The step includes storing the inspection image and the first determination result information in a database according to the degree of similarity between the inspection image and the similar image. A method for recording and monitoring outlier detection data for an outlier detection guide.

2. The step of providing the aforementioned detailed determination information is: The process includes providing at least one piece of information from among the worker information, which identifies the worker responsible for determining whether or not there are outliers in the similar image, and the second determination result information, which identifies the result of the determination regarding the presence or absence of outliers in the similar image. A method for monitoring the determination record for an outlier determination guide according to claim 1.

3. The step of detecting similar images is: The process includes measuring the raw data similarity between at least one over-checked image pre-stored in the database and the checked image, A method for monitoring the determination record for an outlier determination guide according to claim 1.

4. The step of detecting similar images is: The steps include obtaining a test feature vector, which is a feature vector for the test image, and an overtest feature vector, which is a feature vector for the overtest image, based on a predetermined image feature extraction model, The further step includes measuring the similarity between feature vectors, which is the similarity between the acquired test feature vector and the overtested feature vector. A method for monitoring the determination record for an outlier determination guide according to claim 3.

5. The aforementioned image feature extraction model is This includes the Outlier Detection Model, a deep learning model specializing in anomaly detection using a patch feature-based learning method. A method for monitoring the determination record for an outlier determination guide according to claim 4.

6. The step of detecting similar images is: The step further includes calculating a decision similarity that identifies the final similarity between the test image and the overtest image based on the raw data similarity and the feature vector similarity, A method for monitoring the determination record for an outlier determination guide according to claim 4.

7. The step of detecting similar images is: The step of detecting similar images based on the determination similarity and a preset similarity threshold is further included. A method for monitoring the determination record for an outlier determination guide according to claim 6.

8. The step of detecting similar images is: The steps include determining whether the number of similar images detected satisfies a predetermined number, If the number of images does not meet the predetermined number, the further step includes detecting additional similar images. A method for monitoring the determination record for an outlier determination guide according to claim 7.

9. The step of detecting additional similar images is: The steps include adjusting the aforementioned similarity threshold, The step of detecting similar images based on the adjusted similarity threshold is included, A method for monitoring the determination record for an outlier determination guide according to claim 8.

10. The step of providing the aforementioned similar image and detailed determination information is: The step of further providing the aforementioned decision similarity is included, A method for monitoring the determination record for an outlier determination guide according to claim 6.

11. The step of providing the aforementioned similar image and detailed determination information is: The steps include detecting additional similar images according to the ratio between the second judgment result information for at least one similar image, The step includes further providing an additional image which is a similar image that has been detected. A method for monitoring the determination record for an outlier determination guide according to claim 2.

12. The step of providing the aforementioned similar image and detailed determination information is: The step includes aligning the detected similar images based on at least one of the following: the information of the person in charge and the degree of similarity between the inspection image and the similar images. A method for monitoring the determination record for an outlier determination guide according to claim 2.

13. The step of storing in the aforementioned database is: The process further includes the step of storing the inspection image and the first determination result information in a database, depending on whether the first determination result information and the second determination result information are identical. A method for monitoring the determination record for an outlier determination guide according to claim 2.

14. The step further includes filtering the detected similar images according to already set conditions. A method for monitoring the determination record for an outlier determination guide according to claim 1.

15. The step of acquiring the aforementioned inspection image is: The process includes the step of acquiring the image based on an outlier detection model, which is a deep learning model specialized in anomaly detection using a patch feature-based learning method. A method for monitoring the determination record for an outlier determination guide according to claim 1.

16. At least one Vision Inspection Unit, At least one Inspection Monitoring Processing Unit, At least one monitoring interface (I / F) unit, It includes at least one inspection database, The steps include controlling the vision inspection unit to acquire a predetermined inspection image, The steps include controlling the inspection monitoring processing unit to detect at least one similar image having a similarity (similarity) of the acquired inspection image or higher than a predetermined standard, The steps include controlling the monitoring interface (I / F) unit to provide detailed determination information, which is information that identifies the detected similar image and the determination content regarding the presence or absence of outliers in each of the similar images, The steps include: controlling the monitoring interface (I / F) unit to obtain first judgment result information, which is information that identifies the result of determining whether or not there are outliers in the inspection image; The steps include controlling the inspection monitoring processing unit to store the inspection image and the first determination result information in the inspection database according to the degree of similarity between the inspection image and the similar image. A judgment record monitoring system for outlier detection guides.

17. At least one Vision Inspection Unit, At least one Inspection Monitoring Processing Unit, At least one monitoring interface (I / F) unit, It includes at least one inspection database, The vision inspection unit is controlled to acquire a predetermined inspection image. The inspection monitoring processing unit is controlled to detect at least one similar image having a similarity (similarity) of the acquired inspection image or higher than a predetermined standard. The monitoring interface (I / F) unit is controlled to provide detailed determination information, which is information that identifies the detected similar image and the determination of whether or not there are outliers in each of the similar images. The monitoring interface (I / F) unit is controlled to obtain first judgment result information, which is information that identifies the result of determining whether or not there are outliers in the inspection image. The inspection monitoring processing unit is controlled to store the inspection image and the first judgment result information in the inspection database according to the degree of similarity between the inspection image and the similar image. Computing device.

Images