Method and system for predicting process defects
The method and system address class imbalance in anomaly detection by constructing an AI model that effectively predicts defects using process data, improving detection accuracy and reducing costs through fair evaluation and robust performance across varying manufacturing environments.
Patent Information
- Application Number
- KR1020250147047
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2024-10-22
- Filing Date
- 2025-10-13
- Publication Date
- 2026-11-11
- Estimated Expiration
- 2045-10-13
AI Technical Summary
Conventional anomaly detection technologies struggle with class imbalance issues, leading to low prediction accuracy and difficulty in flexibly responding to changing process conditions, resulting in increased costs and delays in defect detection during product manufacturing.
A method and system that utilizes process data to construct an artificial intelligence model capable of determining product abnormalities by addressing class imbalance through feature engineering, Mahalanobis distance classification, and contrastive learning, enabling fair evaluation and robust performance across various environments.
Enables early detection of defects, reducing inspection costs and time, optimizing production processes, and enhancing the reliability and universality of the AI model for diverse manufacturing conditions.
Smart Images

Figure 112025114212967-PAT00004_ABST
Abstract
Description
Technology Field
[0001] The present invention relates to a method and system for predicting process defects, and provides a method and system for predicting process defects capable of determining whether a product is abnormal by constructing an artificial intelligence model applicable to various process environments. Background Technology
[0002] Product production proceeds sequentially through the Component Assembly Line, which is a component-level process, the Sub Assembly Line, which combines components, and the Total Assembly Line, which assembles the final product.
[0003] In this regard, the display manufacturing process consists of a series of continuous steps requiring numerous precision processes and high-level quality control, and minute environmental changes or operational deviations occurring at each stage can directly affect product quality. In particular, due to the process characteristics of display products that demand high precision, problems such as bubble formation and curing defects may occur depending on changes in minute variables.
[0004] Traditionally, defects occurring during the product manufacturing process were primarily identified retrospectively through inspection equipment or operator judgment. Consequently, this frequently led to issues such as increased operating costs for inspection equipment, production delays, and resource waste resulting from defect detection late in the process. Furthermore, some defects were difficult to detect visually or only became apparent after a period of time, making early response challenging.
[0005] To address these issues, active attempts have recently been made to utilize AI-based anomaly detection technology to predict the likelihood of defects in real-time during the process and to respond proactively. However, conventional anomaly detection technologies are prone to degraded performance due to class imbalance, where the proportion of defective data is extremely small compared to the total data. Furthermore, prediction accuracy can be low because models often fail to sufficiently learn key features, such as time intervals between processes and interaction variables like temperature and pressure. Moreover, conventional models are generally trained with fixed parameters, making it difficult to flexibly respond to real-time changing process conditions or equipment status.
[0006] Therefore, there is still a need for technology that analyzes various process data collected during the product manufacturing process to determine whether a product is defective. The problem to be solved
[0007] The present invention aims to provide a process defect prediction method and system capable of determining whether a product is abnormal by analyzing process data collected during the product production process.
[0008] More specifically, the present invention aims to provide a process defect prediction method and system that solves class imbalance problems and can effectively detect product abnormalities by utilizing process data.
[0009] Furthermore, the present invention aims to provide a process defect prediction method and system capable of solving class imbalance problems and performing fair evaluation of artificial intelligence models.
[0010] Furthermore, the present invention aims to provide a method and system for predicting process defects that can perform a fair evaluation of the performance of an artificial intelligence model and construct an artificial intelligence model applicable to various process environments. means of solving the problem
[0011] To solve the problem described above, a computerized process defect prediction method according to the present invention, comprising the following steps, may include: specifying process data including data corresponding to at least one product produced through a plurality of process steps; configuring a training data set and a verification data set, respectively, using the process data; training at least one artificial intelligence model using the training data set so that at least one artificial intelligence model to be trained determines whether there is an abnormality in the product; inputting the verification data set into at least one artificial intelligence model trained with the training data set; obtaining a prediction value for the verification data set from the at least one trained artificial intelligence model; and evaluating the abnormality detection performance of the at least one trained artificial intelligence model based on the prediction value.
[0012] In an embodiment, the method further includes the step of performing feature engineering on the process data, and the process data may be configured to include a plurality of different features and a plurality of samples having a value corresponding to each of the plurality of features.
[0013] In the embodiment, in the step of performing the feature engineering, feature engineering is performed on the process data to generate a derived variable at each of the plurality of process steps, and each of the training data set and the validation data set may further include the derived variable and a value corresponding to the derived variable.
[0014] In an embodiment, in the step of generating the derived variable, the derived variable may be generated using at least some of the features corresponding to each of the plurality of process steps among the plurality of features included in the process data and values corresponding to the at least some of the features.
[0015] In the embodiment, the step of classifying each of the plurality of samples according to a pre-set classification criterion may be further included.
[0016] In an embodiment, the classifying step may include: dividing the plurality of samples into a first group or a second group based on a value corresponding to a specific feature among the plurality of features; calculating a first covariance matrix for the first group and a second covariance matrix for the second group, respectively; calculating a first mean vector for the first group and a second mean vector for the second group, respectively; calculating a Mahalanobis distance between the first mean vector and the second mean vector for each of the plurality of samples; and classifying the plurality of samples based on the preset classification criteria and the calculated Mahalanobis distance.
[0017] In an embodiment, the step of calculating the Mahalanobis distance may be a step of calculating, for each of the plurality of samples, a first Mahalanobis distance based on the first average vector calculated from the first group and a second Mahalanobis distance based on the second average vector calculated from the second group, respectively.
[0018] In an embodiment, the specific feature is a feature indicating whether the produced product is normal or defective, and the value corresponding to the specific feature is configured to have a first value or a second value depending on whether the produced product is normal or defective, and the first group may include a sample among the plurality of samples that includes the first value for the specific feature, and the second group may include a sample among the plurality of samples that includes the second value for the specific feature.
[0019] In an embodiment, the previously set classification criteria are set based on the difference in the Mahalanobis distance between the normal distribution and the abnormal distribution of each of the plurality of samples, the normal distribution is defined based on the first covariance matrix and the first mean vector calculated for the first group, and the abnormal distribution can be defined based on the second covariance matrix and the second mean vector calculated for the second group.
[0020] In the embodiment, in the classification step, at least some of the samples among the plurality of samples, in which the difference between the Mahalanobis distance between the normal distribution and the abnormal distribution satisfies the first classification criterion among the preset classification criteria, can be classified as the first sample.
[0021] In the embodiment, in the classification step, at least some of the samples among the plurality of samples, in which the difference between the Mahalanobis distance between the normal distribution and the abnormal distribution satisfies the second classification criterion among the preset classification criteria, can be classified as the second sample.
[0022] In an embodiment, the plurality of samples includes a first sample and a second sample classified according to a preset classification criterion, and in the step of constituting the verification data set, at least a portion of each of the first sample and the second sample may be included in the verification data set such that the composition ratio of each of the first sample and the second sample satisfies a preset composition ratio criterion.
[0023] In an embodiment, the step of configuring the verification data set may be a step of configuring the verification data set such that the composition ratio of each of the first sample and the second sample satisfies the preset composition ratio standard, based on a first index corresponding to the first sample and a second index corresponding to the second sample.
[0024] In the embodiment, in the training step, the at least one artificial intelligence model to be trained can be subjected to contrastive learning using the training data set in order to determine whether the product is defective.
[0025] In an embodiment, the at least one artificial intelligence model to be trained includes a plurality of machine learning models and a plurality of deep learning models, and in the training step, the plurality of machine learning models and the plurality of deep learning models can each be trained using the training data set.
[0026] In an embodiment, in the evaluation step, a sample corresponding to at least one product included in the verification data set is input to each of the plurality of learned machine learning models and the plurality of learned deep learning models, a plurality of predicted values for the sample are obtained from each of the plurality of learned machine learning models and the plurality of learned deep learning models, and the anomaly detection performance of each of the plurality of learned machine learning models and the plurality of learned deep learning models can be evaluated based on the plurality of predicted values.
[0027] In an embodiment, the step of determining a final predicted value for the sample using the plurality of predicted values may be further included.
[0028] In an embodiment, the step of determining the final predicted value may be configured such that a plurality of predicted values obtained from a plurality of trained machine learning models are ensembled, and a final predicted value for the sample is determined using the ensemble result and the top N predicted values obtained from the plurality of trained deep learning models, and the plurality of trained deep learning models perform post-correction on the ensemble result.
[0029] A process defect prediction system according to the present invention, comprising a memory configured to store executable instructions and one or more processors configured to perform operations by executing one or more instructions, specifies process data including data corresponding to at least one product produced through a plurality of process steps, and respectively configures a training data set and a verification data set using the process data, and in order to enable at least one artificial intelligence model to be trained to determine whether there is an abnormality in the product, trains at least one artificial intelligence model using the training data set, inputs the verification data set to at least one artificial intelligence model trained with the training data set, obtains a prediction value for the verification data set from at least one trained artificial intelligence model, and evaluates the abnormality detection performance of at least one trained artificial intelligence model based on the prediction value.
[0030] A program according to the present invention is a program that is executed by one or more processes in an electronic device and can be stored on a computer-readable recording medium, and may include instructions for performing the following steps: specifying process data including data corresponding to at least one product produced through a plurality of process steps; configuring a training data set and a verification data set, respectively, using the process data; training at least one artificial intelligence model using the training data set so that at least one artificial intelligence model to be trained determines whether there is an abnormality in the product; inputting the verification data set into at least one artificial intelligence model trained with the training data set; obtaining a prediction value for the verification data set from at least one trained artificial intelligence model; and evaluating the abnormality detection performance of at least one trained artificial intelligence model based on the prediction value. Effects of the invention
[0031] As described above, according to the process defect prediction method and system of the present invention, an artificial intelligence model can be constructed to determine whether a product is defective by utilizing process data collected during the product production process. Through this, the present invention can determine whether a product is defective in advance, thereby reducing the costs (e.g., equipment operation costs, labor costs, etc.) and inspection time required for product inspection, while simultaneously increasing the efficiency of product production and achieving overall process optimization.
[0032] Furthermore, according to the process defect prediction method and system of the present invention, an artificial intelligence model is constructed to determine whether a product is defective by considering class imbalances in process data, and the possibility of defects occurring during the production process can be determined in advance using the constructed artificial intelligence model. Through this, the present invention can reduce the burden of the inspection process and reduce overall manufacturing quality control costs.
[0033] Furthermore, according to the process defect prediction method and system of the present invention, key characteristics such as volume change and pressure change are analyzed based on physical variables (e.g., volume, time, speed, temperature, pressure, etc.), and important features can be extracted at each process stage. Through this, the present invention generates derived variables at each of the multiple process stages, thereby ensuring data diversity and enabling the model to efficiently learn the diversity of data characteristics between processes or the complex correlations between physical characteristics. In other words, the present invention allows for the quantitative analysis of changes at each process stage by utilizing derived variables, enabling the early detection of process abnormality signals and the prediction of the likelihood of defect occurrence in advance.
[0034] Furthermore, according to the process defect prediction method and system of the present invention, a verification data set can be constructed by evenly distributing samples of different classes, and an artificial intelligence model can be verified and evaluated using various evaluation indicators and verification strategies. Through this, the present invention enables fair performance evaluation that is not biased toward a specific class, and can enhance the robustness and reliability of the artificial intelligence model. Accordingly, the artificial intelligence model can be applied usefully and stably to various process environments and data characteristics in actual field conditions.
[0035] Furthermore, according to the process defect prediction method and system of the present invention, the artificial intelligence model is not biased toward a specific process or class and can learn the diversity of data characteristics between processes, thereby improving generalization performance under various production conditions when applied to actual processes. In other words, the present invention can increase the universality of the model by performing various modeling that can be applied in diverse process environments, and thus can contribute significantly to improving productivity and strengthening quality competitiveness throughout the manufacturing industry. Brief explanation of the drawing
[0036] FIG. 1a illustrates an example of a block diagram of a computing system in which the present invention can be implemented. FIG. 1b illustrates an example of a block diagram of a computing device that may be included in a user computing device, a server computing system, and a training computing system, as an embodiment of a computing system in which the present invention can be implemented. FIG. 1c illustrates an example of a block diagram from another perspective of a computing device, which is one of the components of a computing system. FIGS. 2, FIGS. 3a, and FIGS. 3b are conceptual diagrams for explaining a process defect prediction system according to the present invention. FIGS. 4, FIGS. 5 and FIGS. 6 are flowcharts illustrating a process defect prediction method according to the present invention. FIGS. 7, FIGS. 8, FIGS. 9, and FIGS. 10 are conceptual diagrams for explaining a process defect prediction method according to the present invention. Figure 11 shows formulas related to the process defect prediction method according to the present invention. FIG. 12 illustrates a structural diagram and a data flow diagram of a process defect prediction method and system utilized according to an embodiment of the present invention. Specific details for implementing the invention
[0037] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Identical or similar components are assigned the same reference number regardless of the drawing symbols, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably solely for the ease of drafting the specification and do not have distinct meanings or roles in themselves. Furthermore, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of related prior art could obscure the essence of the embodiments disclosed in this specification, such detailed description will be omitted. Additionally, the attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification; the technical concept disclosed in this specification is not limited by the attached drawings, and it should be understood that they include all modifications, equivalents, and substitutions that fall within the spirit and technical scope of the present invention.
[0038] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.
[0039] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.
[0040] Singular expressions include plural expressions unless the context clearly indicates otherwise.
[0041] In this application, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.
[0042] Hereinafter, the present invention will be examined in more detail with reference to the attached drawings. FIG. 1a illustrates an example of a block diagram of a computing system in which the present invention can be implemented. FIG. 1b illustrates an example of a block diagram of a computing device that may be included in a user computing device, a server computing system, and a training computing system as an embodiment of a computing system in which the present invention can be implemented. FIG. 1c illustrates an example of a block diagram of a computing device in another aspect, which is one of the components of a computing system.
[0043] Furthermore, FIGS. 2, 3a, and 3b are conceptual diagrams for explaining a process defect prediction system according to the present invention. FIGS. 4, 5, and 6 are flowcharts for explaining a process defect prediction method according to the present invention. FIGS. 7, 8, 9, and 10 are conceptual diagrams for explaining a process defect prediction method according to the present invention, and FIG. 11 is a formula related to the process defect prediction method according to the present invention. FIG. 12 illustrates a structural diagram and a data flowchart of a process defect prediction method and system utilized according to an embodiment of the present invention.
[0044] Meanwhile, FIG. 1a illustrates an example of a block diagram of a computing system in which the present invention can be implemented. In this regard, the process defect prediction system according to the present invention can be implemented through a computing device described below and can perform data processing related to the process defect prediction method described in this specification.
[0045] Referring to FIG. 1a, a computing system (10000) that performs a method of determining whether a product is defective by analyzing process data collected during a product production process according to one embodiment of the present invention may include at least one computing device. At this time, the at least one computing device may be a single processor or a multi-processor computing device.
[0046] The components of at least one computing device of the present invention may include various hardware components such as one or more processors, memory, other hardware, and a system bus (not shown) that connects various system components so that they can transmit and receive data to and from each other (e.g., telecommutatively connected, physically connected, electrically connected), and the components of at least one computing device are not limited thereto and may be very diverse.
[0047] Meanwhile, at least one computing device included in a computing system (10000) that performs a method of determining whether a product is defective by analyzing process data collected during the product production process may be connected to communicate via a network (1070). For example, at least one computing device included in the computing system (10000) may be clustered or may be part of a local area network (LAN). Additionally, at least one computing device may be part of a wide area network (WAN) or connected to at least one of a client-server network and a peer-to-peer network within the cloud.
[0048] Meanwhile, when at least one computing device is used in at least one of a network environment and a cloud computing environment, the at least one computing device may be connected to at least one of a public and private network through a network interface or adapter. In one embodiment, other communication connection devices, such as a modem, may be used to establish communication through the network. The modem may be at least one of an internal modem and an external modem, and may be connected to a system bus through a network interface or a specific mechanism, etc. A wireless network component consisting of an interface and an antenna may be coupled to the network through a device such as an access point, a peer computer, etc. In the present invention, the method of connecting at least one computing device to communicate through the network (1070) is not limited, and it may be connected to communicate in a manner different from the described example.
[0049] Furthermore, other computer-type devices and / or systems not shown in FIG. 1a may also interact technically with at least one computing device or other system through one or more connections to the network (1070) via a network interface. Here, the network interface may include network interface equipment such as a physical network interface controller (NIC) or a virtual network interface (VIF).
[0050] The network (1070) of the present invention may include various forms such as the Internet, WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), 5G (5th Generation Mobile Telecommunication), Bluetooth (Bluetooth™ RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra-Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi Direct, Wireless USB (Wireless Universal Serial Bus), etc., and in the present invention, data transmission may be performed based on standard communication protocols such as TCP / IP, HTTP, SSL, etc.
[0051] A computing system (10000) that performs a method of determining whether a product is defective by analyzing process data collected during the production process of a product according to the present invention may include at least one of a user computing device (1010), a training computing system (1050), and a server computing system (1030).
[0052] A user computing device (1010) according to the present invention may be understood as a computing device comprising at least one processor (1011) and a memory (1012) that perform a method of determining whether there is an abnormality in a product by analyzing process data collected during the production process of a product. For example, the user computing device (1010) may include at least one computing device among a smartphone, a smart TV, a laptop computer, a desktop computer, a digital broadcasting terminal, a PDA (personal digital assistants), a PMP (portable multimedia player), a navigation device, a slate PC, a tablet PC, an ultrabook, a wearable device (e.g., a smartwatch, a smart glass, and a head-mounted display).
[0053] At least one processor (1011) constituting the user computing device (1010) may include one or more general-purpose processors and / or one or more special-purpose processors. For example, at least one processor (1011) constituting the user computing device (1010) may be composed of at least one or a plurality of electrically connected processors among a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), a neural network processing unit (NPU), an arithmetic logic unit (ALU), a floating-point arithmetic unit (FPU), an application integrated circuit, an application semiconductor (ASIC), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, and / or other electrical units for performing functions.
[0054] Furthermore, at least one processor (1011) may be configured to execute computer-readable instructions contained in memory (1012) and / or other instructions described herein.
[0055] The memory (1012) constituting the user computing system (1010) according to the present invention may include volatile memory, non-volatile memory, fixed media, removable media, magnetic media, optical media, semiconductor media and / or other types of physically durable storage media.
[0056] For example, the memory (1012) may include one or more non-transient / transient computer-readable storage media such as RAM, ROM, HDD (Hard Disk Drive), SSD (Solid State Disk), SSD (Silicon Disk Drive), EEPROM, EPROM, flash memory device, magnetic disk, and combinations thereof, and may include web storage of a server that performs the storage function of memory on the internet. This memory (1012) may store data and instructions necessary for the at least one processor (1011) to perform the operation of an application that analyzes process data collected during the production process of a product to determine whether the product is defective.
[0057] A user computing device (1010) may include one or more user input components (1021) that detect user input. For example, the user input component (1021) may also be referred to as a user interface module. The user input component (1021) may include a touch screen, a computer mouse, a keyboard, a keypad, a touchpad, a trackball, a joystick, a voice recognition module, or other similar devices. However, the present invention does not limit the type of user input component (1021). In this case, the user input component (1021) in the present invention does not necessarily mean a hardware means, but can be understood as a channel for receiving input from a user. Meanwhile, the user of the present invention may refer to an automated agent, script, playback software, etc., that operates on behalf of one or more people.
[0058] A user can interact with a computing system (10000) including at least one computing device through input text, touch, voice, movement, computer vision, gestures and / or other forms of input / output using a user input component (1021). For example, the user input component (1021) may include one or more of a command line interface (CLI), a graphical user interface (GUI), a natural user interface (NUI), a voice command interface and / or other user interface (UI) representations.
[0059] Between the user input component (1021) and the user computing device (1010), one or more application programming interface (API) calls may be made based on user input received from the user interface and / or network.
[0060] Here, the expression "based on" may be interpreted to include cases where it is based on the use of a specific configuration, modified from, derived from, influenced by, dependent on, or otherwise derived from a specific configuration. In some embodiments, an API call may be configured for a specific API, which may be interpreted or converted into an API call configured for another API. Here, an API may refer to a defined interface or connection between computers or between computer programs.
[0061] In one embodiment, the user computing device (1010) may store at least one machine learning model (1020). For example, the user computing device (1010) may be various machine learning models, such as a plurality of neural networks (e.g., deep neural networks) that perform the role of determining whether there is an abnormality in the product by analyzing process data collected during the product production process, or other types of machine learning models including non-linear models and / or linear models, and may be composed of a combination thereof.
[0062] According to an embodiment of the present invention, a user computing device (1010) may perform a method of determining whether a product is defective by analyzing process data collected during the production process of a product using a local or / and external machine learning model (1020). Alternatively, the user computing device (1010) may perform a method of determining whether a product is defective by analyzing process data collected during the production process of a product using a machine learning model (1040) provided by a server.
[0063] Additionally, according to another embodiment of the present invention, a server computing system (1030) communicating with a user computing device (1010) may provide a result of determining whether a product is abnormal (e.g., normal or defective) to the user computing device (1010) via an application or / and the web, in accordance with a request from a user received through the user computing device (1010).
[0064] In addition, according to another embodiment of the present invention, at least a part of a user computing device (1010) and a server computing system (1030) are interconnected to perform a method of determining whether there is an abnormality in a product from process data collected during the production process of a product, thereby providing the result of determining whether there is an abnormality in a product to the user.
[0065] Additionally, according to various embodiments of the present invention, a user computing device (1010) and / or a server computing system (1030) can learn a machine learning model (1020, 1040) performed in a method of determining whether a product is defective by analyzing process data collected during the production process of a product through interaction with a training computing system (1050) that is communicatedly connected via a network (1070). In this case, the training computing system (1050) may be a computing system separate from the server computing system (1030). Alternatively, in some embodiments, the training computing system (1050) may be a part of the server computing system (1030) or a part of the user computing device (1010).
[0066] Meanwhile, the server computing system (1030) may include at least one processor (1031) and memory (1032). Here, the processor (1031) may be composed of at least one or a plurality of electrically connected processors among a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), a neural network processing unit (NPU), an application integrated circuit, an application semiconductor (ASIC), an arithmetic logic unit (ALU), a floating-point arithmetic unit (FPU), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, and / or other electrical units for performing functions. For example, at least one processor (1031) may include a circuit and a transistor configured to execute instructions from memory (1032).
[0067] The memory (1032) constituting the server computing system (1030) according to the present invention may include volatile memory, non-volatile memory, fixed media, removable media, magnetic media, optical media, semiconductor media, and / or other types of physically durable storage media. For example, the memory (1032) may include one or more non-transient / transient computer-readable storage media such as RAM, ROM, HDD (Hard Disk Drive), SSD (Solid State Disk), SSD (Silicon Disk Drive), EEPROM, EPROM, flash memory device, magnetic disk, etc., and combinations thereof, and may include web storage of a server that performs the storage function of memory over the internet. Additionally, the server computing system (1030) may further include a data storage (data store). For example, the data storage may be composed of at least one of a relational database, a NoSQL database, a data warehouse, and a local file system.
[0068] In the memory (1032) constituting the server computing system (1030) according to the present invention, data and instructions necessary for the at least one processor (1031) to perform the operation of an application that analyzes process data collected during the production process of a product and determines whether the product is defective may be stored.
[0069] In one embodiment, the server computing system (1030) may be composed of a single device or a plurality of computing devices, and may be configured to operate according to a sequential or parallel computing architecture. Additionally, a distributed processing system may be configured with a plurality of networked devices.
[0070] Meanwhile, the training computing system (1050) may include at least one processor (1051) and memory (1052). The model trainer (1060) is a logical component that executes the training of at least one machine learning model (1020, 1040) and may be implemented in the form of hardware, firmware, or software. For example, the model trainer (1060) may be executed by the processor (1051) after loading training data (1061) stored in a storage device into memory (1052). For example, the model trainer (1060) may be configured to execute one or more operations (e.g., model training, model reconstruction, model validation, model testing) on at least one machine learning model.
[0071] The machine learning model of the present invention may include at least one of a statistical model, an algorithm, a neural network (NN), a convolutional neural network (CNN), a generative neural network (GNN), a Word2Vec model, a Bag of Words model, a TF-IDF (document frequency-inverse document frequency) model, a GPT (Generative Pre-trained Transformer) model (or other autoregressive models), a PPO (Proximal Policy Optimization) model, a nearest neighbor model (e.g., a k-nearest neighbor model), a linear regression model, a K-means clustering model, a Q-learning model, a TD (Temporal Difference) model, a Deep Adversarial Network model, and all other types of models further described herein.
[0072] Specifically, the model trainer (1060) may execute operations to train a machine learning model, and said operations may include at least one of adding, removing, and modifying model parameters. At this time, the training of the machine learning model may be at least one of supervised learning, semi-supervised learning, and unsupervised learning. In one embodiment, the training of the machine learning model may include the step of repeatedly inputting training data (1061) based on epochs and repeatedly performing the machine learning model training process configured in this way. Here, an epoch may refer to a unit in which the entire set of training data (1061) undergoes forward and backpropagation processing once. In some implementations, different levels of training methods (e.g., supervised learning, semi-supervised learning, unsupervised learning) may be used for different epochs.
[0073] The training data (1061) of the present invention may include input data and / or data previously output from at least one machine learning model (e.g., recursive learning feedback).
[0074] At least one parameter of a machine learning model may include at least one of a seed value, a model node, a model layer, an algorithm, a function, connections between different machine learning models, connections between parameters, machine learning model constraints, and other digital components that influence the output of the machine learning model. In this case, model connections between different machine learning models may include or represent relationships between model parameters and / or models, which may be dependent or interdependent, hierarchical, and / or static or dynamic. The combinations and configurations of model parameters described herein may be too complex to be maintained or utilized by human cognitive abilities.
[0075] In the present invention, the machine learning parameters described according to the embodiments are not limited, and a single machine learning model may further include a plurality of model parameters.
[0076] Meanwhile, FIG. 1b illustrates an example of a block diagram of a computing device (1100) that may be included in a user computing device (1010), a server computing system (1030), and a training computing system (1050), as an embodiment of a computing system (10000) in which the present invention can be implemented.
[0077] As illustrated in FIG. 1b, the computing device (1100) may include at least one application (e.g., Application 1 to Application N), and each of the at least one application may include a machine learning library and a model execution environment for performing a method to determine whether a product is abnormal by analyzing process data collected during the production process of a machine learning-based product. The at least one application included in the computing device (1100) may communicate with the sensor, context manager, device state manager, or additional component(s) within the computing device (1100) via an Application Programming Interface (API). In one embodiment, the at least one application may interface with device components, such as receiving sensor data or state data or transmitting prediction results to an output device via a public or private API.
[0078] Meanwhile, FIG. 1c illustrates an example of a block diagram in another aspect of a computing device (1200), which is one of the components of a computing system (10000) that performs a method of determining whether a product is defective by analyzing process data collected during the production process of a product according to an embodiment of the present invention.
[0079] A computing device (1200) according to the present invention may include at least one application (e.g., Application 1 to Application N), and at least one application may communicate with a central intelligence layer (1210). Each application may interact with a shared model within the central intelligence layer (1210) through an API (e.g., a common API).
[0080] The central intelligence layer (1210) includes one or more machine learning models and may share them among multiple applications or provide them independently to each. In one embodiment, the central intelligence layer (1210) may be integrated as part of an operating system or implemented as a separate logical layer.
[0081] Additionally, the central intelligence layer (1210) can communicate with the central device data layer (1220). The central device data layer (1220) can integrate and store data corresponding to at least one product included in the process data collected during the production process of a product stored within the computing device (1200), and provide this as input data necessary to determine whether there is an abnormality in the product. Each device component (e.g., sensor, state manager, etc.) can communicate with the central device data layer (1220) through a private API, etc.
[0082] The technology described in this specification may be composed of a single or multiple computing devices, and a machine learning model that performs a method to determine whether a product is defective by analyzing process data collected during the product production process may be executed sequentially or in parallel on a single component or multiple distributed components. The data storage, machine learning model, and application may be distributed and operated locally or over a network, and these configurations can be flexibly applied to various system architectures.
[0083] Meanwhile, the present invention relates to a method and system for predicting process defects capable of determining whether a product is defective by analyzing process data collected during the production process of a product. More specifically, the present invention relates to a method and system for predicting process defects capable of resolving class imbalance issues and effectively detecting product defects by utilizing process data.
[0084] The process defect prediction system according to the present invention includes at least one artificial intelligence model, and the present invention aims to determine whether a product is defective by constructing an artificial intelligence model applicable to various process environments.
[0085] In this case, the artificial intelligence model mentioned in the present invention may include various models that can be utilized depending on various situations or purposes. For example, the artificial intelligence model of the present invention may include at least one of a machine learning (ML) model, a deep learning model, a deep neural network (DNN), a language model (LM), a large language model (LLM), a super-large foundation model, a generative artificial intelligence (Generative AI) model, a transformer-based model, a supervised learning (SL) model, a reinforcement learning (RL) model, a special purpose model (e.g., a time series forecasting model (e.g., ARIMA model, SARIMA model, etc.), a time series foundation model, a graph neural network (GNN), a multimodal model, a natural language processing (NLP) model, a computer vision model, a speech recognition / synthesis model, a recommendation system model, etc.).
[0086] Furthermore, the artificial intelligence model used in the process defect prediction system according to the present invention may be implemented with a single or multiple components. More specifically, the artificial intelligence model mentioned in the present invention may be implemented with a single component (1) or with multiple components (2, 3, 4 or more, etc.).
[0087] In this regard, when multiple artificial intelligence models are implemented in the present invention, the multiple artificial intelligence models may be implemented as identical models (e.g., implemented as a first large-scale language model and a second large-scale language model). In this case, the multiple artificial intelligence models (150) may be implemented to perform different functions (or roles) in the present invention (e.g., the first large-scale language model performs a first function and the second large-scale language model performs a second function).
[0088] Alternatively, multiple artificial intelligence models may each be implemented as different models (for example, at least one artificial intelligence model is implemented as a machine learning model, and at least one other artificial intelligence model is implemented as a deep learning model). In this case, the multiple artificial intelligence models may be physically separated models (or modules) or multiple functional units within a single hardware. However, the present specification does not limit the implementation of artificial intelligence models to either a singular or multiple forms, and for convenience, it will be described as "at least one artificial intelligence model."
[0089] Meanwhile, as illustrated in FIG. 2, the process defect prediction system (100) according to the present invention can determine whether there is an abnormality (21) in a product (20, or product to be inspected, product to be manufactured) that is subject to inspection. More specifically, the process defect prediction system (100) can collect process data for a product (20) produced through a plurality of process steps and analyze the collected process data to determine (or determine, predict, detect, etc.) whether the product (20) is normal (21a) or defective (21b). For example, the type of product (20) subject to inspection in the present invention may be a display product. Here, “Disp.1” and “Disp.2” may represent a plurality (2) of display units (or screens), and this may represent a display arrangement in a display manufacturing process related to a vehicle HUD, wearable device, multi-screen device, etc. However, the type of product subject to inspection in the present invention is not necessarily limited to the examples mentioned, and it is obvious that various other products may be included.
[0090] The process defect prediction system (100) according to the present invention may include at least one of an input unit (110), an output unit (120), a communication unit (130), a storage unit (140), a data processing unit (150), a data set configuration unit (160), a prediction unit (170), and a control unit (180). However, the components of the process defect prediction system (100) according to the present invention are not limited thereto and may further include various hardware components that perform the same or similar roles as the functions described in the present specification.
[0091] A process defect prediction system (100) according to the present invention may include at least one processor and at least one memory containing computer program code. In this case, the memory may perform the role of the storage unit (140) above. In the present invention, the memory and the program code may cooperate with the processor to perform a series of processes described below.
[0092] Although not illustrated, the process defect prediction system (100) according to the present invention may include one or more processors, and such processors may include one or more general-purpose processors and / or one or more special-purpose processors (e.g., digital signal processor, tensor processing unit (TPU), graphics processing unit (GPU), neural network processing unit (NPU), application integrated circuit, application semiconductor (ASIC), field programmable gate array (FPGA), quantum processing unit (or quantum processor, QPU), etc.). One or more processors may be configured to execute instructions, computer-readable instructions, and / or other instructions described herein that are stored (or included) in the storage unit (140). The process defect prediction method and system according to the present invention may perform data processing described below in cooperation with memory and at least one processor. The processor may perform a series of operations and data processing using data and information stored in memory. In this case, memory may be a component of the storage unit (140).
[0093] In addition, the process defect prediction system (100) according to the present invention can perform data processing and calculation processes using quantum gates, quantum entanglement, and quantum superposition states, taking into consideration implementation in a quantum computer environment. For example, the present invention can perform parallel operations based on qubits, and such quantum operations can operate complementarily with existing classical computers.
[0094] Such quantum computers may include parallel computation using qubits and high-speed data processing devices utilizing quantum entanglement, and hardware-based computational optimization using FPGAs and ASICs is possible. In addition, quantum computers may utilize quantum processors capable of qubit-based parallel computation, and data processing efficiency can be improved through a hybrid structure with existing classical computers.
[0095] Meanwhile, the input unit (110) can be configured in various ways as a means of data input. For example, the input unit (110) can be configured to receive user input. The input unit (110) can be configured to receive user input from a user terminal (not shown). Here, “receiving input” may mean receiving an input signal (or selection signal) corresponding to the user’s input based on input made by the user through the configuration of the input unit provided in the user terminal.
[0096] Here, the user terminal may include at least one of a mobile phone, a smartphone, a notebook computer, a laptop computer, a slate PC, a tablet PC, an ultrabook, a desktop computer, a digital broadcasting terminal, a PDA (personal digital assistants), a PMP (portable multimedia player), a navigation device, and a wearable device (e.g., a smartwatch, a smart glass, a head-mounted display).
[0097] In addition, the input unit (110) in the present invention does not necessarily mean a hardware means, but can be understood as a channel for receiving input from a user.
[0098] The input unit (110) may also be referred to as a user interface module. The input unit (110) may include a touch screen, a computer mouse, a keyboard, a keypad, a touchpad, a trackball, a joystick, a voice recognition module, or other similar devices. However, the present invention does not limit the type of input unit (110).
[0099] Here, user input may include documents, text, images (or videos), voice, etc. In this case, the process defect prediction system (100) may further include a module that converts voice into text.
[0100] Next, the output unit (120) can output information through an output unit configuration (e.g., a display unit, a touch screen, a speaker, etc.) provided in a user terminal linked to the process defect prediction system (100) according to the present invention. For example, the output unit (120) can output a page (or service page) linked to the process defect prediction system (100) according to the present invention to the display unit of the user terminal. In addition, the output unit (120) does not necessarily mean a hardware means, but can be understood as a channel for outputting results to a user.
[0101] Next, the communication unit (130) may be connected via a wireless or wired network to a user terminal, a server (e.g., a central server, an external server, etc.), a device, and at least one network, etc., to receive or transmit overall data and information necessary for the operation of the process defect prediction system (100) according to the present invention.
[0102] The communication unit (130) can support various communication methods depending on the communication standard of the communicating device.
[0103] For example, the communication unit (130) may be configured to communicate with a communication target using at least one of the following technologies: WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), 5G (5th Generation Mobile Telecommunication), Bluetooth (Bluetooth™ RFID (Radio Frequency Identification), Infrared Communication (Infrared Data Association; IrDA), UWB (Ultra-Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus).
[0104] Next, the storage unit (140, or memory) serves to store various data related to the present invention and may include one or more non-transient computer-readable storage media that can be read and / or accessed by at least one of one or more processors.
[0105] One or more computer-readable storage media may include volatile and / or non-volatile storage components, such as optical, magnetic, organic, or other memory or disk storage devices. In some examples, the storage unit (140) may be implemented using a single physical device (e.g., one optical, magnetic, organic, or other memory or disk storage device), whereas in other examples, the storage unit (140) may be implemented using two or more physical devices.
[0106] The storage unit (140) may include computer-readable instructions and additional data. The storage unit (140) may include a storage necessary to perform at least some of the methods, scenarios, and techniques described herein and / or at least some of the functions of the device and network.
[0107] Furthermore, at least a portion of the storage unit (140) may be a cloud storage or a cloud server. At least a portion of the data and training data corresponding to user input received from the input unit (110) may be stored in the storage unit (140). Additionally, process data including data corresponding to at least one product produced through a plurality of process steps may be stored in the storage unit (140).
[0108] In one embodiment, as illustrated in FIG. 3a, process data (200) may be composed of a format such as table data (or table, table, tabular data, tabular data, etc.). Table data may be structured in a table form that includes row information representing information on various heterogeneous features of individual samples and column information representing information on one feature of multiple samples. More specifically, table data may be configured to include rows (or records, tuples, samples, etc.) and columns (or features, columns, fields, features, characteristics, attributes, variables, predictors, etc.). That is, table data refers to a table-shaped data structure composed of rows and columns, and may be a unit for storing and managing data in a relational database. Additionally, table data may be used to systematically organize and manage data, such as Excel sheets or database tables. Here, a column refers to a set of data arranged vertically in the table data, and each column can represent a specific attribute (or field) of the table data. Additionally, a row refers to a line of data arranged horizontally in the table data, and each row can represent a single record or tuple in the table data. In other words, each row represents a single record of the table data, and each column can represent a specific attribute of that record.
[0109] In this case, the process data (200) may be configured to include multiple different features (e.g., “workorder”, “Production Qty Collect Result”, “Collect Date_Dam”, “Collect Date_AtuoClave”, “Total_Time_Diff”, “target”) and multiple samples having values corresponding to (or for) each of the multiple features. The process data (200) may include data corresponding to at least one product produced through multiple process steps (or data corresponding to each of the multiple products). For example, the process data (200) may include at least one of the following: a process name (e.g., the name of each process step during product production (Dam, Fill, AutoClave, etc.), time information (e.g., the work Tact Time, Collect Time, etc. of each process), environmental variables (e.g., the coordinate value of each sensor, the speed during resin dispensing, the Temp of the Chamber, etc.), and defect status (e.g., in the target feature, Normal is displayed if normal, and Abnormal is displayed if defective). However, the format of the process data (200) in the present invention is not necessarily limited to the examples mentioned, and it is obvious that it can be implemented through various data formats other than those mentioned (e.g., time series data, event / log data, image data, video data, text data, numerical data, categorical data, handwritten data, etc.).
[0110] That is, the storage unit (140) is sufficient as a space where information necessary for the operation of the process defect prediction system (100) according to the present invention is stored, and it can be understood that there are no restrictions on the physical space.
[0111] Furthermore, the storage unit (140) may store a computer program including computer program instructions. Furthermore, the storage unit (140) may store a computer program including computer program instructions that control the operation of the system (100) or control the operation of the control unit (180) when loaded into the processor of the system (100).
[0112] Next, the data processing unit (150) may be configured to perform preprocessing on process data (200) collected during the production process of a product to be manufactured. For example, the data processing unit (150) may perform preprocessing on the process data (200) to perform at least one of data cleaning, processing of missing values, normalization, and transformation.
[0113] Additionally, the data processing unit (150) can perform the role of cleaning the process data (200) to process errors, missing values, and single values in the process data (200), and to detect (or identify) and remove outliers or duplicate records (or data). For example, the data processing unit (150) can remove features that interfere with calculations among the data (or features) included in the process data (200).
[0114] Additionally, the data processing unit (150) can perform the role of generating at least one derived variable (or derived feature, derived data, etc.) by using (or processing) the process data (200). This data processing expands the data that an artificial intelligence model can learn, and can also be understood as a feature engineering process. For example, the data processing unit (150) can analyze key characteristics (or features), such as volume change and pressure change, based on physical variables (e.g., volume, time, speed, temperature, pressure, etc.), and extract at least one (or at least some) important feature for each process step among the multiple features included in the process data (200). Then, the data processing unit (150) can generate a derived variable for each process step using the extracted at least one feature and the value corresponding to the feature.
[0115] Furthermore, the data processing unit (150) can perform the role of classifying each of the multiple samples included in the process data (200, or training data set (310)) according to a pre-set classification criterion. For example, the data processing unit (150) can classify each of the multiple samples into a first sample (e.g., Easy sample) or a second sample (e.g., Hard sample) according to a classification criterion set based on the difficulty level of each of the multiple samples. More specific details regarding this will be described later.
[0116] Next, the data set configuration unit (160) may be configured to perform the role of configuring the training data set (310) and the verification data set (320), respectively, using the process data (200).
[0117] In one embodiment, the data set configuration unit (160) may configure a validation data set (Validation Set, 320) such that the first sample and the second sample are evenly included in each fold during the training of the artificial intelligence model (or during the training process, e.g., cross-check or cross-validation). In the present invention, “configured to be evenly included” may mean that when dividing the data (distributing, splitting, etc.), the difficulty distribution is evenly included in each fold. That is, in the present invention, the first sample and the second sample classified according to difficulty are distributed evenly to each fold, and the artificial intelligence model (150) is not biased only toward data having a specific difficulty, thereby enabling the acquisition of stable and highly reliable evaluation results.
[0118] Here, a fold may refer to a piece (or block) that divides the data in cross-validation. For example, let us assume that there are N (e.g., 5) folds that equally contain a first sample and a second sample. In this case, the present invention may use the first fold (Fold 1) among the N (or K) folds for the validation of the artificial intelligence model, and the second to fifth folds (Fold 2 to Fold 5) for training. Subsequently, the present invention may use the second fold for validation and use the remaining folds (first fold, third fold, fourth fold, fifth fold, etc.), excluding the second fold, for training. That is, this process is repeated a number of times (e.g., 5 times) corresponding to the number of folds (e.g., 5) so that all folds are used for model validation. Finally, the performance of the model can be evaluated by averaging the performance results obtained from each fold. In the present invention, the term "fold" may also be named "data fold," "data fold set," "fold set," or "sub-data set," etc.
[0119] That is, the data set configuration unit (160) can configure the verification data set (320) such that N folds all have a data configuration (or distribution) of similar difficulty during the verification process of the artificial intelligence model (150).
[0120] Next, the prediction unit (170) may be configured to perform the role of predicting whether there is an abnormality in the product. To this end, the prediction unit (170) may include at least one machine learning model and at least one deep learning model. In this case, the machine learning model or deep learning model included in the prediction unit (170) may be implemented as a single or multiple component.
[0121] For example, as illustrated in FIG. 3b, the prediction unit (170) may include a plurality (or a plurality) of machine learning models (171a, 171b, 171c, 171d). The plurality of machine learning models (171a, 171b, 171c, 171d) may include at least one of a first machine learning model (171a), a second machine learning model (171b), a third machine learning model (171c), and a fourth machine learning model (171d). In order to utilize the importance of features and characteristics that exhibit superior performance in high-dimensional data, the machine learning models of the present invention may include at least one of an XGBoost model, a LightGBM model, and a CatBoost model. In this case, the XGBoost model and the CatBoost model are used to perform only classification tasks (e.g., class prediction), while the LightGBM model is used to perform classification tasks and regression tasks (e.g., continuous value prediction) simultaneously. That is, at least some of the multiple machine learning models (171a, 171b, 171c, 171d) may perform the role of predicting whether a product is abnormal (e.g., normal or defective) (i.e., classification), and the remaining models may be used to perform the role of predicting whether a product is abnormal and the role of predicting continuous values (e.g., estimating the probability that a product (or sample) is abnormal as a value between 0 and 1, or quantifying how far it is from normal, etc.) (i.e., regression).
[0122] As another example, the prediction unit (170) may include a plurality (or a plurality) of deep learning models (172a, 172b, 172c). The plurality of deep learning models (172a, 172b, 172c) may include at least one of a first deep learning model (171a), a second deep learning model (172b), and a third deep learning model (172c). In the present invention, a Transformer-based model capable of assigning weights to reflect a tendency to show high dependence on a specific feature (e.g., Autoclave) among a plurality of features of process data (200) may be used as a deep learning model. Additionally, in the present invention, a CLOF model utilizing Contrastive Loss is applied to the deep learning model to effectively learn the distribution of normal samples corresponding to normal products (normal data) and defective samples corresponding to defective products (defective data). Furthermore, the deep learning model of the present invention may further include a Deep Belief Network (DBN).
[0123] The prediction unit (170) can determine (or generate, extract, calculate, etc.) a final prediction value (e.g., “Prediction”, 175) regarding whether a product (a sample corresponding to the product) is abnormal through post-correction (or post-processing) using multiple deep learning models (172a, 172b, 172c) on the ensemble result between multiple machine learning models (171a, 171b, 171c, 171d) and said ensemble result (173). Here, an ensemble can be understood as a technique for determining a final prediction by combining (or averaging) the results (or probabilities) predicted independently by multiple artificial intelligence models. This prediction unit (170) can be used to determine the final prediction result (or final prediction value) by combining the result values (or prediction values) output from each of the multiple machine learning models (171a, 171b, 171c, 171d). That is, the multiple deep learning models (or trained deep learning models) can be configured to perform post-correction on the ensemble result (173).
[0124] In one embodiment, the prediction unit (170) may construct a first ensemble result (or first ensemble value) by weighting the prediction values of each of the plurality of machine learning models (171a, 171b, 171c, 171d) for a sample corresponding to at least one product included in the training data set (310) and / or verification data set (320) at a certain ratio. At this time, in the present invention, among the plurality of machine learning models (171a, 171b, 171c, 171d), one machine learning model (171d) that had less overfitting to a specific feature (e.g., AutoClave) compared to other machine learning models may be designated as a target for weight increase, and the weight for the designated machine learning model (171d) may be increased (or raised).
[0125] At this time, the prediction unit (170) can specify (or search, extract, etc.) an optimal prediction probability threshold value based on the target distribution (e.g., normal : abnormal = 16 : 1) of the training data set (310) of a plurality of deep learning models (172a, 172b, 172c). The specified threshold value may include a value that converts the lower K% of samples with a prediction probability value related to the number of normal samples to 0, and a value that converts the upper K% of samples with a prediction probability value related to the number of abnormal samples to 1. Based on this, the present invention may store common conversion target sample indices for the upper and lower parts of the plurality of deep learning models (172a, 172b, 172c) in the storage unit (140). The final ensemble result can be generated by a process of modifying (or post-correcting) the prediction values in the first ensemble result, which is obtained by weighted averaging the prediction values obtained from each of the multiple machine learning models (171a, 171b, 171c, 171d), with the upper transformation sample index set to 1 and the lower transformation sample index set to 0. In this way, the present invention maximizes the generalization performance of the model by fusing the heterogeneous characteristics of each model, such as machine learning models and deep learning models.
[0126] Next, the control unit (180) can perform the role of controlling the overall operation of the process defect prediction system (100) related to the present invention. The control unit (180) can process signals, data, information, etc. that are input or output through the components of the process defect prediction system (100) described above, or perform a series of data processing to provide or process appropriate information and functions to the user. The control unit (180) can be physically implemented by the processor described above.
[0127] Meanwhile, the present invention aims to provide a method and system for predicting process defects capable of determining whether a product is defective by analyzing process data collected during the product manufacturing process. More specifically, the present invention aims to determine whether a product is defective by constructing an artificial intelligence model applicable to various process environments. Below, we will examine in more detail the construction of an artificial intelligence model optimized for various process environments and the method for predicting process defects using the constructed artificial intelligence model.
[0128] In the present invention, process data including data corresponding to at least one product produced through a plurality of process steps is specified (S410), and a process of configuring a training data set and a verification data set, respectively, using the process data can be carried out (S420, see FIG. 4).
[0129] In the present invention, the method or criteria for specifying process data (200) may vary. The control unit (180) may specify process data (200) through various methods or criteria.
[0130] In one embodiment, the control unit (180) stores various data collected during the production process of a product to be manufactured in a storage unit (140, or memory), and can identify the data stored in the storage unit (140) as process data (200).
[0131] In another embodiment, the control unit (180) collects (or receives) a data set corresponding to at least one product produced through a plurality of process steps from a user terminal (10) or various sources (e.g., a database (DB), a server linked to the process defect prediction system (100), a central server, an external server, etc.), and can identify the collected data set as process data (200).
[0132] As described above, process data (200) may be configured to include multiple different features and multiple samples having values corresponding to each of the multiple features (see FIG. 3a). That is, process data (200) may be configured to include multiple samples having values for multiple different features. In this case, a sample represents at least one data unit, and the sample may contain data values of multiple features (or columns) (i.e., multiple fields or attributes, etc.). That is, a sample may mean that each data included in a data set or a data unit contains data values of multiple features (or categories). In a database, this may also be referred to as a “row.” For example, in an Excel sheet, each row represents a sample, and each column of the row may represent data values of various features within the sample.
[0133] The control unit (180) can perform preprocessing on process data (200) and generate at least one derived variable using (or processing) the process data (200). More specifically, the control unit (180) can perform preprocessing on process data (200) using the data processing unit (150) and generate at least one derived variable from the process data (200).
[0134] In one embodiment, the data processing unit (150) may perform preprocessing on the process data (200) to perform at least one of data cleaning, missing value processing, normalization, and transformation.
[0135] In another embodiment, the data processing unit (150) can perform the role of cleaning the process data (200) to process errors, missing values, and single values in the process data (200), and to detect (or identify) and remove outliers or duplicate records (or data). For example, the data processing unit (150) can remove features that interfere with calculations among the data (or features) included in the process data (200).
[0136] Additionally, the data processing unit (150) can generate derived variables by performing feature engineering on process data (i.e., preprocessed process data). More specifically, the data processing unit (150) can generate derived variables at each of the multiple process steps by performing feature engineering on process data (200) configured to include multiple samples having values for multiple different features (S501, see FIG. 5).
[0137] Here, generating derived variables at each of multiple process steps through feature engineering may mean generating new variables by combining data (e.g., time, pressure, temperature, speed, etc.) collected (or observed) at each of the multiple process steps.
[0138] The data processing unit (150) can generate a derived variable using at least some features corresponding to each of the multiple process steps among the multiple features included in the process data (200) and a value corresponding to said at least some features. More specifically, the data processing unit (150) can extract at least one (or at least some) important feature for each process step among the multiple features included in the process data (200), and generate a derived variable using the extracted feature and a value corresponding to said extracted feature. For example, the feature extracted from each of the multiple process steps may include a feature extracted by analyzing major characteristics (or features) such as volume change and pressure change based on physical variables (e.g., volume, time, speed, temperature, pressure, etc.). Here, the at least some features and the value corresponding to said at least some features may correspond to either “at least one feature and a value corresponding to that at least one feature” or “multiple features and a value corresponding to each of the multiple features.”
[0139] In one embodiment, as illustrated in FIG. 7(a), the data processing unit (150) can perform feature engineering on at least some of the features (e.g., features related to Volume, Time, Speed, and Thickness, 701) corresponding to a first process step (e.g., “Dam”) among the plurality of features included in the process data (200) and the values corresponding to the features (701) to generate at least one derived variable (e.g., “Thickness diff”, “Volume change per stage”, etc., 701a).
[0140] In another embodiment, as illustrated in FIG. 7(b), the data processing unit (150) can perform feature engineering on at least some of the features corresponding to the second process step (e.g., “Fill”) among the plurality of features included in the process data (200) (e.g., features related to Volume, Time, and Speed, 711) and values corresponding to the features (711) to generate at least one derived variable (e.g., “Volume Change per Hour”, “Hour times Speed”, etc., 711a).
[0141] In another embodiment, as illustrated in FIG. 7(c), the data processing unit (150) can perform feature engineering on at least some of the features corresponding to the third process step (e.g., “AutoClave”) among the plurality of features included in the process data (200) (e.g., features related to Volume, Time, Temperature, and Pressure, 721) and values corresponding to said features (721) to generate at least one derived variable (e.g., “Pressure change per hour”, “Volume times Pressure”, etc., 721a).
[0142] In this way, the data processing unit (150) can generate derived variables using the value of the original data (value corresponding to the original feature) included in the process data (200). In this case, the data processing unit (150) can generate derived variables from the process data (200) using various methods (e.g., physical formulas and data calculation methods, etc.). The process and meaning of generating derived variables from the data processing unit (150) are as follows (see Tables 1 and 2). For example, Tables 1 and 2 may show the physical formulas and data calculation methods used to generate derived variables, and the types of original data and access methods used to derive derived variables.
[0143] Attribute Name explanation official # Dam Cure_HD_Dam The value calculated as the difference between the CURE start position and end position (operational height) df_train['Cure_HD_Dam'] = df_train['CURE END POSITION Cure_time_Dam The value calculated by dividing the CURE operating height by the CURE speed to determine the operating time. df_train['Cure_time_Dam'] = df_train['Cure_HD_Dam'] / df_train['CURE SPEED Collect Result_Dam'] Cure_position_category_Dam Values classified as 'Up' or 'Down' based on the CURE termination position df_train['Cure_position_category_Dam'] = df_train['CURE END POSITION Theta_d_1_Dam Values generated based on θ (angle) and velocity u_c for CURE hysteresis calculation def f_H(x): return np.arccos(1 - 2 * np.tanh(5.16 * (x / (1 + 1.31 * x**0.99))**0.706))u_c = df_train['CURE SPEED Collect Result_Dam']theta_e_1 = df_train['CURE START POSITION ΘCollect Result_Dam']mu = 1gamma = 1df_train['Theta_d_1_Dam'] = f_H((mu * u_c / gamma) + f_H(theta_e_1)) Discharged_Resin_distance_1_Dam Distance traveled in Stage 1 while resin is being discharged df_train['Discharged_Resin_distance_1_Dam'] = df_train['DISCHARGED SPEED OF RESIN Collect Result_Dam'] * df_train['DISCHARGED TIME OF RESIN(Stage1) Collect Result_Dam'] Discharged_Resin_distance_2_Dam Distance traveled in Stage 2 while resin is being discharged df_train['Discharged_Resin_distance_2_Dam'] = df_train['DISCHARGED SPEED OF RESIN Collect Result_Dam'] * df_train['DISCHARGED TIME OF RESIN(Stage2) Collect Result_Dam'] Discharged_Resin_distance_3_Dam Distance traveled in Stage 3 while Resin is being discharged df_train['Discharged_Resin_distance_3_Dam'] = df_train['DISCHARGED SPEED OF RESIN Collect Result_Dam'] * df_train['DISCHARGED TIME OF RESIN(Stage3) Collect Result_Dam'] TotalTime_Discharged_Resin_Dam Total time for resin to be discharged over 3 stages df_train['TotalTime_Discharged_Resin_Dam'] = df_train['DISCHARGED TIME OF RESIN(Stage1) Collect Result_Dam'] + df_train['DISCHARGED TIME OF RESIN(Stage2) Collect Result_Dam'] + df_train['DISCHARGED TIME OF RESIN(Stage3) Collect Result_Dam'] Dispense_volume_change1_Dam Change in Dispenser Volume from Stage 1 to Stage 2 df_train['Dispense_volume_change1_Dam'] = df_train['Dispense Volume(Stage2) Collect Result_Dam'] - df_train['Dispense Volume(Stage1) Collect Result_Dam'] Dispense_volume_change2_Dam Change in Dispenser Volume from Stage 2 to Stage 3 df_train['Dispense_volume_change2_Dam'] = df_train['Dispense Volume(Stage3) Collect Result_Dam'] - df_train['Dispense Volume(Stage2) Collect Result_Dam'] Dispense_volume_change1_abs_Dam Absolute value of the change in Dispenser volume from Stage 1 to Stage 2 df_train['Dispense_volume_change1_abs_Dam'] = df_train['Dispense_volume_change1_Dam'].abs() Dispense_volume_change2_abs_Dam Absolute value of the change in Dispenser volume from stage 2 to stage 3 df_train['Dispense_volume_change2_abs_Dam'] = df_train['Dispense_volume_change2_Dam'].abs() Total_Resin_volume1_Dam Amount of resin applied in Step 1 df_train['Total_Resin_volume1_Dam'] = df_train['Discharged_Resin_distance_1_Dam'] * df_train['Dispense Volume(Stage1) Collect Result_Dam'] Total_Resin_volume2_Dam Amount of resin applied in step 2 df_train['Total_Resin_volume2_Dam'] = df_train['Discharged_Resin_distance_2_Dam'] * df_train['Dispense Volume(Stage2) Collect Result_Dam'] Total_Resin_volume3_Dam Amount of resin applied in step 3 df_train['Total_Resin_volume3_Dam'] = df_train['Discharged_Resin_distance_3_Dam'] * df_train['Dispense Volume(Stage3) Collect Result_Dam'] Stage1_Dam_Volume_Speed Stage 1 Resin discharge rate per hour df_train['Stage1_Dam_Volume_Speed'] = df_train['Dispense Volume(Stage1) Collect Result_Dam'] / df_train['DISCHARGED TIME OF RESIN(Stage1) Collect Result_Dam'] Stage2_Dam_Volume_Speed Stage 2 Resin discharge rate per hour df_train['Stage2_Dam_Volume_Speed'] = df_train['Dispense Volume(Stage2) Collect Result_Dam'] / df_train['DISCHARGED TIME OF RESIN(Stage2) Collect Result_Dam'] Stage3_Dam_Volume_Speed Stage 3 Resin discharge rate per hour df_train['Stage3_Dam_Volume_Speed'] = df_train['Dispense Volume(Stage3) Collect Result_Dam'] / df_train['DISCHARGED TIME OF RESIN(Stage3) Collect Result_Dam'] # Fill Discharged_Resin_distance_1_Fill1 Distance traveled in Stage 1 while resin is being discharged df_train['Discharged_Resin_distance_1_Fill1'] = df_train['DISCHARGED SPEED OF RESIN Collect Result_Fill1'] * df_train['DISCHARGED TIME OF RESIN(Stage1) Collect Result_Fill1'] Discharged_Resin_distance_2_Fill1 Distance traveled in Stage 2 while resin is being discharged df_train['Discharged_Resin_distance_2_Fill1'] = df_train['DISCHARGED SPEED OF RESIN Collect Result_Fill1'] * df_train['DISCHARGED TIME OF RESIN(Stage2) Collect Result_Fill1'] Discharged_Resin_distance_3_Fill1 Distance traveled in Stage 3 while Resin is being discharged df_train['Discharged_Resin_distance_3_Fill1'] = df_train['DISCHARGED SPEED OF RESIN Collect Result_Fill1'] * df_train['DISCHARGED TIME OF RESIN(Stage3) Collect Result_Fill1'] TotalTime_Discharged_Resin_Fill1 Total time for resin to be discharged over 3 stages df_train['TotalTime_Discharged_Resin_Fill1'] = df_train['DISCHARGED TIME OF RESIN(Stage1) Collect Result_Fill1'] + df_train['DISCHARGED TIME OF RESIN(Stage2) Collect Result_Fill1'] + df_train['DISCHARGED TIME OF RESIN(Stage3) Collect Result_Fill1'] Dispense_volume_change1_Fill1 Change in Dispenser Volume from Stage 1 to Stage 2 df_train['Dispense_volume_change1_Fill1'] = df_train['Dispense Volume(Stage2) Collect Result_Fill1'] - df_train['Dispense Volume(Stage1) Collect Result_Fill1'] Dispense_volume_change2_Fill1 Change in Dispenser Volume from Stage 2 to Stage 3 df_train['Dispense_volume_change2_Fill1'] = df_train['Dispense Volume(Stage3) Collect Result_Fill1'] - df_train['Dispense Volume(Stage2) Collect Result_Fill1'] Dispense_volume_change1_abs_Fill1 Absolute value of the change in Dispenser volume from Stage 1 to Stage 2 df_train['Dispense_volume_change1_abs_Fill1'] = df_train['Dispense_volume_change1_Fill1'].abs() Dispense_volume_change2_abs_Fill1 Absolute value of the change in Dispenser volume from stage 2 to stage 3 df_train['Dispense_volume_change2_abs_Fill1'] = df_train['Dispense_volume_change2_Fill1'].abs() Total_Resin_volume1_Fill1 Amount of resin applied in Step 1 df_train['Total_Resin_volume1_Fill1'] = df_train['Discharged_Resin_distance_1_Fill1'] * df_train['Dispense Volume(Stage1) Collect Result_Fill1'] Total_Resin_volume2_Fill1 Amount of resin applied in step 2 df_train['Total_Resin_volume2_Fill1'] = df_train['Discharged_Resin_distance_2_Fill1'] * df_train['Dispense Volume(Stage2) Collect Result_Fill1'] Total_Resin_volume3_Fill1 Amount of resin applied in step 3 df_train['Total_Resin_volume2_Fill1'] = df_train['Discharged_Resin_distance_2_Fill1'] * df_train['Dispense Volume(Stage2) Collect Result_Fill1'] Stage1_Fill1_Volume_Speed Stage 1 Resin discharge rate per hour df_train['Stage1_Fill1_Volume_Speed'] = df_train['Dispense Volume(Stage1) Collect Result_Fill1'] / df_train['DISCHARGED TIME OF RESIN(Stage1) Collect Result_Fill1'] Stage2_Fill1_Volume_Speed Stage 2 Resin discharge rate per hour df_train['Stage2_Fill1_Volume_Speed'] = df_train['Dispense Volume(Stage2) Collect Result_Fill1'] / df_train['DISCHARGED TIME OF RESIN(Stage2) Collect Result_Fill1'] Stage3_Fill1_Volume_Speed Stage 3 Resin discharge rate per hour df_train['Stage3_Fill1_Volume_Speed'] = df_train['Dispense Volume(Stage3) Collect Result_Fill1'] / df_train['DISCHARGED TIME OF RESIN(Stage3) Collect Result_Fill1'] Fill1_X_1 Difference in X coordinates between Stage 1 and 2 df_train['Fill1_X_1'] = df_train['HEAD NORMAL COORDINATE X AXIS(Stage1) Collect Result_Fill1'] - df_train['HEAD NORMAL COORDINATE X AXIS(Stage2) Collect Result_Fill1'] Fill1_X_2 Stage 1과 3의 X 좌표 차이 df_train['Fill1_X_2'] = df_train['HEAD NORMAL COORDINATE X AXIS(Stage1) Collect Result_Fill1'] - df_train['HEAD NORMAL COORDINATE X AXIS(Stage3) Collect Result_Fill1'] Fill1_X_3 Stage 2와 3의 X 좌표 차이 df_train['Fill1_X_3'] = df_train['HEAD NORMAL COORDINATE X AXIS(Stage2) Collect Result_Fill1'] - df_train['HEAD NORMAL COORDINATE X AXIS(Stage3) Collect Result_Fill1'] Fill1_Y_1 Stage 1과 2의 Y 좌표 차이 df_train['Fill1_Y_1'] = df_train['HEAD NORMAL COORDINATE Y AXIS(Stage1) Collect Result_Fill1'] - df_train['HEAD NORMAL COORDINATE Y AXIS(Stage2) Collect Result_Fill1'] Fill1_Y_2 Stage 1과 3의 Y 좌표 차이 df_train['Fill1_Y_2'] = df_train['HEAD NORMAL COORDINATE Y AXIS(Stage1) Collect Result_Fill1'] - df_train['HEAD NORMAL COORDINATE Y AXIS(Stage3) Collect Result_Fill1'] Fill1_Y_3 Stage 2와 3의 Y 좌표 차이 df_train['Fill1_Y_3'] = df_train['HEAD NORMAL COORDINATE Y AXIS(Stage2) Collect Result_Fill1'] - df_train['HEAD NORMAL COORDINATE Y AXIS(Stage3) Collect Result_Fill1'] Fill1_dist_1 XY coordinate distance between Step 1 and Step 2 df_train['Fill1_dist_1'] = np.sqrt(df_train['Fill1_X_1']**2 +df_train['Fill1_Y_1']**2) Fill1_dist_2 XY coordinate distance between Step 1 and Step 3 df_train['Fill1_dist_2'] = np.sqrt(df_train['Fill1_X_2']**2 +df_train['Fill1_Y_2']**2) Fill1_dist_3 XY coordinate distance between step 2 and step 3 df_train['Fill1_dist_3'] = np.sqrt(df_train['Fill1_X_3']**2 +df_train['Fill1_Y_3']**2) Vector_Fill1_Stage1 Vector size of Stage 1 df_train['Vector_Fill1_Stage1']= calculate_magnitude(df_train['HEAD NORMAL COORDINATE AXIS(Stage1) Collect Result_Fill1']) Vector_Fill1_Stage2 Vector size of Stage 2 df_train['Vector_Fill1_Stage2']= calculate_magnitude(df_train['HEAD NORMAL COORDINATE X AXIS(Stage2) Collect Result_Fill1'], df_train['HEAD NORMAL COORDINATE Y AXIS(Stage2) Collect Result_Fill1'], df_train['HEAD NORMAL COORDINATE Z AXIS(Stage2) Collect Result_Fill1']) Vector_Fill1_Stage3 Stage 3의 벡터 크기 df_train['Vector_Fill1_Stage3']= calculate_magnitude(df_train['HEAD NORMAL COORDINATE X AXIS(Stage3) Collect Result_Fill1'], df_train['HEAD NORMAL COORDINATE Y AXIS(Stage3) Collect Result_Fill1'], df_train['HEAD NORMAL COORDINATE Z AXIS(Stage3) Collect Result_Fill1']) Angle_x_Fill1_Stage1 Stage 1의 X축 각도 df_train['Angle_x_Fill1_Stage1'],df_train['Angle_y_Fill1_Stage1'],df_train['Angle_z_Fill1_Stage1'] = calculate_angles(df_train['HEAD NORMAL COORDINATE X AXIS(Stage1) Collect Result_Fill1'], df_train['HEAD NORMAL COORDINATE Y AXIS(Stage1) Collect Result_Fill1'], df_train['HEAD NORMAL COORDINATE Z AXIS(Stage1) Collect Result_Fill1']) Angle_x_Fill1_Stage2 Stage 2의 X축 각도 df_train['Angle_x_Fill1_Stage2'] ,df_train['Angle_y_Fill1_Stage2'],df_train['Angle_z_Fill1_Stage2'] = calculate_angles(df_train['HEAD NORMAL COORDINATE COORDINATE Y AXIS(Stage2) Collect Result_Fill1'], df_train['HEAD NORMAL COORDINATE Z AXIS(Stage2) Collect Result_Fill1']) Angle_x_Fill1_Stage3 X-axis angle of Stage 3 df_train['Angle_x_Fill1_Stage3'] ,df_train['Angle_y_Fill1_Stage3'],df_train['Angle_z_Fill1_Stage3'] = calculate_angles(df_train['HEAD NORMAL COORDINATE COORDINATE Y AXIS(Stage3) Collect Result_Fill1'], df_train['HEAD NORMAL COORDINATE Z AXIS(Stage3) Collect Result_Fill1']) # AutoClave AutoClave_1st_Pressure_str The value obtained by converting the primary pressure value to a string and then converting the last digit to a float. df_train['AutoClave_1st_Pressure_str'] = df_train['1st Pressure 1st Pressure Unit Time_AutoClave'].astype(str).str[-1].astype('float') AutoClave_2nd_Pressure_str The value obtained by converting the secondary pressure value to a string and then converting the last digit to a float. df_train['AutoClave_2nd_Pressure_str'] = df_train['2nd Pressure Unit Time_AutoClave'].astype(str).str[-1].astype('float') AutoClave_3rd_Pressure_str The value obtained by converting the 3rd pressure value to a string and then converting the last digit to a float. df_train['AutoClave_3rd_Pressure_str'] = df_train['3rd Pressure Unit Time_AutoClave'].astype(str).str[-1].astype('float') AutoClave_Pressure_str_Sum The sum of the last digits of the 1st, 2nd, and 3rd pressure values df_train['AutoClave_Pressure_str_Sum']= df_train['AutoClave_1st_Pressure_str'] + df_train['AutoClave_2nd_Pressure_str'] + df_train['AutoClave_3rd_Pressure_str'] Pressure_Time_slope1_AutoClave Slope of the pressure-time curve for primary pressure df_train['Pressure_Time_slope1_AutoClave'] = df_train['1st Pressure Collect Result_AutoClave'] / (df_train['1st Pressure 1st Pressure Unit Time_AutoClave'] + 1) Pressure_Time_slope2_AutoClave The slope of the pressure-time curve between the primary and secondary pressures df_train['Pressure_Time_slope2_AutoClave'] = (df_train['2nd Pressure Collect Result_AutoClave'] - df_train['1st Pressure Collect Result_AutoClave']) / (df_train['2nd Pressure Unit Time_AutoClave'] + 1) Pressure_Time_slope3_AutoClave The slope of the pressure-time curve between the secondary and tertiary pressures df_train['Pressure_Time_slope3_AutoClave'] = (df_train['3rd Pressure Collect Result_AutoClave'] - df_train['2nd Pressure Collect Result_AutoClave']) / (df_train['3rd Pressure Unit Time_AutoClave'] + 1) Total_Pressure_change_AutoClave Total pressure change of primary and tertiary pressure values df_train['Total_Pressure_change_AutoClave'] = df_train['3rd Pressure Collect Result_AutoClave'] - df_train['1st Pressure Collect Result_AutoClave'] Total_Pressure_AutoClave The sum of the 1st, 2nd, and 3rd pressure values df_train['Total_Pressure_AutoClave'] = df_train['1st Pressure Collect Result_AutoClave'] + df_train['2nd Pressure Collect Result_AutoClave'] + df_train['3rd Pressure Collect Result_AutoClave'] Average_Pressure_AutoClave Average of 1st, 2nd, and 3rd pressure values df_train['Average_Pressure_AutoClave'] = (df_train['1st Pressure Collect Result_AutoClave'] + df_train['2nd Pressure Collect Result_AutoClave'] + df_train['3rd Pressure Collect Result_AutoClave']) / 3 Total_pressure_time_AutoClave Total of 1st, 2nd, and 3rd pressure durations df_train['Total_pressure_time_AutoClave'] = df_train['1st Pressure 1st Pressure Unit Time_AutoClave'] + df_train['2nd Pressure Unit Time_AutoClave'] + df_train['3rd Pressure Unit Time_AutoClave'] Area_under_pressure_time_AutoClave Area under the pressure-time curve for the 1st, 2nd, and 3rd pressure values pressure_values = df_train[['1st Pressure Collect Result_AutoClave','2nd Pressure Collect Result_AutoClave','3rd Pressure Collect Result_AutoClave']]df_train['Area_under_pressure_time_AutoClave'] = np.trapz(pressure_values, axis = 1) Time_AutoClave_else The difference between total pressure time and chamber temperature time df_train['Time_AutoClave_else'] = df_train['Chamber Temp. Unit Time_AutoClave'] - df_train['Total_pressure_time_AutoClave']
[0144] 1. Workorder 과 Equipment 를 묶어서 그룹화df_train['Work_Equip'] = df_train['Equipment'] + df_train['Workorder']df_test['Work_Equip'] = df_test['Equipment'] + df_test['Workorder']2. Workorder 그룹별 Production Qty를 시간순으로 배치#Production Qty Transformationfor lst in tqdm(df_train['Workorder'].unique()): tr = df_train[df_train['Workorder'] == lst] te = df_test[df_test['Workorder'] == lst] tr['Production_Sequence_Ratio'] = tr['Production Qty Collect Result'] / tr['Production Qty Collect Result'].max() te['Production_Sequence_Ratio'] = te['Production Qty Collect Result'] / tr['Production Qty Collect Result'].max() tr['sin_sequence'] = np.sin(2 * np.pi * tr['Production Qty Collect Result'].astype(float) / 59.0) te['sin_sequence'] = np.sin(2 * np.pi * te['Production Qty Collect Result'].astype(float) / 59.0) tr['cos_sequence'] = np.cos(2 * np.pi * tr['Production Qty Collect Result'].astype(float) / 59.0) te['cos_sequence'] = np.cos(2 * np.pi * te['Production Qty Collect Result'].astype(float) / 59.0) df_train.loc[tr.index,'Production_Sequence_ratio'] = tr['Production_Sequence_Ratio'] df_test.loc[te.index,'Production_Sequence_ratio'] = te['Production_Sequence_Ratio'] df_train.loc[tr.index,'sin_sequence'] = tr['sin_sequence'] df_test.loc[te.index,'sin_sequence'] = te['sin_sequence'] df_train.loc[tr.index,'cos_sequence'] = tr['cos_sequence'] df_test.loc[te.index,'cos_sequence'] = te['cos_sequence']3. Workorder 안의 공정이름 식별을 위한 소분화#문자열 정보 추출df_train['Wo_Number'] = df_train['Workorder'].apply(lambda x : x[:1]).astype(str)df_train['number_alpha'] = df_train['Workorder'].apply(lambda x : x[:2]).astype(str)df_train['alpha'] = df_train['Workorder'].apply(lambda x : x[1]).astype(str)df_train['Wo_Main'] = df_train['Workorder'].apply(lambda x : x[:3]).astype(str)df_train['Wo_Sub'] = df_train['Workorder'].apply(lambda x : x[3]).astype(str)df_train['Wo_Detail_1'] = df_train['Workorder'].apply(lambda x : x[4]).astype(str)df_train['Wo_Detail_2'] = df_train['Workorder'].apply(lambda x : x[5:]).astype(str)df_train['last'] = df_train['Workorder'].apply(lambda x : x[-1]).astype(str)df_train['sj_1'] = df_train['Workorder'].apply(lambda x : x[:4]).astype(str)df_train['sj_5'] = df_train['Workorder'].apply(lambda x : x[:8]).astype(str)df_train['sj_6'] = df_train['Workorder'].apply(lambda x : x[2]).astype(str)df_train['sj_7'] = df_train['Workorder'].apply(lambda x : x[5]).astype(str)df_train['sj_8'] = df_train['Workorder'].apply(lambda x : x[4:8]).astype(str)df_train['sj_9'] = df_train['Workorder'].apply(lambda x : x[-5:-2]).astype(str)df_test['Wo_Number'] = df_test['Workorder'].apply(lambda x : x[:1]).astype(str)df_test['number_alpha'] = df_test['Workorder'].apply(lambda x : x[:2]).astype(str)df_test['alpha'] = df_test['Workorder'].apply(lambda x : x[1]).astype(str)df_test['Wo_Main'] = df_test['Workorder'].apply(lambda x : x[:3]).astype(str)df_test['Wo_Sub'] = df_test['Workorder'].apply(lambda x : x[3]).astype(str)df_test['Wo_Detail_1'] = df_test['Workorder'].apply(lambda x : x[4]).astype(str)df_test['Wo_Detail_2'] = df_test['Workorder'].apply(lambda x : x[5:]).astype(str)df_test['last'] = df_test['Workorder'].apply(lambda x : x[-1]).astype(str)df_test['sj_1'] = df_test['Workorder'].apply(lambda x : x[:4]).astype(str)df_test['sj_5'] = df_test['Workorder'].apply(lambda x : x[:8]).astype(str)df_test['sj_6'] = df_test['Workorder'].apply(lambda x : x[2]).astype(str)df_test['sj_7'] = df_test['Workorder'].apply(lambda x : x[5]).astype(str)df_test['sj_8'] = df_test['Workorder'].apply(lambda x : x[4:8]).astype(str)df_test['sj_9'] = df_test['Workorder'].apply(lambda x : x[-5:-2]).astype(str)4. 절대 온도 활용 Featuredf_train['K_TEMP_AutoClave'] = df_train['Chamber Temp. Collect Result_AutoClave'] + 273df_test['K_TEMP_AutoClave'] = df_test['Chamber Temp. Collect Result_AutoClave'] + 2735. 절대 온도 변환후 PV=NRT 를 활용한 featuredf_train['Auto_vol_1_AutoClave'] = (df_train['Chamber Temp.Collect Result_AutoClave'] + 273) / df_train['1st Pressure Collect Result_AutoClave']df_train['Auto_vol_2_AutoClave'] = (df_train['Chamber Temp. Collect Result_AutoClave'] + 273) / df_train['2nd Pressure Collect Result_AutoClave']df_train['Auto_vol_3_AutoClave'] = (df_train['Chamber Temp. Collect Result_AutoClave'] + 273) / df_train['3rd Pressure Collect Result_AutoClave']df_test['Auto_vol_1_AutoClave'] = (df_test['Chamber Temp. Collect Result_AutoClave'] + 273) / df_test['1st Pressure Collect Result_AutoClave']df_test['Auto_vol_2_AutoClave'] = (df_test['Chamber Temp. Collect Result_AutoClave'] + 273) / df_test['2nd Pressure Collect Result_AutoClave']df_test['Auto_vol_3_AutoClave'] = (df_test['Chamber Temp. Collect Result_AutoClave'] + 273) / df_test['3rd Pressure Collect Result_AutoClave']6. 반응 속도의 온도 의존도를 나타낸 아레니우스 방정식을 활용한 featuredf_train['Arrhenius_AutoClave'] = np.exp(1 / (df_train['Chamber Temp. Collect Result_AutoClave'] + 273) - 1 / 373.15)df_test['Arrhenius_AutoClave'] = np.exp(1 / (df_test['Chamber Temp. Collect Result_AutoClave'] + 273) - 1 / 373.15) 7. Dispense Volume per Unit Time (Flow Rate) df_train['Q_1'] = df_train['Dispense Volume(Stage1) Collect Result_Dam'] / df_train['DISCHARGED TIME OF RESIN(Stage1) Collect Result_Dam'] 8. Flow Rate * Time (Total Flow Rate) df_train['np_1'] = df_train['Q_1'] * df_train['1st Pressure 1st Pressure Unit Time_AutoClave'] 9. Thickness / Flow Rate df_train['Thickness_Flow Rate_1'] = df_train['THICKNESS 1 Collect Result_Dam'] / df_train['Q_1'] 10. Borrowed from Pv=nrt -> Volume * Pressure df_train['pv_2'] = df_train['volume_sum_stage_2'] * df_train['2nd Pressure Collect Result_AutoClave'] 11. Motion efficiency (relative velocity) on straight and circular paths df_train['speed_1'] = df_train['Stage1 Line1 Distance Speed Collect Result_Dam'] / df_train['Stage1 Circle1 Distance Speed Collect Result_Dam'] 12. Other time features#Time Featuresdf_train['Collect Date_Dam'] = pd.to_datetime(df_train['Collect Date_Dam'])df_train['Collect Date_Fill1'] = pd.to_datetime(df_train['Collect Date_Fill1'])df_train['Collect Date_Fill2'] = pd.to_datetime(df_train['Collect Date_Fill2'])df_train['Collect Date_AutoClave'] = pd.to_datetime(df_train['Collect Date_AutoClave'])##Time diffdf_train['Dam_Fill1_Time_Diff'] = (df_train['Collect Date_Fill1'] - df_train['Collect Date_Dam']).dt.total_seconds()df_train['Fill1_Fill2_Time_Diff'] = (df_train['Collect Date_Fill2'] - df_train['Collect Date_Fill1']).dt.total_seconds()df_train['Fill2_Auto_Time_Diff'] = (df_train['Collect Date_AutoClave'] - df_train['Collect Date_Fill2']).dt.total_seconds()df_train['Total_Time_Diff'] = (df_train['Collect Date_AutoClave'] - df_train['Collect Date_Dam']).dt.total_seconds()##Fill1&Fill2 Matchdf_train['Fill1_Fill2_match'] = (df_train['Collect Date_Fill1'] == df_train['Collect Date_Fill2']).astype(int)##Time Transformdf_train['year_Dam'] = df_train['Collect Date_Dam'].dt.yeardf_train['month_Dam'] = df_train['Collect Date_Dam'].dt.monthdf_train['day_Dam'] = df_train['Collect Date_Dam'].dt.daydf_train['weekday_Dam'] = df_train['Collect Date_Dam'].dt.weekdaydf_train['hour_Dam'] = df_train['Collect Date_Dam'].dt.hourdf_train['minute_Dam'] = df_train['Collect Date_Dam'].dt.minutedf_train['second_Dam'] = df_train['Collect Date_Dam'].dt.seconddf_train['year_Fill1'] = df_train['Collect Date_Fill1'].dt.yeardf_train['month_Fill1'] = df_train['Collect Date_Fill1'].dt.monthdf_train['day_Fill1'] = df_train['Collect Date_Fill1'].dt.daydf_train['weekday_Fill1'] = df_train['Collect Date_Fill1'].dt.weekdaydf_train['hour_Fill1'] = df_train['Collect Date_Fill1'].dt.hourdf_train['minute_Fill1'] = df_train['Collect Date_Fill1'].dt.minutedf_train['second_Fill1'] = df_train['Collect Date_Fill1'].dt.seconddf_train['year_Fill2'] = df_train['Collect Date_Fill2'].dt.yeardf_train['month_Fill2'] = df_train['Collect Date_Fill2'].dt.monthdf_train['day_Fill2'] = df_train['Collect Date_Fill2'].dt.daydf_train['weekday_Fill2'] = df_train['Collect Date_Fill2'].dt.weekdaydf_train['hour_Fill2'] = df_train['Collect Date_Fill2'].dt.hourdf_train['minute_Fill2'] = df_train['Collect Date_Fill2'].dt.minutedf_train['second_Fill2'] = df_train['Collect Date_Fill2'].dt.seconddf_train['sin_hour_Dam'] = np.sin(2 * np.pi * df_train['hour_Dam'] / 23.0)df_train['cos_hour_Dam'] = np.cos(2 * np.pi * df_train['hour_Dam'] / 23.0)df_train['sin_minute_Dam'] = np.sin(2 * np.pi * df_train['minute_Dam'] / 59.0)df_train['cos_minute_Dam'] = np.cos(2 * np.pi * df_train['minute_Dam'] / 59.0)df_train['sin_date_Dam'] = -np.sin(2 * np.pi * (df_train['month_Dam']+df_train['day_Dam'] / 31) / 12)df_train['cos_date_Dam'] = -np.sin(2 * np.pi * (df_train['month_Dam']+df_train['day_Dam'] / 31) / 12)df_train['sin_month_Dam'] = -np.sin(2 * np.pi * df_train['month_Dam'] / 12.0)df_train['cos_month_Dam'] = -np.cos(2 * np.pi * df_train['month_Dam'] / 12.0)##Cycling Transformdf_train['sin_hour_Fill1'] = np.sin(2 * np.pi * df_train['hour_Fill1'] / 23.0)df_train['cos_hour_Fill1'] = np.cos(2 * np.pi * df_train['hour_Fill1'] / 23.0)df_train['sin_minute_Fill1'] = np.sin(2 * np.pi * df_train['minute_Fill1'] / 59.0)df_train['cos_minute_Fill1'] = np.cos(2 * np.pi * df_train['minute_Fill1'] / 59.0)df_train['sin_date_Fill1'] = -np.sin(2 * np.pi * (df_train['month_Fill1']+df_train['day_Fill1'] / 31) / 12)df_train['cos_date_Fill1'] = -np.sin(2 * np.pi * (df_train['month_Fill1']+df_train['day_Fill1'] / 31) / 12)df_train['sin_month_Fill1'] = -np.sin(2 * np.pi * df_train['month_Fill1'] / 12.0)df_train['cos_month_Fill1'] = -np.cos(2 * np.pi * df_train['month_Fill1'] / 12.0)df_train['sin_hour_Fill2'] = np.sin(2 * np.pi * df_train['hour_Fill2'] / 23.0)df_train['cos_hour_Fill2'] = np.cos(2 * np.pi * df_train['hour_Fill2'] / 23.0)df_train['sin_minute_Fill2'] = np.sin(2 * np.pi * df_train['minute_Fill2'] / 59.0)df_train['cos_minute_Fill2'] = np.cos(2 * np.pi * df_train['minute_Fill2'] / 59.0)df_train['sin_date_Fill2'] = -np.sin(2 * np.pi * (df_train['month_Fill2']+df_train['day_Fill2'] / 31) / 12)df_train['cos_date_Fill2'] = -np.sin(2 * np.pi * (df_train['month_Fill2']+df_train['day_Fill2'] / 31) / 12)df_train['sin_month_Fill2'] = -np.sin(2 * np.pi * df_train['month_Fill2'] / 12.0)df_train['cos_month_Fill2'] = -np.cos(2 * np.pi * df_train['month_Fill2'] / 12.0)#Time Featuresdf_test['Collect Date_Dam'] = pd.to_datetime(df_test['Collect Date_Dam'])df_test['Collect Date_Fill1'] = pd.to_datetime(df_test['Collect Date_Fill1'])df_test['Collect Date_Fill2'] = pd.to_datetime(df_test['Collect Date_Fill2'])df_test['Collect Date_AutoClave'] = pd.to_datetime(df_test['Collect Date_AutoClave'])##Time diffdf_test['Dam_Fill1_Time_Diff'] = (df_test['Collect Date_Fill1'] - df_test['Collect Date_Dam']).dt.total_seconds()df_test['Fill1_Fill2_Time_Diff'] = (df_test['Collect Date_Fill2'] - df_test['Collect Date_Fill1']).dt.total_seconds()df_test['Fill2_Auto_Time_Diff'] = (df_test['Collect Date_AutoClave'] - df_test['Collect Date_Fill2']).dt.total_seconds()df_test['Total_Time_Diff'] = (df_test['Collect Date_AutoClave'] - df_test['Collect Date_Dam']).dt.total_seconds()##Fill1&Fill2 Matchdf_test['Fill1_Fill2_match'] = (df_test['Collect Date_Fill1'] == df_test['Collect Date_Fill2']).astype(int)##Time Transformdf_test['year_Dam'] = df_test['Collect Date_Dam'].dt.yeardf_test['month_Dam'] = df_test['Collect Date_Dam'].dt.monthdf_test['day_Dam'] = df_test['Collect Date_Dam'].dt.daydf_test['weekday_Dam'] = df_test['Collect Date_Dam'].dt.weekdaydf_test['hour_Dam'] = df_test['Collect Date_Dam'].dt.hourdf_test['minute_Dam'] = df_test['Collect Date_Dam'].dt.minutedf_test['second_Dam'] = df_test['Collect Date_Dam'].dt.seconddf_test['year_Fill1'] = df_test['Collect Date_Fill1'].dt.yeardf_test['month_Fill1'] = df_test['Collect Date_Fill1'].dt.monthdf_test['day_Fill1'] = df_test['Collect Date_Fill1'].dt.daydf_test['weekday_Fill1'] = df_test['Collect Date_Fill1'].dt.weekdaydf_test['hour_Fill1'] = df_test['Collect Date_Fill1'].dt.hourdf_test['minute_Fill1'] = df_test['Collect Date_Fill1'].dt.minutedf_test['second_Fill1'] = df_test['Collect Date_Fill1'].dt.seconddf_test['year_Fill2'] = df_test['Collect Date_Fill2'].dt.yeardf_test['month_Fill2'] = df_test['Collect Date_Fill2'].dt.monthdf_test['day_Fill2'] = df_test['Collect Date_Fill2'].dt.daydf_test['weekday_Fill2'] = df_test['Collect Date_Fill2'].dt.weekdaydf_test['hour_Fill2'] = df_test['Collect Date_Fill2'].dt.hourdf_test['minute_Fill2'] = df_test['Collect Date_Fill2'].dt.minutedf_test['second_Fill2'] = df_test['Collect Date_Fill2'].dt.seconddf_test['sin_hour_Dam'] = np.sin(2 * np.pi * df_test['hour_Dam'] / 23.0)df_test['cos_hour_Dam'] = np.cos(2 * np.pi * df_test['hour_Dam'] / 23.0)df_test['sin_minute_Dam'] = np.sin(2 * np.pi * df_test['minute_Dam'] / 59.0)df_test['cos_minute_Dam'] = np.cos(2 * np.pi * df_test['minute_Dam'] / 59.0)df_test['sin_date_Dam'] = -np.sin(2 * np.pi * (df_test['month_Dam']+df_test['day_Dam'] / 31) / 12)df_test['cos_date_Dam'] = -np.sin(2 * np.pi * (df_test['month_Dam']+df_test['day_Dam'] / 31) / 12)df_test['sin_month_Dam'] = -np.sin(2 * np.pi * df_test['month_Dam'] / 12.0)df_test['cos_month_Dam'] = -np.cos(2 * np.pi * df_test['month_Dam'] / 12.0)##Cycling Transformdf_test['sin_hour_Fill1'] = np.sin(2 * np.pi * df_test['hour_Fill1'] / 23.0)df_test['cos_hour_Fill1'] = np.cos(2 * np.pi * df_test['hour_Fill1'] / 23.0)df_test['sin_minute_Fill1'] = np.sin(2 * np.pi * df_test['minute_Fill1'] / 59.0)df_test['cos_minute_Fill1'] = np.cos(2 * np.pi * df_test['minute_Fill1'] / 59.0)df_test['sin_date_Fill1'] = -np.sin(2 * np.pi * (df_test['month_Fill1']+df_test['day_Fill1'] / 31) / 12)df_test['cos_date_Fill1'] = -np.sin(2 * np.pi * (df_test['month_Fill1']+df_test['day_Fill1'] / 31) / 12)df_test['sin_month_Fill1'] = -np.sin(2 * np.pi * df_test['month_Fill1'] / 12.0)df_test['cos_month_Fill1'] = -np.cos(2 * np.pi * df_test['month_Fill1'] / 12.0)df_test['sin_hour_Fill2'] = np.sin(2 * np.pi * df_test['hour_Fill2'] / 23.0)df_test['cos_hour_Fill2'] = np.cos(2 * np.pi * df_test['hour_Fill2'] / 23.0)df_test['sin_minute_Fill2'] = np.sin(2 * np.pi * df_test['minute_Fill2'] / 59.0)df_test['cos_minute_Fill2'] = np.cos(2 * np.pi * df_test['minute_Fill2'] / 59.0)df_test['sin_date_Fill2'] = -np.sin(2 * np.pi * (df_test['month_Fill2']+df_test['day_Fill2'] / 31) / 12)df_test['cos_date_Fill2'] = -np.sin(2 * np.pi * (df_test['month_Fill2']+df_test['day_Fill2'] / 31) / 12)df_test['sin_month_Fill2'] = -np.sin(2 * np.pi * df_test['month_Fill2'] / 12.0)df_test['cos_month_Fill2'] = -np.cos(2 * np.pi * df_test['month_Fill2'] / 12.0).
[0145] In this case, the training data set (310) and the validation data set (320), each configured through the process described below, may further include the derived variables discussed above and the values corresponding to the derived variables. That is, in the present invention, the performance (ability) of a model determining normal or defective status can be improved by generating derived variables through feature engineering to reduce dependency on specific features (e.g., AutoClave).
[0146] Meanwhile, the data processing unit (150) can configure the training data set (310) and the verification data set (320) respectively using the process data (200).
[0147] At this time, the data processing unit (150) may configure a verification data set (320) such that different multiple samples are equally included in each fold during the training of the artificial intelligence model (or during the training process (e.g., cross-check or cross-validation)). Here, the different multiple samples may include at least one of a first sample and a second sample, and the first sample and the second sample may be understood as samples classified based on different difficulty levels. That is, the first sample and the second sample may be understood as samples having different difficulty levels (or corresponding to, or corresponding to, etc.).
[0148] In this regard, the data processing unit (150) can classify a plurality of samples according to a pre-set classification criterion.
[0149] First, the data processing unit (150) can separate data by label. For example, the label may include at least one of a label assigned to a normal sample corresponding to a normal product (e.g., Normal) and a label assigned to a defective (or abnormal) sample corresponding to a defective product (e.g., AbNormal).
[0150] Specifically, the data processing unit (150) can classify (or divide) multiple samples into a first group (or normal group) or a second group (or defective group, abnormal group, etc.) based on a value corresponding to a specific feature among multiple features (S503, see FIG. 5). Here, the specific feature may include a feature (e.g., target) indicating whether the produced product is normal or defective. The value corresponding to the specific feature may be configured to have a first value (e.g., 0 (Normal)) or a second value (e.g., 1 (AbNormal)) depending on whether the produced product is normal or defective.
[0151] For example, the data processing unit (150) may classify a sample (or a first sample, a normal sample, etc.) that includes a first value for a specific feature among a plurality of samples as a sample to be included in a first group, and may include the classified sample that includes the first value in the first group. That is, the first group may include at least one sample that includes a first value for a specific feature among a plurality of samples.
[0152] As another example, the data processing unit (150) may classify a sample (or a second sample, a defective sample, etc.) that includes a second value for a specific feature among a plurality of samples as a sample to be included in a second group, and may include the classified sample that includes the second value in the second group. That is, the second group may include at least one sample that includes a second value for a specific feature among a plurality of samples.
[0153] Subsequently, the data processing unit (150) can calculate the covariance matrix and inverse matrices. The covariance matrix can be calculated for each of the first group and the second group. At this time, small normalization values are added to the diagonal so that the matrix can be inverted, which is necessary because small changes in the data or noise can lead to a singular matrix.
[0154] That is, the data processing unit (150) can calculate a first covariance matrix for the first group and a second covariance matrix for the second group, respectively (S505, see FIG. 5).
[0155] Next, the data processing unit (150) can calculate a first average vector for the first group and a second average vector for the second group, respectively (S507, see FIG. 5).
[0156] For example, the data processing unit (150) can calculate a first average vector for all (or all) samples having a first value for a specific feature included in the first group and a second average vector for all samples having a second value for a specific feature included in the second group, respectively. These average vectors can represent the center points of the distribution of normal samples and the distribution of defective samples, respectively, and can serve as reference points for distance calculation.
[0157] Furthermore, the data processing unit (150) can calculate the distance between the first mean vector and the second mean vector for each of the plurality of samples (S509, see FIG. 5). Here, the calculated distance may be the Mahalanobis distance. Here, the Mahalanobis distance has the scale-invariant characteristic of considering the covariance matrix, which means that it is not important whether the variable is measured in different units.
[0158] More specifically, Mahalanobis distance is a statistical measure used to determine the distance between a point and a distribution. Since Mahalanobis distance considers the correlation between variables, it can be effective in distinguishing between normal and abnormal samples within datasets containing multivariate data. Measuring Mahalanobis distance can be usefully applied to identify outliers because the distance can be calculated relative to a reference distribution. In other words, Mahalanobis distance
[0159] The formula for calculating this Mahalinobis distance can be represented as shown in (a) of Fig. 11.
[0160] In this regard, the data processing unit (150) can classify samples into different difficulty levels (e.g., easy or difficult) depending on how close the samples are to the distribution of normal and abnormal data by utilizing the Mahalanobis distance. In the present invention, the Mahalanobis distance from each sample included in the data set (i.e., process data) to the mean of the normal samples and the mean of the abnormal samples can be calculated.
[0161] More specifically, the data processing unit (150) can calculate, for each of the plurality of samples, a first Mahalanobis distance based on a first mean vector calculated from a first group and a second Mahalanobis distance based on a second mean vector calculated from a second group. That is, two distance values (a first Mahalanobis distance and a second Mahalanobis distance) can be calculated for each of the plurality of samples. At this time, the Mahalanobis distance can be calculated by applying the inverse matrix of the covariance matrix of each of the first group and the second group. This can also be understood as being performed using the inverse covariance matrix of a normal distribution (or normal distribution) and an abnormal distribution to integrate the variance-covariance structure.
[0162] Finally, the data processing unit (150) can classify a plurality of samples based on a pre-set classification criterion and a calculated Mahalanobis distance (S511, see FIG. 5). In the present invention, the pre-set classification criterion can be set based on the difference in Mahalanobis distance between the normal distribution and the abnormal distribution (or defective distribution) of each of the plurality of samples. That is, the pre-set classification criterion can be set based on the difference in Mahalanobis distance between the normal distribution and the abnormal distribution of each sample.
[0163] Here, the normal distribution can be defined based on the first covariance matrix and the first mean vector calculated for the first group. Additionally, the non-normal distribution can be defined based on the second covariance matrix and the second mean vector calculated for the second group.
[0164] In one embodiment, the data processing unit (150) may classify at least some of the samples among a plurality of samples as the first samples, wherein the difference between the Mahalanobis distance to the normal distribution (first Mahalanobis distance) and the Mahalanobis distance to the abnormal distribution (second Mahalanobis distance) satisfies the first classification criterion among the pre-set classification criteria. For example, the first classification criterion may include cases where the absolute difference between the distance between the normal distribution and the abnormal distribution is greater than a pre-set threshold (e.g., 0.3). That is, the data processing unit (150) may classify at least some of the samples where the difference between the Mahalanobis distance between the normal distribution and the abnormal distribution is greater than the pre-set threshold as the first samples (easy samples). Since a larger difference means that the sample is closer to one distribution, it may be easier to classify it as normal or defective.
[0165] In another embodiment, the data processing unit (150) may classify at least some of the samples among a plurality of samples as second samples, wherein the difference between the Mahalanobis distance to a normal distribution (first Mahalanobis distance) and the Mahalanobis distance to an abnormal distribution (second Mahalanobis distance) satisfies the second classification criterion among the pre-set classification criteria. For example, the second classification criterion may include cases where the absolute difference between the distances between the normal distribution and the abnormal distribution is less than a pre-set threshold. That is, the data processing unit (150) may classify at least some of the samples in which the difference between the Mahalanobis distances between the normal distribution and the abnormal distribution is less than a pre-set threshold as second samples (hard samples). This small difference may mean that it is difficult to classify the sample as normal or bad because it is ambiguous since the sample is close to both distributions.
[0166] Thus, the present invention utilizes the Mahalanobis distance to efficiently classify multiple samples according to difficulty, thereby preventing biased learning from one specific class toward another and supporting balanced learning of the model. As an example, through the final result after adding features (e.g., “easy_hard”) associated with the first and second samples, these sample types can be distinguished in subsequent analysis or model training.
[0167] Meanwhile, the control unit (180) can configure a verification data set (320) using the data set configuration unit (160). The data set configuration unit (160) can equally include multiple samples having different difficulty levels in each of the verification data set (320). As seen above, the multiple samples may include a first sample and a second sample classified according to a pre-set classification criterion.
[0168] In one embodiment, as illustrated in FIG. 8, the data set configuration unit (160) may include at least a portion of each of the first sample (801) and the second sample (802) in the verification data set (320) so that the configuration ratio of each of the first sample (801) and the second sample (802) satisfies a preset configuration ratio standard (e.g., “3 : 2 - first sample : second sample”).
[0169] In this case, the data set configuration unit (160) can configure the verification data set (320) based on the first index corresponding to the first sample (801) and the second index corresponding to the second sample (802), such that the configuration ratio of each of the first sample (801) and the second sample (802) satisfies a preset configuration ratio standard (S513, see FIG. 5).
[0170] At this time, the data set configuration unit (160) can configure the verification data set (320) so that the first sample (801) and the second sample (802) are evenly included in each fold during the training of the artificial intelligence model (or during the training process (e.g., cross-check or cross-validation)).
[0171] Here, a fold may refer to a piece (or block) that divides the data in cross-validation. For example, let us assume that there are N (e.g., 5) folds that equally contain a first sample (801) and a second sample (802). In this case, the present invention may use the first fold (Fold 1) among the N (or K) folds for the validation of the artificial intelligence model, and the second to fifth folds (Fold 2 to Fold 5) for training. Subsequently, the present invention may use the second fold for validation, and the remaining folds excluding the second fold (first fold, third fold, fourth fold, fifth fold, etc.) may be used for training. That is, this process is repeated a number of times (e.g., 5 times) corresponding to the number of folds (e.g., 5) so that all folds are used for model validation. That is, in the present invention, based on the entire data (process data), the data is divided into a predetermined number of folds (e.g., 5), and at each cross-validation step (or at each split or epoch), one fold is used as the validation data set (320) and the remainder as the training data set (310) to train and validate at least one artificial intelligence model. As an example, in the present invention, after repeating training and validation a total of N times, the results of all splits are finally averaged, or the performance of the model is evaluated based on the best split result.
[0172] In this case, the training data set (310) may include at least some of the multiple (N) folds that are not being used for training the artificial intelligence model. Additionally, the verification data set (320) may include at least some of the folds that are not being used for training the artificial intelligence model. As another example, the training data set (310) may also be configured equally through the same process as the configuration method of the verification data set (320). However, the present invention does not limit the configuration method of the training data set (310) to any one.
[0173] Meanwhile, in the present invention, in order for at least one artificial intelligence model to be trained to determine whether there is an abnormality in a product, a process may be performed to train at least one artificial intelligence model using a training data set (S430), input a verification data set to at least one artificial intelligence model trained with the training data set (S440), obtain a prediction value for the verification data set from at least one trained artificial intelligence model (S450), and evaluate the abnormality detection performance (or abnormality detection classification performance, abnormality determination performance, defect detection performance, etc.) of at least one trained artificial intelligence model based on the prediction value (S460, see FIG. 4).
[0174] As seen above, the data set configuration unit (160) can configure the verification data set (320) such that when training the artificial intelligence model, the number (or ratio) of each of the first sample (801) and the second sample (802) in each fold is included in a number (or ratio) that satisfies a preset ratio standard.
[0175] At this time, at least some of the N folds included in the verification data set (320) may be used as the training data set (310), and at least some of the other folds, excluding the at least some of the folds used as the training data set (310), may be used as the verification data set (320). Accordingly, in the present invention, configuring the verification data set (320) can be understood as configuring the training data set (310) and the verification data set (320) respectively such that the ratio (or number) of the first sample (801) and the second sample (802) in each of the N folds is included in a ratio (or number) that satisfies a pre-set ratio standard.
[0176] The control unit (180) may train at least one artificial intelligence model by using at least some of the plurality (or plurality) of folds as a training data set (310) so that at least one artificial intelligence model to be trained determines whether the product is defective. As illustrated in FIG. 9, the at least one artificial intelligence model to be trained may include a plurality of machine learning models (171a, 171b, 171c, 171d) and a plurality of deep learning models (172a, 172b, 172c). The control unit (180) may train the plurality of machine learning models (171a, 171b, 171c, 171d) and the plurality of deep learning models (172a, 172b, 172c) respectively using the training data set (310) to determine whether the produced product is normal or defective (S601, see FIG. 6). In this case, the control unit (180) can fine-tune multiple machine learning models (171a, 171b, 171c, 171d) and multiple deep learning models (172a, 172b, 172c) respectively by applying contrastive learning so that normal samples and defective samples included in the learning data set (310) are learned to move away from each other and can be distinguished well even if they have similar distributions.
[0177] In one embodiment, let us assume that there are a plurality of folds, from the first to the fifth fold. The control unit (180) can use the first fold as a learning data set (310) to learn a plurality of machine learning models (171a, 171b, 171c, 171d) and a plurality of deep learning models (172a, 172b, 172c), respectively. In this case, the control unit (180) inputs normal samples and defective samples included in the learning data set (i.e., the first fold) into a plurality of machine learning models (171a, 171b, 171c, 171d) and a plurality of deep learning models (172a, 172b, 172c), respectively, and can compare and learn the plurality of machine learning models (171a, 171b, 171c, 171d) and the plurality of deep learning models (172a, 172b, 172c) respectively so that the distance between the normal samples and the defective samples is maximized (or becomes far apart). Afterward, when the training of a plurality of machine learning models (171a, 171b, 171c, 171d) and a plurality of deep learning models (172a, 172b, 172c) is completed through the first fold, the control unit (180) can use at least one fold different from the fold used for training (e.g., the second fold) as a training data set (310) to comparatively train the plurality of machine learning models (171a, 171b, 171c, 171d) and the plurality of deep learning models (172a, 172b, 172c), respectively.
[0178] Next, the control unit (180) can perform evaluation (or verification) on each of the plurality of machine learning models (171a, 171b, 171c, 171d) and the plurality of deep learning models (172a, 172b, 172c) that have been trained (or comparatively trained) with the training data set (310) using the verification data set (320). Based on the training through the training data set (310), the control unit (180) can obtain the plurality of trained machine learning models (171a, 171b, 171c, 171d) and the plurality of trained deep learning models (172a, 172b, 172c).
[0179] To this end, the control unit (180) may input a sample corresponding to at least one product included in the verification data set (320) into a plurality of trained machine learning models (171a, 171b, 171c, 171d) and a plurality of trained deep learning models (172a, 172b, 172c), respectively (S603, see FIG. 6). Then, the control unit (180) may obtain a plurality of predicted values for the sample from each of the plurality of trained machine learning models (171a, 171b, 171c, 171d) and the plurality of trained deep learning models (172a, 172b, 172c) (S605, see FIG. 6), and evaluate the anomaly detection performance of each of the plurality of trained machine learning models and the plurality of trained deep learning models based on the plurality of predicted values (S607, see FIG. 6).
[0180] In one embodiment, let us assume that there are a plurality of folds, from the first to the fifth fold. When learning for each of the multiple machine learning models (171a, 171b, 171c, 171d) and multiple deep learning models (172a, 172b, 172c) is performed through at least one of the multiple folds, the control unit (180) can verify each of the multiple learned machine learning models (171a, 171b, 171c, 171d) and multiple learned deep learning models (172a, 172b, 172c) by using at least one other fold, excluding the fold used for learning each of the multiple learned machine learning models (171a, 171b, 171c, 171d) and multiple learned deep learning models (172a, 172b, 172c), as verification data (320).
[0181] In this case, the control unit (180) inputs normal samples and defective samples included in a verification data set (e.g., a first fold) into a plurality of trained machine learning models (171a, 171b, 171c, 171d) and a plurality of trained deep learning models (172a, 172b, 172c), respectively, and can obtain a plurality of predicted values for the samples from each of the plurality of trained machine learning models (171a, 171b, 171c, 171d) and the plurality of trained deep learning models (172a, 172b, 172c). Afterward, when verification (or evaluation) of multiple learned machine learning models (171a, 171b, 171c, 171d) and multiple learned deep learning models (172a, 172b, 172c) is completed through the first fold, the control unit (180) can verify each of the multiple learned machine learning models (171a, 171b, 171c, 171d) and multiple learned deep learning models (172a, 172b, 172c) by using at least one fold different from the fold used for verification (e.g., the second fold) as a verification data set (320).
[0182] At this time, there may be various evaluation methods (or evaluation indicators) and evaluation strategies for verifying each of the plurality of learned machine learning models (171a, 171b, 171c, 171d) and the plurality of learned deep learning models (172a, 172b, 172c) in the present invention.
[0183] For example, as illustrated in FIGS. 10(a) and (b), the evaluation indicators in the present invention may include at least one of the F1 evaluation indicator, the MCC evaluation indicator, and the BER evaluation indicator. F1 (or F1 score) is an evaluation indicator used to quantify the performance that combines the precision and recall of a model. Additionally, MCC is not affected by data imbalance, allowing for a fair evaluation by considering all values of TP, TN, FP, and FN (see FIG. 11(b)). Furthermore, BER is the average error rate in each class; as a favorable indicator in the face of class imbalance, it allows for an evaluation that is not biased toward a specific class because it considers errors for each class separately (see FIG. 11(c)). In this case, FIG. 10(a) may show an evaluation result (1010, or first evaluation result) obtained (or calculated) through evaluation of each of the multiple learned deep learning models (172a, 172b, 172c), and FIG. 10(b) may show an evaluation result (1020, or second evaluation result) obtained (or calculated) through evaluation of each of the multiple learned machine learning models (171a, 171b, 171c, 171d).
[0184] Furthermore, the control unit (180) can determine the final predicted value for the sample using a plurality of predicted values (S609, see FIG. 6).
[0185] The output (or output data) of the plurality of learned machine learning models (171a, 171b, 171c, 171d) described above may include a plurality of predicted values. For example, as illustrated in FIG. 9, the plurality of predicted values may include at least one of the predicted value (901) of the first machine learning model, the predicted value (902) of the second machine learning model, the predicted value (903) of the third machine learning model, and the predicted value (904) of the fourth machine learning model for at least one sample included in the verification data set (320). In this case, the control unit (180) may ensemble the plurality of predicted values (901, 902, 903, 904) obtained from each of the plurality of learned machine learning models (171a, 171b, 171c, 171d) to generate an ensemble result (910).
[0186] Additionally, the control unit (180) can generate predictions (920) of the top N (or K) selections using multiple prediction values obtained from each of the multiple learned deep learning models (172a, 172b, 172c). For example, the prediction of the top N selections (920) may refer to the process of selecting samples to be converted based on a threshold value using multiple prediction values obtained from each of the multiple learned deep learning models (172a, 172b, 172c). Here, the top N selections refer to samples (abnormal candidates) corresponding to the N% range with the highest prediction probability, and conversely, the bottom N% selections refer to samples (normal candidates) existing in the range with the lowest prediction probability. That is, the top N selection predictions (920) may include the process of extracting the top N% of probability values predicted by multiple learned deep learning models (172a, 172b, 172c) (samples with a high probability of being abnormal) and expanding them to defects (e.g., 1) in the final prediction.
[0187] Finally, the control unit (180) can determine (or calculate, extract, generate, etc.) a final prediction value (930) for a sample using the ensemble result (910) and the top N prediction values (i.e., the prediction of the top N selections (920)). For example, the final prediction value (930) can be understood as the result of combining the prediction values of each of the multiple learned machine learning models (171a, 171b, 171c, 171d) and the multiple learned deep learning models (172a, 172b, 172c), representing the probability value (e.g., normal is 0, defective is 1) that the product is normal or defective.
[0188] As discussed above, the present invention can improve the generalization performance of the model by incorporating a validation strategy that classifies samples into easy or difficult based on the Mahalanobis distance. Furthermore, the present invention can effectively widen the margin between normal samples and anomalous samples within a feature-engineered feature space. By utilizing these improved features, the present invention can enhance prediction accuracy through post-processing that combines an ensemble and a deep learning model. In other words, the present invention can more accurately solve the defect detection problem through this probabilistic approach to defect prediction. Thus, the present invention can build a high-performance model capable of identifying defects with greater confidence by significantly improving prediction accuracy through ensemble, providing a powerful solution for estimating defect probabilities.
[0189] Meanwhile, according to another embodiment of the process defect prediction system and method according to the present invention described above, the present invention utilizes both raw data (Raw Data, or original data, raw data, etc.) and simulation data in the inference (or prediction) process to determine whether there is an abnormality in a product in various process environments.
[0190] For example, FIG. 12 may illustrate a framework of the present invention that utilizes both raw data (e.g., “Raw Data”, 1201) and simulation data (e.g., “Simulation Data”, 1202). Here, the simulation data (1202) is data generated through simulation based on domain knowledge, which can expand data diversity and potentially generate abnormal samples that are not present in the raw data.
[0191] When input data (raw data (1201) and simulation data (1202)) is received (or input) by the process defect prediction system (100), the original data (1201) and simulation data (1202) can both be preprocessed through the data processing unit (150). For example, the data processing unit (150) can perform preprocessing to ensure data consistency, quality, and feature extraction.
[0192] In addition, the process defect prediction system (100) may input preprocessed input data into at least one trained artificial intelligence model. For example, the artificial intelligence model may include a plurality of trained machine learning models (171a, 171b, 171c, 171d) and a plurality of trained deep learning models (172a, 172b, 172c). In this case, the machine learning model or deep learning model used in the present invention (or in the inference process of the present invention) may be a model selected by taking into account i) reducing dependence on a specific feature (AutoClave) and ii) the overlap between the distribution of normal samples and the distribution of abnormal samples. That is, it may be a model selected by considering pre-set selection criteria (e.g., characteristics of the process and characteristics of the data, etc.). For example, it may include at least some machine learning models or deep learning models selected through the evaluation results using the verification data set (320) described above. That is, in the present invention, at least one artificial intelligence model for determining whether there is an abnormality in a product can be selectively utilized based on the evaluation results through the verification data set (320).
[0193] Here, each of the selected multiple trained machine learning models (171a, 171b, 171c, 171d) is used for a specific task, and some may be composed of classifiers and others of regression models. The multiple trained machine learning models (171a, 171b, 171c, 171d) are machine learning models that are frequently used for tabular data and can be very efficient for processing structured data. When the process defect prediction system (100) obtains multiple predicted values (1211, 1212, 1213, 1214) generated (or predicted) for input data from each of the multiple trained machine learning models (171a, 171b, 171c, 171d), it can combine the multiple predicted values (e.g., 1211, 1212, 1213, 1214) in an ensemble manner. Ensembles can improve overall accuracy and robustness by combining predictions from multiple models.
[0194] Additionally, the selected multiple trained deep learning models (172a, 172b, 172c) can process complex and unstructured data and may include useful models used for function learning, pattern recognition, and anomaly detection. As described above, when the process defect prediction system (100) obtains multiple prediction values generated (or predicted) for input data from each of the multiple trained deep learning models (172a, 172b, 172c), it can generate top K selected predictions (1230) using the multiple prediction values.
[0195] Finally, a final prediction value (1240) can be generated by combining an ensemble result (1220) obtained by ensembling multiple prediction values (1211, 1212, 1213, 1214) obtained from each of multiple trained machine learning models (171a, 171b, 171c, 171d) and a prediction (1230) of the top K selections of multiple trained deep learning models (172a, 172b, 172c).
[0196] At this time, a feedback loop may be established in the process defect prediction system (100) to continuously improve the simulation data generation process. Based on the verification of anomalies in the simulation, the process defect prediction system (100) may provide feedback (1241) to adjust the simulation model, thereby improving the quality and relevance of the simulation data in the future.
[0197] As such, the pipeline illustrated in FIG. 12 can improve the robustness and accuracy of predictions by combining the advantages of machine learning and deep learning with simulation-based data augmentation. The feedback loop ensures that the simulation data maintains relevance and accuracy, thereby continuously improving pipeline performance.
[0198] Meanwhile, according to another embodiment of the process defect prediction system and method of the present invention described above, scheduling can be performed with the goal of reducing process delays and producing products quickly and efficiently. For example, the present invention can optimize process scheduling (or production planning) through Linear Programming and Mixed Integer Programming (MILP) by considering resource and process constraints.
[0199] Furthermore, according to another embodiment of the process defect prediction system and method according to the present invention described above, a system (100) capable of extracting defects from a multimodal perspective through various data such as cameras as well as sensor data can be constructed. For example, in the present invention, the system (100) can be constructed in a direction that models various scenarios while performing simulations for various events by conducting Discrete Event Simulation (DED).
[0200] As described above, according to the process defect prediction method and system of the present invention, an artificial intelligence model can be constructed to determine whether a product is defective by utilizing process data collected during the product production process. Through this, the present invention can determine whether a product is defective in advance, thereby reducing the costs (e.g., equipment operation costs, labor costs, etc.) and inspection time required for product inspection, while simultaneously increasing the efficiency of product production and achieving overall process optimization.
[0201] Furthermore, according to the process defect prediction method and system of the present invention, an artificial intelligence model is constructed to determine whether a product is defective by considering class imbalances in process data, and the possibility of defects occurring during the production process can be determined in advance using the constructed artificial intelligence model. Through this, the present invention can reduce the burden of the inspection process and reduce overall manufacturing quality control costs.
[0202] Furthermore, according to the process defect prediction method and system of the present invention, key characteristics such as volume change and pressure change are analyzed based on physical variables (e.g., volume, time, speed, temperature, pressure, etc.), and important features can be extracted at each process stage. Through this, the present invention generates derived variables at each of the multiple process stages, thereby ensuring data diversity and enabling the model to efficiently learn the diversity of data characteristics between processes or the complex correlations between physical characteristics. In other words, the present invention allows for the quantitative analysis of changes at each process stage by utilizing derived variables, enabling the early detection of process abnormality signals and the prediction of the likelihood of defect occurrence in advance.
[0203] Furthermore, according to the process defect prediction method and system of the present invention, a verification data set can be constructed by evenly distributing samples of different classes, and an artificial intelligence model can be verified and evaluated using various evaluation indicators and verification strategies. Through this, the present invention enables fair performance evaluation that is not biased toward a specific class, and can enhance the robustness and reliability of the artificial intelligence model. Accordingly, the artificial intelligence model can be applied usefully and stably to various process environments and data characteristics in actual field conditions.
[0204] Furthermore, according to the process defect prediction method and system of the present invention, the artificial intelligence model is not biased toward a specific process or class and can learn the diversity of data characteristics between processes, thereby improving generalization performance under various production conditions when applied to actual processes. In other words, the present invention can increase the universality of the model by performing various modeling that can be applied in diverse process environments, and thus can contribute significantly to improving productivity and strengthening quality competitiveness throughout the manufacturing industry.
[0205] Meanwhile, the present invention described above can be implemented based on a quantum computer. The present invention implemented based on a quantum computer may include a qubit-based quantum processor and quantum memory, and may include software and hardware interfaces optimized for quantum computation.
[0206] Quantum processors in quantum computers utilize qubits to efficiently process complex operations through parallel computation, quantum entanglement, and quantum superposition, which cannot be performed by the binary bits of classical computers. Quantum processors process data using quantum gates and can provide exponential speed improvements for specific problems.
[0207] Meanwhile, the present invention described above can be implemented as a program that is executed by one or more processes on a computer and can be stored on a computer-readable medium (or recording medium).
[0208] Furthermore, the present invention described above can be implemented as computer-readable code or instructions on a medium on which a program is recorded. That is, the present invention can be provided in the form of a program.
[0209] Meanwhile, computer-readable media include all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include HDD (Hard Disk Drive), SSD (Solid State Disk), SSD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.
[0210] Furthermore, the computer-readable medium may be a server or cloud storage that includes a storage and is accessible to an electronic device via communication. In this case, the computer may download the program according to the present invention from the server or cloud storage via wired or wireless communication.
[0211] A computer program may reach the system (100) through various suitable transmission mechanisms. The transmission mechanism may be, for example, a computer-readable storage medium, a computer program product, a memory device, a recording medium such as a CD-ROM or DVD, or a product that tangibly embodies the computer program. The transmission mechanism may be a signal configured to reliably transmit the computer program through air or an electrical connection. The system (100) may propagate or transmit the computer program as a computer data signal.
[0212] Furthermore, references to 'computer-readable storage media,' 'computer program products,' 'computer programs embodied in a tangible form,' etc., or to 'controller,' 'computer,' 'processor,' etc., should be understood to include not only computers with various architectures such as single / multi-processor architectures and sequential (Von Neumann) / parallel architectures, but also specialized circuits such as Field-Programmable Gate Arrays (FPGAs), Application Specific Circuits (ASICs), signal processing units, and other devices. References to computer programs, instructions, code, etc., should be understood to include software for programmable processors or firmware, such as programmable content for hardware devices, whether it is instructions for a processor or configuration settings for a fixed-function device, gate array, or programmable logic device.
[0213] Furthermore, in the present invention, the computer described above is an electronic device equipped with a processor, namely a CPU (Central Processing Unit), and no special limitations are placed on its type.
[0214] Meanwhile, the above detailed description should not be interpreted restrictively in all respects but should be considered exemplary. The scope of the invention shall be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the invention are included within the scope of the invention.
Claims
Claim 1 A computerized process defect prediction method comprising the following steps: specifying process data including data corresponding to at least one product produced through a plurality of process steps; performing feature engineering on the process data to generate a derived variable using at least some features corresponding to each of the plurality of process steps and values corresponding to the at least some features; constructing a training data set and a verification data set, respectively, using the process data; training at least one artificial intelligence model using the training data set so that at least one artificial intelligence model to be trained determines whether there is an abnormality in the product; inputting the verification data set into at least one artificial intelligence model trained with the training data set; and obtaining a predicted value for the verification data set from the at least one trained artificial intelligence model. A method for predicting process defects, comprising the step of evaluating the anomaly detection performance of at least one learned artificial intelligence model based on the predicted value, wherein each of the training data set and the verification data set further includes the derived variable and a value corresponding to the derived variable, and wherein in the step of configuring each of the verification data sets, the verification data set containing the derived variable is configured such that a first sample and a second sample classified according to different difficulty levels are evenly included in each fold of cross-validation. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 A method for predicting process defects according to claim 1, further comprising the step of classifying a plurality of different features included in the process data and a plurality of samples having values corresponding to each of the plurality of features, according to a pre-set classification criterion. Claim 6 A method for predicting process defects according to claim 5, wherein the classifying step comprises: a step of classifying the plurality of samples into a first group or a second group based on a value corresponding to a specific feature among the plurality of features; a step of calculating a first covariance matrix for the first group and a second covariance matrix for the second group, respectively; a step of calculating a first mean vector for the first group and a second mean vector for the second group, respectively; a step of calculating a Mahalanobis distance between the first mean vector and the second mean vector for each of the plurality of samples; and a step of classifying the plurality of samples based on the previously set classification criteria and the calculated Mahalanobis distance. Claim 7 A process defect prediction method according to claim 6, wherein the step of calculating the Mahalanobis distance is characterized by, for each of the plurality of samples, calculating a first Mahalanobis distance based on the first average vector calculated from the first group and a second Mahalanobis distance based on the second average vector calculated from the second group. Claim 8 A method for predicting process defects according to claim 6, wherein the specific feature is a feature indicating whether the produced product is normal or defective, and the value corresponding to the specific feature is configured to have a first value or a second value depending on whether the produced product is normal or defective, and the first group includes a sample among the plurality of samples that includes the first value for the specific feature, and the second group includes a sample among the plurality of samples that includes the second value for the specific feature. Claim 9 A method for predicting process defects according to claim 6, wherein the previously established classification criteria are set based on the difference in Mahalanobis distances for the normal distribution and the abnormal distribution of each of the plurality of samples, the normal distribution is defined based on the first covariance matrix and the first mean vector calculated for the first group, and the abnormal distribution is defined based on the second covariance matrix and the second mean vector calculated for the second group. Claim 10 A method for predicting process defects according to claim 9, wherein in the classification step, at least some of the samples among the plurality of samples are classified as first samples, wherein the difference between the Mahalanobis distance between the normal distribution and the abnormal distribution satisfies the first classification criterion among the previously set classification criteria. Claim 11 A method for predicting process defects according to claim 9, wherein in the classification step, at least some of the samples among the plurality of samples are classified as second samples if the difference between the normal distribution and the abnormal distribution satisfies the second classification criterion among the previously set classification criteria. Claim 12 A method for predicting process defects according to claim 1, wherein a plurality of samples included in the process data include the first sample and the second sample, and in the step of constituting the verification data set, at least a portion of each of the first sample and the second sample is included in the verification data set such that the composition ratio of each of the first sample and the second sample satisfies a preset composition ratio standard. Claim 13 A method for predicting process defects according to claim 12, wherein the step of configuring the verification data set is characterized by configuring the verification data set such that the composition ratio of each of the first sample and the second sample satisfies the preset composition ratio standard, based on a first index corresponding to the first sample and a second index corresponding to the second sample. Claim 14 A method for predicting process defects according to claim 1, wherein in the training step, the at least one artificial intelligence model to be trained is subjected to contrastive learning using the training data set in order to determine whether the product is defective. Claim 15 A method for predicting process defects according to claim 1, wherein the at least one artificial intelligence model to be trained includes a plurality of machine learning models and a plurality of deep learning models, and wherein, in the training step, the plurality of machine learning models and the plurality of deep learning models are each trained using the training data set. Claim 16 A method for predicting process defects according to claim 15, wherein in the evaluation step, a sample corresponding to at least one product included in the verification data set is input to each of the plurality of learned machine learning models and the plurality of learned deep learning models, a plurality of predicted values for the sample are obtained from each of the plurality of learned machine learning models and the plurality of learned deep learning models, and the anomaly detection performance of each of the plurality of learned machine learning models and the plurality of learned deep learning models is evaluated based on the plurality of predicted values. Claim 17 A process defect prediction method according to claim 16, further comprising the step of determining a final predicted value for the sample using the plurality of predicted values. Claim 18 A process defect prediction method according to claim 17, wherein in the step of determining the final predicted value, a plurality of predicted values obtained from the plurality of trained machine learning models are ensembled, and a final predicted value for the sample is determined using the ensemble result and the top N predicted values obtained from the plurality of trained deep learning models, and the plurality of trained deep learning models are configured to perform post-correction on the ensemble result. Claim 19 A process defect prediction system comprising a memory configured to store executable instructions and one or more processors configured to perform operations by executing one or more instructions, wherein the system specifies process data including data corresponding to at least one product produced through a plurality of process steps, performs feature engineering on the process data to generate a derived variable using at least some features corresponding to each of the plurality of process steps and a value corresponding to the at least some features, respectively, using the process data, constructs a training data set and a verification data set, respectively, and trains at least one artificial intelligence model using the training data set to enable at least one artificial intelligence model to determine whether there is an abnormality in the product, inputs the verification data set to at least one artificial intelligence model trained with the training data set, obtains a prediction value for the verification data set from the at least one trained artificial intelligence model, and evaluates the anomaly detection performance of the at least one trained artificial intelligence model based on the prediction value, wherein each of the training data set and the verification data set further includes the derived variable and a value corresponding to the derived variable, and when constructing the verification data set, the verification data set containing the derived variable is according to different difficulty levels A process defect prediction system characterized by being configured such that classified first and second samples are evenly included in each fold of cross-validation. Claim 20 A program that is executed by one or more processes in an electronic device and stored on a computer-readable recording medium, wherein the program comprises: a step of specifying process data including data corresponding to at least one product produced through a plurality of process steps; a step of performing feature engineering on the process data to generate a derived variable using at least some features corresponding to each of the plurality of process steps and values corresponding to the at least some features; a step of constructing a training data set and a verification data set, respectively, using the process data; a step of training at least one artificial intelligence model using the training data set so that at least one artificial intelligence model to be trained determines whether there is an abnormality in the product; a step of inputting the verification data set into at least one artificial intelligence model trained with the training data set; and a step of obtaining a predicted value for the verification data set from the at least one trained artificial intelligence model. A program stored on a computer-readable recording medium, comprising instructions for performing a step of evaluating the anomaly detection performance of at least one learned artificial intelligence model based on the predicted value, wherein each of the training data set and the verification data set further includes the derived variable and a value corresponding to the derived variable, and wherein in the step of configuring each of the verification data sets, the verification data set containing the derived variable is configured such that a first sample and a second sample classified according to different difficulty levels are equally included in each fold of cross-verification.
Citation Information
Patent Citations
Abnormality determination system, motor control device, and abnormality determination device
JP6569927B1
Method and appratus for predicting process failure
KR1020220168316A
Device and method for updating ai predition model to prediction of product failure
KR1020230018869A
Method and apparatus for selective ensemble prediction based on dynamic model combination
KR1020230135838A
Method and system for detecting the quality abnormality based on the artificial intelligence, and controlling facility
KR1020240025792A
