System and method for automatically determining optimization process algorithms using machine learning models

Abstract

To provide a process optimization system that can determine the optimal process scenario among several process scenarios with different order of unit processes.SOLUTION: A process optimization system may include the steps of: performing evaluation for each unit process according to evaluation criteria by a reading module while each execution comprising one or more unit processes performed in sequence is performed, and, while accumulating in the order of the unit processes, collecting execution data generated by the evaluation and communicating the execution data to the reading module; and generating instruction data using execution data for the best execution determined from among multiple executions performed by the determination module by changing the order of the unit processes based on trial data of the reading module.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a system and method for automatically determining an optimization process algorithm using a machine learning model. [Background technology]

[0002] The process of manufacturing one product, which consists of several unit processes, processes materials mechanically, physically, and chemically, and changes their structure, properties, and external shape to produce a finished product or intermediate product. As these processes became more complex, the work instructions proposed by engineers during the design process included a significant number of items, and errors and defects that did not link with actual on-site inspections and manufacturing equipment occurred frequently.

[0003] For this reason, it has become even more difficult to reflect the know-how of actual workers, rather than engineers, in work instructions. For example, it may be found that a process sequence different from that specified in the work instructions is more efficient due to the know-how of experienced workers, but in many cases it is not actually possible to immediately apply this to work instructions in terms of cost and responsibility, and so it is not documented in manuals.

[0004] Therefore, there have been various attempts to improve process efficiency and increase corporate productivity (Patent Document 1, Patent Document 2, etc.), but these only propose suitable process sequences theoretically using methods such as regression models, and there are limitations in that it is difficult to immediately apply these to the workplace.

[0005] Even if such work instructions were created, the only way to do so would be to sample and inspect after all work had been completed, or to have workers manually collect internal data from inspection equipment after interrupting work and understand it based on their own knowledge.

[0006] This created a need for a system that could generate optimized work instructions in real time without interrupting the work site. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Republic of Korea Patent No. 10-1441230 [Patent Document 2] Republic of Korea Patent Publication No. 10-2003-0068676 Summary of the Invention [Problem to be solved by the invention]

[0008] An object of the present invention is to provide a process optimization system capable of determining an optimal process scenario from among several process scenarios in which the sequences of unit processes are different.

[0009] However, the technical problem that the present embodiment aims to achieve is not limited to the above-mentioned technical problem, and other technical problems may exist. [Means for solving the problem]

[0010] According to one embodiment of the present invention, the artificial intelligence based process optimization system may include a reading module for collecting execution data generated by performing an overall process for manufacturing a product, the overall process being composed of one or more unit processes performed in a sequential order, the unit processes being performed in a different order, and evaluating each unit process according to an evaluation criterion during each execution, and accumulating and evaluating the execution data in the order of the unit processes; and a judgment module for determining an optimal execution from among a plurality of executions performed in a different order of the unit processes based on the trial data of the reading module, and generating instruction data based on the execution data for the optimal execution.

[0011] The reading module can be evaluated by determining one or more of the evaluation criteria: required time or defect rate.

[0012] The reading module can identify each trial through the progression sequence of the unit processes.

[0013] The reading module can identify each unit process on the target object of the unit process and record the order of progress of the unit processes.

[0014] The reading module may include a photographing module that photographs a unit process of the object and generates image data; and a machine learning model that reads the image data transmitted from the photographing module, recognizes the object, and identifies the unit process corresponding to the object.

[0015] When the machine learning model recognizes a new object whose external shape has changed as each unit process is completed, it reads the corresponding unit process as completed and inputs the entire process list for the product. When all unit processes for the objects in the list are completed, it reads each trial as completed and inputs the trial data collected for each trial to the judgment module.

[0016] The machine learning model measures the similarity with the image data based on normal image data already collected for the object, and the reading module can read the image data as normal image data using a preset similarity criterion and collect new normal images as the trial data.

[0017] The reading module may include a search module that searches the list for a list including the object first recognized by the machine learning model and inputs the list to the machine learning model.

[0018] The reading module includes a search module that searches for an object associated with the object if there is no normal image data already collected for the object, and can measure the similarity between the image data and the object searched for by the search module using normal image data already collected for an object associated with the object.

[0019] The judgment module can be configured to receive a format from an MES system, merge trial data for the optimal execution with the format to generate the instruction data, and output the instruction data in real time.

[0020] An artificial intelligence based process optimization method according to one embodiment of the present invention may include a step of: an overall process for manufacturing a product is composed of one or more unit processes that are performed in a sequential order, the unit processes are performed in a different order, and while each operation is being performed, a reading module evaluates each unit process according to an evaluation criterion, and while accumulating the evaluation in the order of the unit processes, collecting and transmitting the generated operation data to the reading module; and a step of generating instruction data from the operation data for an optimal operation determined by the judgment module from among a plurality of operations performed by changing the order of the unit processes based on the trial data of the reading module.

[0021] The reading module includes an imaging module that captures an image of a unit process of an object to generate image data; and a machine learning module that determines and evaluates at least one of a required time or a defect rate as the evaluation criteria, reads the image data, recognizes that the object changes as the unit process progresses, evaluates the required time, recognizes the outer shape of the object being deformed, reads the assembly degree and assembly direction of the unit process, and evaluates the defect rate. The judgment module can judge an execution in which the total required time of the unit processes recorded in the execution data is the smallest to be the optimal execution. Effect of the Invention

[0022] The present invention has the following advantages.

[0023] First, by automatically monitoring and evaluating the progress of processes, the optimal process sequence can be determined based on the field experience of workers, even between unit processes that are not dependent on each other.

[0024] Second, the optimal process scenario can be determined as the process proceeds naturally without the need for separate simulations.

[0025] However, the effects of the present invention are not limited to those described above, and effects not mentioned can also be clearly understood by a person having ordinary skill in the art to which the present invention belongs from this specification and the accompanying drawings. [Brief description of the drawings]

[0026] [Figure 1] FIG. 1 is a relationship diagram of an artificial intelligence-based process optimization system according to an embodiment of the present invention. [Diagram 2] 1 is a flow chart of an artificial intelligence based process optimization method according to one embodiment of the present invention. [Diagram 3] FIG. 2 is a diagram illustrating a process for implementing an artificial intelligence-based process optimization system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] Preferred embodiments of the present invention will be described in more detail, but already known technical aspects will be omitted or simplified for the sake of brevity.

[0028] FIG. 1 is an artificial intelligence-based process optimization system according to one embodiment of the present invention.

[0029] As shown in FIG. 1, an artificial intelligence-based process optimization management system according to an embodiment of the present invention includes a reading module 100, a sensing module 200, and an output module 300.

[0030] An artificial intelligence-based process optimization management system according to an embodiment of the present invention provides a process optimization system that can determine an optimal process scenario from among a plurality of process scenarios having different unit process sequences.

[0031] Specifically, an AI-based process optimization management system according to one embodiment of the present invention determines an optimal implementation from among multiple implementations in which the order of progress of each unit process is changed when an overall process for manufacturing a product is composed of one or more unit processes that are performed in a sequential order and the unit processes are performed in a different order.

[0032] During the process of deriving the optimal implementation and even after the optimal implementation has been derived, data is recorded so that it is possible to detect whether the ongoing process is being carried out successfully based on the recorded data, and by reading the data using a processor, it is possible to detect the degree to which the implementation deviates from the optimal implementation.

[0033] The reading module 100 is capable of collecting and reading image data relating to processes performed on objects provided for the manufacture of a product.

[0034] Here, the object can mean materials or parts used to make a product through a manufacturing process.

[0035] That is, the reading module 100 can read image data of a process currently in progress in accordance with existing instruction data and generate work data.

[0036] The reading module 100 may include a photography module 110 and a machine learning model 120 .

[0037] The image capture module 110 may capture an image of the progress of an object to generate image data. The image capture module 110 may include a plurality of cameras installed on a process line, and may transmit the captured image data of the corresponding cameras to the machine learning model 120.

[0038] The camera used by the imaging module 110 can sense the progress of an object through various radio waves such as infrared rays, visible light, X-rays, gamma rays, electromagnetic waves, ultrasound, etc. The image data can be a two-dimensional image, a three-dimensional image, a pre-processed ROI image, a cropped image, etc.

[0039] The machine learning model 120 can recognize and read objects in image data and generate work data for the progress process.

[0040] The machine learning model 120 can recognize that the target object is changed as each unit process is performed, read the order in which the processes proceed, and generate work data including sequence information regarding such order of proceeding.

[0041] The machine learning model 120 can recognize the object of the unit process through image data generated by photographing the unit process while the unit process is being performed in a series, and can thereby identify the type of the unit process. If the object is completely changed, the unit process is also identified as different, so it can be read that the next unit process was performed after the previous unit process was completed. The machine learning model 120 can grasp the order in which all unit processes are performed using the above method, identify the type of process execution, i.e., which process the whole process is performed, and generate work data including order information of the execution of the corresponding processes.

[0042] Similarly, the machine learning model 120 can recognize the contour of an object that is deformed as the process progresses to read the working state of the process, and generate working data including state information regarding such working state.

[0043] That is, when an object is identified, the machine learning model 120 can read the working state of the unit process from the degree of deformation of the object recognized through image data before and after the progress of the corresponding unit process. The outer shape can include not only the shape but also the color, surface, interior, etc.

[0044] For example, if the process is an assembly operation with a part as the object, the result of the first unit process performed on the first part as the object may be read from the degree to which the outer shape of the part is deformed as a result of the assembly operation. After the machine learning model 120 recognizes the object through the image data, it may numerically measure the operation state, such as the direction in which the first part is assembled or the assembly strength, and generate operation data from the recorded state information.

[0045] Alternatively, the machine learning model 120 may generate work data using status information that is a numerical value recorded by comparing image data relating to the object with previously collected image data of normal assembly to measure the degree of similarity.

[0046] In addition, when there is no normal image data already collected for an object, the machine learning model 120 can use a search module that searches for objects associated with the object, and measure the similarity with the image data using normal image data already collected for the object associated with the object searched for by the search module.

[0047] The machine learning model 120 can then detect the work area where the object process is performed and generate work data including position information.

[0048] Here, the work area may refer to a location where a specific unit process of a type of work line is carried out, or may refer to a portion of an area within a facility where a process is carried out.

[0049] Of course, a GPS module that may be included in the machine learning model 120 can acquire location and generate operation data that includes location information.

[0050] As a result, defect information that may have been previously generated may be matched with location information and stored as work data, making it possible to search for defect information related to defects that occurred in the area of ​​the corresponding work from the location information.

[0051] In addition, the machine learning model 120 can read the time required for a process by recognizing that the object is changed while the process is being performed, and can measure the error between the time information, which is the required time, and the time information in the instruction data and the time required for optimal process execution. Thereafter, the detection module 200 can evaluate the error as the proficiency of the worker, excluding environmental factors such as temperature and humidity in the area of ​​the corresponding work.

[0052] Meanwhile, the reading module 100 can collect sensing data corresponding to quality factors or external factors using the sensing module, and can read the process of the object based on a preset criterion.

[0053] The sensing module can collect sensing data from the outside through various common sensors such as a microphone, a proximity sensor, an ultrasonic sensor, a gyro sensor, a vibration sensor, a temperature and humidity sensor, a pressure sensor, an impact sensor, and a gas sensor.

[0054] The quality factors may be pressure applied to the object, the moving speed of the object, vibration, temperature, humidity, specific gravity, shrinkage, strength of the object, weather environment, lighting environment, equipment life, equipment information, worker skill, material characteristics, number of cycles, etc.

[0055] The reading module 100 can collect sensing data for general process optimization and use regression analysis or machine learning to determine relationships between quality variables that are affected by the collected sensing data among the set quality variables.

[0056] When a functional relationship exists between the quality factor and the quality variable, an optimization algorithm can be used to find a quality factor that will result in a desired value for the output variable, which is the quality variable, as an input variable, and this can be reflected in the instruction data in the case of the input variable value for the optimal implementation.

[0057] The detection module 200 compares the operation data transmitted from the reading module 100 with the instruction data to determine whether the process is normal or defective, and if defective, generates defective information.

[0058] Here, the defect information may be classified into types of defects such as defects in the progress order, defects in the work state, and defects in the required time.

[0059] The detection module 200 can compare the work data with the instruction data and generate defect information indicating one or more of the defect types, i.e., sequence defect, status defect if the work status is poor, or time defect if the required time is poor.

[0060] Preferably, for the working conditions, there may be a preset error range or similarity criterion for the working to distinguish between normal and bad cases.

[0061] The detection module 200 compares the status information in the work data with the numerical value of the status information in the instruction data, and detects it as normal if it determines that it is within a preset level of error range, and generates defective information indicating a defective condition if it determines that it is outside the error range.

[0062] The detection module 200 can analyze the sensing data for the unit process where the defect occurred, and compare the quality factors based on the optimal implementation on the instruction data to find and record the type of quality factor that caused the defect, and can store the same together with the defect information.

[0063] The output module 300 can output the defect information generated by the detection module 200 .

[0064] When defect information is generated during a process, position information searched from the operation data can be input in real time to the output module 300. That is, a signal for defect information can be output to the operation data in an operation area on the position information of a corresponding unit process where a defect occurs.

[0065] Here, the signal may be a visual, auditory, tactile signal, etc., but is not limited thereto.

[0066] For example, the output module 300 may be installed in a work area identified from the position information of a unit process in which defect information was generated during the process, and may include a display that outputs a signal to visually notify an operator that a defect has occurred, an audio module that issues an audible alarm, and a vibration module that is attached to the operator and outputs vibrations.

[0067] The output module 300 may include an input module as necessary, and an operator may input information regarding the occurrence of atypical defects.

[0068] In addition, the output module 300 may include a worker terminal that is preset or can be detected by the system as being working in the vicinity of a work area identified from the position information of a unit process in which defect information is generated during the process.

[0069] The output module 300 can output quality factors that are presumed to be the cause of the defect together with the defect information.

[0070] For example, in the case of an assembly process, when the second unit process is performed to weaken the assembly fastening strength of the parts and the reading module 100 generates work data including status information that is a numerical value measuring the assembly fastening strength, the detection module 200 compares the work data with the instruction data to detect a defective work state and generate defect information related thereto. The output module 300 can output defect information in the region of the work where the defect occurred, and can also output quality factors estimated to be the cause of the defect, if necessary.

[0071] On the other hand, the reading module 100 can generate the instruction data.

[0072] As an example, a process for a number of products can be divided into a number of unit processes, and the work data acquired while performing the corresponding unit processes a number of times, preferably in different orders, can be analyzed, and the work data of the execution requiring the least amount of time can be determined to be the optimized process order for the corresponding products.

[0073] The reading module 100 can transmit instruction data for the optimal implementation for such products to be stored in a database.

[0074] The manner in which the reading module 100 generates the indication data is described in detail below.

[0075] The reading module 100 may further include a determining module 130 .

[0076] The determination module 130 can generate instruction data based on the execution data for the optimal execution determined by the determination module from among a plurality of executions performed by changing the order of the unit processes.

[0077] While the unit processes are performed in series, each unit process is evaluated according to the evaluation criteria, and the unit processes are cumulatively evaluated in order to collect performance data.

[0078] Here, the above-mentioned work data and execution data may be information generated by the same category, but in the method of generating instruction data, it is described as execution data to distinguish it from the data generated by the reading module 100.

[0079] For example, a unit process can refer to one of a multi-step component assembly process to make a product.

[0080] Here, the evaluation criteria are factors that affect the yield and quality of a process, such as the required time and the defect rate, and can be set as necessary.

[0081] The reading module 100 can identify the type of unit process through information collected from the unit processes while the unit processes are being performed in series, and can identify the type of process execution for each process by grasping the progress order of the unit processes.

[0082] Here, the process execution means a kind of scenario that is tried by changing the order of unit processes. Even if the unit processes are the same, the process scenario is different if the order is different. In the present invention, in order to determine the optimal process execution from process executions with different progression orders of unit processes, information is collected from the object, the unit processes are identified, the order is grasped, and the execution data for each process execution is mapped.

[0083] For example, while a unit process is being carried out, the required time can be evaluated from the time when the unit process is identified by judging the object to the time when the unit process is completed. At the same time or at a different time, the completion state of the unit process can be grasped and the presence or absence of defects can be evaluated.

[0084] Then, when the next unit process is performed, the required time and the presence or absence of defects can be evaluated in the same manner. At the same time or at a different time, the required time from the start of the entire process to the present and the presence or absence of defects can be cumulatively evaluated to collect execution data.

[0085] Here, the completion status of a unit process can be evaluated as completed when the target object is changed, and can also be understood based on data regarding the degree of fastening, fastening direction, release, etc. of assembled parts.

[0086] More specifically, there may be a total process list or list for a product manufactured using the object, and the reading module 100 inputs this and evaluates it each time a unit process of an object is completed. When all unit processes in the total process list have been evaluated, it may determine that the total process or one operation is completed, and input the collected operation data to the judgment module 130, thereby ending collection.

[0087] In addition, the search module included in the reading module can search a database in which various types of overall process lists are collected for a list including the object initially recognized by the machine learning model, and input the list into the machine learning model 120.

[0088] Here, the machine learning model 120 can be used to judge an object based on information collected from a unit process, and identify whether the corresponding unit process is complete, whether there is a defect, or the type of the unit process.

[0089] At this time, the machine learning model 120 continuously learns and improves images of objects using one or more object detection algorithms of vision fitting, edge, color, and location through the installed application, and can identify the type of unit process, recognize the assembly or missing state of the object, and determine whether the corresponding unit process is defective or complete.

[0090] Specifically, the machine learning model 120 learns and improves images that represent the shape, color, etc. of objects using deep learning or machine learning, which are algorithmic artificial intelligence technologies that classify or learn the characteristics of the corresponding image data by themselves, and can classify and detect objects and identify the type of unit process.

[0091] The machine learning model 120 can process data using a processor and can preprocess image data of an object to generate learning data when detecting the object.

[0092] Here, the learning data may be labeled with the assembly state, defect state, and unit process type information for the target object together with the preprocessed image data.

[0093] Information about the object to be labeled may be collected through feedback from the operator. That is, if many defective cases of the object are not found initially, the part information of the object may be input using an input interface to label the corresponding image data, or information about whether assembly is complete, whether there is a defect, etc. may be input to label the object.

[0094] The labeled data can be used to train a classifier model and applied to the machine learning model 120.

[0095] In addition, image data collected through a camera while a unit process is being performed can be pre-processed, and then a classifier model can be used to detect whether an object is defective, whether the object is assembled, and the like.

[0096] Meanwhile, in one embodiment of the present invention, normal image data may be collected in advance when a unit process on an object is normally performed without any defects.

[0097] The machine learning model 120 measures the similarity between the normal image data and newly collected image data based on the normal image data, reads the image data as normal image data using a preset similarity standard, and collects new normal images as trial data.

[0098] If necessary, features can be extracted from the collected normal image data so that the machine learning model 120 can learn only the normal image data.

[0099] At this time, if the product is improved or newly released, some or all of the parts and processes used in the product may be different from those in existing products, and therefore image data may not already be collected about the object.

[0100] The reading module 100 can collect normal image data for the improved new product process by using the machine learning model 120 to measure the similarity between image data of an object of the improved product and normal image data collected for a previous version of the product, in order to immediately determine the improved new product process without collecting image data.

[0101] In addition to the above, the machine learning model 120 may include hardware, software, and / or interfaces that utilize general deep learning vision or machine vision (MV) for surface defect inspection of wafers, display products, PCB defect inspection, LED chip packages, and other products in industrial factory automation processes.

[0102] The process of collecting trial data by the reading module 100 can be repeated until all unit processes have been completed.

[0103] The determination module 130 determines an optimal process execution from among a plurality of process executions based on the collected execution data.

[0104] Specifically, when the reading module 100 collects implementation data for the process implementation identified by the reading module 100 and implementation data for each of the multiple process implementations, the judgment module 130 can use this to determine whether the identified process implementation is an optimal process implementation.

[0105] According to one embodiment of the present invention, any one of the optimal process executions may be the case where the sum of the total unit process durations is the smallest.

[0106] In other words, using the machine learning model 120 of the reading module 100, execution data measured against the evaluation criteria, which are the required time and defect rate of each unit process, can be collected each time each unit process is performed, and if the total required time of the unit processes for any process execution is the minimum time based on the execution data, the judgment module 130 can judge the corresponding process execution to be the optimal process execution.

[0107] In addition, the above-mentioned machine learning model 120 can be used to check the work status according to the process, calculate the defect rate, and reflect this in the evaluation.

[0108] Meanwhile, a big data database for process execution may be constructed by labeling and storing the execution data collected from the reading module 100 with each process execution. The big data database may be constructed by managing execution data including the time required for process execution, defect rate, etc., in a history manner.

[0109] In the big data database, information of a worker in charge may be labeled with execution data while a corresponding process is being performed and stored.

[0110] The determination module 130 can then generate instruction data with execution data for the optimal process execution. Specifically, an existing work instruction format can be obtained from an MES (Manufacturing Execution System) system, and the optimal process execution can be merged into the corresponding format to generate instruction data with a new work instruction. The new work instruction can be output via the output module 300, such as displaying the work table where the corresponding process is performed.

[0111] If data on process execution by different workers were collected by reflecting information about the workers, it would be possible to reduce errors in the time required for process execution and defect rates caused by differences in worker skill levels, and increase the reproducibility of work instructions that reflect optimal process execution.

[0112] In other words, when information regarding the worker's proficiency level, etc. is reflected in the execution data, the judgment module 130 can distinguish between execution data of process execution by skilled workers and execution data of process execution by unskilled workers and determine the optimal process execution for each level of proficiency.

[0113] Preferably, by managing the execution data differently for each individual worker, it would be possible to propose an optimal process execution for each worker.

[0114] FIG. 3 illustrates a process for implementing an artificial intelligence-based process optimization management system according to an embodiment of the present invention.

[0115] Hereinafter, one embodiment of an artificial intelligence based process optimization system will be described with reference to FIG.

[0116] FIG. 3 shows a process in which image data collected by the imaging module 110 during the process of assembling part B to part A is read, and execution data is generated and collected by the machine learning model 120 of the reading module 100.

[0117] As shown in FIG. 3, in the above-mentioned artificial intelligence-based process optimization management system, the machine learning model 120 first detects the target object, part A, and identifies the unit process type A. The reading module 100 can then search for a list of all parts (A, B, C, D) of the product that includes the target object of the identified unit process A.

[0118] Then, it is confirmed whether the unit process A identified through the machine learning model 120 has been completed.

[0119] The reading module 100 can collect execution data of the unit process A by evaluating in real time the time required for the specified unit process A to be completed and the presence or absence of defects.

[0120] When the next unit process B is performed, it can be detected and recognized that the object has been changed through the machine learning model 120. That is, the machine learning model 120 can determine whether the object has been changed by grasping through the image that a new part B has been added to the object and the shape, color, etc. have been changed.

[0121] Similarly, the reading module 100 can collect execution data by evaluating in real time whether or not the unit process B specified by the changed object is completed, the required time, and whether or not there is a defect.

[0122] Moreover, the reading module 100 can record and store the fact that unit process A is carried out first, followed by unit process B.

[0123] When the unit process B is completed, the reading module 100 can accumulate and evaluate the time required from the beginning to the present, the presence or absence of defects, and the like in real time to collect execution data.

[0124] The above process can be carried out in a series of unit processes until all the parts of the product (A, B, C, D) are assembled.

[0125] Here, the execution of a plurality of processes may be a sequence of unit processes. If the entire list of parts is A, B, C, D, then there are 4! kinds of process executions, such as ABCD, ABDC, ACBD, ACDB, ADBC, ADCB, BACD, BADC, ..., DCBA.

[0126] Each process execution may be evaluated through the above-mentioned process and execution data may be collected. When a certain execution is performed repeatedly, the required time or the presence or absence of defects for the corresponding process execution may be cumulatively collected and stored.

[0127] Of course, not all process administrations need to be evaluated; some types of processes may not be evaluated due to inability to proceed or operator choice.

[0128] When sufficient implementation data is accumulated by the machine learning model 120, the determination module 130 can determine an optimal process implementation from among the process implementations evaluated based on the collected implementation data.

[0129] For example, as shown in Table 1 below, in the case where the ABCD process is performed 30 times with an average required time of 1 minute 20 seconds, an average error of 10 seconds, and a defect rate of 5%, and the BCAD process is performed 25 times with an average required time of 1 minute 40 seconds, an average error of 30 seconds, and a defect rate of 7%, if the preset condition prioritizes the minimum required time, the judgment module 130 can judge that ABCD, which has the minimum required time, is the optimal process execution.

[0130] [Table 1]

[0131] The AI-based process optimization system according to an embodiment of the present invention can automatically monitor and evaluate the progress of a process, thereby determining the optimal process order based on the field experience of workers even between unit processes that are not dependent on each other, and can determine the optimal process execution while the process naturally progresses without a separate simulation.

[0132] FIG. 2 shows a flow chart of an artificial intelligence-based process optimization management method according to one embodiment of the present invention.

[0133] An artificial intelligence based process optimization management method according to one embodiment of the present invention may include a step of receiving, into a reading module, instruction data recorded regarding an optimal execution determined from among a plurality of executions performed by changing the order of unit processes; a step of the reading module generating image data regarding a process to be performed on an object and generating work data resulting from reading the image data so as to correspond to the instruction data; a step of a detection module receiving the work data and comparing it with the instruction data to generate defect information regarding the process; and a step of an output module receiving and outputting the defect information.

[0134] Throughout this specification, the terms machine learning model, deep learning based model, computational model, neural network, network function, deep neural network, and neural network may be used interchangeably.

[0135] A deep neural network (DNN) may refer to a neural network that includes multiple hidden layers in addition to an input layer and an output layer. A deep neural network can be used to grasp latent structures of data. That is, it can grasp latent structures of photos, text, videos, audio, and music (e.g., what objects are in the photo, what is the content and emotion of the text, what is the content and emotion of the audio, etc.). Deep neural networks can include convolutional neural networks (CNNs), recurrent neural networks (RNNs), autoencoders, generative adversarial networks (GANs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), Q networks, U networks, Siamese networks, generative adversarial networks (GANs), and the like. The above description of deep neural networks is merely exemplary, and the present disclosure is not limited thereto.

[0136] In one embodiment of the present disclosure, the network function may also include an autoencoder. The autoencoder may be a type of artificial neural network for outputting output data similar to input data. The autoencoder may include at least one hidden layer, and an odd number of hidden layers may be arranged between the input and output layers. The number of nodes in each layer may be reduced from the number of nodes in the input layer to an intermediate layer called a bottleneck layer (encoding), and then expanded symmetrically from the bottleneck layer to the output layer (symmetric to the input layer). The autoencoder may also perform nonlinear dimensionality reduction. The number of input layers and output layers may correspond to the dimensions after preprocessing of the input data. The number of hidden layer nodes included in the encoder in the autoencoder structure may have a structure in which the number of nodes decreases the farther away from the input layer. If the number of nodes in the bottleneck layer (the layer with the fewest nodes located between the encoder and the decoder) is too small, a sufficient amount of information may not be transmitted, so it may be maintained at a certain number or more (e.g., more than half of the input layer).

[0137] A neural network may be trained by at least one of the following methods: supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. Training a neural network may be a process of applying knowledge to the neural network to make it perform a particular operation.

[0138] A neural network can be trained in a direction that minimizes output errors. In training of a neural network, training data is repeatedly input to the neural network, the output of the neural network for the training data and the target error are calculated, and the error of the neural network is backpropagated from the output layer to the input layer of the neural network in a direction to reduce the error, thereby updating the weights of each node of the neural network. In the case of supervised learning, training data with a correct answer label is used for each training data (i.e., labeled training data), and in the case of unsupervised learning, each training data may not be labeled with a correct answer. That is, for example, in the case of supervised learning for data classification, the training data may be data in which each training data is labeled with a category. The labeled training data is input to the neural network, and an error may be calculated by comparing the output (category) of the neural network with the label of the training data. As another example, in the case of unsupervised learning for data classification, an error may be calculated by comparing the training data, which is the input, with the neural network output. The calculated error may be backpropagated in the reverse direction (i.e., from the output layer to the input layer) in the neural network, and the connection weights of each node in each layer of the neural network may be updated by backpropagation. The amount of change in the connection weights of each node to be updated may be determined by a learning rate. The calculation of the neural network for the input data and the backpropagation of the error may constitute a learning cycle (epoch). The learning rate may be applied differently depending on the number of iterations of the neural network learning cycle. For example, in the early stages of learning of the neural network, a high learning rate may be used to increase efficiency by enabling the neural network to quickly achieve a certain level of performance, and a low learning rate may be used in the later stages of learning to increase accuracy.

[0139] In learning a neural network, the learning data may generally be a subset of the actual data (i.e., data to be processed using the trained neural network), and therefore there may be a learning cycle in which the error of the learning data decreases but the error increases for the actual data. Overfitting is a phenomenon in which the learning data is over-learned in this way, resulting in an increase in the error for the actual data. For example, a neural network that has learned cats by showing a yellow cat cannot recognize that it is a cat even if it sees a cat other than a yellow cat, which may be a type of overfitting. Overfitting may act as a cause of increasing the error of a machine learning algorithm. Various optimization methods may be used to prevent such overfitting. To prevent overfitting, methods such as increasing the amount of learning data, regularization, dropout that deactivates some of the nodes of the network during the learning process, and the use of a batch normalization layer may be applied.

[0140] Although the present disclosure has been described generally as being implemented by computing devices, those skilled in the art will appreciate that the present disclosure may be implemented in combination with computer-executable instructions and / or other program modules that may be executed on one or more computers and / or in a combination of hardware and software.

[0141] Generally, program modules include routines, programs, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Those skilled in the art will also appreciate that the methods of the present disclosure may be practiced with other computer system configurations, including single-processor or multi-processor computer systems, minicomputers, mainframe computers, as well as personal computers, handheld computing devices, microprocessor-based or programmable consumer electronics, and the like, each of which may be operatively connected to one or more associated devices.

[0142] The described embodiments of the present disclosure may also be practiced in distributed computing environments where certain tasks are performed by remote processing devices linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

[0143] A computer typically includes a variety of computer-readable media. Any medium accessible by a computer can be a computer-readable medium, including volatile and non-volatile media, transitory and non-transitory media, removable and non-removable media. By way of example and not limitation, computer-readable media can include computer-readable storage media and computer-readable transmission media. Computer-readable storage media include volatile and non-volatile media, transitory and non-transitory media, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, DVD (digital video disk) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information accessible by a computer.

[0144] Various embodiments presented herein may be embodied as a method, apparatus, or article of manufacture using standard programming and / or engineering techniques. The term "article of manufacture" includes a computer program, carrier, or media accessible from any computer-readable storage device. For example, computer-readable storage media include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, etc.), optical disks (e.g., CDs, DVDs, etc.), smart cards, and flash memory devices (e.g., EEPROMs, cards, sticks, key drives, etc.). Additionally, various storage media presented herein include one or more devices and / or other machine-readable media for storing information.

[0145] The specific order or hierarchy of steps in the processes presented is an example of an example approach. Based on design priorities, the specific order or hierarchy of steps in the processes may be rearranged within the scope of the present disclosure. The accompanying method claims present elements of the various steps in a sample order, but are not meant to be limited to the specific order or hierarchy presented.

[0146] The description of the embodiments presented is provided to enable any person of ordinary skill in the art to which the disclosure pertains to the present disclosure to utilize or practice the present disclosure. Various modifications to these embodiments will be apparent to those of ordinary skill in the art to which the disclosure pertains, and the general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure is not intended to be limited to the embodiments presented herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. [Explanation of symbols]

[0147] 100 Read Module 110 Shooting module 120 Machine Learning Models 130 Judgment Module 200 Detection Module 300 Output Module

Claims

1. A reading module for collecting execution data generated by evaluating each unit process according to an evaluation criterion while the entire process for manufacturing a product is composed of one or more unit processes performed in a sequential order, the unit processes being performed in a different order, and accumulating the evaluation of the unit processes in order while evaluating each unit process according to an evaluation criterion during each execution; and A determination module determines an optimal execution from among a plurality of executions performed by changing the order of unit processes based on the execution data of the reading module, and generates instruction data based on the execution data for the optimal execution, The reading module includes: Identify each step through the sequence of unit processes, In addition, each unit process is identified by the object that is the target of the unit process, and the progress order of the unit processes is recorded; Also, an imaging module that captures an image of a unit process of an object to generate image data; and A machine learning model that reads the image data transmitted from the photographing module, recognizes the object, and identifies a unit process corresponding to the object; The machine learning model comprises: As each unit process is completed, if a new object with a changed shape is recognized, the corresponding unit process is marked as completed. An entire process list of the product is input, and when all unit processes of the objects in the list are completed, each operation is read as completed, and operation data collected for each operation is input to the judgment module; The determination module includes: The execution in which the total required time or defect rate of the unit process recorded in the execution data is the smallest is determined to be the optimal execution. An artificial intelligence-based process optimization system.

2. The reading module includes: As the evaluation criteria, one or more of the required time or the defect rate is determined and evaluated.

2. The artificial intelligence based process optimization system of claim 1.

3. The machine learning model comprises: Measure a similarity between the image data and normal image data already collected for the object; The reading module reads the image data as normal image data using a preset similarity criterion, and collects a new normal image as the execution data.

2. The artificial intelligence based process optimization system of claim 1.

4. The reading module includes: and a search module for searching the list for a list including the object first recognized by the machine learning model and inputting the list to the machine learning model.

4. The artificial intelligence based process optimization system of claim 3.

5. The reading module includes: a search module for searching for an object associated with the object if no normal image data has already been collected for the object; The search module measures the similarity between the image data and the object associated with the object searched for by using normal image data already collected.

4. The artificial intelligence based process optimization system of claim 3.

6. The determination module includes: A format is input from the MES system, execution data for the optimal execution is merged with the format to generate the instruction data, and the instruction data is transmitted to be output in real time.

6. An artificial intelligence based process optimization system according to claim 4 or 5.

7. The entire process for manufacturing a product is composed of one or more unit processes that are performed in a series, and the unit processes are performed in a different order, and during each operation, a reading module evaluates each unit process according to an evaluation criterion, and collects and transmits the evaluation data generated by accumulating the unit processes in order to a judgment module; and A step of generating instruction data based on execution data of the reading module, the execution data being determined by the judgment module from among a plurality of executions performed by changing the order of unit processes; The reading module includes: An imaging module that captures an image of a unit process of an object to generate image data; and As the evaluation criteria, one or more of the required time or the defective rate is determined and evaluated; reading the image data to recognize that the object is changed as the unit process proceeds, and estimating the required time; A machine learning module that recognizes the outer shape of the object to be deformed, reads the assembly degree and assembly direction of the unit process, and evaluates the defect rate; The determination module includes: The execution in which the total required time or defect rate of the unit process recorded in the execution data is the smallest is determined to be the optimal execution. The artificial intelligence-based process optimization method according to the present invention is characterized in that

Images