Process Integration Management Method and System Using Machine Learning Model

Abstract

To provide a process optimization management system that can manage a process by determining the optimal execution from among several executions that differ in the order of unit processes that are part of the process, generating work instructions for the optimized process, and informing workers whether they are performing the optimized work as instructed.SOLUTION: A process optimization management system may include: a reading module that generates work data, which is a result of reading image data related to a process performed on an object, corresponding to instruction data upon input of the instruction data recorded for the best execution determined from among multiple executions performed in a different order of progression of unit processes that are part of the process for manufacturing a product; a detection module that compares the work data with the instruction data communicated from the reading module to generate poor information for the process; and an output module that outputs the poor information.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a process integration management method and system using a machine learning model.

Background Art

[0002] The steps of a process consisting of several unit processes for manufacturing a single product process materials mechanically, physically, and chemically, changing the structure, properties, and outer shape to produce a finished product or an intermediate product. As such processes become more complex, the work instructions proposed by engineers in the design process include a considerably large number of items, and errors and defects that are not linked to the actual on-site inspections and manufacturing equipment have become frequent.

[0003] This has made it even more difficult to reflect the know-how of actual workers rather than engineers in the work instructions. For example, although it may be revealed that a process sequence different from the work instructions is more efficient based on the know-how of skilled workers, it is often practically difficult and not manualized in terms of cost and responsibility to immediately apply this to the work instructions.

[0004] Therefore, there have been various attempts (such as Patent Document 1 and Patent Document 2) to improve process efficiency and enhance a company's productivity, but these only propose an appropriate process sequence by using a regression model theoretically, and there has been a limit in that it is difficult to immediately apply this to the work site.

[0005] Even if such work instructions are constructed, in order to determine whether an optimization process that minimizes defects has been performed according to the work instructions at the work site, sampling and inspection are performed after all the work has been completed, or after the work is interrupted, the worker manually collects internal data from the inspection device one by one, and the cause of the defect is grasped based on the knowledge of the worker. Eventually, even if a defect occurs during the process, the cause analysis can only be performed after the product is completed, so the cause analysis is also difficult, and mass production of defective products cannot be prevented.

[0006] Therefore, there is a need for a system that derives an optimized work instruction in real time without interrupting the work site and ensures compliance with it.

Prior Art Documents

Patent Documents

[0007]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0008] An object of the present invention is to determine an optimal implementation from among several implementations in which the order of unit processes, which are part of a process, is different, generate a work instruction for the optimized process, and inform the operator whether the work is being performed optimally as instructed, thereby providing a process optimization management system that can manage the process.

[0009] However, the technical problems to be achieved by the present embodiment are not limited to the above-described technical problems, and there may be other technical problems.

Means for Solving the Problems

[0010] An artificial intelligence-based process optimization management system according to an embodiment of the present invention includes a reading module that generates work data, which is a result of reading image data related to a process performed on an object, so as to correspond to instruction data when the instruction data recorded for the optimal implementation determined from among a plurality of implementations in which the order of unit processes, which are part of a process for manufacturing a product, is changed, is input; a detection module that receives the work data from the reading module, compares it with the instruction data, and detects whether there is normality or defectiveness in the process; and an output module that outputs the defect information.

[0011] When the detection result is defective, the detection module generates defect information for the above process. The output module outputs the defect information. The defect information can be generated by comparing any one or more of order information, which is the order of the unit process in the order of progress, or state information, which is the working state of the process, as the type of defect.

[0012] The reading module may include a photographing module that photographs the process of the object to generate the image data; and a machine learning model that recognizes and reads the object from the image data to generate working data of the process.

[0013] The machine learning model generates working data including the order information, which is the result of reading the order of progress of the process by recognizing that the object changes as each unit process progresses. When the time information of the working data and the instruction data is different, the detection module can generate defect information indicating an order defect.

[0014] The machine learning model generates working data including the state information, which is the result of reading the working state of the process by recognizing the outer shape of the object that is deformed as the process progresses. When the state information of the working data and the instruction data is different, the detection module can generate defect information indicating a state defect.

[0015] The machine learning model detects the working area where the process of the object is performed and generates working data including position information. The detection module can search the working data for the position information corresponding to the generated defect information.

[0016] When the defect information is generated during the execution of the process, the output module inputs the position information retrieved from the working data in real time, identifies the unit process in which the defect information is generated, and can output a signal for the defect information to the working area specified from the position information of the corresponding unit process.

[0017] The above output module can include a display installed in the work area specified from the position information of the unit process in which the above defective information is generated during the above process.

[0018] When there is no normal image data that has already been collected for the above object, it includes a search module that searches for an object associated with the above object, and the similarity with the above image data can be measured using the normal image data that has already been collected for the object associated with the object searched by the above search module.

[0019] An artificial intelligence-based process optimization management method according to an embodiment of the present invention includes a step of inputting instruction data recorded for the optimal implementation determined from among a plurality of implementations in which the reading module changed the order of unit processes; a step of generating image data related to the process performed on the object by the reading module, and generating work data, which is the result of reading the image data, so as to correspond to the instruction data; a step of generating defective information related to the process by comparing the work data transmitted to the detection module with the instruction data; and a step of outputting the defective information by transmitting it to the output module.

[0020] The above reading module can analyze the work data obtained while a plurality of implementations are performed for each product, determine that the work data of the implementation with the minimum required time is the optimal implementation for the corresponding product, and transmit it so as to store the instruction data for the optimal implementation for each product in the database.

Effect of the Invention

[0021] According to the present invention, there are the following effects.

[0022] First, by automatically monitoring and evaluating the progress of the process, it is possible to determine the optimal process order based on the on-site experience of the operator even among unit processes that are not mutually dependent.

[0023] Second, it is possible to determine whether work is being carried out while naturally progressing the process without separate simulation and while maintaining the optimal process order.

[0024] Third, since the vision technology can be utilized to immediately notify the occurrence of defects, the time and cost required for separately performing defect detection can be reduced.

[0025] Fourth, since data is collected for each unit process which is a part of the overall process to detect defects, it is possible to easily identify the unit process in which the defect has occurred.

[0026] However, the effects of the present invention are not limited to the effects described above, and effects not mentioned can also be clearly understood by those having ordinary knowledge in the technical field to which the present invention pertains from the present specification and the accompanying drawings.

Brief Description of the Drawings

[0027]

Figure 1

Figure 2

Figure 3

Modes for Carrying Out the Invention

[0028] An artificial intelligence-based process optimization management system according to an embodiment of the present invention aims to provide a process optimization system capable of determining an optimal process scenario from a plurality of process scenarios with different unit process orders.

[0029] Specifically, the artificial intelligence-based process optimization management system according to an embodiment of the present invention consists of one or more unit processes in which the overall process for manufacturing a product is performed in a series of orders. When the order of the unit processes is changed, the optimal execution is determined from a plurality of executions in which the order of each unit process is changed.

[0030] Based on the process of deriving the optimal execution and the data recorded even after the optimal execution is derived, data is recorded so that it can be detected whether the current ongoing process is being performed successfully, and by reading it with a processor, it can be detected how much deviation there is from the optimal execution.

[0031] The reading module 100 can collect and read image data related to the process performed on the object provided for product manufacturing.

[0032] Here, the object can mean the material and parts for making a product through the manufacturing process.

[0033] That is, the reading module 100 can read the image data of the current ongoing process in accordance with the existing instruction data to generate work data.

[0034] The reading module 100 can include a photographing module 110 and a machine learning model 120.

[0035] The photographing module 110 can photograph the ongoing process of the object to generate image data. The photographing module 110 can include a plurality of cameras installed on the process line, and can transmit the captured image data of the corresponding camera to the machine learning model 120.

[0036] The cameras used by the photographing module 110 can sense the ongoing process of the object through various radio waves such as infrared rays, visible light, X-rays, gamma rays, electromagnetic waves, and ultrasonic waves. The image data can be a two-dimensional image, a three-dimensional image, a pre-processed ROI image, a cropped image, etc.

[0037] The machine learning model 120 can recognize and read the object in the image data to generate the work data of the progress process.

[0038] The machine learning model 120 can recognize that the object is changed while each unit process is being performed, read the progress order of the process, and generate work data including order information regarding such progress order.

[0039] The machine learning model 120 can recognize the object of the corresponding unit process through the image data generated by photographing the unit process while a series of unit processes are being performed, and thereby the type of the unit process can be specified. If the object is completely changed, since the unit process is also specified as different, after the previous unit process is completed, it can be read that the next unit process is performed. The machine learning model 120 can grasp the order until all the unit processes are performed by the above method, specify the type of the process implementation of the whole process, and generate work data including the order information of the corresponding process implementation.

[0040] Similarly, the machine learning model 120 can recognize the outer shape of the object that is deformed as the process progresses, read the working state of the process, and generate work data including state information regarding such working state.

[0041] That is, when the object is specified, the machine learning model 120 can read the working state of the unit process through the degree of deformation of the outer shape of the object recognized through the image data before and after the progress of the corresponding unit process. The outer shape can include not only the shape but also the hue, surface, interior, etc.

[0042] For example, when the process is an assembly operation targeting a component, the result of the first unit process performed on the first component, which is the object, can be read through the degree of deformation of the outer shape of the component as a result of the assembly operation. After the machine learning model 120 recognizes the object through the image data, it can measure numerically the working state, which is the direction in which the first component is assembled or the assembly strength, and generate working data with the recorded state information.

[0043] Alternatively, the machine learning model 120 can generate working data using, as state information, a numerical value obtained by measuring the similarity by comparing the image data related to the object with the already collected normal assembly image data.

[0044] Also, when there is no already collected normal image data for the object, the machine learning model 120 can measure the similarity with the image data using the normal image data already collected for the object associated with the object searched by the search module that searches for the object associated with the object.

[0045] And the machine learning model 120 can detect the working area where the process of the object is performed and generate working data including position information.

[0046] Here, the working area may mean the place where a specific unit process in a kind of production line is performed, or may mean a part of the area within the equipment where the process is performed.

[0047] Of course, the position can be obtained by the GPS module that may be included in the machine learning model 120, and working data including position information can be generated.

[0048] Thereby, the defective information that may exist in the previously generated ones may be matched with the position information and stored as working data, and it becomes possible to search for defective information related to the defect that occurred in the corresponding working area from the position information.

[0049] In addition, the machine learning model 120 can recognize that the object is changed while the process is being performed, read the required time of the process, and measure the error from the required time of the optimal process implementation by comparing the time information, which is the required time, with the time information on the instruction data. Thereafter, by the detection module 300, the corresponding error can be evaluated as the proficiency level of the operator, excluding environmental factors such as the temperature and humidity of the area where the operation is performed.

[0050] On the other hand, the reading module 100 can collect sensing data corresponding to quality factors or external factors using a sensing module, and read about the process of the object based on a preset standard.

[0051] The sensing module can collect external sensing data through various common sensors such as a microphone, proximity sensor, ultrasonic sensor, gyro sensor, vibration sensor, temperature and humidity sensor, pressure sensor, impact sensor, and gas sensor.

[0052] And the quality factors can be the pressure applied to the object, the moving speed of the object, vibration, temperature, humidity, specific gravity of the object, shrinkage, strength, weather environment, lighting environment, equipment life, equipment information, operator proficiency, material characteristics, number of cycles, etc.

[0053] The reading module 100 can collect sensing data for general process optimization, and can utilize regression analysis or machine learning to clarify the relationship between the quality variables affected by the collected sensing data among the set quality variables.

[0054] When a functional relationship is established between the quality factors and the quality variables, the quality factors for which the output variables, which are the quality variables, become the desired values can be obtained as input variables using an optimization algorithm, and in the case of the input variable values for the optimal implementation, this can be reflected in the instruction data.

[0055] The detection module 200 can read the work data transmitted from the reading module 100, compare it with the instruction data, determine whether the process is normal or defective, and generate defective information in case of a defect.

[0056] Here, the defective information may include defects with respect to the progress order as the type of defect, defects with respect to the working state, defects with respect to the required time, and the like.

[0057] The detection module 200 can compare the work data with the instruction data, compare one or more of the types of defects, and generate defective information that it is an order defect, a state defect if the working state is defective, or a time defect if the required time is defective.

[0058] Preferably, for the working state, there may be a preset error range or similarity criterion for the work to distinguish between the normal case and the defective case.

[0059] The detection module 200 compares the numerical values of the state information on the work data with the state information on the instruction data, and if it is determined to be within the preset error range, it detects it as normal, and if it is determined to be outside the error range, it can generate defective information that it is a state defect.

[0060] The detection module 200 can analyze the sensing data for the unit process in which a defect has occurred, compare it with the quality factor by the optimal implementation on the instruction data, find and record the type of quality factor that causes the defect, and can store it together with the defective information.

[0061] The output module 300 can output the defective information generated by the detection module 200.

[0062] When defective information is generated during the process, the output module 300 can receive the position information retrieved from the work data in real time. That is, in the work data, a signal for the defective information can be output to the work area on the position information of the corresponding unit process where the defect has occurred.

[0063] Here, the signal can be a visual, auditory, tactile signal, etc., but is not particularly limited.

[0064] For example, the output module 300 can be installed in the work area specified from the position information of the unit process in which defective information is generated during the process, and includes a display that outputs a signal visually notifying the operator that a defect has occurred, an acoustic module that gives an alarm by sound, and a vibration module that is attached to the operator and outputs vibration.

[0065] The output module 300 can be equipped with an input module as needed, and information regarding the occurrence of non-standard defects can be input from the operator.

[0066] Also, the output module 300 can include a worker terminal that is preset to work around the work area specified from the position information of the unit process in which defective information is generated during the process, or can be sensed by the system.

[0067] The output module 300 can output quality factors estimated to be the cause of the defect together with the defective information.

[0068] For example, in the case of an assembly process, if the work data including the state information which is the numerical value of the assembly fastening strength measured by the reading module 100 is generated when the assembly fastening strength of the parts becomes weak in the second unit process, the detection module 200 compares the work data with the instruction data and detects a work state defect, and defective information regarding this can be generated. The output module 300 can output the defective information to the work area where the corresponding defect has occurred, and can output the quality factors estimated to be the cause of the defect together if necessary.

[0069] On the other hand, the reading module 100 can generate the above instruction data.

[0070] As an example, the processes for a plurality of products can be divided into a plurality of unit processes, and preferably, the operation data obtained while performing the processes a plurality of times by changing the order of the corresponding unit processes is analyzed, and it can be determined that the operation data of the execution with the minimum required time is the optimized process order of the corresponding product.

[0071] The reading module 100 can be transmitted to store the instruction data for the optimal execution for each such product in the database.

[0072] Hereinafter, a method for the reading module 100 to generate instruction data will be described in detail.

[0073] The reading module 100 can further include a determination module 130.

[0074] Based on the execution data, the determination module 130 can generate instruction data with the execution data for the optimal execution determined from among the plurality of executions performed by the determination module by changing the order of the unit processes.

[0075] While a series of unit processes are being performed, the execution data can be collected by evaluating each unit process according to the evaluation criteria and accumulating the evaluations in the order of the unit processes.

[0076] Here, the above-described operation data and execution data can be information generated by the same range, but in the method of generating instruction data, it is described as execution data in order to distinguish it from the data generated by the reading module 100.

[0077] For example, a unit process can mean one of the component assembly processes of a plurality of steps for manufacturing one product.

[0078] Here, the evaluation criteria are factors that affect the yield, quality, etc. of the process, such as the required time and the defect rate, and can be set as necessary.

[0079] The reading module 100 can identify the type of unit process through the information collected from the unit process during a series of unit processes, understand the progress order of the unit process, and identify the type of process execution for which process execution it is based on the progress order of the unit process.

[0080] Here, process execution means a kind of scenario attempted by changing the order of unit processes. Even for the same unit process, the process scenario is different due to different orders. In the present invention, in order to determine the optimal process execution from among process executions with different progress orders of unit processes, information is collected from the object to identify the unit process, understand the order, and map it to the execution data for each process execution.

[0081] For example, while advancing the unit process, the required time can be evaluated from the time when the object is determined and the unit process is identified until the corresponding unit process is completed. At the same time or at a different time, the completion status of the unit process can be grasped to evaluate the presence or absence of defects.

[0082] After that, when the next unit process is performed, the required time and the presence or absence of defects can be evaluated in the same way. At the same time or at a different time, the required time and the presence or absence of defects from the start of the entire process to the present can be accumulated and evaluated to collect execution data.

[0083] Here, the completion status of the unit process can be evaluated that the corresponding unit process is completed when the object is changed, and can also be grasped based on data related to the fastening degree, fastening direction, detachment, etc. of the assembled parts.

[0084] More appropriately, there can be an overall process list or list for the product manufactured using the object. When the reading module 100 is input with this and evaluates each unit process of the overall process list every time the unit process of one object is completed, and all the unit processes of the overall process list are evaluated, it is determined that the overall process or one execution is completed, and the collection can be terminated while inputting the collected execution data to the determination module 130.

[0085] In addition, the search module included in the reading module can search for a list including the object first recognized by the machine learning model in a database in which various types of overall process lists are collected, and input it to the machine learning model 120.

[0086] Here, based on the information collected from the unit process using the machine learning model 120, an object can be determined, and the completion, defect, or type of the corresponding unit process can be specified.

[0087] At this time, the machine learning model 120 is continuously improved by learning an image related to the object using any one or more object detection algorithms of vision fitting, edge, color, and location via the installed application, and can specify the type of unit process, recognize the assembly and missing states of the object, and determine the defect or completion of the corresponding unit process.

[0088] Specifically, the machine learning model 120 uses deep learning or machine learning, which is an algorithm artificial intelligence technology that classifies or learns the features of the corresponding image data by itself, to learn and improve an image in which the shape, hue, etc. of the object are represented, classify and detect the object, and specify the type of unit process.

[0089] The machine learning model 120 can process data by a processor, and can preprocess the image data of the object to generate learning data when detecting the object.

[0090] Here, the learning data may be labeled with the preprocessed image data, the assembly state, the defective state, and the unit process type information for the object.

[0091] Information about the object to be labeled may be collected based on the operator's feedback. That is, if a large number of defective cases of the object are not initially found, the component information of the object can be input using the input interface and labeled with the corresponding image data, or the assembly completion status, defect presence or absence, etc. can be input for labeling.

[0092] The classifier model can be trained using the labeled data and applied to the machine learning model 120.

[0093] Then, after preprocessing the image data collected via the camera during the performance of the unit process, the object and the presence or absence of defects in the object, the assembly completion status, etc. can be detected using the classifier model.

[0094] On the other hand, in one embodiment of the present invention, normal image data collected when the unit process for the object is performed normally without defects can be collected in advance.

[0095] The machine learning model 120 measures the similarity between the newly collected image data and the above normal image data, reads the image data as normal image data using a preset similarity criterion, and can collect a new normal image as trial data.

[0096] Features can be extracted from the normal image data collected as needed, and the machine learning model 120 can be trained only with the normal image data.

[0097] At this time, when the product is improved or newly launched, some or all of the components or processes of the object used in the product may be different from the existing product. Therefore, there may be no previously collected image data for the object.

[0098] The reading module 100 measures the similarity between the image data of the object of the improved product taken by using the normal image data collected by the machine learning model 120 for the previous version of the product and the image data of the object of the improved product in order to immediately determine the process of the improved new product without collecting the image data, so that the normal image data for the process of the improved new product can be collected.

[0099] In addition to the above description, the machine learning model 120 may utilize general deep learning vision or machine vision (MV), and may be equipped with hardware, software, or an interface used for surface defect inspection of wafers, display products, PCB defect inspection, LED chip packaging, and inspection of other products in the factory automation process of the industrial body.

[0100] The process of collecting the trial data by the above reading module 100 can be repeated until the progress of all unit processes is completed.

[0101] The determination module 130 determines the optimal process implementation from among a plurality of process implementations based on the collected implementation data.

[0102] Specifically, when the implementation data for the process implementation specified by the reading module 100 and the implementation data of each of the plurality of process implementations are collected by the reading module 100, the determination module 130 can utilize this to determine whether the specified process implementation is the optimal process implementation.

[0103] According to an embodiment of the present invention, any one of the optimal process implementations may be the case where the total sum of the required times for all unit processes is the minimum.

[0104] That is, each time a unit process is performed, the execution data measured against the evaluation criteria of the required time and defect rate of each unit process can be collected using the machine learning model 120 of the reading module 100. When the total required time of the unit process in any process execution is the minimum time on the execution data, the corresponding process execution can be determined as the optimal process execution by the determination module 130.

[0105] In addition, the working state of each process can be jointly confirmed by the above-mentioned machine learning model 120 to calculate the defect rate, which can be reflected in the evaluation.

[0106] On the other hand, the execution data collected from the reading module 100 can be labeled and stored for each process execution to construct a big data database for process execution. The big data database can be constructed by managing the execution data including the required time and defect rate of the process execution in a history.

[0107] In the above big data database, the responsible operator information can be labeled and stored together with the execution data during the corresponding process execution.

[0108] After that, the determination module 130 can generate instruction data with the execution data for the optimal process execution. Specifically, the existing work instruction format can be obtained from the MES (Manufacturing Execution System) system, and the optimal process execution can be merged into the corresponding format to generate instruction data with a new work instruction. The new work instruction can be output via an output module 300 such as the display of the work table where the corresponding process is performed.

[0109] If the operator information is reflected and collected in the execution data for the process execution performed with a different operator, the error in the required time and defect rate of the process execution due to the difference in the proficiency of the operator can be reduced, and the reproduction rate of the work instruction reflecting the optimal process execution can be increased.

[0110] In other words, when information regarding the proficiency level of an operator, etc. is reflected in the execution data, the determination module 130 can determine the optimal process execution for each proficiency level by distinguishing between the execution data of the process execution by a skilled operator and the execution data of the process execution by an unskilled operator.

[0111] Preferably, by separately managing the execution data so as to be different for each individual operator, an optimal process execution can be proposed for each operator.

[0112] FIG. 3 shows the process in which an artificial intelligence-based process optimization management system according to an embodiment of the present invention is implemented.

[0113] Hereinafter, with reference to FIG. 3, an embodiment of an artificial intelligence-based process optimization system will be described.

[0114] FIG. 3 shows the process in which the video data collected by the imaging module 110 is generated and collected as execution data by the machine learning model 120 of the reading module 100 while the process of assembling part B to part A is being performed.

[0115] As shown in FIG. 3, in the above-described artificial intelligence-based process optimization management system, the machine learning model 120 first identifies the unit process type A while detecting part A, which is the object, and the reading module 100 can search for a list (A, B, C, D) of all the parts of the product that includes the object of the identified unit process A.

[0116] Then, it is confirmed whether the unit process A identified via the machine learning model 120 is completed.

[0117] The reading module 100 can collect the execution data of the unit process A by evaluating in real time the time required until the identified unit process A is completed and the presence or absence of defects.

[0118] When the next unit process B is performed, it can be detected and recognized through the machine learning model 120 that the object has been changed. That is, the machine learning model 120 can grasp and determine whether the object has been changed, such as when a new part B is added to the object and its shape, hue, etc. are changed, through an image.

[0119] Similarly, the reading module 100 can collect implementation data by evaluating in real time whether the unit process B specified by the changed object is completed, the required time, and whether there are defects.

[0120] In addition, the reading module 100 can record and store the fact that the unit process A to the unit process B have been performed in order.

[0121] When the unit process B is completed, the reading module 100 can collect implementation data by accumulating and evaluating in real time the required time from the beginning to the present, whether there are defects, etc.

[0122] The above process can be carried out until the progress of a series of unit processes ends when all the parts (A, B, C, D) of the product are assembled.

[0123] Here, the implementation of multiple processes may be in the unit process permutation. If the overall part list is A, B, C, D, then there are 4! types of process implementations in the order of A - B - C - D, A - B - D - C, A - C - B - D, A - C - D - B, A - D - B - C, A - D - C - B, B - A - C - D, B - A - D - C,..., D - C - B - A.

[0124] Each process implementation may be evaluated through the above process and implementation data may be collected. When a certain type of implementation is repeated, the required time or whether there are defects for the corresponding process implementation can be accumulated, collected, and stored.

[0125] Of course, it is not necessary to evaluate all process implementations. There may be cases where the progress is impossible or some types of processes are not evaluated by the operator's choice.

[0126] When sufficient execution data is accumulated by the machine learning model 120, the determination module 130 can determine the optimal process execution from the process executions evaluated based on the collected execution data.

[0127] For example, as shown in Table 1 below, when the determination module 130 performs the A-B-C-D process execution 30 times, with an average required time of 1 minute and 20 seconds, an average error of 10 seconds, and a defect rate of 5%, and the B-C-A-D process execution is performed 25 times, with an average required time of 1 minute and 40 seconds, an average error of 30 seconds, and a defect rate of 7%, if the preset condition gives priority to the minimum required time, A-B-C-D with the minimum required time can be determined as the optimal process execution.

[0128]

Table 1

[0129] The artificial intelligence-based process optimization system according to an embodiment of the present invention can automatically monitor and evaluate the progress of the process, thereby determining the optimal process order even among unit processes that are not mutually dependent, based on the on-site experience of the operator, and can determine the optimal process execution while naturally advancing the process without separate simulation.

[0130] FIG. 2 shows a flowchart of an artificial intelligence-based process optimization management method according to an embodiment of the present invention.

[0131] The artificial intelligence-based process optimization management method according to an embodiment of the present invention may include a step of inputting instruction data recorded for the optimal execution determined from among a plurality of executions performed by changing the order of unit processes by a reading module; a step of generating image data related to the process performed on the object by the reading module and generating work data, which is the result of reading the image data, to correspond to the instruction data; a step of generating defect information related to the process by a detection module that transmits the work data and compares it with the instruction data; and a step of outputting the defect information by an output module that transmits the defect information.

[0132] Throughout this specification, a machine learning model, a deep learning-based model, an arithmetic model, a neural network, a network function, a deep neural network, and a neural network can be used interchangeably with the same meaning.

[0133] A deep neural network (DNN) can mean a neural network that includes multiple hidden layers in addition to an input layer and an output layer. By using a deep neural network, the latent structures of data can be grasped. That is, the latent structures of photos, articles, videos, voices, and music (for example, what objects are in a photo, what the content and emotion of an article are, what the content and emotion of a voice are, etc.) can be grasped. The deep neural network can include a convolutional neural network (CNN), a recurrent neural network (RNN), an auto encoder, a GAN (Generative Adversarial Networks), a restricted boltzmann machine (RBM), a deep belief network (DBN), a Q-network, a U-network, a Sham network, a Generative Adversarial Network (GAN), and the like. The description of the deep neural network above is only an example, and the present disclosure is not limited thereto.

[0134] In one embodiment of the present disclosure, the network function can also include an autoencoder. The autoencoder can be a type of artificial neural network for outputting output data similar to the input data. The autoencoder can include at least one hidden layer, and an odd number of hidden layers can be arranged between the input and output layers. The number of nodes in each layer can be reduced from the number of nodes in the input layer to an intermediate layer called the bottleneck layer (encoding), and then symmetrically expanded from the bottleneck layer to the output layer (symmetric to the input layer). The autoencoder can also perform non-linear dimensionality reduction. The number of input and output layers can correspond to the dimensions after preprocessing the input data. The number of nodes in the hidden layer included in the encoder within the autoencoder structure can have a structure that decreases as it is farther from the input layer. If the number of nodes in the bottleneck layer (the layer with the fewest nodes located between the encoder and the decoder) is too small, there is a possibility that a sufficient amount of information will not be transmitted, so it can be maintained above a certain number (for example, more than half of the input layer, etc.).

[0135] The neural network can be trained in at least one of supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. The training of the neural network can be a process of applying knowledge for the neural network to perform a specific operation to the neural network.

[0136] A neural network can be trained in a direction that minimizes the error of the output. In neural network training, training data is repeatedly input into the neural network, the error between the output of the neural network for the training data and the target is calculated, and the error of the neural network is backpropagated from the output layer to the input layer in a direction to reduce the error, and the weights of each node in the neural network are updated. In the case of supervised learning, learning data with correct labels is used for each piece of training data (i.e., labeled learning data), and in the case of unsupervised learning, the correct answer may not be labeled for each piece of training data. That is, for example, the training data in the case of supervised learning related to data classification can be data in which each piece of training data is labeled with a category. The error can be calculated by inputting the labeled learning data into the neural network and comparing the output (category) of the neural network with the label of the learning data. As another example, in the case of unsupervised learning related to data classification, the error can be calculated by comparing the input training data with the neural network output. The calculated error is backpropagated in the reverse direction (i.e., from the output layer to the input layer) in the neural network, and the connection weights of each node in each layer of the neural network can be updated by backpropagation. The change amount of the connection weight of each node to be updated can be determined by the learning rate. The calculation of the neural network for the input data and the backpropagation of the error can constitute an epoch. The learning rate can be applied to vary depending on the number of repetitions of the neural network training epoch. For example, at the initial stage of neural network training, a high learning rate can be used to enhance efficiency by enabling the neural network to quickly secure a predetermined level of performance, and at the later stage of training, a low learning rate can be used to improve accuracy.

[0137] In neural network learning, generally, the training data may sometimes be a subset of the actual data (i.e., the data to be processed using the learned neural network). Therefore, there may be a learning cycle in which the error of the training data decreases, but the error increases for the actual data. Overfitting is a phenomenon in which the neural network over-learns the training data and the error for the actual data increases. For example, a neural network that has learned cats by showing yellow cats may not be able to recognize that other non-yellow cats are also cats, which can be a type of overfitting. Overfitting can act as a cause for increasing the error of machine learning algorithms. Various optimization methods can be used to prevent such overfitting. To prevent overfitting, methods such as increasing the training data, regularization, dropout (deactivating some of the network nodes during the learning process), and utilization of a batch normalization layer can be applied.

[0138] Although the present disclosure has been described as generally being implemented by a computing device, those skilled in the art will understand that the present disclosure can be implemented in combination with computer-executable instructions and / or other program modules that can be executed on one or more computers and / or in a combination of hardware and software.

[0139] In general, a program module includes routines, programs, components, data structures, etc. that perform a particular task or implement a particular abstract data type. Also, those skilled in the art will understand that the methods of the present disclosure can be implemented in other computer system configurations, including single-processor or multi-processor computer systems, minicomputers, mainframe computers, of course, personal computers, handheld computing devices, microprocessor-based or programmable household appliances, etc. (each of which can operate connected to one or more associated devices).

[0140] The described embodiments of the present disclosure can also be implemented in a distributed computing environment where any task is performed by a remote processing device connected via a communication network. In a distributed computing environment, program modules can be located in both local and remote memory storage devices.

[0141] Computers typically include a variety of computer-readable media. Media accessible by a computer can be any kind of computer-readable media, and such computer-readable media can include volatile and non-volatile media, transitory and non-transitory media, removable and non-removable media. By way of example and not limitation, computer-readable media can include computer-readable storage media and computer-readable transmission media. Computer-readable storage media are volatile and non-volatile media, transitory and non-transitory media, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media include RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, DVD (digital video disk) or other optical disk storage devices, magnetic cassettes, magnetic tape, magnetic disk storage devices or other magnetic storage devices, or any other media that can be used to store desired information accessible by a computer, but are not limited thereto.

[0142] The various embodiments presented herein can be embodied as a method, an apparatus, or an article of manufacture using standard programming and / or engineering techniques. The term "article of manufacture" includes a computer program, carrier, or media accessible from any computer-readable storage device. For example, computer-readable storage media include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, etc.), optical disks (e.g., CDs, DVDs, etc.), smart cards, and flash memory devices (e.g., EEPROMs, cards, sticks, key drives, etc.). Further, the various storage media presented herein include one or more devices and / or other machine-readable media for storing information.

[0143] The specific order or hierarchical structure of the steps in the process presented is an example of an exemplary approach. It should be understood that, based on the design priorities, the specific order or hierarchical structure of the steps in the process can be rearranged within the scope of the present disclosure. The appended method claims provide the elements of the various steps in a sample order, but are not meant to be limited to the specific order or hierarchical structure presented.

[0144] The description of the presented embodiments is provided so that any person of ordinary skill in the art of the present disclosure can make use of or implement the present disclosure. Various modifications to these embodiments will be apparent to those of ordinary skill in the art of the present disclosure, and the general principles defined herein can be applied to other embodiments without departing from the scope of the present disclosure. Accordingly, the present disclosure is not limited to the embodiments presented herein, but is to be accorded the widest scope consistent with the principles and novel features presented herein.

Description of Symbols

[0145] 100 Reading Module 110 Photographing Module 120 Machine Learning Model 130 Determination Module 200 Detection Module 300 Output Module

Claims

1. When input with instruction data recorded for the optimal operation determined from a plurality of operations performed by changing the order of unit operations that are part of the process for manufacturing a product, a reading module that generates work data, which is the result of reading image data regarding the process performed on the object, to correspond to the instruction data; A detection module that receives the work data from the reading module, compares it with the instruction data, and detects whether the process is normal or defective; and, An output module that outputs defect information, The instruction data is The overall process for manufacturing a product consists of one or more unit operations performed in a series of orders. The unit operations are performed with the order changed, and while each operation is being performed, the reading module evaluates each unit operation according to evaluation criteria, accumulates the evaluations in the order of the unit operations, collects the generated work data, and transmits it to the determination module. Based on the work data of the reading module, the determination module generates data generated from the work data for the optimal operation determined from among a plurality of operations performed with the order of the unit operations changed, The optimal operation is determined by the determination module to be the operation with the minimum total required time or defect rate of the unit operations recorded in the work data. The detection module is When the detection result is defective, it generates defect information for the process, The output module outputs the defect information, The defect information is Generated by comparing one or more of the order information, which is the order of progress of the unit operation, or the state information, which is the working state of the process, as the type of defect, Also, the reading module is A photographing module that photographs the progress process of the object to generate the image data; and, A machine learning model that recognizes and reads the object from the image data to generate work data of the progress process, Also, the machine learning model is As each unit operation progresses, it recognizes that the object is changed and generates work data including the order information read for the progress order of the process, The detection module is When the order information of the work data and the instruction data is different, it generates defect information indicating an order defect An artificial intelligence-based process optimization management system characterized by this.

2. The machine learning model is Generate work data including the state information obtained by recognizing the outer shape of the object to be deformed as the process progresses and reading the working state of the process. The detection module: When it is determined that the state information of the work data and the instruction data is similar using a preset error criterion, it detects normal; when they are different, it detects a defective state and generates defect information. The artificial intelligence-based process optimization management system according to claim 1.

3. The machine learning model: Detect the work area where the process of the object is performed and generate work data including position information. The detection module: Search for the position information corresponding to the generated defect information from the work data. The artificial intelligence-based process optimization management system according to claim 2.

4. The output module: When the defect information is generated during the process, input the position information retrieved from the work data in real time, identify the unit process in which the defect information is generated, and output a signal for the defect information to the work area specified from the position information of the corresponding unit process. The artificial intelligence-based process optimization management system according to claim 3.

5. The output module: Includes a display installed in the work area specified from the position information of the unit process in which the defect information is generated during the process. The artificial intelligence-based process optimization management system according to claim 4.

6. When there is no normal image data already collected for the object, includes a search module that searches for an object associated with the object, and measures the similarity with the image data using the normal image data already collected for the object associated with the object searched by the search module. Measure the similarity with the image data using the normal image data already collected for the object associated with the object searched by the search module. The artificial intelligence-based process optimization management system according to claim 2.

7. A step of inputting the instruction data recorded for the optimal work determined from among a plurality of operations performed by changing the order of unit processes by the reading module; A step of generating image data related to the process performed on the object by the reading module and generating work data, which is the result of reading the image data, to correspond to the instruction data; A step of generating defect information related to the process by the detection module by transmitting the work data and comparing it with the instruction data; and A step of outputting by transmitting the defect information by the output module. The instruction data is composed of one or more unit processes in which the entire process for manufacturing a product is performed in a series of orders, the unit processes are worked in a changed order, and while each operation is being performed, the reading module evaluates each unit process according to evaluation criteria, collects the work data generated by evaluating while accumulating in the order of the unit processes, and transmits it to the determination module. Based on the work data of the reading module, it is data generated from the work data for the optimal work determined by the determination module from among a plurality of operations performed with the order of the unit processes changed. The optimal work is determined by the determination module to be the work with the minimum total required time or defect rate of the unit processes recorded in the work data. The detection module generates defect information for the process when the detection result is defective. The output module outputs the defect information. The defect information is generated by comparing any one or more of the order information which is the order of progress of the unit process or the state information which is the working state of the process as the type of defect. Also, the reading module a photographing module that photographs the progress process of the object to generate the image data; and includes a machine learning model that recognizes and reads the object from the image data to generate work data of the progress process. Also, the machine learning model recognizes that the object is changed as each unit process progresses, and generates work data including the order information obtained by reading the order of progress of the process. The detection module generates defect information indicating that there is an order defect when the order information of the work data and the instruction data is different. An artificial intelligence-based process optimization management method characterized by the above.

8. The reading module transmits to store the instruction data for the optimal work for each product in a database. The artificial intelligence-based process optimization management method according to Claim 7.

Images