Multi-agent-based system and method for autonomous operation of manufacturing process

Abstract

The present disclosure provides a multi-agent-based system for autonomous operation of a manufacturing process. The system comprises: a coordinating agent for generating a command for a task associated with a manufacturing process and transmitting the command to any one of a plurality of sub-agents; a first sub-agent for generating context information associated with the task by using a first machine learning model; a second sub-agent for generating action plan information based on the context information by using a second machine learning model; a third sub-agent for generating result information by performing the task on the basis of the action plan information; and a fourth sub-agent for generating evaluation information obtained by evaluating a result of the task on the basis of the result information.

Description

Multi-agent-based manufacturing process autonomous operation system and method

[0001] The present disclosure relates to a multi-agent-based manufacturing process autonomous operation system and method.

[0002] Manufacturing processes can involve various procedures, such as process control, quality monitoring, and safety management. Consequently, operating a manufacturing process requires a significant amount of manpower and time. In particular, problems in manufacturing process operations may arise due to variables such as human error, differences in competence among field engineers, and variations in work methods.

[0003] The present disclosure provides a multi-agent-based manufacturing process autonomous operation system (device) and a method for solving the above-mentioned problems.

[0004] A multi-agent-based manufacturing process autonomous operation system according to one embodiment of the present disclosure may include a coordinating agent that generates a command for a task associated with a manufacturing process and transmits it to one of a plurality of sub-agents; a first sub-agent that generates situation information associated with the task using a first machine learning model; a second sub-agent that generates action plan information based on the situation information using a second machine learning model; a third sub-agent that generates result information by performing a task based on the action plan information; and a fourth sub-agent that generates evaluation information by evaluating the result of the task based on the result information.

[0005] According to one embodiment of the present disclosure, a coordination agent can update a command for a task based on evaluation information.

[0006] According to one embodiment of the present disclosure, the system further includes a natural language processing unit that receives a query including natural language associated with a task and generates an embedding vector associated with the query, and a coordination agent can generate a command for the task based on the embedding vector.

[0007] According to one embodiment of the present disclosure, a coordination agent generates a result associated with a task based on evaluation information, and a natural language processing unit can generate a response including natural language based on the result.

[0008] According to one embodiment of the present disclosure, the system further includes a fifth sub-agent that monitors data associated with a manufacturing process to generate monitoring information, and the coordination agent can generate a command for a task based on the monitoring information.

[0009] According to one embodiment of the present disclosure, the first machine learning model can collect base data associated with a task from at least one of equipment or a database associated with a manufacturing process.

[0010] According to one embodiment of the present disclosure, the database may include a first database storing a vector containing text or an image associated with a manufacturing process, a second database storing structured data associated with a manufacturing process, and a third database storing a knowledge graph associated with a manufacturing process.

[0011] According to one embodiment of the present disclosure, each of the first machine learning model and the second machine learning model may include at least one of a fine-tuned language model or a vision language model associated with a manufacturing process.

[0012] According to one embodiment of the present disclosure, a second machine learning model may be provided. It may be trained to take task and situation information as inputs and output action plan information for performing a task.

[0013] According to one embodiment of the present disclosure, a second sub-agent can generate action plan information based on situation information, tasks, numerical data associated with a manufacturing process, and environmental data associated with a manufacturing process.

[0014] According to one embodiment of the present disclosure, a third sub-agent may perform a simulation associated with a task based on action plan information and generate result information of the task based on the result of the simulation.

[0015] According to one embodiment of the present disclosure, a third sub-agent can generate an execution command associated with a task based on action plan information, transmit the execution command to equipment associated with the task, and generate result information of the task based on the execution result of the task based on the execution command.

[0016] According to one embodiment of the present disclosure, the fourth sub-agent can determine whether the task goal has been achieved based on result information and generate evaluation information based on whether the task goal has been achieved.

[0017] According to one embodiment of the present disclosure, if the goal of the task is not achieved, the fourth sub-agent may send a request to the second sub-agent to modify the action plan information.

[0018] According to one embodiment of the present disclosure, each of the first to fourth sub-agents may include at least one of a planning unit that analyzes a situation related to a manufacturing process and establishes an action plan based on input data using a machine learning model, a control unit that controls at least one of equipment or systems related to the manufacturing process based on the action plan, a storage unit that stores input data and an action plan, and a utility unit that assists the control unit.

[0019] According to one embodiment of the present disclosure, a task is associated with the production of a good product of a first equipment in a manufacturing process, and a first sub-agent generates situation information associated with the product of the first equipment based on real-time data collected from the first equipment, a second sub-agent generates action plan information associated with the production recipe of the first equipment based on the situation information associated with the product of the first equipment, a third sub-agent generates result information regarding the product of the first equipment based on the action plan information associated with the production recipe of the first equipment, and a fourth sub-agent can generate evaluation information evaluating the result of the production of a good product of the first equipment based on the result information regarding the product of the first equipment.

[0020] According to one embodiment of the present disclosure, the task is associated with the movement of a second piece of equipment in a manufacturing process, and a first sub-agent generates situational information associated with the environment of the second piece of equipment based on real-time data collected from the second piece of equipment, a second sub-agent generates action plan information associated with the movement path of the second piece of equipment based on the situational information associated with the environment of the second piece of equipment, a third sub-agent controls the second piece of equipment to move the second piece of equipment based on the action plan information associated with the movement path of the second piece of equipment to generate result information associated with the location of the second piece of equipment, and a fourth sub-agent can generate evaluation information evaluating the movement result of the second piece of equipment based on the result information associated with the location of the second piece of equipment.

[0021] According to one embodiment of the present disclosure, the task is associated with establishing a production plan for a manufacturing process, and a first sub-agent collects data on a product associated with the production plan to generate situational information, a second sub-agent generates action plan information including a plurality of production plans based on the situational information, a third sub-agent performs a simulation of the plurality of production plans based on the action plan information, and a fourth sub-agent can generate evaluation information that evaluates the results of the simulation of the plurality of production plans.

[0022] A multi-agent-based manufacturing process autonomous operation method according to one embodiment of the present disclosure may include the steps of: generating a command for a task associated with a manufacturing process and transmitting it to one of a plurality of sub-agents; generating situation information associated with the task using a first machine learning model; generating action plan information based on the situation information using a second machine learning model; generating result information by performing a task based on the action plan information; and generating evaluation information that evaluates the result of the task based on the result information.

[0023] A computer program stored on a computer-readable recording medium may be provided to execute a method according to one embodiment of the present disclosure on a computer.

[0024] According to one embodiment of the present disclosure, all intellectual tasks that can be performed by an operator can be performed autonomously by utilizing a plurality of agents. In addition, by the interaction of a plurality of agents, the manufacturing process can be operated autonomously through appropriate responses even if changes occur in the process environment, products, or personnel.

[0025] According to one embodiment of the present disclosure, queries and tasks can be easily entered in natural language, even if programming language or manufacturing expertise is required. Accordingly, operator intervention can be minimized.

[0026] The effects of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art to which the present disclosure pertains (referred to as "person skilled in the art") from the description in the claims.

[0027] Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, wherein similar reference numerals indicate similar elements, but are not limited thereto.

[0028] FIG. 1 shows an example of an autonomous manufacturing process operating system according to one embodiment of the present disclosure.

[0029] FIG. 2 is a schematic diagram showing a configuration in which an information processing system is connected to communicate with a plurality of user terminals to operate a manufacturing process according to one embodiment of the present disclosure.

[0030] FIG. 3 is a block diagram showing the internal configuration of a user terminal and an information processing system according to one embodiment of the present disclosure.

[0031] FIG. 4 is a diagram illustrating an example of performing a task based on user input according to one embodiment of the present disclosure.

[0032] FIG. 5 is a diagram illustrating an example of performing a task based on process monitoring according to one embodiment of the present disclosure.

[0033] FIG. 6 is a drawing showing an example of the internal configuration of an agent according to one embodiment of the present disclosure.

[0034] FIG. 7 is a diagram illustrating an example of a method for improving the production of defective products in a manufacturing process using an autonomous operating system according to one embodiment of the present disclosure.

[0035] FIG. 8 is a drawing illustrating an example of a method in which an autonomous manufacturing process operating system moves equipment according to one embodiment of the present disclosure.

[0036] FIG. 9 is a diagram illustrating an example of a method for a manufacturing process autonomous operation system to generate a production plan according to one embodiment of the present disclosure.

[0037] FIG. 10 is a flowchart illustrating an example of a multi-agent-based manufacturing process autonomous operation method according to one embodiment of the present disclosure.

[0038] Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the attached drawings. However, in the following description, specific descriptions regarding widely known functions or configurations will be omitted if there is a risk that the gist of the present disclosure may be unnecessarily obscured.

[0039] In the attached drawings, identical or corresponding components are assigned the same reference numerals. Additionally, in the description of the following embodiments, the description of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.

[0040] The advantages and features of the disclosed embodiments and the methods for achieving them will become clear by referring to the embodiments described below in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below but may be implemented in various different forms, and the embodiments provided are merely to make the present disclosure complete and to fully inform those skilled in the art of the scope of the invention.

[0041] The terms used in this specification will be briefly explained, and the disclosed embodiments will be described in detail. The terms used in this specification have been selected to be as generally used as possible, taking into account their functions in this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this disclosure should be defined not merely by their names, but based on their meanings and the content throughout this disclosure.

[0042] In this specification, singular expressions include plural expressions unless the context clearly specifies them as singular. Additionally, plural expressions include singular expressions unless the context clearly specifies them as plural. Throughout the specification, when a part is described as including a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0043] Additionally, the terms 'module' or 'part' as used in the specification refer to software or hardware components, and the 'module' or 'part' performs certain roles. However, the meaning of 'module' or 'part' is not limited to software or hardware. The 'module' or 'part' may be configured to reside in an addressable storage medium or configured to run on one or more processors. Thus, as an example, the 'module' or 'part' may include components such as software components, object-oriented software components, class components, and task components, and at least one of processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, or variables. The components and the functions provided within the 'module' or 'part' may be combined into a smaller number of components and 'modules' or 'parts', or further separated into additional components and 'modules' or 'parts'.

[0044] According to one embodiment of the present disclosure, a ‘module’ or ‘part’ may be implemented as a processor and memory. The term ‘processor’ should be broadly interpreted to include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, etc. In some environments, the term ‘processor’ may refer to an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), etc. The term ‘processor’ may also refer to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors combined with a DSP core, or any other combination of such configurations. Additionally, the term ‘memory’ should be broadly interpreted to include any electronic component capable of storing electronic information. 'Memory' may refer to various types of processor-readable media, such as Random Access Memory (RAM), Read-Only Memory (ROM), Non-Volatile Random Access Memory (NVRAM), Programmable Read-Only Memory (PROM), Erasable-Programmable Read-Only Memory (EPROM), Electrically Erasable PROM (EEPROM), Flash Memory, Magnetic or Optical Data Storage Devices, Registers, etc. If a processor can read information from memory and / or write information to memory, the memory is said to be in an electronic communication state with the processor. Memory integrated into a processor is in an electronic communication state with the processor.

[0045] In the present disclosure, the 'system' may include at least one of a server device and a cloud device, but is not limited thereto. For example, the system may be composed of one or more server devices. As another example, the system may be composed of one or more cloud devices. As yet another example, the system may be configured and operated with both a server device and a cloud device.

[0046] In the present disclosure, 'display' may refer to any display device associated with a computing device, for example, any display device capable of displaying any information / data controlled by or provided by the computing device.

[0047] In the present disclosure, 'each of a plurality of A' or 'each of a plurality of A' may refer to each of all components included in a plurality of A, or each of some components included in a plurality of A.

[0048] FIG. 1 shows an example of an autonomous manufacturing process operating system according to one embodiment of the present disclosure.

[0049] Referring to FIG. 1, in one embodiment, a manufacturing process autonomous operation system may include a coordinator agent (110) that generates commands for tasks associated with the manufacturing process and a plurality of sub-agents. Here, the coordinator agent (110) may supervise the manufacturing process and transmit the generated commands to any one of the plurality of sub-agents. Additionally, the plurality of sub-agents may control equipment / facilities of the manufacturing process, monitor the status of the process, or utilize a manufacturing AI model, but are not limited thereto. Additionally, the plurality of sub-agents may include a first sub-agent to a fourth sub-agent (120 to 150).

[0050] In one embodiment, the first sub-agent (120) may generate situational information associated with a task using a first machine learning model. Here, the situational information associated with the task may include information that needs to be identified to perform the task. For example, if the task is to improve the production of defective products, the situational information associated with the task may include, but is not limited to, information associated with the process of producing defective products, information associated with the equipment of producing defective products, environmental information of the process of producing defective products, information on the cause of producing defective products, etc., and may include data that can be collected to perform the task. Such data may include various forms such as text, images, and voice.

[0051] In one embodiment, the first machine learning model may collect base data associated with a task from at least one of equipment or a database associated with a manufacturing process. For example, the database may include, but is not limited to, a first database storing vectors embedding text or images associated with a manufacturing process, a second database storing structured data associated with a manufacturing process, and a third database storing a knowledge graph associated with a manufacturing process. Additionally, the first machine learning model may include at least one of a language model or a vision language model that is fine-tuned in association with a manufacturing process.

[0052] In one embodiment, the second sub-agent (130) may generate action plan information based on situational information using a second machine learning model. Here, the second machine learning model may be trained to take a task and situational information generated by the first sub-agent (120) as inputs and output action plan information for performing the task. Additionally, the second machine learning model may include at least one of a language model or a vision language model that is fine-tuned in relation to the manufacturing process. Furthermore, the second sub-agent (130) may generate action plan information based on situational information, a task, numerical data related to the manufacturing process (e.g., temperature, pressure, etc. in the process), and environmental data related to the manufacturing process (e.g., temperature, humidity, etc. inside the factory).

[0053] In one embodiment, the third sub-agent (140) can generate result information by performing a task based on action plan information generated by the second sub-agent (130). Specifically, the third sub-agent (140) can perform a simulation associated with the task based on the action plan information and generate result information of the task based on the result of the simulation. Alternatively, the third sub-agent (140) can generate an execution command associated with the task (e.g., equipment movement command, facility operation command, etc.) based on the action plan information and transmit the execution command to the equipment associated with the task. Additionally, the third sub-agent (140) can generate result information of the task based on the execution result of the task based on the execution command. Here, the third sub-agent (140) can receive the execution result from the equipment associated with the task or generate the execution result based on data associated with said equipment.

[0054] In one embodiment, the fourth sub-agent (150) may generate evaluation information that evaluates the results of a task based on result information generated by the third sub-agent (140). Specifically, the fourth sub-agent (150) may determine whether the goal of the task has been achieved based on the result information. Additionally, the fourth sub-agent (150) may generate evaluation information based on whether the goal has been achieved.

[0055] In one embodiment, if the objective of the task is not achieved, the fourth sub-agent (150) may send a request to the second sub-agent (130) to modify the action plan information. In this case, the fourth sub-agent (150) may also send the request to the second sub-agent (130) through the coordination agent (110). The coordination agent (110) may update the command for the task based on the evaluation information.

[0056] In one embodiment, a plurality of sub-agents may utilize manufacturing AI models. Here, the manufacturing AI models may include a quality optimization AI model that predicts product quality using numerical data generated during the production process (e.g., temperature, pressure, etc. in the process), a production optimization AI model that recommends an optimal production recipe using numerical data generated during the production process and environmental data (e.g., factory temperature, humidity, etc.), a vision inspection AI model that photographs the appearance of a product using an industrial camera and inspects for product defects based on the captured image, and an industrial safety AI model that detects whether a person has entered a hazardous area within the factory through a camera and controls warnings and the operation of facilities / equipment. For example, a second sub-agent (130) may use a production optimization AI model to generate action plan information associated with a recipe that can produce an optimal product. As another example, a third sub-agent (140) may use a quality optimization AI model to predict the quality of a product produced according to a production recipe.

[0057] In FIG. 1, there are four sub-agents, but this is not limited to this, and sub-agents may be added or some omitted.

[0058] With this configuration, all intelligent tasks that factory workers can perform can be executed autonomously by utilizing multiple agents. Furthermore, through the interaction of multiple agents, the manufacturing process can be operated autonomously by responding appropriately even when changes occur in the process environment, products, or personnel.

[0059] FIG. 2 is a schematic diagram showing a configuration in which an information processing system (230) is connected to communicate with a plurality of user terminals (210_1, 210_2, 210_3) in order to autonomously operate a manufacturing process according to one embodiment of the present disclosure.

[0060] Referring to FIG. 2, a plurality of user terminals (210_1, 210_2, 210_3) can be connected to an information processing system (230) capable of providing manufacturing process autonomous operation services through a network (220). Here, the plurality of user terminals (210_1, 210_2, 210_3) may include a user terminal receiving manufacturing process autonomous operation services.

[0061] In one embodiment, the information processing system (230) may include one or more server devices and / or databases capable of storing, providing, and executing computer-executable programs (e.g., downloadable applications) and data associated with providing autonomous operation services for manufacturing processes, or one or more distributed computing devices and / or distributed databases based on cloud computing services.

[0062] The manufacturing process autonomous operation service provided by the information processing system (230) can be provided to the user through a manufacturing process autonomous operation application, a web browser, a web browser extension, etc. installed on each of the plurality of user terminals (210_1, 210_2, 210_3). For example, the information processing system (230) can provide information or perform corresponding processing in response to a task execution request related to the manufacturing process received from the user terminals (210_1, 210_2, 210_3) through the manufacturing process autonomous operation application, etc.

[0063] Multiple user terminals (210_1, 210_2, 210_3) can communicate with an information processing system (230) through a network (220). The network (220) can be configured to enable communication between the multiple user terminals (210_1, 210_2, 210_3) and the information processing system (230). Depending on the installation environment, the network (220) may be configured as a wired network such as Ethernet, Power Line Communication, telephone line communication device and RS-serial communication, a mobile communication network, a Wireless LAN (WLAN), Wi-Fi, Bluetooth and ZigBee, or a combination thereof. The communication method is not limited and may include not only communication methods utilizing communication networks that the network (220) may include (e.g., mobile communication network, wired internet, wireless internet, broadcasting network, satellite network, etc.) but also short-range wireless communication between user terminals (210_1, 210_2, 210_3).

[0064] In FIG. 2, a mobile phone terminal (210_1), a tablet terminal (210_2), and a PC terminal (210_3) are illustrated as examples of user terminals, but are not limited thereto. The user terminals (210_1, 210_2, 210_3) may be any computing device capable of wired and / or wireless communication and capable of installing and running a manufacturing process autonomous operation application or a web browser, etc. For example, user terminals may include an AI speaker, a smartphone, a mobile phone, a navigation system, a computer, a laptop, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), a tablet PC, a game console, a wearable device, an IoT (Internet of Things) device, a VR (Virtual Reality) device, an AR (Augmented Reality) device, a set-top box, etc. Additionally, FIG. 2 illustrates three user terminals (210_1, 210_2, 210_3) communicating with an information processing system (230) through a network (220), but is not limited thereto, and may be configured so that a different number of user terminals communicate with an information processing system (230) through a network (220).

[0065] In one embodiment, the information processing system may include a plurality of agents. For example, the information processing system may include a coordination agent and a plurality of subordinate agents. Alternatively, each of the plurality of information processing systems may perform and process tasks associated with the manufacturing process as each of the plurality of agents.

[0066] In FIG. 2, a configuration in which a user's request (e.g., a task execution request) is transmitted to an information processing system (230) through a user terminal (210_1, 210_2, 210_3) is illustrated as an example, but is not limited thereto. A user's request may be provided to an information processing system (230) through an input device associated with the information processing system (230) without passing through the user terminal (210_1, 210_2, 210_3), and a result of processing the user's request (e.g., a task execution result) may be provided to the user through an output device (e.g., a display, etc.) associated with the information processing system (230).

[0067] FIG. 3 is a block diagram showing the internal configuration of a user terminal (210) and an information processing system (230) according to one embodiment of the present disclosure.

[0068] Referring to FIG. 3, the user terminal (210) may refer to any computing device capable of running applications, web browsers, etc., and capable of wired / wireless communication, and may include, for example, the mobile phone terminal (210_1), tablet terminal (210_2), PC terminal (210_3) of FIG. 2. The user terminal (210) may include memory (312), a processor (314), a communication module (316), and an input / output interface (318). Similarly, the information processing system (230) may include memory (332), a processor (334), a communication module (336), and an input / output interface (338). The user terminal (210) and the information processing system (230) may be configured to communicate information and / or data through a network (220) using their respective communication modules (316, 336). Additionally, the input / output device (320) may be configured to input information and / or data to the user terminal (210) or output information and / or data generated from the user terminal (210) through the input / output interface (318).

[0069] The memory (312, 332) may include any non-transient computer-readable recording medium. According to one embodiment, the memory (312, 332) may include a permanent mass storage device such as ROM (read-only memory), a disk drive, a solid-state drive (SSD), or flash memory. As another example, a permanent mass storage device such as ROM, an SSD, flash memory, or a disk drive may be included in the user terminal (210) or information processing system (230) as a separate permanent storage device distinct from the memory. Additionally, an operating system and at least one program code may be stored in the memory (312, 332).

[0070] These software components may be loaded from a computer-readable recording medium separate from memory (312, 332). This separate computer-readable recording medium may include a recording medium that can be directly connected to the user terminal (210) and the information processing system (230), for example, a computer-readable recording medium such as a floppy drive, disk, tape, DVD / CD-ROM drive, or memory card. As another example, the software components may be loaded into memory (312, 332) via a communication module (316, 336) rather than a computer-readable recording medium. For example, at least one program may be loaded into memory (312, 332) based on a computer program installed by files provided through a network (220) by developers or a file distribution system that distributes installation files for the application.

[0071] The processor (314, 334) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processor (314, 334) by memory (312, 332) or a communication module (316, 336). For example, the processor (314, 334) may be configured to execute instructions received according to program code stored in a recording device such as memory (312, 332).

[0072] The communication module (316, 336) may provide a configuration or function for the user terminal (210) and the information processing system (230) to communicate with each other via the network (220), and may provide a configuration or function for the user terminal (210) and / or the information processing system (230) to communicate with another user terminal or another system (e.g., a separate cloud system). For example, a request or data (e.g., a task execution request) generated by the processor (314) of the user terminal (210) according to program code stored in a recording device such as memory (312) may be transmitted to the information processing system (230) via the network (220) under the control of the communication module (316). Conversely, a control signal or command provided under the control of the processor (334) of the information processing system (230) can be received by the user terminal (210) through the communication module (336) and the network (220) via the communication module (316) of the user terminal (210).

[0073] The input / output interface (318) may be a means for interfacing with an input / output device (320). As an example, the input device may include a device such as a camera including an audio sensor and / or an image sensor, a keyboard, a microphone, or a mouse, and the output device may include a device such as a display, a speaker, or a haptic feedback device. As another example, the input / output interface (318) may be a means for interfacing with a device in which the configuration or function for performing input and output is integrated into one, such as a touchscreen. For example, when the processor (314) of the user terminal (210) processes instructions of a computer program loaded in memory (312), a service screen configured using information and / or data provided by an information processing system (230) or another user terminal may be displayed on a display through the input / output interface (318). In FIG. 3, the input / output device (320) is depicted as not being included in the user terminal (210), but is not limited thereto and may be configured as a single device with the user terminal (210). Additionally, the input / output interface (338) of the information processing system (230) may be a means for interfacing with a device (not shown) for input or output that is connected to the information processing system (230) or that the information processing system (230) may include. In FIG. 3, the input / output interface (318, 338) is shown as an element configured separately from the processor (314, 334), but is not limited thereto, and the input / output interface (318, 338) may be configured to be included in the processor (314, 334).

[0074] The user terminal (210) and the information processing system (230) may include more components than those of FIG. 3. In one embodiment, the user terminal (210) may be implemented to include at least some of the input / output devices (320) described above. Additionally, the user terminal (210) may include other components such as a transceiver, a GPS (Global Positioning System) module, a camera, various sensors, a database, etc. For example, if the user terminal (210) is a smartphone, it may include components that are generally included in a smartphone, and may be implemented to include various components such as an accelerometer, a gyroscope, a microphone module, a camera module, various physical buttons, buttons using a touch panel, input / output ports, and a vibrator for vibration.

[0075] While a program or application for autonomous operation services for manufacturing processes is in operation, the processor (314) may receive text, images, video, voice and / or actions, etc., that are input or selected through an input device such as a touch screen, keyboard, audio sensor and / or image sensor, camera, microphone, etc., connected to an input / output interface (318), and may store the received text, images, video, voice and / or actions, etc. in memory (312) or provide them to an information processing system (230) through a communication module (316) and a network (220).

[0076] The processor (314) of the user terminal (210) may be configured to manage, process, and / or store information and / or data received from an input / output device (320), another user terminal, an information processing system (230), and / or a plurality of external systems. The information and / or data processed by the processor (314) may be provided to the information processing system (230) through a communication module (316) and a network (220). The processor (314) of the user terminal (210) may transmit information and / or data to the input / output device (320) through an input / output interface (318) to output it. For example, the processor (314) may output or display the received information and / or data on a screen associated with the user terminal (210).

[0077] The processor (334) of the information processing system (230) may be configured to manage, process, and / or store information and / or data received from a plurality of user terminals (210) and / or a plurality of external systems. The information and / or data processed by the processor (334) may be provided to the user terminals (210) through a communication module (336) and a network (220).

[0078] FIG. 4 is a diagram illustrating an example of performing a task based on user input according to one embodiment of the present disclosure.

[0079] Referring to FIG. 4, in one embodiment, a user may input a query associated with a task through a user terminal (410). Here, the query may include natural language. In this case, a natural language processing unit (420) may generate an embedding vector associated with the query. Accordingly, a coordination agent (430) may generate a command for the task based on the embedding vector generated by the natural language processing unit (420). Such a command for the task may be transmitted to at least one of a plurality of sub-agents so that the task entered by the user may be performed.

[0080] In one embodiment, the natural language processing unit (420) can generate a response to user input received through the user terminal (410). Specifically, the coordination agent (430) can generate a result associated with a task based on results (e.g., evaluation information, etc.) generated by a plurality of sub-agents. Additionally, the natural language processing unit (420) can generate a response containing natural language based on the results. Accordingly, a user-friendly response can be provided.

[0081] In one embodiment, when the coordination agent (430) outputs a programming language as a result, the natural language processing unit (420) can convert the language into natural language and provide it to the user. For example, when the coordination agent (430) outputs an SQL query, which is a programming language, as a result, the subordinate agent can connect to the database (440) and execute (invoke) the SQL query. Additionally, the subordinate agent can receive the query result from the database (440) and transmit it to the natural language processing unit (420). The natural language processing unit (420) can convert the query result into natural language and provide it to the user.

[0082] In one embodiment, the natural language processing unit (420) can directly generate a response to user input. Specifically, the natural language processing unit (420) can generate a text embedding vector for a query using a language model. Additionally, the natural language processing unit (420) can retrieve information associated with the text embedding vector from a database (440). Furthermore, the natural language processing unit (420) can generate a response to user input by inputting the query result, retrieved information, and prompt into the language model.

[0083] In one embodiment, the database (440) may include a first database that stores a vector containing embedded text or an image associated with a manufacturing process, a second database that stores structured data associated with a manufacturing process, and a third database that stores a knowledge graph associated with a manufacturing process. Here, the structured data stored in the second database may include at least one of numerical values ​​used in the process, sensing data generated from equipment / facility (e.g., temperature, pressure, etc.), statistical values ​​associated with the manufacturing process, or process autonomous operation data. Additionally, the third database may store data that can be expressed in the form of a knowledge graph and may store relationships between the data together. For example, the knowledge graph may include nodes associated with a problem, phenomenon, or response, and edges representing the relationship between each node associated with a cause or means of solution.

[0084] In one embodiment, the language model may be trained using a specialized language dataset associated with the manufacturing process. Fine-tuning techniques such as Parameter Efficient Fine-tuning (LoRA), Supervised Fine-tuning (Instruction Tuning), and Alignment Fine-tuning (Direct Preference Optimization, DPO) may be used for training the language model, but are not limited thereto. The natural language processing unit (420) may use such a language model to embed a query containing natural language and generate a response containing natural language based on a result associated with a task output by the coordination agent (430).

[0085] With this configuration, even workers lacking programming language or manufacturing expertise can easily input queries and tasks in natural language. As a result, worker intervention is minimized, and the user experience can be enhanced.

[0086] FIG. 5 is a diagram illustrating an example of performing a task based on process monitoring according to one embodiment of the present disclosure.

[0087] Referring to FIG. 5, in one embodiment, a coordinating agent (530) may generate a command for a task based on manufacturing process monitoring information received from a subordinate agent. Specifically, a monitoring agent (520) may generate monitoring information by monitoring data associated with the manufacturing process. Here, the data associated with the manufacturing process may include at least one of data associated with equipment / facilities used in the manufacturing process or data associated with products produced. The monitoring agent (520) may collect data associated with the manufacturing process in real time from a plurality of equipment (510_1 to 510_n) and detect data in which a change exceeding a threshold value occurs. Here, the monitoring agent (520) may detect abnormal signs using a rule-based detection method (e.g., Upper Control Limit, Lower Control Limit, etc.) or a statistical-based detection method (e.g., PCA, Hoteling's T-square, etc.), but is not limited thereto. Additionally, the monitoring agent (520) may generate monitoring information based on the detected data and transmit it to the coordinating agent (530). In this case, the coordination agent (530) can generate a command for the task based on the monitoring information.

[0088] In one embodiment, a coordination agent (530) may receive an abnormal sign regarding at least one of a plurality of equipment (510_1 to 510_n) from a monitoring agent (520). In this case, the coordination agent (530) may transmit an abnormal sign analysis command to a subordinate agent, an equipment failure prediction agent (540). The equipment failure prediction agent (540) may predict the failure probability and / or remaining lifespan at a specific point in time (e.g., after 1 hour, after 1 day, after 10 days, etc.) using a failure prediction model for each of the plurality of equipment (510_1 to 510_n). Here, the failure prediction model may be a model using at least one of survival analysis, a Cox proportional hazards model, or a Kaplan-Meier Estimator, but is not limited thereto. Additionally, the equipment failure prediction agent (540) may infer the cause of the failure and transmit the analysis result to the coordination agent (530). These analysis results may include guidance information for predictive maintenance.

[0089] In one embodiment, the coordination agent (530) may transmit a command for a task to at least one of a plurality of sub-agents. The sub-agent to which the command is transmitted may be determined based on a plurality of scenarios entered through prompt engineering.

[0090] In one embodiment, the monitoring agent (520) and the equipment failure prediction agent (540) perform complementary roles, and the coordination agent (530) can prevent unexpected accidents by linking the monitoring agent (520) and the equipment failure prediction agent (540). Additionally, the coordination agent (530) can assist the equipment management personnel in performing planned predictive maintenance at an appropriate time. For example, the monitoring agent (520) can detect abnormal signs in multiple pieces of equipment (510_1 to 510_n) in real time and transmit them to the coordination agent (530). Additionally, the equipment failure prediction agent (540) can transmit the failure probability (e.g., 90%) and the failure cause prediction result (e.g., valve malfunction) at a specific point in time (e.g., 10 days later) to the coordination agent (530). Accordingly, the coordination agent (530) can synthesize this and automatically generate a predictive maintenance report, and provide the predictive maintenance report to a designated equipment management officer.

[0091] FIG. 6 is a drawing showing an example of the internal configuration of an agent (600) according to one embodiment of the present disclosure.

[0092] Referring to FIG. 6, in one embodiment, each agent (a plurality of sub-agents or coordination agents) of the manufacturing process autonomous operation system may include at least one of a planning unit (610), a control unit (620), a storage unit (630), and a utility unit (640).

[0093] First, the planning unit (610) can use a machine learning model to analyze situations related to the manufacturing process based on input data and establish an action plan. Here, the input data may be related to tasks entered by the user or tasks generated based on process monitoring results. Additionally, the planning unit (610) can extract data necessary to establish an action plan from the database (650) to infer and analyze situations related to the manufacturing process. Accordingly, the planning unit (610) can establish an action plan to satisfy the conditions necessary to perform the task.

[0094] In one embodiment, the planning unit (610) may include a manufacturing-specific machine learning model. Here, the machine learning model may include at least one of a language model or a vision language model that is fine-tuned in relation to the manufacturing process. The machine learning model may describe the situation and context associated with the manufacturing process and establish a plan of action. Additionally, the machine learning model may perform understanding and description of the manufacturing process environment, and evaluation and selection of action options.

[0095] The control unit (620) can control at least one of the equipment (660) or system associated with the manufacturing process based on an action plan. Here, the equipment (660) associated with the manufacturing process may include machines and robots used in the manufacturing process. For example, the control unit (620) may transmit a control command to the equipment to move a specific piece of equipment based on an action plan, but is not limited thereto.

[0096] The storage unit (630) can store input data, reasoning history, and action plans. In this case, the storage unit (630) is responsible for short-term and long-term memory and can store the information in the database (650).

[0097] The utility unit (640) may provide utility tools necessary for the task. Here, the utility tools may include a calendar, calculator, code interpreter, search tool (web search API), programming language writing tool, translation tool, industrial robot control framework, etc. For example, the utility unit (640) may use a programming writing tool to generate commands converted into a language applicable to the equipment (660) to which the control command is transmitted. As another example, the utility unit (640) may use a translation tool to provide necessary information in multiple languages.

[0098] In FIG. 6, the agent (600) is shown to include a planning unit (610), a control unit (620), a storage unit (630), and a utility unit (640), but is not limited thereto, and some components may be omitted or added. Additionally, at least one function of the planning unit (610), the control unit (620), the storage unit (630), or the utility unit (640) may be enhanced through fine tuning.

[0099] FIG. 7 is a diagram illustrating an example of a method for improving the production of defective products in a manufacturing process using an autonomous operating system according to one embodiment of the present disclosure.

[0100] Referring to FIG. 7, in one embodiment, the production of defective products may be detected in the manufacturing process (S710). In this case, the coordination agent may generate a command for the production of good products by the equipment producing the defective products. Here, the coordination agent may generate the command based on monitoring information regarding the equipment. Alternatively, the coordination agent may generate the command based on user input associated with the production of good products by a specific piece of equipment in the manufacturing process. The coordination agent may transmit a command to a first sub-agent to determine the status of the production of defective products.

[0101] Subsequently, real-time data can be collected (S720). Here, the real-time data may correspond to monitoring information received by the coordination agent. Accordingly, the coordination agent may collect the real-time data and transmit it to the first sub-agent. Alternatively, the first sub-agent may directly collect the real-time data associated with the equipment.

[0102] Subsequently, the first subordinate agent can generate situational information related to the product of the equipment based on real-time data collected from the equipment (S730). Here, the real-time data collected from the equipment may include actual measured values ​​of the product, inspection results of the product, and setting values ​​set by the equipment to produce the product. Additionally, the situational information related to the product of the equipment may include the good / defective status of the product, whether there are abnormalities in the setting values, whether there are abnormalities in the actual measured values, etc.

[0103] The generated situation information can be transmitted to a coordination agent. For example, in response to the production (or continuous production) of defective products, the coordination agent can transmit a command to a second sub-agent to modify the manufacturing recipe.

[0104] Accordingly, the second sub-agent can generate action plan information associated with a production recipe based on situation information associated with the equipment's products (S740). Here, the situation information can be converted into a format suitable for the second sub-agent so that it can be utilized by the second sub-agent. The second sub-agent can use a machine learning model to extract variables associated with the cause of defective products being produced in the current situation and generate action plan information associated with a manufacturing recipe for producing good products.

[0105] Subsequently, the third sub-agent can generate result information regarding the product of the equipment based on action plan information associated with the equipment's production recipe (S750). Specifically, the third sub-agent can generate result information by predicting it using a quality optimization AI model. Here, the result information may include the predicted defect results and defect probabilities of the product to be produced through the production recipe.

[0106] Subsequently, the fourth sub-agent can generate evaluation information that evaluates the production results of good products of the equipment based on result information regarding the products of the equipment (S760). In this case, the fourth sub-agent can determine whether the production results of good products can achieve a goal (or a predetermined threshold for the probability of producing good products) based on the evaluation information (S770). If the goal cannot be achieved, the process can be repeated starting from the real-time data collection step (S720). Alternatively, if the goal cannot be achieved, the process can be repeated starting from the action plan information generation step associated with the production recipe (S740). Accordingly, the action plan information generated by the second sub-agent may be changed.

[0107] If the goal can be achieved, the adjustment agent can apply the improved production recipe to the equipment based on the action plan information (S780). In this case, the utility unit of the adjustment agent (e.g., 640 in FIG. 6) can write a programming language so that the production recipe can be applied to the equipment. Additionally, the control unit of the adjustment agent (e.g., 620 in FIG. 6) can transmit control commands associated with the written programming language to the equipment. Accordingly, the equipment can produce good products according to the improved production recipe.

[0108] FIG. 8 is a drawing illustrating an example of a method in which an autonomous manufacturing process operating system moves equipment according to one embodiment of the present disclosure.

[0109] Referring to FIG. 8, in one embodiment, user input for moving equipment may be received (S810). Here, the user input may be transmitted to a control agent as described above in FIG. 4. Accordingly, the control agent may generate a command for moving the equipment.

[0110] Subsequently, real-time data associated with the equipment may be collected (S820). Here, the equipment and real-time data may include, but are not limited to, images captured through a camera mounted on the equipment, location information of the equipment, location information of a product associated with a task, obstacle detection information measured through at least one of Lidar, ultrasound, or infrared, tracking information and speed information of a dynamic object measured through radar. Accordingly, a coordination agent may collect real-time data and transmit it to a first sub-agent. Alternatively, the first sub-agent may directly collect real-time data associated with the equipment.

[0111] Subsequently, the first sub-agent can generate situational information related to the environment of the equipment based on real-time data related to the equipment (S830). Specifically, the first sub-agent may use a machine learning model to generate information such as the distance and angle between the equipment and the product based on the collected real-time data, but is not limited thereto.

[0112] After that, the second sub-agent can generate action plan information related to the movement path of the equipment based on situational information related to the environment of the equipment (S840). Here, the action plan information may include step-by-step action plans such as moving the equipment, loading products, and unloading.

[0113] After that, the third sub-agent can control the equipment to move based on action plan information associated with the equipment's production recipe (S850). Here, the third sub-agent can move the equipment according to a step-by-step action plan. Additionally, the third sub-agent can generate result information associated with the equipment's location (S860).

[0114] After that, the fourth sub-agent can generate evaluation information that evaluates the movement results of the equipment based on result information associated with the location of the equipment (S870). Specifically, the fourth sub-agent can evaluate the results of the step-by-step action plan based on images collected through a camera mounted on the equipment.

[0115] After that, the fourth sub-agent can determine whether the result of the equipment's movement has achieved the final goal based on the evaluation information (S880). If the final goal has not been achieved, the process can be repeated starting from the real-time data collection step (S720). Alternatively, if the final goal has not been achieved, the process can be repeated starting from the action plan information generation step (S840) associated with the equipment's movement path.

[0116] In one embodiment, if the final goal is achieved, the fourth sub-agent may transmit to the coordinating agent whether the final goal has been achieved. The coordinating agent may provide the user with results associated with the achievement of the final goal (e.g., an image captured by a camera mounted on the equipment or text indicating that the equipment has moved to the final goal).

[0117] FIG. 9 is a diagram illustrating an example of a method for a manufacturing process autonomous operation system to generate a production plan according to one embodiment of the present disclosure.

[0118] Referring to FIG. 9, in one embodiment, a user input requesting the establishment of a production plan for a manufacturing process may be received (S910). Here, the user input may be transmitted to a coordination agent as described above in FIG. 4. Accordingly, the coordination agent may generate a command for the movement of equipment and transmit it to a subordinate agent.

[0119] Subsequently, data regarding products associated with the production plan may be collected (S920). Here, the data regarding the products may include, but is not limited to, product production time, the amount of raw materials required to produce the products, and product delivery schedules. Accordingly, a coordination agent may collect data regarding products associated with the production plan and transmit it to a first sub-agent. Alternatively, the first sub-agent may directly collect data regarding products associated with the production plan.

[0120] After that, the first sub-agent can generate situation information based on data regarding products associated with the production plan (S930). After that, the second sub-agent can generate action plan information including multiple production plans based on data regarding products associated with the production plan using a machine learning model (S940).

[0121] After that, the third sub-agent can perform simulations of multiple production plans based on action plan information (S950). The third sub-agent can transmit the result information of simulating production plans for all combinations to the fourth sub-agent.

[0122] After that, the fourth sub-agent can generate evaluation information that evaluates the results of simulations for multiple production plans (S960). Here, the evaluation information evaluates the objective function for each simulation, and the objective function can be determined in advance based on the manufacturing time, the amount of raw materials consumed, etc.

[0123] Subsequently, the coordination agent can output final production plan information based on the evaluation information (S970). Specifically, the coordination agent can determine the production plan that received the evaluation closest to the objective as the final production plan based on the evaluation information of multiple simulation results. Accordingly, the coordination agent can provide the determined final production plan information to the user in natural language.

[0124] FIG. 10 is a flowchart illustrating an example of a multi-agent-based manufacturing process autonomous operation method (1000) according to one embodiment of the present disclosure.

[0125] Referring to FIG. 10, in one embodiment, a manufacturing process autonomous operation method (1000) may be performed by a processor included in at least one of a coordination agent or a plurality of sub-agents. The manufacturing process autonomous operation method (1000) may be initiated by generating a command for a task associated with the manufacturing process and transmitting it to any one of the plurality of sub-agents (S1010).

[0126] After that, the processor can generate situation information associated with the task using a first machine learning model (S1020).

[0127] After that, the processor can generate action plan information based on situation information using a second machine learning model (S1030).

[0128] In addition, the processor can generate result information by performing a task based on action plan information (S1040).

[0129] Based on the result information, the processor can generate evaluation information that evaluates the result of the task (S1050).

[0130] The method described above may be provided as a computer program stored on a computer-readable recording medium for execution on a computer. The medium may continuously store a program executable by a computer, or temporarily store it for execution or download. Additionally, the medium may be various recording or storage means in the form of a single or multiple hardware components combined, and may not be limited to a medium directly connected to a computer system but may exist distributed over a network. Examples of media may include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and media configured to store program instructions, including ROM, RAM, and flash memory. Furthermore, other examples of media may include recording or storage media managed by app stores that distribute applications or sites and servers that supply or distribute various other software.

[0131] The methods, operations, or techniques of the present disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those skilled in the art will understand that the various exemplary logical blocks, modules, circuits, and algorithmic steps described in connection with the disclosure herein may be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate such interchangeability between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been generally described above in terms of their functional aspects. Whether such functions are implemented in hardware or in software depends on the design requirements imposed on the specific application and the overall system. Those skilled in the art may implement the functions described in various ways for each specific application, but such implementations should not be construed as departing from the scope of the present disclosure.

[0132] In a hardware implementation, the processing units used to perform the techniques may be implemented in one or more ASICs, DSPs, GPUs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described in this disclosure, computers, or a combination thereof.

[0133] Accordingly, the various exemplary logic blocks, modules, and circuits described in connection with the present disclosure may be implemented or performed by any combination of general-purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or those designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, for example, a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors coupled with a DSP core, or any other combination of configurations.

[0134] In firmware and / or software implementations, techniques may be implemented as instructions stored on a computer-readable medium such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic or optical data storage devices, etc. The instructions may be executable by one or more processors, and may cause the processor(s) to perform specific aspects of the functions described in this disclosure.

[0135] When implemented in software, the techniques may be stored on a computer-readable medium as one or more instructions or code, or transmitted through a computer-readable medium. Computer-readable media include both computer storage media and communication media, including any medium that facilitates the transmission of a computer program from one place to another. Storage media may be any available medium that can be accessed by a computer. As a non-limiting example, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to transfer or store desired program code in the form of instructions or data structures and can be accessed by a computer. Additionally, any connection is appropriately made to the computer-readable medium.

[0136] For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line, or wireless technologies such as infrared, radio, and microwave are included within the definition of a medium. As used herein, disk and disc include CD, laser disc, optical disc, DVD (digital versatile disc), floppy disk, and Blu-ray disc, wherein disks usually play data magnetically, whereas discs play data optically using a laser. The above combinations should also be included within the scope of computer-readable media.

[0137] The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other known form of storage medium. An exemplary storage medium may be connected to a processor so that the processor can read information from the storage medium or write information to the storage medium. Alternatively, the storage medium may be integrated into the processor. The processor and the storage medium may exist within an ASIC. The ASIC may exist within a user terminal. Alternatively, the processor and the storage medium may exist as separate components within the user terminal.

[0138] Although the embodiments described above have been described as utilizing aspects of the subject matter disclosed herein in one or more standalone computer systems, the present disclosure is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or a distributed computing environment. Furthermore, aspects of the subject matter in the present disclosure may be implemented in a plurality of processing chips or devices, and storage may be similarly affected across a plurality of devices. Such devices may include PCs, network servers, and portable devices.

[0139] Although the present disclosure has been described in relation to some embodiments, various modifications and changes may be made without departing from the scope of the present disclosure as understood by a person skilled in the art to which the invention of the present disclosure pertains. Furthermore, such modifications and changes should be considered to fall within the scope of the claims appended to this specification.

Claims

1. In a multi-agent-based manufacturing process autonomous operation system, A coordination agent that generates commands for tasks related to a manufacturing process and transmits them to one of a plurality of sub-agents; A first sub-agent that generates situation information associated with the task using a first machine learning model; A second sub-agent that generates action plan information based on the above situation information using a second machine learning model; A third sub-agent that generates result information by performing the above task based on the above action plan information; and A fourth sub-agent that generates evaluation information evaluating the result of the above task based on the above result information. A multi-agent-based manufacturing process autonomous operation system including 2. In Paragraph 1, The above-mentioned coordination agent updates commands for the above-mentioned task based on the above-mentioned evaluation information, in a multi-agent-based manufacturing process autonomous operation system.

3. In Paragraph 1, A natural language processing unit that receives a query including natural language associated with the above task and generates an embedding vector associated with the query Includes more, A multi-agent-based manufacturing process autonomous operation system in which the above-mentioned coordination agent generates commands for the above-mentioned task based on the above-mentioned embedding vector.

4. In Paragraph 3, The above coordination agent generates a result associated with the above task based on the above evaluation information, and The above natural language processing unit generates a response including natural language based on the above result, a multi-agent-based manufacturing process autonomous operation system.

5. In Paragraph 1, A fifth sub-agent that generates monitoring information by monitoring data related to the above manufacturing process Includes more, The above-mentioned coordination agent generates commands for the above-mentioned task based on the above-mentioned monitoring information, in a multi-agent-based manufacturing process autonomous operation system.

6. In Paragraph 1, A multi-agent-based manufacturing process autonomous operation system in which the first machine learning model collects base data associated with the task from at least one of the equipment or database associated with the manufacturing process.

7. In Paragraph 6, A multi-agent-based manufacturing process autonomous operation system comprising a first database storing vectors embedding text or images associated with the manufacturing process, a second database storing structured data associated with the manufacturing process, and a third database storing a knowledge graph associated with the manufacturing process.

8. In Paragraph 1, A multi-agent-based manufacturing process autonomous operation system, wherein each of the first machine learning model and the second machine learning model comprises at least one of a fine-tuned language model or a vision language model associated with the manufacturing process.

9. In Paragraph 1, A multi-agent-based manufacturing process autonomous operation system, wherein the second machine learning model is trained to take the task and situation information as inputs and output action plan information for performing the task.

10. In Paragraph 1, The above second sub-agent is, A multi-agent-based manufacturing process autonomous operation system that generates action plan information based on the above situation information, the above task, numerical data associated with the above manufacturing process, and environmental data associated with the above manufacturing process.

11. In Paragraph 1, The above third sub-agent is, Based on the above action plan information, perform a simulation associated with the above task, and A multi-agent-based manufacturing process autonomous operation system that generates result information of the above task based on the results of the above simulation.

12. In Paragraph 1, The above third sub-agent is, Based on the above action plan information, generate execution commands associated with the above task, and Transmit the execution command to the equipment associated with the above task, and A multi-agent-based manufacturing process autonomous operation system that generates result information of the task based on the execution result of the task based on the execution command.

13. In Paragraph 1, The above-mentioned fourth sub-agent is, Based on the above result information, determine whether the goal of the above task has been achieved, and A multi-agent-based manufacturing process autonomous operation system that generates evaluation information based on whether the goal of the above task has been achieved.

14. In Paragraph 13, The above-mentioned fourth sub-agent is, A multi-agent-based manufacturing process autonomous operation system that, if the objective of the above task is not achieved, transmits a request to modify the action plan information to the second sub-agent.

15. In Paragraph 1, Each of the above first to fourth sub-agents is, A planning unit that uses a machine learning model to analyze situations related to the manufacturing process based on input data and establishes an action plan; A control unit that controls at least one of the equipment or systems associated with the manufacturing process based on the above action plan; A storage unit that stores the above input data and the above action plan; and Utility unit assisting the above control unit A multi-agent-based manufacturing process autonomous operation system comprising at least one of the following.

16. In Paragraph 1, The above task is associated with the production of good products of the first equipment of the above manufacturing process, and The first sub-agent above generates situation information related to the products of the first equipment based on real-time data collected from the first equipment, and The second sub-agent above generates action plan information associated with the production recipe of the first equipment based on situation information associated with the product of the first equipment, and The above-mentioned third sub-agent generates result information regarding the product of the first equipment based on action plan information associated with the production recipe of the first equipment, and A multi-agent-based manufacturing process autonomous operation system in which the above-mentioned fourth sub-agent generates evaluation information evaluating the good product production results of the above-mentioned first equipment based on result information regarding the products of the above-mentioned first equipment.

17. In Paragraph 1, The above task is associated with the movement of the second equipment of the above manufacturing process, and The first sub-agent above generates situation information related to the environment of the second equipment based on real-time data collected from the second equipment, and The second sub-agent generates action plan information associated with the movement path of the second equipment based on situational information associated with the environment of the second equipment, and The third sub-agent controls the second equipment to move based on action plan information associated with the movement path of the second equipment, thereby generating result information associated with the location of the second equipment. A multi-agent-based manufacturing process autonomous operation system in which the above-mentioned fourth sub-agent generates evaluation information evaluating the movement results of the second equipment based on result information associated with the location of the second equipment.

18. In Paragraph 1, The above task is related to the establishment of a production plan for the above manufacturing process, and The above-mentioned first sub-agent collects data on products associated with the above-mentioned production plan to generate situational information, and The above-mentioned second sub-agent generates action plan information including a plurality of production plans based on the above-mentioned situation information, and The above third sub-agent performs a simulation of the plurality of production plans based on the action plan information, and The above-mentioned fourth sub-agent generates evaluation information that evaluates the results of simulations for the above-mentioned plurality of production plans, in a multi-agent-based manufacturing process autonomous operation system.

19. As a multi-agent-based manufacturing process autonomous operation method, A step of generating a command for a task associated with a manufacturing process and transmitting it to one of a plurality of sub-agents; A step of generating situation information associated with the above task using a first machine learning model; A step of generating action plan information based on the above situation information using a second machine learning model; A step of generating result information by performing the above task based on the above action plan information; and A step of generating evaluation information that evaluates the results of the above task based on the above result information A multi-agent-based manufacturing process autonomous operation method including 20. A computer-readable recording medium storing a computer program for executing the method according to paragraph 19 on a computer.

Images