Optimization method of distributed fuzzy flexible manufacturing system integrating assembly and logistics

Abstract

The invention discloses an optimization method for a distributed fuzzy flexible manufacturing system integrating assembly and logistics, and the method comprises the steps: constructing intelligent agents for three stages of the distributed fuzzy flexible manufacturing system, and training the intelligent agents in each stage; and for a to-be-optimized instance, determining a scheduling scheme and maximum fuzzy completion time of a first stage by using the trained first-stage intelligent agent, determining an assembly sequence and a whole assembly period of workpieces in a second stage by using the trained second-stage intelligent agent, and obtaining a transportation route by using the trained third-stage intelligent agent. According to the invention, through constructing a multi-agent reinforcement learning system composed of a workpiece agent, a factory agent, a machine agent, a task agent, a work station agent and a transportation agent, collaborative optimization control of production, assembly and transportation three-stage tasks in a distributed fuzzy flexible manufacturing system is realized.

Description

Technical Field

[0001] The present invention relates to an optimization method for a distributed fuzzy flexible manufacturing system integrating assembly and logistics, and belongs to the technical field of deep reinforcement learning and production scheduling. Background Art

[0002] With the rapid development of industrial intelligence and the Industrial Internet of Things (IIoT), production scheduling has become a key research topic in the field of intelligent manufacturing. By optimizing resource allocation, improving production efficiency and profitability, and achieving lean production, this technology plays a crucial role in enhancing a company's market competitiveness. Flexible manufacturing systems (FMSs) can improve production efficiency, reduce costs, shorten cycle times, and enhance product quality and operational flexibility. However, the increasing complexity of dynamic demand and the high flexibility of these systems present increasingly challenging challenges. In recent years, the increasing importance of cross-regional collaborative production has garnered significant attention in the research of distributed FMSs. In a distributed FMS, finished workpieces from various factories are transported to a central assembly center for further processing and assembled into final products via assembly lines. Traditional linear assembly lines are primarily used for high-volume production, with workstations arranged in a linear layout, with relatively balanced workloads at each station and subject to predetermined priority constraints. However, with the increasing prevalence of just-in-time production strategies, U-shaped assembly lines have gradually replaced the traditional linear layout, achieving significant improvements in production efficiency, profitability, and product quality. Finished products are then transported to customer locations by vehicle, requiring optimized transportation routes while meeting capacity constraints. However, dynamic disruptions, such as machine failures, workpiece insertions, and parameter fluctuations, introduce uncertainty into distributed flexible manufacturing systems; additional consideration of fuzzy processing time is crucial for addressing these uncertainties. Solving this integrated optimization problem is crucial for improving production efficiency, reducing costs, and enhancing customer satisfaction. The coordinated integration of distributed fuzzy flexible job shop scheduling, U-shaped assembly lines, and capacity-constrained vehicle routing constitutes the highly complex and interconnected distributed fuzzy flexible job shop scheduling-U-shaped assembly line-capacitated vehicle routing problem (DFFJS-UA-CVRP). The complex coupling of dynamic constraints and uncertainties in the production, assembly, and transportation stages poses significant challenges to solving the problem. Effectively addressing this problem will not only improve operational performance but also create a significant competitive advantage in the rapidly evolving field of distributed flexible manufacturing systems.

[0003] In recent years, the application of deep reinforcement learning (DRL) in complex scheduling has evolved from the static environment of traditional job shops to the dynamic environment of flexible job shops. While it can effectively handle uncertainties within a single factory, such as process interchangeability and random processing time, its performance in distributed fuzzy flexible manufacturing systems that integrate assembly and logistics has significant limitations. Because single-agent DRL struggles to fully perceive the cross-domain resource competition in multi-factory collaboration and effectively coordinate the coupling effects of the workpiece assembly phase and the capacity constraints of the transportation phase, it often falls into local optimality due to incomplete global observation when faced with the three-stage DFFJS-UA-CVRP optimization problem. Summary of the Invention

[0004] The present invention provides an optimization method for a distributed fuzzy flexible manufacturing system that integrates assembly and logistics. By constructing a multi-agent reinforcement learning system consisting of a workpiece agent, a factory agent, a machine agent, a task agent, a workstation agent, and a transportation agent, the coordinated optimization control of the three-stage tasks of production, assembly, and transportation in the distributed fuzzy flexible manufacturing system is achieved.

[0005] The technical solution of the present invention is:

[0006] According to a first aspect of the present invention, a method for optimizing a distributed fuzzy flexible manufacturing system that integrates assembly and logistics is provided, comprising: constructing intelligent agents for the three stages of the distributed fuzzy flexible manufacturing system respectively, and training the intelligent agents for each stage; for the instance to be optimized, using the trained first-stage intelligent agent to determine the scheduling plan and the maximum fuzzy completion time for the first stage, using the trained second-stage intelligent agent to determine the assembly sequence of workpieces in the second stage and the entire assembly cycle, and using the trained third-stage intelligent agent to obtain the transportation route.

[0007] It also includes: processing the actual workpiece according to the determined scheduling plan, assembling the processed workpiece according to the determined assembly sequence after the cumulative processing time in the first stage reaches the production time threshold, and transporting the assembled products to the customer according to the obtained transportation route.

[0008] The method specifically comprises the following steps:

[0009] S1. Set up factory agents, workpiece agents, and machine agents for the production phase of the distributed fuzzy flexible manufacturing system; set up task agents and workstation agents for the assembly phase; and set up transportation agents for the transportation phase;

[0010] S2, set the maximum number of training iterations LT for each agent;

[0011] S3. Generate a first random instance with the same scale as the production stage according to the scale of the production stage;

[0012] S4. Extract the workpiece state at decision time t based on the first random instance Input the workpiece state at the decision time t to the workpiece agent and calculate the action of the workpiece agent Get the processing workpiece i at the decision time t;

[0013] S5. Determine whether the workpiece i at the decision time t has been assigned to a factory: If the workpiece i has been assigned to a factory f, then the action of the factory agent at the decision time t Otherwise, according to the first random instance, extract the factory state at decision time t Input the factory state at decision time t to the factory agent and calculate the action of the factory agent Get the factory f that processes workpiece i at decision time t;

[0014] S6. Based on the first random instance, extract the machine state at the decision time t Input the machine state at decision time t To the machine agent, calculate the action of the machine agent Get the processing machine m in factory f that can process process j for workpiece i at decision time t;

[0015] S7, the workpiece agent, factory agent and machine agent make decisions and obtain joint actions The workpiece i to be processed at the decision time t is determined, along with the assigned factory f and the processing machine m within factory f. The currently available process steps of workpiece i are processed on machine m in factory f, and the production phase is transferred to the next decision time t+1.

[0016] S8. Collecting experience tuples of artifact agents in the production environment Among them, r t JFM represents the rewards of the workpiece agent, factory agent, and machine agent at the decision time t;

[0017] S9. Collecting experience tuples of factory agents in the production environment

[0018] S10. Collecting Experience Tuples of Machine Agents in Production Environments

[0019] S11, updating the network parameters of the workpiece agent, factory agent and machine agent respectively;

[0020] S12, repeat S4-S11 until all workpieces in the first random instance are processed;

[0021] S13, repeat S3-S12 until the maximum number of training iterations LT is reached;

[0022] S14. Generate a second random instance with the same scale as the assembly stage according to the scale of the assembly stage;

[0023] S15. Extract the task state at decision time t based on the second random instance Input the task state at decision time t to the task agent and calculate the action of the task agent Get the assembly workpiece i at the decision time t;

[0024] S16. Extract the workstation status at decision time t based on the second random instance Input the workstation status at decision time t To the workstation agent, calculate the action of the workstation agent Get the assembly workstation k that assembles workpiece i at decision time t;

[0025] S17. Get a joint action after the task agent and the workstation agent make a decision The workpiece i and workstation k to be assembled at the decision time t are determined, and the workpiece i is assembled at the workstation k. The assembly phase is transferred to the next decision time t+1;

[0026] S18. Collecting experience tuples of task agents in the assembly environment Among them, r t TS represents the rewards of the task agent and the workstation agent at the decision time t;

[0027] S19. Collecting experience tuples of workstation agents in an assembly environment

[0028] S20, updating the network parameters of the task agent and the workstation agent respectively;

[0029] S21, repeat S15-S20 until all the workpieces in the second random instance are assembled;

[0030] S22, repeat S14-S21 until the maximum number of training iterations LT is reached;

[0031] S23. Generate a third random instance with the same size as the transportation stage according to the transportation stage size;

[0032] S24. Based on the third random instance, extract the transportation status at the decision time t. Input the transportation state at the decision time t to the transportation agent and calculate the action of the transportation agent Get the customer g that vehicle v delivers to at decision time t;

[0033] S25. Get the action after the transport agent makes the decision The customer g that vehicle v delivers to at decision time t is determined, the product of vehicle v is transported to customer g, and the transportation phase is transferred to the next decision time t+1;

[0034] S26. Collecting experience tuples of transport agents in the transport environment Among them, r t V represents the reward of the transport agent at the decision time t;

[0035] S27, updating the network parameters of the transport agent;

[0036] S28, repeat S24-S27 until all deliveries are completed for the customers in the second random instance;

[0037] S29, repeat S23-S28 until the maximum number of training iterations LT is reached;

[0038] S30, using the trained factory agent, workpiece agent, and machine agent to determine the scheduling plan for the production stage and the maximum fuzzy completion time;

[0039] S31. Using the trained task agent and workstation agent, determine the assembly sequence of workpieces in each workstation during the assembly phase and the entire assembly cycle.

[0040] S32. Using the trained transport agent, obtain a transport route;

[0041] S33, processing the actual workpiece according to the determined scheduling plan;

[0042] S34. Calculate the production time threshold;

[0043] S35. When the cumulative processing time of the production stage reaches the production time threshold, assembling the processed workpieces according to the determined assembly sequence;

[0044] S36: transport the assembled products to the customer according to the obtained transportation route.

[0045] described Of which: JBP t i : Decision variable indicating whether all processes of workpiece i have been completed at the decision time t: If it has been completed, then JBP t i =1, otherwise JBP t i =0;N nIndicates the total number of artifacts; JPT t i : represents the average fuzzy processing time of the current process j of workpiece i on all available machines at the decision time t; JPP t i : represents the fuzzy completion time of the predecessor process of the current processing process of workpiece i in factory f at the decision time t; TPT t i : represents the cumulative fuzzy processing time of workpiece i at the decision time t.

[0046] described in: N represents the total load of all machines in factory f at the decision time t; f Indicates the total number of factories; PST t f : It means that at the decision time t, all unprocessed processes in factory f are processed in sequence, and the available machine with the shortest processing time is selected for each process, and the maximum fuzzy completion time PST of factory f is finally calculated. t f .

[0047] described Among them: APT t m : represents the fuzzy processing time of the current machinable process j of workpiece i on machine m at the decision time t; N m Indicates the total number of machines; EAT t m : represents the earliest fuzzy available time of machine m in factory f at the decision time t.

[0048] described in: Indicates whether workpiece i has been assembled at the decision time t. If workpiece i has been assembled, then otherwise N n Indicates the total number of artifacts; JAT t i : represents the assembly time of workpiece i at the decision time t.

[0049] described Among them, ACW t k N represents the total assembly time of workstation k at the decision time t; k Indicates the total number of workstations.

[0050] described in: N represents the demand of customer g at the decision time t; g Indicates the total number of customers; represents the Euclidean distance between the vehicle v and the customer g at the decision-making moment t; represents the Euclidean distance between the assembly center and the customer g at the decision-making moment t; represents whether the remaining load capacity of the vehicle v is greater than the demand of the customer at the decision-making moment t.

[0051] According to a second aspect of the present application, an optimization system of a distributed fuzzy flexible manufacturing system integrating assembly and logistics is provided, comprising: a training module for constructing an agent for each of the three stages of the distributed fuzzy flexible manufacturing system and training the agents for each stage; an obtaining module for, for a to-be-optimized instance, determining a scheduling scheme and a maximum fuzzy completion time of the first stage using the trained first-stage agent, determining an assembly sequence of the workpieces and an entire assembly cycle in the second stage using the trained second-stage agent, and obtaining a transportation route using the trained third-stage agent.

[0052] The present application has the beneficial effects that: the present application constructs a multi-agent reinforcement learning system composed of workpiece agents, factory agents, machine agents, task agents, workstation agents and transportation agents, each agent has rich state feature expression capability, and learns collaboratively under the guidance of three unified reward functions, can effectively represent the resource dependence and task coupling relationship between multiple stages, thereby improving the adaptability of the system to key constraints such as cross-factory workpiece allocation, flexible processing, workpiece priority dependence in the assembly link and transportation capacity limitation. Compared with the traditional single-agent method, the present application breaks through the limitations of local observation and isolated decision-making, significantly enhances the global coordination and robustness of the distributed fuzzy flexible manufacturing system, and can make the production, assembly and transportation in the distributed fuzzy flexible manufacturing system more efficient and stable. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 is a system framework diagram of a distributed flexible manufacturing system;

[0054] Figure 2 is a framework diagram of the multi-agent reinforcement learning proposed by the present application;

[0055] Figure 3 is a flowchart of the method of the present application;

[0056] Figure 4 is a loss value change curve of the factory agent;

[0057] Figure 5 is a loss value change curve of the workpiece agent;

[0058] Figure 6 is a loss value change curve of the machine agent;

[0059] Figure 7 is the loss value change curve of the task agent;

[0060] Figure 8 Change curve of the loss value of the workstation agent;

[0061] Figure 9 is the loss value change curve of the transportation agent. DETAILED DESCRIPTION

[0062] To make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other in any way.

[0063] Example 1: Figures 1-9 As shown, according to the first aspect of an embodiment of the present invention, an optimization method for a distributed fuzzy flexible manufacturing system that integrates assembly and logistics is provided, including: constructing intelligent agents for the three stages of the distributed fuzzy flexible manufacturing system respectively, and training the intelligent agents for each stage; for the instance to be optimized, using the trained first-stage intelligent agent to determine the scheduling plan and the maximum fuzzy completion time of the first stage, using the trained second-stage intelligent agent to determine the assembly sequence of the workpiece in the second stage and the entire assembly cycle, and using the trained third-stage intelligent agent to obtain the transportation route.

[0064] Furthermore, it also includes: processing the actual workpiece according to the determined scheduling plan, assembling the processed workpiece according to the determined assembly sequence after the cumulative processing time in the first stage reaches the production time threshold, and transporting the assembled products to the customer according to the obtained transportation route.

[0065] According to a second aspect of the embodiment of the present application, there is provided an optimization system of an integrated assembly and logistics distributed fuzzy flexible manufacturing system, comprising: a training module configured to construct an agent for each of three stages of the distributed fuzzy flexible manufacturing system and train the agent for each stage; an obtaining module configured to, for a to-be-optimized instance, determine a scheduling scheme and a maximum fuzzy completion time for the first stage using the trained agent for the first stage, determine an assembly sequence and an entire assembly cycle for a workpiece in the second stage using the trained agent for the second stage, and obtain a transportation route using the trained agent for the third stage. Further, the optimization system further comprises an executing module configured to process an actual workpiece according to the determined scheduling scheme, assemble the processed workpiece according to the determined assembly sequence after the cumulative processing time of the first stage reaches a production time threshold, and transport the assembled product to a customer according to the obtained transportation route.

[0066] An optional embodiment of the present application is described as follows, an optimization method of an integrated assembly and logistics distributed fuzzy flexible manufacturing system, specifically comprising the following steps:

[0067] S1, setting a factory agent, a workpiece agent and a machine agent for a production stage of the distributed fuzzy flexible manufacturing system, setting a task agent and a workstation agent for an assembly stage, and setting a transportation agent for a transportation stage;

[0068] S2, setting a maximum training iteration number LT for each agent;

[0069] S3, generating a first random instance with the same scale as the production stage according to the scale of the production stage; the first random instance satisfies the same total number of factories, the same total number of workpieces and the same total number of machines, and different fuzzy processing times of processes under the LT iteration times;

[0070] S4, extracting a workpiece state at a decision time t according to the first random instance inputting the workpiece state at the decision time t to the workpiece agent, and calculating an action of the workpiece agent according to a double-depth Q network, i.e., according to formula (38) obtaining a processed workpiece i at the decision time t;

[0071] the wherein: JBP t i represents a decision variable indicating whether all processes of the workpiece i have been completed at the decision time t; if the processes have been completed, JBP t i = 1, otherwise JBP t i = 0; N n represents the total number of workpieces; JPT t i: represents the average fuzzy processing time of the current process j of workpiece i on all available machines at the decision time t; JPP t i : represents the fuzzy completion time of the predecessor process of the current processing process of workpiece i in factory f at the decision time t; if the current process j of workpiece i is the first process, that is, there is no predecessor process, then JPP t i =0;TPT t i : represents the cumulative fuzzy processing time of workpiece i at the decision time t.

[0072] S5. Determine whether the workpiece i at the decision time t has been assigned to a factory: If the workpiece i has been assigned to a factory f, then the action of the factory agent at the decision time t Otherwise, according to the first random instance, extract the factory state at decision time t Input the factory state at the decision time t to the factory agent, and calculate the action of the factory agent according to the double-depth Q network, that is, according to formula (38) Get the factory f that processes workpiece i at decision time t;

[0073] Furthermore, the Among them: TLM t f : represents the total load of all machines in factory f at the decision time t; PST t f : It means that at the decision time t, all unprocessed processes in factory f are processed in sequence, and the available machine with the shortest processing time is selected for each process, and the maximum fuzzy completion time PST of factory f is finally calculated. t f .

[0074] S6. Based on the first random instance, extract the machine state at the decision time t Input the machine state at decision time t To the machine agent, according to the double-depth Q network, the action of the machine agent is calculated according to formula (38) Get the processing machine m in factory f that can process process j for workpiece i at decision time t;

[0075] Furthermore, the Among them: APT t m : represents the fuzzy processing time of the current machinable process j of workpiece i on machine m at the decision time t; EAT t m : represents the earliest fuzzy available time of machine m in factory f at the decision time t.

[0076] S7, the workpiece agent, factory agent and machine agent make decisions and obtain joint actions At decision time t, workpiece i is determined to be processed, along with its assigned factory f and processing machine m within factory f. Workpiece i is processed on machine m in factory f at the currently available process step, and the production phase shifts to the next decision time t+1. The available process steps for workpiece i are processed sequentially according to the order represented by the process numbers.

[0077] S8. Collecting experience tuples of artifact agents in the production environment Among them, r t JFM represents the rewards of the workpiece agent, factory agent, and machine agent at the decision time t;

[0078] S9. Collecting experience tuples of factory agents in the production environment

[0079] S10. Collecting Experience Tuples of Machine Agents in Production Environments

[0080] S11, respectively update the network parameters of the workpiece agent, factory agent and machine agent according to the gradient descent method in the dual-depth Q network;

[0081] S12, repeat S4-S11 until all workpieces in the first random instance are processed;

[0082] S13, repeat S3-S12 until the maximum number of training iterations LT is reached;

[0083] S14. Generate a second random instance with the same scale as the assembly stage based on the scale of the assembly stage; under LT iterations, the second random instance satisfies the following conditions: the total number of workpieces and the total number of workstations are the same, but the workpiece assembly time and workpiece assembly priority are different; and the total number of workpieces is equal to the number of tasks;

[0084] S15. Extract the task state at decision time t based on the second random instance Input the task state at the decision time t to the task agent, and calculate the action of the task agent according to the dual-depth Q network, that is, according to formula (38) Get the assembly workpiece i at the decision time t;

[0085] Furthermore, the in: Indicates whether workpiece i has been assembled at the decision time t. If workpiece i has been assembled, then otherwise represents the assembly time of workpiece i at the decision time t.

[0086] S16. Extract the workstation status at decision time t based on the second random instance Input the workstation status at decision time t To the workstation agent, according to the double-depth Q network, the action of the workstation agent is calculated according to formula (38) Get the assembly workstation k that assembles workpiece i at decision time t;

[0087] Furthermore, the in, ACW t k represents the total assembly time of workstation k at the decision time t; S k represents the set of artifacts on workstation k.

[0088] S17. Get a joint action after the task agent and the workstation agent make a decision The workpiece i and workstation k to be assembled at the decision time t are determined, and the workpiece i is assembled at the workstation k. The assembly phase is transferred to the next decision time t+1;

[0089] S18. Collecting experience tuples of task agents in the assembly environment Among them, r t TS represents the rewards of the task agent and the workstation agent at the decision time t;

[0090] S19. Collecting experience tuples of workstation agents in an assembly environment

[0091] S20, respectively updating the network parameters of the task agent and the workstation agent according to the gradient descent method in the dual-depth Q network;

[0092] S21, repeat S15-S20 until all the workpieces in the second random instance are assembled;

[0093] S22, repeat S14-S21 until the maximum number of training iterations LT is reached;

[0094] S23. Generate a third random instance with the same scale as the transportation stage based on the scale of the transportation stage; in LT iterations, the third random instance satisfies the same total number of customers but different customer demands and customer horizontal and vertical coordinates; the customer horizontal and vertical coordinates are the customer locations;

[0095] S24. Based on the third random instance, extract the transportation status at the decision time t. Input the transportation state at the decision time t to the transportation agent, and calculate the action of the transportation agent according to the double-depth Q network, that is, according to formula (38) Get the customer g that vehicle v delivers to at decision time t;

[0096] described

[0097] in: represents the demand of customer g at the decision time t; represents the Euclidean distance between vehicle v and customer g′ at the decision time t; represents the Euclidean distance between the assembly center and customer g at the decision time t; Indicates whether the remaining cargo capacity of vehicle v is greater than the customer demand at the decision time t.

[0098] S25. Get the action after the transport agent makes the decision The customer g that vehicle v delivers to at decision time t is determined, the product of vehicle v is transported to customer g, and the transportation phase is transferred to the next decision time t+1;

[0099] S26. Collecting experience tuples of transport agents in the transport environment Among them, r t V represents the reward of the transport agent at the decision time t;

[0100] S27, updating the network parameters of the transport agent according to the gradient descent method in the dual-depth Q network;

[0101] S28, repeat S24-S27 until all deliveries are completed for the customers in the second random instance;

[0102] S29, repeat S23-S28 until the maximum number of training iterations LT is reached;

[0103] S30, using the trained factory agent, workpiece agent, and machine agent to determine the scheduling plan for the production stage and the maximum fuzzy completion time The scheduling plan for the production stage is the sequence of workpiece processes on each machine m in each factory f;

[0104] S31. Use the trained task agent and workstation agent to determine the assembly sequence of the workpieces in each workstation during the assembly phase and the entire assembly cycle CT.

[0105] S32. Using the trained transport agent, obtain a transport route;

[0106] S33, processing the actual workpiece according to the scheduling plan determined in S30;

[0107] S34, calculating the production time threshold TV according to formula (21);

[0108] S35, when the cumulative processing time of the production stage reaches the production time threshold TV, the processed workpieces are assembled according to the assembly sequence determined in S31;

[0109] S36, the assembled product is transported to the customer according to the transportation route obtained in S32.

[0110] The optimization problem of the distributed fuzzy flexible manufacturing system is referred to as DFFJS-UA-CVRP, and an innovative six-agent collaborative decision architecture is proposed for DFFJS-UA-CVRP. Problem description:

[0111] The distributed fuzzy manufacturing system is composed of N f configurations of the same but different geographical locations of the factory, each factory is internally configured with N m machines. The system also contains an assembly center with a U-shaped layout, and N k assembly stations are arranged in the U-shaped assembly line on both sides. In the production stage, the system allocates N n workpieces to N f factories for processing. Each workpiece i contains n i processes, all of which must be processed in sequence within the same factory. Each process can be assigned to one or more feasible processing machines, and the fuzzy processing time is affected by the selected machine. After the factory completes the workpiece processing, it is transported to the assembly center (transportation cost is ignored in this model). When the cumulative processing time of a factory reaches the preset threshold TV, the workpiece assembly begins. In the assembly stage, each workpiece is assigned to N k assembly stations for assembly based on the workpiece assembly priority relationship and the load balancing principle. In the transportation stage, the completed products are transported by N v vehicles to N g geographically dispersed customers, and the capacity of each vehicle is p. The demand of each customer is different, and the total demand of all customers is N d .

[0112] The present application realizes the collaborative optimization of the production, assembly and transportation stages by constructing a multi-agent system, and the overall architecture is as follows:

[0113] In the production stage, three agents work collaboratively. The factory agent is responsible for assigning production tasks to the most suitable factory, which will consider the equipment condition, current task quantity and production capacity of each factory to ensure reasonable and balanced task allocation. The workpiece agent is responsible for selecting workpieces that need to be processed in priority. The machine agent is responsible for arranging specific processing machines for each workpiece, which will monitor the running state of each device in real time and select the most suitable machine for processing.

[0114] During the assembly phase, two collaborative assembly agents are deployed. The task agent determines the order of assembly workpieces, ensuring a smooth and efficient assembly process. The workstation agent assigns workpieces to the various workstations along the U-shaped assembly line, making appropriate arrangements based on the workstation's current load.

[0115] During the transportation phase, a dedicated agent is deployed to plan transportation routes. This agent considers factors such as customer location, delivery time requirements, and vehicle loading capacity to create the optimal route, ensuring on-time delivery while minimizing transportation costs.

[0116] All these intelligent agents are trained using reinforcement learning technology, and they share information and cooperate with each other. The complex manufacturing process is broken down into multiple relatively independent but interrelated stages. Each stage has a dedicated intelligent agent responsible for optimization, while ensuring overall coordination and consistency through information sharing. The system can be flexibly adjusted according to actual conditions. It is suitable for both stable production environments and changing needs in emergencies. Compared with traditional methods, it can significantly improve production efficiency and reduce operating costs. The architecture of the distributed fuzzy flexible manufacturing system is shown in the attached figure. Figure 1 As shown. The symbolic definitions involved in the optimization problem of the distributed fuzzy flexible manufacturing system are shown in Tables 1 to 4, and the mathematical model established based on this optimization problem is shown in Equations (1) to (22). Based on this mathematical model, the TCS can be obtained under the multi-agent reinforcement learning framework proposed in this invention (see the second column of Table 6). It should be noted that the fuzzy processing time used in this invention is determined by triangular fuzzy numbers.

[0117] Table 1

[0118]

[0119] Table 2

[0120]

[0121] Table 3

[0122]

[0123] Table 4

[0124]

[0125] The established mathematical model is as follows:

[0126]

[0127]

[0128]

[0129] Formula (1) ensures that all processes of the same workpiece are completed in the same factory. Formula (2) defines the order constraint between different processes of the same workpiece. Formula (3) shows that the first process of any workpiece can be processed from time zero. Formula (4) ensures that the completion time of a process is no earlier than its start time. Formulas (5) and (6) stipulate that each machine in the factory can only process one process at a time. Formula (7) ensures that each process is assigned to only one machine for processing. Formula (8) defines the fuzzy completion time of all workpieces in each factory. Formula (9) stipulates the fuzzy completion time of all workpieces in all factories. Formula (10) ensures that each workpiece must and can only be assigned to one workstation. Formulas (11) and (12) ensure that all workpieces are fully assigned and processes are not repeated between workstations. Formula (13) requires that the workpiece allocation must satisfy the priority relationship constraint between workpieces. Formula (14) defines the assembly cycle of the assembly line. Formula (15) ensures that the loading capacity of each vehicle does not exceed its maximum cargo capacity. Equation (16) stipulates that each vehicle must return to the assembly center after completing the delivery task. Equations (17) and (18) ensure that each customer point is served exactly once and that the number of vehicles entering and leaving is equal. Equation (19) defines the calculation method of the Euclidean distance of vehicle v from customer g′ to g. Equation (20) defines the total transportation distance. Equation (21) defines the production time threshold. When the production time reaches the production time threshold, each factory begins to transport the finished workpiece to the assembly center for assembly. As shown in Equation (22), the objective function of DFFJS-UA-CVRP aims to minimize the total cost of the entire process from production to delivery of the final product to the customer; where CV represents the vehicle's unit distance travel time, and the unit time costs of the production, assembly, and transportation stages are {CS1, CS2, CS3} respectively.

[0130] In a multi-agent reinforcement learning framework for optimizing distributed fuzzy flexible manufacturing systems integrating assembly and logistics, the action space defines the set of executable decisions for each agent, which directly affects the efficiency and effectiveness of the framework in solving the DFJS-UA-CVRP problem. The factory agent needs to assign each workpiece to the appropriate factory. Since all factories have the same configuration, the workpiece can be assigned to any factory. Therefore, the action space of the factory agent at the decision time t is Including all factories, At the initial decision time t = 0, the first process of all workpieces is ready. At this time, the action space of the workpiece agent is Contains all artifacts, As the decision-making process progresses and the workpiece is processed step by step, the action space The set of machines that can process the workpiece i is gradually reduced to an empty set, indicating that all workpieces have been processed. When the workpiece agent selects a workpiece i, the machine agent needs to make a choice from the set of machines that can process the workpiece in the current process j. Therefore, the action space of the machine agent at the decision time t is After entering the assembly phase, the task agent needs to select the workpiece that meets the priority constraints from the currently assembleable workpieces, so the action space Only the workpieces that meet the conditions are included. Since the workpieces can be assembled at any workstation, the action space of the workstation agent at the decision time t is Including all workstations, In the transportation phase, the action space of the transportation agent at the decision time t Includes all unserved customers whose demand does not exceed the vehicle's capacity, as well as the assembly center, since the vehicle must return to the assembly center and each customer can only be served once,

[0131] State features are used to capture key information about the environment, enabling the agent to make wise decisions. At the decision time t, the state s input to each agent is t It is composed of the features of all candidate actions in the current action space. At the decision time t, the state of the workpiece agent is expressed as Action Space Each candidate action in contains four features;

[0132] (1)JBP t i : Indicates whether all processes of workpiece i have been completed at the decision time t. If they have been completed, JBP t i =1, otherwise JBP t i =0;

[0133] (2)JPT t i : represents the average fuzzy processing time of the currently machinable process j of workpiece i on all available machines at the decision time t;

[0134]

[0135] Where, AM i,j Indicates processable O i,j The collection of machines, O i,j represents the j-th process of workpiece i; Indicates process O i,j Fuzzy processing time on machine m; |AM i,j | indicates AM i,j size;

[0136] (3)JPP t i denotes the fuzzy completion time of the predecessor j' of the current processing operation j of the workpiece i in the factory f at the decision-making moment t; if the current processable operation j of the workpiece i is the first operation, i.e. no predecessor j', then JPP t i = 0, otherwise:

[0137]

[0138] wherein, denotes the fuzzy completion time of the operation O i,j′ on the machine m in the factory f;

[0139] (4) TPT t i denotes the cumulative fuzzy processing time of the workpiece i at the decision-making moment t;

[0140]

[0141] wherein, TP i denotes the number of operations that have been processed by the workpiece i; X i,j,m,f is the decision variable, if the operation O i,j is processed on the machine m in the factory f, X i,j,m,f = 1, otherwise X i,j,m,f = 0; N m denotes the total number of machines.

[0142] When the workpiece agent determines the action at the decision-making moment t according to the input state features, after that, if the workpiece i has not been processed and is in the unassigned factory state, the factory agent needs to assign a factory for the workpiece i, and the state of the factory agent at the decision-making moment t is represented as Each candidate action in the action space contains two features:

[0143] (1) denotes the total load of all machines in the factory f at the decision-making moment t;

[0144]

[0145] (2) PST t f denotes the maximum fuzzy completion time of the factory f at the decision-making moment t, which is calculated by sequentially processing all unprocessed operations in the factory f and selecting the machine with the shortest processing time for each operation; t f

[0146] ​Subsequently, the machine agent is responsible for selecting the appropriate processing machine for the current process j of workpiece i in factory f. The state of the machine agent at the decision time t is expressed as Action Space Each candidate action in contains two features:

[0147] (1) APT t m : represents the fuzzy processing time of the current machinable step j of workpiece i on machine m at the decision time t;

[0148]

[0149] (2) EAT t m : represents the earliest fuzzy available time of machine m in factory f at the decision time t.

[0150]

[0151] In the assembly phase, the task agent selects the workpiece to be assembled at the decision time t according to the priority constraint relationship between the workpieces. The state of the task agent at this time is expressed as Action Space Each candidate action in contains two features:

[0152] (1) Indicates whether workpiece i has been assembled at the decision time t. If workpiece i has been assembled, then otherwise

[0153] (2) JAT t i : represents the assembly time of workpiece i at the decision time t.

[0154] JAT t i =q i (29)

[0155] Among them, q i represents the assembly time of workpiece i.

[0156] When the task agent completes the decision, the workstation agent needs to assign a suitable workstation to the workpiece i selected by the task agent. The state of the workstation agent at the decision time t is expressed as Action Space Each candidate action in contains only one feature:

[0157] (1)ACW t k : represents the total assembly time of workstation k at the decision time t;

[0158]

[0159] Among them, S k represents the set of workpieces on workstation k; N k Indicates the total number of workstations.

[0160] When transporting products to customers, the transport agent needs to plan a reasonable route for vehicle v, ensuring that the total delivery volume does not exceed the vehicle capacity and that the route is as short as possible. Therefore, the state of the transport agent at the decision time t is expressed as Action Space Each candidate action contains four features:

[0161] (1) represents the demand of customer g at the decision time t;

[0162]

[0163] (2) represents the Euclidean distance between vehicle v and customer g′ at the decision time t;

[0164]

[0165] Where, d g Indicates the demand of customer g; dist g′,g,v represents the Euclidean distance of vehicle v from customer g′ to g;

[0166] (3) represents the Euclidean distance between the assembly center and customer g at the decision time t;

[0167]

[0168] Where, dist g,0,v The Euclidean distance of vehicle v from customer g to the assembly center;

[0169] (4) Indicates whether the remaining cargo capacity of vehicle v is greater than the customer demand at the decision time t;

[0170]

[0171] Where p represents the maximum cargo capacity of the vehicle, PA v Represents the set of customers delivered by vehicle v.

[0172] The states comprehensively represent the production, assembly, and transportation environments, enabling intelligent agents to make optimal decisions at each stage of manufacturing, assembly, and transportation. These features enable efficient and collaborative decision-making by integrating key information such as workpiece processing status, machine availability, and logistics constraints.

[0173] The reward function guides the agent to learn the optimal strategy by quantifying the effectiveness of its actions, achieving a balance between immediate benefits and long-term goals, which is crucial for coordinated operations. In the production phase, the workpiece agent, factory agent, and machine agent share the unified goal of minimizing the maximum fuzzy completion time: at the decision time t, the unprocessed processes are selected in turn according to the principle of the shortest fuzzy processing time and assigned to the corresponding machines for simulated processing, generating a virtual maximum fuzzy completion time. When the workpiece agent, factory agent and machine agent complete the decision at the decision time t, the joint action is obtained. After the execution state transition enters the new state, the virtual maximum fuzzy completion time at decision time t+1 is calculated The rewards r of the workpiece agent, factory agent, and machine agent at the decision time t t FJM is the difference between the virtual maximum fuzzy completion time between the two decision moments, as shown in Equation (35). In the assembly phase, the task agent and the workstation agent are responsible for determining the assembly sequence of the workpiece and assigning workstations. At the decision moment t, the reward r of the task agent and the workstation agent is t TS It is defined as the difference in assembly cycle between consecutive decision moments, as shown in formula (36). For all unassembled workpieces, each unassembled workpiece is simulated and assembled on all workstations to obtain the assembly cycle. The assembly efficiency is improved by minimizing the assembly cycle. The path planning of the vehicle agent aims to minimize the total driving distance D, and its reward r t V Designed as the negative of the total travel distance, as shown in Equation (37). At decision time t, for all undelivered customer points, a shortest path first method is used to simulate delivery and calculate the total travel distance, thereby incentivizing the agent to select the optimal path. This multi-stage collaborative reward mechanism improves efficiency throughout the entire process, from production schedule optimization, assembly cycle control, and transportation path planning.

[0174]

[0175] r t TS =CT(t)-CT(t+1) (36)

[0176] r t V =-D(t) (37)

[0177] The Dual Deep Q-Network (DDQN) effectively addresses the inherent overestimation bias of traditional Deep Q-Networks (DQNs) by decoupling the action selection and value assessment processes. This approach employs a dual-network architecture: an online network responsible for action selection and a target network for value assessment, thereby mitigating the risk of Q-value overestimation and ensuring learning stability. Within this framework, each agent uses the standard DDQN formulation, which comprises three core components:

[0178] Action function: Each agent evaluates the state s through an online Q network t To determine its action a t :

[0179] a t =argmax a Q(s t ,a;θ) (38)

[0180] Where Q(s t ,a;θ) represents the state s of the online network t Next action a t The Q value of is estimated, and θ is the parameter of the online network.

[0181] Target Q value calculation: Calculate the target Q value using the target network to reduce overestimation:

[0182] y t =r t +γQ(s t+1 ,argmax a Q(s t+1 ,a;θ);θ′) (39)

[0183] Where r t represents the reward at decision time t, γ is the discount factor, Q(s t+1 ,a;θ) is the state of the online network s t+1 The Q value of the next action a is estimated, and θ′ is the parameter of the target network.

[0184] Loss function: The loss function of each agent is based on the mean squared error between the target Q value and the Q value predicted by the online network:

[0185]

[0186] where y t is the target Q value, Q(s t+1 ,a;θ) is the state of the online network s t Next action a t The Q-value estimation of .

[0187] The execution process of the multi-agent reinforcement learning framework for optimizing the distributed fuzzy flexible manufacturing system integrated with assembly and logistics is shown in FIG. Figure 3 As shown in FIG. Figure 4 As shown in FIG. Figure 9 The training curves on instances are shown, the x-axis represents the number of iterations, and the y-axis represents the average loss value of each iteration, and the convergence performance of each agent is significant. The pseudo code of the gradient descent method is shown in Table 5. Table 6 shows the simulation results of the multi-agent reinforcement learning framework and the genetic algorithm on instances, and the scale of each instance is N f ×N n ×N m ×N k ×N g ×N d As shown in Table 6, in different scale instances, the multi-agent reinforcement learning framework has better scheduling performance than the genetic algorithm in most cases, especially in large-scale instances, the difference between the two results is more significant, which shows that the framework has stronger optimization ability and adaptability in complex environment.

[0188] Table 5

[0189]

[0190] Table 6

[0191] Example Scale Multi-agent Reinforcement Learning Framework Genetic Algorithm 2×6×6×3×30×105 (11010,11588,12166) (11619,12231,12843) 2×6×6×3×40×140 (13258,13955,14652) (15039,15832,16625) 2×15×8×4×50×169 (50128,52765,55402) (53582,56400,59218) 2×15×8×4×60×198 (33898,35680,37462) (36841,38782,40723) 2×15×4×4×70×236 (60800,63998,67196) (69297,72946,76595) 2×10×15×4×80×270 (53787,56620,59453) (60266,63440,66614) 2×20×5×5×90×305 (77216,81278,85340) (90187,94936,99685) 2×20×10×5×100×332 (216800,228208,239616) (227060,239012,250964) 2×20×10×5×110×367 (19230,20243,21256) (209534,220560,231586) 2×20×15×5×120×401 (194435,204670,214905) (208453,219426,230399) 3×6×6×3×30×105 (10412,10962,11512) (11020,11602,12184) 3×6×6×3×40×140 (12194,12838,13482) (13975,14712,15449) 3×15×8×4×50×169 (43064,45332,47600) (47482,49982,52482) 3×15×8×4×60×198 (30512,32120,33728) (34208,36010,37812) 3×15×4×4×70×236 (49590,52202,54814) (63019,66338,69657) 3×10×15×4×80×270 (49683,52300,54917) (56675,59660,62645) 3×20×5×5×90×305 (65626,69082,72538) (79177,83346,87515) 3×20×10×5×100×332 (183998,193684,203370) (20056,21114,22172) 3×20×10×5×110×367 (172777,181872,190967) (200469,211022,221575) 3×20×15×5×120×401 (168530,177402,186274) (186358,196168,205978)

[0192] It can be known from the above technical solutions that the multi-agent reinforcement learning provided by the present application provides a new technical path for solving the complex collaborative optimization problem of DFFJS-UA-CVRP. The core advantage of the present application is that its distributed decision-making architecture can naturally map the multi-agent characteristics in the actual manufacturing system: in the production stage, the workpiece agent, the factory agent and the machine agent can autonomously optimize the local resource scheduling and coordinate the cross-factory task allocation; in the assembly stage, the task agent and the workstation agent can adjust the U-shaped line workpiece assembly sequence and the workstation allocation in real time; in the transportation stage, the transportation agent can optimize the vehicle path planning based on global information. This collaborative mechanism not only overcomes the limitations of single-agent system in information perception and decision-making dimension, but also effectively balances the conflict between local efficiency and global goal based on the game theory and distributed optimization collaborative learning framework.

[0193] The specific embodiments of the present invention are described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in this field without departing from the scope of the present invention.

Claims

1. An optimization method for a distributed fuzzy flexible manufacturing system integrating assembly and logistics, characterized in that: include: Intelligent agents are constructed for the three stages of the distributed fuzzy flexible manufacturing system and trained for each stage; For the instance to be optimized, the trained first-stage agent is used to determine the scheduling plan and maximum fuzzy completion time of the first stage. The trained second-stage agent is used to determine the assembly sequence of the workpieces in the second stage and the entire assembly cycle. The trained third-stage agent is used to obtain the transportation route.

2. The optimization method of the distributed fuzzy flexible manufacturing system integrating assembly and logistics according to claim 1 is characterized in that: Also includes: The actual workpiece is processed according to the determined scheduling plan. After the cumulative processing time in the first stage reaches the production time threshold, the processed workpiece is assembled according to the determined assembly sequence, and the assembled products are transported to the customer according to the obtained transportation route.

3. The optimization method of the distributed fuzzy flexible manufacturing system integrating assembly and logistics according to claim 1 is characterized in that: The specific steps include: S1. Set up factory agents, workpiece agents, and machine agents for the production phase of the distributed fuzzy flexible manufacturing system; set up task agents and workstation agents for the assembly phase; and set up transportation agents for the transportation phase; S2, set the maximum number of training iterations LT for each agent; S3. Generate a first random instance with the same scale as the production stage according to the scale of the production stage; S4. Extract the workpiece state at decision time t based on the first random instance Input the workpiece state at the decision time t to the workpiece agent and calculate the action of the workpiece agent Get the processing workpiece i at the decision time t; S5. Determine whether the workpiece i at the decision time t has been assigned to a factory: If the workpiece i has been assigned to a factory f, then the action of the factory agent at the decision time t Otherwise, according to the first random instance, extract the factory state at decision time t Input the factory state at decision time t to the factory agent and calculate the action of the factory agent Get the factory f that processes workpiece i at decision time t; S6. Based on the first random instance, extract the machine state at the decision time t Input the machine state at decision time t To the machine agent, calculate the action of the machine agent Get the processing machine m in factory f that can process process j for workpiece i at decision time t; S7, the workpiece agent, factory agent and machine agent make decisions and obtain joint actions The workpiece i to be processed at the decision time t is determined, along with the assigned factory f and the processing machine m within factory f. The currently available process steps of workpiece i are processed on machine m in factory f, and the production phase is transferred to the next decision time t+1. S8. Collecting experience tuples of artifact agents in the production environment Among them, r t JFM represents the rewards of the workpiece agent, factory agent, and machine agent at the decision time t; S9. Collecting experience tuples of factory agents in the production environment S10. Collecting Experience Tuples of Machine Agents in Production Environments S11, updating the network parameters of the workpiece agent, factory agent and machine agent respectively; S12, repeat S4-S11 until all workpieces in the first random instance are processed; S13, repeat S3-S12 until the maximum number of training iterations LT is reached; S14. Generate a second random instance with the same scale as the assembly stage according to the scale of the assembly stage; S15. Extract the task state at decision time t based on the second random instance Input the task state at decision time t to the task agent and calculate the action of the task agent Get the assembly workpiece i at the decision time t; S16. Extract the workstation status at decision time t based on the second random instance Input the workstation status at decision time t To the workstation agent, calculate the action of the workstation agent Get the assembly workstation k that assembles workpiece i at decision time t; S17. Get a joint action after the task agent and the workstation agent make a decision The workpiece i and workstation k to be assembled at the decision time t are determined, and the workpiece i is assembled at the workstation k. The assembly phase is transferred to the next decision time t+1; S18. Collecting experience tuples of task agents in the assembly environment Among them, r t TS represents the rewards of the task agent and the workstation agent at the decision time t; S19. Collecting experience tuples of workstation agents in an assembly environment S20, updating the network parameters of the task agent and the workstation agent respectively; S21, repeat S15-S20 until all the workpieces in the second random instance are assembled; S22, repeat S14-S21 until the maximum number of training iterations LT is reached; S23. Generate a third random instance with the same size as the transportation stage according to the transportation stage size; S24. Based on the third random instance, extract the transportation status at the decision time t. Input the transportation state at the decision time t to the transportation agent and calculate the action of the transportation agent Get the customer g that vehicle v delivers to at decision time t; S25. Get the action after the transport agent makes the decision Determine the customer g that vehicle v is delivering to at decision time t, and The product of vehicle v is transported to customer g, and the transportation phase shifts to the next decision time t+1; S26. Collecting experience tuples of transport agents in the transport environment Among them, r t V represents the reward of the transport agent at the decision time t; S27, updating the network parameters of the transport agent; S28, repeat S24-S27 until all deliveries are completed for the customers in the second random instance; S29, repeat S23-S28 until the maximum number of training iterations LT is reached; S30, using the trained factory agent, workpiece agent, and machine agent to determine the scheduling plan for the production stage and the maximum fuzzy completion time; S31. Using the trained task agent and workstation agent, determine the assembly sequence of workpieces in each workstation during the assembly phase and the entire assembly cycle. S32. Use the trained transport agent to obtain a transport route. S33, processing the actual workpiece according to the determined scheduling plan; S34. Calculate the production time threshold; S35. When the cumulative processing time of the production stage reaches the production time threshold, assembling the processed workpieces according to the determined assembly sequence; S36: transport the assembled products to the customer according to the obtained transportation route.

4. The optimization method of a distributed fuzzy flexible manufacturing system integrating assembly and logistics according to claim 3 is characterized in that: described in: JBP t i : Decision variable indicating whether all processes of workpiece i have been completed at the decision time t: If it has been completed, then JBP t i =1, otherwise JBP t i =0;N n Indicates the total number of artifacts; JPT t i : represents the average fuzzy processing time of the currently machinable process j of workpiece i on all available machines at the decision time t; JPP t i : represents the fuzzy completion time of the predecessor process of the current processing process of workpiece i in factory f at the decision time t; TPT t i : represents the cumulative fuzzy processing time of workpiece i at the decision time t.

5. The optimization method of the distributed fuzzy flexible manufacturing system integrating assembly and logistics according to claim 3 is characterized in that: described in: N represents the total load of all machines in factory f at the decision time t; f represents the total number of factories; PST t f : It means that at the decision time t, all unprocessed processes in factory f are processed in sequence, and the available machine with the shortest processing time is selected for each process, and the maximum fuzzy completion time PST of factory f is finally calculated. t f .

6. The optimization method of the distributed fuzzy flexible manufacturing system integrating assembly and logistics according to claim 3 is characterized in that: described in: APT t m : represents the fuzzy processing time of the current machinable process j of workpiece i on machine m at the decision time t; N m Indicates the total number of machines; E-A-T t m : represents the earliest fuzzy available time of machine m in factory f at the decision time t.

7. The optimization method of a distributed fuzzy flexible manufacturing system integrating assembly and logistics according to claim 3 is characterized in that: described in: Indicates whether workpiece i has been assembled at the decision time t. If workpiece i has been assembled, then otherwise N n Indicates the total number of artifacts; JAT t i : represents the assembly time of workpiece i at the decision time t.

8. The optimization method of a distributed fuzzy flexible manufacturing system integrating assembly and logistics according to claim 3 is characterized in that: described Among them, ACW t k N represents the total assembly time of workstation k at the decision time t; k Indicates the total number of workstations.

9. The optimization method of a distributed fuzzy flexible manufacturing system integrating assembly and logistics according to claim 3, characterized in that: described in: N represents the demand of customer g at the decision time t; g Indicates the total number of customers; represents the Euclidean distance between vehicle v and customer g′ at the decision time t; represents the Euclidean distance between the assembly center and customer g at the decision time t; Indicates whether the remaining cargo capacity of vehicle v is greater than the customer demand at the decision time t.

10. An optimization system for a distributed fuzzy flexible manufacturing system integrating assembly and logistics, characterized in that: include: The training module is used to construct intelligent agents for the three stages of the distributed fuzzy flexible manufacturing system and train the intelligent agents in each stage; The acquisition module is used to determine the scheduling plan and maximum fuzzy completion time of the first stage for the instance to be optimized using the trained first-stage intelligent agent, determine the assembly sequence of the workpieces in the second stage and the entire assembly cycle using the trained second-stage intelligent agent, and obtain the transportation route using the trained third-stage intelligent agent.

Images