Dynamic scheduling method of SMT adaptive production line based on deep reinforcement learning

Abstract

The invention discloses a dynamic scheduling method of an SMT adaptive production line based on deep reinforcement learning. Comprising the following steps: constructing an adaptive reconstruction SMT production line, training a plurality of agents by adopting a deep reinforcement learning model, and integrating the plurality of agents into an EMAS; an EMAS is adopted to complete a decision problem of ARFSP, on an SMS station, SMS unit reconstruction, CAP and CPSP are subjected to joint decision making, and on an SMT workshop, task operation is sorted, and an SMT workshop scheduling scheme is obtained. According to the invention, the SMT-FSP is used as a system problem to be solved through integrating the end-to-end multi-agent system of a plurality of deep reinforcement learning models, and the production efficiency of an SMT production line is improved.

Description

Technical Field

[0001] The present invention relates to the field of SMT production technology, and in particular to a dynamic scheduling method for an SMT adaptive production line based on deep reinforcement learning. Background Art

[0002] In recent years, the demand for printed circuit boards (PCBs) has been increasing. The scheduling of the surface mount technology (SMT) workshop, which is the key to its production efficiency, is a complex workshop scheduling problem (flow-shop scheduling problem, FSP) involving two levels, the workshop and the process, and there are multiple sub-problems at the process level. SMT-FSP (SMT flow-shop scheduling problem) is a multi-level and multi-sub-problem dynamic optimization decision-making problem: 1. At the workshop level, it is necessary to solve the scheduling of multiple tasks in the workshop; 2. At the bottleneck station level, because the SMS station needs to mount a large number of components, they need to be assigned to different SMS machines, and the completion time of different mounting sequences will also be different; at the same time, workshop management is also affected by various dynamic events, such as changes in task operations, machine failures, etc.

[0003] The SMT production line (SMTL) includes 5 stations, which are composed of production equipment such as Solder Paste Printer (SPP), Solder Paste Inspection (SPI), Surface Mount System (SMS), Reflow Soldering Machine (RSM), and Auto Optical Inspection (IAO) in series. The SMS station is an SMS unit composed of at least one identical SMS machine. Except for the SMS station, other stations are only equipped with one machine. Each SMT workshop is equipped with at least one SMTL. SMTL is usually equipped with multiple SMS machines. These serial SMS units are the bottleneck of the SMT production line and the part with the highest equipment cost.

[0004] At present, the research on these problems is mainly focused on certain parts, and few research results are found to solve these problems. According to production requirements, SMTL needs to maintain a distributed assembly line structure, and ordinary flexible production lines cannot meet its production requirements. In order to improve the solution efficiency and quality of SMT-FSP, this paper proposes a dynamic scheduling method for SMT adaptive production lines based on deep reinforcement learning, which enables SMT production lines to be adaptively reconstructed online for different task operations and dynamic events. Summary of the invention

[0005] The purpose of the present invention is to provide a dynamic scheduling method for an SMT adaptive production line based on deep reinforcement learning. The present invention solves SMT-FSP as a system problem by integrating an end-to-end multi-agent system of multiple deep reinforcement learning models, thereby improving the production efficiency of the SMT production line.

[0006] The technical solution of the present invention is a dynamic scheduling method for an SMT adaptive production line based on deep reinforcement learning, comprising the following steps:

[0007] Step 1: Build an adaptive reconfigurable SMT production line, use a deep reinforcement learning model to train multiple agents, and integrate multiple agents into a multi-agent system;

[0008] Step 2: Use a multi-agent system to complete the adaptive reconstruction of the SMT workshop scheduling problem: at the SMS workstation, make joint decisions on the SMS unit reconstruction, component allocation problem, and component placement order problem; in the SMT workshop, sort the tasks and obtain the SMT workshop scheduling plan.

[0009] In the above-mentioned dynamic scheduling method of the SMT adaptive production line based on deep reinforcement learning, in step one, the intelligent agent includes an adaptive reconstruction agent, an SMS agent and an SMT agent.

[0010] In the aforementioned dynamic scheduling method of the SMT adaptive production line based on deep reinforcement learning, in step 2, the adaptive reconstruction agent is used to provide a set of reconstruction alternative solutions; the SMS agent is used to obtain the solution to the component allocation problem, component placement sequence problem and production time of each solution in the reconstruction solution set; the SMT agent is used to determine the sequencing problem of the SMT workshop task operations and obtain the SMT workshop scheduling solution.

[0011] In the aforementioned dynamic scheduling method of the SMT adaptive production line based on deep reinforcement learning, in step 2, the use of a multi-agent system to complete the adaptive reconstruction of the SMT workshop scheduling problem includes:

[0012] Decision 1: Based on the number of SMS machines currently operating normally, an adaptive reconstruction agent is used to obtain a set of reconstruction alternatives for the PCB board on the SMS station; the set of reconstruction alternatives satisfies the resource constraints of the batching trough and the nozzle;

[0013] Decision 2: Use the SMS agent to obtain the solution of the component allocation problem and component placement order problem for each solution in the set of reconstructed alternative solutions, and obtain the minimum production time for each solution at the SMS station;

[0014] Decision 3: Use the SMT agent to calculate the preparation time for each solution and obtain the minimum production time for each task at the SMS station;

[0015] Decision 4: Use the SMT agent to sort the SMT workshop tasks with the weighted minimum delay time as the goal and obtain the SMT workshop scheduling plan.

[0016] In the aforementioned dynamic scheduling method of the SMT adaptive production line based on deep reinforcement learning, in decision 1, the set of reconstruction alternatives is expressed as:

[0017]

[0018] in, is the reconstruction alternative; w is the number of machines currently operating normally in the SMS group.

[0019] In the above-mentioned dynamic scheduling method of the SMT adaptive production line based on deep reinforcement learning, in decision 1, the resource constraints of the dispensing slot and the nozzle are:

[0020] w≤m; (2)

[0021]

[0022]

[0023] Where m is the number of machines in the SMS unit; ct i For PCB i The number of component types on the pcb i is the PCB board identified by the i index; nt i For PCB i fs is the number of nozzle types required on the SMS machine; fs is the number of batching slots on the SMS machine; ns is the type of nozzle on the SMS machine; for The number of SMS machines.

[0024] In the aforementioned dynamic scheduling method of the SMT adaptive production line based on deep reinforcement learning, in decision 2, the minimum production time of each solution at the SMS station is calculated as follows:

[0025]

[0026]

[0027]

[0028]

[0029] Among them, sx is the x-axis coordinate of the origin of the PCB board on the SMS machine; sy is the y-axis coordinate of the origin of the PCB board on the SMS machine; For components On the PCB i The x-axis coordinate on For PCB i The upper type is comp l The nth component, comp l is the component type identified by the l index; For components On the PCB i The y-axis coordinate on ; for The qth SMS machine in; For components In the The x-axis coordinate of the batching tank on the SMS machine; For components In the The y-axis coordinate of the batching tank on the SMS machine; For In the kth Components processed on SMS machines For components production time; From SMS head to components The x-axis distance of the batching trough; From SMS head to components The y-axis distance of the batching trough; From SMS head to components The x-axis distance of the mounting position; From SMS head to components The y-axis distance of the placement position; v is the moving speed of the SMS head; t1 l For components Placement time; t2 l For components The crawl time; To allocate to Number of components processed by each SMS machine; for production time.

[0030] In the aforementioned dynamic scheduling method of the SMT adaptive production line based on deep reinforcement learning, in decision 3, the minimum production time of each task operation at the SMS station is calculated as follows:

[0031]

[0032] in, For the job j Medium PCB i The shortest production time, job j is the task job identified by the j index; for job j Medium PCB i average preparation time.

[0033] In the aforementioned dynamic scheduling method of the SMT adaptive production line based on deep reinforcement learning, in decision 4, the calculation process of sorting the SMT workshop tasks with the weighted minimum delay time as the goal and obtaining the SMT workshop scheduling plan is as follows:

[0034]

[0035] Among them, p j for job j Production time; j for job j The order quantity; L j for job j Delay time; T j for job j The start time of j for job j deadline; f is the workshop scheduling plan; r is the number of task operations; ω j for job j The weight of .

[0036] Compared with the prior art, the adaptively reconfigurable SMT production line of the present invention has higher efficiency and stability in a manufacturing environment where dynamic events occur relative to the static organizational structure of the SMT production line. When some SMS machines fail, the SMT production line can continue to operate; at the same time, when facing different task operations, the multi-agent system can be adaptively reconstructed, which effectively improves the production efficiency of the SMT production line; in addition, compared with the current SMS workstation optimization that only considers the two sub-problems of component allocation and component placement sequence, the present invention jointly optimizes the SMT production line reconstruction with it, increasing the level and depth of optimization; the present invention solves SMT-FSP as a system problem through an end-to-end multi-agent system that integrates multiple deep reinforcement learning models, thereby improving the production efficiency of the SMT production line. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a schematic diagram of the reconstruction alternative and production flow of the present invention;

[0038] Figure 2 It is a schematic diagram of the structure of the decision-making process of the present invention;

[0039] Figure 3 It is a schematic diagram of the structure of the multi-agent system of the present invention;

[0040] Figure 4 It is a schematic diagram of the process of the present invention;

[0041] Figure 5 It is a statistical chart of the cumulative value of none for different penalty numbers and number of components;

[0042] Figure 6 This is a training comparison chart of masks and penalties;

[0043] Figure 7 It is a schematic diagram of the scheduling plan for the SMT workshop. DETAILED DESCRIPTION

[0044] The present invention is further described below in conjunction with the accompanying drawings and embodiments, but they are not intended to limit the present invention.

[0045] Embodiment: A dynamic scheduling method for an SMT adaptive production line based on deep reinforcement learning comprises the following steps:

[0046] Step 1: Build an adaptive reconfiguration SMT line (ARSL), use a deep reinforcement learning model to train multiple agents, and integrate multiple agents into an end-to-end multi-agent system integrating multiple DRL models (EMAS), such as Figure 3 As shown in the figure, an end-to-end approach is adopted to solve the Adaptive Reconfiguration SMT flow shop scheduling problem (ARFSP) as a system problem. The whole process from raw data input to final task output is automatically processed without manual division into multiple stages, which simplifies the model design and improves the overall performance.

[0047] In step 1, the agents include an adaptive reconfiguration agent (Adaptive Reconfiguration Agent, hereinafter referred to as ARA), an SMS agent (SMS Agent, hereinafter referred to as SMSA) and an SMT agent (SMT Agent, hereinafter referred to as SMTA).

[0048] ARSL is to design the bottleneck process and SMS unit with the highest failure rate into a reconfigurable structure, such as Figure 1 As shown in the figure. ARSL can adaptively reconfigure online in the face of dynamic events, ensuring that the SMT production line can continue to operate when some SMS machines stop. At the same time, it can optimize not only the component allocation problem (CAP) and the component placement sequence problem (CPSP) for different tasks, but also jointly optimize with the deeper SMT production line structure. ARSL not only enables the SMT production line to respond to dynamic events, but also improves the production efficiency and operation stability of the SMT production line.

[0049] Step 2: Use EMAS to complete the adaptive reconstruction SMT workshop scheduling problem: at the SMS workstation, make joint decisions on SMS unit reconstruction, CAP and CPSP; in the SMT workshop, sort the tasks and obtain the SMT workshop scheduling plan.

[0050] In step 2, the ARA is used to provide a set of reconstruction alternative solutions; the SMSA is used to obtain the CAP, CPSP solutions and production time of each solution in the reconstruction solution set; the SMTA is used to determine the sequencing problem of SMT workshop task operations and obtain the SMT workshop scheduling plan.

[0051] In ARFSP, two dynamic events need to be considered, and these two dynamic events may occur simultaneously in the manufacturing environment; the two dynamic events are:

[0052] 1) Task changes: including the insertion of task operations and changes in order quantity;

[0053] 2) Failure: In this embodiment, only random failures of bottleneck SMS machines are considered; when a failure occurs, if the SMS unit is determined to need to be rescheduled, EMAS will start the rescheduling calculation; the number of unfinished tasks interrupted by the failure will be included in the rescheduling calculation as a new task.

[0054] The operation of SMS units has the following conditions:

[0055] 1) Multiple components need to be assembled on a PCB, and multiple identical components are allowed;

[0056] 2) Each SMS machine has a limited number of dispensing slots and nozzles, which are fs and ns respectively. When allocating component placement tasks, the limitations of both dispensing slots and nozzles must be considered;

[0057] 3) The head of the SMS machine can only grab and place one component during operation;

[0058] 4) Multiple identical components of a PCB can be assigned to different SMS machines, and there is no priority restriction for each component;

[0059] 5) Considering the preparation time including material preparation and replacement, nozzle installation, etc., the production preparation time of the same PCB may be different under different order quantities;

[0060] 6) A task job can only have one PCB order, and the task job cannot be split unless there is a fault.

[0061] In step 2, the ARFSP is completed using EMAS. Figure 2 As shown, this includes the following decisions:

[0062] Decision 1: Based on the number of SMS machines currently operating normally, ARA is used to obtain a set of PCB board reconstruction alternatives on the SMS station; the set of reconstruction alternatives satisfies the resource constraints of the batching trough and the nozzle;

[0063] In decision 1, the set of reconstruction alternatives is expressed as:

[0064]

[0065] in, is the reconstruction alternative; w is the number of machines currently operating normally in the SMS group.

[0066] In decision 1, the resource constraints of the batching trough and the nozzle are:

[0067] w≤m; (2)

[0068]

[0069]

[0070] Where m is the number of machines in the SMS unit; ct i For PCB i The number of component types on the pcb i is the PCB board identified by the i index; nt i For PCB i fs is the number of nozzle types required on the SMS machine; fs is the number of batching slots on the SMS machine; ns is the type of nozzle on the SMS machine; for The number of SMS machines.

[0071] like Figure 1 As shown, pcb i The set of refactoring alternatives is:

[0072] RS i={(4,1),(4,2),(4,4),...,(3,1),(3,3)...};

[0073] In (4,1), w=4 means that there are 4 SMS machines currently operating normally. It means that one SMS machine is a group, each SMS machine has the same working content and completes the PCB independently. i All operations at the SMS station are equivalent to 4 parallel machines. When w = 4, 3 reconstruction alternatives are obtained: {(4,1), (4,2), (4,4)}; when one SMS machine fails, w = 4, and 2 reconstruction alternatives are obtained: {(3,1), (3,3)}.

[0074] Decision 2: Use SMSA to obtain the CAP and CPSP solutions of each solution in the reconstruction alternative solution set, and obtain the minimum production time of each solution at the SMS station;

[0075] In decision 2, the minimum production time of each solution at the SMS station is calculated as follows:

[0076]

[0077] During production, the head of the SMS machine first moves to the batching trough to grab the component, and then moves to the mounting position of the component on the PCB to place the component.

[0078] For example: The initial coordinates of the machine head are (0,0). On the SMS machine, the first component in the installation order is represented by Component type is comp a , the distance the head needs to move during installation is calculated as follows:

[0079]

[0080] On the same machine, the type of the element with order k-1 (k ≥ 2) is comp b , expressed as The type of the element with order k is comp l , expressed as The distance the head needs to move during placement is calculated as follows:

[0081]

[0082] In the XY motion mechanism of the machine head, the calculation of the machine head movement time depends on the motor with the longer movement distance of the two drive motors, because it takes more time to complete the movement, therefore, the completion element The movement time of placement is calculated as follows:

[0083]

[0084] Among them, sx is the x-axis coordinate of the origin of the PCB board on the SMS machine; sy is the y-axis coordinate of the origin of the PCB board on the SMS machine; For components On the PCB i The x-axis coordinate on For PCB i The upper type is comp l The nth component, comp l is the component type identified by the l index; For components On the PCB i The y-axis coordinate on ; for The qth SMS machine in; For components In the The x-axis coordinate of the batching tank on the SMS machine; For components In the The y-axis coordinate of the batching tank on the SMS machine; For In the kth Components processed on SMS machines For components production time; From SMS head to components The x-axis distance of the batching trough; From SMS head to components The y-axis distance of the batching trough; From SMS head to components The x-axis distance of the mounting position; From SMS head to components The y-axis distance of the placement position; v is the moving speed of the SMS head; t1 l For components Placement time; t2 l For components The crawl time; To allocate to Number of components processed by each SMS machine; for production time.

[0085] Decision 3: Use SMTA to calculate the preparation time for each solution and obtain the minimum production time for each task at the SMS station;

[0086] In decision 3, the minimum production time of each task operation at the SMS station is calculated as follows:

[0087]

[0088] in, For the job j Medium PCB i The shortest production time, job j is the task job identified by the j index; for job j Medium PCB i average preparation time.

[0089] Decision 4: Use SMTA to sort the SMT workshop tasks with the weighted minimum delay time as the goal and obtain the SMT workshop scheduling plan.

[0090] In decision 4, the calculation process of sorting the SMT workshop tasks with the weighted minimum delay time as the goal and obtaining the SMT workshop scheduling plan is as follows:

[0091]

[0092]

[0093] Among them, p j for job j Production time; j for job j The order quantity; L j for job j Delay time; T j for job j The start time of j for job j deadline; f is the workshop scheduling plan; r is the number of task operations; ω j for job j The weight of .

[0094] The scheduling work of EMAS is mainly divided into three categories: SMS pre-scheduling, task scheduling and fault scheduling, such as Figure 4 As shown, the basic process is described as follows:

[0095] 1) SMS pre-scheduling: In order to reduce the calculation time and increase the EMAS response speed, pre-decision can be made for new products that will enter the SMT production line to obtain a set of reconstruction alternatives for the SMS station.

[0096] Step 1.1: When receiving the new product PCB i Information, ARA first generates pcbi The reconstruction alternative set RS i .

[0097] Step 1.2: SMSA obtains RS i The CAP and CPSP solutions and their production time for each scheme are obtained and stored in the database.

[0098] 2) Task Scheduling: Because the order quantity affects the preparation time, especially for small batch orders, the impact is particularly serious. When the SMT workshop receives a series of new tasks or the existing tasks change, its workflow is as follows:

[0099] Step 2.1: SMTA calculates the preparation time for each task based on the products and order quantities corresponding to each task.

[0100] Step 2.2: Read the feasible solutions in the set of reconstructed alternative solutions under the current manufacturing environment status from the database, and select the optimal solution taking into account the preparation time.

[0101] Step 2.3: Calculate and obtain the SMT workshop scheduling plan.

[0102] 3) Fault scheduling: When some SMS machines fail and are determined to need to be rescheduled, the fault scheduling calculation begins. The process is as follows:

[0103] Step 3.1: SMTA queries the database to see whether all unfinished tasks in the current manufacturing environment have corresponding reconstruction alternatives. If all are available, go to Step 2.1; if none are available, go to Step 1.1.

[0104] Step 3.2: Calculate the SMT workshop scheduling plan. In this step, there are 3 possibilities:

[0105] 1) If all remaining tasks have feasible reconstruction alternatives, a new SMT workshop scheduling solution is calculated.

[0106] 2) If some tasks have feasible reconstruction alternatives, these tasks will be rescheduled and other tasks will be suspended.

[0107] 3) If no feasible reconstruction alternatives can be obtained for all tasks, the production line will be shut down.

[0108] Furthermore, there are many challenges in using deep reinforcement learning algorithms for scheduling problems, such as: 1. Scheduling problems are very complex, with high dimensions, huge solution space, and strong coupling between actions; 2. There are a large number of illegal actions; for example, the assigned processes, equipment, and task operations cannot be assigned again, so a large number of illegal actions will be generated, and the illegal actions will continue to grow with the increase in the number of steps until the assignment is completed. To this end, careful consideration needs to be given to the action, state, environment, and algorithm design. For the problem of a large number of illegal actions, the method of using invalid masks usually has good scalability, while the method of invalid behavior penalty will fail. Due to the greater complexity of the shop scheduling problem (JSP), in order to verify the effectiveness of the invalid mask in the shop scheduling problem, the algorithm of this embodiment designs the corresponding invalid mask and verifies it; in terms of action design, for SMSA, because of the strong correlation between CAP and CPSP, therefore, under the consideration of reconstruction alternatives, a joint decision-making method is adopted, and the action design is to select a component to be assigned to a machine in an SMS unit; for SMTA, it is to select a task operation. The state information includes different types, multi-level and multi-dimensional information such as products, equipment and processes. In order to meet the data structure requirements of the policy network, the algorithm of this embodiment specially designs a state flattening module to pre-process the data.

[0109] When a new PCB or a fault triggers a reconstruction, ARA generates a set of feasible reconstruction alternatives based on product information. A feasible reconstruction alternative must satisfy the three constraints of formula (3) to formula (5), and its calculation model is shown in Algorithm 1; where nan represents an infeasible solution.

[0110]

[0111]

[0112] SMSA will solve the pcb i Reconstruction Alternative Set RS i CAP, CPSP solutions and minimum production time for all solutions in The calculation process is shown in Algorithm 2.

[0113]

[0114]

[0115] In order to analyze the effectiveness of invalid masking and invalid behavior punishment, a comparative analysis of the two methods was conducted.

[0116] First, the effectiveness of the penalty method was analyzed. The penalty value and the number of components were selected as the two most important factors for correlation comparison during the experimental design. Among them, there were four penalty values, namely -1, -10, -100 and -200, and three numbers of components, namely 6, 10 and 20.

[0117] Table 1 shows the results of the two factor combination tests. None in Table 1 means that there is no feasible solution in one iteration (i.e., a complete scheduling solution cannot be obtained). Table 1 shows the iteration number where none first appears, the proportion of none, and the average feedback value. As the number of components increases, the solution space and action space will increase rapidly. As can be seen from Table 1, the iteration number where none first appears is decreasing, and the proportion of none is increasing. At the same time, as the number of iterations increases, the total number of none is increasing rapidly. Figure 5 As shown in Figure 2, because the iterations exceed 1000, the proportion of feasible solutions decreases rapidly, so Figure 5 Only data within 1000 iterations are shown, and instances with penalty values ​​above -200 and the number of components exceeding 20 are not tested.

[0118] After 15,000 iterations of testing, the penalty method for multiple instances could not converge, while the invalid mask method under the same conditions could converge. Figure 6 shown.

[0119] Table 1 Experimental data of penalty method

[0120]

[0121] When a series of tasks arrive or dynamic events need to be rescheduled, SMTA will start the sorting calculation. Under the current task status and manufacturing environment, it will determine the sorting problem of the tasks to be produced with the weighted minimum delay time as the goal. The calculation process is shown in Algorithm 3.

[0122]

[0123]

[0124] Four PCB products with different numbers of components are selected below to compare the production time of different reconstruction alternatives at different preparation times. The specific data is shown in Table 2. From the data in Table 2, it can be seen that for different task operation parameters, the production time of different reconstruction alternatives is different; in addition, when the number of components is smaller, the proportion of the impact of the reconstruction alternative on the production time is greater.

[0125] Table 2 Example of production time of SMS station

[0126]

[0127]

[0128] The task arrangement of each scheme in Table 2 in the SMS unit is shown in Table 3. In the scheduling plan sorting diagram in the right column, the first number in the bracket represents the serial number of the component on the PCB board (i.e., i), and the second number represents the ID number of the component (i.e., l); the row of numbers below the sorting diagram represents the processing order; the two SMS machines are represented as M1 and M2 respectively.

[0129] Table 3SMS unit dispatch plan

[0130]

[0131] After obtaining the processing data of the SMS station, the scheduling plan for all pending production tasks can be calculated. Table 4 shows the relevant data of each task operation. The obtained scheduling plan is as follows: Figure 7 As shown in Table 5, the target value is 15.7. When job 6 completed 60% of the order volume, an SMS machine broke down and the estimated repair time was long, so EMAS triggered a rescheduling. The updated production data of the remaining unfinished tasks is shown in Table 5, and the new scheduling scheme is as follows: Figure 7 As shown, the new target value becomes 125.54.

[0132] Table 4 Tasks waiting to be processed

[0133]

[0134] Table 5 Tasks that have not been processed when the failure occurs

[0135]

[0136] In summary, the present invention verifies the feasibility of applying deep reinforcement learning algorithms in complex multi-level and multi-sub-problem decision-making with many illegal actions, and successfully solves the dynamic scheduling of SMT workshops as a system problem as a whole, achieving end-to-end; compared with the static organizational structure of the SMT production line, ARSL has higher efficiency and stability in the manufacturing environment where dynamic events occur. When some SMS machines fail, the SMT production line can continue to operate. At the same time, when facing different task operations, EMAS can be adaptively reconstructed, effectively improving the production efficiency of the SMT production line. In addition, compared with the current SMS station optimization that only considers the two sub-problems of CAP and CPSP, the present invention reconstructs the SMT production line and jointly optimizes it, increasing the level and depth of optimization; the present invention solves SMT-FSP as a system problem through an end-to-end multi-agent system that integrates multiple deep reinforcement learning models, thereby improving the production efficiency of the SMT production line.

Claims

1. A dynamic scheduling method for SMT adaptive production line based on deep reinforcement learning, characterized by: The following steps are involved: Step 1: Build an adaptive reconfigurable SMT production line, use a deep reinforcement learning model to train multiple agents, and integrate multiple agents into a multi-agent system; Step 2: Use a multi-agent system to complete the adaptive reconstruction of the SMT workshop scheduling problem: at the SMS workstation, make joint decisions on the SMS unit reconstruction, component allocation problem, and component placement order problem; in the SMT workshop, sort the tasks and obtain the SMT workshop scheduling plan.

2. The dynamic scheduling method of the SMT adaptive production line based on deep reinforcement learning according to claim 1 is characterized in that: In step one, the agents include an adaptive reconstruction agent, an SMS agent and an SMT agent.

3. The dynamic scheduling method of the SMT adaptive production line based on deep reinforcement learning according to claim 2 is characterized in that: In step two, the adaptive reconstruction agent is used to provide a set of reconstruction alternative solutions; the SMS agent is used to obtain the solutions to the component allocation problem, component placement sequence problem and production time for each solution in the reconstruction solution set; the SMT agent is used to determine the sequencing problem of SMT workshop task operations and obtain the SMT workshop scheduling solution.

4. The dynamic scheduling method of the SMT adaptive production line based on deep reinforcement learning according to claim 3 is characterized in that: In step 2, the use of a multi-agent system to complete the adaptive reconfiguration of the SMT workshop scheduling problem includes the following decisions: Decision 1: Based on the number of SMS machines currently operating normally, an adaptive reconstruction agent is used to obtain a set of reconstruction alternatives for the PCB board on the SMS station; the set of reconstruction alternatives satisfies the resource constraints of the batching trough and the nozzle; Decision 2: Use the SMS agent to obtain the solution of the component allocation problem and component placement order problem for each solution in the set of reconstructed alternative solutions, and obtain the minimum production time for each solution at the SMS station; Decision 3: Use the SMT agent to calculate the preparation time for each solution and obtain the minimum production time for each task at the SMS station; Decision 4: Use the SMT agent to sort the SMT workshop tasks with the weighted minimum delay time as the goal and obtain the SMT workshop scheduling plan.

5. The dynamic scheduling method of the SMT adaptive production line based on deep reinforcement learning according to claim 4 is characterized in that: In decision 1, the set of reconstruction alternatives is expressed as: in, is the reconstruction alternative; w is the number of machines currently operating normally in the SMS group.

6. The dynamic scheduling method of the SMT adaptive production line based on deep reinforcement learning according to claim 5 is characterized in that: In decision 1, the resource constraints of the batching trough and the nozzle are: w≤m; (2) Where m is the number of machines in the SMS unit; ct i For PCB i The number of component types on the pcb i is the PCB board identified by the i index; nt i For PCB i fs is the number of nozzle types required on the SMS machine; fs is the number of batching slots on the SMS machine; ns is the type of nozzle on the SMS machine; for The number of SMS machines.

7. The dynamic scheduling method of the SMT adaptive production line based on deep reinforcement learning according to claim 6 is characterized in that: In decision 2, the minimum production time of each solution at the SMS station is calculated as follows: Among them, sx is the x-axis coordinate of the origin of the PCB board on the SMS machine; sy is the y-axis coordinate of the origin of the PCB board on the SMS machine; For components On the PCB i The x-axis coordinate on For PCB i The upper type is comp l The nth component, comp l is the component type identified by the l index; For components On the PCB i The y-axis coordinate on ; for The qth SMS machine in; For components In the The x-axis coordinate of the batching trough on the SMS machine; For components In the The y-axis coordinate of the batching tank on the SMS machine; For In the kth Components processed on SMS machines For components production time; From SMS head to components The x-axis distance of the batching trough; From SMS head to components The y-axis distance of the batching trough; From SMS head to components The x-axis distance of the mounting position; From SMS head to components The y-axis distance of the placement position; v is the moving speed of the SMS head; t1 l For components Placement time; t2 l For components The crawl time; To allocate to Number of components processed by an SMS machine; f s i for production time.

8. The dynamic scheduling method of the SMT adaptive production line based on deep reinforcement learning according to claim 7 is characterized in that: In decision 3, the minimum production time of each task operation at the SMS station is calculated as follows: in, For the job j Medium PCB i The shortest production time, job j is the task job identified by the j index; for job j Medium PCB i average preparation time.

9. The dynamic scheduling method of the SMT adaptive production line based on deep reinforcement learning according to claim 8 is characterized in that: In decision 4, the calculation process of sorting the SMT workshop tasks with the weighted minimum delay time as the goal and obtaining the SMT workshop scheduling plan is as follows: Among them, p j for job j Production time; j for job j The order quantity; L j for job j Delay time; T j for job j The start time of j for job j deadline; f is the workshop scheduling plan; r is the number of task operations; ω j for job j The weight of .

Images