A PCB drilling workshop intelligent scheduling method based on deep reinforcement learning

Abstract

The application discloses a PCB drilling workshop intelligent scheduling method based on deep reinforcement learning, constructs a multi-target parallel machine scheduling model, and abstracts the scheduling process into a Markov decision process, first acquires device information and work order information, establishes optimization targets including total drilling output, device utilization rate, type changing time length, urgent order delivery rate and avoidable idle time, sets device compatibility constraints, stack plate number range constraints, processing time constraints, device capacity constraints and earliest completion time constraints, constructs a state space including global state and enhanced state, defines an action space including No-Op actions, filters unactionable actions through an action mask mechanism, adopts a Dueling-DQN network as a value function approximation model, and combines experience replay, a target network, a masked time difference update, a multi-device random order decision and a dynamic candidate set refreshing mechanism to realize intelligent scheduling optimization in a multi-device parallel distribution scenario of a PCB drilling workshop.

Description

Technical Field

[0001] This invention relates to the field of intelligent manufacturing and production scheduling optimization technology, specifically to an intelligent scheduling method for PCB drilling workshops based on deep reinforcement learning. Background Technology

[0002] PCB drilling workshop scheduling is a typical discrete manufacturing scheduling problem. It has obvious high-dimensional, dynamic and strongly constrained characteristics due to the large number of equipment, complex work order attributes, and the fact that the processing is affected by a variety of factors such as equipment precision, board type, number of holes, area, minimum hole diameter, board thickness and stacking process.

[0003] In actual production, drilling equipment with different precision levels can process different types of work orders. High-precision equipment can be backward compatible with ordinary work orders, while ordinary equipment cannot process high-precision work orders. At the same time, there is a changeover time between different plate types and a changeover time between the same plate type. The work order processing time is also affected by the total number of holes required, the number of drill spindles, and the number of stacked plates. This makes the scheduling decision significantly nonlinear and time-varying.

[0004] Currently, workshop scheduling still largely relies on manual experience or simple rules, making it difficult to simultaneously consider multiple objectives such as total drilling output, equipment utilization rate, rush order delivery rate, changeover losses, and avoidable idle time. Furthermore, in scenarios where multiple machines are idle at the same time, the number of candidate work orders is large, and real-time order insertion occurs, traditional methods are unable to quickly generate high-quality scheduling solutions.

[0005] Furthermore, directly inputting all pending work orders into the scheduling model will cause the state space and action space to expand rapidly, reducing the solution efficiency. If rules such as equipment compatibility, work order status, and planned time limits are not explicitly incorporated into the decision-making process, it is also easy to cause problems of illegal or inefficient allocation. Summary of the Invention

[0006] This paper designs an intelligent scheduling method for PCB drilling workshops based on deep reinforcement learning, including the following steps: To address the challenges of varying equipment precision levels, diverse work order attributes, nonlinear processing times due to stacking processes, frequent board changes, and time-consuming and inaccurate manual scheduling in PCB drilling workshops, this study acquires equipment and work order information from the workshop. A multi-objective scheduling mathematical model is established for parallel allocation of multiple devices. The workshop scheduling process is abstracted as a Markov decision process. A candidate work order set and action set are generated for currently idle devices. An action mask is constructed based on the candidate work order status, equipment precision matching relationship, and planned time limit. A Dueling-DQN network is used to evaluate the action value of the current device. When multiple idle devices exist simultaneously, the idle device set is randomly shuffled, and decisions are made for each device individually. After all idle devices have completed their decisions at the current time, the system time is advanced, and the agent strategy is updated, resulting in a scheduling scheme that satisfies both equipment compatibility and process constraints. The acquisition of equipment information and work order information in the PCB drilling workshop refers to: acquiring equipment accuracy level, equipment operating status, equipment idle status, current remaining processing time and equipment processing capacity, as well as work order model, priority, area, board thickness, minimum drill bit diameter, total number of holes required, delivery date, earliest processing time and accuracy requirements; The aforementioned establishment of a multi-objective scheduling mathematical model refers to: based on the process characteristics of the PCB drilling workshop, taking total drilling output, equipment utilization rate, changeover time, rush order delivery rate, and avoidable idle time as optimization objectives, and setting equipment compatibility constraints, stacking number range constraints, processing time constraints, equipment capacity constraints, and earliest completion time constraints.

[0007] Furthermore, a deep reinforcement learning-based intelligent scheduling method for PCB drilling workshops is proposed, wherein a multi-objective scheduling mathematical model is established, and the PCB drilling workshop satisfies the following modeling assumptions: All drilling equipment in the workshop has the same single-axis drilling speed and the same number of drill shafts; The equipment is only subject to planned maintenance. Once a work order begins processing, it cannot be interrupted. The actual processing time of a work order is affected by the number of stacked boards; The board replacement time is fixed when switching between the same board type; The changeover time between different plate types is fixed; The parameters in a multi-objective scheduling mathematical model should include at least: Total number of holes required for the work order Work order area Work order priority Work order material thickness Minimum drill bit diameter for work order Single-axis reference drilling speed Maximum number of drill spindles per unit Planning timeframe Work order delivery time Earliest processing time for work orders And equipment from work orders Switch to work order The time required for model change ; The optimization objectives in the mathematical model include: The expression for maximizing the total number of boreholes is: ,in, For work orders A binary variable indicating whether completion is possible; To maximize equipment utilization, the expression is: ,in, For work orders Is it in the device? Binary variables processed on the upper part, For work order change or board replacement time; The expression for minimizing the total transformation time is: ,in, Penalty for changing models within a unit of time. Indicates device Work order and work orders The time required for changing models or boards between different components; The expression for maximizing the delivery rate of rush orders is: ; The expression for minimizing the always avoidable idle time is: ,in, For equipment At any moment The idle state indicator variable, This is a non-empty indicator variable for the queue of processes; The multi-objective scheduling mathematical model satisfies the following constraints: equipment compatibility constraints, which restrict high-precision work orders from being assigned to equipment that does not meet the precision requirements; The range constraint for the number of stacked plates is expressed as follows: ,and: ,in, For work orders The number of stacked plates, For work orders Maximum number of stacked plates; The processing time constraint is expressed as: ,in, For work orders Processing time; Equipment capacity constraints are expressed as follows: The earliest completion time constraint is expressed as: ,in, For work orders Completion time; The mathematical model constructs a comprehensive optimization objective through a weighted approach, and its expression is as follows: ,in, , , , and These are the weighting coefficients for the five sub-objectives, which are derived from the scores given by the actual companies.

[0008] Furthermore, a method for intelligent scheduling in PCB drilling workshops based on deep reinforcement learning is proposed. The abstraction of the workshop scheduling process into a Markov decision process refers to: constructing a state space containing global state and augmentation state, an action space containing candidate work order actions and No-Op actions, and a reward function corresponding to the scheduling effect; The aforementioned generating a candidate work order set and constructing an action set refers to: for the currently idle equipment, generating a candidate work order set from the currently processable work orders, and constructing an action set based on the candidate work order set, including the No-Op action of assigning a certain candidate work order and not assigning a work order; The aforementioned action mask refers to: based on whether the candidate work order is empty, whether the work order is in a pending processing state, whether the equipment accuracy meets the work order accuracy requirements, and whether the completion time after allocation exceeds the daily plan limit, to mask inactive actions, so as to ensure that action selection is only performed within the range of active actions; The Markov decision process is represented as follows: ,in For state space, For the action space, For the reward function, Discount factor; The global state is represented as follows: ,in, Indicates the progress of time within the day. This indicates the percentage of idle time for devices with different levels of precision. Indicates priority The percentage of holes to be processed This indicates the proportion of holes to be processed to the total number of holes; The enhanced state is represented as follows: ,in, For equipment The accuracy level, For the first Feature vectors of candidate work orders; The candidate work order feature vector is represented as follows: ,in, For feasibility, For equipment and work order matching indicator variables, This is the normalized value for the number of holes. Prioritize work orders. For the accuracy level of the work order, This is the area normalized value; The action space is represented as follows: Among them, actions A No-Op action indicates that no work order will be assigned to the device in this instance. To action Corresponding to the first in the candidate work order set To the Work order.

[0009] Furthermore, a method for intelligent scheduling in PCB drilling workshops based on deep reinforcement learning is proposed. The device at time The candidate work order set is represented as The candidate work order set is obtained by heuristically sorting the processable work orders according to priority, matching degree and area and then truncating them.

[0010] Furthermore, a method for intelligent scheduling in PCB drilling workshops based on deep reinforcement learning is proposed. The action mask is represented as follows: The inaction includes at least the following: the candidate position is empty, the work order is not in a pending state, the equipment accuracy cannot meet the work order accuracy requirements, and the completion time after allocation exceeds the daily plan limit. The action selection rule is expressed as follows: This ensures that the optimal choice is made only within the set of possible actions.

[0011] Furthermore, a method for intelligent scheduling in PCB drilling workshops based on deep reinforcement learning is proposed. The reward function includes allocation phase rewards, time progression phase rewards, and completion rewards: The allocation phase reward is used to evaluate the feasibility and priority rationality of a single allocation action; The time-progression phase reward is used to reflect the percentage of busy equipment, the percentage of avoidable idle equipment, and the percentage of replacement equipment. The completion reward is used to settle rewards at the end of the round based on the completion rate of high-priority orders, the completion rate of total number of holes, and the completion rate of area.

[0012] The reward for the allocation phase is represented as follows: ,in, This indicates the priority difference between the currently selected work order and the best priority work order in the candidate set.

[0013] The reward for the time progression phase is represented as follows: ,in, For the percentage of busy devices, To avoid the proportion of idle devices, The percentage of replacement equipment. This represents the time difference between the next event and the current event.

[0014] The completion reward is represented as follows: ,in, The actual number of holes completed for high-priority work orders. For high-priority work orders, the required number of holes is specified. The total number of holes completed. The total number of holes required. The total completed area, This represents the total required area. Its total reward function can be expressed as: .

[0015] Furthermore, a method for intelligent scheduling in PCB drilling workshops based on deep reinforcement learning is proposed. The specific details of constructing the Dueling-DQN intelligent scheduling algorithm are as follows: The construction of the Dueling-DQN intelligent scheduling algorithm refers to the following: the algorithm includes candidate work order generation, enhanced state construction, action value assessment, decision-making for multiple idle devices, time advancement, experience replay, and strategy update. The Dueling-DQN network comprises a shared feature layer, a value branch, a dominance branch, and an aggregation layer. The value branch outputs the state value, the dominance branch outputs the action advantage, and the aggregation layer obtains the Q-value for each action. The standard Q-function is defined as: ,in For future discount returns; The value branch outputs the state value. This indicates the degree of good or bad in the current state; The advantage branch output action advantage This indicates the advantage of choosing a certain action relative to the average level. The aggregation layer obtains the Q-value of each action based on the state value and action advantage, and its expression is: The Dueling-DQN network architecture details, defined as follows: , Specific network layer definitions: Feature layer: , Value Stream: , Advantage Stream: , in ; Input Dimensions : Global status dimension + device feature dimension + candidate work order feature dimension; Output dimensions: (No-Op + N candidate work orders); Network initialization: Uniform Xavier initialization is used to ensure the unbiasedness of the initial Q-value estimation; The training process employs an experience replay mechanism and a target network mechanism: The experience replay mechanism transfers samples by storing state. Training is performed using an experience replay pool and random sampling to break temporal correlation and improve sample efficiency; Define quintuple transitions ,in The action mask for the next state, with a playback pool storage capacity of B. When the buffer is full, FIFO (First-In, First-Out) is used. The experience replay pool has the following functions: breaking temporal correlation, specifically, state transitions at adjacent times are highly correlated, and directly using them for training will cause the network to overfit the current trajectory. The replay pool randomly samples a batch of data, destroying the correlation and allowing the network to learn more general patterns; improving sample efficiency, that is, the same transition may be sampled multiple times, effectively increasing the sample size, especially in the early exploration stage, a small number of high-quality transitions can be repeatedly used.

[0016] The parameter update is performed using the difference objective, and its objective value is expressed as: , in, It is a delayed copy of policy_net, and the loss function is expressed as: The Adam optimizer is used for gradient descent updates.

[0017] Furthermore, a method for intelligent scheduling in PCB drilling workshops based on deep reinforcement learning is proposed. The aforementioned random shuffling of the idle device set and decision-making for each device refers to: when there are multiple idle devices at the same time, the idle device set is randomly sorted, and candidate work order refresh, action mask calculation, state construction and action selection are performed for each device in a random order, so as to reduce the resource allocation deviation caused by the fixed device order; The decision order of the multiple idle devices is achieved by random shuffling, where the set of idle devices is represented as follows: The set is randomly shuffled, and decisions are made on a per-unit basis. When making decisions on multiple idle devices at the same time, the current set of available work orders is re-acquired before each device decision to achieve dynamic candidate set refresh and avoid subsequent devices repeatedly selecting work orders that have been occupied by the preceding device. The aforementioned advancement of system time and updating of agent strategy refers to: advancing system time after all idle devices have completed their decisions at the current moment, and updating agent strategy based on allocation stage rewards, time advancement stage rewards, and completion rewards, thereby obtaining an intelligent scheduling scheme that satisfies process constraints and equipment compatibility constraints.

[0018] The beneficial effects of this invention are as follows: The complex PCB drilling workshop scheduling process is abstracted into a Markov Decision Process (MDP), and intelligent decision-making is achieved using a Dueling-DQN network. This method enables millisecond-level scheduling response, reduces reliance on human experience, significantly shortens scheduling time, avoids human error, and significantly improves the accuracy and scientific rigor of scheduling results. The model precisely sets multi-dimensional constraints such as equipment compatibility, stack size range, processing time, equipment capacity, and earliest completion time. By incorporating nonlinear factors such as stack size, it ensures that the generated scheduling scheme fully conforms to actual production conditions, avoiding on-site execution conflicts caused by infeasible solutions. A state space containing global and enhanced states is constructed, and No-Op (no-operation) actions are innovatively introduced into the action space. Simultaneously, an action masking mechanism is used to filter out infeasible actions limited by equipment capacity and process constraints in real time. This not only significantly reduces the agent's action search space, avoiding ineffective exploration and accelerating the convergence speed of reinforcement learning models, but also allows the agent to proactively relinquish equipment when there are no suitable processing tasks, avoiding efficiency losses caused by forced allocation. A multi-objective parallel machine scheduling model was constructed, incorporating total drilling output, equipment utilization, changeover time, rush order delivery rate, and avoidable idle time. Through multi-device random sequential decision-making and masked temporal differential updates, the agent can autonomously find the optimal balance point among multiple conflicting optimization objectives, ensuring both high output and high utilization rate while effectively reducing time losses caused by frequent changeovers. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the overall process of the method of the present invention. Figure 2 This is a schematic diagram illustrating the construction of the global state and the enhanced state in this invention; Figure 3 This is a flowchart of the multi-device random sequential decision-making and dynamic candidate set refresh process in this invention; Figure 4 This is a schematic diagram of the Dueling-DQN network structure in this invention; Figure 5 Accumulate reward convergence curves during the training process; Figure 6 Gantt chart showing the daily production targets for drilling rig equipment; Figure 7 This is a Gantt chart showing the production tasks for the material number on that day. Detailed Implementation

[0020] This invention provides an intelligent scheduling method for PCB drilling workshops based on deep reinforcement learning. It is applicable to production scenarios with multiple drilling machines of different precision levels, multiple work orders competing for equipment resources in parallel, and the presence of stacking processes, type and board change constraints, priority constraints, and dynamic disturbances. This method establishes a multi-objective scheduling mathematical model, abstracts the workshop scheduling process into a Markov decision process, and constructs an intelligent scheduling algorithm based on Dueling-DQN to output a scheduling scheme that satisfies equipment compatibility constraints and process constraints.

[0021] In a small-scale embodiment, a PCB drilling workshop simulation environment containing 3 drilling machines and 20 orders is constructed to illustrate the modeling method, decision-making process, and scheduling effect verification process of the present invention. Among them, the machines numbered 0 and 1 are ordinary drilling machines, the machine numbered 2 is a high-precision drilling machine, the high-precision drilling machine can process high-precision orders and is also backward compatible with ordinary orders, while the ordinary drilling machine is only used to process regular PCB orders.

[0022] Each device maintains status information such as current task, running status, device idle status, current drill bit diameter, remaining processing time, cumulative effective processing time, and cumulative changeover time. The processing capacity of the device is determined by the single-axis reference drilling speed and the maximum number of drill axes per device. In this embodiment, the single-axis reference drilling speed can be set to 1666 holes / minute, and the maximum number of available drill axes per device can be set to 6.

[0023] Regarding work order data, the 20 orders can be automatically constructed by the random generator within the environment or imported from the enterprise's order master data. Each work order includes at least fields such as model, priority, area, plate thickness, minimum drill bit diameter, total number of holes required, delivery date, earliest processing time, and accuracy requirements, and can further include attribute information such as material quantity, high-precision attributes, and plate model.

[0024] In this embodiment, orders are divided into different precision levels, some orders are marked as high precision orders, and are strictly restricted to be assigned only to high precision drilling equipment; the order priority can be set to a discrete level from 1 to 4, the smaller the priority value, the higher the priority, which is used to characterize the scheduling priority relationship between urgent orders and ordinary orders, the corresponding relationship is (1: urgent, 2: extremely urgent, 3: important, 4: normal), the specific order information is shown in Table 1 below.

[0025] Table 1 Order Information Table Regarding the board type switching rules, when the board types of continuous processing work orders are the same, only a board change operation needs to be performed, and the board change time can be set to 10 minutes; when the board types of continuous processing work orders are different, a board type switching operation needs to be performed, and the board type switching time can be set to 40 minutes, thereby reflecting the time difference between continuous processing of the same board type and switching across board types in a real workshop.

[0026] Regarding the calculation of processing time, the processing time for each work order is jointly determined by the total number of holes required, the single-axis reference drilling speed, the maximum number of drill axes, and the number of stacked plates; the number of stacked plates is constrained by the work order plate thickness and the minimum drill bit diameter, and its processing time can be calculated according to... Calculate, where, Where r is the total number of holes required in the work order, and r is the single-axis reference drilling speed. This represents the maximum number of drill spindles for the equipment. This represents the number of stacked plates in the work order.

[0027] To achieve a unified description of scheduling objectives, a multi-objective scheduling mathematical model for PCB drilling workshops is first established. The mathematical model takes total drilling output, equipment utilization rate, changeover time, rush order delivery rate, and avoidable idle time as optimization objectives, and sets constraints such as equipment compatibility, stacking number range, processing time, equipment capacity, and earliest completion time.

[0028] After establishing the mathematical model, the workshop scheduling process is abstracted into a Markov decision process, in which... For state space, For the action space, For the reward function, This serves as a discount factor; through this abstraction, complex scheduling problems can be transformed into an iterative decision-making process of "perceiving the current state - selecting an action - obtaining a reward - transitioning to the next state".

[0029] To facilitate the display of scheduling results from an order management perspective, this embodiment further outputs an order scheduling table, as shown in Table 2. Table 2 includes the order ID, whether it is an automotive order, priority, compatible board type, total number of part numbers, expected completion status for the day, number of completed part numbers, expected start time, expected completion time, and assigned equipment ID, which is used to display the overall production scheduling results at the order level.

[0030] Table 2 Order Scheduling Table At this point, the priority correspondence in the table is (1: Urgent, 2: Top Urgent, 3: Important, 4: Normal). As can be seen from Table 2, high-priority orders are prioritized. Some orders are expected to be completed within normal shifts, some are expected to be completed during overtime, and a small number of orders remain pending. This reflects the comprehensive balance of priority constraints, equipment compatibility constraints, and delivery time requirements in this invention. Taking order TEST-D1-004 as an example, this is a non-automotive sheet general order with a priority of "Urgent" (highest priority), totaling 2 part numbers. As can be seen from the scheduling table, due to its highest priority, the agent places it at the beginning of the daily work start, with an expected start time of minute 0. Furthermore, because it is a general order, it is reasonably assigned to two general drilling rigs numbered 1 and 2 for processing, with an expected completion time of minute 150.

[0031] To facilitate the display of work results from the perspective of equipment execution, this embodiment further outputs a part number scheduling table, as shown in Table 3. Table 3 includes information such as equipment ID, part number ID, compatible plate type, whether it is an automotive order, priority, estimated start time, estimated end time, completion status, and number of stacked plates, which are used to reflect the specific work sequence and execution arrangement of each drilling rig.

[0032] Table 3. Material Number Scheduling Table As can be seen from Table 3, tasks on the same device are arranged sequentially in chronological order, with a model change interval between different models, while high-precision orders are scheduled to be executed on compatible devices.

[0033] To further visualize the equipment-level work schedule, a Gantt chart of the drilling rig's daily production tasks was drawn based on the scheduling results in Table 2, as shown below. Figure 6 As shown; Figure 6 The diagram illustrates the task distribution of three drilling rigs within a single day's production cycle, the distinction between regular and high-precision orders, and the normal closing timeline, providing a clear picture of equipment utilization and changeover intervals. The diagram shows that identical patterns represent the same plate type; two orders with the same pattern have a changeover time of approximately 10 minutes, while different patterns representing different plate types require a changeover time of 40 minutes. The red border marks automotive orders, which are high-precision orders.

[0034] To further illustrate the start, waiting, and completion status of orders at different levels, a Gantt chart of the daily production tasks for each part number is drawn based on the order results in Table 3, as shown below. Figure 7 As shown; Figure 7In the diagram, gray sections represent pending or waiting status, green task blocks represent normal completion, orange task blocks represent completion with overtime, and red vertical dashed lines represent normal off-duty times. This visually reflects the lifecycle and delivery time fulfillment status of each order on the daily timeline.

[0035] In addition to the small-scale embodiments described above, the present invention can also be applied to large-scale scheduling scenarios in real PCB factories. In such scenarios, the system can directly read the production schedule and material master data files of the enterprise for several consecutive days, perform unified modeling of a large number of equipment and part numbers, and calculate the actual processing constraints based on attributes such as material size, area, board thickness, minimum hole diameter, and whether it is a high-precision board.

[0036] In real-world factory applications, the processing time of a work order is not solely determined by the total number of holes required, but is highly dependent on the number of stacked plates. Therefore, the system can load a real drilling stacking rule table, dynamically match the maximum allowed number of stacked plates based on the material plate thickness range and the minimum hole diameter, and accurately calculate the processing time of a single task in conjunction with the equipment's baseline drilling rate, thereby making the simulation scheduling results closer to the real production process.

[0037] Furthermore, to adapt to continuous rolling scheduling scenarios, the system can load the initial occupancy status of equipment during daily scheduling initialization. Specifically, it reads the equipment occupancy data passed down from the previous shift, deducts the corresponding occupancy time for equipment that is undergoing changeover or processing, and sets its status to external occupancy status. Scheduling is then opened after the initial occupancy time is exhausted, thereby making the cross-day scheduling connection smoother.

[0038] For extremely large work orders with an excessive total number of holes required, the system can also set a maximum hole count threshold for a single task. When the total number of holes required for a work order exceeds this threshold, the task splitting logic is automatically triggered, splitting it into multiple sub-tasks with suffix identifiers, and processing them in parallel across multiple devices to reduce the risk of global congestion caused by long-term monopolization of a single device.

[0039] In addition, to improve the responsiveness to actual production disturbances, this invention can also introduce an emergency order insertion mechanism. For work orders marked as emergency tasks, their physical arrival time can be preset. Before the trigger time, the work order is in an invisible waiting state. When the time reaches the trigger node, the environmental state is forcibly refreshed, the emergency work order is converted to a pending state and given the highest priority. The intelligent agent quickly executes rescheduling decisions based on the current busy / idle status of the equipment.

[0040] from Figures 5 to 7As can be seen from the results in Tables 1 to 3, the intelligent scheduling method based on Dueling-DQN proposed in this invention can effectively handle the complex constraints of the PCB drilling workshop. Under the premise of meeting constraints such as equipment compatibility, stacking process, and changeover, it achieves multiple optimization goals such as maximizing total output, prioritizing urgent orders, high equipment utilization, and minimizing changeover losses.

[0041] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent scheduling in a PCB drilling workshop based on deep reinforcement learning, characterized in that, Includes the following steps: This paper obtains equipment information and work order information in the PCB drilling workshop, establishes a multi-objective scheduling mathematical model for the parallel allocation of multiple devices, abstracts the workshop scheduling process into a Markov decision process, generates a candidate work order set and constructs an action set for the currently idle devices, constructs an action mask based on the candidate work order status, equipment accuracy matching relationship and planned time limit, and uses a Dueling-DQN network to evaluate the action value of the current devices. When there are multiple idle devices at the same time, the idle device set is randomly shuffled and decisions are made for each device. After all idle devices have completed their decisions at the current time, the system time is advanced and the agent strategy is updated, thereby obtaining a scheduling scheme that satisfies equipment compatibility constraints and process constraints. The acquisition of equipment information and work order information in the PCB drilling workshop refers to: acquiring equipment accuracy level, equipment operating status, equipment idle status, current remaining processing time and equipment processing capacity, as well as work order model, priority, area, board thickness, minimum drill bit diameter, total number of holes required, delivery date, earliest processing time and accuracy requirements; The aforementioned establishment of a multi-objective scheduling mathematical model refers to: based on the process characteristics of the PCB drilling workshop, taking total drilling output, equipment utilization rate, changeover time, rush order delivery rate, and avoidable idle time as optimization objectives, and setting equipment compatibility constraints, stacking number range constraints, processing time constraints, equipment capacity constraints, and earliest completion time constraints. The abstraction of the workshop scheduling process into a Markov decision process refers to: constructing a state space containing global state and augmentation state, an action space containing candidate work order actions and No-Op actions, and a reward function corresponding to the scheduling effect; The aforementioned generating a candidate work order set and constructing an action set refers to: for the currently idle equipment, generating a candidate work order set from the currently processable work orders, and constructing an action set based on the candidate work order set, including the No-Op action of assigning a certain candidate work order and not assigning a work order; The aforementioned action mask refers to: based on whether the candidate work order is empty, whether the work order is in a pending processing state, whether the equipment accuracy meets the work order accuracy requirements, and whether the completion time after allocation exceeds the daily plan limit, to mask non-actionable actions, so as to ensure that action selection is only performed within the range of actionable actions; The construction of the Dueling-DQN intelligent scheduling algorithm refers to the following: the algorithm includes candidate work order generation, enhanced state construction, action value assessment, decision-making for multiple idle devices, time advancement, experience replay, and strategy update. The aforementioned random shuffling of the idle device set and decision-making for each device refers to: when there are multiple idle devices at the same time, the idle device set is randomly sorted, and candidate work order refresh, action mask calculation, state construction and action selection are performed for each device in a random order, so as to reduce the resource allocation deviation caused by the fixed device order; The aforementioned advancement of system time and updating of agent strategy refers to: advancing system time after all idle devices have completed their decisions at the current moment, and updating agent strategy based on allocation stage rewards, time advancement stage rewards, and completion rewards, thereby obtaining an intelligent scheduling scheme that satisfies process constraints and equipment compatibility constraints.

2. The intelligent scheduling method for PCB drilling workshop based on deep reinforcement learning as described in claim 1, characterized in that, The specific details of establishing the multi-objective scheduling mathematical model are as follows: The PCB drilling workshop satisfies the following modeling assumptions: all drilling equipment in the workshop has the same single-axis drilling speed and the same number of drilling axes; only planned maintenance is considered for the equipment; once a work order starts processing, it cannot be interrupted; the actual processing time of a work order is affected by the number of stacked boards. The board changeover time is fixed when switching between the same board type; the board changeover time is fixed when switching between different board types; The parameters in the mathematical model include at least: total number of holes required for the work order, work order area, work order priority, work order plate thickness, minimum drill bit diameter for the work order, single-axis reference drilling speed, maximum number of drill axes per machine, planning time cycle, work order delivery date, earliest processing time for the work order, and changeover time for the equipment when switching between different work orders. The optimization objectives in the mathematical model include: maximizing the total number of boreholes, maximizing equipment utilization, minimizing the total changeover time, maximizing the rush order delivery rate, and minimizing the total avoidable idle time; Furthermore, the multi-objective scheduling mathematical model should satisfy the following constraints: equipment compatibility constraints, which restrict high-precision work orders from being assigned to equipment with insufficient precision; stacking plate range constraints; processing time constraints; equipment capacity constraints; and earliest completion time constraints. In summary, the mathematical model constructs a comprehensive optimization objective through a weighted approach.

3. The intelligent scheduling method for PCB drilling workshop based on deep reinforcement learning as described in claim 1, characterized in that, The abstraction of the workshop scheduling process into a Markov decision process refers to: constructing a state space containing global state and augmentation state, an action space containing candidate work order actions and No-Op actions, and a reward function corresponding to the scheduling effect; The workshop scheduling process is abstracted as a Markov decision process, and its process representation is as follows: ,in For state space, For the action space, For the reward function, Discount factor; The global status includes: daily time progress, idle percentage of equipment with different precision, percentage of holes to be processed for each priority, and percentage of holes to be processed to the total number of holes. The enhanced status includes: the current accuracy level of the device and the feature vector of the candidate work order; The candidate work order feature vector includes a feasibility bit, a device-work order matching indicator variable, a hole number normalized value, a work order priority, a work order accuracy level, and an area normalized value. The action space of the No-Op action is represented as follows: Among them, actions A No-Op action indicates that no work order will be assigned to the device in this instance. To action Corresponding to the first in the candidate work order set To the One work order; The reward function includes allocation stage rewards, time progression stage rewards, and completion rewards.

4. The intelligent scheduling method for PCB drilling workshop based on deep reinforcement learning as described in claim 1, characterized in that, For currently idle equipment, the candidate work order set is obtained by heuristically sorting the processable work orders according to priority, matching degree and area and then truncating them.

5. The intelligent scheduling method for PCB drilling workshop based on deep reinforcement learning as described in claim 1, characterized in that, The action mask is represented as follows: The inactive actions include: the candidate position is empty, the work order is not in a pending state, the equipment accuracy cannot meet the work order accuracy requirements, and the completion time after allocation exceeds the daily plan limit. The action selection rule is expressed as follows: ， This ensures that the optimal choice is made only within the set of possible actions.

6. The intelligent scheduling method for PCB drilling workshop based on deep reinforcement learning as described in claim 3, characterized in that, The reward function includes allocation stage rewards, time progression stage rewards, and completion rewards, wherein: The allocation phase reward is used to evaluate the feasibility and priority rationality of a single allocation action, and it is determined based on the priority difference between the currently selected work order and the best priority work order in the candidate set. The time-progression phase reward is used to reflect the percentage of busy equipment, the percentage of avoidable idle equipment, and the percentage of equipment to be replaced. It is determined based on the time difference between the next event time and the current time and the relevant equipment percentage. The completion reward is used to settle rewards at the end of the round based on the completion rate of high-priority orders, the completion rate of total number of holes, and the completion rate of area.

7. The intelligent scheduling method for PCB drilling workshop based on deep reinforcement learning as described in claim 1, characterized in that, The specific details of constructing the Dueling-DQN intelligent scheduling algorithm are as follows: The Dueling-DQN network comprises a shared feature layer, a value branch, a dominant branch, and an aggregation layer. The shared feature layer is used to extract representations of the input features. The value branch outputs state values, the dominant branch outputs action advantages, and the aggregation layer combines state values ​​and action advantages to obtain the Q-value of each action. The network input dimension is the sum of the global state dimension, the device feature dimension, and the candidate work order feature dimension, and the output dimension is the sum of the No-Op actions and the number of candidate work orders. During training, an experience replay mechanism and a target network mechanism are employed: the experience replay mechanism stores state transfer samples in the experience replay pool and performs training by random sampling, and replaces them with a first-in-first-out strategy when the buffer is full; and a differential target is used to update parameters to optimize the network.

8. The intelligent scheduling method for PCB drilling workshop based on deep reinforcement learning as described in claim 1, characterized in that, The process of randomly shuffling the set of idle devices and making decisions for each device individually, with multiple idle devices selected using the following specific methods: When multiple idle devices exist at the same time, the set of idle devices is randomly shuffled to achieve a random sorting of the decision order, and decisions are made on each device in a random order. When making decisions on multiple idle devices at the same time, the current set of available work orders is retrieved again before each device makes a decision, so as to achieve dynamic candidate set refresh and avoid subsequent devices repeatedly selecting work orders that have been occupied by the preceding device.

Images