Resource allocation and process planning optimization method under complex constraint conditions

Abstract

The invention relates to a resource allocation and process planning optimization method under complex constraint conditions, which comprises the following steps of: representing various constraint conditions of a process planning process by utilizing a double-layer directed acyclic graph, and generating and constructing an initial feasible solution set meeting all the constraint conditions; initializing a state space, wherein the state space comprises state variables of a process number, an average process length, reward value fluctuation and fitness; noise is collected and coupled to a Q network, action selection is carried out, and low-level heuristic algorithm options are output; generating a new taboo sequence by using the selected low-level heuristic algorithm, and optimizing the current feasible solution; switching a low-level heuristic algorithm according to the improvement amplitude and the execution duration; and outputting an optimal processing feature sequence and process division combination after multiple times of training. Compared with the prior art, the method has the advantages that multiple process decisions can be optimized at the same time, sub-optimal results caused by separate optimization processes are avoided, processing time and resource consumption are remarkably reduced, and production efficiency and scheme adaptability are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent production planning, and in particular to a method for optimizing resource allocation and process planning under complex constraints. Background Art

[0002] Process planning in industrial manufacturing involves the sorting of machining features, the division of processes, and the allocation of machining resources. Traditional process planning methods mostly use pre-set fixed rules: for example, first locking the positions of some key machining features, and then sorting the remaining features and allocating resources. This strategy may be effective in specific scenarios, but it lacks flexibility and adaptability overall, and it is difficult to cope with changing production needs. In addition, due to the complexity of process planning problems, existing technologies often solve the three sub-problems of sorting machining features, dividing processes, and allocating resources separately, resulting in the inability to globally optimize the process planning scheme, resulting in resource waste and inefficiency. As the manufacturing industry develops towards high-variety, small-batch, and customized directions, the above limitations pose greater challenges to the production efficiency and resource utilization of enterprises.

[0003] In order to solve the optimization problem of complex process planning, researchers have introduced various meta-heuristic algorithms and hyper-heuristic algorithms. Meta-heuristic algorithms (such as genetic algorithms, particle swarm optimization, etc.) perform well in solving NP (Non-deterministic Polynomial) difficult combinatorial optimization problems and have been used in process planning optimization to improve solution efficiency and result quality. However, a single meta-heuristic algorithm is usually only applicable to specific scenarios or optimization stages, and its performance may drop sharply when the problem environment or optimization stage changes. In contrast, the hyper-heuristic algorithm provides a framework for high-level strategies to dynamically control multiple underlying heuristic algorithms. By constructing high-level selection strategies and underlying algorithm pools, it can flexibly switch algorithms at different stages, thereby taking into account global search and local optimization. This approach has demonstrated flexible and effective performance in many fields.

[0004] However, the direct application of hyper-heuristic algorithm to process planning with complex constraints still faces the following bottlenecks: ① Lack of unified process planning coding method and solving strategy: there is no standardized method to uniformly encode and solve the process planning problem in the existing technology, especially when the constraint condition is complex, which becomes an obstacle to the application of hyper-heuristic algorithm. For example, for the three related sub-problems of machining feature sequencing, process partitioning and resource allocation, the lack of unified representation makes it difficult to integrate them into an optimization framework for solving. ② It is difficult to determine the jumping time of low-level heuristic algorithm: in the hyper-heuristic framework, the upper layer needs to decide when to switch the algorithm in the algorithm pool. However, if the jumping time of low-level algorithm is not planned properly, it may cause the algorithm to stay in local optimum for too long or jump too early, resulting in insufficient search. This uncertainty of jumping time limits the adaptive ability of the algorithm in different optimization stages. ③ It is difficult to determine the balance between exploration and development: in the optimization process, different stages need to balance between global exploration (finding new solutions) and local development (improving current solutions). If there is no effective strategy to balance exploration and development, the algorithm may be too greedy to fall into local optimum, or too random to converge, thereby affecting the improvement of global optimization performance.

[0005] In summary, the existing process planning optimization technology has the problems of poor adaptability, lack of unified optimization framework and difficulty in algorithm regulation, which cannot meet the optimization demand under complex constraint conditions and is not conducive to reducing processing time and resource consumption. SUMMARY

[0006] The purpose of the present application is to overcome the defects of the prior art and provide a method for simultaneously optimizing resource allocation and process planning under complex constraint conditions, which can realize objective description of optimization stages, solve the problem of selecting heuristic algorithm under different optimization stages, accelerate optimization efficiency and reduce production cost.

[0007] The purpose of the present application can be realized by the following technical scheme: a resource allocation and process planning optimization method under complex constraint conditions, comprising the following steps:

[0008] S1, using a double-layer directed acyclic graph to represent various constraint conditions of the process planning process, and generating an initial feasible solution set that meets all the constraint conditions;

[0009] S2, initializing the state space, including state variables of process number, average process length, reward value fluctuation and fitness;

[0010] S3, collecting noise and coupling it to the Q network to select action, and outputting low-level heuristic algorithm options;

[0011] S4, generating a new tabu sequence using the selected low-level heuristic algorithm to optimize the current feasible solution;

[0012] S5. Switch to the low-level heuristic algorithm based on the improvement and execution time, and then return to step S3;

[0013] S6. After multiple trainings, the optimal processing feature sequence and process division combination are output.

[0014] Furthermore, the various constraints in the process planning process in step S1 include priority constraints of machining features, joint machining constraints of machining features, sequence constraints for machine tool use, and machining time constraints.

[0015] Furthermore, the specific process of step S1 is as follows:

[0016] S11. Collect processing feature table Machine tool / tool ​​collection And determine the constraint set, including the pre-order constraint Joint constraints Machine tool priority constraints Processing time constraints

[0017] S12. By joint constraint Merge the features that need to be "bundled" into directed acyclic subgraphs; if two directed acyclic subgraphs have an intersection, that is, the same feature appears in different subgraphs, they are merged;

[0018] S13, treat each directed acyclic subgraph as a node, and add the machine tool node with the machine tool priority to the same graph, using the pre-order constraint and machine priority constraints Add directed edges between nodes to obtain a two-layer directed acyclic graph;

[0019] S14. If a loop appears in the two-layer directed acyclic graph, an edge is deleted based on processing experience to break the loop, ensuring that the topological sorting is feasible, and obtaining a two-layer directed acyclic topological graph;

[0020] S15. Eliminate the subgraphs and features to be processed one by one according to the taboo sequence to generate and construct an initial feasible solution set, wherein the taboo sequence refers to the code used to record the currently selected processing feature subgraph or processing feature sequence during the optimization search process.

[0021] Furthermore, the specific process of step S15 is: using a forward topological sorting algorithm to generate an initial feasible solution on a two-layer directed acyclic topological graph, wherein all constraints are followed during the topological sorting process: a node, that is, a processing feature or a feature subset, can only be arranged in the processing sequence when all its predecessor dependencies have been arranged. In specific implementation, sorting is started from the node with an in-degree of 0, that is, no predecessor dependency, and it is added to the current process sequence. Then, the node is removed and the in-degree of the relevant node is updated. This process is repeated until all feature nodes are processed, that is, a complete processing feature sequence is obtained. If the accumulated processing time of a certain process during the sorting process exceeds the preset upper limit, a new process is divided at the exceeding point.

[0022] Furthermore, the step S2 of initializing the state space is specifically as follows:

[0023]

[0024] in, Indicates the number of current processes, Indicates the average length of the current process; Indicates the change in reward value compared to the previous step; f t It represents the fitness under the current solution, and its calculation formula is:

[0025] f t =1000000 / (F-200000)

[0026] Among them, 1000000 and 200000 are experimental experience hyperparameters formulated based on the cost of machine tools and cutting tools. F represents the processing cost under the current processing feature sequence division, which can be specifically expressed as:

[0027]

[0028] in, represents the number of the m-th type of machine tools in the k-th process, represents the price of using the mth type of machine tool, is a binary variable, which takes the value 1 if and only if the type m machine tool is used for the k-th process, otherwise it takes the value 0; The unit price of the tool selected for the i-th processing feature; x ij is a binary decision variable, which takes the value 1 if and only if the i-th feature is placed in the j-th position, otherwise it takes the value 0; h. They represent the total number of operations, the total number of machine tool types, the total number of machining features, and the total number of machining positions, respectively.

[0029] By introducing the above state space description, the key indicators of the optimization process can be objectively reflected, so that the decision-making of reinforcement learning can "perceive" the current optimization stage. For example, when the algorithm falls into a local optimum, the number of processes and the average process length may stagnate and the fitness improvement will slow down, which will be reflected in the state The changes are minimal. In the early stages of the search, fitness is allowed to fluctuate within a certain range to facilitate extensive exploration. This state representation allows high-level reinforcement learning agents to dynamically adjust their decision-making strategies based on the current state.

[0030] Furthermore, the specific process of step S3 is as follows:

[0031] First proceed The noise is sampled and then used The selection of low-level heuristic algorithms is performed, where s represents the current state, a represents the action to be evaluated, θ represents the parameters of the Q network, that is, the weight of the neural network, ∈ represents the trainable noise parameter, and low-level heuristic algorithms include genetic algorithm, whale group optimization algorithm, differential evolution algorithm, taboo search algorithm and simulated annealing algorithm. By selecting the action a with the largest Q value t , that is, select the corresponding low-level heuristic algorithm as the optimization strategy adopted in the current iteration.

[0032] By introducing trainable noise into the Noisy DQN (Deep Q-Network), reinforcement learning agents can automatically balance exploration and exploitation when making decisions. This noise injection increases the randomness of action selection, preventing premature entrapment in local optima caused by greedy strategies. Furthermore, the Deep Q-Network continuously adjusts its assessment of the value of each algorithm through training, allowing decisions to converge toward the optimal strategy as experience accumulates. This enables a dynamic, high-level algorithm selection mechanism that adaptively selects the most appropriate low-level heuristic algorithm for different optimization stages, improving global search efficiency and convergence speed.

[0033] Furthermore, step S4 specifically utilizes the low-level heuristic algorithm selected in step S3 to generate a new taboo sequence, and generates a new solution in combination with a two-layer directed acyclic graph. During the optimization search process, when a specific subgraph or feature is selected in a certain step, it is recorded in the taboo sequence. Within a certain iteration cycle, subsequent steps will avoid selecting the same element again. By introducing the taboo sequence mechanism, the encoding process of the low-level heuristic algorithm is converted into the generation of taboo sequences and the solution process of the topological sorting of the two-layer directed acyclic graph: whenever a new taboo sequence is generated, a new set of processing sequence plans is generated accordingly, including a processing feature sequence and corresponding process division and resource allocation plans.

[0034] This dynamic adjustment of taboo rules effectively avoids premature repetition of previously explored areas of the solution space during the search process, prompting the algorithm to more fully explore the entire feasible solution space. Compared to traditional fixed encoding methods, this taboo sequence encoding method can reduce repeated switching and idle time during the machining process, resulting in a more optimized process solution.

[0035] Furthermore, in step S5, if the improvement of the current solution is less than a preset improvement threshold, the switching of the low-level heuristic algorithm is triggered;

[0036] If the execution time of the current low-level heuristic algorithm reaches the preset time threshold, the switch of the low-level heuristic algorithm is triggered;

[0037] When the switching is triggered, the process returns to step S3 to reselect an action in the next iteration, thereby realizing the adaptive switching of the low-level heuristic algorithm.

[0038] Furthermore, the calculation formula for the improvement of the current solution is:

[0039]

[0040] in, Indicates the current solution Compared to iteration count n(p) Previous solution If the improvement is less than the preset improvement threshold, the low-level heuristic algorithm is switched, count n(p) An integer hyperparameter representing the number of iterations.

[0041] Furthermore, the specific process of step S6 is as follows:

[0042] Repeat the reinforcement learning decision-making and optimization process of steps S3 to S5 to form a continuous training round. When a single training round reaches the predetermined stopping condition, the training round is terminated and a final process plan is obtained. The fitness of the plan is compared with the global optimal fitness record: if the current plan is better, the global optimal solution is updated and added to the initial solution set;

[0043] Afterwards, the next training round is started as needed: another initial solution is selected from the initial solution set as a new starting point, and steps S2 to S5 are repeated for optimization, and continuous training is performed to obtain a better solution.

[0044] Compared with the prior art, the present invention has the following advantages:

[0045] The present invention first establishes a two-layer directed acyclic graph model to represent the various constraints of the process planning process and generates a set of feasible process solutions that meet all constraints. It then uses a Noisy DQN-based sequence selection hyper-heuristic algorithm to search and optimize the set of feasible solutions, dynamically selecting the optimal heuristic strategy to simultaneously determine the optimal sequence of processing features, process division scheme, and resource allocation scheme. Finally, it outputs the optimal process solution that meets all constraints and minimizes processing cost and time. This allows for the simultaneous optimization of multiple process decisions, avoiding the suboptimal results that would result from performing each optimization process separately, significantly reducing processing time and resource consumption, and improving production efficiency and the adaptability of the solution.

[0046] The present invention will process number Average process length Reward value fluctuations And the fitness f t The coupling is a four-element state vector, which can depict the scale, structural complexity and quality evolution trend of the process plan in real time. This refined state representation not only avoids the search blind spots caused by traditional single-indicator drive, but also enables the reinforcement learning agent to accurately perceive "when global exploration is needed and when local deep digging is appropriate", thereby quickly expanding the solution space in the early stage and accelerating the convergence to the optimal neighborhood in the later stage, which improves the overall search efficiency and solution quality stability. The present invention integrates the three sub-problems of processing feature sorting, process grouping and resource allocation for optimization, avoiding the local optimality and resource waste that may be caused by the independent optimization of each sub-process, and improving the overall optimality of the process planning results.

[0047] This method utilizes a hyper-heuristic algorithm driven by a Noisy DQN for high-level decision-making. It dynamically switches between different heuristic algorithm strategies as needed during the optimization process, balancing the conflict between global search and local optimization, accelerating convergence, and improving the algorithm's adaptability to diverse problem instances. By introducing a noisy network at the high level to achieve a balance between exploration and exploitation, and implementing an algorithm switching mechanism at the low level to prevent stagnation, the method effectively prevents local optima and premature convergence, enhancing the algorithm's robustness and generalization capabilities.

[0048] The present invention uses a dual-threshold trigger mechanism: comparing the fitness improvement ratio (i.e., improvement amplitude) within the window and the continuous execution time of the monitored single algorithm, it can evaluate the marginal contribution of the current low-level heuristic algorithm in real time. Once it is detected that the improvement is saturated or has run for too long, it will immediately switch to the heuristic algorithm with higher potential benefits in the algorithm pool. This strategy not only avoids the waste of resources caused by long-term stagnation of a single algorithm, but also prevents search oscillations caused by too frequent switching, ensuring the optimization rhythm of "continuous gain, timely jump" throughout the process, and effectively improving the overall optimization efficiency and the global optimal hit rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 Schematic diagram of the method flow of the present invention;

[0050] Figure 2 This is a diagram of the Noisy DQN training method for reinforcement learning;

[0051] Figure 3 Schematic diagram comparing the optimization results of the method proposed in the present invention and the classic heuristic optimization algorithm in the embodiment. DETAILED DESCRIPTION

[0052] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0053] Example

[0054] like Figure 1 As shown, a resource allocation and process planning optimization method for complex constraint conditions includes the following steps:

[0055] S1. Use a two-layer directed acyclic graph to represent various constraints of the process planning process and generate an initial feasible solution set that satisfies all constraints.

[0056] S2, initialize the state space, including the state variables of process number, average process length, reward value fluctuation and fitness;

[0057] S3, collects noise and couples it to the Q network to select actions and output low-level heuristic algorithm options;

[0058] S4, using the selected low-level heuristic algorithm to generate a new taboo sequence and optimize the current feasible solution;

[0059] S5. Switch to the low-level heuristic algorithm based on the improvement and execution time, and then return to step S3;

[0060] S6. After multiple trainings, the optimal processing feature sequence and process division combination are output.

[0061] Applying the above solution to practice, the main contents are:

[0062] 1. Use a two-layer directed acyclic graph to construct an initial feasible solution set. Specifically, the execution process is as follows:

[0063] S11. Collect ① processing feature table ②Machine tool / tool ​​collection ③ Constraint set - including pre-order constraints Joint Constraints Machine tool priority constraints Processing time constraints

[0064] S12. By joint constraint First, the features that need to be "bundled and processed" are merged into directed acyclic subgraphs; if two directed acyclic subgraphs have an intersection (the same feature appears in different subgraphs), they are merged;

[0065] S13. Treat each directed acyclic subgraph as a node; add the machine tool nodes with machine tool priorities to the same graph, and add directed edges between the nodes using the pre-order constraint and the machine tool priority constraint to obtain a two-layer directed acyclic graph;

[0066] S14. If a loop appears in the two-layer directed acyclic graph, delete an edge to break the loop based on processing experience to ensure that the topological sorting is feasible;

[0067] S15. Eliminate the subgraphs and features to be processed one by one according to the taboo sequence. After constructing the double-layer directed acyclic topological graph model, use the forward topological sorting algorithm to generate an initial feasible solution on the graph. All constraints are followed during the topological sorting process: a node (processing feature or feature subset) can only be arranged in the processing sequence when all its predecessor dependencies have been arranged. In specific implementation, sorting can be started from the node with an in-degree of 0 (no predecessor dependency), adding it to the current process sequence, then removing the node and updating the in-degree of the relevant nodes. This process is repeated until all feature nodes are processed, and a complete processing feature sequence is obtained. If the accumulated processing time of a certain process during the sorting process exceeds the preset upper limit, a new process is divided at the excess point.

[0068] Second, initialize the state space with the number of processes, average process length, reward value fluctuation and fitness as state variables. The state space is expressed as:

[0069]

[0070] in, Indicates the number of current processes, Indicates the average length of the current process; Indicates the change in reward value compared to the previous step; f t It represents the fitness under the current solution, and its calculation formula is:

[0071] f t =1000000 / (F-200000)

[0072] Among them, 1,000,000 and 200,000 are experimental hyperparameters formulated based on machine tool and tool costs. F represents the processing cost under the current processing feature sequence division, which can be specifically expressed as:

[0073]

[0074] wherein, denotes the number of the mth type of machine tool in the kth process, denotes the usage price of the mth type of machine tool, is a binary variable, which takes 1 if and only if the mth type of machine tool is used in the kth process, otherwise 0; the unit price of the selected tool for the ith machining feature; x ij is a binary decision variable, which takes 1 if and only if the ith feature is arranged in the jth position, otherwise 0. h、 denote the total number of processes, machine tool types, machining features and machining positions, respectively.

[0075] By introducing the above state space description, the key indicators in the optimization process can be objectively reflected, so that the decision of reinforcement learning can "perceive" the current optimization stage. When the algorithm falls into a local optimum, the number of processes and the average process length may stagnate and the fitness improvement tends to be slow, which will be reflected as the state changes very little. In the early stage of search, in order to strive for extensive exploration, the fitness is allowed to fluctuate within a certain range. Through such state representation, the high-level reinforcement learning agent can dynamically adjust the decision strategy based on the current state.

[0076] III. Noisy DQN, when selecting actions, first performs noise sampling, and then uses to select low-level heuristic algorithms, including but not limited to genetic algorithm, whale optimization algorithm, differential evolution algorithm, tabu search algorithm and simulated annealing algorithm. Among them, s represents the current state, a represents the action to be evaluated, θ represents the parameters of the Q network (the weights of the neural network), and ∈ represents the trainable noise parameter. The action a t with the maximum Q value, i.e. the corresponding low-level heuristic algorithm, is selected as the optimization strategy for the current iteration. Due to the existence of noise, the agent can try different algorithms more in the early stage of training (increase exploration), and as the training progresses, if some algorithm strategies repeatedly prove to be better, the Q network will give them higher Q values, so that the agent is more inclined to choose these algorithms in the later stage (realize exploitation). Through Noisy DQN, the high-level algorithm selection strategy can automatically adjust according to the change of state, and select the most suitable bottom algorithm for different optimization stages. It should be noted that in actual implementation, the parameters of the Q network will be updated according to the reward signal obtained after each iteration, in order to gradually approach the true action value function. The continuous training of this reinforcement learning makes the algorithm selection strategy continuously optimized, and finally converges to a decision strategy with excellent performance.

[0077] By introducing trainable noise through Noisy DQN, the reinforcement learning agent can automatically balance exploration and exploitation when making decisions: on the one hand, the injection of noise increases the randomness of action selection, avoiding premature falling into local optimum caused by greedy strategy; on the other hand, the deep Q network constantly adjusts the value evaluation of each algorithm through training, so that the decision tends to be more optimal strategy with experience accumulation. In this way, a dynamic high-level algorithm selection mechanism is realized, which can adaptively select the most suitable low-level heuristic algorithm for different optimization stages, improving the global search efficiency and convergence speed.

[0078] IV. Using the selected low-level heuristic algorithm to generate a new taboo sequence and combining it with the double-layer directed acyclic graph to generate a new solution. The so-called "taboo sequence" is a code used to record the currently selected machining feature subgraph or machining feature sequence in the optimization search process. When a particular subgraph or feature is selected at a certain step, it is recorded in the taboo sequence, and the subsequent steps will avoid selecting the same element within a certain iteration period, thereby reducing invalid repeated search. By introducing the taboo sequence mechanism, the coding process of the low-level heuristic algorithm is essentially transformed into the generation of a taboo sequence and the solution of a double-layer directed acyclic topological sorting process: each time a new taboo sequence is generated, a new set of machining order schemes is correspondingly produced. This dynamic adjustment of taboo rules can effectively avoid repeating the exploration of the solution space region in the search process, and promote the algorithm to more fully explore the entire feasible solution space. This taboo sequence coding method can reduce the cost of repeated switching and idling in the machining process compared to the traditional fixed coding method, and the generated process plan is more optimal.

[0079] In actual implementation, if the genetic algorithm is selected, the current solution population is selected, crossed and mutated to generate a new scheme; if the taboo search algorithm is selected, a better scheme is searched in the neighborhood of the current process plan (such as exchanging the order of adjacent features in the sequence, adjusting the process ownership of a certain feature, etc.), and this change (action) is recorded in the taboo list to avoid undoing the change in the next step. Regardless of which algorithm is used, a new process plan (i.e. a new machining feature sequence and the corresponding process division and resource allocation scheme) will be generated after a certain calculation. Denote this new solution as new_solution. Then, compare the fitness of new_solution and the previous current_solution: if the fitness of the new solution is better than that of the current solution, i.e. new f current, then accept new_solution as an update to the current solution; otherwise, the algorithm can choose whether to accept the inferior solution (for example, the simulated annealing algorithm has a certain probability of accepting an inferior solution to escape the local optimum) or directly retain the original solution unchanged according to the specific algorithm strategy. At the same time, the feature sequence corresponding to new_solution is added to the taboo list as a new taboo sequence to avoid the recurrence of the same feature processing sequence scheme in the future search process. The taboo list is set to a certain length. When the list capacity exceeds the limit, the sequences added earlier will be removed to ensure that the algorithm still has the opportunity to visit the previously explored area after a long run. Through this mechanism, it can effectively prevent the cycle in the search process and expand the exploration depth of the solution space.

[0080] 5. When deciding whether to switch to a low-level heuristic algorithm, the decision is made based on the improvement of the current solution and the execution time of the algorithm. The improvement calculation formula is:

[0081]

[0082] in, Indicates the current solution Compared to iteration count n(p) Previous solution If the improvement is less than a certain threshold, the low-level heuristic algorithm is switched, count n(p) An integer hyperparameter representing the number of iterations. In addition, when the execution time of a low-level heuristic algorithm reaches a certain threshold, the heuristic algorithm is also switched. The above two switching conditions can be used separately or in combination. In this embodiment, the two conditions are in an "or" relationship. As long as one of them is met, the next step of changing the algorithm will be triggered. When the switch is triggered, the high level will re-select the action from the algorithm pool in the next iteration (which may be different from the previous selection), thereby realizing adaptive switching of the algorithm. It should be noted that the switching mechanism does not affect the continuous training and decision optimization of the reinforcement learning agent. On the contrary, the attempt of different algorithms will provide the agent with more diverse experience data, enrich the training samples of the Q network, and enable it to have a more comprehensive study of the effects of each algorithm in different situations. This provides a guarantee for the maturity of subsequent high-level decision-making strategies.

[0083] 6. If Figure 2As shown, the reinforcement learning decision and optimization process of steps three to five above are repeated to form a continuous training round. When a single training round reaches a predetermined stopping condition (such as a set maximum number of iterations, fitness no longer improving, etc.), the round is terminated and a final process plan is obtained. Then, the fitness of the plan is compared with the global optimal fitness record: if the current plan is better, the global optimal solution is updated. After that, the next training round can be started as needed: another initial plan (preferably including the just-updated excellent plan or other not-yet-fully-optimized plans) is selected from the initial solution set as the new starting point, and steps two to five are repeated for optimization, constantly trying to find a better plan. In this process, after each round of training is completed, the excellent solution (if any) produced by the current round is added to the initial solution pool, so that the initial solution pool gradually becomes richer. This approach can integrate the excellent partial solutions found in previous training into subsequent search, playing a role in drawing inferences from one thing to another and enhancing diversity. After several rounds of training, the strategy of the reinforcement learning agent gradually matures, and the probability of finding a global optimal or near-optimal solution is greatly increased. Finally, the process planning scheme with the highest fitness is selected from all training rounds to be the final output result, which contains the best execution order of the machining features, the allocation of each feature to a process, and the selection of machine tools and tools used in each process, and can meet all the constraints of the problem. To verify the effectiveness of the present scheme, the method proposed in the present application (NoisyDQN) is compared with other classical heuristic optimization algorithms (including genetic algorithm, differential evolution algorithm, tabu search algorithm, simulated annealing algorithm, random extraction algorithm, whale optimization algorithm) in a comparative experiment, and the results are shown in Figure 3 As shown, the method proposed in the present application can achieve the optimal fitness.

Claims

1. A method for optimizing resource allocation and process planning under complex constraints, characterized in that: The following steps are involved: S1. Use a two-layer directed acyclic graph to represent various constraints of the process planning process and generate an initial feasible solution set that satisfies all constraints. S2, initialize the state space, including the state variables of process number, average process length, reward value fluctuation and fitness; S3, collects noise and couples it to the Q network to select actions and output low-level heuristic algorithm options; S4, using the selected low-level heuristic algorithm to generate a new taboo sequence and optimize the current feasible solution; S5. Switch to the low-level heuristic algorithm based on the improvement and execution time, and then return to step S3; S6. After multiple trainings, the optimal processing feature sequence and process division combination are output.

2. The method for optimizing resource allocation and process planning under complex constraints according to claim 1, characterized in that: The various constraints in the process planning process in step S1 include priority constraints of machining features, joint machining constraints of machining features, sequence constraints for machine tool use, and machining time constraints.

3. The method for optimizing resource allocation and process planning under complex constraints according to claim 2, characterized in that: The specific process of step S1 is: S11. Collect processing feature table Machine tool / tool ​​collection And determine the constraint set, including the pre-order constraint Joint Constraints Machine tool priority constraints Processing time constraints S12. By joint constraint Merge the features that need to be "bundled" into directed acyclic subgraphs; if two directed acyclic subgraphs have an intersection, that is, the same feature appears in different subgraphs, they are merged; S13, treat each directed acyclic subgraph as a node, and add the machine tool node with the machine tool priority to the same graph, using the pre-order constraint and machine priority constraints Add directed edges between nodes to obtain a two-layer directed acyclic graph; S14. If a loop appears in the two-layer directed acyclic graph, an edge is deleted based on processing experience to break the loop, ensuring that the topological sorting is feasible, and obtaining a two-layer directed acyclic topological graph; S15. Eliminate the subgraphs and features to be processed one by one according to the taboo sequence to generate and construct an initial feasible solution set, wherein the taboo sequence refers to the code used to record the currently selected processing feature subgraph or processing feature sequence during the optimization search process.

4. The method for optimizing resource allocation and process planning under complex constraints according to claim 3, characterized in that: The specific process of step S15 is: using a forward topological sorting algorithm to generate an initial feasible solution on a two-layer directed acyclic topological graph, wherein all constraints are followed during the topological sorting process: a node, i.e., a processing feature or a feature subset, can be arranged in the processing sequence only when all its preceding dependencies have been arranged. In specific implementation, sorting is started from the node with an in-degree of 0, i.e., no preceding dependencies, and it is added to the current process sequence. Then, the node is removed and the in-degree of the related nodes is updated. This process is repeated until all feature nodes are processed, i.e., a complete processing feature sequence is obtained. If the accumulated processing time of a certain process during the sorting process exceeds the preset upper limit, a new process is divided at the exceeding point.

5. The method for optimizing resource allocation and process planning under complex constraints according to claim 1, characterized in that: The step S2 of initializing the state space is specifically as follows: in, Indicates the number of current processes, Indicates the average length of the current process; Indicates the change in reward value compared to the previous step; f t It represents the fitness under the current solution, and its calculation formula is: f t =1000000 / (F-200000) Among them, 1000000 and 200000 are experimental experience hyperparameters formulated based on the cost of machine tools and cutting tools. F represents the processing cost under the current processing feature sequence division, which can be specifically expressed as: in, represents the number of the m-th type of machine tools in the k-th process, represents the price of using the mth type of machine tool, is a binary variable, which takes the value 1 if and only if the type m machine tool is used for the k-th process, otherwise it takes the value 0; The unit price of the tool selected for the i-th processing feature; x ij is a binary decision variable, which takes the value 1 if and only if the i-th feature is placed in the j-th position, otherwise it takes the value 0; h. They represent the total number of operations, the total number of machine tool types, the total number of machining features, and the total number of machining positions, respectively.

6. The method for optimizing resource allocation and process planning under complex constraints according to claim 3, characterized in that: The specific process of step S3 is: First proceed The noise is sampled and then used The selection of low-level heuristic algorithms is performed, where s represents the current state, a represents the action to be evaluated, θ represents the parameters of the Q network, that is, the weight of the neural network, ∈ represents the trainable noise parameter, and low-level heuristic algorithms include genetic algorithm, whale group optimization algorithm, differential evolution algorithm, taboo search algorithm and simulated annealing algorithm. By selecting the action a with the largest Q value t , that is, select the corresponding low-level heuristic algorithm as the optimization strategy adopted in the current iteration.

7. The method for optimizing resource allocation and process planning under complex constraints according to claim 6, characterized in that: The step S4 specifically utilizes the low-level heuristic algorithm selected in step S3 to generate a new taboo sequence, and generates a new solution in combination with a two-layer directed acyclic graph. During the optimization search process, when a specific subgraph or feature is selected in a certain step, it is recorded in the taboo sequence. Within a certain iteration cycle, subsequent steps will avoid selecting the same element again. By introducing the taboo sequence mechanism, the encoding process of the low-level heuristic algorithm is converted into the generation of taboo sequences and the solution process of the topological sorting of the two-layer directed acyclic graph: whenever a new taboo sequence is generated, a new set of processing sequence plans is generated accordingly, including a processing feature sequence and corresponding process division and resource allocation plans.

8. The method for optimizing resource allocation and process planning under complex constraints according to claim 7, characterized in that: In step S5, if the improvement of the current solution is less than a preset improvement threshold, switching to a low-level heuristic algorithm is triggered; If the execution time of the current low-level heuristic algorithm reaches the preset time threshold, the switch of the low-level heuristic algorithm is triggered; When the switching is triggered, the process returns to step S3 to reselect an action in the next iteration, thereby realizing the adaptive switching of the low-level heuristic algorithm.

9. The method for optimizing resource allocation and process planning under complex constraints according to claim 8, characterized in that: The calculation formula for the improvement of the current solution is: in, Indicates the current solution Compared to iteration count n(p) Previous solution If the improvement is less than the preset improvement threshold, the low-level heuristic algorithm is switched, count n(p) An integer hyperparameter representing the number of iterations.

10. The method for optimizing resource allocation and process planning under complex constraints according to claim 1, characterized in that: The specific process of step S6 is as follows: Repeat the reinforcement learning decision-making and optimization process of steps S3 to S5 to form a continuous training round. When a single training round reaches the predetermined stopping condition, the training round is terminated and a final process plan is obtained. The fitness of the plan is compared with the global optimal fitness record: if the current plan is better, the global optimal solution is updated and added to the initial solution set; Afterwards, the next training round is started as needed: another initial solution is selected from the initial solution set as a new starting point, and steps S2 to S5 are repeated for optimization, and continuous training is performed to obtain a better solution.

Images