Construction method of heterogeneous workpiece system production scheduling scheme, production scheduling method, computer system and computer readable storage medium

Abstract

The invention belongs to the field of workpiece processing production scheduling, and particularly relates to a construction method of a heterogeneous workpiece system production scheduling scheme, a production scheduling method, a computer system and a computer readable storage medium. The construction method comprises the following steps: 1) selecting a workpiece needing to be processed currently, determining a current processing procedure of the workpiece according to a serial logic sequence of the processing procedures of the workpiece, and determining a decision moment of the procedure according to a timeline of the processing procedure; (2) inputting a state space which corresponds to the decision-making moment and contains the current processing procedure, the occupation state of each processing device and the occupation state of each processing resource into a decision-making model corresponding to the workpiece to obtain a decision-making result used for determining the processing device used by the processing procedure, updating a timeline of the processing procedure and a locking time period of processing equipment and processing resources; and repeating the steps 1)-2) until all the workpieces are processed, so as to complete the construction of the production scheduling scheme.

Description

Technical Field

[0001] The present invention belongs to the field of workpiece processing production scheduling, and specifically relates to a method for constructing a production scheduling plan for a heterogeneous workpiece system, a production scheduling method, a computer system and a computer-readable storage medium. Background Art

[0002] Production scheduling is essentially a scheduling problem under complex spatiotemporal constraints, which essentially requires a feasible and highly complex combinatorial optimization solution. Researchers widely use optimization algorithms and intelligent optimization algorithms to solve them. However, if you want to make fast and accurate decisions in highly dynamic and real-time scenarios, traditional offline static scheduling solutions lack environmental adaptability and real-time feedback mechanisms, and are difficult to meet the needs of rapid response. They are often limited by real-time requirements due to computational complexity.

[0003] In the prior art, a Markov model is often constructed according to time steps and a state space corresponding to the time step is set on this basis, and a decision model of the workpiece is trained accordingly; when constructing a production scheduling plan, each workpiece in each time step of its corresponding timeline is judged whether the current processing is completed. If completed, the state space corresponding to the current time step is input into the decision model of the workpiece to determine the processing equipment that will perform the workpiece processing in the next step, thereby realizing real-time scheduling; however, the various time steps of this method are completely serial, that is, it is assumed that a processing equipment can only process a single process of a workpiece in each period of time, and it is impossible to consider the situation where a processing equipment can process more than two processes of a workpiece at the same time, so that the constructed production scheduling plan cannot fully utilize the processing capacity of the processing equipment, resulting in a longer total duration of the constructed production scheduling plan and low overall efficiency. Summary of the invention

[0004] The purpose of the present invention is to provide a method for constructing a production scheduling plan for a heterogeneous workpiece system, a production scheduling method, a computer system and a computer-readable storage medium, so as to solve the problem that the method for constructing a production scheduling plan in a completely serial manner between each time step in the prior art cannot take into account the situation where a processing equipment can simultaneously process more than two processes of a workpiece, so that the constructed production scheduling plan cannot fully utilize the processing capacity of the processing equipment, resulting in a longer total time for the constructed production scheduling plan and low overall efficiency.

[0005] In order to achieve the above object, the present invention provides a method for constructing a production scheduling scheme for a heterogeneous workpiece system, comprising:

[0006] 1) Select the workpiece to be processed, determine the current processing procedure of the workpiece according to the serial logical order of the processing procedures of the workpiece, and determine the decision time of the procedure according to the timeline of the processing procedure;

[0007] 2) Input the state space corresponding to the decision moment, including the current processing procedure, the occupancy status of each processing equipment, and the occupancy status of each processing resource, into the decision model obtained by training corresponding to the heterogeneous workpiece type to which the workpiece belongs, and obtain the decision result for determining the processing equipment used for the processing procedure, and update the timeline of the processing procedure and the locking time period of the processing equipment and processing resources; repeat 1)-2) until all workpieces are processed to complete the construction of the production scheduling plan;

[0008] The method for determining the state space corresponding to the decision moment includes: using the decision moment, the timeline of the processing process, and the locked time period of the processing equipment and processing resources to obtain the occupation state of each processing equipment and the occupation state of each processing resource in the state space corresponding to the decision moment of the process;

[0009] The method for determining the serial logical order of the workpiece processing procedures includes: setting the logical order of the predecessor processing procedure before the successor processing procedure; setting the serial logical order between the processing procedures that do not have a predecessor-successor relationship according to a set sorting method;

[0010] The method of setting the timeline of the process includes: setting corresponding parallel timelines for each process set that is parallel in time sequence of a single workpiece; setting corresponding serial timelines for each process set that is serial in time sequence of the remaining workpiece; each process set contains at least one processing process and different processing processes in the set are serial in time sequence;

[0011] The method for updating the timeline of the processing procedure includes: if the decision result of executing the procedure does not require changing the processing equipment where the workpiece is located, then directly superimpose the execution time of the procedure onto its corresponding timeline; otherwise, take the end time of the timeline with the longest total time among the timelines of all the procedures of the workpiece currently to be processed as the basis, unify the end time of the timelines of all the procedures of the workpiece, and then superimpose the execution time of the procedure onto its corresponding timeline.

[0012] Further, the workpiece to be processed is selected by selecting the workpiece with the highest current priority from the workpieces to be processed;

[0013] The method for determining the current priority of a workpiece includes: for workpieces whose processing procedures have not been fully completed, the total time required to execute the remaining processing procedures of the workpiece itself is used, and the workpiece with a longer total time has a higher priority; if the total time is equal, the priority of these workpieces with equal total time is set according to the preset workpiece number order.

[0014] Furthermore, each time a decision result for determining the processing equipment used for the processing step is obtained, the current priority of the workpiece is updated according to the situation after the decision result is executed.

[0015] Furthermore, the setting order is determined according to the time consumption of the processing procedure and the preset process number; the shorter the processing time consumption, the earlier the setting order is; if the processing procedures consume the same time consumption, the order of these processing procedures with the same time consumption is set according to the preset process number order.

[0016] Furthermore, the execution time of the process is determined by:

[0017] a) taking the intersection of the idle time period of the processing equipment used in the process determined according to the decision result and the non-locked time period of the processing resources required to execute the process; determining the processing start time corresponding to the decision time of the process; the idle time period includes both the non-locked time period of the processing equipment and the locked time period of the processing equipment that meets the set condition; the set condition is that the workpiece processed corresponding to the locked time period is the same workpiece as the workpiece corresponding to the process;

[0018] b) Determine the processing end time of the process in combination with the processing start time of the process and the time required for the execution of the process itself; check whether the processing time period from the processing start time to the processing end time of the process is included in the intersection, and if so, determine the execution time of the process according to the processing time period; otherwise, postpone the processing start time of the process, and repeat b) until the execution time of the process is determined.

[0019] Furthermore, the state space also includes the execution state of each processing step of the workpiece currently to be processed, the distance of the workpiece currently to be processed relative to each processing equipment, and the processing equipment where the workpiece currently to be processed is located;

[0020] During the training process of the decision model, the reward function corresponding to the output decision result is used to update the decision model parameters; the reward function includes a first function item corresponding to the completion status of the processing procedure corresponding to the current decision result, a second function item corresponding to the waiting time of the processing procedure corresponding to the current decision result, a third function item corresponding to the transfer time of the processing procedure corresponding to the current decision result, and a fourth function item corresponding to the load balancing status of the processing procedure corresponding to the current decision result;

[0021] The value of the first function item when the processing procedure corresponding to the decision result is successfully completed is greater than the value of the first function item when the processing procedure corresponding to the decision result is not successfully completed; the shorter the waiting time of the processing procedure corresponding to the decision result, the greater the value of the second function item; the shorter the transfer time of the processing procedure corresponding to the decision result, the greater the value of the third function item; the smaller the difference between the load rate of each processing resource corresponding to the processing procedure corresponding to the decision result and the load rate of all processing resources, the greater the value of the fourth function item.

[0022] Furthermore, each decision model is trained using the DQN algorithm.

[0023] The above technical solution of the present invention provides a new method for constructing a production scheduling solution for a heterogeneous workpiece system, and its beneficial effects include:

[0024] By determining whether there is a predecessor-successor relationship and the set sorting method, the various processing procedures of a single workpiece that may be in parallel in time sequence are uniformly converted into an overall serial logic that conforms to the actual processing sequence. In this way, the sequence of moments that need to be executed serially when building a production scheduling plan can be modified to a logical sequence that needs to be executed serially. In combination with each decision, the state space corresponding to the decision moment of the current processing procedure of the current workpiece is input into the setting of the decision model corresponding to the workpiece, so that the time step is no longer used as the standard for dividing the front and back state space of the Markov model, but the execution of the decision step corresponding to each procedure is used to divide the front and back state space; at the same time, a parallel timeline is set for a set of processes that are parallel in time sequence. When the execution of the workpiece process When the processing equipment does not change, the process execution time is directly superimposed on the timeline corresponding to the process. Therefore, for a workpiece whose processing equipment does not change and whose processing processes are adjacent in serial logical order but parallel in timing, their respective times are only superimposed on their respective corresponding parallel timelines instead of all superimposed on the serial timeline, thereby achieving the effect of simultaneously processing different processes of the workpiece on the same processing equipment. In addition, on the basis of dividing the before and after state space by the execution of the decision step corresponding to each process, since the processes of different types of workpieces in heterogeneous workpieces are different, the decision models corresponding to each different type of workpiece need to be obtained separately through their own training processes to ensure that the decision model is adapted to the process conditions of the workpiece itself.

[0025] The present invention also provides a production scheduling method for a heterogeneous workpiece system. During the construction process of the production scheduling plan for the heterogeneous workpiece system, the actual production scheduling of the heterogeneous workpiece system is controlled by using the information obtained from the construction process; the construction of the production scheduling plan for the heterogeneous workpiece system is implemented according to the above-mentioned method for constructing the production scheduling plan for the heterogeneous workpiece system.

[0026] The above-mentioned method for production scheduling of a heterogeneous workpiece system of the present invention can achieve the same beneficial effects as the above-mentioned method for constructing a production scheduling solution for a heterogeneous workpiece system.

[0027] The present invention also provides a computer system, wherein the processor is used to execute executable program instructions, and the executable program instructions are used to be executed to implement the above-mentioned heterogeneous workpiece system production scheduling method.

[0028] The computer system of the present invention can achieve the same beneficial effects as the above-mentioned heterogeneous workpiece system production scheduling method.

[0029] The present invention also provides a computer-readable storage medium, in which computer program instructions are stored. The computer program instructions are used to implement the above-mentioned heterogeneous workpiece system production scheduling method when executed.

[0030] The computer-readable storage medium of the present invention can achieve the same beneficial effects as the above-mentioned heterogeneous workpiece system production scheduling method. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a principle block diagram of a method for constructing a production scheduling solution for a heterogeneous workpiece system in an embodiment of the method for constructing a production scheduling solution for a heterogeneous workpiece system of the present invention;

[0032] Figure 2 A schematic diagram of the training principle of training a decision model by using a DQN algorithm in an embodiment of a method for constructing a production scheduling solution for a heterogeneous workpiece system of the present invention;

[0033] Figure 3 A Gantt chart showing scheduling decisions made by two heterogeneous workpieces in the method for constructing a production scheduling solution for a heterogeneous workpiece system according to the construction method in this embodiment under the guidance of a decision model according to the production scheduling solution for a heterogeneous workpiece system constructed by the construction method in this embodiment;

[0034] Figure 4 A schematic diagram of the time sequence relationship of the processes required to be executed for the workpiece type A in the embodiment of the method for constructing the production scheduling plan for the heterogeneous workpiece system of the present invention;

[0035] Figure 5 A schematic diagram of the time sequence relationship of the processes required to be executed for the workpiece type B in the embodiment of the method for constructing the production scheduling plan for the heterogeneous workpiece system of the present invention;

[0036] Figure 6 This is an example diagram of the total processing time of the heterogeneous workpiece system production scheduling solution constructed by the construction method of this embodiment and the classical combinatorial optimization algorithm genetic algorithm under different numbers of workpieces in the construction method embodiment of the heterogeneous workpiece system production scheduling solution of the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0038] Embodiment of a method for constructing a production scheduling scheme for a heterogeneous workpiece system

[0039] This embodiment provides a technical solution for a method for constructing a production scheduling plan for a heterogeneous workpiece system. The method uses the decision steps of the process rather than the time steps to realize the construction of the production scheduling plan, and sets a parallel timeline to represent the processes that can be logically executed in parallel, thereby fully considering the situation where the same machine executes more than two processing processes in parallel.

[0040] The scheme refers to Figure 1 ,include:

[0041] 1) Select the workpiece to be processed, determine the current processing procedure of the workpiece according to the serial logical order of the processing procedures of the workpiece, and determine the decision time of the procedure according to the timeline of the processing procedure;

[0042] 2) Input the state space corresponding to the decision moment, including the current processing procedure, the occupancy status of each processing equipment, and the occupancy status of each processing resource, into the decision model obtained by training corresponding to the heterogeneous workpiece type to which the workpiece belongs, and obtain the decision result for determining the processing equipment used for the processing procedure, and update the timeline of the processing procedure and the locking time period of the processing equipment and processing resources; repeat 1)-2) until all workpieces are processed to complete the construction of the production scheduling plan;

[0043] Among them, the method for determining the state space corresponding to the decision moment includes: using the decision moment, the timeline of the processing procedure, and the locked time period of the processing equipment and processing resources to obtain the occupancy status of each processing equipment and the occupancy status of each processing resource in the state space corresponding to the decision moment of the procedure; updating the locked time period of the processing equipment and processing resources can directly add the time period in which the procedure occupies the processing equipment and processing resources for processing as new locked time periods of the processing equipment and processing resources, and the workpieces processed corresponding to these locked time periods are the workpieces corresponding to the procedure; in other embodiments, other locked time period update methods that can reflect the occupancy of the processing equipment and processing resources by the processing procedure can also be used.

[0044] The method of determining the serial logical order of the workpiece processing procedures includes: setting the logical order of the predecessor processing procedure before the successor processing procedure; setting the serial logical order between the processing procedures that do not have a predecessor-successor relationship according to the set order;

[0045] How to set the timeline of the process (i.e. Figure 1 The timeline allocation rules in the method include: setting corresponding parallel timelines for each set of process steps that are parallel in time sequence of a single workpiece; setting corresponding serial timelines for each set of process steps that are serial in time sequence of the remaining workpiece; each set of process steps contains at least one processing step and different processing steps in the set are serial in time sequence;

[0046] The method of updating the timeline of the processing procedure includes: if the decision result of executing the procedure does not require changing the processing equipment where the workpiece is located (for example, according to the serial logical order of the processing procedures of a certain workpiece, the previous processing procedure X of the current processing procedure Y of the workpiece is processed by processing equipment A, and the decision result corresponding to the execution of the current processing procedure Y is also processed by processing equipment A, then it means that the decision result of executing procedure Y does not require changing the processing equipment where the workpiece is located), the execution time of the procedure is directly added to its corresponding timeline; otherwise, the end time of the timeline with the longest total time among all the timelines of the workpiece currently to be processed is used as a reference, and after unifying the end time of the timelines of all the procedures of the workpiece, the execution time of the procedure is added to its corresponding timeline.

[0047] In summary, the method for constructing a production scheduling solution for a heterogeneous workpiece system in this embodiment converts the various processing procedures of a single workpiece that may have a temporally parallel relationship into an overall serial logic that conforms to the actual processing sequence by determining whether there is a predecessor-successor relationship and the set sorting method. Combined with the setting of the state space input decision model corresponding to the decision moment of the current processing procedure, the effect of dividing the previous and next state spaces by the execution of the decision steps corresponding to each procedure is achieved; and parallel timelines and corresponding update methods are set for a set of temporally parallel procedures to achieve the effect of simultaneously processing different procedures of the workpiece on the same processing equipment; in addition, the decision models corresponding to each different type of workpiece need to be obtained separately through their own training processes to ensure that the decision models are adapted to the process conditions of the heterogeneous workpiece itself.

[0048] Specifically, the setting order in the serial logic order is determined according to the time consumption of the processing procedure and the preset process number; the shorter the processing procedure is, the higher the setting order is; if the processing procedures are the same in time consumption, the order of these processing procedures with the same time consumption is set according to the preset process number sequence. Figure 1 The predecessor process priority rule and short process priority rule in set the serial logic order.

[0049] like Figure 1As shown in FIG. 1 , in a processing topological sequence (specifically, a temporal topological sequence) of a workpiece, there are two groups of temporally parallel processing procedures. These two groups of processing procedures can be divided into two temporally parallel process sets, one of which has two temporally serial processing procedures and the other has three temporally serial processing procedures. In the processing topological sequence, there are also two temporally serial processing procedures that have a temporally serial relationship with the two groups of temporally parallel processing procedures. These two processing procedures can be divided into two temporally serial process sets or into the same process set. The temporally serial processing procedures can be divided into the same process set to reduce the number of set timelines and correspondingly reduce the number of heterogeneous processes. The number of variables that need to be paid attention to in the process of constructing the production scheduling plan of the component system, so in this embodiment, these two processing steps are classified into the same process set; but in principle, since a serial timeline is set for the process set that is serial in time, during real-time scheduling and statistics of the total processing time, the execution order of the processes corresponding to each serial timeline in time is consistent with the sequential relationship between the serial timelines, and the duration of each serial timeline is also directly superimposed. Therefore, whether the processing steps that are serial in time are divided into two process sets that are serial in time and serial timelines are set separately, or they are classified into the same process set and share the same timeline, it does not affect the final presentation effect of the timeline in real-time scheduling and statistics of the total processing time. According to the timeline allocation rules, corresponding parallel timelines are set for the above two process sets that are parallel in time, that is, Figure 1 Timeline 1 and timeline 2 in the process set are used to set timeline 3 for other process sets; then, the processing steps in these three process sets are combined according to the serial logical order set by the predecessor process priority rule and the short process priority rule, and the result is Figure 1 Serial logic in equivalent serial.

[0050] In the serial logic of the equivalent serial, each processing procedure has a corresponding timeline. When calculating the total duration of parallel timelines 1 and 2, the duration of the longest timeline between timelines 1 and 2 is selected and added to the total duration. In this way, parallel processing operations are achieved in terms of total duration statistics. In the scheduling process, if the decision result of executing the second procedure in timeline 1 does not require changing the processing equipment where the workpiece is located, for example, according to the serial logic order of the processing procedures of the workpiece, the next procedure of the first procedure in timeline 1 happens to be the second procedure in timeline 1, the decision result of the second procedure in timeline 1 is consistent with the decision result of the first procedure in timeline 1. If the decision result of executing the second process in timeline 2 does not require changing the processing equipment where the workpiece is located, for example, according to the serial logical order of the processing procedures of the workpiece, the next process of the first process in timeline 2 happens to be the second process in timeline 2, and the decision result of the second process in timeline 2 corresponds to the same processing equipment as the decision result of the first process in timeline 2, then the execution time of the second process in timeline 2 is directly added to timeline 2; since timeline 1 and timeline 2 are in a parallel relationship with the same start time, this timeline update method can ensure the timeliness. The first process and the second process in timeline 1 are executed in parallel with the first process and the second process in timeline 2 in terms of timing; and on this basis, if the decision result of executing the third process in timeline 2 requires changing the processing equipment where the workpiece is located, for example, according to the serial logical order of the processing processes of the workpiece, the next process of the second process in timeline 1 happens to be the third process in timeline 1, and the decision result of the third process in timeline 2 corresponds to different processing equipment than the decision result of the second process in timeline 2, then the end time of the timeline with the longest total time between timeline 1 and timeline 2 (i.e., the timeline of all processes of the workpiece currently to be processed) needs to be used as the benchmark. , after unifying the end time of timeline 1 and timeline 2, the execution time of the third process in timeline 2 is added to timeline 2; for example, the first and second processes in timeline 1 take a total of 1 minute, and the first and second processes in timeline 2 take a total of 40 seconds. Then, the end time of timeline 1 is used as the benchmark to unify the end time of timeline 1 and timeline 2, which is equivalent to the time taken after the end of the first and second processes in timeline 2 is regarded as 1 minute, and then the execution time of the third process in timeline 2 is added to the unified timeline 2. The final total duration of timeline 2 is the duration corresponding to 1 minute plus the execution time of the third process.When performing actual scheduling, the preparation time before the processing equipment executes the processing procedure is not considered. The scheduling according to timeline 1 and timeline 2 means that a certain processing equipment processes the procedures in timeline 1 and timeline 2 at the same time. After 40 seconds, the first and second procedures in timeline 2 are completed. At this time, since the third procedure in timeline 2 needs to be transported to another processing equipment for processing, the end time of timeline 1 (where the first and second procedures have been completed) is used as the benchmark to unify the end time of timeline 1 and the end time of timeline 2 (where the first and second procedures have been completed). This is equivalent to controlling the third procedure in timeline 2 to wait for the first and second procedures in timeline 1 to be completed on the current processing equipment before starting to transport it to another processing equipment for processing, which is in line with the normal logic of actual scheduling. In addition, the method of determining the execution time of the procedure includes:

[0051] a) taking the intersection of the idle time period of the processing equipment used in the process determined according to the decision result and the non-locked time period of the processing resources required to execute the process; determining the processing start time corresponding to the decision time of the process; the idle time period includes both the non-locked time period of the processing equipment and the locked time period of the processing equipment that meets the set condition; the set condition is that the workpiece processed corresponding to the locked time period is the same workpiece as the workpiece corresponding to the process;

[0052] b) Determine the processing end time of the process in combination with the processing start time of the process and the time required for the execution of the process itself; check whether the processing time period from the processing start time to the processing end time of the process is included in the intersection, and if so, determine the execution time of the process according to the processing time period; otherwise, postpone the processing start time of the process, and repeat b) until the execution time of the process is determined.

[0053] Therefore, it can be seen that the execution time of the process needs to include the time required to wait for the processing equipment or processing resources to end their occupancy before the process is executed and the time required for the execution of the process itself; wherein, step a) first uses the idle time period of the processing equipment of the process and the non-locked time period of the processing resources to take the intersection, and determines the time period when the processing equipment and processing resources to be used by the process are idle; it should be noted that, considering that the same processing equipment has the ability to process different processes of the same workpiece, the idle time period of the processing equipment of the process includes not only the time period when the processing equipment does not perform any processing tasks (i.e., the non-locked time period of the processing equipment), but also includes the time period when the processing equipment is occupied by the workpiece corresponding to the current process (i.e., the locked time period in the locked time period of the processing equipment that meets the set conditions); for example, the current process number is 1, referred to as process 1; the workpiece number corresponding to process 1 is B, referred to as workpiece B; the processing equipment determined to process process 1 according to the decision result is m, then the idle time period of the processing equipment used for process 1 includes both the non-locked time period of processing equipment m and the locked time period when processing equipment m is occupied by workpiece B. In this way, it can be ensured that the determined idle time period is compatible with the situation where the same equipment processes different processes of the same workpiece at the same time.

[0054] Step b) means judging whether the processing start time is valid starting from the processing start time corresponding to the workpiece decision time (i.e., the earliest processing start time among the possible processing start times of the process). If the processing start time can ensure that the process is completed within a time period when the equipment and resources are idle, then the processing start time is valid. If the processing start time is invalid, then the processing start time will continue to be postponed, and whether the new processing start time is valid will continue to be judged until a valid processing start time is found. The valid processing start time can be used to more reasonably determine the execution time of the process.

[0055] Specifically, in combination with the above content, the pseudo code corresponding to the determination of the serial logic order and the updating method of the timeline of the processing procedure in this embodiment is shown in Table 1 below:

[0056] Table 1

[0057]

[0058] The state space of the above-mentioned input decision model is the state space of the Markov process model corresponding to the workpiece processing; the decision result output by the decision model for determining the processing equipment used in the processing procedure is the action space of the Markov process model; the Markov process model is usually expressed as a five-tuple <s, a, P (s'|s, a), γ, R (s'|s, a)>; wherein s is the state space, which is expressed as s (n) in the present invention, that is, the state space before the execution of the nth decision step (each decision step obtains a decision result), and the state space after the execution of the nth decision step is s (n + 1), that is, the state space before the execution of the n + 1th decision step; a is the action space, which is usually expressed as a n , that is, the decision result obtained in the nth decision step; P(s'|s,a) is the state transition probability, which describes the probability that the agent transfers from the current state space to another state space after performing a certain action. Since the change of the state space after completing a process in this embodiment is unique, the state transition probability is taken as a constant 1; R(s'|s,a) is the reward function, which can also be expressed as r(n) in this embodiment, which is the instant feedback provided by the workpiece after taking a certain action in each state. That is, the reward obtained after executing action a in state s and transferring to state s'. In this embodiment, the reward function corresponds to the output decision result, and will be used each time the parameters of the decision model are updated in the process of training the decision model; γ is a discount factor, which is a constant in the iterative process of training the decision model. It will be used each time the parameters of the decision model are updated in the process of training the decision model. It is usually set to a real number between 0 and 1 according to the specific situation, and is used to determine the importance of future rewards relative to immediate rewards; when γ is close to 1, more emphasis is placed on future rewards; when γ is close to 0, more attention is paid to immediate rewards; in this embodiment, the discount factor is compared through a controlled variable method, the same environment and artifact set are set, the average reward function curves obtained by training with different discount factors are compared, and the value that makes the average reward function perform best in the experimental results is taken to determine it. In other embodiments, it can also be set according to actual needs or experience.

[0059] Specifically, since the production scheduling of heterogeneous workpiece systems is usually carried out in time waves, and a wave needs to go through the process of processing-transportation-reprocessing, the Markov process model of this embodiment focuses on the optimal allocation of processing procedures and processing equipment and processing resources for all workpieces in a single wave. The agreed constraints of this Markov process model are as follows:

[0060] (1) Only one machine can be selected for processing workpieces at the same process stage;

[0061] (2) The sum of the total number of processes assigned to the machine during the entire process is the total number of processes required to be performed by all workpieces processed during the entire process;

[0062] (3) The completion time of a workpiece in the corresponding process is equal to the sum of its start time and the process processing time, and a process cannot be interrupted after it starts processing;

[0063] (4) Each machine can only serve one workpiece at a time.

[0064] In this embodiment, in addition to the above-mentioned current processing steps, the occupancy status of each processing equipment and the occupancy status of each processing resource, the state space also includes the execution status of each processing step of the workpiece currently to be processed, the distance of the workpiece currently to be processed relative to each processing equipment and the processing equipment where the workpiece currently to be processed is located; it is specifically expressed by the following formula:

[0065]

[0066] Where s(n) is the state space corresponding to the workpiece currently to be processed before the nth decision step is executed; n is the decision step number, is the occupancy status of processing equipment m in step n, Indicates that processing equipment m is not occupied in step n, otherwise is the occupancy status of processing resource k in step n, and its value has the same meaning as the processing equipment; is the execution status of the ith process of workpiece j in step n, Indicates that the status of the i-th process of workpiece j in step n is executed, otherwise d jm (n) is the distance of workpiece j relative to processing equipment m in step n; is the location of workpiece j in step n. For convenience, we use indicates that workpiece j is at processing equipment m; w j is the number of the processing step currently required for workpiece j, for example, w j =i means that the current workpiece j needs to perform the i-th processing operation.

[0067] Action Space a n The specific formula is as follows:

[0068] a n =m,m=0,1,...M

[0069] In the formula, m represents the serial number of the processing equipment, and M is the maximum serial number of the processing equipment; in particular, when m=0 (i.e., a n =0) indicates that the current processing step of the workpiece is continued in the original processing equipment (i.e. the processing equipment where the workpiece is currently located, i.e. the processing equipment of the previous processing step of the workpiece).

[0070] The decision model corresponding to each different type of workpiece in the heterogeneous workpieces is obtained through their own training process, that is, the decision model obtained by training corresponds to the type of heterogeneous workpiece to which the workpiece belongs; during the training process of the decision model, the reward function corresponding to the output decision result is used to update the decision model parameters; the reward function includes a first function item corresponding to the completion status of the processing procedure corresponding to the current decision result, a second function item corresponding to the waiting time of the processing procedure corresponding to the current decision result, a third function item corresponding to the transportation time of the processing procedure corresponding to the current decision result, and a fourth function item corresponding to the load balancing status of the processing procedure corresponding to the current decision result;

[0071] The value of the first function item when the processing procedure corresponding to the decision result is successfully completed is greater than the value of the first function item when the processing procedure corresponding to the decision result is not successfully completed; the shorter the waiting time of the processing procedure corresponding to the decision result, the greater the value of the second function item; the shorter the transfer time of the processing procedure corresponding to the decision result, the greater the value of the third function item; the smaller the difference between the load rate of each processing resource corresponding to the processing procedure corresponding to the decision result and the load rate of all processing resources, the greater the value of the fourth function item. Specifically, the determination method of the reward function r(n) is expressed by the following formula:

[0072]

[0073] In the formula, α, β, γ, is the hyperparameter that controls the weights of various rewards, and α·r t (n) is the first function term corresponding to the completion of the processing procedure corresponding to this decision result; β·r s (n) is the second function term corresponding to the waiting time of the processing procedure corresponding to this decision result; λ·r c (n) is the third function term corresponding to the time consumption of the processing procedure transfer corresponding to this decision result; That is, the fourth function item corresponding to the load balancing situation of the processing procedure corresponding to this decision result.

[0074] In this embodiment, the first function item corresponding to the completion status of each iteration of the processing step is manually judged according to the decision result outputted in each iteration. If the decision result of this decision can make the step successfully processed, r t (n) takes the value of a first set constant, which is greater than 0; if the process fails, then r t (n) takes a second set constant which is smaller than the first set constant, and the second set constant is usually 0. s (n), r c(n) The waiting time of the processing procedure and the transportation time of the processing procedure corresponding to the decision result of this decision can be determined respectively; r l (n) can be calculated according to the following formula:

[0075]

[0076] In the formula, K represents the number of all processing resources, x k is the load rate of the kth processing resource; μ is the mean load rate of all processing resources.

[0077] Specifically, each decision model is trained using the DQN algorithm. The training principle is as follows: Figure 2 Since the method of training the decision model of the action space of the output Markov process model based on the state space of the input Markov process model through the DQN algorithm belongs to the prior art and is usually implemented through the existing DQN algorithm encapsulation module, it will not be repeated here.

[0078] Considering that deadlock may occur when a workpiece waits for a certain processing equipment for a long time, causing it to be idle for a long time or even permanently, thereby significantly extending the completion time of the entire processing process, it is crucial to ensure the balance of the processing process and avoid potential deadlock in the optimization solution of workpiece processing; therefore, in this embodiment, in order to prevent the balance in the processing process and avoid potential guarantee deadlock, an anti-deadlock mechanism based on processing priority is designed, which is specifically reflected in the method of selecting the workpiece currently to be processed, that is, the method of selecting the workpiece currently to be processed is to select the workpiece with the highest current priority from the workpieces to be processed; the method of determining the current priority of the workpiece includes: for workpieces whose processing procedures have not been fully executed, according to the total time required for the execution of the remaining processing procedures of the workpiece itself, the longer the total time is, the higher the priority of the workpiece; if the total time is equal, the priority of these workpieces with equal total time is set according to the preset workpiece number order.

[0079] Therefore, the above-mentioned selection method of selecting the workpiece with the highest current priority from the workpieces to be processed and the method of determining the current priority of the workpiece are equivalent to dividing the processing priority of each workpiece in this batch according to the actual production plan and the processing progress of the workpiece, and setting the priority of the workpiece with lagging processing progress (that is, the total time required for the execution of the remaining processing steps is relatively longer) higher, ensuring that the workpiece with lagging processing progress is processed first, and will not continue to be idle for a long time to cause deadlock, and at the same time balance the processing progress of each workpiece. In this embodiment, taking the method of setting the priority of these workpieces with equal total time according to the preset workpiece number order as an example of arranging them in ascending order according to the preset workpiece number order, the pseudo code of the process of selecting the workpiece currently to be processed is shown in Table 2 below:

[0080] Table 2

[0081]

[0082]

[0083] In other embodiments, other methods of setting the priority of these workpieces with equal total time length according to a preset workpiece number order may also be adopted; if there is no need to consider the processing progress balance and anti-deadlock effect between the workpieces, other priority determination methods may also be adopted, and even other methods of selecting the workpiece currently to be processed may be adopted, and this is not limited in the present invention.

[0084] In addition, in order to ensure that the workpiece currently to be processed selected according to the priority each time can adapt to the changes in the state space after the decision result is executed, each time a decision result is obtained to determine the processing equipment used for the processing procedure, the current priority of the workpiece is updated according to the situation after the execution of the decision result.

[0085] In summary, the pseudo code of the overall process of completing the construction of the production scheduling plan can be represented by the following Table 3:

[0086] Table 3

[0087]

[0088] Figure 3 The figure shows a Gantt chart of scheduling decisions made according to the heterogeneous workpiece system production scheduling solution constructed according to the construction method of this embodiment under the guidance of the decision model for two heterogeneous workpieces, wherein the horizontal axis represents time and the vertical axis represents different processes. Figure 3 The actual guarantee process of two types of four heterogeneous workpieces (0, 1, 2, and 3 represent four heterogeneous workpieces, among which the heterogeneous workpieces represented by 0, 1, and 2 are of the same type, referred to as workpiece type A; the heterogeneous workpiece represented by 3 is of another type, referred to as workpiece type B) is described. The decision model allocates actual processing equipment to the workpiece at each decision moment and indicates the decision-making behavior of the workpiece, thereby completing the production scheduling plan for the entire process and minimizing the total processing time.

[0089] The schematic diagrams of the timing relationship of the processes required to be executed for the two heterogeneous workpieces (i.e., workpiece type A and workpiece type B) set in this embodiment are respectively as shown in FIG. Figure 4 and Figure 5 As shown, Figure 4 and Figure 5The dark process box indicates that the process requires processing resources during processing, and the light process box indicates that the process does not require processing resources during processing, that is, the required processing resources are empty; in order to intuitively show the superiority of the construction method of the heterogeneous workpiece system production scheduling solution of this embodiment, the construction method is compared with the classic combinatorial optimization algorithm genetic algorithm, and the different heterogeneous workpiece system production scheduling solutions constructed by the two methods under the same environment and resource settings are measured, and the different construction speeds and final construction qualities of the two are compared. The final results are shown in Table 4 and Figure 6 As shown:

[0090] Table 4

[0091] Number of workpieces This embodiment constructs the method Genetic Algorithms 1 0.3620s 92.2158s 2 0.3980s 148.3864s 3 0.4869s 196.6379s 4 0.5694s 208.4142s 5 0.7539s 300.6371s

[0092] Table 4 shows the time required for the construction method of this embodiment and the classic combinatorial optimization algorithm genetic algorithm to construct a production scheduling solution for a heterogeneous workpiece system under different numbers of workpieces; Figure 6 The construction method of this embodiment is shown in the case of different numbers of workpieces ( Figure 6 The data corresponding to "Mine" in the above example) and the classic combinatorial optimization algorithm genetic algorithm ( Figure 6 The total processing time of the production scheduling plan for the heterogeneous workpiece system constructed by the data corresponding to the "GA" in the figure); by comparison, it can be seen that the construction method of the production scheduling plan for the heterogeneous workpiece system of this embodiment can ensure a higher construction quality (that is, a shorter total processing time) while ensuring a superior construction speed.

[0093] Embodiment of a production scheduling method for a heterogeneous workpiece system

[0094] This embodiment provides a production scheduling method for a heterogeneous workpiece system. During the process of constructing the production scheduling plan for the heterogeneous workpiece system, the actual production scheduling of the heterogeneous workpiece system is controlled by using the information obtained from the construction process; the construction of the production scheduling plan for the heterogeneous workpiece system is implemented according to the method for constructing the production scheduling plan for the heterogeneous workpiece system in the above-mentioned embodiment of the method for constructing the production scheduling plan for the heterogeneous workpiece system.

[0095] Since the specific principles and beneficial effects of the heterogeneous workpiece system production scheduling method in this embodiment have been described in detail in the above-mentioned embodiment of the method for constructing a heterogeneous workpiece system production scheduling solution, they will not be repeated here.

[0096] Computer System Embodiment

[0097] This embodiment provides a technical solution for a computer system, which includes a processor having executable program instructions stored therein, and the executable program instructions are used to be executed to implement the heterogeneous workpiece system production scheduling method in the above-mentioned heterogeneous workpiece system production scheduling method embodiment.

[0098] Since the specific working principle and effect of the computer system in this embodiment have been described in detail in the above-mentioned embodiment of the production scheduling method for a heterogeneous workpiece system, they will not be repeated here.

[0099] Computer Readable Storage Medium Embodiments

[0100] This embodiment provides a technical solution of a computer-readable storage medium, in which computer program instructions are stored. The computer program instructions are used to implement the heterogeneous workpiece system production scheduling method in the above-mentioned heterogeneous workpiece system production scheduling method embodiment when executed.

[0101] Since the specific working principle and effect of the computer-readable storage medium in this embodiment have been described in detail in the above-mentioned embodiment of the production scheduling method for a heterogeneous workpiece system, they will not be repeated here.

[0102] It should be understood that the above specific embodiments of the present invention are only used to illustrate or explain the principles of the present invention, but do not constitute a limitation to the present invention.

Claims

1. A method for constructing a production scheduling scheme for a heterogeneous workpiece system, characterized in that: include: 1) Select the workpiece to be processed, determine the current processing procedure of the workpiece according to the serial logical order of the processing procedure of the workpiece, and determine the decision time of the procedure according to the timeline of the processing procedure; 2) Input the state space corresponding to the decision moment, including the current processing procedure, the occupancy status of each processing equipment, and the occupancy status of each processing resource, into the decision model trained corresponding to the heterogeneous workpiece type to which the workpiece belongs, and obtain the decision result for determining the processing equipment used for the processing procedure, and update the timeline of the processing procedure and the locked time period of the processing equipment and processing resources; repeat 1)-2) until all workpieces are processed to complete the construction of the production scheduling plan; The method for determining the state space corresponding to the decision moment includes: using the decision moment, the timeline of the processing process, and the locked time period of the processing equipment and processing resources to obtain the occupation state of each processing equipment and the occupation state of each processing resource in the state space corresponding to the decision moment of the process; The method for determining the serial logical order of the workpiece processing procedures includes: setting the logical order of the predecessor processing procedure before the successor processing procedure; setting the serial logical order between the processing procedures that do not have a predecessor-successor relationship according to a set sorting method; The method of setting the timeline of the process includes: setting corresponding parallel timelines for each process set that is parallel in time sequence of a single workpiece; setting corresponding serial timelines for each process set that is serial in time sequence of the remaining workpiece; each process set contains at least one processing process and different processing processes in the set are serial in time sequence; The method for updating the timeline of the processing procedure includes: if the decision result of executing the procedure does not require changing the processing equipment where the workpiece is located, then directly superimpose the execution time of the procedure onto its corresponding timeline; otherwise, take the end time of the timeline with the longest total time among the timelines of all the procedures of the workpiece currently to be processed as the basis, unify the end time of the timelines of all the procedures of the workpiece, and then superimpose the execution time of the procedure onto its corresponding timeline.

2. The method for constructing a production scheduling scheme for a heterogeneous workpiece system according to claim 1, characterized in that: The method of selecting the workpiece to be processed currently is to select the workpiece with the highest current priority from the workpieces to be processed; The current priority of the workpiece is determined in the following manner: for a workpiece whose processing steps have not been fully completed, the total time required for executing the remaining processing steps of the workpiece itself is used, and the longer the total time is, the higher the priority of the workpiece; If the total durations are equal, the priorities of the workpieces with the same total durations are set according to a preset order of workpiece numbers.

3. The method for constructing a production scheduling scheme for a heterogeneous workpiece system according to claim 2, characterized in that: Each time a decision result for determining the processing equipment used for the processing step is obtained, the current priority of the workpiece is updated according to the situation after the decision result is executed.

4. The method for constructing a production scheduling scheme for a heterogeneous workpiece system according to claim 1 or 2, characterized in that: The setting order is determined according to the time consumption of the processing procedure and the preset process number; the shorter the processing time consumption, the earlier the setting order is; if the processing procedures consume the same time consumption, the order of these processing procedures with the same time consumption is set according to the preset process number order.

5. The method for constructing a production scheduling solution for a heterogeneous workpiece system according to claim 1 or 2, characterized in that: The execution time of an operation is determined by: a) Taking the intersection of the idle time period of the processing equipment used in the process determined according to the decision result and the non-locked time period of the processing resources required to execute the process; determining the processing start time corresponding to the decision time of the process; the idle time period includes both the non-locked time period of the processing equipment and the locked time period of the processing equipment that meets the set condition; the set condition is that the workpiece processed corresponding to the locked time period is the same workpiece as the workpiece corresponding to the process; b) Determine the processing end time of the process in combination with the processing start time of the process and the time required for the execution of the process itself; check whether the processing time period from the processing start time to the processing end time of the process is included in the intersection, and if so, determine the execution time of the process according to the processing time period; otherwise, postpone the processing start time of the process, and repeat b) until the execution time of the process is determined.

6. The method for constructing a production scheduling solution for a heterogeneous workpiece system according to claim 1 or 2, characterized in that: The state space also includes the execution state of each processing step of the workpiece currently to be processed, the distance of the workpiece currently to be processed relative to each processing equipment, and the processing equipment where the workpiece currently to be processed is located; During the training process of the decision model, the reward function corresponding to the output decision result is used to update the decision model parameters; the reward function includes a first function item corresponding to the completion status of the processing procedure corresponding to the current decision result, a second function item corresponding to the waiting time of the processing procedure corresponding to the current decision result, a third function item corresponding to the transfer time of the processing procedure corresponding to the current decision result, and a fourth function item corresponding to the load balancing status of the processing procedure corresponding to the current decision result; The value of the first function item when the processing procedure corresponding to the decision result is successfully completed is greater than the value of the first function item when the processing procedure corresponding to the decision result is not successfully completed; the shorter the waiting time of the processing procedure corresponding to the decision result, the greater the value of the second function item; the shorter the transfer time of the processing procedure corresponding to the decision result, the greater the value of the third function item; the smaller the difference between the load rate of each processing resource corresponding to the processing procedure corresponding to the decision result and the load rate of all processing resources, the greater the value of the fourth function item.

7. The method for constructing a production scheduling solution for a heterogeneous workpiece system according to claim 1 or 2, characterized in that: Each decision model is trained using the DQN algorithm.

8. A production scheduling method for a heterogeneous workpiece system, characterized in that: During the construction of the production scheduling plan for the heterogeneous workpiece system, the actual production scheduling of the heterogeneous workpiece system is controlled through the information obtained from the construction process; the construction of the production scheduling plan for the heterogeneous workpiece system is implemented according to the construction method of the production scheduling plan for the heterogeneous workpiece system according to any one of claims 1-7.

9. A computer system comprising a processor, wherein the processor is configured to execute executable program instructions, wherein: The executable program instructions are used to be executed to implement the heterogeneous workpiece system production scheduling method according to claim 8.

10. A computer-readable storage medium, wherein computer program instructions are stored in the storage medium, characterized in that: The computer program instructions are used to implement the heterogeneous workpiece system production scheduling method according to claim 8 when being executed.

Images