Flexible job shop scheduling method, device, equipment and medium

Abstract

The application belongs to the technical field of manufacturing, and particularly relates to a flexible job shop scheduling method, device, equipment and medium. The application aims to solve the problem that the existing job shop scheduling method leads to low efficiency of determining a scheduling result when the number of workpieces and machines is large. The flexible job shop scheduling method, device, equipment and medium provided by the application input the machining information and the workpiece state information into a proximal strategy optimization model to obtain a scheduling result after obtaining the machining information and the workpiece state information, wherein the proximal strategy optimization model is a pre-trained calculation model for determining a scheduling result according to machining information and workpiece state information. The scheme schedules the workpieces through the proximal strategy optimization model, and effectively improves the efficiency of obtaining the scheduling result.

Description

Technical Field

[0001] This application belongs to the field of manufacturing technology, specifically relating to flexible workshop scheduling methods, devices, equipment, and media. Background Technology

[0002] With the development of technology and the improvement of people's living standards, people have higher requirements for the quantity and speed of product production. Therefore, workshop scheduling schemes have attracted much attention in the product production process. Flexible work-shop scheduling is a general workshop scheduling method, widely used in industries such as semiconductor manufacturing, automobile assembly, and textiles. It is one of the most common scheduling modes in modern industrial manufacturing environments and has many applications in actual production and daily life.

[0003] The flexible job shop scheduling problem is typically described as follows: multiple jobs need to be processed on multiple machines; each job contains one or more operations; the order of operations for each job is predetermined; each operation can be performed on at least one machine; and the processing time for each operation varies depending on the machine. In existing technologies, flexible job shop scheduling methods usually employ operations research or linear programming to determine the constraints and then solve the problem. However, when there are many jobs and machines, determining the scheduling scheme is slow.

[0004] In summary, existing workshop scheduling methods are inefficient in determining scheduling results when there are a large number of workpieces and machines. Summary of the Invention

[0005] In order to solve the above-mentioned problems in the prior art, namely, the problem that the scheduling efficiency is low when the number of workpieces and machines is large, the embodiments of this application provide a flexible job shop scheduling method, device, equipment and medium.

[0006] In a first aspect, embodiments of this application provide a flexible job shop scheduling method, including:

[0007] Acquire processing information and workpiece status information. The processing information includes the number of workpieces, the number of machines, the process for each workpiece, the selectable machine identifier for each process of each workpiece, and the processing time for each process of each workpiece on the selectable machines. The workpiece status information includes the group identifier for each workpiece, the transport time adjacency matrix for the current process, the number of processes, the time limit information between processes, the processing order identifier for the current machine, the current time, the current process feature information, the current machine characteristic information, the current machine processing time, and the transport time adjacency matrix for the current machine.

[0008] The processing information and the workpiece state information are input into the near-end strategy optimization model to obtain the scheduling result. The near-end strategy optimization model is a pre-trained computational model used to determine the scheduling result based on the processing information and the workpiece state information.

[0009] In the preferred embodiment of the above-mentioned flexible workshop scheduling method, the step of inputting the processing information and the workpiece status information into the near-end strategy optimization model to obtain the scheduling result includes:

[0010] Based on the processing information, the workpiece status information, and the determined scheduling information, a workpiece to be processed and its scheduling information are determined from all the workpieces.

[0011] Update the workpiece status information and the number of workpieces to be processed;

[0012] If the updated number of workpieces to be processed is equal to the number of workpieces, the scheduling result is generated based on the scheduling information of each workpiece to be processed.

[0013] In the preferred embodiment of the above-mentioned flexible workshop scheduling method, the method further includes:

[0014] If the updated number of workpieces to be processed is less than the number of workpieces, based on the processing information, the updated workpiece status information, and the determined scheduling information, one workpiece to be processed and its corresponding scheduling information are re-determined from all workpieces except the workpiece to be processed. The workpiece status information and the number of workpieces to be processed are updated until the updated number of workpieces to be processed equals the number of workpieces. The scheduling result is then generated based on the scheduling information of each workpiece to be processed.

[0015] In the preferred embodiment of the above-mentioned flexible workshop scheduling method, the step of determining a workpiece to be processed and its scheduling information from all workpieces based on the processing information, the workpiece status information, and the determined scheduling information includes:

[0016] Based on the processing information, the workpiece status information, and the determined scheduling information, determine multiple reward values ​​corresponding to the scheduling of each workpiece using multiple preset scheduling rules;

[0017] The workpiece with the highest reward value among all reward values ​​corresponding to all workpieces is determined as the workpiece to be processed;

[0018] The scheduling information is generated according to the preset scheduling rule corresponding to the maximum reward value.

[0019] In the preferred embodiment of the above-mentioned flexible workshop scheduling method, before acquiring processing information and workpiece status information, the method further includes:

[0020] Acquire multiple sets of training data, each set including processing information and workpiece status information;

[0021] The model is trained based on the multiple sets of training data and the initial reinforcement learning model until the number of training iterations equals the preset number of training iterations, thus obtaining the near-end policy optimization model.

[0022] In the preferred embodiment of the above-mentioned flexible workshop scheduling method, the step of training the model based on the multiple sets of training data and the initial reinforcement learning model until the number of training iterations equals the preset number of training iterations, to obtain the proximal policy optimization model, includes:

[0023] Select one set of training data from the multiple sets of training data;

[0024] The training data is input into the initial reinforcement learning model to obtain the scheduling training results;

[0025] Update training times;

[0026] If the updated number of training iterations equals the preset number of training iterations, the proximal policy optimization model is obtained;

[0027] If the updated number of training iterations is less than the preset number of training iterations, then the initial reinforcement learning model is updated to obtain the trained reinforcement learning model; and a new set of training data is selected from the multiple sets of training data and input into the trained reinforcement learning model to update the number of training iterations until the updated number of training iterations equals the preset number of training iterations, thus obtaining the proximal policy optimization model.

[0028] Secondly, embodiments of this application provide a flexible workshop scheduling device, comprising:

[0029] The acquisition module is used to acquire processing information and workpiece status information. The processing information includes the number of workpieces, the number of machines, the process of each workpiece, the selectable machine identifier for each process of each workpiece, and the processing time of each process of each workpiece on the selectable machines. The workpiece status information includes the group identifier of each workpiece, the transport time adjacency matrix of the current process, the number of processes, the time limit information between processes, the processing order identifier of the current machine, the current time, the current process feature information, the current machine characteristic information, the current machine processing time, and the transport time adjacency matrix of the current machine.

[0030] The processing module is used to input the processing information and the workpiece state information into the near-end strategy optimization model to obtain the scheduling result. The near-end strategy optimization model is a pre-trained computational model used to determine the scheduling result based on the processing information and the workpiece state information.

[0031] Thirdly, embodiments of this application provide an electronic device, including:

[0032] Processor, memory, communication interface;

[0033] The memory is used to store the executable instructions of the processor;

[0034] The processor is configured to execute the flexible job shop scheduling method of any one of the first aspects by executing the executable instructions.

[0035] Fourthly, embodiments of this application provide a readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the flexible job shop scheduling method described in any of the first aspects.

[0036] Fifthly, embodiments of this application provide a computer program product, including a computer program, which, when executed by a processor, is used to implement the flexible job shop scheduling method described in any of the first aspects.

[0037] Those skilled in the art will understand that the flexible job shop scheduling method, apparatus, equipment, and medium provided in this application, after acquiring processing information and workpiece status information, inputs the processing information and workpiece status information into a near-end strategy optimization model to obtain scheduling results. The near-end strategy optimization model is a pre-trained computational model used to determine the scheduling results based on the processing information and workpiece status information. This solution effectively improves the efficiency of obtaining scheduling results by scheduling workpieces through a near-end strategy optimization model. Attached Figure Description

[0038] A preferred embodiment of the flexible job shop scheduling method of this application will now be described with reference to the accompanying drawings. The drawings are as follows:

[0039] Figure 1a This is a flowchart illustrating an embodiment of the flexible workshop scheduling method provided in this application;

[0040] Figure 1b The Gantt chart provided for this application;

[0041] Figure 2 A flowchart illustrating Embodiment 2 of the flexible workshop scheduling method provided in this application;

[0042] Figure 3 A flowchart illustrating Embodiment 3 of the flexible workshop scheduling method provided in this application;

[0043] Figure 4 This is a schematic diagram of the structure of an embodiment of the flexible workshop scheduling device provided in this application;

[0044] Figure 5 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0045] First, those skilled in the art should understand that these embodiments are merely for explaining the technical principles of this application and are not intended to limit the scope of protection of this application. Those skilled in the art can make adjustments as needed to adapt to specific application scenarios.

[0046] Secondly, it should be noted that in the description of the embodiments of this application, the terms "inner" and "outer" and other terms indicating the direction or positional relationship are based on the direction or positional relationship shown in the drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or component must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this application.

[0047] Furthermore, it should be noted that, in the description of the embodiments of this application, unless otherwise explicitly specified and limited, the terms "connected" and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the embodiments of this application according to the specific circumstances.

[0048] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0049] With the development of technology, the number of products produced in workshops is increasing, which puts forward higher requirements for production efficiency. In order to improve production efficiency, a good scheduling method is indispensable. Flexible workshop scheduling is a method of scheduling workpieces.

[0050] Solving the flexible job shop problem yields scheduling methods. The flexible job shop scheduling problem is typically described as follows: multiple jobs need to be processed on multiple machines; each job contains one or more operations; the order of operations for each job is predetermined; each operation can be performed on at least one machine; and the processing time varies depending on the machine. In existing technologies, flexible job shop scheduling methods usually employ operations research or linear programming to determine the constraints before solving the problem. However, when there are many jobs and machines, determining the scheduling scheme is slow.

[0051] Therefore, existing workshop scheduling methods, which use operations research or linear programming, suffer from low efficiency in determining scheduling results when there are a large number of workpieces and machines.

[0052] This embodiment provides a flexible job shop scheduling method. To improve the efficiency of determining the scheduling result, a near-end strategy optimization model can be trained based on the processing information and workpiece status information used for training. Then, the processing information and workpiece status information of the workpieces to be scheduled are input into the near-end strategy optimization model to obtain the scheduling result. By using a near-end measurement optimization model, the efficiency of determining the scheduling result can be effectively improved.

[0053] The principles and features of the embodiments of this application are described below with reference to the accompanying drawings. The examples given are only used to explain the embodiments of this application and are not intended to limit the scope of the embodiments of this application.

[0054] It should be noted that the executing entity of this application can be a computer, a server, or a terminal device or other device capable of model training and model application. This application does not limit these devices and can be selected according to the actual situation.

[0055] The following describes the application scenarios of the flexible workshop scheduling method provided in this application.

[0056] In this scenario, the user has a batch of workpieces that need to be processed in the workshop. Since multiple workpieces need to be processed on multiple machines; each workpiece contains one or more processes; the process sequence of each workpiece is predetermined; each process can be processed on at least one processing machine; and the processing time of each process varies depending on the processing machine, a flexible workshop scheduling method is needed to determine the scheduling result.

[0057] The user inputs the processing information and workpiece status information corresponding to this batch of workpieces into the near-end strategy optimization model in the computer. The processing information includes the number of workpieces, the number of machines, the operation of each workpiece, the selectable machine identifier for each operation of each workpiece, and the processing time of each operation of each workpiece on the selectable machines. The workpiece status information includes the group identifier of each workpiece, the transport time adjacency matrix of the current operation, the number of operations, the time limit information between operations, the processing order identifier of the current machine, the current time, the current operation feature information, the current machine feature information, the current machine processing time, and the transport time adjacency matrix of the current machine.

[0058] The near-end strategy optimization model determines a workpiece to be processed and its scheduling information from all workpieces based on processing information, workpiece status information, and the determined scheduling information; and then updates the workpiece status information and the number of workpieces to be processed.

[0059] Determine if the updated number of workpieces to be processed is equal to the number of workpieces; if the updated number of workpieces to be processed is equal to the number of workpieces, it means that each workpiece has been scheduled. Generate a scheduling result based on the scheduling information of each workpiece to be processed.

[0060] If the updated number of workpieces to be processed is less than the number of workpieces, then based on the processing information, the updated workpiece status information, and the determined scheduling information, a new workpiece to be processed and its corresponding scheduling information are determined from all workpieces except the workpieces to be processed. The workpiece status information and the number of workpieces to be processed are updated, and the updated number of workpieces to be processed equals the number of workpieces. Based on the scheduling information of each workpiece to be processed, a scheduling result is generated.

[0061] The computer may control the machine to process the workpiece based on the scheduling results. Furthermore, the computer can generate a Gantt chart of the scheduling results for display by the user.

[0062] It should be noted that the embodiments of this application do not limit the actual form of various devices in the application scenario, nor do they limit the interaction method between devices. In the specific application of the solution, it can be set according to actual needs.

[0063] The technical solution of this application will now be described in detail through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0064] Figure 1aThis is a flowchart illustrating an embodiment of the flexible job shop scheduling method provided in this application. This embodiment describes how a computer inputs workpiece processing information and workpiece status information into a near-end strategy optimization model to obtain scheduling results. The method in this embodiment can be implemented through software, hardware, or a combination of both. Figure 1a As shown, the flexible workshop scheduling method specifically includes the following steps:

[0065] S101: Obtain processing information and workpiece status information.

[0066] A user has a batch of workpieces that need to be processed. After obtaining the scheduling result using the flexible job shop scheduling method, the user can proceed with the processing based on the scheduling result. Therefore, the user needs to input the processing information and workpiece status information into the computer.

[0067] In this step, after the user inputs the processing information and workpiece status information into the computer, the computer can obtain the processing information and workpiece status information. The processing information includes the number of workpieces, the number of machines, the number of operations for each workpiece, the selectable machine identifier for each operation of each workpiece, and the processing time for each operation of each workpiece on the selectable machines. The workpiece status information includes the group identifier for each workpiece, the transport time adjacency matrix for the current operation, the number of operations, the time limit information between operations, the processing order identifier on the current machine, the current time, the characteristic information of the current operation, the characteristic information of the current machine, the processing time of the current machine, and the transport time adjacency matrix on the current machine.

[0068] S102: Input the processing information and workpiece status information into the near-end strategy optimization model to obtain the scheduling result.

[0069] In this step, after the computer obtains the processing information and workpiece status information, it inputs them into the near-end strategy optimization model. Since the near-end strategy optimization model is a pre-trained computational model used to determine the scheduling result based on the processing information and workpiece status information, the scheduling result can be obtained.

[0070] The computer can identify a workpiece to be processed from all workpieces based on processing information and workpiece status information, and obtain scheduling information for that workpiece. It then updates the workpiece status information by removing the relevant information from the workpiece to be processed and updating the status information of the remaining workpieces according to the current situation. This process continues until all workpieces are processed, and the scheduling information for each workpiece is obtained, thus yielding the scheduling result.

[0071] In addition, the computer can generate Gantt charts based on the scheduling results for users to view, for example, Figure 1b The Gantt chart provided for this application, such as Figure 1bAs shown, there are three workpieces that need to be processed on three machines. The white block represents workpiece A, the black block represents workpiece B, and the shaded block represents workpiece C. These three workpieces need to be processed on machines 1, 2, and 3. The processing information and workpiece status information of the three workpieces are input into the near-end strategy optimization model. After obtaining the scheduling result, the model can be generated. Figure 1b Gantt chart.

[0072] The flexible job shop scheduling method provided in this embodiment obtains scheduling results by inputting workpiece processing information and workpiece status information into a near-end measurement optimization model. Compared to existing technologies that use operations research or linear programming to determine scheduling results, this solution can obtain scheduling results using a near-end strategy optimization model, effectively improving computational efficiency and the efficiency of obtaining scheduling results. Furthermore, existing technologies require adjustments to parameters in operations research or linear programming methods and recalculation when processing different batches of workpieces, as processing information changes. This solution eliminates the need to adjust the near-end strategy optimization model, resulting in greater applicability and improved overall efficiency.

[0073] Figure 2 This is a flowchart illustrating a second embodiment of the flexible job shop scheduling method provided in this application. Based on the above embodiments, this application describes how the computer inputs the processing information and the workpiece status information into a near-end strategy optimization model to obtain the scheduling result. Figure 2 As shown, the flexible workshop scheduling method specifically includes the following steps:

[0074] S201: Based on the processing information, workpiece status information, and the determined scheduling information, determine one workpiece to be processed and its scheduling information from all workpieces.

[0075] In this step, the computer inputs the processing information and workpiece status information into the near-end strategy optimization model. The near-end strategy optimization model determines a workpiece to be processed and its scheduling information from all workpieces based on the processing information, workpiece status information, and the determined scheduling information.

[0076] Specifically, based on processing information, workpiece status information, and determined scheduling information, multiple reward values ​​are determined for each workpiece using multiple preset scheduling rules.

[0077] First, based on the established scheduling information, determine the maximum completion time. That is, based on the established scheduling information, determine the corresponding completion time, and then use the longest of these three completion times as the maximum completion time. For example, if three workpieces have been identified as workpieces to be processed, determine the corresponding completion time based on the scheduling information of these three workpieces, and then use the longest of these three completion times as the maximum completion time.

[0078] Furthermore, for each workpiece, since there are multiple preset scheduling rules in the near-end strategy optimization model, the corresponding scheduling information can be determined by using these preset scheduling rules, processing information, and workpiece status information. Then, based on these scheduling information, the corresponding candidate completion time can be determined, and the difference between the maximum completion time and the candidate completion time can be calculated to obtain the reward value. In this way, multiple reward values ​​can be obtained for each workpiece when scheduling using multiple preset scheduling rules.

[0079] The formula for calculating the reward value is: r t =C mint -C jk , where r t C represents the reward value. mint C represents the maximum completion time. jk Indicates the estimated completion time.

[0080] The workpiece with the highest reward value among all the reward values ​​corresponding to all workpieces is identified as the workpiece to be processed. This ensures a shorter processing time and improves processing efficiency. Scheduling information is generated based on the preset scheduling rules corresponding to the highest reward value.

[0081] It should be noted that the preset scheduling rules can be first-in-first-out rules, last-in-first-out rules, shortest processing time rules, longest processing time rules, shortest number of processing steps rules, longest number of processing steps rules, shortest remaining processing time rules, longest remaining processing time rules, etc. This application embodiment does not limit the preset scheduling rules, and can be selected according to the actual situation.

[0082] S202: Update workpiece status information and the number of workpieces to be processed.

[0083] In this step, after the computer obtains a workpiece to be processed and the corresponding scheduling information, it needs to update the workpiece status information and the number of workpieces to be processed in order to obtain the next workpiece to be processed.

[0084] Updating workpiece status information requires deleting the information corresponding to the workpiece to be processed as determined by the computer from the original workpiece status information. Then, for workpieces that have not been determined to be processed, the corresponding workpiece status information is updated according to the current operating status.

[0085] Updating the number of workpieces to be processed means incrementing the number of workpieces to be processed by one.

[0086] S203: Determine whether the updated number of workpieces to be processed is equal to the number of workpieces; if the updated number of workpieces to be processed is equal to the number of workpieces, proceed to step S204; if the updated number of workpieces to be processed is less than the number of workpieces, proceed to step S205.

[0087] In this step, after the computer updates the workpiece status information and the number of workpieces to be processed, in order to determine whether any workpieces have not had their scheduling information determined, it is necessary to determine whether the updated number of workpieces to be processed is equal to the number of workpieces. If the updated number of workpieces to be processed is equal to the number of workpieces, it means that scheduling information has been generated for all workpieces. If the updated number of workpieces to be processed is less than the number of workpieces, it means that some workpieces have not had their scheduling information determined.

[0088] S204: Generate scheduling results based on the scheduling information of each workpiece to be processed.

[0089] In this step, if the updated number of workpieces to be processed equals the total number of workpieces, it means that scheduling information has been generated for all workpieces. Based on the scheduling information for each workpiece to be processed, a scheduling result is generated. The scheduling information for each workpiece to be processed is then merged to generate the final scheduling result.

[0090] S205: Based on the processing information, the updated workpiece status information, and the determined scheduling information, re-determine one workpiece to be processed and its corresponding scheduling information from all workpieces except the workpiece to be processed, update the workpiece status information and the number of workpieces to be processed, until the updated number of workpieces to be processed equals the number of workpieces, and generate a scheduling result based on the scheduling information of each workpiece to be processed.

[0091] In this step, if the updated number of workpieces to be processed is less than the number of workpieces, it means that some workpieces have not had their scheduling information determined. It is necessary to continue to determine the workpieces to be processed and their corresponding scheduling information. This requires re-determining one workpiece to be processed and its corresponding scheduling information from all workpieces except the workpieces to be processed, based on the processing information, the updated workpiece status information, and the determined scheduling information. The workpiece status information and the number of workpieces to be processed are then updated until the updated number of workpieces to be processed equals the number of workpieces. Based on the scheduling information of each workpiece to be processed, a scheduling result is generated.

[0092] The flexible workshop scheduling method provided in this embodiment determines each workpiece to be processed and its corresponding scheduling information in sequence based on processing information, workpiece status information and determined scheduling information, and then generates scheduling results, which effectively improves the efficiency of obtaining scheduling results.

[0093] Figure 3 This is a flowchart illustrating a third embodiment of the flexible job shop scheduling method provided in this application. Based on the above embodiments, this application describes the training process of the near-end strategy optimization model. Figure 3 As shown, the flexible workshop scheduling method specifically includes the following steps:

[0094] S301: Acquire multiple sets of training data.

[0095] In this step, if the user wants to optimize the model using a proximal policy, they need to train the initial reinforcement learning model with training data by inputting multiple sets of training data into the computer. The computer will then obtain multiple sets of training data, each set including processing information and workpiece status information.

[0096] S302: Train the model based on multiple sets of training data and the initial reinforcement learning model until the number of training iterations equals the preset number of training iterations, and obtain the near-end policy optimization model.

[0097] In this step, after obtaining multiple sets of training data, the model is trained based on the multiple sets of training data and the initial reinforcement learning model until the number of training iterations equals the preset number of training iterations, thus obtaining the near-end policy optimization model.

[0098] Specifically, a set of training data is selected from multiple sets of training data. This training data is then input into the initial reinforcement learning model to obtain the scheduled training results. The number of training iterations is then updated accordingly.

[0099] If the updated number of training iterations equals the preset number of training iterations, the proximal policy optimization model is obtained.

[0100] If the updated number of training iterations is less than the preset number of training iterations, the initial reinforcement learning model is updated to obtain the trained reinforcement learning model. Then, a new set of training data is selected from multiple sets of training data and input into the trained reinforcement learning model to update the number of training iterations until the updated number of training iterations equals the preset number of training iterations, thus obtaining the proximal policy optimization model.

[0101] It should be noted that the preset number of training sessions can be 30,000, 50,000, or 70,000. This application embodiment does not limit it and can be set according to the actual situation.

[0102] It should be noted that the process of updating the initial reinforcement learning model requires calculating the advantage function and discounted reward, and then calculating the evaluator network loss function value; then calculating the probability ratio, and then calculating the actor network loss function value; and finally updating the initial reinforcement learning model based on the evaluator network loss function value and the actor network loss function value.

[0103] The training process of the near-end policy optimization model can be viewed as a Markov decision process, which can be represented by a quintuple (S, A, P, R, γ), where S represents a finite set of states, A represents a finite set of actions, P represents the state transition probability, R represents the reward function, and γ represents the discount factor. In a Markov decision process, the state space, action space, and reward function need to be defined. The state space is n*10, where n represents the number of jobs, and 10 represents the amount of information included in the job's state information. Each job is described by 10 state features, which are the information included in the job's state information. The action space includes all job identifiers and one no-operation. The reward function is r. t =C mint -C jk , where r t C represents the reward value. mint C represents the maximum completion time. jk This indicates the estimated completion time. In the interaction between the agent and the scheduling environment, the environment sends state observations and rewards to the agent, which then sends back actions to interact with the environment and transition to a new state. The period from entering the first state to the last action sent is called a training round. After each round, the production scheduling model is reset and enters the next round.

[0104] The flexible job shop scheduling method provided in this embodiment trains an initial reinforcement learning model based on multiple sets of processing information and workpiece status information until the number of training iterations equals the preset number of training iterations, thereby obtaining a near-end policy optimization model. Using the near-end policy optimization model for flexible job shop scheduling can effectively improve the efficiency of obtaining scheduling results.

[0105] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0106] Figure 4 This is a structural schematic diagram of an embodiment of the flexible workshop scheduling device provided in this application. Figure 4 As shown, the flexible workshop scheduling device 40 includes:

[0107] The acquisition module 41 is used to acquire processing information and workpiece status information. The processing information includes the number of workpieces, the number of machines, the process of each workpiece, the machine identifier of each process of each workpiece, and the processing time of each process of each workpiece on the selectable machines. The workpiece status information includes the group identifier of each workpiece, the transport time adjacency matrix of the current process, the number of processes, the time limit between processes, the processing order identifier of the current machine, the current time, the current process feature information, the current machine feature information, the current machine processing time, and the transport time adjacency matrix of the current machine.

[0108] Processing module 42 is used to input the processing information and the workpiece state information into the near-end strategy optimization model to obtain the scheduling result. The near-end strategy optimization model is a pre-trained computational model used to determine the scheduling result based on the processing information and the workpiece state information.

[0109] Furthermore, the processing module 42 is specifically used for:

[0110] Based on the processing information, the workpiece status information, and the determined scheduling information, a workpiece to be processed and its scheduling information are determined from all the workpieces.

[0111] Update the workpiece status information and the number of workpieces to be processed;

[0112] If the updated number of workpieces to be processed is equal to the number of workpieces, the scheduling result is generated based on the scheduling information of each workpiece to be processed.

[0113] Furthermore, the processing module 42 is also used for:

[0114] If the updated number of workpieces to be processed is less than the number of workpieces, based on the processing information, the updated workpiece status information, and the determined scheduling information, one workpiece to be processed and its corresponding scheduling information are re-determined from all workpieces except the workpiece to be processed. The workpiece status information and the number of workpieces to be processed are updated until the updated number of workpieces to be processed equals the number of workpieces. The scheduling result is then generated based on the scheduling information of each workpiece to be processed.

[0115] Furthermore, the processing module 42 is also used for:

[0116] Based on the processing information, the workpiece status information, and the determined scheduling information, determine multiple reward values ​​corresponding to the scheduling of each workpiece using multiple preset scheduling rules;

[0117] The workpiece with the highest reward value among all reward values ​​corresponding to all workpieces is determined as the workpiece to be processed;

[0118] The scheduling information is generated according to the preset scheduling rule corresponding to the maximum reward value.

[0119] Furthermore, the acquisition module 41 is also used to acquire multiple sets of training data, each set of training data including processing information and workpiece status information;

[0120] Furthermore, the processing module 42 is also used to train the model based on the multiple sets of training data and the initial reinforcement learning model until the number of training iterations equals the preset number of training iterations, thereby obtaining the near-end policy optimization model.

[0121] Furthermore, the processing module 42 is also used for:

[0122] Select one set of training data from the multiple sets of training data;

[0123] The training data is input into the initial reinforcement learning model to obtain the scheduling training results;

[0124] Update training times;

[0125] If the updated number of training iterations equals the preset number of training iterations, the proximal policy optimization model is obtained;

[0126] If the updated number of training iterations is less than the preset number of training iterations, then the initial reinforcement learning model is updated to obtain the trained reinforcement learning model; and a new set of training data is selected from the multiple sets of training data and input into the trained reinforcement learning model to update the number of training iterations until the updated number of training iterations equals the preset number of training iterations, thus obtaining the proximal policy optimization model.

[0127] The flexible workshop scheduling device provided in this embodiment is used to execute the technical solutions in any of the aforementioned method embodiments. Its implementation principle and technical effects are similar, and will not be described again here.

[0128] Figure 5 A schematic diagram of the structure of the electronic device provided in this application. Figure 5 As shown, the electronic device 50 includes:

[0129] Processor 51, memory 52, communication interface 53;

[0130] The memory 52 is used to store the executable instructions of the processor 51;

[0131] The processor 51 is configured to execute the technical solutions in any of the foregoing method embodiments by executing the executable instructions.

[0132] Optionally, the memory 52 can be either standalone or integrated with the processor 51.

[0133] Optionally, when the memory 52 is a device independent of the processor 51, the electronic device 50 may further include:

[0134] Bus 54, memory 52 and communication interface 53 are connected to processor 51 through bus 54 and complete communication with each other. Communication interface 53 is used to communicate with other devices.

[0135] Optionally, the communication interface 53 can be implemented using a transceiver. The communication interface is used to enable communication between the database access device and other devices (e.g., clients, read-write databases, and read-only databases). The memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk drive.

[0136] Bus 54 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0137] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0138] The electronic device is used to execute the technical solutions in any of the foregoing method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0139] This application also provides a readable storage medium storing a computer program thereon, which, when executed by a processor, implements the technical solutions provided in any of the foregoing embodiments.

[0140] This application also provides a computer program product, including a computer program, which, when executed by a processor, is used to implement the technical solutions provided in any of the foregoing method embodiments.

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A flexible job shop scheduling method, characterized by, The method comprises the following steps: obtaining processing information and workpiece state information, wherein the processing information comprises workpiece quantity, machine quantity, work procedure of each workpiece, machine identifier of each work procedure of each workpiece, processing time of each work procedure of each workpiece on the optional machine, the workpiece state information comprises grouping identifier of each workpiece, transportation time adjacency matrix of the current work procedure, work procedure quantity, time length limit information between work procedures, processing order identifier of the current machine, current time, current work procedure characteristic information, current machine characteristic information, current machine processing time length, and transportation time adjacency matrix of the current machine; inputting the processing information and the workpiece state information into a proximal strategy optimization model to obtain a scheduling result, wherein the proximal strategy optimization model is a pre-trained calculation model for determining the scheduling result according to the processing information and the workpiece state information; the step of inputting the processing information and the workpiece state information into the proximal strategy optimization model to obtain the scheduling result comprises the following steps: determining one workpiece to be processed and scheduling information of the workpiece to be processed from all workpieces according to the processing information, the workpiece state information and the determined scheduling information; updating the workpiece state information and the quantity of workpieces to be processed; if the quantity of the workpieces to be processed after the update is equal to the workpiece quantity, generating the scheduling result according to the scheduling information of each workpiece to be processed.

2. The method of claim 1, wherein, The method further comprises the following steps: if the quantity of the workpieces to be processed after the update is less than the workpiece quantity, re-determining one workpiece to be processed and corresponding scheduling information from all workpieces except the workpiece to be processed according to the processing information, the updated workpiece state information and the determined scheduling information, updating the workpiece state information and the quantity of workpieces to be processed, until the quantity of the workpieces to be processed after the update is equal to the workpiece quantity, and generating the scheduling result according to the scheduling information of each workpiece to be processed.

3. The method of claim 1, wherein, The step of determining one workpiece to be processed and scheduling information of the workpiece to be processed from all workpieces according to the processing information, the workpiece state information and the determined scheduling information comprises the following steps: determining a plurality of reward values corresponding to the scheduling of each workpiece by a plurality of preset scheduling rules according to the processing information, the workpiece state information and the determined scheduling information; determining the workpiece corresponding to the maximum reward value among all reward values corresponding to all workpieces as the workpiece to be processed; generating the scheduling information according to the preset scheduling rule corresponding to the maximum reward value.

4. The method of claim 1, wherein, Before the step of obtaining the processing information and the workpiece state information, the method further comprises the following steps: obtaining a plurality of groups of training data, each group of training data comprising processing information and workpiece state information; performing model training according to the plurality of groups of training data and an initial reinforcement learning model until the number of training times is equal to a preset number of training times, to obtain the proximal strategy optimization model.

5. The method of claim 4, wherein, The step of performing model training according to the plurality of groups of training data and the initial reinforcement learning model until the number of training times is equal to the preset number of training times, to obtain the proximal strategy optimization model, comprises the following steps: selecting one group of training data from the plurality of groups of training data; inputting the training data into the initial reinforcement learning model to obtain a scheduling training result; updating the training times; if the updated training times are equal to the preset training times, obtaining the proximal policy optimization model; if the updated training times are less than the preset training times, updating the initial reinforcement learning model to obtain a trained reinforcement learning model, and reselecting a set of training data from the multiple sets of training data to input into the trained reinforcement learning model, updating the training times, until the updated training times are equal to the preset training times, and obtaining the proximal policy optimization model.

6. A flexible job shop scheduling apparatus characterized by comprising: comprising: an acquisition module, configured to acquire processing information and workpiece state information, the processing information including workpiece quantity, machine quantity, workpiece process, machine identifier selectable for each workpiece process, and processing time of each workpiece process on the selectable machine, and the workpiece state information including grouping identifier of each workpiece, transport time adjacency matrix at a current process, process quantity, process time length limitation information, processing order identifier at a current machine, current time, current process feature information, current machine characteristic information, current machine processing time length, and transport time adjacency matrix at the current machine; a processing module, configured to input the processing information and the workpiece state information into a proximal policy optimization model to obtain a scheduling result, wherein the proximal policy optimization model is a pre-trained calculation model for determining the scheduling result according to the processing information and the workpiece state information; the processing module is further configured to: determine a workpiece to be processed and scheduling information of the workpiece to be processed from all workpieces according to the processing information, the workpiece state information, and the determined scheduling information; update the workpiece state information and the quantity of workpieces to be processed; if the updated quantity of workpieces to be processed is equal to the quantity of workpieces, generate the scheduling result according to the scheduling information of each workpiece to be processed.

7. An electronic device, comprising: comprising: a processor, a memory, and a communication interface; the memory is configured to store executable instructions of the processor; wherein the processor is configured to execute the flexible job shop scheduling method of any one of claims 1 to 5 by executing the executable instructions.

8. A readable storage medium, having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the flexible job shop scheduling method of any one of claims 1 to 5.

9. A computer program product, characterised in that, comprising a computer program, which is executed by the processor to implement the flexible job shop scheduling method of any one of claims 1 to 5.

Images