Intelligent production scheduling method and system for chemical fiber products based on reinforcement learning

Abstract

The invention relates to the technical field of chemical fiber production management, in particular to an intelligent production scheduling method and system for chemical fiber products based on reinforcement learning, and the method comprises the steps: obtaining production scheduling data of the chemical fiber products, screening production scheduling features, calculating order priorities, and forming an order queue in positive order arrangement; a production scheduling constraint condition is set, an order allocation matrix and production resources are initialized, the production resources are allocated according to the order queue and the production scheduling constraint, the order tasks are traversed to update the production resources, the order tasks and the allocation matrix, and a preliminary production scheduling plan is generated; a reinforcement learning algorithm based on a Markov decision process is introduced, a total delay delivery value and a resource utilization rate are comprehensively considered, an initial production scheduling plan is optimized, and a final production scheduling plan is output, so that dynamic real-time control is further realized, a resource allocation strategy can be dynamically optimized, and environment change can be responded in real time; and the flexibility and effect of production scheduling of chemical fiber products are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of chemical fiber production management, and specifically to a method and system for intelligent production scheduling of chemical fiber products based on reinforcement learning. Background Art

[0002] Chemical fiber product scheduling refers to the process by which chemical fiber enterprises organize, plan, allocate and coordinate various tasks in the production process based on factors such as market demand, order status and production capacity. In this process, it is necessary to ensure that the product production plan is highly consistent with customer needs, effectively identify and foresee potential risks in production, and take corresponding countermeasures to ensure the stability, efficiency and quality of chemical fiber product production.

[0003] Traditional production scheduling methods usually adopt a plan locking approach, which means that once the production schedule is determined, the production schedule cannot be modified for a period of time in the future. Although this approach can ensure the stability of the plan to a certain extent, it also leads to problems such as low equipment utilization and poor responsiveness. Some existing solutions have introduced optimization algorithms in the production scheduling process, which greatly improves equipment utilization and can make adjustments according to changes in production tasks and resources. However, these existing solutions only focus on optimizing a single goal and it is difficult to consider the conflict and balance between multiple goals at the same time. On the other hand, many existing production scheduling solutions usually need to go through a lot of calculations and optimization processes again when making adjustments based on real-time monitoring data, resulting in a slow response speed of the production scheduling system and difficulty in quickly adapting to sudden changes in the production environment, so there are still certain limitations in terms of effect and flexibility.

[0004] To this end, an intelligent production scheduling method and system for chemical fiber products based on reinforcement learning is proposed. Summary of the invention

[0005] The purpose of the present invention is to provide a method and system for intelligent production scheduling of chemical fiber products based on reinforcement learning, which obtains production scheduling data of chemical fiber products, screens production scheduling features and calculates order priorities to form an order queue arranged in positive order; sets production scheduling constraints, initializes order allocation matrix and production resources, allocates production resources according to order queue and production scheduling constraints, traverses order tasks to update production resources, order tasks and allocation matrix, and generates a preliminary production schedule; introduces a reinforcement learning algorithm based on Markov decision process (MDP), comprehensively considers total delayed delivery value and resource utilization, optimizes the preliminary production schedule, outputs the final production schedule, further realizes dynamic real-time control, can dynamically optimize resource allocation strategy, respond to environmental changes in real time, and improves the flexibility and effect of chemical fiber product scheduling.

[0006] To achieve the above object, the present invention provides the following technical solutions: The first part is an intelligent production scheduling method for chemical fiber products based on reinforcement learning, including: Acquire production scheduling data of chemical fiber products, wherein the production scheduling data of chemical fiber products includes order tasks, production resources, historical production records and production environment; and screen and obtain production scheduling features according to the production scheduling data of chemical fiber products; Calculate order priorities according to the production scheduling characteristics, and arrange the order priorities in positive order to obtain an order queue; Setting a production scheduling constraint condition, wherein the production scheduling constraint condition is used to select the production resources; Initialize an order allocation matrix and the production resources, wherein the order allocation matrix is ​​used to allocate the production resources according to the order queue and the production scheduling constraints; traverse the order tasks, and update the production resources, the order tasks and the order allocation matrix to obtain a preliminary production scheduling plan; Establishing an MDP framework, the MDP framework is used to describe the process of optimizing the preliminary production scheduling plan by a reinforcement learning algorithm; the MDP framework includes a scheduling state, a scheduling action, and a scheduling feedback; the scheduling state includes the scheduling characteristics and the order queue, and the scheduling action includes allocating the production resources and adjusting the order queue; according to the scheduling state and the scheduling action, a total delayed delivery value and a resource utilization rate are calculated, and the total delayed delivery value and the resource utilization rate are weightedly summed to obtain the scheduling feedback; The preliminary production schedule is used as the initial production scheduling action, and the final production schedule is output using the reinforcement learning algorithm according to the MDP framework.

[0007] Furthermore, the production scheduling characteristics include order characteristics, production resource characteristics and time characteristics; the order characteristics include the remaining quantity of products, the remaining time for delivery, product specifications and order status; the production resource characteristics include the equipment status, equipment availability time and the equipment operating speed corresponding to the product specifications; the time characteristics include the current time and the total remaining time of the task; Among them, the order status includes a completed state and an uncompleted state; the equipment status includes an unoccupied state, an occupied state and a faulty state.

[0008] Furthermore, the calculation process of the order priority includes: ; in, For orders The order priority of For orders The remaining time for delivery, For orders The remaining quantity of the product, For orders The device running speed, For orders Product specifications, The order serial number.

[0009] Furthermore, the production scheduling constraints include: Device status constraint, used to filter the device status to be unoccupied; Equipment availability time constraint: if the equipment availability time is greater than 0, the equipment availability time is reduced according to the order allocation ratio and the equipment operation speed; if the equipment availability time is equal to 0, the equipment availability time is initialized to the maximum availability time after n hours; The order allocation ratio constraint is expressed as: ; in, For orders On the device The above order allocation ratio; Indicates the current moment, Indicates the time interval, Indicates the order allocation ratio at the next moment; Indicates the order allocation ratio at the current moment; Complete the order constraint, expressed as: ; in, is the total number of allocations.

[0010] Furthermore, the calculation process of the production scheduling feedback includes: Divide the remaining quantity of the product by the order allocation ratio and the equipment operating speed to get the actual production time of the order; Calculate the difference between the actual production time of the order and the remaining delivery time to obtain a delayed delivery value; if the delayed delivery value is a negative value, record the delayed delivery value as zero; Accumulating all the delayed delivery values ​​to obtain the total delayed delivery value; According to the production scheduling characteristics, the ratio of the actual use time of all equipment to the available time of the equipment is calculated to obtain the resource utilization rate; The total delayed delivery value and the resource utilization rate are normalized and weightedly summed to obtain the production scheduling feedback.

[0011] The second part is an intelligent production scheduling system for chemical fiber products based on reinforcement learning, including: A data acquisition module is used to acquire production scheduling data of chemical fiber products; and to screen and obtain production scheduling features according to the production scheduling data of chemical fiber products; A queue setting module calculates order priorities according to the production scheduling characteristics, and arranges the order priorities in positive order to obtain an order queue; A constraint setting module is used to set production scheduling constraints, where the production scheduling constraints are used to select production resources; A preliminary production scheduling plan generation module initializes an order allocation matrix and the production resources, wherein the order allocation matrix is ​​used to allocate the production resources according to the order queue and the production scheduling constraints; traverses order tasks, and updates the production resources, the order tasks and the order allocation matrix to obtain a preliminary production scheduling plan; The final production schedule generation module establishes an MDP framework, which is used to describe the process of optimizing the preliminary production schedule through a reinforcement learning algorithm; the MDP framework includes a scheduling state, a scheduling action, and a scheduling feedback; the scheduling state includes the scheduling characteristics and the order queue, and the scheduling action includes allocating the production resources and adjusting the order queue; according to the scheduling state and the scheduling action, the total delayed delivery value and the resource utilization are calculated, and the total delayed delivery value and the resource utilization are weightedly summed to obtain the scheduling feedback; the preliminary production schedule is used as the initial scheduling action, and the final production schedule is output using the reinforcement learning algorithm according to the MDP framework.

[0012] Furthermore, the production scheduling characteristics include order characteristics, production resource characteristics and time characteristics; the order characteristics include the remaining quantity of products, the remaining time for delivery, product specifications and order status; the production resource characteristics include the equipment status, equipment availability time and the equipment operating speed corresponding to the product specifications; the time characteristics include the current time and the total remaining time of the task; Among them, the order status includes a completed state and an uncompleted state; the equipment status includes an unoccupied state, an occupied state and a faulty state.

[0013] Furthermore, the calculation process of the order priority includes: ; in, For orders The order priority of For orders The remaining time for delivery, For orders The remaining quantity of the product, For orders The device running speed, For orders Product specifications, The order serial number.

[0014] Furthermore, the production scheduling constraints include: Device status constraint, used to filter the device status to be unoccupied; Equipment availability time constraint: if the equipment availability time is greater than 0, the equipment availability time is reduced according to the order allocation ratio and the equipment operation speed; if the equipment availability time is equal to 0, the equipment availability time is initialized to the maximum availability time after n hours; The order allocation ratio constraint is expressed as: ; in, For orders On the device The above order allocation ratio; Indicates the current moment, Indicates the time interval, Indicates the order allocation ratio at the next moment; Indicates the order allocation ratio at the current moment; Complete the order constraint, expressed as: ; in, is the total number of allocations.

[0015] Furthermore, the calculation process of the production scheduling feedback includes: Divide the remaining quantity of the product by the order allocation ratio and the equipment operating speed to get the actual production time of the order; Calculate the difference between the actual production time of the order and the remaining delivery time to obtain a delayed delivery value; if the delayed delivery value is a negative value, record the delayed delivery value as zero; Accumulating all the delayed delivery values ​​to obtain the total delayed delivery value; According to the production scheduling characteristics, the ratio of the actual use time of all equipment to the available time of the equipment is calculated to obtain the resource utilization rate; The total delayed delivery value and the resource utilization rate are normalized and weightedly summed to obtain the production scheduling feedback.

[0016] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention analyzes the production scheduling data of chemical fiber products, screens out the production scheduling characteristics, and calculates the order priority based on factors such as the remaining delivery time, the remaining number of products, and the equipment running speed, and arranges the order priorities in positive order to form an order queue, which can quickly determine the order of importance of each order, reasonably arrange production tasks, and improve the effect of chemical fiber product production scheduling. At the same time, production scheduling constraints are set, including equipment status constraints, equipment available time constraints, and order allocation ratio constraints, and unoccupied equipment is dynamically screened to ensure the continuity of production, thereby improving the flexibility and effect of chemical fiber product production scheduling.

[0017] 2. The present invention describes the production scheduling process as a reinforcement learning optimization process by constructing an MDP framework. The MDP framework includes production scheduling status, production scheduling actions and production scheduling feedback. The production scheduling status includes order characteristics, production resource characteristics and time characteristics. The production scheduling actions include allocating production resources and dynamically adjusting the order queue. The production scheduling feedback is a weighted evaluation based on the delayed delivery value and resource utilization. The MDP framework can effectively describe complex factors such as resource changes, equipment failures and order demand uncertainty, ensuring that the production scheduling plan can still be efficiently executed under a variety of constraints, thereby improving the flexibility and effectiveness of chemical fiber product scheduling.

[0018] 3. The present invention uses the preliminary production schedule as the initial decision input, which effectively speeds up the early optimization speed and reduces resource waste. Combined with the reinforcement learning algorithm of the MDP framework, it can output the optimized production schedule based on the real-time changes in the production schedule status, delayed delivery values ​​and resource utilization feedback. The present invention realizes iterative optimization and dynamic adaptation of the production schedule strategy by continuously adjusting the order allocation and resource scheduling scheme, which can minimize the risk of delayed order delivery, while improving the overall utilization of resources, and flexibly responding to emergencies such as insertion of orders, withdrawal of orders and equipment damage, thereby improving the flexibility and effectiveness of chemical fiber product scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of a process flow of a method for intelligent production scheduling of chemical fiber products based on reinforcement learning provided by an embodiment of the present invention; Figure 2 A schematic diagram of a process for screening production scheduling features provided by an embodiment of the present invention; Figure 3 A schematic diagram of the structure of the MDP framework provided by an embodiment of the present invention; Figure 4 A schematic diagram of the structure of an intelligent production scheduling system for chemical fiber products based on reinforcement learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] See also Figures 1 to 4 The present invention provides a method and system for intelligent production scheduling of chemical fiber products based on reinforcement learning, and the technical solution is as follows: In the chemical fiber manufacturing industry, production scheduling is a key link to ensure production efficiency and on-time delivery. Existing production scheduling plans are committed to improving the stability, efficiency and quality of chemical fiber product production. However, when equipment fails, materials are in short supply or order demand fluctuates, some of them are based on static rule algorithms and cannot quickly reallocate resources, which can easily lead to delivery delays. At the same time, although some production scheduling plans attempt to introduce optimization algorithms, they do not fully consider key factors that affect corporate profits, such as energy consumption and inventory, and the optimization effect is limited. To solve the above problems, it is necessary to introduce a new intelligent production scheduling method and system for chemical fiber products to dynamically optimize resource allocation strategies and improve the flexibility and effectiveness of chemical fiber product scheduling.

[0022] Embodiment 1 is as follows: like Figure 1 As shown, a method for intelligent production scheduling of chemical fiber products based on reinforcement learning includes: Reference Figure 1 S10 in which: obtaining production scheduling data of chemical fiber products, wherein the production scheduling data of chemical fiber products includes order tasks, production resources, historical production records and production environment; and screening and obtaining production scheduling features according to the production scheduling data of chemical fiber products; The process of obtaining production scheduling data for chemical fiber products specifically includes: The order tasks are automatically exported through the enterprise resource planning (ERP) system, and the order tasks include the product quantity, the delivery deadline and the product specifications. The equipment information is monitored and recorded in real time through the production management system (MES), the chemical fiber material information is queried through the warehouse management system (WMS), and the personnel information is queried through the personnel management system. The production resources are combined with this information to obtain the production resources, which include the number of personnel, personnel hours, equipment operating speed, equipment available time, chemical fiber material inventory and unit chemical fiber material demand. The historical production records are obtained through the production log, and the historical production records include the historical equipment failure probability, historical personnel efficiency, historical material demand and historical production time. The production environment is obtained through environmental monitoring sensors, and the production environment includes temperature and humidity.

[0023] Furthermore, the production scheduling characteristics include order characteristics, production resource characteristics and time characteristics; the order characteristics include the remaining quantity of products, the remaining time for delivery, product specifications and order status; the production resource characteristics include the equipment status, equipment availability time and the equipment operating speed corresponding to the product specifications; the time characteristics include the current time and the total remaining time of the task; Among them, the order status includes a completed state and an uncompleted state; the equipment status includes an unoccupied state, an occupied state and a faulty state.

[0024] Figure 2 A schematic diagram of a process for screening production scheduling features provided in an embodiment of the present invention.

[0025] Specifically, Figure 2 As shown, first, through ERP, MES, WMS and personnel management systems, the order data, production resource data, historical production record data and environmental data in each system are integrated to break the information island and realize data circulation and sharing. Then, these data are deduplicated, standardized and screened to obtain the processed chemical fiber product production scheduling data, which can improve the integrity and accuracy of the data. Then, based on the current time, the remaining processing volume and delivery remaining time of the order are dynamically calculated, and the remaining time of all order tasks is summarized to obtain the total task remaining time. Since the data such as the number of personnel, personnel hours, chemical fiber material inventory and unit chemical fiber material demand are relatively stable in this embodiment, they are no longer considered as key variables in the optimization process. Finally, according to the key production scheduling features screened, the production scheduling feature vector is integrated to provide data support for the subsequent optimization of the production scheduling strategy, thereby improving the flexibility and effect of chemical fiber product scheduling.

[0026] Reference Figure 1 S20: Calculate order priorities according to the production scheduling characteristics, and arrange the order priorities in positive order to obtain an order queue.

[0027] Furthermore, the order priority is expressed as: ; in, For orders The order priority of For orders The remaining time for delivery, For orders The remaining quantity of the product, For orders The device running speed, For orders Product specifications, The order serial number.

[0028] Specifically, assume that there are scheduling characteristics as shown in Table 1. Table 1 gives three orders, each of which includes four scheduling characteristics. Among them, according to the number of products and the equipment running speed, the order completion time is calculated to be 10, 25 and 30 hours respectively. The remaining delivery time is divided by the order completion time to obtain the order priority, which is 5, 4 and 5 respectively. If the order priorities are the same, they are sorted in descending order according to the remaining delivery time to obtain the order queue. By placing higher priority orders first, the risk of delayed delivery can be reduced and high priority orders can be completed within a limited time. In addition, sorting by order priority can also reasonably schedule production resources, optimize the use of equipment, and improve resource utilization in the production process, thereby improving the flexibility and effectiveness of chemical fiber product scheduling.

[0029] Table 1 Example of production scheduling characteristics

[0030] Reference Figure 1 S30: setting a production scheduling constraint condition, wherein the production scheduling constraint condition is used to select the production resources; Among them, production scheduling constraints refer to specific restrictions on equipment, personnel, time, etc. in the production process. The purpose of setting them is to fit the actual production scheduling process of chemical fiber products, and to ensure that production can proceed smoothly and resources can be reasonably utilized, so as to improve the flexibility and effectiveness of chemical fiber product scheduling.

[0031] Furthermore, the production scheduling constraints include: Device status constraint, used to filter the device status to be unoccupied; Assume that during the production scheduling of chemical fiber products, there are two devices in the workshop, namely device A and device B, both of which can be used for the next order task. Device A is currently producing another order, while device B is idle. According to the device status constraint, only device B can be selected for production scheduling, which avoids idle equipment or repeated scheduling, ensures that order tasks can be carried out on time, and thus improves the production scheduling effect.

[0032] Equipment availability time constraint: if the equipment availability time is greater than 0, the equipment availability time is reduced according to the order allocation ratio and the equipment operation speed; if the equipment availability time is equal to 0, the equipment availability time is initialized to the maximum availability time after n hours, expressed as: ; in, For equipment Equipment availability time, For orders On the device The order allocation ratio on For orders The device running speed, For equipment The maximum device availability time, For the current moment, Indicates the current time Hours later.

[0033] In this embodiment, Set to 4 hours. By setting the equipment available time constraint, it can ensure that each task is executed within the time range that the equipment can withstand, avoid the problem of over-scheduling or equipment overload, and ensure that the equipment is fully utilized within its available time. Assume that in the production scheduling of chemical fiber products, the maximum available time of equipment A is 24 hours. For order 1, the number of products is 100 pieces, and the equipment runs at a speed of 10 pieces / hour. It takes 10 hours to process order 1. After 10 hours, the available time of equipment A is 14 hours. For order 2, the number of products is 200 pieces, and the equipment runs at a speed of 10 pieces / hour. It takes 20 hours to process order 2. Since the available time of equipment A is insufficient, only 140 pieces are allocated to use equipment A. After 14 hours, the available time of equipment A is 0 hours. After waiting for 4 hours, it will continue to recover to the maximum available time.

[0034] The order allocation ratio constraint is expressed as: ; in, For orders On the device The above order allocation ratio; Indicates the current moment, Indicates the time interval, Indicates the order allocation ratio at the next moment; Indicates the order allocation ratio at the current moment; The allocation ratio represents the proportion of the order on the device, that is, the ratio of the number of products currently allocated on the device to the total number of products. Each time a task is allocated, the order allocation ratio on the device will decrease accordingly, which reflects the progress of the task execution and will not exceed the processing capacity or resource limit of the device.

[0035] also, Also meets: ; in, For orders The equipment operating speed, that is, the time required after the order is allocated in proportion does not exceed the equipment available time.

[0036] Complete the order constraint, expressed as: ; in, is the total number of allocations, is the device serial number. The sum of the allocation ratios is 1, which ensures that each order will only be completed once, and no tasks will be allocated to other devices when the order is completed. After the order is completed, the order status changes to completed.

[0037] Specifically, in this embodiment, the operating parameters of all devices are the same, that is, all devices have no difference in performance, processing capacity and work efficiency. Therefore, in the process of scheduling production of chemical fiber products, there is no need to make special distinctions or settings for different devices. Through the constraints of equipment status and equipment available time, it is ensured that production tasks can be carried out on time and the utilization rate of equipment is improved. Through the constraints of order allocation ratio, the phenomenon of equipment failure or shutdown due to excessive scheduling or excessive load is reduced. By completing the order constraints, duplicate order allocation and task omissions are avoided, thereby improving the flexibility and effect of chemical fiber product scheduling and ensuring the efficiency and stability of chemical fiber product production.

[0038] Reference Figure 1 S40: Initialize the order allocation matrix and the production resources, the order allocation matrix is ​​used to allocate the production resources according to the order queue and the production scheduling constraint conditions; traverse the order tasks, and update the production resources, the order tasks and the order allocation matrix to obtain a preliminary production scheduling plan; Specifically, suppose there is an order queue containing 2 orders and 2 devices: Order 1, 100 pieces, the equipment speed corresponding to the product specification is 10 pieces / hour; Order 2, 50 pieces, the equipment speed corresponding to the product specification is 5 pieces / hour; Device 1 and Device 2, Device 1 has 20 hours remaining, and Device 2 has 15 hours remaining.

[0039] Initialize the order allocation matrix, all the values ​​in the matrix are 0. According to the production scheduling constraints, the high priority order 1 is assigned to the device with the highest remaining time, then order 1 is assigned to device 1, and order 2 is assigned to device 2. After the update, the remaining time of device 1 is 10 hours, and the remaining time of device 2 is 5 hours. Because the remaining time of the device satisfies the complete order 1 and order 2, the status of order 1 and order 2 are both changed to completed. The updated order allocation matrix is ​​expressed as: ; in, It is an order allocation matrix, that is, order 1 is completely allocated to equipment 1, and order 2 is completely allocated to equipment 2. The matrix can track the allocation of each order, thereby ensuring the reasonable execution of the production schedule and improving the flexibility and effect of chemical fiber product scheduling.

[0040] Reference Figure 1 S50 in: establishing an MDP framework, the MDP framework is used to describe the process of optimizing the preliminary production scheduling plan by a reinforcement learning algorithm; the MDP framework includes a scheduling state, a scheduling action and a scheduling feedback; the scheduling state includes the scheduling characteristics and the order queue, and the scheduling action includes allocating the production resources and adjusting the order queue; according to the scheduling state and the scheduling action, a total delayed delivery value and a resource utilization rate are calculated, and the total delayed delivery value and the resource utilization rate are weightedly summed to obtain the scheduling feedback; Among them, the MDP framework is a Markov decision process framework, which can model decision-making problems in an uncertain scheduling environment.

[0041] Figure 3 A schematic diagram of the structure of the MDP framework provided by an embodiment of the present invention.

[0042] like Figure 3 As shown in the figure, the MDP framework includes three core elements: state (scheduling state), action (scheduling action) and reward function (scheduling feedback). The scheduling environment is the actual environment that interacts with the reinforcement learning algorithm, including equipment, raw materials, and tasks. It will generate a scheduling state based on the current production conditions and tell the reinforcement learning algorithm the current situation of the chemical fiber factory. The scheduling state is a specific description of the current scheduling environment and is the basis for the reinforcement learning algorithm to make decisions. The scheduling action is the behavior taken by the reinforcement learning algorithm under the current scheduling state, for example, choosing to assign an order to a certain device. Scheduling feedback is a criterion for evaluating the quality of the current scheduling action. For example, delayed delivery may result in a negative reward.

[0043] The process of optimizing the preliminary production schedule using the reinforcement learning algorithm can be expressed as: S501: The reinforcement learning algorithm observes the current production scheduling status; S502: Select the optimal production scheduling action according to the preliminary production scheduling plan, including allocating equipment resources and adjusting the order queue; S503: Execute the production scheduling action and transfer from the current production scheduling state to a new production scheduling state; S504: Calculate the production scheduling feedback brought by the current action, and evaluate the pros and cons of the action based on the delayed delivery value and resource utilization.

[0044] S505: Adjust the production scheduling action according to the production scheduling feedback, thereby updating the preliminary production scheduling plan.

[0045] By repeatedly executing S501 to S505, the reinforcement learning algorithm will find the optimal production scheduling strategy according to the dynamically changing production scheduling status, thereby improving the flexibility and effect of chemical fiber product scheduling. In addition, the reinforcement learning algorithm also follows the production scheduling constraints in the process of optimizing the preliminary production scheduling plan.

[0046] Furthermore, the calculation process of the production scheduling feedback includes: Divide the remaining quantity of the product by the order allocation ratio and the equipment running speed to get the actual production time of the order, expressed as: ; in, For orders The actual production time of the order, For orders The remaining quantity of the product, For orders On the device The order allocation ratio on For orders The device running speed, Waiting time for the device.

[0047] The difference between the actual production time of the order and the remaining delivery time is calculated to obtain the delayed delivery value; if the delayed delivery value is a negative value, the delayed delivery value is recorded as zero, which is expressed as: ; in, For orders The delayed delivery value, For orders The remaining time for delivery, For orders The actual production time of the order, Select a function for Maximum.

[0048] All the delayed delivery values ​​are accumulated to obtain the total delayed delivery value, which is expressed as: ; in, is the total delayed delivery value, is the total number of orders, The order serial number.

[0049] According to the production scheduling characteristics, the ratio of the actual use time of all equipment to the available time of the equipment is calculated to obtain the resource utilization rate, which is expressed as: ; in, is the resource utilization rate, is the actual usage time of device j, is the device availability time of device j, is the total number of devices.

[0050] The total delayed delivery value and the resource utilization rate are normalized and weighted summed to obtain the production scheduling feedback, which is expressed as: ; in, For production scheduling feedback, and is the weight factor.

[0051] Specifically, in this embodiment, and They are set to 0.6 and 0.4 respectively. Assuming that in order 1, the remaining number of chemical fiber products is 100 pieces, the remaining order allocation ratio is 0.5, the equipment running speed is 10 pieces / hour, the remaining delivery time is 12 hours, and the maximum available time of the equipment is 24 hours. Without waiting for the equipment to start, the total delayed delivery value is calculated to be 8, and the resource utilization rate is 0.833. The production scheduling feedback comprehensively considers the total delayed delivery value and resource utilization rate. Based on the production scheduling feedback, it can help the reinforcement learning algorithm to dynamically adjust the production scheduling strategy, promote the full use of production equipment and reduce the delivery delay of orders, thereby improving the flexibility and effect of chemical fiber product scheduling.

[0052] Reference Figure 1 S60: taking the preliminary production schedule as the initial production scheduling action, and outputting a final production schedule using the reinforcement learning algorithm according to the MDP framework.

[0053] Among them, in this embodiment, the reinforcement learning algorithm selects the Deep Q-Network (DQN) algorithm. Because DQN can process high-dimensional state space, it is suitable for the optimization of production scheduling of chemical fiber products with much more complex equipment and multiple orders. And DQN can optimize the delayed delivery value and resource utilization by maximizing the reward function (production scheduling feedback), thereby improving the flexibility and effect of chemical fiber product scheduling.

[0054] Specifically, in this embodiment, the parameters of DQN include a discount factor of 0.95 and a learning rate of 0.001. For 500 chemical fiber products, 5 devices. Among them, these 500 orders are not dispatched at the same time, but randomly dispatched, which is used to determine whether the present invention can respond to changes in order tasks. The preliminary production schedule is compared with the final production schedule. As shown in Table 2, the experimental results of the preliminary production schedule and the final production schedule are given. It can be seen that through the reinforcement learning algorithm, the delayed delivery value and resource utilization are optimized as a whole, and it can quickly respond to problems such as adding orders, thereby improving the flexibility and effect of chemical fiber product scheduling.

[0055] Table 2 Comparison between the preliminary production schedule and the final production schedule

[0056] In summary, the present invention optimizes the ordering of orders and the order of resource allocation by screening production scheduling features and calculating order priorities, ensuring that high-priority orders can be produced in a timely manner, and improving the overall production scheduling effect. Then, according to the actual production scheduling environment of the actual chemical fiber products, the corresponding constraints are set to generate a preliminary production schedule, which can help the reinforcement learning algorithm find the final strategy faster, thereby improving the flexibility and effect of chemical fiber product scheduling. On the basis of preliminary production scheduling, the MDP framework is used to design the production scheduling status, production scheduling actions and production scheduling feedback. The total delayed delivery value and resource utilization rate are introduced in the production scheduling feedback, and weighted summation is performed, so that the optimization goal comprehensively considers the timeliness of order delivery and the efficient use of resources, effectively reducing the risk of delayed order delivery and improving customer satisfaction. Through the continuous iterative optimization of the reinforcement learning algorithm, the resource allocation and order queue are dynamically adjusted to adapt to the changes in the production environment and improve the flexibility and effect of chemical fiber product scheduling.

[0057] Embodiment 2 is as follows: A small chemical fiber product manufacturing plant produces a wide variety of chemical fiber materials, covering textile fibers of different specifications, materials and uses. Due to the instability of market demand, the types, quantities and delivery times of orders often change, resulting in frequent adjustments to production scheduling. In addition, during the scheduling process, the factory often faces problems of idle resources and stagnant production lines, which further delay the overall production progress. In order to solve these problems, the factory urgently needs to introduce a more flexible and efficient scheduling method.

[0058] like Figure 4 As shown, an intelligent production scheduling system for chemical fiber products based on reinforcement learning includes: Reference Figure 4 The data acquisition module in the embodiment acquires the production scheduling data of chemical fiber products; and according to the production scheduling data of chemical fiber products, obtains the production scheduling characteristics by screening; Reference Figure 4 The queue setting module in the embodiment calculates order priorities according to the production scheduling characteristics, and arranges the order priorities in positive order to obtain an order queue; Reference Figure 4 The constraint setting module in the production scheduling module is used to set production scheduling constraints, and the production scheduling constraints are used to select production resources; Reference Figure 4The preliminary production scheduling plan generation module in the initialization order allocation matrix and the production resources, the order allocation matrix is ​​used to allocate the production resources according to the order queue and the production scheduling constraints; traverse the order tasks, and update the production resources, the order tasks and the order allocation matrix to obtain a preliminary production scheduling plan; Reference Figure 4 A final production scheduling generation module is provided in the embodiment of the present invention, and an MDP framework is established, wherein the MDP framework is used to describe the process of optimizing the preliminary production scheduling by a reinforcement learning algorithm; the MDP framework includes a scheduling state, a scheduling action and a scheduling feedback; the scheduling state includes the scheduling characteristics and the order queue, and the scheduling action includes allocating the production resources and adjusting the order queue; according to the scheduling state and the scheduling action, a total delayed delivery value and a resource utilization rate are calculated, and the total delayed delivery value and the resource utilization rate are weightedly summed to obtain the scheduling feedback; the preliminary production scheduling plan is used as the initial scheduling action, and the reinforcement learning algorithm is used according to the MDP framework to output a final production scheduling plan.

[0059] Furthermore, the production scheduling characteristics include order characteristics, production resource characteristics and time characteristics; the order characteristics include the remaining quantity of products, the remaining time for delivery, product specifications and order status; the production resource characteristics include the equipment status, equipment availability time and the equipment operating speed corresponding to the product specifications; the time characteristics include the current time and the total remaining time of the task; Among them, the order status includes a completed state and an uncompleted state; the equipment status includes an unoccupied state, an occupied state and a faulty state.

[0060] Among them, equipment can refer to a group of equipment, including equipment required for chemical fiber products.

[0061] Specifically, in this embodiment, the product of the unit chemical fiber material quantity and the product quantity is calculated to obtain the material demand, and the future material demand is output using the prediction model based on the historical production records and the production environment. If the material demand or the future material demand is less than the chemical fiber material inventory, a reminder is given to replenish the material, thereby improving the flexibility and effect of chemical fiber product scheduling. Among them, the prediction model can be an LSTM model and an SVM model.

[0062] Furthermore, the calculation process of the order priority includes: ; in, For orders The order priority of For orders The remaining time for delivery, For orders The remaining quantity of the product, For orders The device running speed, For orders Product specifications, The order serial number.

[0063] Furthermore, the production scheduling constraints include: Device status constraint, used to filter the device status to be unoccupied; Equipment availability time constraint: if the equipment availability time is greater than 0, the equipment availability time is reduced according to the order allocation ratio and the equipment operation speed; if the equipment availability time is equal to 0, the equipment availability time is initialized to the maximum availability time after n hours; The order allocation ratio constraint is expressed as: ; in, For orders On the device The above order allocation ratio; Indicates the current moment, Indicates the time interval, Indicates the order allocation ratio at the next moment; Indicates the order allocation ratio at the current moment; Complete the order constraint, expressed as: ; in, is the total number of allocations, Is the device serial number.

[0064] Specifically, other production resource constraints can also be introduced, such as blanks, outsourced parts, tools and personnel, etc. Through capacity verification, resource completeness inspection and shift schedule setting, the adequacy and reasonable allocation of production resources can be ensured.

[0065] Furthermore, the calculation process of the production scheduling feedback includes: Divide the remaining quantity of the product by the order allocation ratio and the equipment operating speed to get the actual production time of the order; Calculate the difference between the actual production time of the order and the remaining delivery time to obtain a delayed delivery value; if the delayed delivery value is a negative value, record the delayed delivery value as zero; Accumulating all the delayed delivery values ​​to obtain the total delayed delivery value; According to the production scheduling characteristics, the ratio of the actual use time of all equipment to the available time of the equipment is calculated to obtain the resource utilization rate; The total delayed delivery value and the resource utilization rate are normalized and weightedly summed to obtain the production scheduling feedback.

[0066] Table 3 Convergence speed comparison

[0067] Specifically, in this embodiment, the reinforcement learning algorithm also selects the DQN algorithm, and inputs it into the Q-learning algorithm according to the obtained preliminary production schedule in the same implementation environment of a certain factory as comparison plan one. The DQN algorithm that does not use the preliminary production schedule is used as comparison plan two, and is compared with the original system of a certain factory. The comparison results are shown in Tables 3 and 4. Table 3 gives the convergence speed results of different comparison schemes, and Table 4 gives the production scheduling results of different comparison schemes. It can be seen that the present invention optimizes production scheduling through reinforcement learning and realizes algorithm convergence first, which means that the optimal production scheduling strategy can be found the fastest. In addition, the present invention achieves the highest resource utilization and improves the factory scheduling effect. In the production scheduling process, delayed delivery is effectively reduced, and the flexibility of chemical fiber product scheduling is improved.

[0068] Table 4 Comparison of production scheduling results

[0069] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for intelligent production scheduling of chemical fiber products based on reinforcement learning, characterized in that: include: Acquire production scheduling data of chemical fiber products, wherein the production scheduling data of chemical fiber products includes order tasks, production resources, historical production records and production environment; and screen and obtain production scheduling features according to the production scheduling data of chemical fiber products; Calculate order priorities according to the production scheduling characteristics, and arrange the order priorities in positive order to obtain an order queue; Setting a production scheduling constraint condition, wherein the production scheduling constraint condition is used to select the production resources; Initializing an order allocation matrix and the production resources, wherein the order allocation matrix is ​​used to allocate the production resources according to the order queue and the production scheduling constraint condition; Traversing the order tasks, and updating the production resources, the order tasks and the order allocation matrix to obtain a preliminary production schedule; Establishing an MDP framework, the MDP framework is used to describe the process of optimizing the preliminary production scheduling plan by a reinforcement learning algorithm; the MDP framework includes a scheduling state, a scheduling action, and a scheduling feedback; the scheduling state includes the scheduling characteristics and the order queue, and the scheduling action includes allocating the production resources and adjusting the order queue; According to the production scheduling status and the production scheduling action, a total delayed delivery value and a resource utilization rate are calculated, and a weighted sum of the total delayed delivery value and the resource utilization rate is performed to obtain the production scheduling feedback; The preliminary production schedule is used as the initial production scheduling action, and the final production schedule is output using the reinforcement learning algorithm according to the MDP framework.

2. According to claim 1, a method for intelligent production scheduling of chemical fiber products based on reinforcement learning is characterized in that: The production scheduling features include order features, production resource features and time features; the order features include the remaining quantity of products, the remaining time for delivery, product specifications and order status; the production resource features include the equipment status, equipment availability time and the equipment operating speed corresponding to the product specifications; the time features include the current time and the total remaining time of the task; Among them, the order status includes a completed state and an uncompleted state; the equipment status includes an unoccupied state, an occupied state and a faulty state.

3. The intelligent production scheduling method for chemical fiber products based on reinforcement learning according to claim 1 is characterized in that: The calculation process of the order priority includes: ； in, For orders The order priority of For orders The remaining time for delivery, For orders The remaining quantity of the product, For orders The device running speed, For orders Product specifications, The order serial number.

4. The intelligent production scheduling method for chemical fiber products based on reinforcement learning according to claim 1 is characterized in that: The production scheduling constraints include: Device status constraint, used to filter the device status to be unoccupied; Equipment availability time constraint: if the equipment availability time is greater than 0, the equipment availability time is reduced according to the order allocation ratio and the equipment operation speed; if the equipment availability time is equal to 0, the equipment availability time is initialized to the maximum availability time after n hours; The order allocation ratio constraint is expressed as: ； in, For orders On the device The above order allocation ratio; Indicates the current moment, Indicates the time interval, Indicates the order allocation ratio at the next moment; Indicates the order allocation ratio at the current moment; Complete the order constraint, expressed as: ； in, is the total number of allocations.

5. The intelligent production scheduling method for chemical fiber products based on reinforcement learning according to claim 1 is characterized in that: The calculation process of the production scheduling feedback includes: Divide the remaining quantity of the product by the order allocation ratio and the equipment operating speed to get the actual production time of the order; Calculate the difference between the actual production time of the order and the remaining delivery time to obtain a delayed delivery value; if the delayed delivery value is a negative value, record the delayed delivery value as zero; Accumulating all the delayed delivery values ​​to obtain the total delayed delivery value; According to the production scheduling characteristics, the ratio of the actual use time of all equipment to the available time of the equipment is calculated to obtain the resource utilization rate; The total delayed delivery value and the resource utilization rate are normalized and weightedly summed to obtain the production scheduling feedback.

6. An intelligent production scheduling system for chemical fiber products based on reinforcement learning, characterized in that: include: Data acquisition module, to obtain production scheduling data of chemical fiber products; According to the production scheduling data of the chemical fiber products, the production scheduling characteristics are screened and obtained; A queue setting module calculates order priorities according to the production scheduling characteristics, and arranges the order priorities in positive order to obtain an order queue; A constraint setting module is used to set production scheduling constraints, where the production scheduling constraints are used to select production resources; A preliminary production scheduling generation module is used to initialize an order allocation matrix and the production resources, wherein the order allocation matrix is ​​used to allocate the production resources according to the order queue and the production scheduling constraints; Traversing the order tasks, and updating the production resources, the order tasks and the order allocation matrix to obtain a preliminary production schedule; The final production scheduling generation module establishes an MDP framework, wherein the MDP framework is used to describe the process of optimizing the preliminary production scheduling by a reinforcement learning algorithm; the MDP framework includes a scheduling state, a scheduling action, and a scheduling feedback; the scheduling state includes the scheduling characteristics and the order queue, and the scheduling action includes allocating the production resources and adjusting the order queue; According to the production scheduling status and the production scheduling action, a total delayed delivery value and a resource utilization rate are calculated, and a weighted sum of the total delayed delivery value and the resource utilization rate is performed to obtain the production scheduling feedback; The preliminary production schedule is used as the initial production scheduling action, and the final production schedule is output using the reinforcement learning algorithm according to the MDP framework.

7. The intelligent production scheduling system for chemical fiber products based on reinforcement learning according to claim 6 is characterized in that: The production scheduling features include order features, production resource features and time features; the order features include the remaining quantity of products, the remaining time for delivery, product specifications and order status; the production resource features include the equipment status, equipment availability time and the equipment operating speed corresponding to the product specifications; the time features include the current time and the total remaining time of the task; Among them, the order status includes a completed state and an uncompleted state; the equipment status includes an unoccupied state, an occupied state and a faulty state.

8. The intelligent production scheduling system for chemical fiber products based on reinforcement learning according to claim 6 is characterized in that: The calculation process of the order priority includes: ； in, For orders The order priority of For orders The remaining time for delivery, For orders The remaining quantity of the product, For orders The device running speed, For orders Product specifications, The order serial number.

9. The intelligent production scheduling system for chemical fiber products based on reinforcement learning according to claim 6 is characterized in that: The production scheduling constraints include: Device status constraint, used to filter the device status to be unoccupied; Equipment availability time constraint: if the equipment availability time is greater than 0, the equipment availability time is reduced according to the order allocation ratio and the equipment operation speed; if the equipment availability time is equal to 0, the equipment availability time is initialized to the maximum availability time after n hours; The order allocation ratio constraint is expressed as: ； in, For orders On the device The above order allocation ratio; Indicates the current moment, Indicates the time interval, Indicates the order allocation ratio at the next moment; Indicates the order allocation ratio at the current moment; Complete the order constraint, expressed as: ； in, is the total number of allocations.

10. The intelligent production scheduling system for chemical fiber products based on reinforcement learning according to claim 6, characterized in that: The calculation process of the production scheduling feedback includes: Divide the remaining quantity of the product by the order allocation ratio and the equipment operating speed to get the actual production time of the order; Calculate the difference between the actual production time of the order and the remaining delivery time to obtain a delayed delivery value; if the delayed delivery value is a negative value, record the delayed delivery value as zero; Accumulating all the delayed delivery values ​​to obtain the total delayed delivery value; According to the production scheduling characteristics, the ratio of the actual use time of all equipment to the available time of the equipment is calculated to obtain the resource utilization rate; The total delayed delivery value and the resource utilization rate are normalized and weightedly summed to obtain the production scheduling feedback.

Images