Factory dynamic production scheduling optimization method and system based on deep reinforcement learning

Abstract

The invention provides a factory dynamic production scheduling optimization method and system based on deep reinforcement learning, and the method comprises the steps: firstly obtaining a real-time production monitoring data set of a target factory, including a plurality of production link monitoring sequences which are composed of parameters such as an equipment operation state, an order execution progress and environment interference; extracting features of the real-time production monitoring data set to obtain a target production feature set containing equipment state dynamic and other features, calling a pre-trained deep reinforcement learning strategy network, performing dynamic constraint matching processing to generate dynamic production constraint features, and performing iterative optimization on the basis of the dynamic production constraint features and a preset production scheduling optimization objective function strategy to obtain a real-time production monitoring data set; and generating a real-time production scheduling optimization strategy set containing sequences such as equipment scheduling priorities and the like, finally inputting the real-time production scheduling optimization strategy set into a production scheduling control system, triggering an instruction dynamic adjustment operation, updating deep reinforcement learning strategy network parameters according to feedback data, and realizing dynamic production scheduling optimization of the factory.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning, and in particular, to a method and system for optimizing factory dynamic production scheduling based on deep reinforcement learning. Background Art

[0002] In the modern industrial production system, factory production scheduling optimization has always been a key link to improve production efficiency, reduce costs, and enhance the competitiveness of enterprises. Traditional factory production scheduling methods mainly rely on manual experience or simple rule algorithms. When facing complex and changing production environments, these methods expose many insurmountable limitations.

[0003] On the one hand, traditional production scheduling methods often rely on static production plans and preset rules, and cannot obtain and process dynamic information in the factory production process in real time. In actual production, the operating status of equipment changes continuously over time, and there may be situations such as failures and efficiency fluctuations; the progress of order execution is also affected by various factors, such as delays in raw material supply and changes in customer demands; at the same time, there are also interference factors in the production environment such as temperature, humidity, and power supply. However, traditional methods are difficult to effectively integrate these multi-dimensional information that changes in real time, resulting in the disconnection between the production scheduling plan and the actual production situation, and unable to make timely adaptive adjustments, thus affecting production efficiency and the timeliness of order delivery.

[0004] On the other hand, existing production scheduling optimization technologies based on simple algorithms or models are very limited in dealing with complex production constraints and multi-objective optimization problems. These technologies usually can only consider single or a few optimization objectives, such as equipment utilization rate or order delivery time, and it is difficult to achieve global optimality in multiple dimensions such as equipment scheduling, order allocation, and resource adjustment. Moreover, they lack the ability to perceive and match dynamic production constraints in real time, and cannot dynamically generate appropriate constraint conditions according to the real-time changes in the production process, resulting in the lack of flexibility and adaptability of the production scheduling strategy and being unable to cope with emergencies and production anomalies.

[0005] That is to say, existing factory production scheduling methods and technologies have serious deficiencies in dealing with complex and dynamic production environments, achieving multi-objective optimization, and self-learning and evolution, and are difficult to meet the requirements of modern industrial production for efficient, flexible, and intelligent production scheduling. Summary of the Invention

[0006] In view of the above-mentioned problems, in combination with the first aspect of the present invention, embodiments of the present invention provide a method for optimizing factory dynamic production scheduling based on deep reinforcement learning, and the method includes: Obtain the real-time production monitoring data set of the target factory, where the real-time production monitoring data set includes multiple production link monitoring sequences, and each production link monitoring sequence is composed of at least one equipment operation status parameter, an order execution progress parameter, and an environmental interference parameter; Extract features from the real-time production monitoring data set to obtain the target production feature set of each production link monitoring sequence, where the target production feature set includes equipment status dynamic features, order priority features, and environmental constraint features; Call the pre-trained deep reinforcement learning policy network to perform dynamic constraint matching processing on the target production feature set to generate dynamic production constraint features corresponding to the current production link monitoring sequence; Based on the dynamic production constraint features and the preset production scheduling optimization objective function, perform policy iteration optimization processing to generate a real-time production scheduling optimization policy set, where the real-time production scheduling optimization policy set includes an equipment scheduling priority sequence, an order allocation weight sequence, and a resource adjustment ratio sequence; Input the real-time production scheduling optimization policy set into the production scheduling control system of the target factory, trigger the dynamic adjustment operation of the production scheduling execution instruction, and update the network parameters of the deep reinforcement learning policy network based on the adjusted execution feedback data.

[0007] On the other hand, an embodiment of the present invention further provides a factory dynamic production scheduling optimization system based on deep reinforcement learning, including a processor and a machine-readable storage medium. The machine-readable storage medium is connected to the processor. The machine-readable storage medium is used to store programs, instructions, or codes, and the processor is used to execute the programs, instructions, or codes in the machine-readable storage medium to implement the above method.

[0008] Based on the above aspects, the embodiments of the present application effectively integrate and deeply mine the multi-dimensional heterogeneous information in the real-time production monitoring data set. The target production feature set obtained through feature extraction not only accurately depicts the dynamic evolution of the equipment state, the dynamic adjustment requirements of the order priority, and the real-time changes of the environmental constraints, but also provides a high-fidelity input representation for the dynamic modeling of the production system at the global level. The application of the deep reinforcement learning policy network can, based on the dynamic constraint matching processing mechanism, generate dynamic production constraint features that are highly adapted to the current production situation in real time, thus breaking through the limitations of traditional scheduling methods that are difficult to cope with complex dynamic environments. Further, through policy iteration optimization processing, the dynamic production constraint features are deeply integrated with the preset scheduling optimization objective function to generate a real-time scheduling optimization policy set including equipment scheduling priority sequences, order allocation weight sequences, and resource adjustment ratio sequences. This real-time scheduling optimization policy set realizes the optimal allocation of production resources at the global level, not only considering the maximization of equipment utilization, but also taking into account the timeliness of order delivery and the refined control of production costs, thereby achieving a comprehensive improvement in production efficiency in multiple objective optimization dimensions. By feeding back the real-time scheduling optimization policy set to the scheduling control system and triggering dynamic adjustment operations, a closed-loop feedback mechanism for production execution and policy optimization is constructed, enabling the scheduling policy to continuously evolve with the real-time changes in the production process, significantly enhancing the adaptive ability and robustness of the production system. Particularly importantly, by updating the network parameters of the deep reinforcement learning policy network based on the adjusted execution feedback data, the deep integration of the policy learning mechanism and production practice is realized. This characteristic of online learning and continuous optimization can continuously accumulate production experience and improve the level of policy intelligence, thereby forming a self-evolving and continuously optimizing intelligent scheduling paradigm at the global logic level. This paradigm not only breaks through the limitations of traditional scheduling methods that rely on manual experience and static models, but also achieves the global optimal decision-making for production scheduling in a dynamic and complex environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 FIG. is a schematic flowchart of the execution process of the factory dynamic scheduling optimization method based on deep reinforcement learning provided by an embodiment of the present invention.

[0010] Figure 2 FIG. is a schematic diagram of exemplary hardware and software components of the factory dynamic scheduling optimization system based on deep reinforcement learning provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0011] The present invention will be specifically described below with reference to the accompanying drawings of the specification. Figure 1 FIG. is a flowchart of the factory dynamic scheduling optimization method based on deep reinforcement learning provided by an embodiment of the present invention. The factory dynamic scheduling optimization method based on deep reinforcement learning will be introduced in detail below.

[0012] Step S110: Obtain the real-time production monitoring data set of the target factory. The real-time production monitoring data set includes multiple production link monitoring sequences, and each production link monitoring sequence is composed of at least one equipment operation status parameter, order execution progress parameter, and environmental interference parameter.

[0013] In this embodiment, taking the scenario of a digital power battery factory as an example, its production process includes multiple production links, such as battery cell manufacturing, battery module assembly, battery PACK packaging, etc. Each production link has a corresponding production link monitoring sequence.

[0014] Taking the battery cell manufacturing link as an example, the equipment operation status parameters may include the temperature, pressure, current, voltage, etc. of the equipment. For example, at a certain moment, the temperature of equipment A is 50 degrees Celsius, the pressure is 200 Pascals, the current is 10 amperes, and the voltage is 3.7 volts. The order execution progress parameters may include the completed workload of the order, the remaining workload, the consumed resource quantity, etc. Suppose there is currently an order to produce 1000 battery cells, 300 have been completed, 700 are remaining uncompleted, and 50 kilograms of raw materials have been consumed during the production of these 300 cells. The environmental interference parameters may include environmental temperature, humidity, energy supply stability, external events, etc. For example, the current environmental temperature is 25 degrees Celsius, the humidity is 60%, the voltage fluctuation of the energy supply is within ±5%, and no external events have occurred in the past hour.

[0015] By monitoring the above parameters in real time, the monitoring sequence of this production link can be obtained. For other production links, such as battery module assembly and battery PACK packaging, the monitoring sequences of their respective production links are obtained in the same way, and finally the real-time production monitoring data set of the target factory is formed.

[0016] Step S120: Perform feature extraction on the real-time production monitoring data set to obtain the target production feature set of each production link monitoring sequence. The target production feature set includes equipment status dynamic features, order priority features, and environmental constraint features.

[0017] The following will describe in detail the specific process of extracting the target production feature set for each production link monitoring sequence.

[0018] Step S121: Perform time-series fluctuation analysis on the equipment operation status parameters in the production link monitoring sequence, and extract the fluctuation amplitude feature and trend correlation feature of the equipment operation status parameters within a continuous time window.

[0019] In this embodiment, taking equipment A in the battery cell manufacturing link as an example, perform time-series fluctuation analysis on its equipment operation status parameters. The specific sub-steps are as follows: Step S1211: Divide the device operation status parameters into multiple consecutive time window sequences according to a preset time window length. Each time window contains a fixed number of sampling points of the device operation status parameters.

[0020] Suppose the preset time window length is 1 hour. Since the device operation status parameters are sampled once per minute, each time window contains 60 sampling points of the device operation status parameters. Taking the temperature parameter of Device A as an example, within 24 hours of a day, it can be divided into 24 consecutive time window sequences, and each time window contains 60 temperature sampling values within that hour.

[0021] Step S1212: Calculate the actual fluctuation amplitude of the device operation status parameters within each time window, generate the actual average fluctuation amplitude and the actual maximum fluctuation amplitude of each time window, and calculate the window-level difference degree between the actual average fluctuation amplitude and the preset reference average fluctuation amplitude to generate the window-level load deviation coefficient of each time window.

[0022] For the temperature parameter of Device A within each time window, when calculating the actual average fluctuation amplitude, first calculate the average value of the 60 temperature sampling values within this time window, then find the absolute value of the difference between each sampling value and the average value, and then calculate the average value of the above absolute values to obtain the actual average fluctuation amplitude of this time window. For example, within the first time window, the average value of the 60 temperature sampling values is 52 degrees Celsius, and the average value of the absolute values of the differences between each sampling value and 52 degrees Celsius is 2 degrees Celsius, then the actual average fluctuation amplitude of this time window is 2 degrees Celsius. The actual maximum fluctuation amplitude of this time window can be calculated in the same way, assuming it is 4 degrees Celsius. The preset reference average fluctuation amplitude is 1.5 degrees Celsius. By comparing the actual average fluctuation amplitude of 2 degrees Celsius with the reference average fluctuation amplitude of 1.5 degrees Celsius and calculating the difference degree, the window-level load deviation coefficient of this time window is obtained. For example, the difference degree is 2 minus 1.5 equals 0.5 degrees Celsius. According to a certain rule, convert this difference degree into the window-level load deviation coefficient. Suppose the rule is that the difference degree is divided by the reference average fluctuation amplitude, then the window-level load deviation coefficient is 0.5 divided by 1.5, approximately equal to 0.33.

[0023] Step S1213: Perform overrun detection on the actual maximum fluctuation amplitude and the preset reference maximum fluctuation amplitude threshold to generate the maintenance trigger flag for each time window.

[0024] Suppose the preset reference maximum fluctuation threshold is 3 degrees Celsius. In the first time window, the actual maximum fluctuation is 4 degrees Celsius. Since 4 degrees Celsius is greater than 3 degrees Celsius, it indicates that the temperature fluctuation of device A in this time window exceeds the normal range. At this time, a maintenance trigger flag of 1 is generated, indicating that the device maintenance situation needs to be concerned. If the actual maximum fluctuation is less than or equal to 3 degrees Celsius, the maintenance trigger flag is 0, indicating that the device is operating within the normal fluctuation range.

[0025] Step S1214: Perform time-series aggregation processing on the window-level load deviation coefficients of consecutive time windows to generate time-series data of the device load deviation coefficient.

[0026] In this embodiment, the window-level load deviation coefficients of 24 time windows in a day can be arranged in chronological order to form the time-series data of the device load deviation coefficient. For example, the window-level load deviation coefficient of the first time window is 0.33, the second time window is 0.25, the third time window is 0.4, etc. The above window-level load deviation coefficients are arranged in sequence to form a time series containing 24 data points.

[0027] Step S1215: Perform frequency statistics and duration analysis on the maintenance trigger flags of consecutive time windows to generate device maintenance warning indicators.

[0028] In this embodiment, the number of maintenance trigger flags with a value of 1 in 24 time windows of a day can be counted. Suppose it is 5. At the same time, analyze whether the time windows with the maintenance trigger flag of 1 are continuous. If there are 3 consecutive time windows with the maintenance trigger flag of 1, it indicates that the device may have a relatively serious fluctuation problem. According to the above statistical and analysis results, generate device maintenance warning indicators according to certain rules. For example, the rule is that when the number of maintenance trigger flags with a value of 1 exceeds 4 and there are 3 consecutive time windows with a value of 1, the device maintenance warning indicator is high; when the number of maintenance trigger flags with a value of 1 is between 2 and 4, the device maintenance warning indicator is medium; when the number of maintenance trigger flags with a value of 1 is less than 2, the device maintenance warning indicator is low. Suppose the conditions for high warning are met in this example, then the device maintenance warning indicator is high.

[0029] Step S1216: Fit a trend line to the device operating state parameters in each time window, extract the trend line slope and residual fluctuation range, and generate time-window-level parameters of trend correlation features.

[0030] In this embodiment, for the temperature parameters of Device A within each time window, a suitable method is used for trend line fitting, such as the least squares method. Suppose that within the first time window, the slope of the trend line obtained through fitting is 0.1 degree Celsius per minute, and the residual fluctuation range is between ±0.5 degrees Celsius. The values of the above slope and residual fluctuation range constitute the time window-level parameters of the trend correlation feature for this time window.

[0031] Step S1217: Combine the time series data of the device load deviation coefficient and the device maintenance warning index into the fluctuation amplitude feature, and combine the trend line slope and the residual fluctuation range into the trend correlation feature.

[0032] In this embodiment, the time series data of the device load deviation coefficient obtained previously and the device maintenance warning index are combined together to form the fluctuation amplitude feature. For example, the time series data of the device load deviation coefficient is a sequence containing 24 data points, and the device maintenance warning index is high. Combining them into a multi-dimensional data set gives the fluctuation amplitude feature. Similarly, the trend line slope and the residual fluctuation range of each time window are combined together to form the trend correlation feature. For instance, the trend line slopes and the residual fluctuation ranges of 24 time windows in a day respectively form two sequences containing 24 data points, and combining them into a multi-dimensional data set gives the trend correlation feature.

[0033] Step S122: Calculate the degree of difference between the fluctuation amplitude feature and the preset device operation reference parameters to generate the device load deviation coefficient and the device maintenance warning index.

[0034] In this embodiment, after obtaining the fluctuation amplitude feature, it is compared with the preset device operation reference parameters. The preset device operation reference parameters include the standard ranges of the load deviation coefficient and the maintenance warning index during normal device operation. For the device load deviation coefficient, the time series data of the device load deviation coefficient in the fluctuation amplitude feature is compared point by point with the reference device load deviation coefficient to calculate the degree of difference. Suppose the reference device load deviation coefficient is between 0.1 - 0.2, and some data points in the time series data of the device load deviation coefficient in the fluctuation amplitude feature exceed this range. A comprehensive device load deviation coefficient is recalculated based on the degree and range of the exceedance. For the device maintenance warning index, the device maintenance warning index in the fluctuation amplitude feature is compared with the reference maintenance warning index. If the device maintenance warning index in the fluctuation amplitude feature is high while the reference is low, it indicates that there are significant risks in the device operation, and the device maintenance warning index is updated after comprehensive consideration.

[0035] Step S123: Perform similarity matching between the trend correlation feature and the historical device failure mode database to determine the device failure risk level.

[0036] In this embodiment, the trend correlation feature obtained previously can be used to perform similarity matching with the historical device failure mode database. The historical device failure mode database stores the trend correlation features corresponding to various device failures. For example, when a device has a certain failure, the slope of the trend line and the residual fluctuation range in its trend correlation feature will show a specific pattern. Thus, the trend correlation feature of the current device A can be compared with the above historical device failure modes in the historical device failure mode database to calculate the similarity. Suppose the calculated similarity with a certain failure mode is 80%. According to the level of similarity, the device failure risk level is determined. If the similarity is between 70% - 100%, the device failure risk level is high; between 30% - 70%, the device failure risk level is medium; between 0 - 30%, the device failure risk level is low. In this example, the device failure risk level is high.

[0037] Step S124: Combine the device load deviation coefficient, the device maintenance warning index, and the device failure risk level into the device status dynamic feature.

[0038] In this embodiment, the device load deviation coefficient, the device maintenance warning index, and the device failure risk level calculated previously can be combined together to form the device status dynamic feature. For example, the device load deviation coefficient is 0.3, the device maintenance warning index is high, and the device failure risk level is high. By combining them into a multi - dimensional data set, the device status dynamic feature is obtained.

[0039] Step S125: Perform an urgency evaluation process on the order execution progress parameter, extract the difference ratio feature between the remaining order processing time and the preset delivery deadline, and generate an order priority feature in combination with the order type attribute. The order priority feature includes the order delivery urgency, the order resource occupancy weight, and the order type - related constraint.

[0040] In this embodiment, first, the difference ratio feature between the remaining processing time of the order and the preset delivery deadline is extracted. Taking an order in the battery cell manufacturing process as an example, if there is currently an order to produce a certain number of battery cells, and the number of completed cells is known, subtracting the number of completed cells from the total number of cells to be produced can obtain the number of remaining cells to be produced. To estimate the time required to complete the production of the remaining cells, that is, the remaining processing time of the order, the average production rate of cells in the past period can be statistically analyzed. Dividing the number of remaining cells to be produced by the average production rate can obtain the remaining processing time of the order. The preset delivery deadline is a time node clearly defined at the time of order signing. Dividing the remaining processing time of the order by the preset delivery deadline, the resulting value is the difference ratio feature. For example, for the current order to produce 1000 battery cells, 300 have been completed, and 700 are remaining. If the average production rate in the past period is 100 cells per hour, then the remaining processing time of the order is 7 hours. If the preset delivery deadline is 10 hours from the current moment, the difference ratio feature is 7 divided by 10, which is equal to 0.7.

[0041] When determining the order delivery urgency, both the difference ratio feature and the order type attribute play key roles. For the difference ratio feature, if its value is small, it indicates that the delivery deadline is approaching, and the order is relatively more urgent. The order type attribute is divided into ordinary orders and urgent orders, etc. Urgent orders themselves mean that they need to be processed preferentially and have a higher degree of urgency. Considering these two factors together, if the difference ratio feature is small and the order is an urgent order, then the order delivery urgency will be very high; if the difference ratio feature is large and it is an ordinary order, the order delivery urgency will be relatively low. A judgment rule can be set. When the difference ratio feature is less than a certain set value, such as 0.3, and the order is an urgent order, the order delivery urgency is determined to be high; when the difference ratio feature is between 0.3 and 0.7, regardless of the order type, the order delivery urgency is determined to be medium; when the difference ratio feature is greater than 0.7 and it is an ordinary order, the order delivery urgency is determined to be low.

[0042] The determination of the order resource occupancy weight is based on the amount of resources already consumed by the order in the production process and the amount of resources expected to be consumed. For example, when producing the 300 completed cells of this order, a certain amount of raw materials has been consumed, and it is expected that a certain amount of raw materials will be consumed to complete the remaining 700 cells. Adding these two parts together gives the total amount of raw materials required. Then, comparing the total amount of raw materials required for this order with the resource requirements of other orders, if the amount of resources required for this order is relatively large, then its resource occupancy weight is relatively large; if it is relatively small, the resource occupancy weight is relatively small. For example, if the average raw material requirement for other orders is 120 kg, and this order requires a total of 150 kg of raw materials, it indicates that the resource occupancy weight of this order is relatively large.

[0043] The order type association constraint involves whether the order has special production process requirements, whether specific equipment is needed, etc. Some orders may require the use of special battery manufacturing processes or must use specific models of production equipment. These special requirements will affect the production arrangement of the order. For example, if an order has special production process requirements, workers who master the process and the corresponding equipment need to be arranged for production; if specific equipment is required and the quantity of this equipment is limited, the production progress of the order will be restricted.

[0044] Finally, combining the order delivery urgency, the order resource occupancy weight, and the order type association constraint forms the order priority feature. This order priority feature can comprehensively and synthetically reflect the importance and priority of each order in the production scheduling, providing an important basis for subsequent production scheduling.

[0045] Step S126: Perform impact factor analysis processing on the environmental interference parameters, extract the environmental temperature change rate, the energy supply volatility rate, and the external event trigger frequency, and generate environmental constraint features. The environmental constraint features include the temperature adaptability coefficient, the energy stability level, and the event response priority.

[0046] In this embodiment, during the battery cell manufacturing process, the environmental temperature is a key environmental interference parameter. To obtain the environmental temperature change rate, the environmental temperature needs to be monitored over a period of time. The specific method is to record the environmental temperature at the start time and the end time, then subtract the temperature at the start time from the temperature at the end time to obtain the change value of the environmental temperature during this period. Divide this change value by the time interval to obtain the environmental temperature change rate. For example, within 1 hour, the environmental temperature rises from 25 degrees Celsius to 26 degrees Celsius, then the environmental temperature change rate is (26 - 25) degrees Celsius divided by 1 hour, which is equal to 1 degree Celsius per hour.

[0047] The calculation of the energy supply volatility rate depends on monitoring the fluctuations of parameters such as the voltage and current of the energy supply. In actual production, the stability of the energy supply directly affects the normal operation of the equipment. The maximum and minimum values of the energy supply parameters can be recorded over a period of time, and the maximum value minus the minimum value is used to obtain the fluctuation range. Then, based on this fluctuation range and combined with the standard range of the energy supply parameters during normal operation, the energy supply volatility rate is calculated. Suppose the normal fluctuation range of the voltage of the energy supply is ±3%, and the actually monitored voltage fluctuation is within ±5%, then the energy supply volatility rate can be determined according to this fluctuation situation.

[0048] The external event trigger frequency is the number of external events that occur within a certain period of time. External events include equipment failures, sudden power outages, raw material supply interruptions, etc. For example, if 2 external events occur within a day, then the external event trigger frequency is 2 times per day.

[0049] Next, based on the extracted environmental temperature change rate, energy supply volatility, and external event trigger frequency, generate a temperature adaptability coefficient, an energy stability level, and an event response priority respectively.

[0050] For the temperature adaptability coefficient, it is necessary to determine it in combination with the temperature adaptation range of the equipment. Different production equipment has different requirements for the environmental temperature. Generally, equipment will have a normal operating temperature range. If the environmental temperature change rate is small and the current environmental temperature is within the adaptation range of the equipment, then the equipment can operate stably and the temperature adaptability coefficient is relatively high; conversely, if the environmental temperature change rate is large or the environmental temperature exceeds the adaptation range of the equipment, the equipment may malfunction or its performance may decline, and the temperature adaptability coefficient is relatively low. For example, if the equipment can operate normally at an environmental temperature of 20 - 30 degrees Celsius, the current environmental temperature is 25 degrees Celsius, and the environmental temperature change rate is small, then the temperature adaptability coefficient can be set to a relatively high value, such as 0.8.

[0051] The energy stability level is divided according to the energy supply volatility. Different energy supply volatility intervals can be preset, corresponding to different energy stability levels. If the energy supply volatility is within a small range, it indicates that the energy supply is relatively stable and the energy stability level is relatively high; if the energy supply volatility is large, the energy stability level is relatively low. For example, when the energy supply volatility is within ±3%, the energy stability level can be determined as high; when it is between ±3% - ±5%, the energy stability level is medium; when it exceeds ±5%, the energy stability level is low.

[0052] The event response priority needs to be determined according to the external event trigger frequency and the severity of the event. If the external event trigger frequency is high and the severity of the event is large, such as multiple equipment failures occurring and affecting the production progress, then the event response priority is high; if the external event trigger frequency is low and the severity of the event is small, such as only occasional minor equipment failures that do not affect normal production, then the event response priority is low. For example, if multiple serious equipment failures occur within a day, resulting in production stagnation, then the event response priority can be set to high; if only one minor raw material supply delay occurs within a week, which does not affect the overall production progress, the event response priority can be set to low.

[0053] Finally, combine the temperature adaptability coefficient, the energy stability level, and the event response priority together to form an environmental constraint feature, which comprehensively reflects the degree of environmental factors' constraints on the production process. By accurately evaluating the environmental constraint feature, the production plan can be reasonably arranged, production efficiency can be improved, and production failures and losses caused by environmental factors can be reduced.

[0054] Step S127: Perform feature fusion processing on the device state dynamic features, the order priority features, and the environmental constraint features to generate the target production feature set.

[0055] In this embodiment, the device state dynamic features, the order priority features, and the environmental constraint features obtained previously can be subjected to feature fusion processing. For example, a weighted splicing method can be adopted, and different weights are assigned according to the importance of different features. For example, the weight of the device state dynamic features is 0.4, the weight of the order priority features is 0.3, and the weight of the environmental constraint features is 0.3. These three features are spliced according to the weights to form a multi-dimensional data set, and the target production feature set is obtained.

[0056] Step S130: Invoke the pre-trained deep reinforcement learning policy network to perform dynamic constraint matching processing on the target production feature set to generate dynamic production constraint features corresponding to the current production link monitoring sequence.

[0057] After obtaining the target production feature set, a pre-trained deep reinforcement learning policy network is invoked for dynamic constraint matching processing. The specific process is as follows: Step S131: Input the target production feature set into the state encoding module of the deep reinforcement learning policy network to perform spatial mapping processing on the device state dynamic features and generate a device state encoding vector.

[0058] In this embodiment, the target production feature set is input into the state encoding module of the deep reinforcement learning policy network. For the device state dynamic features, the state encoding module will perform spatial mapping processing on them. For example, the device state dynamic features are a multi-dimensional data set including the device load deviation coefficient, the device maintenance warning index, and the device failure risk level. The state encoding module can map it into a high-dimensional vector space to generate a device state encoding vector. Suppose the device state dynamic features are mapped to a 10-dimensional vector after mapping, and this vector is the device state encoding vector.

[0059] Step S132: Perform weight assignment processing on the order priority features and generate an order priority encoding vector in combination with historical order execution efficiency data.

[0060] The specific sub-steps for performing weight assignment processing on the order priority features are as follows: Step S1321: Obtain the order completion time distribution, the resource consumption ratio, and the customer satisfaction score in the historical order execution efficiency data.

[0061] In this embodiment, the order completion time distribution, resource consumption ratio, and customer satisfaction score can be extracted from the historical order execution efficiency data. For example, the historical order completion time distribution shows that most orders are completed within 5 - 7 days; in terms of the resource consumption ratio, the resource consumption of different types of orders is different. An ordinary order consumes an average of 100 kilograms of raw materials, and an urgent order consumes an average of 120 kilograms of raw materials; in terms of the customer satisfaction score, based on the feedback and evaluation of customers, the average score is 80 points.

[0062] Step S1322: Generate a time urgency weight according to the proportional relationship between the order completion time distribution and the remaining processing time of the current order.

[0063] In this embodiment, assume that the remaining processing time of the current order is 3 days. According to the historical order completion time distribution, most orders are completed within 5 - 7 days. Calculate the ratio of the remaining processing time of the current order to the average completion time of historical orders (assumed to be 6 days), that is, 3 days divided by 6 days equals 0.5. According to this proportional relationship, generate a time urgency weight according to certain rules. For example, when the ratio is less than 0.5, the time urgency weight is greater than 0.8; when the ratio is between 0.5 - 0.8, the time urgency weight is between 0.5 - 0.8; when the ratio is greater than 0.8, the time urgency weight is less than 0.5. In this example, the time urgency weight is 0.8.

[0064] Step S1323: Generate a resource adaptability weight according to the difference degree between the resource consumption ratio and the resource occupancy weight of the current order.

[0065] In this embodiment, assume that an ordinary order in historical orders consumes an average of 100 kilograms of raw materials, and the current order is an ordinary order, with an estimated consumption of 110 kilograms of raw materials. Calculate the difference degree between the resource consumption ratio and the resource occupancy weight of the current order, that is, 110 kilograms minus 100 kilograms equals 10 kilograms. According to this difference degree, generate a resource adaptability weight according to the set rules. For example, when the difference degree is less than 10 kilograms, the resource adaptability weight is greater than 0.8; when the difference degree is between 10 - 20 kilograms, the resource adaptability weight is between 0.5 - 0.8; when the difference degree is greater than 20 kilograms, the resource adaptability weight is less than 0.5. In this example, the resource adaptability weight is 0.7.

[0066] Step S1324: Generate a customer priority weight according to the correlation degree between the customer satisfaction score and the attribute of the current order type.

[0067] In this embodiment, assume that the current order is an urgent order. The customer satisfaction score of historical urgent orders is relatively high, with an average of 85 points, while the average customer satisfaction score of the overall historical orders is 80 points. This indicates that the current order type (urgent order) is associated with a relatively high customer satisfaction. According to this degree of association, a customer priority weight is generated according to certain rules. If the degree of association is high, the customer priority weight can be set to 0.9; if the degree of association is average, it is set to 0.7; if the degree of association is low, it is set to 0.5. In this example, since the urgent order corresponds to a relatively high customer satisfaction, the customer priority weight is set to 0.9.

[0068] Step S1325: Normalize the time urgency weight, the resource adaptability weight, and the customer priority weight to generate an order priority weight vector.

[0069] In this embodiment, the time urgency weight 0.8, the resource adaptability weight 0.7, and the customer priority weight 0.9 obtained previously are normalized. Normalization is to make the sum of the above weights equal to 1 to ensure the rationality of subsequent calculations. First, calculate the sum of these three weights, that is, 0.8 + 0.7 + 0.9 = 2.4. Then, calculate the proportion of each weight in the sum. After normalization, the time urgency weight is 0.8÷2.4≈0.33, the resource adaptability weight is 0.7÷2.4≈0.29, and the customer priority weight is 0.9÷2.4≈0.38. Combine these three normalized weights into a vector, that is, the order priority weight vector [0.33, 0.29, 0.38].

[0070] Step S1326: Concatenate the order priority weight vector with the order type association constraint to generate the order priority coding vector.

[0071] In this embodiment, the order type association constraint may include information such as special production process requirements and specific equipment requirements. Assume that the order type association constraint is represented by a 3D vector, such as [1, 0, 1], where 1 indicates the presence of this constraint and 0 indicates the absence. Concatenate the order priority weight vector [0.33, 0.29, 0.38] with the order type association constraint vector [1, 0, 1] to obtain a 6D order priority coding vector [0.33, 0.29, 0.38, 1, 0, 1].

[0072] Step S133: Normalize the environmental constraint features to generate an environmental constraint coding vector.

[0073] In this embodiment, the environmental constraint features include a temperature adaptability coefficient, an energy stability level, and an event response priority. Assume that the temperature adaptability coefficient is 0.8, the energy stability level is quantified as 0.6 (for example, in the range of 0 - 1, where 0 represents extremely unstable and 1 represents extremely stable), and the event response priority is quantified as 0.3 (for example, in the range of 0 - 1, where 0 represents low priority and 1 represents high priority). To uniformly process the features with different ranges and meanings above, normalization is performed. First, calculate the sum of these three values, 0.8 + 0.6 + 0.3 = 1.7. Then, calculate the proportion of each value in the sum respectively. After normalization, the temperature adaptability coefficient is 0.8 ÷ 1.7 ≈ 0.47, the energy stability level after normalization is 0.6 ÷ 1.7 ≈ 0.35, and the event response priority after normalization is 0.3 ÷ 1.7 ≈ 0.18. Combine these three normalized values into a vector, that is, the environmental constraint encoding vector [0.47, 0.35, 0.18].

[0074] Step S134: Invoke the constraint matching module of the deep reinforcement learning policy network to perform joint attention calculation on the device state encoding vector, the order priority encoding vector, and the environmental constraint encoding vector to generate a joint attention weight distribution.

[0075] In this embodiment, the constraint matching module of the deep reinforcement learning policy network can perform joint attention calculation on the device state encoding vector, the order priority encoding vector, and the environmental constraint encoding vector. The joint attention mechanism is used to determine the importance of each encoding vector when generating dynamic production constraint features. Assume that the device state encoding vector is a 10 - dimensional vector, the order priority encoding vector is a 6 - dimensional vector, and the environmental constraint encoding vector is a 3 - dimensional vector. The constraint matching module can calculate an attention weight for each dimension of each vector through a series of calculations (such as linear transformation, activation function, etc.). For example, for the first dimension of the device state encoding vector, the calculated attention weight is 0.1; for the third dimension of the order priority encoding vector, the attention weight is 0.2, etc. Finally, a joint attention weight distribution is obtained, which represents the relative importance of each dimension of each encoding vector when generating dynamic production constraint features.

[0076] Step S135: Based on the joint attention weight distribution, perform dynamic weighted aggregation processing on the device state encoding vector, the order priority encoding vector, and the environmental constraint encoding vector to generate the dynamic production constraint features.

[0077] In this embodiment, the device status encoding vector, the order priority encoding vector, and the environmental constraint encoding vector can be dynamically weighted and aggregated according to the joint attention weight distribution. For each dimension of the device status encoding vector, its value is multiplied by the corresponding attention weight; the same operation is also performed on the order priority encoding vector and the environmental constraint encoding vector. Then, the three processed vectors are concatenated in a certain manner. For example, after processing, the device status encoding vector is [0.1×a1, 0.2×a2, …, 0.1×a10], the order priority encoding vector is [0.2×b1, 0.3×b2, …, 0.1×b6], and the environmental constraint encoding vector is [0.3×c1, 0.2×c2, 0.1×c3]. They are concatenated into a new vector, which is the dynamic production constraint feature.

[0078] Step S140: Based on the dynamic production constraint feature and a preset production scheduling optimization objective function, perform policy iteration optimization processing to generate a real-time production scheduling optimization policy set, where the real-time production scheduling optimization policy set includes a device scheduling priority sequence, an order allocation weight sequence, and a resource adjustment ratio sequence.

[0079] After obtaining the dynamic production constraint feature, perform policy iteration optimization processing on it and the preset production scheduling optimization objective function. The specific process is as follows: Step S141: Perform policy feasible region matching processing on the dynamic production constraint feature and the production scheduling optimization objective function to determine the initial policy feasible region under the current production state.

[0080] Its specific sub-steps are as follows: Step S1411: Analyze the correlation between the device status dynamic feature and the environmental constraint feature in the dynamic production constraint feature, and extract the device availability constraint interval and the resource supply stability interval. In the actual scenario of battery production, there is a correlation between the device status dynamic feature and the environmental constraint feature, and it is necessary to deeply analyze it to extract key information.

[0081] Specifically, the device load deviation coefficient in the device status dynamic feature reflects the deviation degree of the actual operating load of the device from the ideal load. If the device load deviation coefficient is large, it indicates that the device is running unstably and may have excessive wear or potential faults. The device maintenance warning index is a prediction of whether the device needs maintenance based on the device's operating data and historical fault records. The device failure risk level is an assessment of the possibility of the device failing, considering factors such as the device's aging degree and operating conditions.

[0082] The temperature adaptability coefficient in the environmental constraint features reflects the impact degree of environmental temperature on the operation of the equipment. When the environmental temperature exceeds the suitable operation range of the equipment, the performance of the equipment may decline, or even faults may occur. The energy stability level reflects the fluctuation situation of the energy supply, and a stable energy supply is crucial for the normal operation of the equipment. The event response priority is related to the occurrence frequency and severity of external events (such as natural disasters, sudden equipment failures, etc.).

[0083] To determine the equipment availability constraint interval, first, a large amount of historical data is collected to analyze the fault occurrence situation of the equipment under different equipment states and environmental conditions. For example, when the equipment load deviation coefficient exceeds a certain threshold and the environmental temperature is too high (the temperature adaptability coefficient is low), the probability of equipment failure will increase significantly. Combining the equipment maintenance plan and the time required for repair, the duration range during which the equipment can operate normally within a certain period can be calculated. Assume that in this embodiment, through the analysis of data in the past year, it is found that when the equipment load deviation coefficient is greater than 0.3 and the environmental temperature is higher than 30 degrees Celsius, the equipment may have a fault once every 5 hours on average, and the repair takes 2 hours. Considering the equipment maintenance plan, it can be determined that in this case, the available duration range of the equipment within 24 hours a day may be 4 - 6 hours, which is the equipment availability constraint interval.

[0084] The determination of the resource supply stability interval mainly focuses on the energy supply. In this embodiment, the voltage, current and other parameters of the energy supply are monitored in real time, and their fluctuation situations are recorded. At the same time, combined with the requirements of the equipment for the energy supply, the influence of different fluctuation ranges on the equipment operation is analyzed. For example, some high-precision battery production equipment is very sensitive to voltage fluctuations. When the voltage fluctuation exceeds ±3%, the production precision of the equipment will be affected. Through the analysis of a large amount of historical data, the stable fluctuation range of the energy supply can be determined under the premise of ensuring the normal operation of the equipment. Assume that after analysis, it is found that when the voltage fluctuation of the energy supply is between ±3% - ±5%, the equipment can operate stably, and this is the resource supply stability interval.

[0085] Step S1412: Decompose the production scheduling optimization objective function into multiple sub-objective functions, where the sub-objective functions include the function of maximizing equipment utilization rate, the function of minimizing order delivery delay, and the function of equalizing resource consumption The production scheduling optimization objective function is a comprehensive objective, aiming to achieve the efficient, stable and sustainable development of the entire production process. For the convenience of optimizing it, this embodiment decomposes it into three specific sub-objective functions.

[0086] The core objective of the equipment utilization rate maximization function is to make full use of the production capacity of the equipment and reduce the idle time of the equipment. This embodiment can consider factors such as the production capacity of the equipment, operating efficiency, and maintenance cycle. For example, for a battery cell manufacturing equipment, it can analyze the production quantity per hour, the efficiency under different working modes, and the time required for the equipment to stop for maintenance. By reasonably arranging the production tasks and operating time of the equipment, the equipment can be kept in a highly efficient operating state for as long as possible. This embodiment can establish a mathematical model of equipment utilization rate based on the historical operating data of the equipment, with variables such as the equipment startup time and production quantity, and solve for the production arrangement that maximizes the equipment utilization rate through an optimization algorithm.

[0087] The function of minimizing order delivery delay focuses on how to ensure that orders can be delivered on time and improve customer satisfaction. This embodiment collects information such as the delivery deadline, order quantity, and production difficulty of each order. For urgent orders, this embodiment can give priority to arranging production resources to ensure that they can be completed within the specified time. At the same time, this embodiment can analyze the production process of the order and the time required for each link, and reasonably arrange the production sequence of the order. For example, this embodiment can sort the orders according to the delivery urgency of the orders and give priority to processing the orders with a high degree of urgency. By establishing an evaluation model of order delivery delay, with the difference between the actual delivery time and the delivery deadline of the order as the index, optimize the production scheduling of the order to minimize the order delivery delay.

[0088] The function of balancing resource consumption is committed to reasonably allocating resources among different production links and orders, and avoiding excessive concentration or shortage of resources. In battery production, the resources involved include raw materials, energy, manpower, etc. This embodiment can analyze the demand for various resources in each production link and order, and formulate a reasonable resource allocation plan according to the total amount and supply situation of the resources. For example, for the allocation of raw materials, this embodiment can accurately calculate the quantity of raw materials required for each link according to the production progress and demand of different production links, and replenish them in a timely manner. By establishing a balance model of resource consumption, with variables such as the allocation ratio and consumption speed of resources, optimize the allocation of resources to make resource consumption more balanced.

[0089] Step S1413: Perform a constraint matching process on the equipment availability constraint interval and the equipment utilization rate maximization function to generate the boundary of the first feasible region for equipment scheduling The equipment availability constraint interval stipulates the duration range during which the equipment can operate normally within a certain period of time, while the equipment utilization rate maximization function pursues the efficient use of the equipment during operation. Performing a constraint matching process on the two is to find a scheduling plan that can maximize the equipment utilization rate within the time range when the equipment is available.

[0090] In this embodiment, first, the time period during which the device can be used for production can be determined according to the device availability constraint interval. Then, combined with the device utilization maximization function, the production efficiency of the device in different time periods can be analyzed. For example, the operating efficiency of some devices is relatively high in the morning, while in the afternoon, the efficiency may decrease due to reasons such as device heating. This embodiment can establish a model of the device production efficiency varying with time based on the historical operating data of the device.

[0091] When performing constraint matching, this embodiment can use an optimization algorithm, taking the production task allocation, operating time, etc. of the device as variables, and searching for the solution that maximizes the device utilization within the device availability constraint interval. For example, this embodiment can adopt a genetic algorithm, and through continuous iteration and optimization, find the optimal device scheduling scheme. Suppose the device availability constraint interval is from 9:00 am to 5:00 pm, and the production efficiency of the device is relatively high from 10:00 am to 12:00 pm and from 2:00 pm to 4:00 pm. Through the calculation of the optimization algorithm, this embodiment can determine that production tasks should be preferentially arranged for the device from 10:00 am to 12:00 pm and from 2:00 pm to 4:00 pm, which is the boundary of the first feasible region for device scheduling.

[0092] Step S1414: Align the order delivery delay minimization function with the order priority feature in the dynamic production constraint features to generate the boundary of the second feasible region for order allocation The order priority feature includes information such as the order delivery urgency, the order resource occupancy weight, and the order type association constraint. The above information reflects the importance and special requirements of each order in the production process. The goal of the order delivery delay minimization function is to ensure that orders can be delivered on time and reduce the situation of late delivery.

[0093] This embodiment can first conduct a detailed analysis of the order priority feature. The order delivery urgency can be determined according to the order delivery deadline and the remaining production time. For example, if the delivery deadline of an order is approaching and the remaining production time is short, then the delivery urgency of this order is relatively high. The order resource occupancy weight is related to the quantity of resources such as raw materials, equipment, and manpower required by the order. The order type association constraint includes whether the order has special production process requirements, whether specific equipment is needed, etc.

[0094] When performing the urgency alignment process, this embodiment can combine the order delivery delay minimization function with the order priority feature. For orders with high delivery urgency, this embodiment can give priority to their production arrangements to ensure that they can be completed within the specified time. At the same time, this embodiment can reasonably allocate production resources according to the order resource occupancy weight to avoid order late delivery due to insufficient resources. For orders with special production process requirements or those that require specific equipment, this embodiment can arrange the corresponding resources and equipment in advance.

[0095] By comprehensively analyzing the priority characteristics and delivery delay situations of all orders, this embodiment can determine the allocation scope of orders under different circumstances. For example, for urgent orders, this embodiment can determine that they must be allocated to appropriate equipment for production within the next 2 days; for ordinary orders, this embodiment can determine their allocation sequence and production time within a certain time range according to their delivery urgency and resource requirements. This is the boundary of the second feasible region for order allocation.

[0096] Step S1415: Dynamically adapt the resource consumption equalization function to the resource supply stability interval to generate the boundary of the third feasible region for resource adjustment The goal of the resource consumption equalization function is to reasonably allocate resources among different production links and orders, avoiding excessive concentration or shortage of resources. The resource supply stability interval stipulates the stable supply situation of resources within a certain range. Dynamically adapting the two means adjusting the resource allocation according to the stability of resource supply to achieve balanced resource consumption.

[0097] This embodiment first conducts a detailed analysis of the resource consumption equalization function to determine the resource requirements of different production links and orders for various resources. For example, in battery production, the cell manufacturing link requires a large amount of raw materials and energy, while the module assembly link has a higher demand for manpower and equipment. Then, this embodiment can monitor the resource supply situation in real time and judge whether the resource supply is stable according to the resource supply stability interval.

[0098] When the resource supply is within the stable interval, this embodiment can reasonably allocate resources according to the requirements of the resource consumption equalization function. For example, accurately calculate the quantity of raw materials required for each link according to the production progress and requirements of different production links, and replenish them in a timely manner. When the resource supply fluctuates, this embodiment can dynamically adjust the resource allocation according to the degree and direction of the fluctuation. For example, if the voltage fluctuation of the energy supply exceeds the stable interval, this embodiment preferentially ensures the energy supply to the production links sensitive to voltage, and at the same time reduces the energy allocation to the links with lower voltage requirements.

[0099] By continuously monitoring and adjusting the resource consumption and supply situations, this embodiment can determine the scope and proportion of resource adjustment under different resource supply conditions. For example, when the energy supply stability interval is a voltage fluctuation between ±3% - ±5%, this embodiment can determine that the energy allocation proportion for the battery cell manufacturing link is between 40% - 50%. This is the boundary of the third feasible region for resource adjustment.

[0100] Step S1416: Calculate the joint overlapping region of the first feasible region boundary, the second feasible region boundary, and the third feasible region boundary to generate the initial strategy feasible region The initial policy feasible region refers to the set of policies that simultaneously meet the requirements of equipment scheduling, order allocation, and resource adjustment. To generate this feasible region, this embodiment needs to calculate the joint overlapping region of the first feasible region boundary, the second feasible region boundary, and the third feasible region boundary.

[0101] For example, these three feasible region boundaries can be regarded as three different constraint conditions, and each constraint condition limits the feasible range of the policy. The first feasible region boundary of equipment scheduling stipulates the time periods during which the equipment can perform production scheduling; the second feasible region boundary of order allocation determines the allocation order and production time of orders under different circumstances; the third feasible region boundary of resource adjustment stipulates the allocation ratio of resources among different production links and orders.

[0102] When calculating the joint overlapping region, the constraint conditions represented by these three feasible region boundaries can be integrated. For example, for a certain production time period, if it simultaneously meets the first feasible region boundary of equipment scheduling (the equipment is available during this time period and can achieve a relatively high utilization rate), the second feasible region boundary of order allocation (the orders arranged during this time period can be delivered on time and meet the order priority requirements), and the third feasible region boundary of resource adjustment (the resource allocation during this time period can ensure the balanced consumption of resources), then this time period belongs to the initial policy feasible region.

[0103] This embodiment can use mathematical methods, such as linear programming, non-linear programming, etc., to solve these three feasible region boundaries and find their joint overlapping region. In this way, this embodiment can determine which time, which order, and resource allocation schemes are simultaneously feasible, thereby generating the initial policy feasible region. For example, through calculation, it is found that during the time period from 10 am to 12 pm, equipment A can perform production scheduling, orders 1 and 2 can be produced during this time period, and the resource allocation can ensure the balanced consumption of resources. Then the time period from 10 am to 12 pm belongs to a part of the initial policy feasible region.

[0104] Step S142: Perform dynamic constraint decomposition processing on the initial policy feasible region to generate a multi-objective optimization parameter set, and the multi-objective optimization parameter set includes an equipment load balancing coefficient, an order delivery time threshold, and a resource allocation conflict index.

[0105] The specific sub-steps are as follows: Step S1421: Divide the equipment scheduling priority levels within the initial policy feasible region, and the equipment scheduling priority levels are arranged in descending order according to the product of the equipment failure risk level and the equipment load deviation coefficient to generate an equipment load balancing coefficient.

[0106] In this embodiment, within the initial policy feasible region, the scheduling priority levels of the devices are divided. They are sorted in descending order according to the product of the device failure risk level and the device load deviation coefficient. For example, there are devices A, B, and C. The device failure risk level of device A is high (quantified as 0.8), and the device load deviation coefficient is 0.3. Their product is 0.8×0.3 = 0.24. The device failure risk level of device B is medium (quantified as 0.6), and the device load deviation coefficient is 0.2. The product is 0.6×0.2 = 0.12. The device failure risk level of device C is low (quantified as 0.2), and the device load deviation coefficient is 0.1. The product is 0.2×0.1 = 0.02. Sorted in descending order, they are device A, device B, and device C. According to this sorting situation, considering factors such as the number of devices and production capacity, a device load balance coefficient is generated. For example, if device A has a high production capacity and is ranked in the front, it will be given priority when allocating tasks, which will affect the balance of device loads. The finally generated device load balance coefficient may be 0.7.

[0107] Step S1422: Construct a time urgency matrix based on the reciprocal relationship between the order delivery urgency and the reciprocal of the remaining order processing time, and calculate the delivery time threshold for each order in combination with the order type association constraint.

[0108] In this embodiment, the order delivery urgency is inversely proportional to the remaining order processing time, that is, the shorter the remaining order processing time, the higher the order delivery urgency. A time urgency matrix is constructed based on this relationship. Suppose there are currently three orders. The order delivery urgency of order 1 is high, and the remaining order processing time is 2 days. The order delivery urgency of order 2 is medium, and the remaining order processing time is 4 days. The order delivery urgency of order 3 is low, and the remaining order processing time is 6 days. The constructed time urgency matrix can be expressed as [1 / 2, 1 / 4, 1 / 6]. In combination with the order type association constraint, for example, order 1 is an urgent order with special delivery requirements, order 2 is an ordinary order, and order 3 is a long-term order. Based on the above information, calculate the delivery time threshold for each order. For example, the delivery time threshold for order 1 may be 1 day, the delivery time threshold for order 2 may be 3 days, and the delivery time threshold for order 3 may be 5 days.

[0109] Step S1423: Analyze the inverse correlation relationship between the energy supply volatility and the material replenishment priority within the resource supply stability interval to generate a resource allocation conflict indicator.

[0110] In this embodiment, within the resource supply stability range, the inverse correlation between the energy supply volatility and the material replenishment priority is analyzed. When the energy supply volatility is high, in order to ensure the stability of production, the material replenishment priority should be reduced; conversely, when the energy supply volatility is low, the material replenishment priority can be increased. For example, when the energy supply stability range is that the voltage fluctuation is between ±3% and ±5%, if the current energy supply volatility is ±4%, the material replenishment priority can be appropriately reduced. By analyzing this inverse correlation and combining the total amount and demand of resources, a resource allocation conflict index is generated. If the energy supply fluctuates greatly and the material demand is also large, it may lead to resource allocation conflicts, and the resource allocation conflict index will increase accordingly. Suppose the resource allocation conflict index is 0.6 at this time.

[0111] Step S1424: Normalize and splice the device load balancing coefficient, the order delivery time threshold, and the resource allocation conflict index according to a preset weight ratio to generate the multi-objective optimization parameter set.

[0112] In this embodiment, it is assumed that the preset weight ratio is that the device load balancing coefficient accounts for 0.4, the order delivery time threshold accounts for 0.3, and the resource allocation conflict index accounts for 0.3. First, normalize the device load balancing coefficient, the order delivery time threshold, and the resource allocation conflict index. Suppose the device load balancing coefficient is 0.7, the order delivery time threshold obtains a value of 0.5 through a certain quantization method, and the resource allocation conflict index is 0.6. Splice them according to the weight ratio. After processing, the device load balancing coefficient is 0.4×0.7 = 0.28, the order delivery time threshold after processing is 0.3×0.5 = 0.15, and the resource allocation conflict index after processing is 0.3×0.6 = 0.18. Combine these three processed values into a vector, that is, the multi-objective optimization parameter set [0.28, 0.15, 0.18].

[0113] Step S143: Based on the multi-objective optimization parameter set, perform a policy conflict verification process, reassign the priorities of the optimization objectives with parameter conflicts, and generate an adjusted multi-objective optimization parameter set.

[0114] The specific sub-steps are as follows: Step S1431: Detect the resource occupancy conflict nodes between the device load balancing coefficient and the order delivery time threshold. The resource occupancy conflict nodes are the time period identifiers where the device scheduling requirements do not match the order processing rate.

[0115] In this embodiment, the relationship between the device load balancing coefficient and the order delivery time threshold can be analyzed to detect resource occupation conflict nodes. For example, when the device load balancing coefficient is high, devices may be preferentially allocated to certain orders to achieve load balancing. However, the delivery time threshold for the above orders may be long, and some orders with shorter delivery time thresholds may not be able to obtain device resources in time, resulting in resource occupation conflicts. By analyzing the device scheduling requirements and the order processing rate, the time period identifier when the above conflicts occur is determined. Suppose between 10 am and 11 am, due to the device load balancing allocation method, an order with a shorter delivery time threshold cannot be processed in time, then this time period is a resource occupation conflict node.

[0116] Step S1432: Extract the event response priority in the environmental constraint features at the resource occupation conflict node, and determine whether to trigger the external event intervention mechanism.

[0117] In this embodiment, at the resource occupation conflict node, the event response priority in the environmental constraint features is extracted. If the event response priority is high, the external event intervention mechanism may be triggered. For example, when external events such as device failures and energy supply interruptions occur, the event response priority will increase. Suppose at the resource occupation conflict node between 10 am and 11 am, the event response priority in the environmental constraint features is high, and a device failure is detected, then the external event intervention mechanism is triggered.

[0118] Step S1433: If the external event intervention mechanism is not triggered, dynamically reduce the weight of the device load balancing coefficient according to the current value of the resource allocation conflict index, and increase the priority of the order delivery time threshold.

[0119] In this embodiment, if the external event intervention mechanism is not triggered at the resource occupation conflict node, it is adjusted according to the current value of the resource allocation conflict index. Suppose the resource allocation conflict index is 0.6, indicating that there are certain resource allocation conflicts. At this time, the weight of the device load balancing coefficient is dynamically reduced, for example, from the original 0.4 to 0.3, and at the same time, the priority of the order delivery time threshold is increased, and its weight is increased from 0.3 to 0.4. This can ensure that orders are delivered on time and reduce resource allocation conflicts.

[0120] Step S1434: If the external event intervention mechanism is triggered, freeze the update of the resource allocation conflict index according to the event response priority, and replace the current device load balancing coefficient with the historical average parameter.

[0121] In this embodiment, if the external event intervention mechanism is triggered, the update of the resource allocation conflict metric is frozen according to the event response priority. For example, when the event response priority is high, it indicates that the current external event has a greater impact on production. At this time, the update of the resource allocation conflict metric is suspended to avoid unnecessary adjustments under unstable conditions. Meanwhile, the historical average parameter is used to replace the current device load balancing coefficient. Assume that the average value of the device load balancing coefficient in history is 0.6, and this average value is used to replace the current device load balancing coefficient that may be interfered by external events to ensure a relatively stable basis for device scheduling.

[0122] Step S1435: Re-normalize the adjusted device load balancing coefficient, order delivery time threshold, and resource allocation conflict metric to generate the adjusted multi-objective optimization parameter set.

[0123] In this embodiment, after the previous adjustments, the adjusted device load balancing coefficient, order delivery time threshold, and resource allocation conflict metric are obtained. Assume that the adjusted device load balancing coefficient is 0.3, the weight corresponding to the adjusted order delivery time threshold is 0.4 (such as 0.6 after conversion), and the resource allocation conflict metric remains 0.6 after being frozen. First, calculate the sum of these three values, 0.3 + 0.6 + 0.6 = 1.5. Then, calculate the proportion of each value in the sum respectively. The adjusted device load balancing coefficient after normalization is 0.3 ÷ 1.5 = 0.2, the order delivery time threshold after normalization is 0.6 ÷ 1.5 = 0.4, and the resource allocation conflict metric after normalization is 0.6 ÷ 1.5 = 0.4. Combine these three normalized values into a vector, that is, the adjusted multi-objective optimization parameter set [0.2, 0.4, 0.4].

[0124] Step S144: Input the adjusted multi-objective optimization parameter set into the policy gradient optimizer for iterative optimization to generate a real-time production scheduling optimization policy set including the device scheduling priority sequence, order allocation weight sequence, and resource adjustment ratio sequence.

[0125] In this embodiment, the policy gradient optimizer can take the adjusted multi-objective optimization parameter set as input and find the optimal production scheduling strategy through iterative optimization. In each iteration, the policy gradient optimizer generates a production scheduling strategy according to the current parameter set, and then evaluates the performance of this strategy under the production scheduling optimization objective function. For example, calculate indicators such as equipment utilization rate, order delivery delay, and resource consumption balance under this strategy. According to the evaluation results, the policy gradient optimizer adjusts the parameters of the strategy to make the performance of the strategy generated next time better under the production scheduling optimization objective function. After multiple iterations, it finally converges to a relatively optimal strategy. This strategy includes a device scheduling priority sequence. For example, according to the performance, availability, and current load of the device, it is determined that device A is scheduled prior to device B; an order allocation weight sequence, such as the allocation weight of order 1 is 0.3 and the allocation weight of order 2 is 0.5, etc.; a resource adjustment ratio sequence, such as the energy supply adjustment ratio is 45% and the material replenishment priority is 30%, etc. The above sequences together constitute the real-time production scheduling optimization strategy set.

[0126] Step S150: Input the real-time production scheduling optimization strategy set into the production scheduling control system of the target factory, trigger the dynamic adjustment operation of the production scheduling execution instruction, and update the network parameters of the deep reinforcement learning policy network based on the adjusted execution feedback data.

[0127] For example, the specific process of step S150 is as follows: Step S151: Convert the device scheduling priority sequence into a device control instruction set, and the device control instruction set includes device start / stop time instructions, device load adjustment instructions, and device maintenance trigger instructions.

[0128] In this embodiment, for the device scheduling priority sequence, according to the priority and production requirements of each device therein, it is converted into a specific device control instruction set. For example, the device scheduling priority sequence determines that device A is scheduled prior to device B and starts working at 9 am. Then the device start / stop time instruction in the device control instruction set will clearly indicate that device A starts at 9 am. At the same time, according to the requirements of the production task, the device load adjustment instruction is determined, such as the load of device A is adjusted to 70%. In addition, if the device maintenance warning index shows that the device needs maintenance, the device control instruction set will include a device maintenance trigger instruction, such as instructing device A to perform maintenance at 5 pm.

[0129] Step S152: Convert the order allocation weight sequence into an order processing queue, and the order processing queue includes order batch processing rules, order resource locking strategies, and order delay compensation mechanisms.

[0130] In this embodiment, the order allocation weight sequence reflects the importance of each order in production scheduling. According to this order allocation weight sequence, an order processing queue is generated. For example, the allocation weight of order 1 is 0.3, the allocation weight of order 2 is 0.5, and the allocation weight of order 3 is 0.2. Sorting according to the weight size determines the order of order processing and forms an order processing queue. The order batching rule determines whether an order needs to be batched according to factors such as the order quantity and the production capacity of the equipment. For example, order 1 and order 2 are combined into one batch for processing. The order resource locking strategy locks the corresponding resources according to the order requirements. For example, 50 kilograms of raw materials are locked for order 2. The order delay compensation mechanism takes corresponding compensation measures when an order is delayed, such as providing a certain discount to the customer or expediting the processing of subsequent orders.

[0131] Step S153: Convert the resource adjustment ratio sequence into a resource allocation instruction, where the resource allocation instruction includes an energy supply adjustment ratio, a material replenishment priority, and a manpower scheduling plan.

[0132] In this embodiment, the resource adjustment ratio sequence specifies the allocation ratios of various resources. It is converted into specific resource allocation instructions. For example, if the resource adjustment ratio sequence determines that the energy supply adjustment ratio is 45%, then the resource allocation instruction will instruct the energy supply system to adjust the energy supply to 45% of the level. The material replenishment priority is 30%, which means that during material replenishment, the production link or order corresponding to this priority will be given priority for materials. The manpower scheduling plan arranges the work positions and working hours of personnel according to production requirements and resource adjustment situations. For example, 10 workers are arranged to work in the battery cell manufacturing link for 8 hours.

[0133] Step S154: Monitor the actual production efficiency indicators after the production scheduling control system executes the equipment control instruction set, the order processing queue, and the resource allocation instruction. The actual production efficiency indicators include the equipment utilization deviation value, the order delivery delay rate, and the resource consumption overrun.

[0134] For example, the specific sub-steps of step S154 are as follows: Step S1541: Collect the actual execution time deviation data of the equipment in response to the equipment start-stop time instruction, and calculate the equipment utilization deviation value.

[0135] In this embodiment, after the production scheduling control system executes the equipment control instruction set, the actual execution time deviation data of the equipment's response to the equipment start / stop time instruction is collected. For example, equipment A is instructed to start at 9:00 am, but actually starts at 9:10 am, resulting in a time deviation of 10 minutes. By statistically analyzing the actual operating time and the planned operating time of the equipment over a period of time, the equipment utilization rate deviation value is calculated. Suppose the planned operating time of equipment A is 8 hours, but due to reasons such as start-up delay and mid-course failure, it actually operates for 7.5 hours. The planned equipment utilization rate is 80%, and the actual utilization rate is 75%. Then the equipment utilization rate deviation value is 80% - 75% = 5%.

[0136] Step S1542: Statistically analyze the ratio of the number of orders that are not completed on time in the order processing queue to the total number of orders, and generate the order delivery delay rate.

[0137] In this embodiment, the number of orders that are not completed on time and the total number of orders in the order processing queue are statistically analyzed. For example, there are 10 orders in the order processing queue, and 2 of them are not completed on time. Then the order delivery delay rate is 2÷10 = 20%.

[0138] Step S1543: Monitor the difference between the energy consumption after the execution of the resource allocation instruction and the preset quota, and generate the resource consumption overrun amount.

[0139] In this embodiment, the energy consumption after the execution of the resource allocation instruction can be monitored and compared with the preset energy quota. Suppose the preset energy quota is 1000 kWh, and the actual energy consumption is 1100 kWh. Then the resource consumption overrun amount is 1100 - 1000 = 100 kWh.

[0140] Step S1544: Standardize the equipment utilization rate deviation value, the order delivery delay rate, and the resource consumption overrun amount to generate the actual production efficiency index.

[0141] In this embodiment, in order to uniformly measure the equipment utilization rate deviation value, the order delivery delay rate, and the resource consumption overrun amount, standardization processing is performed. Suppose the equipment utilization rate deviation value is 5%, the order delivery delay rate is 20%, and the resource consumption overrun amount is 100 kWh. These values can be converted to a unified range, such as 0 - 1, through a certain method. For example, the standardized equipment utilization rate deviation value is 0.1, the standardized order delivery delay rate is 0.3, and the standardized resource consumption overrun amount is 0.2. Combine these three standardized values into a vector, that is, the actual production efficiency index [0.1, 0.3, 0.2].

[0142] Step S155: Calculate the policy network loss function based on the difference between the actual production efficiency index and the preset expected efficiency index, and update the network parameters of the deep reinforcement learning policy network through the backpropagation algorithm.

[0143] In this embodiment, the preset expected efficiency index is set according to the production goals and requirements of the factory. For example, it is expected that the deviation value of equipment utilization rate does not exceed 3%, the order delivery delay rate does not exceed 10%, and the excess consumption of resources does not exceed 50 kWh. After standardizing the above expected indexes, they are [0.06, 0.2, 0.1]. Calculate the difference between the actual production efficiency index [0.1, 0.3, 0.2] and the expected efficiency index [0.06, 0.2, 0.1]. For example, obtain the difference value by calculating the Euclidean distance between the two. Calculate the policy network loss function according to this difference value. The loss function reflects the gap between the current production scheduling policy generated by the deep reinforcement learning policy network and the expected goal. Through the backpropagation algorithm, propagate the gradient of the loss function back to each parameter of the deep reinforcement learning policy network, and update the network parameters according to the gradient. For example, adjust the parameters of the state encoding module, constraint matching module, etc., so that the next generated production scheduling policy is closer to the expected goal, and continuously optimize the performance of the deep reinforcement learning policy network.

[0144] Through the above steps, the embodiments of the present application effectively integrate and deeply mine the multi-dimensional heterogeneous information in the real-time production monitoring data set. The target production feature set obtained through feature extraction not only accurately depicts the dynamic evolution of equipment status, the dynamic adjustment requirements of order priorities, and the real-time changes in environmental constraints, but also provides a high-fidelity input representation for the dynamic modeling of the production system at the global level. The application of the deep reinforcement learning policy network can, based on the dynamic constraint matching processing mechanism, generate dynamic production constraint features that are highly adapted to the current production situation in real time, thus breaking through the limitations of traditional scheduling methods that are difficult to handle complex dynamic environments. Further, through policy iteration optimization processing, the dynamic production constraint features are deeply integrated with the preset scheduling optimization objective function to generate a real-time scheduling optimization policy set including equipment scheduling priority sequences, order allocation weight sequences, and resource adjustment ratio sequences. This real-time scheduling optimization policy set achieves the optimal allocation of production resources at the global level, not only considering the maximization of equipment utilization, but also taking into account the timeliness of order delivery and the refined control of production costs, thereby comprehensively improving production efficiency in multiple objective optimization dimensions. By feeding back the real-time scheduling optimization policy set to the scheduling control system and triggering dynamic adjustment operations, a closed-loop feedback mechanism for production execution and policy optimization is constructed, enabling the scheduling policy to continuously evolve with the real-time changes in the production process, significantly enhancing the adaptive ability and robustness of the production system. Particularly importantly, by updating the network parameters of the deep reinforcement learning policy network based on the adjusted execution feedback data, the deep integration of the policy learning mechanism and production practice is realized. This characteristic of online learning and continuous optimization can continuously accumulate production experience and improve the level of policy intelligence, thus forming a self-evolving and continuously optimized intelligent scheduling paradigm in the global logic. This paradigm not only breaks through the limitations of traditional scheduling methods that rely on manual experience and static models, but also achieves the global optimal decision-making of production scheduling in a dynamic and complex environment.

[0145] Figure 2 FIG. shows a schematic diagram of exemplary hardware and software components of a factory dynamic scheduling optimization system 100 based on deep reinforcement learning that can implement the idea of the present application provided by some embodiments of the present application. For example, the processor 120 can be used on the factory dynamic scheduling optimization system 100 based on deep reinforcement learning and is used to execute the functions in the present application.

[0146] The factory dynamic scheduling optimization system 100 based on deep reinforcement learning can be a general-purpose server or a special-purpose server, both of which can be used to implement the factory dynamic scheduling optimization method based on deep reinforcement learning of the present application. Although only one server is shown in the present application, for convenience, the functions described in the present application can be implemented in a distributed manner on multiple similar platforms to balance the processing load.

[0147] For example, the factory dynamic scheduling optimization system 100 based on deep reinforcement learning may include a network port 110 connected to a network, one or more processors 120 for executing program instructions, a communication bus 130, and different forms of storage media 140, such as disks, ROM, or RAM, or any combination thereof. Exemplarily, the factory dynamic scheduling optimization system 100 based on deep reinforcement learning may also include program instructions stored in ROM, RAM, or other types of non-transitory storage media, or any combination thereof. The method of the present application can be implemented according to the above program instructions. The factory dynamic scheduling optimization system 100 based on deep reinforcement learning further includes an input / output (I / O) interface 150 between the computer and other input / output devices.

[0148] For ease of explanation, only one processor is described in the factory dynamic scheduling optimization system 100 based on deep reinforcement learning. However, it should be noted that the factory dynamic scheduling optimization system 100 in the present application may also include multiple processors. Therefore, the steps executed by one processor described in the present application can also be jointly executed or separately executed by multiple processors. For example, if the processor of the factory dynamic scheduling optimization system 100 executes step A and step B, it should be understood that step A and step B can also be jointly executed by two different processors or separately executed in one processor. For example, the first processor executes step A, the second processor executes step B, or the first processor and the second processor jointly execute steps A and B.

[0149] In addition, an embodiment of the present invention further provides a readable storage medium, in which computer-executable instructions are preset. When the processor executes the computer-executable instructions, the above-mentioned factory dynamic scheduling optimization method based on deep reinforcement learning is implemented.

[0150] It should be noted that, in order to simplify the description of the present invention disclosure and thus help the understanding of one or more embodiments of the invention, in the foregoing description of the embodiments of the present invention, sometimes multiple features are merged into one embodiment, drawing, or description thereof.

Claims

1. A method for optimizing factory dynamic production scheduling based on deep reinforcement learning, characterized in that, The method includes: Obtaining a real-time production monitoring data set of a target factory, where the real-time production monitoring data set includes multiple production link monitoring sequences, and each production link monitoring sequence is composed of at least one device operation status parameter, an order execution progress parameter, and an environmental interference parameter; Performing feature extraction on the real-time production monitoring data set to obtain a target production feature set for each production link monitoring sequence, where the target production feature set includes device status dynamic features, order priority features, and environmental constraint features; Invoking a pre-trained deep reinforcement learning policy network to perform dynamic constraint matching processing on the target production feature set to generate dynamic production constraint features corresponding to the current production link monitoring sequence; Performing policy iteration optimization processing based on the dynamic production constraint features and a preset production scheduling optimization objective function to generate a real-time production scheduling optimization policy set, where the real-time production scheduling optimization policy set includes a device scheduling priority sequence, an order allocation weight sequence, and a resource adjustment ratio sequence; Inputting the real-time production scheduling optimization policy set into the production scheduling control system of the target factory, triggering a dynamic adjustment operation of the production scheduling execution instruction, and updating the network parameters of the deep reinforcement learning policy network based on the adjusted execution feedback data.

2. The factory dynamic scheduling optimization method based on deep reinforcement learning according to claim 1, characterized in that The performing feature extraction on the real-time production monitoring data set to obtain a target production feature set for each production link monitoring sequence includes: Performing time-series fluctuation analysis processing on the device operation status parameters in the production link monitoring sequence, and extracting the fluctuation amplitude feature and the trend correlation feature of the device operation status parameters within a continuous time window; Calculating the difference degree between the fluctuation amplitude feature and a preset device operation reference parameter to generate a device load deviation coefficient and a device maintenance warning index; Performing similarity matching between the trend correlation feature and a historical device failure mode database to determine the device failure risk level; Combining the device load deviation coefficient, the device maintenance warning index, and the device failure risk level into device status dynamic features; Performing urgency evaluation processing on the order execution progress parameter, extracting the difference ratio feature between the remaining processing time of the order and a preset delivery deadline, and generating order priority features in combination with the order type attribute, where the order priority features include order delivery urgency, order resource occupancy weight, and order type association constraints; Performing influence factor analysis processing on the environmental interference parameter, extracting the environmental temperature change rate, the energy supply volatility, and the external event trigger frequency, and generating environmental constraint features, where the environmental constraint features include a temperature adaptability coefficient, an energy stability level, and an event response priority; Performing feature fusion processing on the device status dynamic features, the order priority features, and the environmental constraint features to generate the target production feature set.

3. The factory dynamic production scheduling optimization method based on deep reinforcement learning according to claim 2, characterized in that The performing time-series fluctuation analysis processing on the device operation status parameters in the production link monitoring sequence, and extracting the fluctuation amplitude feature and the trend correlation feature of the device operation status parameters within a continuous time window includes: Divide the device operation status parameters into multiple consecutive time window sequences according to a preset time window length, where each time window contains a fixed number of sampling points of the device operation status parameters; Calculate the actual fluctuation amplitude of the device operation status parameters within each time window, generate the actual average fluctuation amplitude and the actual maximum fluctuation amplitude of each time window, and calculate the window-level difference degree between the actual average fluctuation amplitude and the preset reference average fluctuation amplitude to generate the window-level load deviation coefficient of each time window; Perform over-limit detection on the actual maximum fluctuation amplitude and the preset reference maximum fluctuation amplitude threshold to generate the maintenance trigger flag of each time window; Perform time-series aggregation processing on the window-level load deviation coefficients of consecutive time windows to generate time-series data of the device load deviation coefficient; Perform frequency statistics and duration analysis on the maintenance trigger flags of consecutive time windows to generate device maintenance warning indicators; Perform trend line fitting on the device operation status parameters within each time window, extract the trend line slope and the residual fluctuation range, and generate time window-level parameters of the trend correlation features; Combine the time-series data of the device load deviation coefficient and the device maintenance warning indicators into the fluctuation amplitude features, and combine the trend line slope and the residual fluctuation range into the trend correlation features.

4. The factory dynamic production scheduling optimization method based on deep reinforcement learning according to claim 2, wherein The calling of the pre-trained deep reinforcement learning policy network to perform dynamic constraint matching processing on the target production feature set to generate dynamic production constraint features corresponding to the current production link monitoring sequence, including: Input the target production feature set into the state encoding module of the deep reinforcement learning policy network to perform spatial mapping processing on the device state dynamic features to generate a device state encoding vector; Perform weight assignment processing on the order priority features and generate an order priority encoding vector in combination with historical order execution efficiency data; Perform normalization processing on the environmental constraint features to generate an environmental constraint encoding vector; Call the constraint matching module of the deep reinforcement learning policy network to perform joint attention calculation on the device state encoding vector, the order priority encoding vector, and the environmental constraint encoding vector to generate a joint attention weight distribution; Perform dynamic weighted aggregation processing on the device state encoding vector, the order priority encoding vector, and the environmental constraint encoding vector based on the joint attention weight distribution to generate the dynamic production constraint features.

5. The factory dynamic production scheduling optimization method based on deep reinforcement learning according to claim 4, characterized in that The performing of weight assignment processing on the order priority features and generating an order priority encoding vector in combination with historical order execution efficiency data includes: Obtain the order completion time distribution, resource consumption ratio, and customer satisfaction score in the historical order execution efficiency data; Generate a time urgency weight according to the proportional relationship between the order completion time distribution and the remaining processing time of the current order; Generate a resource adaptability weight according to the difference degree between the resource consumption ratio and the current order resource occupancy weight; Generate a customer priority weight according to the correlation degree between the customer satisfaction score and the current order type attribute; Normalize the time urgency weight, the resource adaptability weight, and the customer priority weight to generate an order priority weight vector; Concatenate the order priority weight vector with the order type association constraint to generate the order priority encoding vector.

6. The factory dynamic scheduling optimization method based on deep reinforcement learning according to claim 2, characterized in that, Perform policy iteration optimization processing based on the dynamic production constraint features and a preset scheduling optimization objective function to generate a real-time scheduling optimization policy set, including: Match the dynamic production constraint features with the scheduling optimization objective function to determine the initial policy feasible region under the current production state; Perform dynamic constraint decomposition processing on the initial policy feasible region to generate a multi-objective optimization parameter set, where the multi-objective optimization parameter set includes an equipment load balancing coefficient, an order delivery time threshold, and a resource allocation conflict index; Perform policy conflict verification processing based on the multi-objective optimization parameter set, reassign priorities to the optimization objectives with parameter conflicts, and generate an adjusted multi-objective optimization parameter set; Input the adjusted multi-objective optimization parameter set into a policy gradient optimizer for iterative optimization processing to generate a real-time scheduling optimization policy set including an equipment scheduling priority sequence, an order allocation weight sequence, and a resource adjustment ratio sequence.

7. The method for optimizing the dynamic scheduling of a factory based on deep reinforcement learning according to claim 6, characterized in that, The matching of the dynamic production constraint features with the scheduling optimization objective function to determine the initial policy feasible region under the current production state includes: Analyze the association relationship between the equipment state dynamic features and the environmental constraint features in the dynamic production constraint features, and extract the equipment availability constraint interval and the resource supply stability interval; Decompose the scheduling optimization objective function into multiple sub-objective functions, where the sub-objective functions include a function for maximizing equipment utilization, a function for minimizing order delivery delay, and a function for equalizing resource consumption; Match the equipment availability constraint interval with the function for maximizing equipment utilization to generate the first feasible region boundary for equipment scheduling; Align the urgency of the function for minimizing order delivery delay with the order priority features in the dynamic production constraint features to generate the second feasible region boundary for order allocation; Perform dynamic adaptation processing on the function for equalizing resource consumption and the resource supply stability interval to generate the third feasible region boundary for resource adjustment; Calculate the joint overlapping region of the first feasible region boundary, the second feasible region boundary, and the third feasible region boundary to generate the initial policy feasible region.

8. The factory dynamic production scheduling optimization method based on deep reinforcement learning according to claim 7, characterized in that The dynamic constraint decomposition processing on the initial policy feasible region to generate a multi-objective optimization parameter set includes: Divide equipment scheduling priority levels within the initial policy feasible region, and arrange the equipment scheduling priority levels in descending order according to the product of the equipment failure risk level and the equipment load deviation coefficient to generate an equipment load balancing coefficient; Construct a time urgency matrix based on the reciprocal relationship between the order delivery urgency and the remaining processing time of the order, and calculate the delivery time threshold for each order in combination with the order type association constraint; Analyze the inverse correlation between the energy supply volatility and the material replenishment priority within the resource supply stability interval to generate a resource allocation conflict indicator; Normalize and splice the device load balancing coefficient, the order delivery time threshold, and the resource allocation conflict indicator according to a preset weight ratio to generate the multi-objective optimization parameter set.

9. The factory dynamic production scheduling optimization method based on deep reinforcement learning according to claim 8, characterized in that Perform a policy conflict verification process based on the multi-objective optimization parameter set, and reassign priorities to the optimization objectives with parameter conflicts to generate an adjusted multi-objective optimization parameter set, including: Detect the resource occupancy conflict nodes between the device load balancing coefficient and the order delivery time threshold, where the resource occupancy conflict nodes are the time period identifiers where the device scheduling requirements do not match the order processing rate; Extract the event response priority in the environmental constraint features at the resource occupancy conflict nodes and determine whether to trigger an external event intervention mechanism; If the external event intervention mechanism is not triggered, dynamically reduce the weight of the device load balancing coefficient according to the current value of the resource allocation conflict indicator and increase the priority of the order delivery time threshold; If the external event intervention mechanism is triggered, freeze the update of the resource allocation conflict indicator according to the event response priority and replace the current device load balancing coefficient with the historical average parameter; Re-normalize the adjusted device load balancing coefficient, order delivery time threshold, and resource allocation conflict indicator to generate the adjusted multi-objective optimization parameter set.

10. A factory dynamic production scheduling optimization system based on deep reinforcement learning, characterized in that, The factory dynamic production scheduling optimization system based on deep reinforcement learning includes a processor and a memory. The memory is connected to the processor. The memory is used to store programs, instructions, or codes, and the processor is used to execute the programs, instructions, or codes in the memory to implement the factory dynamic production scheduling optimization method according to any one of claims 1-9 above.

Images